htmlCopy code
<!DOCTYPE html>
<html>
<head>
<title><%= title %></title>
</head>
<body>
<h1><%= message %></h1>
</body>
</html>
4. Test the Template:
Navigate to [Link] to see the rendered template with the title
"Express" and message "Hello there!".
Conclusion
Scaffolding an [Link] application using tools like express-generator helps you
quickly set up a structured project with common directories and files. Templating
engines like Pug and EJS allow you to render dynamic content in your HTML
pages, making your web application more interactive and user-friendly. By using
these tools and techniques, you can efficiently develop and maintain robust
[Link] applications.
UNIT 4
MongoDB is a popular NoSQL database, which contrasts with traditional SQL
databases in several key ways. Below, I'll outline the core concepts of both SQL
and NoSQL databases, focusing on how MongoDB fits into the NoSQL category.
UNIT 1 201
SQL (Relational Databases)
Core Concepts:
1. Tables: Data is stored in tables, which are similar to spreadsheets with rows
and columns.
2. Schema: A fixed schema defines the structure of data in each table (e.g., data
types and constraints).
3. Rows: Each record in a table is a row, and each row must adhere to the table's
schema.
4. Columns: Each column in a table represents a specific attribute of the data.
5. Primary Key: A unique identifier for each row in a table.
6. Foreign Key: A field (or collection of fields) in one table that uniquely identifies
a row in another table, creating a relationship between the tables.
7. SQL Queries: Structured Query Language (SQL) is used to perform CRUD
(Create, Read, Update, Delete) operations and more complex queries (joins,
aggregations).
Example SQL:
sqlCopy code
CREATE TABLE users (
id INT PRIMARY KEY,
name VARCHAR(100),
email VARCHAR(100)
);
INSERT INTO users (id, name, email) VALUES (1, 'John Doe', 'j
ohn@[Link]');
SELECT * FROM users WHERE id = 1;
NoSQL (MongoDB)
UNIT 1 202
Core Concepts:
1. Collections: Data is stored in collections, which are analogous to tables in SQL
but without a fixed schema.
2. Documents: Each record in a collection is a document, typically represented
in JSON (JavaScript Object Notation) or BSON (Binary JSON).
3. Schema Flexibility: MongoDB collections do not enforce a fixed schema,
allowing for flexible and dynamic document structures.
4. Embedded Documents and Arrays: Documents can contain nested
documents and arrays, supporting rich, hierarchical data structures.
5. Indexes: Similar to SQL, indexes improve query performance and can be
created on any field.
6. Replica Sets and Sharding: MongoDB supports high availability and scalability
through replication and sharding.
7. MongoDB Query Language (MQL): MongoDB uses its own query language for
CRUD operations and aggregations.
Example MongoDB:
javascriptCopy code
// Creating a collection and inserting a document
[Link]({
_id: 1,
name: 'John Doe',
email: 'john@[Link]',
address: {
street: '123 Main St',
city: 'Anytown'
},
interests: ['reading', 'traveling']
});
// Querying the collection
UNIT 1 203
[Link]({ _id: 1 });
Key Differences
1. Schema: SQL databases enforce a strict schema, while MongoDB allows a
flexible schema.
2. Data Model: SQL uses tables and rows, whereas MongoDB uses collections
and documents, often containing nested data structures.
3. Scalability: MongoDB is designed for horizontal scaling (sharding), while SQL
databases traditionally scale vertically (more powerful servers).
4. Transactions: SQL databases have robust ACID transaction support, while
MongoDB has improved its transaction capabilities but originally emphasized
eventual consistency.
5. Query Language: SQL databases use SQL, a standardized language, while
MongoDB uses its own query language (MQL).
When to Use SQL vs. NoSQL (MongoDB)
SQL:
When data integrity and complex transactions are crucial.
When the data structure is well-defined and unchanging.
When relationships between entities are complex and need to be enforced
through foreign keys.
NoSQL (MongoDB):
When dealing with large volumes of unstructured or semi-structured data.
When rapid development and iterations are needed, allowing for schema
flexibility.
When horizontal scalability and high availability are priorities.
In summary, MongoDB offers a flexible, scalable alternative to traditional SQL
databases, making it suitable for a wide range of modern applications where data
structures are dynamic, and performance and scalability are critical.
UNIT 1 204
create and manage MongoDB
ChatGPT
Creating and managing a MongoDB database involves several steps, from setting
up the environment to performing basic CRUD operations and ensuring proper
management and maintenance of the database. Here's a comprehensive guide:
1. Setting Up MongoDB
Installation
Windows:
1. Download MongoDB from the official MongoDB website.
2. Follow the installation instructions, ensuring MongoDB is added to the
system PATH.
3. Start the MongoDB service using the command prompt:
bashCopy code
net start MongoDB
macOS:
1. Install MongoDB using Homebrew:
bashCopy code
brew tap mongodb/brew
brew install mongodb-community@5.0
2. Start the MongoDB service:
bashCopy code
brew services start mongodb/brew/mongodb-community
UNIT 1 205
Linux:
1. Import the MongoDB public key:
bashCopy code
wget -qO - [Link]
[Link] | sudo apt-key add -
2. Create a list file for MongoDB:
bashCopy code
echo "deb [ arch=amd64,arm64 ] [Link]
g/apt/ubuntu focal/mongodb-org/5.0 multiverse" | sudo t
ee /etc/apt/[Link].d/[Link]
3. Install MongoDB:
bashCopy code
sudo apt-get update
sudo apt-get install -y mongodb-org
4. Start the MongoDB service:
bashCopy code
sudo systemctl start mongod
2. Basic MongoDB Commands
Connecting to MongoDB
Use the MongoDB shell:
UNIT 1 206
bashCopy code
mongo
Creating a Database
Switch to the database (this creates it if it doesn't exist):
javascriptCopy code
use myDatabase
Creating a Collection
Create a collection within the database:
javascriptCopy code
[Link]("myCollection")
Inserting Documents
Insert a single document:
javascriptCopy code
[Link]({ name: "John Doe", age: 30, ema
il: "john@[Link]" })
Insert multiple documents:
javascriptCopy code
[Link]([
{ name: "Jane Doe", age: 25, email: "jane@[Link]"
},
UNIT 1 207
{ name: "Jim Doe", age: 27, email: "jim@[Link]" }
])
Querying Documents
Find a document:
javascriptCopy code
[Link]({ name: "John Doe" })
Find multiple documents:
javascriptCopy code
[Link]({ age: { $gt: 25 } })
Updating Documents
Update a single document:
javascriptCopy code
[Link](
{ name: "John Doe" },
{ $set: { age: 31 } }
)
Update multiple documents:
javascriptCopy code
[Link](
{ age: { $lt: 30 } },
{ $set: { status: "young adult" } }
UNIT 1 208
)
Deleting Documents
Delete a single document:
javascriptCopy code
[Link]({ name: "John Doe" })
Delete multiple documents:
javascriptCopy code
[Link]({ age: { $lt: 30 } })
3. Managing MongoDB
Indexing
Create an index on a field:
javascriptCopy code
[Link]({ email: 1 })
Backup and Restore
Backup:
bashCopy code
mongodump --db myDatabase --out /path/to/backup
Restore:
UNIT 1 209
bashCopy code
mongorestore /path/to/backup
Monitoring
Use MongoDB's built-in tools like mongostat and mongotop to monitor
performance:
bashCopy code
mongostat
mongotop
4. Security and Access Control
Creating Users
Create a user with read and write access:
javascriptCopy code
use admin
[Link]({
user: "myUser",
pwd: "myPassword",
roles: [ { role: "readWrite", db: "myDatabase" } ]
})
Enabling Authentication
Edit the MongoDB configuration file ( [Link] ) to enable authentication:
yamlCopy code
security:
UNIT 1 210
authorization: enabled
Restart the MongoDB service:
bashCopy code
sudo systemctl restart mongod
5. Replication and Sharding
Setting Up Replication
Configure a replica set in the [Link] file:
yamlCopy code
replication:
replSetName: "myReplicaSet"
Initiate the replica set from the MongoDB shell:
javascriptCopy code
[Link]()
Setting Up Sharding
Enable sharding for the database:
javascriptCopy code
[Link]("myDatabase")
Shard a collection:
UNIT 1 211
javascriptCopy code
[Link]("[Link]", { shardKey:
1 })
Conclusion
Managing MongoDB involves setting up the database, performing CRUD
operations, ensuring data integrity with indexing and backup, securing the
database with authentication, and scaling with replication and sharding. MongoDB
provides powerful features and flexibility for handling modern data requirements.
migration of data into MongoDB
ChatGPT
Migrating data into MongoDB involves several steps, including preparing the data,
choosing the right tools, and performing the actual migration. The process can
vary depending on the source of the data (e.g., relational databases, CSV files,
JSON files). Here’s a comprehensive guide on how to migrate data into MongoDB:
1. Preparing for Migration
Assess Your Data
Identify Source Data: Understand the structure, relationships, and volume of
data you want to migrate.
Determine Data Transformation Needs: Decide if your data needs to be
transformed to fit into MongoDB's document model.
Install MongoDB and Required Tools
Ensure MongoDB is installed and running on your system.
Install necessary tools like mongoimport , mongodump , and mongorestore .
2. Data Migration from Relational Databases
Using MongoDB Atlas Data Migration Tool
UNIT 1 212
MongoDB Atlas provides a data migration tool that helps you migrate data from
SQL databases like MySQL, PostgreSQL, and SQL Server to MongoDB Atlas.
1. Set Up Atlas Account: Create an account and set up a cluster on MongoDB
Atlas.
2. Use Data Migration Tool:
Go to the Data Import & Export section in the Atlas dashboard.
Follow the instructions to connect your source database.
Map tables to MongoDB collections and specify any data transformations.
Using MongoDB Connector for BI
This allows you to import data using SQL queries directly into MongoDB.
1. Install the Connector: Follow the MongoDB BI Connector installation guide.
2. Configure the Connector: Set up the connector to connect to your MongoDB
instance.
3. Run SQL Queries: Use your preferred SQL client to run queries that will fetch
and insert data into MongoDB.
Using Custom Scripts
Write scripts in languages like Python, [Link], or Java to read from the SQL
database and insert into MongoDB.
Example in Python
1. Install Packages:
bashCopy code
pip install pymongo mysql-connector-python
2. Script to Migrate Data:
pythonCopy code
import [Link]
UNIT 1 213
from pymongo import MongoClient
# Connect to MySQL
mysql_conn = [Link](
host="mysql_host",
user="mysql_user",
password="mysql_password",
database="mysql_database"
)
mysql_cursor = mysql_conn.cursor(dictionary=True)
# Connect to MongoDB
mongo_client = MongoClient("mongodb://localhost:27017/")
mongo_db = mongo_client["mongo_database"]
mongo_collection = mongo_db["mongo_collection"]
# Fetch data from MySQL
mysql_cursor.execute("SELECT * FROM mysql_table")
rows = mysql_cursor.fetchall()
# Insert data into MongoDB
for row in rows:
mongo_collection.insert_one(row)
# Close connections
mysql_cursor.close()
mysql_conn.close()
mongo_client.close()
3. Data Migration from CSV/JSON Files
Using mongoimport
mongoimport is a command-line tool that imports content from a JSON, CSV, or TSV
file into MongoDB.
UNIT 1 214
Importing JSON Data
1. JSON File Format:
jsonCopy code
{ "name": "John Doe", "age": 30, "email": "john@[Link]
m" }
{ "name": "Jane Doe", "age": 25, "email": "jane@[Link]
m" }
2. Import Command:
bashCopy code
mongoimport --db myDatabase --collection myCollection --fi
le [Link] --jsonArray
Importing CSV Data
1. CSV File Format:
csvCopy code
name,age,email
John Doe,30,john@[Link]
Jane Doe,25,jane@[Link]
2. Import Command:
bashCopy code
mongoimport --db myDatabase --collection myCollection --ty
pe csv --headerline --file [Link]
4. Post-Migration Steps
UNIT 1 215
Verify Data Integrity
Check Counts: Ensure the number of documents in MongoDB collections
matches the source.
javascriptCopy code
[Link]()
Sample Data Check: Randomly check a few documents to ensure data
correctness.
javascriptCopy code
[Link]().limit(5).pretty()
Create Indexes
Create necessary indexes to optimize query performance.
javascriptCopy code
[Link]({ email: 1 })
Backup
Take a backup of your newly migrated data.
bashCopy code
mongodump --db myDatabase --out /path/to/backup
5. Automating and Scheduling
Using Cron Jobs (Linux) or Task Scheduler (Windows)
Linux: Create a cron job for periodic migrations or backups.
UNIT 1 216
bashCopy code
crontab -e
# Add a line for a daily backup at midnight
0 0 * * * mongodump --db myDatabase --out /path/to/backup
Windows: Use Task Scheduler to schedule similar tasks.
Using ETL Tools
Consider using ETL (Extract, Transform, Load) tools like Apache NiFi, Talend, or
Pentaho for more complex migration needs.
Conclusion
Migrating data into MongoDB requires careful planning and execution. Whether
you're moving data from relational databases, CSV files, or JSON files, MongoDB
provides various tools and methods to facilitate the process. Ensure you verify the
integrity of your data post-migration and set up automated processes for ongoing
data management and backups.
MongoDB with NodeJS
ChatGPT
Using MongoDB with [Link] involves setting up a [Link] application and
connecting it to a MongoDB database. The most popular library for this purpose is
mongoose , which provides a straightforward way to model your data and interact
with MongoDB. Here's a step-by-step guide:
1. Set Up Your Environment
Install [Link] and npm
Ensure [Link] and npm are installed. You can download them from the official
[Link] website.
Install MongoDB
UNIT 1 217
Install MongoDB on your machine or use MongoDB Atlas, the cloud version of
MongoDB. For local installation, follow the MongoDB installation guide.
2. Create a [Link] Project
Initialize a New [Link] Project
1. Open your terminal and create a new directory for your project.
bashCopy code
mkdir myMongoApp
cd myMongoApp
2. Initialize a new [Link] project.
bashCopy code
npm init -y
Install Required Packages
Install mongoose and express (a web framework for [Link]) along with other
dependencies.
bashCopy code
npm install mongoose express body-parser
3. Create the Application Structure
Project Structure
markdownCopy code
myMongoApp
│
UNIT 1 218
├── [Link]
├── models
│ └── [Link]
└── routes
└── [Link]
4. Define the Mongoose Model
models/[Link]
Create a Mongoose model for a User .
javascriptCopy code
const mongoose = require('mongoose');
const userSchema = new [Link]({
name: {
type: String,
required: true
},
email: {
type: String,
required: true,
unique: true
},
age: {
type: Number,
required: true
}
});
[Link] = [Link]('User', userSchema);
UNIT 1 219
5. Create the Express Application
[Link]
Set up your Express application to connect to MongoDB and define basic routes.
javascriptCopy code
const express = require('express');
const mongoose = require('mongoose');
const bodyParser = require('body-parser');
const app = express();
// Middleware
[Link]([Link]());
// Connect to MongoDB
[Link]('mongodb://localhost:27017/myMongoApp', { us
eNewUrlParser: true, useUnifiedTopology: true, useCreateInde
x: true })
.then(() => [Link]('MongoDB connected...'))
.catch(err => [Link](err));
// Routes
const userRoutes = require('./routes/user');
[Link]('/users', userRoutes);
// Start server
const PORT = [Link] || 5000;
[Link](PORT, () => [Link](`Server running on port
${PORT}`));
6. Define Routes for CRUD Operations
routes/[Link]
UNIT 1 220
Create routes for creating, reading, updating, and deleting users.
javascriptCopy code
const express = require('express');
const router = [Link]();
const User = require('../models/user');
// Create a new user
[Link]('/', async (req, res) => {
try {
const user = new User([Link]);
await [Link]();
[Link](201).send(user);
} catch (error) {
[Link](400).send(error);
}
});
// Get all users
[Link]('/', async (req, res) => {
try {
const users = await [Link]();
[Link](200).send(users);
} catch (error) {
[Link](500).send(error);
}
});
// Get a user by ID
[Link]('/:id', async (req, res) => {
try {
const user = await [Link]([Link]);
if (!user) {
return [Link](404).send();
}
[Link](200).send(user);
UNIT 1 221
} catch (error) {
[Link](500).send(error);
}
});
// Update a user by ID
[Link]('/:id', async (req, res) => {
try {
const user = await [Link]([Link].
id, [Link], { new: true, runValidators: true });
if (!user) {
return [Link](404).send();
}
[Link](200).send(user);
} catch (error) {
[Link](400).send(error);
}
});
// Delete a user by ID
[Link]('/:id', async (req, res) => {
try {
const user = await [Link]([Link].
id);
if (!user) {
return [Link](404).send();
}
[Link](200).send(user);
} catch (error) {
[Link](500).send(error);
}
});
[Link] = router;
UNIT 1 222
7. Running the Application
Start the application:
bashCopy code
node [Link]
You can now use tools like Postman to interact with your API. Here are some
example requests:
Create a User:
POST [Link]
Body: { "name": "John Doe", "email": "john@[Link]", "age": 30 }
Get All Users:
GET [Link]
Get a User by ID:
GET [Link]
Update a User by ID:
PUT [Link]
Body: { "name": "John Smith", "email": "johnsmith@[Link]", "age": 31 }
Delete a User by ID:
DELETE [Link]
Conclusion
This guide outlines the steps to create a [Link] application that connects to
MongoDB using mongoose . You set up an Express server, defined Mongoose
models, and implemented CRUD operations through RESTful API routes. This
foundation can be extended with more advanced features, such as authentication,
validation, and more complex data relationships.
services offered by MongoDB
ChatGPT
UNIT 1 223
MongoDB offers a variety of services to help manage, deploy, and optimize
MongoDB databases. Here’s an overview of the key services provided by
MongoDB:
1. MongoDB Atlas
MongoDB Atlas is a fully managed cloud database service that handles the
complexities of deploying, managing, and scaling MongoDB.
Features:
Automated Backups: Scheduled and on-demand backups with point-in-time
recovery.
Global Clusters: Distribute your database globally to provide low-latency
access to data anywhere in the world.
Scalability: Seamlessly scale your database up or down with minimal
downtime.
Monitoring and Alerts: Integrated performance monitoring and alerting to
identify and resolve issues quickly.
Security: Advanced security features, including encryption at rest, encryption
in transit, network isolation, and fine-grained access controls.
Serverless Instances: Flexible pay-per-operation pricing model that
automatically scales with application traffic.
2. MongoDB Realm
MongoDB Realm is a mobile application development platform that provides
synchronization between the client and server, making it easy to build responsive
and offline-first apps.
Features:
Real-Time Sync: Automatic synchronization of data between the client and
MongoDB Atlas.
Data Access Controls: Fine-grained access control to manage data
permissions.
UNIT 1 224
Serverless Functions: Write server-side logic using MongoDB Realm
Functions.
Triggers: Respond to database changes and events with triggers.
GraphQL and REST APIs: Automatically generate APIs to interact with your
MongoDB data.
3. MongoDB Atlas Data Lake
MongoDB Atlas Data Lake allows you to query data across multiple sources,
including your Atlas clusters, Amazon S3, and other data lakes.
Features:
Unified Query Interface: Use the MongoDB Query Language (MQL) to query
across your data lake and databases.
Scalability: Scales automatically to handle large volumes of data.
Data Integration: Integrate data from different sources and formats for
comprehensive analysis.
4. MongoDB Atlas Search
MongoDB Atlas Search is a full-text search service integrated into MongoDB
Atlas, powered by Apache Lucene.
Features:
Rich Text Search: Provides powerful full-text search capabilities, including
autocomplete, relevance scoring, and more.
Seamless Integration: Integrated into your existing MongoDB workflows and
data models.
Scalability: Automatically scales with your MongoDB cluster.
5. MongoDB Charts
MongoDB Charts is a visualization tool that allows you to create and share
visualizations of your MongoDB data.
UNIT 1 225
Features:
Interactive Dashboards: Build and share interactive dashboards to visualize
your data.
Real-Time Data: Visualize live data from your MongoDB collections.
Embed Charts: Embed your charts into your applications or websites.
6. MongoDB Compass
MongoDB Compass is a graphical user interface (GUI) for MongoDB, providing a
visual approach to explore your data.
Features:
Schema Visualization: Visualize your schema and understand your data
structure.
Data Exploration: Query and explore your data using an intuitive interface.
Performance Analysis: Analyze and optimize query performance.
7. MongoDB Ops Manager
MongoDB Ops Manager is an enterprise-grade management platform for
deploying and managing MongoDB on your own infrastructure.
Features:
Automation: Automate routine operations like provisioning, scaling, and
backups.
Monitoring: Monitor your MongoDB deployments with real-time metrics and
alerts.
Backup and Restore: Comprehensive backup and restore capabilities to
protect your data.
8. MongoDB Enterprise Advanced
MongoDB Enterprise Advanced is a subscription package for running MongoDB
in production environments, offering advanced features and support.
UNIT 1 226
Features:
Advanced Security: Includes LDAP integration, Kerberos authentication, and
auditing.
Support: Access to MongoDB’s support team for assistance and
troubleshooting.
Management Tools: Includes Ops Manager and Cloud Manager for database
management.
9. Professional Services and Training
MongoDB Professional Services offers consulting and training to help you get the
most out of MongoDB.
Services:
Consulting: Tailored consulting services to optimize your MongoDB
deployment.
Training: In-person and online training programs to enhance your team's
MongoDB skills.
Certification: Certification programs to validate your MongoDB expertise.
Conclusion
MongoDB provides a comprehensive suite of services to support the entire
lifecycle of your MongoDB deployments, from development and testing to
production and analytics. Whether you are using MongoDB in the cloud with Atlas,
building mobile applications with Realm, or managing on-premise deployments
with Ops Manager, MongoDB offers the tools and services to ensure your success
UNIT 1 227