Lecture Notes

Subject: Database Management Systems Applications

TE Comp. 2012 Course, SPPU

UNIT-V : NOSQL Databases

By : Prof. Sushma R. Vispute

PCET’s Pimpri Chinchwad College of Engineering

Department of Computer Engineering
 Objective Course

To learn and understand NOSQL Data

Models.

To develop an ability to use database

technique such as NOSQL.
 Nearly a quarter of the world's population –
roughly 1.4 billion people – will use the Internet
on a regular basis in 2009.
50 billion photos taken every year
Online Video

4,700,000,000 video streams monthly

England has approximately 4 million
surveillance cameras

1 for approximately every 14 Britons

 Storage requirements: Facebook
• 10,000,000,000 photos
• 2-3 Terabytes of photos are
being uploaded to the site
every day
• One petabyte of photo storage
• Serve over 15 billion photo
images per day
• Photo traffic now peaks at over
300,000 images served per
second
 What is Data
“Collection of raw facts from which conclusions may be drawn”

 Data is converted into more

convenient form i.e. Digital Data
 Increase in data processing capabilities Video

 Lower cost of digital storage

01010101010
 Affordable and faster communication Photo 10101011010
technology 00010101011
01010101010
Book
10101010101
01010101010

 Who creates data? Letter Digital Data

 Individuals
 Businesses
Categories of Data
 Data can be categorized as either
structured or unstructured data
 Over 80% of enterprise
information is unstructured E-Mail Attachments PDFs

X-Rays
Check Unstructured (80%)

Manuals Instant Messages

Images
Documents

Forms
Web Pages

Contracts
Rich Media

Invoices

Audio Video

Structured (20%)
Rows and Columns
Introduction

Enormous data that comes out every day from the web
 Handled by Relational database management systems
(RDBMS).
The relational model is well-suited to client-server
programming and today it is predominant technology for
storing structured data in web and business applications.
Classical relation database follow the ACID Rules
Atomic : A transaction is a logical unit of work which must
be either completed with all of its data modifications, or
none of them is performed.
Consistent : At the end of the transaction, all data must be
left in a consistent state.
Isolated : Modifications of data performed by a transaction
must be independent of another transaction. Unless this
happens, the outcome of a transaction may be erroneous.
Durable : When the transaction is completed, effects of the
modifications performed by the transaction must be
permanent in the system.
 Distributed Systems
• A distributed system consists of multiple computers and
software components that communicate through a
computer network (a local network or by a wide area
network).
• A distributed system can consist of any number of possible
configurations, such as mainframes, workstations, personal
computers, and so on.
• The computers interact with each other and share the
resources of the system to achieve a common goal.
 Advantages of Distributed Computing
Reliability (fault tolerance) : The important advantage of distributed
computing system is reliability. If some of the machines within the system
crash, the rest of the computers remain unaffected and work does not stop.
Scalability : In distributed computing the system can easily be expanded by
adding more machines as needed.
Sharing of Resources : Shared data is essential to many applications such as
banking, reservation system. As data or resources are shared in distributed
system, other resources can be also shared (e.g. expensive printers).
Flexibility: As the system is very flexible, it is very easy to install, implement
and debug new services.
Speed: A distributed computing system can have more computing power and
it's speed makes it different than other systems.
Open system :As it is open system, every service is equally accessible to
every client i.e. local or remote.
Performance: The collection of processors in the system can provide higher
performance (and better price/performance ratio) than a centralized
computer.
 Disadvantages of Distributed Computing
Troubleshooting :
Troubleshooting and diagnosing problems.
Software :
Less software support is the main disadvantage of
distributed computing system.
Networking :
The network infrastructure can create several
problems such as transmission problem,
overloading, loss of messages.
Security :
Easy access in distributed computing system
increases the risk of security and sharing of data
generates the problem of data security
 What is NoSQL ?
NoSQL is a non-relational database management
systems, different from traditional relational database
management systems in some significant ways.
• It is designed for distributed data stores where very
large scale of data storing needs (for example Google
or Facebook which collects terabits of data every day
for their users).
• These type of data storing may not require fixed
schema, avoid join operations and typically scale
horizontally.
Why NoSQL ?
In today’s time data is becoming easier to access and
capture through third parties such as Facebook,
Google+ and others. Personal user information,
social graphs, geo location data, user-generated
content and machine logging data are just a few
examples where the data has been increasing
exponentially. To avail the above service properly,
it is required to process huge amount of data.
Which SQL databases were never designed. The
evolution of NoSql databases is to handle these
huge data properly.
Why NoSQL ?
 NoSQL Products/Projects
[Link] 122
NoSQL Databases
Cassandra

CouchDB

Hadoop & Hbase

MongoDB

StupidDB

Etc.
18 4/3/2026
 RDBMS vs NoSQL
RDBMS NoSQL (Not Only SQL )

1. Structured and organized data 1. Unstructured and unpredictable

2. Structured query language (SQL) data
3. Data and its relationships are 2. No declarative query language
stored in separate tables. 3. No predefined schema
4. Data Manipulation Language, 4. Key-Value pair storage, Column
Data Definition Language Store, Document Store, Graph
5. ACID Property databases
6. Supports Tight Consistency and 5. BASE & CAP Theorem
availability 6. Eventual consistency
7. Prioritizes high performance, high
availability and scalability
 Brief history of NoSQL
The term NoSQL was coined by Carlo Strozzi in the year 1998.
He used this term to name his Open Source, Light Weight,
DataBase which did not have an SQL interface.
In the early 2009, when [Link] wanted to organize an event on
open-source distributed databases, Eric Evans, a Rackspace
employee, reused the term to refer databases which are non-
relational, distributed, and does not conform to atomicity,
consistency, isolation, durability - four obvious features of
traditional relational database systems.
In the same year, the "no:sql(east)" conference held in Atlanta,
USA, NoSQL was discussed and debated a lot.
And then, discussion and practice of NoSQL got a momentum,
and NoSQL saw an unprecedented growth.
 CAP Theorem (Brewer’s Theorem)

You must understand the

CAP theorem when you
talk about NoSql
databases or in fact when
designing any distributed
system. CAP theorem
states that there are three
basic requirements which
exist in a special relation
when designing
applications for a
distributed architecture.
CAP Theorem
Consistency - This means that the data in the
database remains consistent after the execution of
an operation. For example after an update
operation, all clients see the same data.
Availability - This means that the system is always
on (service guarantee availability), no downtime.
Partition Tolerance - This means that the system
continues to function even the communication
among the servers is unreliable, i.e. the servers
may be partitioned into multiple groups that
cannot communicate with one another.
CAP Theorem
CA - Single site cluster, therefore all nodes are always in contact.
When a partition occurs, the system blocks.

CP - Some data may not be accessible, but the rest is still

consistent/accurate.

AP - System is still available under partitioning, but some of the

data returned may be inaccurate
BASE

Instead of adhering to ACID transactions, NoSQL

data stores recognize the CAP theorem, first
explored by Professor Eric Brewer of UC
Berkeley which states that it is impossible for a
distributed computer system to simultaneously
guarantee all three of the properties of consistency,
availability, and partition tolerance.
With this idea in mind, a new set of BASE properties
have been proposed and recognized by most
NoSQL database systems:
The BASE
The BASE acronym was defined by Eric Brewer,
who is also known for formulating the CAP
theorem.
The CAP theorem states that a distributed computer
system cannot guarantee all of the following three
properties at the same time:
Consistency
Availability
Partition tolerance
 The BASE
A BASE system gives up on consistency.
Basically Available indicates that the system does guarantee availability, in
terms of the CAP theorem. The database is basically available such that if
some part of the database becomes unavailable, other parts of the database
continue to function as expected.
Soft state indicates that the state of the system may change over time, even
without input. This is because of the eventual consistency model. Data
may be time-dependent on user interaction with possible expiration after a
period of time. The data must be updated or accessed to remain relevant in
the system.
Eventual consistency indicates that the system will become consistent over
time, given that the system doesn't receive input during that time.
Updated data may not be immediately consistent across the entire system
but with time will become consistent. Therefore, the data is said to be
consistent in the future.
 NoSQL Features (Architecture)
NoSQL data stores are distinguished by several features that
generally stand in contrast to traditional relational
databases.
 Linear Scaling: NoSQL data stores offer much better
scaling that traditional relational databases. Many NoSQL
systems boast linear scaling computational abilities and
petabyte capacities.
 Dynamic Data Models: The data models are often very
flexible or schema-less in design. Referential integrity is
abandoned in favor of restructured or de-normalized data.
Furthermore, the attributes or fields available for storing
data are typically dynamic.
 NoSQL Features (Architecture)
 Replicated and Redundant: Since many NoSQL
database systems target the cloud environment, the
systems allow for failure to be the norm instead of the
exception. In order to accommodate this assumption, data
stores typically provide built-in replication and failover
abilities with fast fault recovery.
 Simple API: The complexity of SQL is abandoned in
favor of simple APIs. Some computation and analysis of
data is moved outside of the database and onto the client
or map-reduce machines which can more easily scale in a
cloud environment. The expensive joins that are
encouraged by SQL are largely abandoned.
 NoSQL Features (Architecture)
 High Throughput: Most NoSQL systems feature
simultaneous high throughput reading and writing across
their entire dataset. Many systems boast lockless designs
that allow for consistent reading and writing speeds
regardless of the number of users reading or updating the
data.
 NoSQL Features (Architecture)
 The linear scaling capabilities of most NoSQL
databases are accomplished through horizontal
scaling ”shared nothing” systems that tend to
abandon the idea of ACID transactions that
feature the following properties:
 Atomicity: All operations that are part of a
transaction are an ”all or nothing” operation. If
any operation in the transaction fails, then all of
the operations must fail. There cannot be a
partial transaction.
 NoSQL Features (Architecture)
 Consistency: Transactions performed by the database
will always leave the data in a consistent state. An
operation cannot leave the database in an inconsistent
state, but rather the database will move from one
consistent state to another.
 Isolation: Each transaction will be isolated from other
transactions, even if they are operating on the same data.
The operations in one transaction will not affect the
intermediate operations of another transaction.
 Durability: A committed transaction will remain
committed even in the event of a system failure. Thus, if
a transaction is committed, it is guaranteed that the data
was successfully stored in the database.
NoSQL Distinguishing Characteristics
Large data volumes
Google’s “big data”
Scalable replication and distribution
Potentially thousands of machines
Potentially distributed around the world
Queries need to return answers quickly
Mostly query, few updates
Asynchronous Inserts & Updates
Schema-less
ACID transaction properties are not needed – BASE
CAP Theorem
Open source development
 NoSQL pros/cons
Advantages :
High scalability
Distributed Computing
Lower cost
Schema flexibility, semi-structure data
No complicated Relationships
Disadvantages
No standardization
Limited query capabilities (so far)
Eventual consistent is not intuitive to program for
ACID vs BASE
ACID BASE

Atomic Basically Available

Consistency Soft state

Isolation Eventual consistency

Durable
Four categories of NOSQL
Databases/Data Models
Key-Value Stores
Document Stores
Wide Column Stores
Graph Databases
Key-Value Stores
Key valued stores are those types of NoSQL database that are scheme
free, and also your values stored as key i.e in one column you will be
having a key “Name” and the value would be “Zack” and in the
second column it’s not necessary mean that you must have the
value of Name again you could store different kind of data in the
same column in different row, and also you could have more column
in one row than previous or vice versa, this is the most common kinds
of NoSQL database that are currently in the market and other kinds of
NoSQL database are built upon the principle of this kinds of NoSQL
database and added some features on that.
-Riak
-redis
-Azure table storage
-Amazon SimpleDB
-Dynomite
-BerkeleyDB
Document Stores
These kinds of NoSQL database are very interesting because instead of
rows and columns your data is stored in documents and these
simply sored in JavaScript like JASON JavaScript Object
Notation and the language of the database that is used internally is
JavaScript.
Since JavaScript is very popular in the web market and most of the
developer are familiar with that language already these kinds of
NoSQL database are very popular with web developer and also these
kinds of NoSQL database are same as Key-Value Stores are Scheme
free.
CouchDB
-MongoDB
-TeraStore
-MarkLogic
- Jackrabbit
Wide Column Stores/Extensible Record Stores
This is another kinds of NoSQL database which they are not Scheme free
they are kinds of semi Structured database which means you need to
specify a groups of columns in these database and they are called
Column Families what it really means is that you will have some
kinds of master Column name and then in each row you could
have different column names which would be different from your
master column name.
-Apache Hbase
- Cassandra
- BigTable
- Hypertable
- Qbase
- OpenNeptune
- Sqrrl Enterprise
Graph Databases
These kinds of NoSQL database are the most specialized kinds of
database they are focused on the relationship between entities rather
than entities themselves, they are mostly famous on social media and
very useful in that field because it’s best to use in social graphs and
taking your social media and your group of friends you could
better understand these kinds of database.
-NeoJ4
-Sones
-InfiniteGraph
-OrientDB
-AllegroGraph
-FlockDB
Categories of NoSQL database
Category Description Name of the database

Document Oriented Data is stored as MongoDB, CouchDB,

documents. An example RethinkDB, RavenDB etc.
format may be like -
FirstName="Arun",
Address="St. Xavier's
Road",
Spouse=[{Name:"Kiran"}],C
hildren=[{Name:"Rihit",
Age:8}]

XML database Data is stored in XML format BaseX, eXist, MarkLogic

Server etc.

Graph databases Data is stored as a Allegro, Neo4J, OrientDB,

collection of nodes, where Virtuoso.
nodes are analogous to
objects in a programming
language. Nodes are
connected using edges.
Categories of NoSQL database
Category Description Name of the database

Key-value store In Key-value-store category Dynamo, FoundationDB,

of NoSQL database, an user MemcacheDB, Redis, Riak.
can store data in schema- etc.
less way. A key may be
strings, hashes, lists, sets,
sorted sets and values are
stored against these keys.

Column store A column is a key value Accumulo, Cassandra,

pair, where the key is HBase etc.
an identifier and the
value stores values
related to the key
(identifier).
Production deployment
There is a large number of companies using NoSQL.
To name a few :
Google
Facebook
Mozilla
Adobe
Foursquare
LinkedIn
Digg
McGraw-Hill Education
Vermont Public Radio
MongoDB
What is MongoDB
MongoDB is an open-source document database that
provides high performance, high availability, and
automatic scaling.
Document Database
A record in MongoDB is a document, which is a data structure composed
of field and value pairs. MongoDB docu-ments are similar to JSON
objects. The values of fields may include other documents, arrays, and
arrays of documents.
The advantages of using documents are:
• Documents (i.e. objects) correspond to native data types in many
programming languages.
• Embedded documents and arrays reduce need for expensive joins.
• Dynamic schema supports fluent polymorphism.
Key Features
High Performance
MongoDB provides high performance data persistence. In particular,
• Support for embedded data models reduces I/O activity on database
system.
• Indexes support faster queries and can include keys from embedded
documents and arrays.
High Availability
To provide high availability, MongoDB’s replication facility, called replica
sets, provide:
• automatic failover.
• data redundancy.
A replica set is a group of MongoDB servers that maintain the same data
set, providing redundancy and increasing data availability.
Key Features
Automatic Scaling
MongoDB provides horizontal scalability as part of
its core functionality.
• Automatic sharding distributes data across a
cluster of machines.
• Replica sets can provide eventually-consistent
reads for low-latency high throughput
deployments.
Start Using.......
A Document- roughly equvivalent to row in RDBMS
Collection – table with dynamic schema
Multiple independant databases
“-id”
 Collage Teacher Database
Creating Teacher Database. Which contain the
information of Teacher_id, name of a teacher,
department of a teacher, salary and status of a
teacher. Here status is whether teacher is approved
by the university or not.

DDL & DML queries on the Teacher Database

and difference between SQL Commands and
mongodb commands.
 SQL Vs MongoDB
SQL Concepts MongoDB Concepts

database database
table Collection
Row Document 0r BSON Document

Column Field
Index Index
Table Join Embedded documents & Linking

Primary key Primary Key

Specify any unique column or column In MongoDB, the primary key is
combination as primary key. automatically set to the field.

aggregation (e.g. group by) aggregation pipeline

 Table Creation in MySql
CREATE TABLE CTeacher_Info( Teacher_id Varchar(10), Teacher_Name
Varchar(10),
Dept_Name Varchar(10), Salary Number(10), Status char(1), PRIMARY KEY (id) );
SQL:
INSERT INTO TeacherInfo(Teacher_id, Teacher_Name, Dept_Name, Salary, Status)
VALUES (“Pic001", “Ravi”, “IT”,30000, “A");
INSERT INTO TeacherInfo(Teacher_id, Teacher_Name, Dept_Name, College_Name,
Status) VALUES (“Pic002", “Mangesh”, “IT”,20000, “A");
INSERT INTO TeacherInfo(Teacher_id, Teacher_Name, Dept_Name, Salary, Status)
VALUES (“Pic003", “Akshay”, “Comp”,25000, “N");
o/p CTeacher_Info

Teacher_id Teacher_Na Dept_Name, Salary Status

me
Pic001 Ravi IT 30000 A
Pic002 Mangesh IT 20000 A
Pic003 Akshay Comp 25000 N
 Creation of Document in MongoDB
{{
{
Teacher_id: “Pic001",
Teacher_Name: “Ravi”,
Dept_Name: “IT”,
Sal: 30000,
status: 'A'
}
OR
[Link](“Teacher_info")

Insert Command:

db.Teacher_info.insert( { Teacher_id: “Pic001", Teacher_Name: “Ravi",

Dept_Name: “IT”, Sal:30000, status: "A" } )
db.Teacher_info.insert( { Teacher_id: “Pic002", Teacher_Name: “Ravi",
Dept_Name: “IT”, Sal:20000, status: "A" } )
db.Teacher_info.insert( { Teacher_id: “Pic003", Teacher_Name: “Akshay",
Dept_Name: “Comp”, Sal:25000, status: “N" } )
 Retrieving data from Mongodb: find()

> db.Teacher_info.find()
Output:
{ "_id" : ObjectId("53a2d8ac8404f005f1acc666"), "Teacher_id" : "pic001",
"Teacher_name" : "Ravi", "Dept_name" : "IT", "sal" : 20000, "status" : "A"
}
{ "_id" : ObjectId("53a2d8fc8404f005f1acc667"), "Teacher_id" : "pic001",
"Teacher_name" : "Ravi", "Dept_name" : "IT", "sal" : 20000, "status" : "A"
}
{ "_id" : ObjectId("53a2d91b8404f005f1acc668"), "Teacher_id" : "pic003",
"Teacher_name" : "Akshay", "Dept_name" : "IT", "sal" : 25000, "status" :
"N" }
{ "_id" : ObjectId("53a2da038404f005f1acc669"), "Teacher_id" : "pic003",
"Teacher_name" : "Akshay", "Dept_name" : "IT", "sal" : 25000, "status" :
"N" }
>
 SQL & Mongodb Commands
SQL SELECT Statements MongoDB find() Statements

SELECT * FROM Teacher_info; db.Teacher_info.find()

SELECT * FROM Teacher_info WHERE db.Teacher_info.find( {sal:

sal = 25000; 25000})

SELECT Teacher_id FROM Teacher_info db.Teacher_info.find(

WHERE Teacher_id = 1; {Teacher_id: "pic001"})
 SQL & Mongodb Commands
SELECT * FROM Teacher_info db.Teacher_info.find({status:{$ne:"A"
WHERE status != "A“; }})
SELECT * FROM Teacher_info db.Teacher_info.find({status:"A",
WHERE status = "A" AND sal = 20000; sal:20000})

SELECT * FROM Teacher_info > db.Teacher_info.find( { $or: [ {

WHERE status = "A" OR sal = 50000; status: "A" } , { sal:50000 } ] } )

SELECT * FROM Teacher_info db. Teacher_info.find( { sal: { $gt:

WHERE sal > 40000 40000 } } )

SELECT * FROM db. Teacher_info.find( { sal: { $gt:

Teacher_infoWHERE sal < 30000 30000 } } )
 SQL & Mongodb Commands
SELECT * FROM Teacher_info db. Teacher_info.find( { status: "A" }
WHERE status = "A" ORDER BY SAL ).sort( { sal: 1 } )
ASC

SELECT * FROM users WHERE status db. Teacher_info.find( { status: "A" }

= "A" ORDER BY SAL DESC ).sort( {sal: -1 } )

SELECT COUNT(*) FROM db. Teacher_info.count()

Teacher_info; or
db. Teacher_info.find().count()

SELECT DISTINCT(Dept_name) db. Teacher_info.distinct(

FROM Teacher_info; “Dept_name" )
 Update
UPDATE Teacher_info SET
Records
db. Teacher_info.update( { sal: { $gt:
Dept_name = “ETC" WHERE sal > 25000 } }, { $set: { Dept_name: “ETC" }
250000 }, { multi: true } )

UPDATE Teacher_infoSET sal = sal + db. Teacher_info.update( { status: "A" }

10000 WHERE status = "A" , { $inc: { sal: 10000 } }, { multi: true } )

Delete Records
DELETE FROM Teacher_info WHERE db.Teacher_info.remove({Teacher_id:
Teacher_id = “pic001" "pic001"});

DELETE FROM Teacher_info; db. Teacher_info.remove({})

 Alter Table in MySql &
MongoDb
MySql:
ALTER TABLE Teacher_info ADD join_date
DATETIME
MongoDb:
At the document level, update() operations can
add fields to existing documents using the $set
operator.
Ex:
db.Teacher_info.update( { }, { $set: { join_date: new
Date() } }, { multi: true} )
 Drop Command

MySql:
DROP TABLE Teacher_info
Mongo:
db.Teacher_info.drop()
 1. MongoDB (Basic Commands)
 Stores data in JSON-like documents (BSON)
 No fixed schema (flexible structure)
 Uses:
o Database → Collection → Document

Example:

{
"name": "Sushma",
"age": 25,
"department": "Computer"
}

2. Start MongoDB
After installation:
Mongosh

This opens MongoDB shell.

3. Basic Database Commands

Show all databases
show dbs

Create / Switch database

use collegeDB // If it doesn’t exist, MongoDB creates it automatically.

Check current database

db

4. Collection Commands
Create collection
[Link]("students")

Show collections
show collections

5. Insert Data
Insert one document
[Link]({
name: "Sushma",
age: 22,
 course: "MCA"
})

Insert multiple documents

[Link]([
{ name: "Amit", age: 23 },
{ name: "Neha", age: 21 }
])

6. Read Data (Query)

Find all data
[Link]()

Pretty format
[Link]().pretty()

Find with condition

[Link]({ age: 22 })

Find one
[Link]({ name: "Sushma" })

7. Update Data
Update one document
[Link](
{ name: "Sushma" },
{ $set: { age: 23 } }
)

Update many
[Link](
{ course: "MCA" },
{ $set: { status: "active" } }
)

8. Delete Data
Delete one
[Link]({ name: "Amit" })

Delete many
[Link]({ age: { $lt: 22 } })
9. Useful Operators
Comparison
{ age: { $gt: 20 } } // greater than
{ age: { $lt: 25 } } // less than

Logical
{ $and: [ { age: 22 }, { course: "MCA" } ] }
{ $or: [ { age: 22 }, { age: 23 } ] }

10. Drop (Delete Completely)

Drop collection
[Link]()

Drop database
[Link]()

11. Practice Examples

use collegeDB

[Link]([
{ name: "Sushma", age: 22 },
{ name: "Rahul", age: 24 }
])

[Link]()

[Link](
{ name: "Rahul" },
{ $set: { age: 25 } }
)

[Link]({ name: "Sushma" })

MongoDB Data Types
Data Type Description Example
String Text values { name: "Sushma" }
Integer Whole numbers { age: 22 }
Double Decimal numbers { marks: 85.5 }
Boolean True/False { isActive: true }
Array List of values { subjects: ["DBMS", "AI"] }
Object Embedded document { address: { city: "Mumbai" } }
Null No value { middleName: null }
ObjectId Unique document ID { _id: ObjectId("...") }
Date Date values { dob: new Date("2002-01-01") }

Relational (Comparison) Operators

Operator Meaning Example
$eq Equal to { age: { $eq: 22 } }
$ne Not equal to { age: { $ne: 22 } }
$gt Greater than { age: { $gt: 20 } }
$gte Greater than or equal { age: { $gte: 22 } }
$lt Less than { age: { $lt: 25 } }
$lte Less than or equal { age: { $lte: 22 } }
$in Match any value in list { age: { $in: [20, 22] } }
$nin Not in list { age: { $nin: [18, 19] } }

Logical Operators
Operator Meaning Example
All conditions must { $and: [ { age: 22 }, {
$and
be true course: "MCA" } ] }
{ $or: [ { age: 22 }, { age:
$or Any condition true
23 } ] }
$not Negates condition { age: { $not: { $gt: 22 } } }
{ $nor: [ { age: 22 }, {
$nor All conditions false
course: "MCA" } ] }
Combined Example
Query:

Find students enrolled for MCA course and age is greater than 20.

[Link]({
$and: [
{ age: { $gt: 20 } },
{ course: "MCA" }
]
})
 Aggregation Pipeline examples in MongoDB with outputs.

Sample Data
[Link]([
{ name: "Sushma", age: 22, marks: 85, course: "MCA" },
{ name: "Rahul", age: 24, marks: 78, course: "MCA" },
{ name: "Neha", age: 21, marks: 92, course: "BCA" },
{ name: "Amit", age: 23, marks: 60, course: "BCA" }
])

1. $match (Filter)
� Query:
[Link]([
{ $match: { marks: { $gt: 80 } } }
])

� Output:
[
{ "name": "Sushma", "age": 22, "marks": 85, "course": "MCA" },
{ "name": "Neha", "age": 21, "marks": 92, "course": "BCA" }
]

2. $project (Select Fields)

� Query:
[Link]([
{ $project: { name: 1, marks: 1, _id: 0 } }
])

� Output:
[
{ "name": "Sushma", "marks": 85 },
{ "name": "Rahul", "marks": 78 },
{ "name": "Neha", "marks": 92 },
{ "name": "Amit", "marks": 60 }
]

3. $group (Average Marks per Course)

� Query:
[Link]([
{
$group: {
_id: "$course",
avgMarks: { $avg: "$marks" }
}
}
])

� Output:
[
{ "_id": "MCA", "avgMarks": 81.5 },
{ "_id": "BCA", "avgMarks": 76 }
]

4. $sort (Descending Order)

� Query:
[Link]([
{ $sort: { marks: -1 } }
])

� Output:
[
{ "name": "Neha", "marks": 92 },
{ "name": "Sushma", "marks": 85 },
{ "name": "Rahul", "marks": 78 },
{ "name": "Amit", "marks": 60 }
]

5. $limit
� Query:
[Link]([
{ $sort: { marks: -1 } },
{ $limit: 2 }
])

� Output:
[
{ "name": "Neha", "marks": 92 },
{ "name": "Sushma", "marks": 85 }
]

6. $count
� Query:
[Link]([
{ $count: "totalStudents" }
])

� Output:
[
{ "totalStudents": 4 }
]

7. $sum (Total Marks per Course)

� Query:
[Link]([
{
$group: {
_id: "$course",
totalMarks: { $sum: "$marks" }
}
}
])

� Output:
[
{ "_id": "MCA", "totalMarks": 163 },
{ "_id": "BCA", "totalMarks": 152 }
]

8. $max and $min

� Query:
[Link]([
{
$group: {
_id: null,
maxMarks: { $max: "$marks" },
minMarks: { $min: "$marks" }
}
}
])

� Output:
[
{ "_id": null, "maxMarks": 92, "minMarks": 60 }
]

9. Combined Pipeline �
� Query:
[Link]([
{ $match: { course: "MCA" } },
{
$group: {
_id: "$course",
avgMarks: { $avg: "$marks" }
}
}
])

� Output:
[
{ "_id": "MCA", "avgMarks": 81.5 }
]

10. Top Student per Course

� Query:
[Link]([
{ $sort: { marks: -1 } },
{
$group: {
_id: "$course",
topStudent: { $first: "$name" },
highestMarks: { $first: "$marks" }
}
}
])

� Output:
[
{ "_id": "BCA", "topStudent": "Neha", "highestMarks": 92 },
{ "_id": "MCA", "topStudent": "Sushma", "highestMarks": 85 }
]
 MapReduce in MongoDB
 Used for processing large data
 Works in 3 steps:
1. Map → Emits key-value pairs
2. Reduce → Aggregates values
3. Result → Final output collection

Concept Meaning
emit(key, value) Sends data from map
Map Function Processes each document
Reduce Function Aggregates values
Finalize Function Optional post-processing
out Output collection name

Sample Data
[Link]([
{ name: "Sushma", course: "MCA", marks: 85 },
{ name: "Rahul", course: "MCA", marks: 78 },
{ name: "Neha", course: "BCA", marks: 92 },
{ name: "Amit", course: "BCA", marks: 60 }
])
1. Total Marks per Course

Map Function:

var mapFunction = function() {

emit([Link], [Link]);
};

Reduce Function:
var reduceFunction = function(key, values) {
return [Link](values);
};

Query:

[Link](
mapFunction,
reduceFunction,
{ out: "total_marks" }
)

Output (in total_marks collection):

[
{ "_id": "MCA", "value": 163 },
{ "_id": "BCA", "value": 152 }
]
2. Count Students per Course

Map:

var mapFunction = function() {

emit([Link], 1);
};

Reduce:

var reduceFunction = function(key, values) {

return [Link](values);
};

Query:

[Link](
mapFunction,
reduceFunction,
{ out: "student_count" }
)

Output:

[
{ "_id": "MCA", "value": 2 },
{ "_id": "BCA", "value": 2 }
]
3. Average Marks per Course
Map:

var mapFunction = function() {

emit([Link], { total: [Link], count: 1 });
};

Reduce:

var reduceFunction = function(key, values) {

var result = { total: 0, count: 0 };
[Link](function(v) {
[Link] += [Link];
[Link] += [Link];
});
return result;
};

Finalize (to calculate average):

var finalizeFunction = function(key, reducedValue) {

return [Link] / [Link];
};

Query:

[Link](
mapFunction,
reduceFunction,
{
out: "avg_marks",
finalize: finalizeFunction
}
)

Output:

[
{ "_id": "MCA", "value": 81.5 },
{ "_id": "BCA", "value": 76 }
]
4. Maximum Marks per Course

Map:

var mapFunction = function() {

emit([Link], [Link]);
};
Reduce:

var reduceFunction = function(key, values) {

return [Link](null, values);
};

Query:

[Link](
mapFunction,
reduceFunction,
{ out: "max_marks" }
)

Output:

[
{ "_id": "MCA", "value": 85 },
{ "_id": "BCA", "value": 92 }
]
 Notes
Subject: Database Management Systems Lab

TE Comp. 2012 Course, SPPU

MongoDB: Indexing, Aggregations, Map-
Reduce
By : Prof. Sushma R. Vispute

PCET’s Pimpri Chinchwad College of Engineering

Department of Computer Engineering
 Indexing

•Indexes support the efficient resolution of queries.

•MongoDB must scan every document of
collection to select those documents that match the
query statement.
•This requires mongodb to process a large volume
of data.

[Link] M
 Indexing

• Indexes are special data structures, that stores

small portion of data set in an easy to traverse
form.

•Index stores the value of specific field or set of

fields ordered by the value of the field as specified
in index.

[Link] M
 Indexing

• Index are created using ensureIndex() method.

Syntax:
db.COLLECTION_NAME.ensureIndex({KEY:1})
Key – name of field on which index is to be
created.
1 – asecending order
-1- descending order

[Link] M
 Indexing

Example-
db.Teacher_Info.ensureIndex({“Teacher_id”:1})

[Link] M
 Aggregation

• Aggregation operation group values from

multiple document together, and can perform
variety of operations on the grouped data to return
a single result.
•To perform aggregation operation aggregate()
method is used.

[Link] M
 Aggregation

Syntax:
Db.COLLECTION_NAME.aggregate(AGGREGAT
E_OPERATION)
Example:
Db.Teacher_Info.aggregate([{$group:{_id:”$DeptN
ame”, Total_sal : {$sum: “$sal”}}}])

[Link] M
 Aggregation Expressions
•SUM
•AVG
•MIN
•MAX
•FIRST
•LAST

[Link] M
 Pipeline
•Pipeline - output of one operation is given input
to another operation.
•Mongodb uses concept of pipeline in
aggregation framework.
•There can be set of possible stages. At each stage
set of documents are taken as input and produce a
resulting set of documents. The resulted set of
documents then in turn again be used for next
stage and so on.

[Link] M
 Stages in aggregation framework

Following are the possible stages in aggregation framework −

$project − Used to select some specific fields from a collection.

$match − This is a filtering operation and thus this can reduce the
amount of documents that are given as input to the next stage.
$group − This does the actual aggregation as discussed above.
$sort − Sorts the documents.
$skip − With this, it is possible to skip forward in the list of documents
for a given amount of documents.
$limit − This limits the amount of documents to look at, by the given
number starting from the current positions.
Pipeline Example

[Link] M
 Map Reduce
• A way to divide large query computations into
smaller chunks.
•May run in multiple processes on multiple
machines.
•Used for processing large data sets.
•MapReduce() command is used to peform
mapreduce operation.

[Link] M
•MapReduce Example

•RED 1
•BLUE 2
•gReen 5
•Red 11 •BLUE 14
•Yellow 2 •GREEN 14
•ReD 17 •RED 42
•bLue 12 •YELLOW 12
•GREEN 9
•Red 13

•Input •Output
•Copyright © 2014 IBM
Corporation
•Map

•RED 1 •RED 1
•BLUE 2 •BLUE 2
•gReen 5 •GREEN 5
•Red 11 •RED 11
•Yellow 2 •YELLOW 2
•ReD 17 •RED 17
•bLue 12 •BLUE 12
•GREEN 9 •MAP •GREEN 9
•Red 13 •RED 13

•Copyright © 2014 IBM

Corporation
•Sort

•RED 1 •BLUE 2
•BLUE 2 •BLUE 12
•GREEN 5 •GREEN 5
•RED 11 •GREEN 9
•YELLOW 2 •RED 1
•RED 17 •RED 11
•BLUE 12 •SORT •RED 17
•GREEN 9 •RED 13
•RED 13 •YELLOW 2

•Copyright © 2014 IBM

Corporation
•Partition

•BLUE 2 •BLUE
•BLUE 2
•BLUE 12 •BLUE
•BLUE 12
•GREEN 5
•GREEN 5
•GREEN 9 •GREEN 9
•RED 1
•RED 11 •RED 1
•PARTITION
•RED 11
•RED 17 •RED 17
•RED 13 •RED 13
•YELLOW 2
•YELLOW
•YELLOW 2

•Copyright © 2014 IBM

Corporation
•Reduce

•BLUE 2
•BLUE 12

•GREEN 5
•GREEN 9 •BLUE 14
•GREEN 14
•RED 1
•RED
•RED 42
11 •REDUCE •YELLOW 2
•RED 17
•RED 13

•YELLOW 2

•Copyright © 2014 IBM

Corporation
•Can I do this in parallel using multiple machines?

•Copyright © 2014 IBM

Corporation
•MapReduce Data Flow

•Copyright © 2014 IBM

Corporation
•MAP: (K1, V1) → list(K2, V2)

•Node1 •Node 2 •Node 3

•bLue 12
•RED 1 •Red 11
•GREEN 9
•BLUE 2 •Yellow 2
•Red 13
•gReen •ReD 17
5

•1 •RED 1 •19 •Red 11 •39 •bLue 12

•6 •BLUE 2 •25 •Yellow 2

•46 •GREEN 9

•12 •gReen 5 •33 •ReD 17 •53 •Red 13

•MAP •MAP •MAP

•RED •1 •RED •11 •BLUE
•BLUE •12

•BLUE
•BLUE •2 •YELLOW
•YELLOW •2 • GREEN •9

•GREEN •5 •RED •17 •RED •13

•Copyright © 2014 IBM

Corporation
•Reduce: (K2, list(V2)) → list(K3, V3)

•Node1 •Node2 •Node 3

•BLUE •12, 2 • GREEN •9,5 •RED •1,11,17,13

•YELLOW •2

•REDUCE •REDUCE •REDUCE

•BLUE •14 •GREEN •14 •RED •42

•YELLOW •2

•Copyright © 2014 IBM

Corporation
 Map Reduce
Following is the syntax of the basic mapReduce command:

>[Link](

function() {emit(key,value);}, //map function

function(key,values) {return reduceFunction}, //reduce function

{
out: collection,
query: document,
sort: document,
limit: number
}
)

[Link] M •Copyright © 2014 IBM

Corporation
 Map Reduce
Simple Syntax-
Db.COLLECTION_NAME.mapReduce(
Function() {emit(key,value);} // map function

Function(key,value){return reduceFunction}, //reduce function.

)

[Link] M
 Map Reduce
The map-reduce function first queries the collection, then maps the result
documents to emit key-value pairs which is then reduced based on the keys
that have multiple values.

In the above syntax:

map is a java script function that maps a value with a key and emits a key-
value pair

reduce is a java script function that reduces or groups all the documents
having the same key

out specifies the location of the map-reduce query result

query specifies the optional selection criteria for selecting documents

sort specifies the optional sort criteria

limit specifies the optional maximum number of documents to be returned

[Link] M
Map Reduce Example
Map Reduce Example
MapReduce Example with Query
MapReduce Example with Query