Course Outcome
Students will be able to use advanced database Programming concepts
Big Data –HADOOP
Unit-IV
Outline
❏ Introduction to NoSQL Database
❏ Types and examples of NoSQL Database
❏ CAP theorem and BASE Properties
❏ Comparative study of SQL and NoSQL
❏ NoSQL Data Models
❏ Case Study-unstructured data from social media
❏ Introduction to Big Data, HADOOP: HDFS, MapReduce
Unit-IV
Introduction to NoSQL Database
NoSQL
is
Non-Relational NoSQL
DBMS
Is used for
Big data,
real-time web apps
Unit-IV
Introduction to NoSQL Database
Relational
Will see
Database
Analytical NoSQL
[OLAP] Databases
Unit-IV
Types of NoSQL Database
Document
Column-Family
Graph
Key-Value
Each type solves a problem that can’t be solved with
Unit-IV relational databases.
examples of NoSQL Database
Key-Value Based Document-Based
Example
Amazon DynamoDB Example
Oracle NoSQL MongoDB
Redis CouchDB
Unit-IV
IBM Cloudant
examples of NoSQL Database
Graph -Based
Column-Based
Example
Example InfoGrid
Cassandra
InfiniteGraph
Hbase
Neo4J
Unit-IV
HyperTable
Practical examples of NoSQL Database
Social networking
Ad-targeting
sites
Session Store E-commerce
Mobile Online shopping
Application sites
Unit-IV User Profile
Information
– C stand for Consistency,
CAP Theorem – A stand for Availability and
– P stand for Partition Tolerance.
❏ “Of three properties of a shared data system: data consistency, system availability
and tolerance to network partitions, only two can be achieved at any given
moment.”
❏ NoSQL database provides BASE property. (ACID Property is not supported)
❏ Consistency - all nodes see the same data at the same time
● Strict Consistency – RDBMS
● Tunable Consistency – Cassandra
● Eventual Consistency – Amazon Dynamo
Unit-IV
Unit-IV
Base properties of CAP theorem
Basically available:
Nodes in the a distributed environment can go
down, but the whole system shouldn’t be affected.
Soft State (scalable):
The state of the system and data changes over time
Eventual Consistency:
Given enough time, data will be consistent across
the distributed system.
Unit-IV
Comparative Study of NoSQL and SQL
SQL NoSQL
RELATIONAL DATABASE MANAGEMENT Non-relational or distributed database
SYSTEM system.
These databases have fixed or static or These databases are best suited for
predefined schema hierarchical data storage.
These databases are best suited for
These databases are not so good for
complex queries
complex queries
Vertically Scalable
Horizontally scalable
Unit-IV
Outline
❏ Review of Last Session
❏ Data Models of NoSQL
❏ Graph based Data Model
❏ Key-Value Pair based Data Model
❏ Column based Data Model
❏ Document based Data Model
❏ Overview Mongo-DB
Unit-IV
NoSQL Data Models
Key-Value Type Document-Based
Redis
MongoDB
value corresponds to a Key.
No table structure is used.
Unit-IV
NoSQL Data Models
Graph -Based
Column-Based
Cassandra
Neo4J
store an information about networks. database using Table. more suitable
application on aggregation and data
Unit-IV warehouse.
NoSQL Data Processing Tools
Graph-Based Data Model
Unit-IV
NoSQL Data Processing Tools
Key-value databases- Redis (CP)
❏ suitable for providing high performance
❏ The maximum of value limit to 1 GB. computing to small amount of data.
❏ main drawback is that capacity of the ❏ Support sql queries.
database is limited by physical memory.
❏ Simple values or data structures by keys
Unit-IV
NoSQL Data Processing Tools
Column-oriented database-Cassandra
❏ Multi-datacenter replication
❏ Support for map/reduce, good for analytics, data warehousing
❏ Tunable consistency & strong availability and partition tolerance (AP)
❏ No single point of failure
❏ Probably the easiest of this list to manage in big/growing clusters
Unit-IV
Cont NoSQL Data Processing Tools
Column-oriented database-Cassandra
Fact reading from database
Unit-IV
NoSQL Data Processing Tools
Document Databases-MongoDB
General Purpose Easy to use
Rich Data Model High Speed
Support complex Easy mapping to
data types object oriented code
Sophisticated query Simple to setup and manage
language reduce able to SQL
Unit-IV
NoSQL Data Models
Unit-VI
NoSQL Data Models
Unit-IV
SQL Vs MongoDB
SQL MongoDB
Unit-IV
Sharding with MongoDB
Replica
Sets
Unit-IV
Type of Data : Variety
❏ Structured, Unstructured, Semi-Structured and Quasi-Structured
Unit-IV
Need of Unstructured Data
❏ Need of databases which are able to store and process big data effectively
❏ demand for high performance when reading and writing
high concurrency applications.
Easy to expand
Big data analysis
High scalability
Data format.
Manageability
Unit-IV
What is Big Data
“A massive volume of both structured and unstructured data that is so large that it's
difficult to process with traditional database and software techniques.”
❏ Web sites with 300+ million unique visitors/month.
Criteria for considering Size ❏ Digital data from sensors used to
data as big data gather climate information
Type of Data
Latency Data
Complexity
Unit-IV ❏ Posts to social networking site
Handling Big Data
Storage
Security
Processing
Analysis
Unit-IV
3 Dimensions of Big Data
Unit-IV
6 Dimensions of Big Data
Unit-IV
References
[1] [Link]
[2] [Link]