Copyright © 2016 Ramez Elmasri and Shamkant B.
Navathe
CHAPTER 24
NOSQL Databases
and Big Data Storage Systems
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe
Introduction
NOSQL
Not only SQL
Most NOSQL systems are distributed databases
or distributed storage systems
Focus on semi-structured data storage, high
performance, availability, data replication, and
scalability
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 3
Introduction (cont’d.)
NOSQL systems focus on storage of “big data”
Typical applications that use NOSQL
Social media
Web links
User profiles
Marketing and sales
Posts and tweets
Road maps and spatial data
Email
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 4
24.1 Introduction to NOSQL Systems
BigTable
Google’s proprietary NOSQL system
Column-based or wide column store
DynamoDB (Amazon)
Key-value data store
Cassandra (Facebook)
Uses concepts from both key-value store and
column-based systems
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 5
Introduction to NOSQL Systems
(cont’d.)
MongoDB and CouchDB
Document stores
Neo4J and GraphBase
Graph-based NOSQL systems
OrientDB
Combines several concepts
Database systems classified on the object model
Or native XML model
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 6
Introduction to NOSQL Systems
(cont’d.)
NOSQL characteristics related to distributed
databases and distributed systems
Scalability
Availability, replication, and eventual consistency
Replication models
Master-slave
Master-master
Sharding of files
High performance data access
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 7
Introduction to NOSQL Systems
(cont’d.)
NOSQL characteristics related to data models
and query languages
Schema not required
Less powerful query languages
Versioning
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 8
Introduction to NOSQL Systems
(cont’d.)
Categories of NOSQL systems
Document-based NOSQL systems
NOSQL key-value stores
Column-based or wide column NOSQL systems
Graph-based NOSQL systems
Hybrid NOSQL systems
Object databases
XML databases
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 9
24.2 The CAP Theorem
Various levels of consistency among replicated
data items
Enforcing serializabilty the strongest form of
consistency
High overhead – can reduce read/write operation
performance
CAP theorem
Consistency, availability, and partition tolerance
Not possible to guarantee all three simultaneously
In distributed system with data replication
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 10
The CAP Theorem (cont’d.)
Designer can choose two of three to guarantee
Weaker consistency level is often acceptable in
NOSQL distributed data store
Guaranteeing availability and partition tolerance
more important
Eventual consistency often adopted
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 11
24.3 Document-Based NOSQL
Systems and MongoDB
Document stores
Collections of similar documents
Individual documents resemble complex objects
or XML documents
Documents are self-describing
Can have different data elements
Documents can be specified in various formats
XML
JSON
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 12
MongoDB Data Model
Documents stored in binary JSON (BSON) format
Individual documents stored in a collection
Example command
First parameter specifies name of the collection
Collection options include limits on size and
number of documents
Each document in collection has unique ObjectID
field called _id
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 13
MongoDB Data Model (cont’d.)
A collection does not have a schema
Structure of the data fields in documents chosen
based on how documents will be accessed
User can choose normalized or denormalized
design
Document creation using insert operation
Document deletion using remove operation
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 14
Figure 24.1 (continues)
Example of simple documents in
MongoDB (a) Denormalized
document design with embedded
subdocuments (b) Embedded
array of document references
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 15
Figure 24.1 (cont’d.)
Example of simple
documents in MongoDB
(c) Normalized documents
(d) Inserting the
documents in Figure
24.1(c) into their
collections
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 16
MongoDB Distributed Systems
Characteristics
Two-phase commit method
Used to ensure atomicity and consistency of
multidocument transactions
Replication in MongoDB
Concept of replica set to create multiple copies on
different nodes
Variation of master-slave approach
Primary copy, secondary copy, and arbiter
Arbiter participates in elections to select new
primary if needed
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 17
MongoDB Distributed Systems
Characteristics (cont’d.)
Replication in MongoDB (cont’d.)
All write operations applied to the primary copy
and propagated to the secondaries
User can choose read preference
Read requests can be processed at any replica
Sharding in MongoDB
Horizontal partitioning divides the documents into
disjoint partitions (shards)
Allows adding more nodes as needed
Shards stored on different nodes to achieve load
balancing
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 18
MongoDB Distributed Systems
Characteristics (cont’d.)
Sharding in MongoDB (cont’d.)
Partitioning field (shard key) must exist in every
document in the collection
Must have an index
Range partitioning
Creates chunks by specifying a range of key values
Works best with range queries
Hash partitioning
Partitioning based on the hash values of each shard
key
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 19
24.4 NOSQL Key-Value Stores
Key-value stores focus on high performance,
availability, and scalability
Can store structured, unstructured, or semi-
structured data
Key: unique identifier associated with a data item
Used for fast retrieval
Value: the data item itself
Can be string or array of bytes
Application interprets the structure
No query language
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 20
DynamoDB Overview
DynamoDB part of Amazon’s Web Services/SDK
platforms
Proprietary
Table holds a collection of self-describing items
Item consists of attribute-value pairs
Attribute values can be single or multi-valued
Primary key used to locate items within a table
Can be single attribute or pair of attributes
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 21
Voldemort Key-Value Distributed
Data Store
Voldemort: open source key-value system similar
to DynamoDB
Voldemort features
Simple basic operations (get, put, and delete)
High-level formatted data values
Consistent hashing for distributing (key, value)
pairs
Consistency and versioning
Concurrent writes allowed
Each write associated with a vector clock
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 22
Figure 24.2 Example of consistent
hashing (a) Ring having three nodes
A, B, and C, with C having greater
capacity. The h(K) values that map to
the circle points in range 1 have their
(k, v) items stored in node A, range 2
in node B, range 3 in node C (b)
Adding a node D to the ring. Items in
range 4 are moved to the node D
from node B (range 2 is reduced) and
node C (range 3 is reduced)
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 23
Examples of Other Key-Value Stores
Oracle key-value store
Oracle NOSQL Database
Redis key-value cache and store
Caches data in main memory to improve
performance
Offers master-slave replication and high
availability
Offers persistence by backing up cache to disk
Apache Cassandra
Offers features from several NOSQL categories
Used by Facebook and others
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 24
24.5 Column-Based or Wide Column
NOSQL Systems
BigTable: Google’s distributed storage system for
big data
Used in Gmail
Uses Google File System for data storage and
distribution
Apache Hbase a similar, open source system
Uses Hadoop Distributed File System (HDFS) for
data storage
Can also use Amazon’s Simple Storage System
(S3)
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 25
Hbase Data Model and Versioning
Data organization concepts
Namespaces
Tables
Column families
Column qualifiers
Columns
Rows
Data cells
Data is self-describing
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 26
Hbase Data Model and Versioning
(cont’d.)
HBase stores multiple versions of data items
Timestamp associated with each version
Each row in a table has a unique row key
Table associated with one or more column
families
Column qualifiers can be dynamically specified
as new table rows are created and inserted
Namespace is collection of tables
Cell holds a basic data item
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 27
Figure 24.3 Examples in Hbase (a) Creating a table called EMPLOYEE with three column
families: Name, Address, and Details (b) Inserting some in the EMPLOYEE table;
different rows can have different self-describing column qualifiers (Fname, Lname,
Nickname, Mname, Minit, Suffix, … for column family Name; Job, Review, Supervisor,
Salary for column family Details). (c) Some CRUD operations of Hbase
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 28
Hbase Crud Operations
Provides only low-level CRUD (create, read,
update, delete) operations
Application programs implement more complex
operations
Create
Creates a new table and specifies one or more
column families associated with the table
Put
Inserts new data or new versions of existing data
items
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 29
Hbase Crud Operations (cont’d.)
Get
Retrieves data associated with a single row
Scan
Retrieves all the rows
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 30
Hbase Storage and Distributed
System Concepts
Each Hbase table divided into several regions
Each region holds a range of the row keys in the
table
Row keys must be lexicographically ordered
Each region has several stores
Column families are assigned to stores
Regions assigned to region servers for storage
Master server responsible for monitoring the
region servers
Hbase uses Apache Zookeeper and HDFS
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 31
24.6 NOSQL Graph Databases and
Neo4j
Graph databases
Data represented as a graph
Collection of vertices (nodes) and edges
Possible to store data associated with both
individual nodes and individual edges
Neo4j
Open source system
Uses concepts of nodes and relationships
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 32
Neo4j (cont’d.)
Nodes can have labels
Zero, one, or several
Both nodes and relationships can have properties
Each relationship has a start node, end node,
and a relationship type
Properties specified using a map pattern
Somewhat similar to ER/EER concepts
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 33
Neo4j (cont’d.)
Creating nodes in Neo4j
CREATE command
Part of high-level declarative query language
Cypher
Node label can be specified when node is created
Properties are enclosed in curly brackets
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 34
Neo4j (cont’d.)
Figure 24.4 Examples in Neo4j using the Cypher language (a) Creating some nodes
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 35
Neo4j (cont’d.)
Figure 24.4 (cont’d.) Examples in Neo4j using the Cypher language
(b) Creating some relationships
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 36
Neo4j (cont’d.)
Path
Traversal of part of the graph
Typically used as part of a query to specify a
pattern
Schema optional in Neo4j
Indexing and node identifiers
Users can create for the collection of nodes that
have a particular label
One or more properties can be indexed
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 37
The Cypher Query Language of
Neo4j
Cypher query made up of clauses
Result from one clause can be the input to the
next clause in the query
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 38
The Cypher Query Language of
Neo4j (cont’d.)
Figure 24.4 (cont’d.) Examples in Neo4j using the Cypher language
(c) Basic syntax of Cypher queries
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 39
The Cypher Query Language of
Neo4j (cont’d.)
Figure 24.4 (cont’d.) Examples in
Neo4j using the Cypher language
(d) Examples of Cypher queries
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 40
Neo4j Interfaces and Distributed
System Characteristics
Enterprise edition versus community edition
Enterprise edition supports caching, clustering of
data, and locking
Graph visualization interface
Subset of nodes and edges in a database graph
can be displayed as a graph
Used to visualize query results
Master-slave replication
Caching
Logical logs
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 41
24.7 Summary
NOSQL systems focus on storage of “big data”
General categories
Document-based
Key-value stores
Column-based
Graph-based
Some systems use techniques spanning two or
more categories
Consistency paradigms
CAP theorem
Copyright © 2016 Ramez Elmasri and Shamkant B. Navathe Slide 24- 42