What is NoSQL?

— Chapter Notes
Big Data Analytics, 2nd Edition — Radha Shankarmani & M. Vijayalakshmi (Wiley)

3.1 What is NoSQL?

NoSQL is a database management system built to store and retrieve huge volumes of unstructured data across a
distributed set of virtual servers, with a focus on scalability, performance, availability, and agility.
It emerged because relational databases struggle to scale and adapt quickly enough for the needs of modern
applications with large numbers of users, objects, and products.
NoSQL stands for “Not only SQL.” Most NoSQL systems are non-relational, schema-free, and avoid JOIN
operations, relying instead on objects, key-value pairs, or tuples.
Examples: SimpleDB, Google BigTable, Hadoop, MapReduce, MemcacheDB — around 150 NoSQL products
exist. Companies such as Netflix, LinkedIn, and Twitter use NoSQL for social-network data analysis.

Key Characteristics of NoSQL

• Next-generation database, different from traditional RDBMS
• “Not only SQL” — can combine SQL and other query languages
• Non-relational and schema-free
• Free of JOIN operations
• Distributed architecture using multiple processors
• Horizontally scalable
• Mostly open-source
• Data files can be easily replicated
• Uses simple APIs
• Can manage huge amounts of data
• Runs on commodity hardware (shared-nothing concept)

3.1.1 Why NoSQL?

Relational databases suit predictable, structured data. Modern applications need an agile system that can
dynamically process unstructured and unpredictable data. NoSQL favors BASE (Basically Available, Soft state,
Eventual consistency) over the strict ACID guarantees of RDBMS.

3.1.2 CAP Theorem (Brewer's Theorem)

A distributed system cannot guarantee all three of the following simultaneously:
1. Consistency — all nodes and their replicas have the same data at the same time
2. Availability — every request is guaranteed a success or failure response
3. Partition Tolerance — the system keeps operating despite network partition failures

3.2 NoSQL Business Drivers

Four pressures push organizations from RDBMS toward NoSQL: Volume, Velocity, Variability, and Agility.

3.2.1 Volume
Query needs outgrow single-CPU RDBMS systems. Horizontal scaling across clusters of commodity machines
(enabled by technologies such as Hadoop, HDFS, MapR, HBase) addresses this.
3.2.2 Velocity
Real-time read/write demand — e.g., traffic bursts on e-commerce and social networking sites — overwhelms
single-CPU RDBMS systems, especially when many columns are indexed.

3.2.3 Variability
RDBMS fixed schemas struggle with uncommon or sparse attributes. Altering a schema (ALTER TABLE)
requires the current transaction to finish and the database to be closed, affecting availability.

3.2.4 Agility
Complex, nested queries in RDBMS require heavy object-relational mapping layers (e.g., Java Hibernate), which
slows development, testing, and change requests.

Desirable Business-Driving Features of NoSQL

4. 24×7 Data availability — fault tolerance through data and function replication; no single point of failure
5. Location transparency — read/write to a storage node regardless of physical location
6. Schema-less data model — accepts structured, semi-structured, and unstructured data
7. Modern-day transaction analysis — does not require strict ACID; consistency is defined by the CAP
theorem instead
8. Architecture that suits Big Data — supports scale of data sources, real-time speed, new storage models
(HDFS, Cassandra, MongoDB, Neo4j), and multiple compute methods
9. Analytics and business intelligence — provides integrated analytics for fast decision-making from large,
complex datasets

3.3 NoSQL Case Studies

Four case studies are discussed, each representing a different NoSQL architectural pattern.
Case Study Pattern Key Points
Amazon DynamoDB Key-Value Handles shopping-cart/session data; scalable, auto storage scaling,
shared-nothing, built-in fault tolerance, strong read consistency, atomic
counters, integrates with Redshift & Elastic MapReduce.
Google BigTable Column-Family Built for massive scale (petabytes, ~100,000 nodes); built atop Google
File System, Chubby Lock, MapReduce; sparse, distributed, persistent,
multi-dimensional sorted map indexed by row key, column key,
timestamp.
MongoDB Document Document-based (not relational) for speed, agility, schema flexibility;
stores JSON-like documents in BSON; supports embedded fields and
indexing.
Neo4j Graph Open-source, Java/Scala-based graph DB; stores nodes and
relationships with properties; efficient for traversing highly connected
data; ACID-compliant, unlike many graph libraries.

3.3.1 Amazon DynamoDB (Key-Value)

Originally, Amazon used an RDBMS for its shopping cart and checkout system. DynamoDB addresses
performance, scalability, and reliability problems that arise as data grows. It automatically saves data across
multiple servers and Availability Zones. Cart and session data are stored in the key-value store; the completed
order is saved in an RDBMS.
Salient features: scalable, automated storage scaling, distributed shared-nothing architecture, built-in fault
tolerance (synchronous replication across zones), flexible schema-free format, efficient indexing via
primary/secondary keys, strong read consistency with atomic counters, secure access control, resource-
consumption monitoring, and integration with Amazon Redshift and Elastic MapReduce.
3.3.2 Google's BigTable (Column-Family)
Built for massive scalability and to run on commodity hardware, handling petabytes of data distributed over
roughly 100,000 nodes. It is built on top of the Google File System, Scheduler, MapReduce, and Chubby Lock
Service. BigTable is a distributed, persistent, multi-dimensional sorted map indexed by row key, column key, and
timestamp.

3.3.3 MongoDB (Document Store)

Designed by Eliot Horowitz and team at 10gen, MongoDB moves from a relational data model to a document-
based model to achieve speed, manageability, agility, schema-less storage, and easier horizontal scaling. It stores
data as JSON-style documents (in BSON format) and supports indexing of embedded fields.

3.3.4 Neo4j (Graph Store)

An open-source graph database implemented in Java and Scala, publicly available since 2007. Neo4j stores nodes
(entities) and relationships (edges), each of which can hold key-value properties. It efficiently traverses highly
connected data and, unlike many graph libraries, supports full ACID transaction compliance, making it suitable
for production use.

3.4 NoSQL Data Architectural Patterns

3.4.1 Types of NoSQL Data Stores
There are four main types of NoSQL data stores: Key-Value Store, Column-Family Store, Document Store, and
Graph Store.

1. Key-Value Store
A simple hash table mapping a key to a value (string, JSON, BLOB, etc.). The key can be artificially generated or
auto-generated.
Core operations: Get(key), Put(key, value), Multi-get(key1, key2, …, keyN), Delete(key).
Rules: all keys are unique, and no queries can be run directly on values.
Weakness: favors Availability and Partition tolerance over Consistency; cannot update part of a value or query on
values; maintaining unique keys becomes harder as data volume grows.
Examples: Redis, Amazon Dynamo, Azure Table Storage, Riak, Memcache.
Typical uses: dictionaries, image stores, lookup tables, query caches.

2. Column-Family (Wide-Column) Store

Popularized by Apache Cassandra (developed at Facebook by Avinash Lakshman and Prashant Malik, open-
sourced in 2008). Data is stored by columns grouped into column families rather than by rows — well suited to
sparse-matrix data and OLAP-style column aggregation.
Data model: <key, value> = <(row key, column family, column name, timestamp), value>.
In Cassandra terminology: a keyspace is roughly equivalent to a database, and a column family is the basic unit of
data organization within it (similar to a table).
Examples: Google BigTable, HBase, Cassandra, Accumulo, HyperTable.

3. Document Store
Extends the key-value idea: the stored “value” is a structured document (typically JSON/BSON), and — unlike
key-value or column-family stores — the contents are automatically indexed, making them searchable.
A document path acts like a key to drill into nested tree structures within a document.
Examples: MongoDB, CouchDB, Couchbase.
MongoDB stores data in BSON format; documents with similar structure are grouped into collections (analogous
to RDBMS tables). Embedded documents and arrays reduce the need for JOINs, improving performance.

4. Graph Store
Stores data as nodes (entities) and edges (relationships), each of which can carry key-value properties. Optimized
for efficiently traversing complex, highly connected relationships rather than computing them expensively at
query time.
Example: Neo4j.

Remaining Chapter Sections (for reference)

The chapter continues with the following topics, which can be covered in a follow-up set of notes:
• 3.5 Variations of NoSQL Architectural Patterns
• 3.6 Using NoSQL to Manage Big Data
• 3.6.1 What is a Big Data NoSQL Solution?
• 3.6.2 Understanding Types of Big Data Problems
• 3.6.3 Analyzing Big Data with a Shared-Nothing Architecture
• 3.6.4 Choosing Distribution Models
• 3.6.5 Four Ways that NoSQL Systems Handle Big Data Problems