Module-4

Document Databases

NoSQL Database (21CS752): Module-3

 Overview

Document Databases, What Is a Document Database?

• Features, Consistency, Transactions, Availability, Query
Features, Scaling, Suitable Use Cases, Event Logging
• Content Management Systems, Blogging Platforms, Web
Analytics or Real-Time Analytics, Ecommerce Applications
• When Not to Use, ComplexTransactions Spanning Different
• Operations, Queries againstVarying Aggregate Structure
2
NoSQL Database (21CS752): Module-3
 Document Databases
1. Documents are the main concept in document databases.
2. The database stores and retrieves documents, which can be XML, JSON,
BSON, and so on.
3. These documents are self-describing, hierarchical tree data structures which
can consist of maps, collections, and scalar values.
4. The documents stored are similar to each other but do not have to be exactly
the same.
5. Document databases store documents in the value part of the key-value
store; think about document databases as key-value stores where the value
3
NoSQL Database (21CS752): Module-3
 Document Databases
1. Let’s look at how terminology compares in Oracle and MongoDB

4
NoSQL Database (21CS752): Module-3
 What Is a Document Database?

1. The above document can

be considered a row in a
traditional RDBMS. Let’s
look at another
document:
5
NoSQL Database (21CS752): Module-3
 Document Databases
1. Looking at the documents, we can see that they are similar, but have
differences in attribute names.
2. This is allowed in document databases.
3. The schema of the data can differ across documents, but these documents
can still belong to the same collection—unlike an RDBMS where every row
in a table has to follow the same schema.
4. We represent a list of citiesvisited as an array, or a list of addresses as list of
documents embedded inside the main document.
5. Embedding child documents as subobjects inside documents provides for
6
NoSQL Database (21CS752): Module-3
 Document Databases
1. If you look at the documents, you will see that some of the attributes are
similar, such as firstname or city.
2. At the same time, there are attributes in the second document which do not
exist in the first document, such as addresses, while likes is in the first
document but not the second.
3. This different representation of data is not the same as in RDBMS where
every column has to be defined, and if it does not have data it is marked as
empty or set to null.

7
NoSQL Database (21CS752): Module-3
 Document Databases
1. In documents, there are no empty attributes; if a given attribute is not found,
we assume that it was not set or not relevant to the document.
2. Documents allow for new attributes to be created without the need to define
them or to change the existing documents
3. Some of the popular document databases we have seen are MongoDB
[MongoDB], CouchDB [CouchDB], Terrastore [Terrastore], OrientDB
[OrientDB], RavenDB [RavenDB], and of course the well-known and often
reviled Lotus Notes [Notes Storage Facility] that uses document storage

8
NoSQL Database (21CS752): Module-3
 Features
1. Consistency
2. Transactions
3. Availability
4. Query Features
5. Scaling

9
NoSQL Database (21CS752): Module-3
 Consistency
1. Consistency in MongoDB database is configured by using the replica sets
and choosing to wait for the writes to be replicated to all the slaves or a given
number of slaves.
2. Every write can specify the number of servers the write has to be propagated
to before it returns as successful.
3. Example: A command like [Link]({ getlasterror : 1 , w :
"majority" }) tells the database how strong is the consistency you want.

10
NoSQL Database (21CS752): Module-3
 Consistency
1. For example, if you have one server and specify the w as majority, the write
will return immediately since there is only one node.
2. If you have three nodes in the replica set and specify w as majority, the write
will have to complete at a minimum of two nodes before it is reported as a
success.
3. You can increase the w value for stronger consistency, but you will suffer on
write performance, since now the writes have to complete at more nodes.
4. Replica sets also allow you to increase the read performance by allowing
reading from slaves by setting slaveOk; this parameter can be set on the
11
NoSQL Database (21CS752): Module-3
 Consistency

12
NoSQL Database (21CS752): Module-3
 Consistency
1. Similar to various options available for read, you can change the settings to
achieve strong write consistency, if desired.
2. By default, a write is reported successful once the database receives it; you
can change this so as to wait for the writes to be synced to disk or to
propagate to two or more slaves.
3. This is known as WriteConcern: You make sure that certain writes are written
to the master and some slaves by setting WriteConcern to
REPLICAS_SAFE.
4. Shown below is code where we are setting the WriteConcern for all writes to
13
NoSQL Database (21CS752): Module-3
 Consistency

14
NoSQL Database (21CS752): Module-3
 Transactions
1. Transactions, in the traditional RDBMS sense, mean that you can start
modifying the database with insert, update, or delete commands over
different tables and then decide if you want to keep the changes or not by
using commit or rollback.
2. These constructs are generally not available in NoSQL solutions—a write
either succeeds or fails.
3. Transactions at the single-document level are known as atomic transactions.
4. Transactions involving more than one operation are not possible, although
there are products such as RavenDB that do support transactions across
15
NoSQL Database (21CS752): Module-3
 Transactions
1. By default, all writes are reported as successful.
2. A finer control over the write can be achieved by usingWriteConcern
parameter.
3. We ensure that order is written to more than one node before it’s reported
successful by using WriteConcern.REPLICAS_SAFE.
4. Different levels of WriteConcern let you choose the safety level during writes;
for example, when writing log entries, you can use lowest level of safety,
[Link]

16
NoSQL Database (21CS752): Module-3
 Transactions

17
NoSQL Database (21CS752): Module-3
 Availability
1. The CAP theorem (“The CAP Theorem,” p. 53) dictates that we can have
only two of Consistency, Availability, and Partition Tolerance.
2. Document databases try to improve on availability by replicating data using
the master-slave setup.
3. The same data is available on multiple nodes and the clients can get to the
data even when the primary node is down.
4. Usually, the application code does not have to determine if the primary node
is available or not.
5. MongoDB implements replication,providing high availability using replica
18
NoSQL Database (21CS752): Module-3
 Availability

1. In a replica set, there are two or more nodes participating in an

asynchronous master-slave replication.
2. The replica-set nodes elect the master, or primary, among themselves.
3. Assuming all the nodes have equal voting rights, some nodes can be favored
for being closer to the other servers, for having more RAM, and so on; users
can affect this by assigning a priority—a number between 0 and
4. 1000—to a node.

19
NoSQL Database (21CS752): Module-3
 Availability

1. All requests go to the master node, and the data is replicated to the slave
nodes.
2. If the master node goes down, the remaining nodes in the replica set vote
among themselves to elect a new master; all future requests are routed to
the new master, and the slave nodes start getting data from the new master.
3. When the node that failed comes back online, it joins in as a slave and
catches up with the rest of the nodes by pulling all the data it needs to get
current
20
NoSQL Database (21CS752): Module-3
 Availability

1. Figure 9.1 is an example configuration of replica sets. We have two nodes,

mongo A and mongo B, running the MongoDB database in the primary data-
center, and mongo C in the secondary datacenter.
2. If we want nodes in the primary datacenter to be elected as primary nodes,
we can assign them a higher priority than the other nodes.
3. More nodes can be added to the replica sets without having to take them
offline.

21
NoSQL Database (21CS752): Module-3
 Availability

22
NoSQL Database (21CS752): Module-3
 Availability
1. The application writes or reads from the primary (master) node. When
connection is established, the application only needs to connect to one node
(primary or not, does not matter) in the replica set, and the rest of the nodes
are discovered automatically.
2. When the primary node goes down, the driver talks to the new primary elected
by the replica set.
3. The application does not have to manage any of the communication failures or
node selection criteria.
4. Using replica sets gives you the ability to have a highly available document
23
data store.
NoSQL Database (21CS752): Module-3
 Availability
1. Replica sets are generally used for data redundancy, automated failover, read
scaling, server maintenance without downtime, and disaster recovery.
2. Similar availability setups can be achieved with CouchDB, RavenDB,
Terrastore, and other products.

24
NoSQL Database (21CS752): Module-3
 Query Features
1. Document databases provide different query features. CouchDB allows you to
query via views—complex queries on documents which can be either
materialized (“Materialized Views,” p. 30) or dynamic (think of them as RDBMS
views which are either materialized or not).
2. With CouchDB, if you need to aggregate the number of reviews for a product
as well as the average rating, you could add a view implemented via map-
reduce (“Basic Map-Reduce,” p. 68) to return the count of reviews and the
average of their ratings.

25
NoSQL Database (21CS752): Module-3
 Query Features
1. When there are many requests, you don’t want to compute the count and
average for every request; instead you can add a materialized view that
precomputes the values and stores the results in the database.
2. These materialized views are updated when queried, if any data was changed
since the last update.
3. One of the good features of document databases, as compared to key-value
stores, is that we can query the data inside the document without having to
retrieve the whole document by its key and then introspect the document.
4. This feature brings these databases closer to the RDBMS query model
26
NoSQL Database (21CS752): Module-3
 Query Features
1. MongoDB has a query language which is expressed via JSON and has
constructs such as $query for the where clause, $orderby for sorting the data,
or $explain to show the execution plan of the query.
2. There are many more constructs like these that can be combined to create a
MongoDB query
3. Let’s look at certain queries that we can do against MongoDB.
4. Suppose we want to return all the documents in an order collection (all rows in
the order table).
5. The SQL for this would be:
27
NoSQL Database (21CS752): Module-3
 Query Features

28
NoSQL Database (21CS752): Module-3
 Query Features
1. Similarly, queries to count, sum, and so on are all available. Since the
documents are aggregated objects, it is really easy to query for documents that
have to be matched using the fields with child objects.
2. Let’s say we want to query for all the orders where one of the items ordered
has a name like Refactoring.
3. The SQL for this requirement would be:

29
NoSQL Database (21CS752): Module-3
 Scaling
1. The idea of scaling is to add nodes or change data storage without simply migrating the
database to a bigger box.
2. We are not talking about making application changes to handle more load; instead, we
are interested in what features are in the database so that it can handle more load.
3. Scaling for heavy-read loads can be achieved by adding more read slaves, so that all the
reads can be directed to the slaves.
4. Given a heavy-read application, with our 3-node replica-set cluster, we can add more
read capacity to the cluster as the read load increases just by adding more slave nodes
to the replica set to execute reads with the slaveOk flag (Figure 9.2).
5. This is horizontal scaling for reads.
30
NoSQL Database (21CS752): Module-3
 Scaling

31
NoSQL Database (21CS752): Module-3
 Scaling
1. Once the new node, mongo D, is started, it needs to be added to the replica set

2. When a new node is added, it will sync up with the existing nodes, join the replica set as
secondary node, and start serving read requests.
3. An advantage of this setup is that we do not have to restart any other nodes, and there is
no downtime for the application either.
4. When we want to scale for write, we can start sharding (“Sharding,” p. 38) the data.
5. Sharding is similar to partitions in RDBMS where we split data by value in a certain
column, such as state or year.

32
NoSQL Database (21CS752): Module-3
 Scaling
1. With RDBMS, partitions are usually on the same node, so the client application does not
have to query a specific partition but can keep querying the base table; the RDBMS
takes care of finding the right partition for the query and returns the data.
2. In sharding, the data is also split by certain field, but then moved to different Mongo
nodes.
3. The data is dynamically moved between nodes to ensure that shards are always
balanced.
4. We can add more nodes to the cluster and increase the number of writable nodes,
enabling horizontal scaling for writes.

33
NoSQL Database (21CS752): Module-3
 Scaling
1. Splitting the data on the first name of the customer ensures that the data is balanced
across the shards for optimal write performance; furthermore, each shard can be a
replica set ensuring better read performance within the shard (Figure 9.3).
2. When we add a new shard to this existing sharded cluster, the data will now be balanced
across four shards instead of three.
3. As all this data movement and infrastructure refactoring is happening, the application will
not experience any downtime, although the cluster may not perform optimally when large
amounts of data are being moved to rebalance the shards

34
NoSQL Database (21CS752): Module-3
 Scaling

35
NoSQL Database (21CS752): Module-3
 Scaling
1. Splitting the data on the first name of the customer ensures that the data is balanced
across the shards for optimal write performance; furthermore, each shard can be a
replica set ensuring better read performance within the shard (Figure 9.3).
2. When we add a new shard to this existing sharded cluster, the data will now be balanced
across four shards instead of three.
3. As all this data movement and infrastructure refactoring is happening, the application will
not experience any downtime, although the cluster may not perform optimally when large
amounts of data are being moved to rebalance the shards

36
NoSQL Database (21CS752): Module-3
 Suitable Use Cases
1. Event Logging:
1. Applications have different event logging needs; within the enterprise, there are
many different applications that want to log events.
2. Document databases can store all these different types of events and can act as a
central data store for event storage.
3. This is especially true when the type of data being captured by the events keeps
changing.
4. Events can be sharded by the name of the application where the event originated or
by the type of event such as order_processed or customer_logged.

37
NoSQL Database (21CS752): Module-3
 Suitable Use Cases
1. Content Management Systems, Blogging Platforms:
1. Since document databases have no predefined schemas and usually understand
JSON documents, they work well in content management systems or applications for
publishing websites, managing user comments, user registrations, profiles, web-
facing documents.
2. Web Analytics or Real-Time Analytics:
1. Document databases can store data for real-time analytics; since parts of the
document can be updated, it’s very easy to store page views or unique visitors, and
new metrics can be easily added without schema changes.

38
NoSQL Database (21CS752): Module-3
 Suitable Use Cases
1. E-Commerce Applications:
1. E-commerce applications often need to have flexible schema for products and
orders, as well as the ability to evolve their data models without expensive
database refactoring or data migration

39
NoSQL Database (21CS752): Module-3
 When Not to Use
1. There are problem spaces where document databases are not the best solution
2. Complex Transactions Spanning Different Operations: If you need to have atomic
cross-document operations, then document databases may not be for you. However,
there are some document databases that do support these kinds of operations, such as
RavenDB.

40
NoSQL Database (21CS752): Module-3
 When Not to Use
1. Queries against Varying Aggregate Structure: Flexible schema means that the
database does not enforce any restrictions on the schema.
2. Data is saved in the form of application entities.
3. If you need to query these entities ad hoc, your queries will be changing (in RDBMS
terms, this would mean that as you join criteria between tables, the tables to join keep
changing).
4. Since the data is saved as an aggregate, if the design of the aggregate is constantly
changing, you need to save the aggregates at the lowest level of granularity—basically,
you need to normalize the data.
5. In this scenario, document databases may not work
41
NoSQL Database (21CS752): Module-3
 End of Module - 04

42
NoSQL Database (21CS752): Module-3