NOSQL DATABASE
MODULE 4
Document Databases
Documents are the main concept in document databases. The database stores and retrieves
documents, which can be XML, JSON, BSON, and so on.
These documents are self-describing, hierarchical tree data structures which can consist
of maps, collections, and scalar values.
Oracle MongoDB
database instance MongoDB instance
schema database
table collection
row document
rowid _id
join DBRef
Some of the popular document databases we have seen are
1. MongoDB,
2. CouchDB,
3. Terrastore,
4. OrientDB,
5. RavenDB,
6. Notes [Notes Storage Facility] that uses document storage.
In the document data model, each document has a key-value pair.
Example,
{
"Name" : "Yashodhra",
"Address" : [{
“State” : “Uttar Pradesh”,
“City” : “Noida”
} ],
"Email" : "yahoo123@[Link]",
"Contact" : "12345"
}
The schema of the data can differ across documents, but these documents can still belong
to the same collection—unlike an RDBMS where every row in a table has to follow the same
schema.
This different representation of data is not the same as in RDBMS where every column has to
be defined, and if it does not have data it is marked as empty or set to null.
In documents, there are no empty attributes; if a given attribute is not found, we assume that
it was not set or not relevant to the document.
Features of Document Databases
1. Consistency
Consistency in MongoDB database is configured by using the replica sets.
Every write can specify the number of servers the write has to be propagated to before it
returns as successful.
A command like [Link]({ getlasterror : 1 , w : "majority" }) tells the database
how strong is the consistency you want.
For example, if you have one server and specify the “w” as majority, the write will return
immediately since there is only one node.
If you have three nodes in the replica set and specify “w” as majority, the write will have to
complete at a minimum of two nodes before it is reported as a success.
We can increase the “w” value for stronger consistency.
Replica sets also allow you to increase the read performance by allowing reading from
slaves by setting slaveOk; this parameter can be set on the connection, or database, or
collection, or individually for each operation.
Mongo mongo = new Mongo("localhost:27017");
[Link]();
Here we are setting slaveOk per operation, so that we can decide which operations can work
with data from the slave node.
DBCollection collection = getOrderCollection();
BasicDBObject query = new BasicDBObject();
[Link]("name", "Martin");
DBCursor cursor = [Link](query).slaveOk();
By default, a write is reported successful once the database receives it; you can change this so
as to wait for the writes to be synced to disk or to propagate to two or more slaves. This is
known as WriteConcern:
You make sure that certain writes are written to the master and some slaves by setting
WriteConcern to REPLICAS_SAFE.
DBCollection shopping = [Link]("shopping");
[Link](REPLICAS_SAFE);
WriteConcern can also be set per operation by specifying it on the save command:
WriteResult result = [Link](order, REPLICAS_SAFE);
2. Transactions
Transactions at the single-document level are known as atomic transactions. Transactions in
NoSQL is — either a write succeeds or fails.
By default, all writes are reported as successful. A finer control over the write can be
achieved by using WriteConcern parameter. We ensure that order is written to more than one
node before it’s reported successful by using WriteConcern.REPLICAS_SAFE.
Different levels of WriteConcern let you choose the safety level during writes;
For example, when writing log entries, you can use the lowest level of safety,
[Link].
final Mongo mongo = new Mongo(mongoURI);
[Link](REPLICAS_SAFE);
DBCollection shopping = [Link](orderDatabase)
.getCollection(shoppingCollection);
try {
WriteResult result = [Link](order, REPLICAS_SAFE);
} catch (MongoException writeException) {
dealWithWriteFailure(order, writeException);
}
3. Availability
The CAP theorem dictates that we can have only two of Consistency, Availability, and
Partition Tolerance. Document databases try to improve on availability by replicating data
using the master-slave setup.
The same data is available on multiple nodes and the clients can get to the data even when the
primary node is down.
MongoDB implements replication, providing high availability using replica sets.
In a replica set, there are two or more nodes participating in an asynchronous master-slave
replication. The replica-set nodes elect the master, or primary, among themselves.
All the nodes have equal voting rights, some nodes can be favored for being closer to the
other servers, for having more RAM, and so on; users can affect this by assigning a
priority—a number between 0 and 1000—to a node.
All requests go to the master node, and the data is replicated to the slave nodes. If the master
node goes down, the remaining nodes in the replica set vote among themselves to elect a new
master; all future requests are routed to the new master, and the slave nodes start getting data
from the new master.
When the node that failed comes back online, it joins in as a slave and catches up with the
rest of the nodes by pulling all the data it needs to get current.
The application writes or reads from the primary (master) node. When connection is
established,the application only needs to connect to one node (primary) in the replica set, and
the rest of the nodes are discovered automatically.
When the primary node goes down, the driver talks to the new primary elected by the replica
set. The application does not have to manage any of the communication failures or node
selection criteria.
Replica sets are generally used for—
● data redundancy,
● automated failover,
● read scaling,
● server maintenance without downtime, and
● disaster recovery.
Similar availability setups can be achieved with CouchDB, RavenDB, Terrastore, and
other products.
4. Query features
MongoDB has a query language which is expressed via JSON and has constructs such
as $query : for the where clause,
$orderby : for sorting the data, or
$explain : to show the execution plan of the query.
Suppose we want to return all the documents in an order collection. The SQL for this would
be:
SELECT * FROM order
The equivalent query in Mongo shell would be:
[Link]()
Selecting the orders for a single customerId of 883c2c5b4e5b would be:
SELECT * FROM order WHERE customerId = "883c2c5b4e5b"
The equivalent query in Mongo to get all orders for a single customerId 883c2c5b4e5b:
[Link]({"customerId":"883c2c5b4e5b"})
Similarly, selecting orderId and orderDate for one customer in SQL would be:
SELECT orderId,orderDate FROM order WHERE customerId = "883c2c5b4e5b"
and the equivalent in Mongo would be:
[Link]({customerId:"883c2c5b4e5b"},{orderId:1,orderDate:1})
Since the documents are aggregated objects, it is really easy to query for documents that have
to be matched using the fields with child objects.
Let’s say we want to query for all the orders where one of the items ordered has a name like
Refactoring. The SQL for this requirement would be:
SELECT * FROM customerOrder, orderItem, product
WHERE
[Link] = [Link]
AND [Link] = [Link]
AND [Link] LIKE '%Refactoring%'
and the equivalent Mongo query would be:
[Link]({"[Link]":/Refactoring/})
The query for MongoDB is simpler because the objects are embedded inside a single
document and you can query based on the embedded child documents.
5. Scaling
The idea of scaling is to add nodes or change data storage without simply migrating the
database to a bigger box.
Scaling for heavy loads —
Scaling for heavy-read loads can be achieved by adding more read slaves, so that all
the reads can be directed to the slaves.
Given a heavy-read application, in 3-node replica-set cluster, we can add more read
capacity to the cluster as the read load increases just by adding more slave nodes to
the replica set using the slaveOk flag. This is horizontal scaling for reads.
Once the new node, mongo D, is started, it needs to be added to the replica set.
[Link]("mongod:27017");
When a new node is added, it will sync up with the existing nodes, join the replica set
as a secondary node, and start serving read requests.
Advantages -
we do not have to restart any other nodes, and there is no downtime for the
application either.
Scaling for Write Loads via Sharding.
Sharding: This involves splitting data based on a specific key (e.g., firstname) and
distributing it across multiple nodes, or "shards." Each shard can also be configured as a
replica set to improve read performance within that shard.
[Link]({ shardcollection: "[Link]", key: {firstname: 1} })
distributes data across shards based on the firstname field.
Shards are automatically balanced by MongoDB to ensure an even distribution of data.
New shards can be added without — application downtime, although performance may be
temporarily affected while rebalancing occurs.
Each shard in a sharded cluster can be set up as a replica set, combining the benefits of
sharding with replication (as seen in Figure 9.3).
This setup enables improved read and write performance, as each shard can serve both as an
independent replica set and a distributed part of the overall dataset.
Sharding can also be based on user location, which places data closer to the users for faster
access, e.g., data for East Coast users in East Coast servers, and West Coast data on the West
Coast.
Suitable use cases
1. Event Logging
Applications have different event logging needs; within the enterprise, there are many
different applications that want to log events.
Document databases can store all these different types of events and can act as a central data
store fornevent storage.
2. Content Management Systems, Blogging Platforms
Since document databases have no predefined schemas and usually understand JSON
documents, they work well in content management systems or applications for publishing
websites, managing user comments, user registrations, profiles, web-facing documents.
3. Web Analytics or Real-Time Analytics
Document databases can store data for real-time analytics; since parts of the document can be
updated, it’s very easy to store page views or unique visitors, and new metrics can be easily
added without schema changes.
4. E-Commerce Applications
E-commerce applications often need to have flexible schema for products and orders, as well
as the ability to evolve their data models without expensive database refactoring or data
migration.
When not to use ?
1. Complex Transactions Spanning Different Operations
Document databases are typically not ideal for applications requiring atomic, multi-document
transactions. While some document databases (like RavenDB) support this, relational
databases are often better suited for applications with high transactional integrity
requirements.
2. Queries Against Varying Aggregate Structures
Flexible schema means that the database does not enforce any restrictions on the schema.
Data is saved in the form of application entities. If you need to query these entities ad hoc,
your queries will be changing.
Since the data is saved as an aggregate, if the design of the aggregate is constantly changing,
you need to save the aggregates at the lowest level of granularity—basically, you need to
normalize the data. In this scenario, document databases may not work.
Benefits of document databases:-
1. Flexible Schema Design
Document databases allow each document to have its own structure, enabling flexibility and
adaptability.
This flexibility reduces the need for costly schema migrations and supports rapid
development, especially for projects with evolving data requirements.
2. Ease of Scalability
Scalability in document databases enables them to handle large volumes of data and high
traffic, making them well-suited for applications needing high availability and scalability,
such as e-commerce or social media platforms.
3. Efficient Data Storage and Retrieval
Documents in a document database store related information together, allowing for faster
access. Data is often stored in a hierarchical structure with nested sub-documents, reducing
the need for joins.
It’s especially useful for applications requiring fast, complex queries, like content
management systems or personalized recommendation engines.
4. Support for Rich Data Types
Document databases can store a wide range of data types, including arrays, nested
documents, and various data structures within a single document.
JSON-like formats make them easy to read and use with many programming languages.