NoSQL Databases

NoSQL stands for "Not Only SQL."

NoSQL databases are designed to store and retrieve data in formats other
than the tabular relations used in relational databases. Common types
include:

Document-oriented databases (e.g., MongoDB, CouchDB): Store data in

document formats like JSON or BSON, making them flexible for
hierarchical data.

Key-value stores (e.g., Redis, DynamoDB): Store simple key-value pairs,

ideal for high-speed retrieval.

Column-family stores (e.g., Cassandra, HBase): Use column families to

group related data, useful in analytics and wide-column storage.

Graph databases (e.g., Neo4j, ArangoDB): Store data as nodes and

relationships, well-suited for highly connected data.
When to Use NoSQL vs. SQL

Use SQL/RDBMS if:

• You need strict data consistency in transactions. (Such as

banking, finance, and inventory management systems)
• The data structure is well-defined and not likely to
change.
• Complex querying and joining of tables are necessary.

Use NoSQL if:

• You need to handle large, unstructured data sets or

schema-less data.
• Data models and requirements may change frequently.
• Scalability and high-speed data access are essential.
Criteria Use SQL (RDBMS) Use NoSQL
Structured, well-defined Unstructured or semi-
Data Type
schema structured data
Fixed schema (changes are Schema-less and highly
Schema Flexibility
complex) flexible

Banking, finance, inventory Social media, IoT, big data

Use Case
management analytics

Simple queries, often key-

Querying Complex queries and joins
based
Strong consistency (ACID Eventual consistency
Consistency
transactions) (BASE)
Vertical (add more Horizontal (add more
Scalability
CPU/RAM) servers)

Performance Goal Accuracy and reliability Speed and scalability

Traditional enterprise Cloud, distributed, or real-

Typical Environment
systems time systems
SQL vs NoSQL

Feature Relational Databases (SQL) NoSQL Databases

Structured tables with rows and Various models (document, key-
Data Model
columns (fixed schema) value, column-family, graph)

Schema Flexibility Fixed schema Schema-less or flexible schema,

easy to change
Horizontal scaling (add servers,
Vertical scaling (add resources to
Scalability often distributed) Automatic
a single server)
Sharding)
Supports BASE (Basically
Data Integrity Strong ACID (Atomicity, Available, Soft-state, Eventually
Consistency, Isolation, Durability) consistent)
Various, often custom (e.g.,
Query Language Structured Query Language (SQL) MongoDB’s BSON, Cassandra
CQL)
Applications requiring strict Big Data, real-time analytics,
Use Cases
consistency content management, social apps
Moving from Serial to Parallel Processing

Serial Processing
• Tasks are performed one after another
• Only one operation happens at a time
• Example: Task1 → Task2 → Task3 → Task4

Parallel Processing
• Multiple tasks are performed at the same time
• Uses several processors, cores, or machines
• Example: Task1, Task2, Task3, Task4 (run simultaneously)

Benefits:
• Increased speed and throughput
• Improves scalability in distributed and big data systems
Analogy:
• Serial → One cashier serving one customer
• Parallel → Multiple cashiers serving customers simultaneously
 Scalability refers to a system’s ability to handle increased load or demand by
adjusting its resources.

Feature Horizontal Scalability Vertical Scalability

Add more machines/nodes. Clusters of
commodity processors.
Approach Upgrade a single machine
Moving from serial to parallel
processing.
System Type Distributed system Centralized system
High (fault tolerance with multiple Dependent on single
Reliability
nodes) machine’s availability
Simpler to manage, but
Complexity Requires distributed management
with single point of failure
Big Data, NoSQL databases, web Relational databases,
Use Cases
applications smaller applications

Vertical Scalability
Vertical scalability (scaling up) involves upgrading the hardware on a single
server, such as adding more CPU power, memory, or storage to handle increased
load.
Automatic Database Sharding

• As data volume grows, a single database may no longer handle all

operations efficiently.
• Database Sharding – breaks a large database into smaller parts called
shards.
• Each shard is stored on a different distributed server.
• Enables systems to scale horizontally and manage massive datasets.
• Helps reduce system downtime and improve performance.
• Commonly used in large-scale web applications and cloud databases.
Distributed computing environment (DCE)
Refers to a network of independent computers that work together to achieve
a common goal, allowing large-scale processing, storage, and analysis by
leveraging multiple machines.

This environment supports distributed processing of tasks across systems,

making it well-suited for handling big data, high-performance applications,
and resource-intensive tasks like scientific simulations, financial modeling,
and cloud services.
Comparing ACID and BASE—two methods of reliable
database transactions
Let’s start with a simple banking example to represent a reliable
transaction. These days, many people have two bank accounts:
savings and checking. If you want to move funds from one account
to the other, it’s likely your bank has a transfer form on their
website.
When you click the Transfer button on the web page, two discrete operations
must happen in unison. The funds are subtracted from your savings account
and then added to your checking account. Transaction management is the
process of making sure that these two operations happen together as a
single unit of work or not at all.

The software also ensures that no reports can be run on the accounts
halfway through the operations. If you run a “combined balance” report
during the transaction, it’d never show a total that drops by 1,000 and then
increases again. If a report starts while the first part of the transaction is in
process, it’ll be blocked until all parts of the transaction are complete.
RDBMS transaction control using ACID Principles
RDBMSs maintain transaction control by using atomic, consistent, independent, and
durable (ACID) properties to ensure transactions are reliable

Atomicity—In the banking transaction example, we said that the exchange of funds
from savings to checking must happen as an all-or-nothing transaction.

Consistency—In the banking transaction example, we talked about the fact that when
moving funds between two related accounts, the total account balance must never
change. This is the principle of consistency. It means that your database must never
have a report that shows the withdrawal from savings has occurred but the addition to
checking hasn’t.

Isolation—Isolation refers to the concept that each part of a transaction occurs

without knowledge of any other transaction. For example, the transaction that adds
funds doesn’t know about the transaction that subtracts funds from an account.

Durability—Durability refers to the fact that once all aspects of a transaction are
complete, it’s permanent. Once the transfer button is selected, you have the right to
spend the money in your checking account. If the banking system crashes that night
and they have to restore the database from a backup tape, there must be some way to
make sure the record of this transfer is also restored.
ACID systems focus on the consistency and integrity of data above
all other considerations.
What if you have a website that relies on computers all over the world? A
computer in Chicago manages your inventory, product photos are on an
image database in Virgina, tax calculations are performed in Seattle, and your
accounting system is in Atlanta.

What if one site goes down? Should you tell your customers to check back in
20 minutes while you solve the problem? Only if your goal is to drive them to
your competitors. Is it realistic to use ACID software for every order that
comes in?

Let’s look at another option. Websites that use the “shopping cart” and
“checkout” constructs have a different primary consideration when it comes
to transaction processing. The issue of reports that are inconsistent for a few
minutes is less important than something that prevents you from taking an
order, because if you block an order, you’ve lost a customer.
Non-RDBMS transaction control using BASE principles
BASE principles are specifically designed to suit the scalability and flexibility
requirements of NoSQL databases

Basic availability: The system guarantees availability of data, meaning it

responds to every request, though it may not reflect the most recent
updates. Rather than providing a guarantee of consistency, it ensures that
some form of the data will always be accessible

Soft-state: The "Soft State" aspect means that the data within the system
may change or be inconsistent for a period of time, even without new input.
In other words, the state of the system is "soft" or flexible rather than being
"hard" or rigidly consistent at any given moment.

Eventual consistency: The system will eventually become consistent, given

enough time. This principle assumes that if no new updates are made to a
given data item, all copies of that item will converge to the same value over
time
• Unlike RDBMSs that focus on consistency, BASE systems focus on
availability
• BASE systems are noteworthy because their number-one objective is to
allow new data to be stored, even at the risk of being out of sync for
a short period of time.

• They relax the rules and allow reports to run even if not all portions of the
database are synchronized.

• BASE systems aren’t considered pessimistic in that they don’t fret about
the details if one process is behind. They’re optimistic in that they
assume that eventually all systems will catch up and become consistent.

BASE systems are ideal for web storefronts, where filling a shopping cart and
placing an order is the main priority.
Q. You are designing a real-time social media application that supports
millions of users posting updates, commenting, and liking posts across
different regions. Given the high volume of interactions, some actions, like
updating the like counts on posts, may not be reflected immediately across
all nodes. However, ensuring that users can post updates and interact with
others without delays is a top priority.

• Explain how BASE properties can support this social media application
in managing high traffic while maintaining user engagement.

• What are the potential trade-offs of this approach?

Answer: BASE Properties in the Social Media Application
Basically Available:
The application must ensure that users can always post updates and interact with
content, even if some background processes (like updating like counts or
comment threads) are still being processed. By prioritizing availability, the system
allows users to submit posts and likes without facing downtime or delays in
accessing the application.
Soft State:
The system can accept that some user actions (like liking a post) may not
immediately reflect the updated state across all nodes or servers. For instance, a
user may like a post, but the like count may not update instantly for all users
viewing that post. This temporary inconsistency is acceptable since the
application is designed to handle updates gradually, allowing for smoother user
interactions.
Eventually Consistent:
After a user interacts with a post (e.g., likes or comments), the system will
eventually propagate these updates throughout the network. This means that
over time, all nodes will synchronize and reflect the correct counts and
interactions. For instance, while one user might see an old like count, the system
will ensure that all counts are consistent after a short period, providing a
seamless experience.
Trade-off

means giving up one thing to gain another, you can’t have both fully at the
same time.

Example:

• If you make a system faster (high availability), you might have to sacrifice
accuracy (data may be slightly outdated).

• If you want perfect accuracy, you might have to sacrifice speed or availability.
Trade-offs Involved

Aspect Description Trade-off Involved

Users may see outdated like High Availability & Speed (system
counts or comment threads. remains responsive)
Data Accuracy Example: After liking a post, the Reduced Real-time Accuracy
count may not update instantly, (data may appear temporarily
causing confusion. inconsistent)

System remains always

accessible, but users might face Continuous Operation
User Experience
delayed feedback after actions Short Delays in Feedback
like liking or commenting.

Developers must handle

Flexible & Scalable System
temporary inconsistencies and
Complexity of Data Management Higher Implementation
ensure eventual consistency
Complexity
through additional logic.
Understanding trade-offs with Brewer’s CAP theorem

Used to make critical decisions while designing a distributed system.

Eric Brewer first introduced the CAP theorem in 2000. The CAP theorem
states that any distributed database system can have at most two of the
following three desirable properties:

Consistency
Availability
Partition tolerance
 Understanding trade-offs with Brewer’s CAP
theorem

Eric Brewer first introduced the CAP theorem in 2000. The CAP theorem states that any distributed
database system can have at most two of the following three desirable properties:

Consistency, Availability,
Partition tolerance
Consistency—Every read receives the most
recent write or an error.

Having a single, up-to-date, readable version of your data available to all clients.

• When one user updates something, others should see that change right away.

• If the data isn’t updated everywhere yet, the system may show an error or wait until
everything matches.
• This isn’t the same as the consistency we talked about in ACID. Consistency here is concerned
with multiple clients reading the same items from replicated partitions and getting consistent
results.
• Some locking mechanism is used until the data is synced across nodes.
• Ex: In an online trading app, all users must see the same stock price at the same time - no
outdated prices.
Understanding trade-offs with Brewer’s CAP
theorem
Eric Brewer first introduced the CAP theorem in 2000. The CAP theorem states
that any distributed database system can have at most two of the following
three desirable properties:
Consistency
Availability
Partition tolerance
High availability Every request receives a non-error message without the
guarantee of containing the most recent write.
The system is always working and ready to respond.

Knowing that the distributed database will always allow database clients to
update items without delay.

Even if servers can’t talk to each other, the system still accepts updates.

Example: A shopping website keeps working during server issues - you can still
place orders, even if the latest stock count updates later.
Understanding trade-offs with Brewer’s CAP
theorem
Eric Brewer first introduced the CAP theorem in 2000. The CAP theorem states
that any distributed database system can have at most two of the following
three desirable properties:

Consistency
Availability
Partition tolerance

Partition tolerance The ability of the system to keep responding to client

requests even if there’s a communication failure between database
partitions.

This is analogous to a person still having an intelligent conversation even

after a link between parts of their brain isn’t working.

Which is almost always required.

Example:- If a server in another city goes offline, your online app still works, and
later all data gets synced.
 Meaning What It
Property (Simple Words) What It Ensures Sacrifices / Example
Limitation

All users see the Data is correct May delay Online trading
responses until app – everyone
Consistency (C) same, latest
data at the and updated data matches must see the
same time. everywhere. across all same stock
servers. price.

The system is May show Shopping site

always up and Users always slightly lets you place
Availability (A) responding, get a quick outdated data an order even if
even if some reply (no (not the newest inventory sync
servers fail. downtime). version). is delayed.

The system May lose Messaging app

keeps working Works despite consistency or still lets you
Partition even if network network availability send messages
Tolerance (P) connections failures or lost during the when some
break between connections. failure. servers are
servers. unreachable.
The CAP theorem helps you understand that once you partition your data, you
must consider the availability-consistency spectrum in a network failure situation.

Then the CAP theorem allows you to determine which options best match your
business requirements.
Normal Operation:

A client writes to the master node and the change is replicated to the slave node. Clients
can read from the slave node, ensuring both consistency and availability as long as the
link is up.​

High-Availability Option:

If the link between master and slave nodes goes down, the system can still accept writes
at the master node, prioritizing availability. However, this risks inconsistent reads from
the slave node because the slave may not have the latest updates.

Example :- Posting photos or status updates on social media. Even if part of the system
(like some servers or connections) is down, you can still post your update. Other users
may not see your new post immediately, but the service remains available and you don’t
experience an error.

Consistency Option:

If consistency is strictly required, the system refuses writes while the link is down. This
maintains data consistency, but sacrifices availability because clients cannot write until
the network partition is resolved.

Example:- Bank transfers during outages. If branches lose connection, new transfers
may be blocked until connections are restored, ensuring that your account balance is not
updated inconsistently or duplicated.
Tools like the CAP theorem can help guide database selection discussions within an
organization and prioritize what properties (consistency, availability, and scalability)
are most important.

If high consistency and update availability are simultaneously required, then a faster
single processor might be your best choice.

If you need the scale-out benefits that distributed systems offer, then you can make
decisions about your need for update availability versus read consistency for each
transaction type.
Q. You are tasked with developing a healthcare management system that enables
patients to schedule appointments, access medical records, and communicate with
healthcare providers. The system is distributed across multiple regions to ensure high
availability and low latency. However, during peak usage times or unexpected outages,
network partitions between data centers can occur. Considering the CAP theorem
(Consistency, Availability, and Partition Tolerance), how would you make design decisions
regarding the system's behavior during network partition scenarios?

a) Which two properties (from Consistency, Availability, and Partition Tolerance) would
you prioritize during network partition scenarios, and why?

b) What are the trade-offs of prioritizing these two properties in the context of a
healthcare management system?
Answer: Prioritized Properties during Network Partition Scenarios

Prioritized Properties: Availability and Partition Tolerance

Availability: In a healthcare system, it is crucial that patients can access

their medical records, schedule appointments, and communicate with
healthcare providers without experiencing downtime. Ensuring that the
system remains available, even during network partitions, helps maintain
continuous access to critical health information and services.

Partition Tolerance: Given the distributed nature of the system across

multiple data centers, the ability to continue operating despite network
partitions is essential. The system should be designed to function
effectively, even when some parts of the network are unable to
communicate.
b) Trade-offs of Prioritizing Availability and Partition Tolerance

Eventual Consistency:
By prioritizing availability, the system may allow temporary inconsistencies
in data. For instance, if a patient updates their medical records, that update
might not immediately reflect in all nodes. This could lead to healthcare
providers accessing outdated information, potentially affecting patient care.

Risk of Conflicting Information:

With an AP system, there is a risk that different data centers might hold
conflicting versions of data during a partition. For example, if two
healthcare providers are accessing patient records simultaneously, one
might see an updated medication list while the other does not, leading to
potential confusion or miscommunication.
Consistency is deprioritized in favor of Availability and Partition
Tolerance (AP) during network partitions because:

Immediate Access to Healthcare Services: In healthcare, accessibility is

often critical. Patients need to access their records, schedule
appointments, or communicate with providers, sometimes urgently. If
strict Consistency were prioritized, a network partition would require the
system to prevent certain actions (like updating records) until all data
centers synchronize, causing delays. This delay could impact patient care,
which is often time-sensitive.
Partition Tolerance is Non-Negotiable in Distributed Systems. Since the
system spans multiple regions to ensure low latency and broad
accessibility, it has to be Partition Tolerant by design to handle inevitable
network issues

Eventual Consistency in Non-Critical Scenarios: Many healthcare

operations can accept a short delay in data consistency during a partition if
it means maintaining service availability.
You are tasked with developing an online banking system that enables customers
to view account balances, make transfers, and manage transactions. The system
is distributed across multiple regions to ensure high availability and low latency.
However, during peak usage times or unexpected outages, network partitions
between data centers can occur. Considering the CAP theorem (Consistency,
Availability, and Partition Tolerance), how would you make design decisions
regarding the system's behavior during network partition scenarios?

Which two properties (from Consistency, Availability, and Partition Tolerance)

would you prioritize during network partition scenarios, and why?

What are the trade-offs of prioritizing these two properties in the context of a
banking system?
You are tasked with developing an online banking system that enables customers
to view account balances, make transfers, and manage transactions. The system
is distributed across multiple regions to ensure high availability and low latency.
However, during peak usage times or unexpected outages, network partitions
between data centers can occur. Considering the CAP theorem (Consistency,
Availability, and Partition Tolerance), how would you make design decisions
regarding the system's behavior during network partition scenarios?

Which two properties (from Consistency, Availability, and Partition Tolerance)

would you prioritize during network partition scenarios, and why?

What are the trade-offs of prioritizing these two properties in the context of a
banking system?
Q. You are designing a global online multiplayer gaming platform where players from
around the world can connect and play real-time games. The platform has servers
located in multiple regions to provide low-latency gaming experiences and support
millions of concurrent users.
During peak gaming hours or unexpected server issues, network partitions may occur
between data centers, causing some regions to temporarily lose communication with
others. However, the game needs to remain accessible for users in all regions, and
players expect continuous, uninterrupted gameplay, even if they are temporarily
isolated from other players or certain game data.

a) Which two properties (from Consistency, Availability, and Partition Tolerance)

would you prioritize during network partition scenarios?

b) What are the trade-offs of prioritizing these two properties in the context of a
global online gaming platform?
Answer is availability and partition tolerance

In the context of the global online multiplayer gaming platform, Consistency

refers to the property where all players, across different regions or data centers,
see the same game state at any given point in time. This means that when an
action is performed by one player, such as making a move or scoring a point, that
action should be reflected consistently across all players’ views of the game