-Manthan

Single Point of Failure

A single point of failure (SPOF) is a part of a system that, if it fails, will cause the
entire system to stop functioning or fail. This means that the system relies on this
component, and there is no backup in place to prevent failure.
Some measures can be taken to reduce their impact like:-

1) Redundancy: Have multiple components in place so that if one node fails, the
others can take over.
2) Load Balancing: Distribute workloads across multiple components to prevent
any one component from becoming overwhelmed.
3) Master-Slave Architecture: Have a backup system in place to ensure that
operations can continue even if a primary system fails. Generally we take
backups of databases.
4) Regular Maintenance: Regular maintenance and testing can help identify and
prevent failures before they occur.
5) Disaster Recovery Planning: Have a plan in place for how to respond to
failures and how to quickly recover operations.

Scaling
Scalability is the measure of the ability of a system to increase or decrease in
performance and cost, in response to changes in application or with increasing load
and traffic on an existing system.

In order to build a scalable system we need to focus on these 3 areas:

1) System is able to handle increased load.
2) Not increase the complexity of the system.
3) Performance of the overall system should not take a hit.

It is of 2 types: -

1) Vertical Scaling(Buy Bigger Machine) -

Vertical scaling, also called “Scaling Up” refers to the process of adding more
resources to a single node in a system, such as increasing the amount of
 memory, storage, or processing power. This is done to improve the
performance and capacity of the system.
Advantages:-
● Interprocess Communication - All the processing occurs in a single
node, hence processing is faster.
● Data remains Consistent
It has some limitations, such as:-
● Finite limit to the amount of resources that can be added to a single
system can handle increased load without a decrease in performance.
Disadvantages:-
● Increased costs as the node becomes more powerful.
● Single Point of Failure

2) Horizontal Scaling(Buy more machines) -

Horizontal scaling, also called "Scaling Out" refers to the process of adding
more nodes to distribute workloads and improve capacity of the given system.
The goal is to spread the workload across multiple nodes, so that the
● Uses Network calls / Remote Procedure Calls(RPCs) - Nodes
communicate with each other using RPCs (RPC is a function call made
between two nodes when they are physically separated).
● Data Inconsistency.
It uses a Load Balancer to distribute incoming traffic across multiple nodes.
Advantages:-
● Resilient - If one machine fails, The requests are transported to other
nodes, hence no single point of failure occurs.
● Scales well as the number of users increase.

Difference between Horizontal and Vertical Scaling:-

Horizontal Scaling Vertical Scaling

1) Process of adding more nodes to 1) Process of adding more

distribute workloads. resources to a single node in a
system.

2) Load Balancer is required 2) No load balancing required.

3) Resilient System 3) Single Point of failure can occur

4) Uses Remote Procedure Calls 4) Interprocess Communication

(RPCs). occurs.

5) Data Inconsistency 5) Data is Consistent

6) It scales well as the number of 6) It can reach a hardware limit

users increases. (Cannot add more resources).
Hybrid Solution is to use Horizontal Scaling.

-Payal

Data Replication and Distributed Consensus

Data Replication is the process of generating numerous copies of data. We then
store these copies also called replicas in various locations for backup, fault
tolerance, and improved overall network accessibility. It uses master Slave
Architecture for generating multiple replicas.

Master Slave Architecture :-

Master-Slave Architecture enables data from one database server (The Master) to
be replicated to one or more other database servers (The Slaves).
There 2 types of data replication techniques:-
1. Synchronous Replication - In a master-slave architecture, the master device is
the sender and the slave device is the receiver. The master sends a request
and waits for a response from the slave before proceeding. The data in
master and slave should always be synchronised.
2. Asynchronous Replication - In a master-slave architecture, the master device
can send requests to the slave device at any time and continue with other
tasks without waiting for a response. The slave device processes the request
and sends a response when it is ready.

Peer to Peer Relation -

In peer to peer relation, master and slave both act as a master. They both have a
right to do write operations on databases.
We need to connect both the master databases so that data remains consistent, But
it can cause a Split Brain Problem.
This problem occurs when nodes communicate directly with each other to maintain
the consistency of data but the communication between nodes is disrupted and each
node may continue to process requests and update its data independently, leading
to multiple instances of data. This can result in data inconsistency as each instance
may have different updates to the same data.
To prevent the split brain problem, strategies like Distributed Consensus Algorithm
can be used.
Distributed Consensus Algorithm - It is a way where multiple nodes agree on a
particular value in order to remain in a consistent state. Each node must reach an
agreement on updates before they are applied.

Advantages of Master Slave Architecture:-

1. Keeps replica of data
2. Scales out read operations (can add multiple number of slaves)
3. Sharding

Heartbeats and Service Discovery in a

Distributed System

Heartbeats in a distributed system are messages sent by a health service between

nodes to verify their status and ensure that they are still operational. The purpose of
heartbeats is to detect failed nodes and allow the system to respond to such failures.
This can be done by marking the failed node as dead and reassigning its tasks to
other nodes, or by triggering a system-wide failover to a backup node. Heartbeats
are used to maintain reliability and availability.

Health service is closely tied to the Service Discovery problem. If a user requests a
service, that service is routed by a load balancer rather than by the service itself, and
it keeps the snapshot of all the data in its database or cache. Now the health service
will see a difference in the snapshot in the load balancer and then open’s up a new
connection with them and check their heartbeat..

Virtualization and Containers

Virtualization refers to the process of using software to create a virtual resource that
runs on a layer separate from the physical hardware. The most common use case of
virtualization is cloud computing.
Several VMs can be run on a computer through virtualization. These VMs are
independent systems but share the same physical IT infrastructure and are managed
by the hypervisor.

Containers are a means of isolating an application from its surroundings by

encapsulating its dependencies and configurations in a single unit.
Compared to machine or server virtualization approaches, a container does not
include images of the operating system. It makes them portable and lightweight with
fundamentally less overhead.
A container is a streamlined technique to test, build, redeploy, and deploy
applications over more than one environment from a local laptop of the developer to
a cloud or even the data center.

How to Avoid the Thundering Herd Problem?

The thundering herd problem can occur when a large number of clients suddenly
access a resource at the same time, such as a server or a database, causing it to
become overwhelmed and potentially fail. The different types of problems could be:-

1) Cascading Failure - Cascading failure refers to a phenomenon where the

failure of a component in a system triggers a chain reaction that leads to the
failure of multiple interdependent components.

TO AVOID CASCADING FAILURE WE CAN USE:

 ➔ Rate limiting: Limiting the rate at which requests are processed by a
system. By restricting the rate at which requests are processed using a
queue, rate limiting can help ensure that a system does not become
overwhelmed and fail as a result of a surge in demand.

2) Going Viral - During sales like black Friday events, load can increase on the
servers due to more Number of users logging in at once.

TO AVOID CRASHING OF SERVERS WE CAN USE:

➔ Pre-scale: Use extra servers and be ready beforehand.
➔ Auto-scale: This type of services can be provided by cloud service
providers. If the load on the system increases, the servers
automatically scale up (increase in number) in order to distribute the
load.
➔ Rate Limiting
★ Auto-scale is much better than pre-scale as installing servers requires a
significant cost and If the number of users are not as much as expected, then
there is no use of more servers.

3) Bulk Job Scheduling - During events like New Year or diwali etc. there are
cron jobs like sending email notifications to users. If all the notifications are
sent at once then the load on servers increases.

TO AVOID THIS WE USUALLY PREFER THE METHOD OF:-

➔ Batch processing: It is a mode of operation where large quantities of
data are processed in groups, rather than one record at a time. This
helps to reduce load and increase efficiency.

4) Popular Post - If a famous person uploads a video and we need to send

notifications to their subscribers. Notifications can be sent using Batch
Processing. But the problem can arrive when all the users start hitting the
page at once increasing its load.

TO AVOID FAILURE IN THESE SITUATIONS WE CAN USE:

➔ Jittering: This method helps to load by not showing some data that is
not of great importance so that other important data can flow
seamlessly. Like in youtube, the number of likes and views are not of
much importance. So we can not show some data in real time but
display an approximate number instead reducing the load.

★ GOOD PRACTICES TO FOLLOW TO AVOID THUNDERING HERD

 1. Caching - If there are lots of common requests on the server then we
can cache them to reduce load on the database.
2. Gradual Deployments - Servers should not be deployed all at once. We
should deploy some servers then inspect them, and if there is a need to
add more servers then deploy more servers based on requirement.
3. Coupling - To improve performance we keep some data of external
service in our own service

How to optimise read and writes in database

In a database we use B+ tree data structure which requires:-
Insertion time - O(log N)
Searching time - O(log N)

A server sends a request to the database and the database acknowledges that it has
received the request. This can cause a lot of unnecessary data overhead.
To scale the database we need to reduce unnecessary exchange of data like
acknowledgements and headers and also reduce I/O calls which helps in freeing
resources and reducing response time.
In order to do it we can do the following:-
1. Condense data queries into single query:
● Advantage:
We can take multiple queries and process them in batches so as to
reduce I/O operations (Number of Acknowledgements reduce).
● Disadvantage:
Additional memory is utilised by the server to condense it.
2. Use linked list:
● Advantage:
Linked list has an insertion time of O(1). So using linked lists in the
database can help increase write operations.
● Disadvantage:
Read operations are slow as search time in the linked list takes O(N)
time.
3. Use Sorted array together with linked list:
● Advantage:
Search time becomes O(log N) and insertion time becomes O(1) hence
improving read and write time giving optimal solution.
We need to sort the data before inserting it in the database so the data is sorted first
and then persisted in the database. Data is processed in batches and these batches
are called sorted chunks. We can apply binary search on them to search for the data
required.
Now searching in these chunks will still be slow as we need to apply binary search to
each and every chunk.
So, we use a hybrid approach - Merging the sorted chunks in a sorted manner until
the sort time is reduced. This would help to reduce the number of sorted chunks and
hence improve the search time.
We can also use bloom filters to reduce search time.
Bloom filters - A Bloom filter is a probabilistic data structure used to test whether an
element is a member of a set or not. It offers an efficient way of checking for the
existence of an item in a large set, while using a small amount of memory. The key
idea behind a Bloom filter is to use multiple hash functions to map an element to
several positions, and set those positions to 1.
However, due to the probabilistic nature of Bloom filters, false positives (reporting
an element as being in the set when it is not) can occur.

FOOD DELIVERY ALGORITHM:

Food Delivery Algorithm is based on location based algorithm. For searching the
location, we assign codes to each and every location i.e pincodes.
Using pin codes we can assign location to a particular post office.

REQUIREMENTS:
1. Measurable Distance:
● Uniform assignment: Distance can be uniformly assigned (In a
particular order) to each and every location.
● Scalable Granularity: Value of latitude and longitude can be extended
to many decimal places.
2. Proximity: It is used to find the people within a particular range or the people
closer to you.

How would you represent a location?

Location can be defined in the form of coordinates (latitude, longitude).

We can measure the distance between two points(coordinates) using Euclidean
Distance. √ ❑

Here Proximity is a problem.

To find the people close to you, you need to find the Euclidean Distance of a point
with every other point is a very costly process. I.e O(N).
If two points are quite similar i.e their first 13 MSB are same that means they are too
close to each other.
Ex: 110010 (5,4)
110011 (5,5)
Here we use QUADTREE to represent a location. In this we divide the entire world
into different partitions.

Range Query in a 2-D plane is a problem.

Range Query in a 1-D plane is good.(O(logN)).
To convert a 2-D plane in a 1-D line, here we use fractals.

Z-Curve Hilbert Curve Alpha Curve

-Rishabh Singh

System Design
What is system design
It is a process of designing the elements of a system such as architecture, modules
and components.
(Application ko define karne ke process ko system design bolte ha, elements matlab
uska architecture, modules, components , different interface or unke beech data flow
kaise hoga usko system design bolte hai)

It is of two type:

1. LLD- low level Design

2. HLD- high level Design

HLD LLD
1. ye main component ko describe karta 1. Jo HLD ke under hume
hai jisko hume développe karna hai apne elements/components likhe hai unke actual
resulting product ke liye design ko describe karne ke process ko
bolenge LLD
2. Esme system architecture details, data 2. Isme actual classes, interfaces kya likhne
base design(matlab cons use karenge), hoge or unke under kya business logic likhna
services , processes and in sab ke beech hoga. Or inka actual implement
relationship

Video 1:

[Link]
VH6EPyvoX&index=1

System Design Basics:

1. Code is written on a computer in form of an algorithm

2. Code is exposed as a service via API (Application Programming Interface) on the Internet.
3. For each request, the computer sends a corresponding response.
4. Server setup may require a database connection, endpoint configuration and disaster
recovery plan(in case of power loss etc.).
5. Hosting the service on a cloud (provided by solution providers like Amazon Web Services)
offers better reliability, configurability and scalability.
6. Scalability : the ability to handle more request by buying more machines and buying bigger
machines is called scalability.
7. Scalability can be increased by either:
a. Vertical Scaling - buying bigger machines to handle more requests faster
b. Horizontal Scaling - buying more machines to handle more requests through load
balancing
 8. Horizontal Scaling has better resilience and faster inter-process communication compared
to Vertical Scaling. However, it requires load balancing and network calls between servers
can be slow.
9. Data consistency can be a challenge in a system with multiple servers.

Horizontal Vertical
1. Load Balancing is required 1. Not required
2. It is resilient , there is no single point of failure 2. There is a single point of failure
3. Network calls (remote procedure calls). It is 3. Inter process communications. It is
slow fast
[Link] inconsistency 4. Data consistent
5. Scales well as users increase/ [Link] limit

Video 2:
System Design Primer : How to start with distributed systems? - YouTube
Pizza shop example:

1. Vertical Scaling: Optimise processes and increase throughput using the same resource.
2. Pre-Processing and Ron job: Preparing before hand at non peak hours.
3. Backups: Keep backups and avoid single point of failure.
4. Horizontal Scaling: Hire more resources.
5. Micro-Service Architecture: we well defines all the responsibilities to a chef and there’s
nothing outside our business use case that they handle.
6. Distributed System (Partitioning) : open a new shop as a backup, route all your request to
this so that any order that are local or very close range to this shop can be handle by it.
7. Load Balancer: It can make intelligent business design and route request in the smart way.
8. Decoupling: separating out concerns so that we can handle separate system more
efficiently.
9. Logging and metrics calculation : log every event and everything for analysis, auditing,
reporting and machine learning algorithms to find sense out of those events.
10. Extensible : we don’t want to re write the code again and again to serve a different purpose.
Example: delivery agent don’t need to know he is delivering a pizza, it could be a burger
tomorrow.

Video 3: Load Balancing.

What is Load Balancing? ⚖️- YouTube

● Consistent Hashing is a concept relevant to hashing objects and it has certain properties
that one needs to know when building systems that can scale to a large extent.
● A server is a computer running a program that serves requests. For example, if someone
wants to use a facial recognition algorithm, they would connect to the server and send a
request. The server would then process the request and send back the result.
● As the number of requests increases, the single server may not be able to handle the load,
so more servers can be added to balance the load.
● The concept of taking n servers and balancing the load evenly on all of them is called load
balancing.
 ● The requests have a unique request ID, which is generated randomly. The request ID is then
hashed to map it to a particular server. The mapping is done by taking the result of the hash
function modulo n, where n is the number of servers.
● Because the request IDs are uniformly random and the hash function is uniformly random,
all the servers are expected to have uniform load.
● If more servers need to be added, the mapping of the requests to the servers will change,
and the requests that were being served previously will be affected. This is where consistent
hashing comes in.
● Consistent Hashing allows for a more stable distribution of requests to the servers even
when new servers are added or removed. It ensures that only a small portion of the requests
are affected and not the entire set of requests.

Consistent Hashing is a technique used in load balancing to evenly distribute requests

across multiple servers. In consistent hashing, each request has a unique request ID which
is hashed to map it to a particular server. The hash function takes the request ID and maps it
to a number, and this number is then taken modulo the number of servers to determine
which server the request should be sent to. The hash function is designed to be uniformly
random, ensuring that the load is evenly distributed across all servers. If a new server is
added, the requests that were previously sent to other servers may now need to be re-routed
to the new server, but consistent hashing ensures that the re-routing of requests is done in
such a way that the load is still evenly distributed across all servers.

consistent hashing is used to evenly distribute the weight across all servers, given n servers
and a request ID that is uniformly random. The request ID is sent to the server, hashed to
produce a value, and then mapped to a particular server by taking the remainder with n (the
number of servers).
● For instance, consider the case where there are 4 servers: s0, s1, s2, and s3. If a request ID
r1 has a value of 10, when it is hashed using the hash function H, it gives the value 3. Then,
taking the remainder of 3 mod 4, gives 3, which maps to the server s3. If another request ID
r2 has a value of 20 and is hashed to give 15, then 15 mod 4 gives 3, which also maps to
server s3.
● This approach ensures that all the servers have uniform load, with each server having a load
factor of 1/n. However, the challenge arises when you need to add more servers. For
example, if you add a new server s4, then the requests which were previously served by
server s3 will have to change as well.
● In consistent hashing, when a new server is added, only a fraction of the keys that were
previously assigned to one server are remapped to the new server. The fraction of keys
remapped depends on the number of servers and the total number of keys. The mapping of
keys to servers is done such that the load is balanced across all servers, even after the
addition of a new server. This approach helps minimize the number of keys that have to be
remapped, making the process of adding new servers more efficient.
● In conclusion, consistent hashing is a way of distributing the load across servers in a
system such that when new servers are added, the load is rebalanced efficiently, with only a
fraction of the keys being remapped.

Video 4: Consistent Hashing

What is Consistent Hashing and Where is it used? - YouTube

Consistent Hashing

Introduction:
 ● Consistent Hashing is a method used in computer networks to distribute requests evenly
among servers in a network.

Problem:

● Load balancing is difficult when adding or removing servers as it completely changes the
local data stored in each server.

Solution:

● The solution is to use a ring of hash function values, where each request is hashed
according to its ID and each server is hashed according to its ID.
● Requests are sent to the nearest server clockwise from their hash value on the ring.
● To improve the load balancing, multiple hash functions can be used.
● Virtual servers can also be used instead of physically adding more servers, reducing cost.

Theoretical Analysis:

● Hashes are uniformly random, so the distance between them is expected to be uniform.
● Load should be uniform, with an expected average of 1/n.

Practical Analysis:

● Practically, there can be skewed distributions if there are not enough servers.
● To mitigate this, multiple hash functions or virtual servers can be used.

Note: Consistent Hashing is a method used to distribute requests evenly among servers in a
network. By using a ring of hash function values, requests are sent to the nearest server clockwise.
This method can be improved by using multiple hash functions or virtual servers.

Video 5 : Message Queue

[Link]
VH6EPyvoX&index=5

Design of a Messaging Queue for a Pizza Shop

Overview:

● A pizza shop receives multiple orders from clients at the same time
● To relieve the client from waiting for an immediate response, the shop gives them a
confirmation of the order placed
● The shop maintains a list of orders and works on making the pizzas as per the priority
● The architecture allows the client and the shop to perform other tasks while the orders are
being processed

Components:

● List of orders: Maintains order number and order details

● Database: To store the list of all orders and their status
● Servers (1 to 4): Handle the orders
● Notifier: Check the heartbeat of each server and redistributes orders if a server crashes

Order Processing:

● Clients place orders and receive a confirmation of the order placed

● The orders are added to the list of orders and stored in the database
● The servers pick up the orders from the list and start working on them
 ● The notifier checks the heartbeat of each server and redistributes orders if a server crashes
● Once the order is completed, it is removed from the list of orders and the client is asked to
pay

Scenario of a Server Crash:

● If a server crashes (e.g., server 3), the notifier checks for its heartbeat
● If the server does not respond, the notifier assumes it is dead and cannot handle orders
● The notifier queries the database to find all orders that are not done and belong to the dead
server
● The notifier redistributes these orders to the remaining servers

Advantages:

● Asynchronous processing allows the client and the shop to perform other tasks while the
orders are being processed
● The priority of the orders can be manipulated as per the requirement
● The notifier helps in redistributing orders if a server crashes, ensuring that the orders are
completed
● The database provides persistence in data and ensures that the list of orders is not lost even
if a server crashes.

Video 6 : Monolith vs microservices

What is a microservice architecture and it's advantages? - YouTube

A monolith architecture is a large system that runs on one or multiple machines, where all clients
are connected to the same machine. The advantages of this architecture include ease of
deployment, faster performance due to the absence of network calls, and simplified testing.
However, it can be difficult for new team members to understand the entire system and
deployments may become complicated as the codebase grows.

On the other hand, microservices are individual business units that have all the data and functions
relevant to a specific service. They talk to their own dedicated databases and may communicate
with a gateway that connects to the clients. The benefits of microservices include easier scalability,
parallel development, and the ability for new team members to focus on a specific service. However,
there can be a large number of moving parts and it can be challenging to manage and maintain the
individual services.

Monolith Architecture:
Advantages:

1. Easy to manage with a small and cohesive team

2. Less moving parts, easier deployments
3. Avoids duplication of code and faster as all logic is in one box
4. This is faster, we are not making any network (remote procedure call) its all in the box , local
calls.
Disadvantages:
5. Requires a lot of context for new team members
6. Complicated deployments( for new change in the code) and frequent monitoring
7. Single point of failure.
8. Tests are more complicated
9. Too much responsibility on each server, causing system collapse if one fails
Microservice Architecture:
Advantages:

1. Easier to scale
2. Assigning tasks based on service, reducing context for new team members
3. Facilitates parallel development
4. Reduced tight coupling between developers and services
Disadvantages:
5. They are not easy to design.
6. Complexity in managing multiple services and intercommunication
7. Increased need for monitoring and coordination between services
8. Increased complexity in testing.

The choice between monolith and microservice depends on the size and nature of the team, the
system requirements, and the scalability goals.

The problems with monolithic architecture are:

1. Inflexibility: Monolithic architecture is rigid and does not allow for changes to be made to
individual components without affecting the entire system.
2. Lack of scalability: Monolithic systems are not scalable and struggle to handle large
amounts of traffic or data.
3. Difficulty in deployment and testing: Monolithic architecture makes it difficult to deploy and
test individual components, as they are tightly integrated with the rest of the system.

On the other hand, microservices architecture, while offering many benefits, also comes with its own
set of challenges, such as:

1. Complexity: Microservices architecture can be complex and difficult to manage, especially

as the number of services increases.
2. Inter-service communication: Inter-service communication and coordination can become
challenging as the number of services increases.
3. Monitoring and debugging: Monitoring and debugging microservices can be difficult, as
there are many moving parts and the interactions between services can be complex.

Video 7: Database sharding

What is Database Sharding? - YouTube

I. Introduction

● Explanation of database optimization

● Explanation of indexing

II. Sharding

● Explanation of sharding and how it works

● Explanation of horizontal partitioning and how it's different from vertical partitioning
● Explanation of sharding as partitioning of data using an attribute of the data, such as user ID
● Explanation of how sharding can improve performance and efficiency of a database

III. Key Considerations

 ● Explanation of the importance of consistency and availability in a database
● Discussion of the problems with joins across shards
● Explanation of inflexibility in number of shards
● Explanation of the solution to overcome this problem through consistent hashing
● Explanation of the use of memcached as a database that uses consistent hashing

IV. Conclusion

● Summary of the key points discussed about sharding and database optimization.

I. Introduction
The introduction explains database optimization, which is the process of making a database
perform better by improving its design and structure. It also explains indexing, which is a database
optimization technique that allows for faster search and retrieval of data by creating a separate data
structure that stores the values of one or more columns in a table.
II. Sharding
Sharding is a database partitioning technique that involves dividing a large database into smaller
parts called shards. The partitioning is done based on an attribute of the data, such as user ID, to
improve performance and efficiency of the database. Sharding can be either horizontal partitioning
or vertical partitioning, with horizontal partitioning dividing a table into multiple smaller tables, while
vertical partitioning divides a table into multiple smaller tables based on columns.
III. Key Considerations
When implementing sharding, it is important to consider consistency and availability, as the
distribution of data across multiple shards can sometimes lead to inconsistent or unavailable data.
Joining data across shards can also pose problems, and inflexibility in the number of shards can
limit the scalability of the database. A solution to this inflexibility is consistent hashing, which
assigns data to shards based on a hashing algorithm, and allows for the addition or removal of
shards without affecting the data distribution. The use of memcached, a database that uses
consistent hashing, is also discussed.
IV. Conclusion
The conclusion summarizes the key points discussed about sharding and database optimization,
highlighting the benefits of sharding and the importance of considering consistency, availability, and
scalability when implementing sharding.

Notes:

● Indexing is a method to improve query performance by creating a sorted list of the data in a
table.
● Sharding is a method of horizontal partitioning data by using an attribute of the data (e.g.
user ID or location) as a key to break the data into pieces and allocate them to different
database servers.
● Consistency is important in databases, meaning the data persisted in it should be what is
read out later, with synchronization to ensure updates are read by new requests.
● Availability is also important, meaning the database should not crash and stay down.
● Joins across shards are a problem as they need to pull data from different shards and join
the data across the network, which is expensive.
● Consistent hashing is a good algorithm to overcome the problem of inflexible shards.
● Sharding can also result in a dynamic number of shards, which can be solved by breaking a
shard that has too much data into smaller pieces.

Video 8: NETFLIX Content Onboarding:

How Netflix onboards new content: Video Processing at scale 🎥 - YouTube
 1. Challenges in uploading new content:
● Storing in different formats (e.g. MP4, AVI) to accommodate different internet
connection speeds.
● Codecs used to compress video and maintain quality-size balance.
● Resolutions to cater to different screens (cell phone, TV, laptop, etc.).
● Multiple formats and resolutions result in multiple videos that need processing.
2. NETFLIX solution:
● Break the original video into chunks to make processing easier.
● Each chunk is processed in different formats and resolutions, making it one task.
● To avoid lags, chunks are broken based on scenes instead of time stamps.
● The video suggestion algorithm takes the entire movie as a set of chunks and
fetches the entire scene together.
3. Processing the chunks:
● Initially, chunks were of equal size (3 minutes each).
● But to improve user experience, chunks are now broken down into fine-grained
shots (4 seconds each).
● These shots are collated into scenes to make video fetching seamless.

● Netflix uses Amazon S3 to store its video content.

● When a user accesses Netflix, it maps to an IP address that is a physical place or a
computer on the internet.
● The servers are usually in the U.S and to improve the user experience, Netflix caches
information to pre-compute and store it locally.
● Netflix extended the caching concept and applied it to ISPs by using Open Connect boxes.
● The Open Connect boxes contain a ton of movies and help in reducing the load on the
servers and the ISPs.
● Around 90% of Netflix traffic is taken care of by these Open Connect boxes provided by the
ISPs.
● The boxes are populated with the latest content by Netflix at low-traffic times such as 4 am
in the night.
● This innovative solution of video processing and video serving is what keeps Netflix running
at scale and helps in providing a better user experience.
● Caching is a technique used to improve the user experience and reduce the load on servers.
In the case of Netflix, they implemented caching by placing "Open Connect" boxes with ISPs.
These boxes contain a large amount of movie content and serve as a cache for the ISP to
quickly retrieve the requested movie. This reduced the load on the Netflix server and saved
bandwidth, resulting in a faster response time and better user experience for the user. The
boxes are populated with the latest content from the Netflix server during low-traffic times,
usually around 4 am. Caching is a revolutionary concept and 90% of Netflix traffic is taken
care of by these boxes. This innovative solution to video processing and serving has been
crucial in allowing Netflix to scale efficiently.

Video 9:

System Design: Tinder as a microservice architecture - YouTube

Tindoer System Design:

1. Storing Profiles:
● Each user can have up to five images in their profile
1. Recommendation System:
● Based on user preferences, matches will be recommended
 ● Number of active users is an important factor to consider
1. Matches:
● When two users match, it will be noted down
● Assumption: 0.1% of users will match for every swipe (typical Indian match rate)
● Number of matches per day can be calculated as: Number of Active Users x (0.1%)
1. Direct Messaging:
● Users can chat with each other once they have matched

Note: Limit the number of features to 4-5 to avoid getting into too much detail during the interview.
The interviewer will guide the discussion towards the details they are interested in.

Storing Images:
File vs Blob

1. Mutability:
● Storing images as a file: Immutable, does not need the mutability feature offered by
databases.
● Storing images as a blob: Mutable, allows for easy updates to the image.
1. Transaction guarantees:
● Storing images as a file: Not required as image updates are not atomic operations.
● Storing images as a blob: Offers transaction guarantees, ensuring data consistency and
integrity.
1. Indexes:
● Storing images as a file: Not necessary, as images are binary objects and cannot be
searched by content.
● Storing images as a blob: Offers the ability to index data, improving search capabilities.
1. Access control:
● Storing images as a file: Access control can be achieved through a secure file system,
although it may be tedious.
● Storing images as a blob: Offers built-in access control mechanisms.

Advantages of storing images as a file:

● Cheaper than storing in a database.

● Designed for storing files and images.
● Faster as large objects are stored separately.
● Static, allowing for easy implementation of a Content Delivery Network (CDN) which allow
fast access.
● Avoiding the risk of doing a "select star" nightmare in databases.

Advantages of storing images as a blob:

● Offers transaction guarantees and access control mechanisms.

When to use files or blobs:

● If practicality and cost-effectiveness are the main concerns, files may be the better option.
● If transaction guarantees and access control are necessary, blobs may be the better option.
● Ultimately, the choice between files or blobs depends on the specific requirements and
constraints of the system being designed.

Storing Profiles Design Notes:

1. Client application: A mobile application where users can send requests by clicking a button.
 2. Profile Service: A service responsible for registering the users with the system by storing
their username and password in a database. It also handles the authentication of the users
for the update profile requests.
3. Email Service: A service used for sending password and other authentication-related emails.
4. Authentication: The process of verifying the identity of a user. The profile service uses the
username and password or token to authenticate the user.
5. Gateway: A service that acts as a mediator between the client and the profile and image
services. It receives requests from the client and sends them to the profile service to
authenticate the user. If the request is authentic, the gateway forwards the request to the
appropriate service and sends the response back to the client.
6. Image Service: A service responsible for storing and processing the user's images. It has a
distributed file system and a database that stores the profile ID, image ID, and URL of the
images.
7. Distributed File System: A file system that stores images across multiple servers.
8. Decoupling: The process of separating different responsibilities and functions of a system
into separate services. The gateway acts as a decoupled system between the client and the
profile and image services, reducing the need for duplicated code and separating the
protocols.
9. Direct Messaging: A type of messaging protocol that allows for direct communication
between different services.
10. Heavy Computations: Intensive computational tasks such as image processing.

1. The system consists of a client application on a mobile device, a profile service, and a
gateway service.
2. The client application allows the user to send a request by clicking a button.
3. The profile service stores the user's username and password in a database and performs
authentication.
4. The profile service may also send emails and perform two-step authentication.
5. To update the user's profile, the client sends the username and a token for authentication.
6. The gateway service acts as an intermediary, taking the request from the client and
checking with the profile service to see if the request is authenticated.
7. If the request is authenticated, the gateway directs it to the appropriate service and
forwards the response back to the client.
8. This approach separates the responsibilities of authentication and request handling and
eliminates duplicated code across services.
9. The profile service updates the user's profile information such as the description, name, and
profile picture.
10. The image service stores and manages the user's profile pictures in a distributed file
system.
11. The image service also has a database that contains references to the profile ID, image ID,
and image URL.
12. The image service is used for heavy computations, such as when all images of a user are
needed, while the profile service is used for regular profile information retrieval.

Messaging:
Introduction to Direct Messaging:

● Direct messaging refers to chat between two clients

● The objective is to connect two clients, User ID 1 and User ID 2, for the purpose of
sending messages

Client-Server Communication Protocol:

 ● Communication between two machines can be achieved through a client-server
communication protocol
● The client communicates with the server and the server responds to the request
● However, this method is not efficient for chat applications as it requires the client to
constantly poll the server for messages

Peer-to-Peer Protocol:

● A better solution for chat applications is to use a peer-to-peer protocol where both
clients are equal
● One example of such a protocol is XMPP (Extensible Messaging and Presence
Protocol)
● Another example is HTTP (Hypertext Transfer Protocol)

Web Socket Connection and TCP:

● XMPP uses a web socket connection for communication

● It is also possible to use a TCP connection to achieve the same purpose

Maintenance of Connection Information:

● The gateway service is responsible for maintaining information on connections

● For each connection, the gateway service must know which user is using the
connection
● It is recommended to decouple the system as much as possible and move the
responsibility of maintaining connection information to another service, such as a
sessions service
● This service can handle sessions and store connection information, mapping the
User ID to the connection
● This information can then be used to determine the connection used by the other
user and send messages accordingly

Conclusion:

● Direct messaging is possible if the requirements of connecting clients and

maintaining connection information are met
● The described architecture can effectively handle direct messaging in a chat
application.

Matching algorithm:

● The chat application stores information on the client device, but the server is the source of
truth for all information.
● The match service is responsible for noting down matches and will store all relevant
information on the server.
● If the client device uninstalls the app, the only information lost will be the number of people
swiped left or right, but this can be regained when the app is reinstalled.

Match Service:

● Keeps a table of user IDs to show which users have matched with each other.
● Checks if a user is authenticated to send a message to another user.
● Communicates with the session service to confirm a user's authentication and send
messages to the correct connection.
● Notes down all matches and stores the information on the server.

Session Service:
 ● Sends messages to the correct connection based on the user's authentication status.

Notes:

● The match service is the key component for noting down matches and maintaining
information about them.
● The client device only stores information about who was swiped left or right, but this
information can be regained if the app is reinstalled.
● The server is the source of truth for all information and ensures that information is not lost
in case the client device is uninstalled.

Recommendation System:

● To recommend people to a user, the system needs to figure out who are the users closest to
the user.
● The profile service has information about the users, such as name, age, gender, and
location. These three attributes (age, gender, and location) are used to make
recommendations.
● Having multiple indexes on these attributes is not possible in a traditional relational
database. Therefore, to optimize the recommendation system based on multiple
parameters, a NoSQL database such as Cassandra or Amazon Dynamo can be used.
● If the NoSQL databases are not preferred, sharding can be used in a relational database.
Sharding is a way of partitioning data based on its values and directing the data to a location
based on those values.
● In sharding, the data can be partitioned based on location, such that all users within a
specific location or a specific chunk of a location are sharded to a particular node.
● The data can then be pulled out from that node and searched based on age and gender
attributes.
● The sharded data can be protected from single point failure using a master-slave
architecture.
1. Sharding involves horizontal partitioning of data based on a property of the data and
directing the data to a location based on its value.
2. Consistent hashing is critical to keep the servers functioning and a master-slave
architecture can be used to prevent single point of failure.
3. Sharding the data based on the location of users can be done by dividing the city into
chunks and directing the users in a chunk to a particular node.
4. This allows for efficient searching among users within the age and gender variables.

Video 11: Distributed Caching

What is Distributed Caching? Explained with Redis! - YouTube
What is Caching?

● Caching is a technique used in computer systems to store frequently used data in memory
for faster access.

Why use Caching?

1. To avoid frequent database queries for commonly used data.

2. To avoid expensive computations.
3. To reduce load on the database.

Benefits of Caching:
 ● Speeds up response times to clients.
● Avoids network calls to the database.
● Avoids expensive computations.

Drawbacks of Caching:

● The hardware used to run a cache is more expensive than a normal database.
● If too much data is stored in the cache, search times can increase, making it less efficient.

Cache Policy:

● The decision of when to load or evict data from a cache is referred to as a cache policy.
● LRU (Least Recently Used) is the most popular cache policy. It states that the most recently
used data should be kept at the top of the cache, and the least recently used data should be
evicted from the cache when space is needed.

Sliding Window based Policies:

● Some policies have been developed that perform even better than LRU.
● These policies are based on sliding windows and dynamically determine which data to keep
in the cache.

Cache Placement
Close to database

● Avoids network latency

● May lead to overloading the database server

Close to servers

● Reduces network latency

● Can consume a lot of memory in the server
● If server fails, the cache also fails
● May lead to data inconsistency between servers

Global cache

● Distributed
● Limited in size
● Faster disk read
● More resilient to server crashes
● Can be scaled independently
● Higher accuracy of data consistency

Write Through vs Write Back Cache

Write Through Cache: Write to cache first and then to database.

● Avoids data inconsistency but may lead to stale data in other cache instances

Write Back Cache: Write to database first and then update cache

● Avoids stale data but may lead to data inconsistency.

Cache is an important component in system design that helps to improve the performance of
applications. There are two types of cache: write-through and write-back.
 ● In a write-through cache, when data is updated in the cache, it is immediately sent to the
database. This ensures that the data in the cache is consistent with the data in the
database. However, the problem with this approach is that if there are other servers that
also have a copy of the same cache in memory, the update may not be reflected in all
instances, leading to data inconsistency.
● On the other hand, in a write-back cache, data is first updated in the database, and then the
cache is updated accordingly. This ensures that all instances of the cache have the same
data. However, this approach can be expensive as it requires constant updates to the cache.
● A hybrid approach, combining elements of both write-through and write-back, is also
possible. This approach involves updating the cache in a write-through manner, but instead
of immediately updating the database, the updates are stored in the cache and sent to the
database in bulk. This reduces the number of network calls and reduces the burden on the
database.
● Grokking the System Design Interview is a well-known online course that covers system
design topics, including load balancing and caching. This course provides hands-on
experience in system design and covers a range of topics, making it a valuable resource for
anyone preparing for a system design interview.
●

Video 12 : WhatsApp

Whatsapp System Design: Chat Messaging Systems for Interviews - YouTube

WhatsApp Architecture Notes:

group messaging
sent + delivered + read receipts
online/ last seen
image sharing
chats are temporary/ permanent/

1. Gateway:
● A user connects to the WhatsApp cloud using an external protocol when talking to the
application.
● The gateway acts as an interface between the user and the internal services of WhatsApp.
● The security mechanisms are handled by the gateway.
1. User-to-Box Mapping:
● The Gateway service needs to store information about which users are connected to which
box.
● This information was stored on the boxes but is expensive to maintain and leads to a lot of
coupling.
1. Sessions Microservice:
● To avoid this coupling, the information about who is connected to which box was decoupled
from the system and sent to the Sessions Microservice.
● The Sessions Microservice acts as a router that figures out where a user exists and routes
messages to the relevant box.
1. Send Message:
● A user sends a message by asking the gateway to send the message to another user.
● The gateway is dumb and sends the request to the Sessions Microservice.
● The Sessions Microservice routes the message to the relevant box, which sends it back to
the user.
1. Real-time Communication:
 ● HTTP is not suitable for real-time communication and long polling can only send messages
every minute or so.
● Websockets are used for real-time communication as they allow peer-to-peer
communication.

In summary, the WhatsApp architecture consists of a Gateway that acts as an interface between the
user and internal services. The Sessions Microservice acts as a router to route messages between
users by storing information about who is connected to which box. Real-time communication is
achieved through the use of websockets as HTTP is not suitable for chat applications.

Building a Chat Messaging Application:

1. Load Balancer: It balances the load across the system, but it will not be covered in detail as
it has already been discussed.
2. Service Discovery or Heartbeat Maintenance: not relevant to the chat application.
3. Authentication Service: it is simple.
4. Profile Service, Image Services, Sending Emails and SMS's: These services are not relevant
to the chat application.
5. Sending Messages: This is the core of the chat application.
6. Group Messaging: In this feature, whenever a user from a group sends a message, it should
go to all other members of the group. For example, if there is a red group with three users,
they are connected to three boxes, and if the session service stores all the information for
all groups, it becomes too complicated for the session service to handle. To resolve this, the
information for who is existing in which group is decoupled in a group service.
7. Group Service: The session service when it gets a message from a user will ask the group
service who the other group members are. The group service can then respond with the
number of members and their user IDs in the group. The session service then runs through
its own database to figure out where these users are connected to and routes the messages
to each of them.
8. Maximum Limit of Group Members: There is a maximum limit of 200 members in a group in
WhatsApp. Chat applications try to contain this to 500-600, as fanning out requests too
much is not practical, especially in real-time messaging.
9. Passing Message: To reduce the memory footprint, the session service passes the message
through a parser microservice, rather than passing the converted message object to the
gateway.
10. Parser Microservice: This microservice sends an unpassed message to any service, and its
responsibility is to pass the unpassed message through a parser. This reduces the
responsibilities of the gateways, which are expensive due to web sockets connected to
actual users.
11. Limiting the Number of Users: The number of users in a group is limited to some number x,
and it is assumed that the sessions can handle web sockets sending messages to relevant
users.

In conclusion, building a chat messaging application requires careful planning and consideration of
various factors such as load balancing, authentication, group messaging, and limiting the number of
users in a group. The session service plays a critical role in routing messages to the relevant users,
and the parser microservice helps reduce the memory footprint and responsibilities of the gateways.

Video 13: API

What is an API and how do you design it? - YouTube
API Design Best Practices:
An API (Application Programming Interface) is a way for external consumers to interact with your
code. It is a contract that outlines what the code will do, rather than how it will do it. To design a
good API, it's important to consider the following factors:

1. Naming: The name of the API should reflect the action it performs. Avoid taking additional
parameters unless necessary.
2. Parameter List: Keep the parameter list as minimal as possible. Add additional parameters
only if it is needed for optimization.
3. Response: Avoid stuffing the response with more information than necessary. The response
should only contain what the caller needs.
4. Error Handling: Define the number of errors that the API can return. Keep it minimal, but
include all the important error cases.
5. Placement: Place the API in the appropriate microservice that handles related tasks.
6. Extensibility: Avoid making the response object overly complex in the hope of making it
extensible in the future. This leads to confusion and unnecessary network requirements.
7. Optimization: If the API is being heavily used and the number of calls is causing
performance issues, consider adding additional parameters for optimization purposes.
However, this should only be done when necessary and the name of the API should reflect
this change.

In summary, a good API design is about finding the balance between providing the necessary
information and making it easy for external consumers to understand how to use it. The key is to
ensure that the API is intuitive and provides clear and concise information, while avoiding any
unnecessary details.

Video 14 : planning and estimation

Capacity Planning and Estimation: How much data does YouTube store daily? - YouTube
Video 15 :
What is the Publisher Subscriber Model? - YouTube
Event-driven services architecture:

● Involves multiple microservices (s1, s2, s3, s4, etc)

● A client is connected to s1 and sends a request
● S1 processes the request and sends two messages to s0 and s2
● The order of the messages to s0 and s2 does not matter
● S2, after processing, sends messages to s3 and s4
● If using a request-response architecture, s2 may wait for s3 and s4 to respond leading to
potential delays and timeouts
● A better approach is to use a publisher-subscriber model with a message broker (e.g. Kafka,
RabbitMQ)
● S1 publishes the message to the message broker and sends a success message to the
client
● The message broker takes responsibility for sending the message to s0 and s2 and ensuring
they receive it even if one of them is down
● The message broker also sends messages to s3 and s4
● Advantages of this architecture include:
● Decouples responsibilities of the services
● Makes the system easier to understand with a single point of failure
● Provides a generic interface for developers
● Offers loose transaction guarantees (at least once)
● Enhances reliability and reduces complexity
 ● Overall, the publisher-subscriber model with a message broker provides a more robust and
efficient way of handling event-driven services compared to a request-response
architecture.

The advantages of using an event-driven architecture with a message broker,

such as Kafka or RabbitMQ, include:

1. Decoupled responsibilities, as services are no longer dependent on each other.

2. Single point of failure, making it easier to understand and manage the system.
3. Developers only need to worry about the interface interaction with the messages.
4. Transaction guarantees, as the message broker provides persistence to ensure messages
will be delivered.

The main disadvantage is that there might be a long wait time for a response to fail if one of the
services is down. This can lead to multiple changes for the same request, which can result in data
inconsistencies.
Disadvantages:
1) An extra layer of interaction slows services
2) Cannot be used in systems requiring strong consistency of data
3) Additional cost to team for redesigning, learning and maintaining the message queues.

Video 16:
Why do Databases fail? AntiPatterns to avoid! - YouTube
Using a database as a message queue is considered an anti-pattern. This approach is often used in
scenarios where servers (s1, s2, and s3) communicate with each other and clients (mobile or
desktop). In this approach, a server sends a message by inserting an entry into the database, and
the other server receives the message by polling the database at a specific interval. However, this
approach has several drawbacks:

1. Polling Interval: Frequent polling puts a lot of load on the database, while long intervals can
result in poor user experience.
2. Read/Write Optimization: Databases are optimized for either reading or writing, but not both.
3. Space and Deletion: The database will get filled up with entries and either needs to be
cleared frequently or updated to mark the messages as completed, which is an expensive
operation.
4. Scalability: With an increasing number of servers, the database will be unable to handle the
load of so many read operations.

In conclusion, using a database as a message queue is not an efficient or scalable solution, and
there are other technologies that are better suited for this purpose, such as message brokers or
message queue systems.

Video 17: Content Delivery Network

System Design: Content Delivery Networks (Simplified) - YouTube
Content Delivery Networks (CDN)

1. CDN's are used to improve the speed and efficiency of delivering content to users from
multiple countries or locations.
2. CDN's make use of caching to store static pages such as HTML pages, reducing the need to
make a request to the server for every single page.
3. CDN's can customize the content sent to different types of devices and locations, which can
help to optimize the user experience.
 4. CDN's aim to serve content quickly, reducing the time it takes for web pages to load and
minimizing the risk of losing a user's interest.
5. CDN's can be a single point of failure, which can lead to the collapse of the whole system if it
crashes.
6. To mitigate this risk, CDN's can be designed as a distributed cache, with multiple nodes
working together in a group consensus.
7. Horizontal sharding of the cache can be done based on location, country, or other factors,
which helps to serve relevant content to specific users.
8. CDN's require a distributed consensus mechanism, such as Paxos or Raft, to ensure the
consistency of the cache.
9. CDN's should be designed to treat requests from different users differently, directing them
to the relevant part of the cache.

In conclusion, CDN's play a crucial role in improving the speed and efficiency of delivering content to
users and optimizing the user experience. To ensure reliability and scalability, CDN's should be
designed as a distributed cache, with a consensus mechanism to maintain consistency, and
sharding to serve relevant content to specific users.

Video 18: Avoid single point of failure

How to avoid a single point of failure in distributed systems ✅ - YouTube

Single Points of Failure in Computing Context:

● A single point of failure (SPOF) is a point in a system where if it fails, the entire system
crashes.
● Mitigating SPOFs is important for creating a resilient architecture and ensuring system
reliability.
● One way to reduce SPOFs is by adding another node, such as adding a backup database or
another profile server.
● Another way is to implement load balancing with multiple load balancers to distribute
requests.
● In case of load balancing, the load balancer itself can be a SPOF, so multiple load balancers
should be used and resolved through DNS.
● To prevent regional disasters, the system can be distributed across multiple regions.
● For a distributed database, the coordinator can also be a SPOF, so multiple coordinators
should be implemented.
● The process of reducing SPOFs should be propagated throughout the entire pipeline of the
distributed system.
● Tools like Netflix's chaos monkey can help test and improve resiliency.

In conclusion, reducing SPOFs is an ongoing process that requires careful planning and
implementation, but is essential for creating a reliable and resilient system.

Video 19: Event Driven System

What's an Event Driven System? - YouTube
Event-driven Architecture is a way of designing systems where services communicate with each
other using events instead of direct request-response interactions.

1. Event Bus: The communication between services is facilitated through an event bus, which
acts as a medium for the services to send and receive events.
2. Producers and Subscribers: The services sending the events are known as producers, and
the services receiving the events are known as subscribers.
3. Data Persistence: Each service stores the events it receives from the event bus in its local
database. This ensures that the data remains accessible even if the service goes down.
 4. Event-Driven Applications: Event-driven architecture is widely used in various applications,
including gateways, gaming systems, and first-person shooter games.
5. Timestamp-based Replication: In games, event-driven architecture allows for the replication
of events and movements at a specific timestamp. This helps in resolving disputes in a fair
manner.
6. Advantages: Event-driven architecture has several advantages, including improved
scalability, decoupled communication between services, and enhanced system resiliency.
7. Limitations: Despite its advantages, event-driven architecture can be challenging to
implement, and it may not be suitable for all types of applications. It is important to carefully
evaluate the requirements before choosing this architecture.

In conclusion, event-driven architecture is a powerful way of designing systems where services

communicate with each other using events. It provides several advantages, but its limitations must
be considered before implementing it in a project.

an example of a first-person shooter game, where event-driven architecture is used to handle the
events of headshots and movements in the game. It is stated that "in a first-person shooter game
like counter-strike, you know about the headshots that we can take." The example then goes on to
describe the scenario where player one takes a headshot, the information is sent to the server, and
the server updates the position of player two. However, due to delay response, player two moves to
a different position and the headshot is not recorded as a win for player one. To resolve this issue,
the example explains that event-driven architecture can be used by taking the events of movements
and shots with their timestamps and replicating them to see if the headshot was taken at the right
position.

Video 20 :

Introduction to NoSQL databases - YouTube

Introduction to NoSQL databases

● NoSQL databases are non-relational databases that store data in a flexible and scalable
manner
● They are becoming increasingly popular due to their ability to handle large amounts of data,
provide high performance and scalability, and accommodate changing data structures

Difference between SQL and NoSQL databases

● SQL databases have a fixed schema and store data in structured tables with well-defined
relationships
● NoSQL databases have a flexible schema and store data as collections of documents, key-
value pairs, or graph data
1. SQL databases use a structured table format for data storage and retrieval, where each
column represents an attribute and each row represents a record. NoSQL databases use
unstructured data formats like JSON, which allow for nesting of objects and more flexible
data models.
2. In SQL databases, a foreign key relationship is used to link data from different tables. In
NoSQL databases, the data is contained in one block and there is no need for foreign key
relationships.
3. Selecting all the data relevant to a user in SQL requires a join operation which can be
expensive, while in NoSQL databases, the data is contained in one block making it easier to
insert and retrieve.
4. The schema in NoSQL databases is flexible and allows for the addition of new attributes
without the need to add a new column. This is not the case in SQL databases.
Advantages of NoSQL databases

1. Scalability: NoSQL databases can handle large amounts of data and provide horizontal
scalability, making it easy to scale out as needed
2. Flexibility: NoSQL databases can accommodate changes in data structure, making it easy to
add new fields and data types
3. Performance: NoSQL databases can provide high performance for read-heavy and write-
intensive applications
4. Cost-effective: NoSQL databases can be more cost-effective than traditional SQL databases
for large scale projects

Use cases for NoSQL databases

1. Big Data: NoSQL databases can handle large amounts of data and provide scalability,
making them a popular choice for big data projects
2. Real-time web applications: NoSQL databases can provide high performance and low
latency, making them suitable for real-time web applications
3. Mobile and Internet of Things (IoT) applications: NoSQL databases can handle a large
number of concurrent users and handle rapidly changing data, making them suitable for
mobile and IoT applications

Conclusion

● NoSQL databases are becoming increasingly popular due to their ability to handle large
amounts of data, provide high performance and scalability, and accommodate changing
data structures
● It's important to choose the right type of NoSQL database for a specific use case and to
understand the trade-offs between performance, scalability, and data consistency
● In conclusion, NoSQL databases can be a valuable tool for organizations looking to manage
and analyze large amounts of data in real-time.

Notes on Cassandra Architecture:

1. Cassandra is a NoSQL database that operates on a cluster of nodes, in this case, 5 nodes.
2. Requests coming into the cluster are assigned to a node based on their request ID. The
request ID can be a numeric value, UUID, or person's name, among others.
3. To assign the request ID to a node, the request ID is hashed and the resulting value is used
to determine the node it should be sent to.
4. A good hash function ensures that requests are distributed evenly across all nodes, allowing
each node to operate at its full capacity.
5. If the hash function is not good and one node ends up receiving too many requests, the
cluster can become overwhelmed. To avoid this, a two-layer cluster can be implemented,
with each layer using a different hash function to distribute the load more evenly.
6. To ensure that important data is not lost, replicas of the data are made and stored on other
nodes in the cluster.
7. Cassandra is used for high-availability and high-performance data storage.

The major advantage of using a hash function in Cassandra is to ensure that requests are
distributed evenly across all nodes and that important data is stored redundantly to ensure its
availability. By using a hash function and replicas, Cassandra can provide high availability and high
performance even under high load conditions.
Quorum is a term used in NoSQL databases, specifically in distributed databases like Cassandra. In
a distributed database, a quorum is the minimum number of nodes in a cluster that must
acknowledge a write operation for it to be considered successful. The purpose of a quorum is to
ensure data consistency and availability in the face of node failures.
For example, if you have a Cassandra cluster with 5 nodes and a quorum of 3, then at least 3 nodes
must respond to a write request before it is considered complete. If fewer than 3 nodes respond, the
write request will fail and the data will not be saved to the cluster. This ensures that even if some
nodes are down or unavailable, there will still be enough nodes to guarantee the reliability and
consistency of the data stored in the cluster.

video 24: Instagram

Designing Instagram:

1. Key features:
● Storing and retrieving images
● Liking and commenting on posts
● Following other users
● Publishing a newsfeed
1. Image storage:
● File system is preferred for cost-effective storage
● CDN can be created for faster access
1. Liking and commenting:
● Likes table represents the posts liked by users, with user ID, timestamp, and active status.
● Parent ID in the likes table represents either a comment or a post, with a type field to
indicate the type.
● Select query can be used to find the number of likes for a post.
1. Newsfeed:
● Select star query for generating news feed is slow and not feasible for large scale.
● Solution: Maintaining a likes column in the posts table that increments whenever a post is
liked.
1. Scalability:
● Flexibility in design is important to accommodate future growth and changes.
● Optimization techniques such as caching and load balancing can be used to improve
performance.
●
1. Database design:
● ER diagram is a useful tool for designing the d

● atabase structure.
● Normalization techniques should be used to ensure consistency and efficient use of
resources.
1. Summary:
● Designing Instagram involves consideration of key features such as image storage, liking
and commenting, following users, and publishing a newsfeed.
● Database design and optimization techniques play a crucial role in ensuring scalability and
performance of the system.

Interviews: Directly give the answer to questions to save time for the interviewer and yourself.

1. System Design: Discussing advanced system design concepts using animations for better
understanding.
2. Instagram: Mobile application that needs to connect to the server-side through a gateway
that encapsulates security mechanisms(like reverse proxy , where to send request to ,
authentication, token etc).
 3. Gateway: Can convert external protocols to an internal protocol for added security.
4. User Feed Service: Provides the top 20 posts for a user and is made resilient and scalable by
having multiple servers.
5. Load Balancer: Routes requests from the user's cellphone to a server and maintains a
snapshot of the entire system.
6. Consistent Hashing: Used to determine which server to send the request to by hashing the
user ID.
7. Dependent Services: User feed service depends on posts and follow services to get
information about the posts and users followed by a user.
8. Other Services: Image, activity, chat, and profile services also exist but are not directly
related to this application.
9. Finding User Feed: Simple way is to get all the users followed by a particular user ID and
retrieve the posts from the posts service.
10. Hidden Services: Other services such as image, activity, chat, and profile services are kept
hidden for now to focus on important features.
11. Future Discussions: The other services and features will be discussed in future videos.

Problem statement: A user needs to view the posts made by people they follow on Instagram.

1. Initial approach: Query the posts service for every user the user follows, which results in a
lot of wasteful computation and the post service gets bombarded with complex queries.
2. Optimizations:
3. a. Expose an API that takes in a set of user IDs and returns all posts in just one go, reducing
load on the database and the service.
4. b. Limit the number of posts that are called from the post service to reduce the load on the
database and the service.
5. The correct solution: Pre-compute the user feed anytime a user feed request comes in and
return the user feed in one shot.
6. Updating the user feed: Whenever a person the user follows posts something, the post
service should send a notification to the user feed service to update the pre-computed user
feed.
7. Storing the user feed: Use cache to store the user feeds as it separates the logic and can be
recomputed if there is a problem with memory. The most recently used users will have their
user feed served by cache.
8. Notifications: The user feed service can send notifications to relevant users whenever a new
post is made. These notifications can be sent using polling or push notifications.
9. Use of cache: The cache will store the user feeds and it will be quickly available to the users.
The cache management policies will take care of the user feeds for infrequently logging in
users.

In conclusion, the correct solution to the user's problem is to pre-compute the user feed and store it
in the cache. This approach is efficient as it reduces the load on the database and the service, and
provides quick access to the user's feed. The user feed service can also send notifications to
relevant users whenever a new post is made.

Follower user Id follow ID timestamp

Gaurav Stephen hawker today

This will answer who follows user x

Which user does user x follows

Video 30 :
System Design: Online Judge for coding contests - YouTube
Remote code execution engine:

● A system that takes code from different users and tests whether it passes the problem
statement.
● The code is submitted to the system and evaluated to give a result, either accepted (AC) or
rejected.
● Common scenario in competitive programming contests, interviews, or practice.

Initial Simple Approach:

● Have a server that takes the code, runs it, and stores the result in a database as either
accepted or rejected.
● Issues: can't handle thousands of users submitting code at the same time (rate-limiting is
not a preferred solution).

Improved Approach:

● Use a message queue to store the code instead of processing it immediately.

● Acknowledge the upload of the file to the user immediately.
● Publish an event to the event bus with the relevant details to execute the file at a later time.
● Make the system asynchronous to handle processing power limitations.
● Server processes events one by one according to its capacity.
● Server subscribes to events and processes them from the tail of the queue.
● Event is executed through a command on the server, given an input and receiving an output.
● The output is checked against the required test case output.
● Result is persisted in a database depending on the output match.

Note: The server is probably going to have a lot of microservices, but for the 10,000 feet view, it can
be seen as a single black box.

Video 33:
System Design: Live Streaming Events like ESPN and Hotstar - YouTube

Video Flow:

● Ingest live video stream

● Use RTMP protocol for reliable video delivery, as it is written over TCP (guarantees no data
loss and ordering)
● Store the original video in multiple databases for backup
● Transform the video into different resolutions and formats in a transformation service
● Use a job scheduler for hundreds of tasks (e.g. converting 8K footage into 720p)
● Assign tasks to worker nodes
● Workers pick up tasks, convert the video, and store the result in a distributed file system
● Publish an event to notify subscribers of task completion
● Subscribers (including the transformation service) can pull the video from the file system
and feed it to end users.

Video 27 :

Containers and Virtualisation in Cloud Computing ☁️- YouTube

Virtualization and Containers: A Guide
I. Introduction
A. Overview
- Virtualization and containers have become important topics in software engineering, particularly in
cloud computing.
B. Background
- Before virtualization, when writing code, developers had to make capacity planning guesses about
the amount of compute, memory, storage, etc. needed for their application and then buy a computer
to run it.
- If the business scaled, the hardware investment would need to be made again, leading to large
upfront costs.
- To mitigate this issue, organizations would let multiple employees use the same computer, but this
would result in resource contention.
C. Problem statement
- Isolate resource usage between applications.
D. Technical details

- Programs require a number of resources such as memory, IO, processing, and disk.
- The operating system is responsible for managing these resources.
- However, the problem of shared compute remains, where one program may take up all the
resources, leaving others without any.
II. Virtual Machines
A. Introduction
- Virtual machines provide a strong boundary between applications.
B. Technical details
- A virtual machine is like an operating system running on top of an operating system (i.e., the real
world is hidden behind a virtual world).
- Each virtual machine has been assigned a set of resources, isolated from other virtual machines.
C. Benefits
- Makes a new business model possible, called cloud computing.
- Companies like Amazon and Google can rent out their spare hardware to small businesses.
- Code does not need to be platform dependent.
- Flexible provisioning of resources.
III. Conclusion
A. Game Changer
- Platform independence is the most significant advantage of virtualization and containers.

Introduction to Containers:

● Containers are a form of virtualization that allows for app isolation.

 ● They are faster than traditional virtual machines in terms of boot times because they are
lightweight and only require the necessary resources such as processing power, memory,
disk, and IO.
● Technologies like Docker have made it easier for developers to specify the required
resources and operating system for their applications without having to worry about the
underlying hardware.
● The process of building and tearing down containers is called mounting and unmounting,
similar to mounting and unmounting disks in traditional virtualization.

Advantages of Containers:

● Easier resource management

● Faster boot times
● More control for developers
● Improved application compatibility

Disadvantages of Containers:

● Potential slow performance

● Possible firewall issues
● Overhead of container management
● Not necessary for simple applications

Conclusion:
Containers provide a flexible and efficient way to virtualize applications, but they may not be
necessary for all cases. It is important to consider the advantages and disadvantages when deciding
whether to use containers in a particular situation.

● The concept of virtualization and containers comes from the problem of capacity planning in
software engineering. Previously, when a developer wrote code, they would have to estimate
the amount of compute, memory, and storage they needed and buy a computer to run it on.
This was a large investment, and if the business scaled, they would have to repeat the
process.
● To address this issue, organizations let employees use the same computer, but this led to
contention for resources. The solution was to isolate resource usage through the operating
system, which would manage the allocation of memory, IO, processing, and disk space to
different programs.
● However, there was still the problem of shared compute where if one program took up too
many resources, it would affect others. Virtual machines provided a solution by creating a
strong boundary between programs. A virtual machine is like an operating system running
on top of an operating system, giving each program its own set of resources and creating a
fake world that they can interact with, without concerning themselves with what other
programs are running in the same hardware.
● The concept of virtualization made cloud computing possible, where large companies like
Amazon and Google can rent out their spare hardware to small businesses. This allows
small businesses to avoid the upfront cost of buying and maintaining a computer and
eliminates the need for code to be platform dependent. The provisioning of resources is
also flexible and dynamic, as a virtual machine can be shut down and restarted as needed.
● Virtualization provides platform independence, flexibility, and dynamic provisioning, making
it a crucial aspect of cloud computing.
video 2 : components of system design

Logical Entities:

1. Database: Technology to store data that can be available to users in the future.
2. Application Layer: Code running on a machine that allows users to interact with the
database.
3. Communication Protocols: Enables communication between different machines so the
components can interact with each other.
4. Presentation Layer (optional): How the system is presented to the user (mobile apps,
desktop apps, websites, etc.).

Tangible Entities:

1. Databases: Options like MongoDB, MySQL, Cassandra, Redis, etc.

2. Application and Services: APIs, RPCs, etc. for code and communication between
components.
3. Presentation Layer: Front-end applications built using frameworks like Amber, React, etc.
for desktop apps, websites, and mobile apps have their own native codebases.
4. Security Mechanisms and Protocols: To secure the system and data and avoid attacks.
5. Instances: Physical computers provided by cloud providers (AWS, GCP, Azure, etc.) to
house all these technologies.

System Overview:

1. Presentation Layer: Where the system is presented to the user through desktop apps,
websites, or mobile apps.
2. Applications: Interact with databases for the exchange of data.
3. Databases: Stores data for the system.
4. Instances: Physical computers that house applications and databases and interact with each
other over network.
5. Communication: Applications interact with each other through APIs and messages.
6. Infrastructure: All components housed inside a cloud provider (AWS, GCP, etc.).

Components of a System:

1. Applications
2. Databases
3. Caches
4. Load Balancers
5. Client Interfaces
6. Network Request
7. Security Layer
8. Infrastructure.

Video 37: online code editor

System Design of an Online Code Editor with @CSDojo - YouTube

Here are the structured notes based on the information provided:
Problem:

● Need a job scheduler to avoid overloading services

● Need a container for security and to avoid overloading services

Situation:

● System works well for evaluation/competition type of situation

● Question is how to extend it to an online IDE where thousands of people may use it at the
same time and expect fast results

Requirement:

● Low latency
● Real-time response to the user
● Quick response from the server
● Avoid processing requests periodically

Solution:

● Change architecture to request-response model

● Get code as text with profile ID and programming language type (e.g. Python, Java, C++) as
parameters
● Server acknowledges receipt of request but doesn't know response yet
● Show thinking button on the browser (product team to decide on better user experience)
● Use task queue on the back-end to isolate speed of inputs, speed of requests, and local
computation power
● Assign requests to available containers (workers)
● Keep requests in queue if no containers are available
● Horizontally scale (add more servers) if necessary
● Once computation result is calculated (standard out, standard error), send back as response
to the server as an event
● Event should have details such as request ID, output, status (success or failure), etc.

1. Introduction:
● The conversation involves discussing a system for adding and executing code in
real-time
● The system involves the use of a stateful server and session-based Linux containers
2. Adding Code:
● Every time a person adds code in the session, the code is associated with a session
ID (e.g. "session ID 123")
● The code is added as a block with line numbers (e.g. line number 5)
3. Executing Code:
● For each browser session, a new Linux container is started and kept alive as long as
the user is present
● The code is executed on the container and the output is sent back to the user as a
response
● The container has its own space, hard disk space, network, and files, making it a
unique operating system
4. Server State and Persistent Storage:
● The system's statefulness and use of Linux containers ensure that code execution
does not need to be recomputed every time a new line is added
 ● If the system crashes, the containers can be restarted by looking at the persistent
storage (e.g. database) and building a container based on the stored code
5. Problem: Storing Results and Timestamps:
● Storing results can be a challenge as they cannot be stored onto variables
● The problem of timestamps (e.g. the real-world factor of time) can be solved by
storing the information that can change the state later on at the start of the code
execution
6. Mitigating System Crashes:
● If an engineer writes the system, crashes may occur quite often
● The system should be scalable as more users and containers are added
● Possible causes of crashes include power loss, data corruption, and network
partition
● To mitigate the effects of crashes, the database stores the code, but in case of a
crash, the user may have to wait for a few seconds
● The goal is to know about a container crash as quickly as possible, without
disrupting the user's experience while they are writing code.

Video 19 : CAP theorem

CAP | Consistency, Availability and Partitioning | System Design Tutorials | Lecture 19 | 2020 -
YouTube
CAP Theorem:

● CAP theorem is a concept in computer science that defines the limitations of distributed
systems in regards to Consistency, Availability, and Partition tolerance.
● It states that it is impossible for a distributed system to simultaneously provide all three of
these guarantees.
● A distributed system can only provide two of the three guarantees at a given time.

Consistency:

● Consistency refers to the property that all nodes in a system see the same data at the same
time.
● It ensures that the system's data is consistent and up-to-date across all nodes.
● In a system with strong consistency, all nodes have the same view of the data, and any
change made to the data is immediately visible to all nodes.

Availability:

● Availability refers to the property that a system is always able to respond to a request,
regardless of the state of the system.
● It ensures that a system is able to serve its clients even in the event of failures or network
partitions.

Partition Tolerance:

● Partition tolerance refers to the property that a system continues to operate even when
there is a network partition or communication failure between nodes.
● It ensures that the system is able to operate even when there is a failure in communication
between nodes.
Choosing between Consistency and Availability:

● The choice between consistency and availability depends on the specific requirements of
the system.
● If consistency is more important, the system should sacrifice some level of availability in
order to ensure that all nodes have the same view of the data.
● If availability is more important, the system should sacrifice some level of consistency in
order to ensure that it is always able to respond to requests.

CAP Theorem in Practice:

● In practice, most systems aim to strike a balance between consistency and availability, and
trade-off some level of partition tolerance.
● Different systems make different trade-offs based on their specific requirements.
● For example, in a banking system, consistency is more important than availability. In such a
system, it is more important to ensure that all nodes have the same view of the data, even if
the system is unavailable for a brief period of time.
● On the other hand, in a social media system, availability is more important than consistency.
In such a system, it is more important to ensure that the system is always available, even if
the data is inconsistent for a brief period of time.

Conclusion:

● CAP theorem is a fundamental concept in computer science that defines the limitations of
distributed systems.
● It states that a distributed system can only provide two of the three guarantees of
Consistency, Availability, and Partition tolerance at a given time.
● The choice between consistency and availability depends on the specific requirements of
the system, and most systems aim to strike a balance between the two.

Ritik

Monolith: A monolithic architecture is the traditional unified model for the design of a software
program. Monolithic, in this context, means "composed all in one piece." According to the
Cambridge dictionary, the adjective monolithic also means both "too large" and "unable to be
changed."

Microservice: Microservices architecture consists of collections of light-weight, loosely-

coupled services. Each service implements a single business capability. Ideally, these
services should be cohesive enough to develop, test, release, deploy, scale, integrate, and
maintain independently.

Advantages of a monolithic architecture

Organizations can benefit from either a monolithic or microservices
architecture, depending on a number of different factors. When developing
using a monolithic architecture, the primary advantage is fast development
speed due to the simplicity of having an application based on one code base.

The advantages of a monolithic architecture include:

Easy deployment – One executable file or directory makes deployment easier.

Development – When an application is built with one code base, it is easier to

develop.

Performance – In a centralized code base and repository, one API can often
perform the same function that numerous APIs perform with microservices.

Simplified testing – Since a monolithic application is a single, centralized unit,

end-to-end testing can be performed faster than with a distributed application.

Easy debugging – With all code located in one place, it’s easier to follow a
request and find an issue.

Disadvantages of a monolithic architecture

As with the case of Netflix, monolithic applications can be quite effective until
they grow too large and scaling becomes a challenge. Making a small change
in a single function requires compiling and testing the entire platform, which
goes against the agile approach today’s developers favor.

The disadvantages of a monolith include:

Slower development speed – A large, monolithic application makes
development more complex and slower.

Scalability – You can’t scale individual components.

Reliability – If there’s an error in any module, it could affect the entire

application’s availability.

Barrier to technology adoption – Any changes in the framework or language

affects the entire application, making changes often expensive and time-
consuming.

Lack of flexibility – A monolith is constrained by the technologies already used

in the monolith.

Deployment – A small change to a monolithic application requires the

redeployment of the entire monolith.

Advantages of microservices

Agility – Promote agile ways of working with small teams that deploy
frequently.

Flexible scaling – If a microservice reaches its load capacity, new instances of

that service can rapidly be deployed to the accompanying cluster to help
relieve pressure. We are now multi-tenanant and stateless with customers
spread across multiple instances. Now we can support much larger instance
sizes.
Continuous deployment – We now have frequent and faster release cycles.
Before we would push out updates once a week and now we can do so about
two to three times a day.

Highly maintainable and testable – Teams can experiment with new features
and roll back if something doesn’t work. This makes it easier to update code
and accelerates time-to-market for new features. Plus, it is easy to isolate and
fix faults and bugs in individual services.

Independently deployable – Since microservices are individual units they allow

for fast and easy independent deployment of individual features.

Technology flexibility – Microservice architectures allow teams the freedom to

select the tools they desire.

High reliability – You can deploy changes for a specific service, without the
threat of bringing down the entire application.

Happier teams – The Atlassian teams who work with microservices are a lot
happier, since they are more autonomous and can build and deploy
themselves without waiting weeks for a pull request to be approved.

Disadvantages of microservices

Development sprawl – Microservices add more complexity compared to a

monolith architecture, since there are more services in more places created by
multiple teams. If development sprawl isn’t properly managed, it results in
slower development speed and poor operational performance.

Exponential infrastructure costs – Each new microservice can have its own
cost for test suite, deployment playbooks, hosting infrastructure, monitoring
tools, and more.

Added organizational overhead – Teams need to add another level of

communication and collaboration to coordinate updates and interfaces.

Debugging challenges – Each microservice has its own set of logs, which
makes debugging more complicated. Plus, a single business process can run
across multiple machines, further complicating debugging.

Lack of standardization – Without a common platform, there can be a

proliferation of languages, logging standards, and monitoring.

Lack of clear ownership – As more services are introduced, so are the number
of teams running those services. Over time it becomes difficult to know the
available services a team can leverage and who to contact for support.

Detecting anamolies using Server Side: Watch Video

Performance metrics are defined as figures and data representative of an organization's

actions, abilities, and overall quality.

Four metrics are: Throughput, Bandwidth, Latency, Response Time.