Foundations of System Design

1.1 What is System Design?

System design is the process of planning and structuring the architecture of a software system based
on user requirements. It defines how components (like front-end, services, databases) work together to
meet functionality, performance and non-functional goals. In other words, system design translates
user needs into a technical blueprint, specifying modules, data flows, and interactions. For example,
designing an online shopping system involves deciding how the user interface, product catalog service,
payment processing, and databases will interact. A good system design balances complexity and
efficiency to meet requirements like capacity, responsiveness, and cost.

Key aspects: Components and interfaces; data models and flow; technology choices and
deployment.
Purpose: Create a scalable, maintainable architecture that can be built by teams and meet non-
functional goals (scalability, availability, etc.).

System design is often split into High-Level Design (HLD) – outlining major modules and their
relationships – and Low-Level Design (LLD) – detailing classes, data structures, and algorithms. HLD
defines the big picture (services, databases, APIs), while LLD drills into the internal logic of each part.

1.2 Goals of System Design

A well-designed system meets key non-functional goals. The primary goals are Scalability, Reliability,
Availability, Maintainability, and Security. Each has trade-offs and impacts architecture choices:

Scalability: The ability to handle increased load (users, data, requests) by adding resources. In
practice, a scalable system can grow horizontally (adding servers, partitions) or vertically (bigger
machines) without performance degradation. For example, an e‑commerce site must scale up
during Black Friday spikes. Techniques include load balancing, caching (Redis, CDNs), data
sharding, and asynchronous processing (message queues). Trade-offs: scalability often increases
complexity and cost. Horizontal scaling requires partitioning data (which can complicate
consistency), and adding caches/CDNs can make invalidation harder. Pitfall: Ignoring bottlenecks
(e.g. a single database) leads to crashes at scale.

Reliability: Ensures the system works correctly over time despite failures. A reliable system
“keeps working correctly even when there are failures (hardware crashes, network issues, bugs)”.
Techniques include redundancy (replicating servers and data), fault tolerance (designing services
 to handle errors gracefully), and automated recovery (circuit breakers, retries). Netflix pioneered
“chaos engineering” (Chaos Monkey) to test automated recovery in production. Trade-offs:
reliability usually means extra servers and code to handle failures, increasing cost and complexity.
For instance, banking systems must not lose transactions even if a server fails. Pitfall: Not
planning for partial failures – e.g. assuming a single network call always succeeds – can cause
cascading outages.

Availability: The system’s uptime and responsiveness to user requests. High availability means
“the system remains operational and accessible despite failures or disruptions”. Achieved via
failover (spinning up backups if one instance dies), load balancing, and geo-replication. For
example, Netflix serves content from a CDN (Open Connect) near users to maximize availability
and minimize latency. Trade-offs: replicating services and data across regions costs resources.
The CAP theorem applies: under partitions, you often sacrifice consistency for availability (e.g.
social media feeds use eventual consistency so the site is always up). Pitfall: Focusing only on
availability can mask data inconsistency; ignoring SLAs (service-level agreements) can lead to
outages.

Maintainability: How easy it is to update, fix, and extend the system over time. Maintainable
systems use modular design, clear APIs, and clean code. For example, a microservices
architecture is often more maintainable than a monolith, since “each service can be updated
independently”. Investing in good documentation, automated tests, and CI/CD pipelines also
improves maintainability. Trade-offs: microservices can improve modularity but add operational
overhead. Poor maintainability (tangled code, no tests) leads to slow development and technical
debt.

Security: Protecting the system from unauthorized access and attacks. This involves ensuring
Confidentiality (data only accessible to authorized users), Integrity (data is accurate and
unaltered), and Availability (the system is not disrupted by attacks). In practice, security measures
include authentication (OAuth, JWT), authorization checks, input validation (to prevent SQL
injection/XSS), encryption (TLS for data-in-transit, AES for at-rest), and rate limiting (to mitigate
DDoS). For example, many designs place an API Gateway at the front to handle auth and SSL,
protecting internal services. Trade-offs: security layers add latency and complexity. Always
design security from the start – reactive fixes often fail. Pitfall: “Rolling your own” crypto or
skipping input validation; forgetting least-privilege principles can expose vulnerabilities.

1.3 Core Architectural Layers

Most large systems adopt a layered architecture. The common layers are:

Presentation Layer (UI): The user interface – web browsers, mobile apps, etc. It handles
displaying data and collecting user input. This layer uses frameworks like React, Angular, or native
mobile SDKs. Its focus is responsiveness and ease-of-use. For instance, Netflix’s front-end uses
[Link] for a fast, dynamic interface. Design considerations include minimizing latency (e.g.
optimize JavaScript, use content caching), and separating static from dynamic content (CDN for
video). The presentation layer should have no business logic – it just calls the backend APIs.
Pitfall: Overloading it with logic or heavy assets can slow the UI.

Application Layer (Business Logic): This is the middle tier (sometimes split into multiple tiers or
microservices). It contains the core functionality: processing inputs, enforcing rules,
orchestrating workflows, and integrating with other systems. In a web system, this often runs on
application servers or cloud services. For example, in an online store this layer handles user
authentication, product catalog search, payment processing, etc. In microservices architectures,
each service (e.g. User Service, Order Service) lives here, with its own APIs. Design trade-offs
include synchronous vs asynchronous flows (e.g. real-time RPC calls vs event-driven queues), and
how much logic to push to client vs server. This layer interacts with the data layer via well-defined
APIs. Netflix’s backend (Java, Spring Boot microservices on AWS) exemplifies this: each service
(User, Video Metadata, Transcoder, Search) encapsulates a piece of business logic. Pitfall: Mixing
unrelated logic in one service makes it hard to scale or update; ignoring failure in downstream
services (lack of circuit breakers) risks cascading faults.

Data Layer (Storage): All storage and retrieval of persistent data happen here. This includes
relational databases (MySQL, PostgreSQL), NoSQL stores (Cassandra, MongoDB), key-value caches
(Redis, Memcached), search indexes (Elasticsearch), and CDNs. For example, Netflix uses
Cassandra for some user-data storage and S3/HDFS for videos. The data layer is designed for
durability, consistency, and performance. Common patterns: replication (multiple copies of data),
sharding/partitioning (splitting large tables by key), read replicas, and caching. Trade-offs:
relational databases provide strong consistency (ACID) but can be harder to scale horizontally;
NoSQL can scale easily but may only guarantee eventual consistency. Indexes speed up reads at
the cost of slower writes. Pitfall: N+1 query problems or lack of caching can kill performance; poor
schema design leads to inefficient queries.

[ Presentation Layer (UI) ]

↓
[ Application Layer ]
↓
[ Data Layer ]

Integration Layer: This layer glues systems together and manages cross-cutting concerns. It
often includes API Gateways, Message Brokers, and Service Buses. For example, an API Gateway
(like Netflix’s Zuul) sits at the edge of the application layer to route requests, handle
authentication, rate limiting, and load balancing. Message brokers (Kafka, RabbitMQ) let services
communicate asynchronously, decoupling producers from consumers. The integration layer may
also handle ETL (extract-transform-load), data validation, and protocol translation between
services. Its role is to ensure seamless data exchange and reliability: e.g. validate and transform
data as it passes between the application and data layers. In complex organizations, an Enterprise
 Service Bus (ESB) or graph-based API layer may aggregate multiple internal/external APIs. Pitfall:
A monolithic ESB can become a bottleneck; using point-to-point service calls without an
integration strategy leads to spaghettis of dependencies.

[Client] --> [API Gateway] --> [Microservice A] --> [DB_A]

|
--> [Microservice B] --> [DB_B]
|
--> [Message Broker / Integration Bus]
|
[Service C] etc.

Each layer can be scaled or updated independently. For instance, caching static content at the edge
(presentation), scaling application servers (application), and adding database replicas (data) are
independent steps. This layered separation also improves security – the presentation and data layers
never talk directly (the application layer mediates, acting like an internal firewall).

1.4 High-Level Design vs Low-Level Design

System design is often discussed in two complementary stages:

High-Level Design (HLD): Focuses on the overall architecture. HLD diagrams show major
components, services, and their interactions – effectively a “map” of the system. It identifies key
modules (web servers, databases, caches, external services), communication patterns, and data
flow. For example, an HLD might show an e‑commerce system with a load balancer, three
microservices (User, Product Catalog, Payment), their databases, and a CDN. HLD addresses
trade-offs: how to meet scale (use microservices or monolith), consistency (choose SQL or
NoSQL), and failover (multi-region deployment). It also includes choice of technologies (e.g. Kafka
for events, Redis for cache). HLD is typically created by architects; it provides enough detail to
guide technology decisions but abstracts away code specifics.

Low-Level Design (LLD): Drills down into the internals of each module identified in HLD. LLD
specifies classes, methods, data structures, and algorithms. It includes API definitions
(endpoints, request/response formats), database schemas (tables, columns, indexes), and the
logic within components. For instance, for a User Service, LLD would detail the user class, its
fields, how registration and authentication work step-by-step, and how errors are handled. LLD
also covers design patterns (like Singleton, Factory), interfaces between classes, and error-
handling strategies. The output of LLD is like a blueprint for developers: class diagrams, sequence
diagrams, and pseudocode.

Differences: HLD is about components and interaction; LLD is about implementation details. HLD
answers “what blocks do we have?” and “how do they communicate?”, while LLD answers “what’s inside
each block?”. For example, Netflix’s HLD might show independent microservices, but its LLD for the
Streaming Service would show the code flow for transcoding and streaming data to the player.

When to use: HLD is used early to validate architecture choices against requirements. LLD is done once
components are set, to ensure clean code. In interviews, you might start with an HLD sketch (like
drawing service boxes) and then dive into one or two components at LLD level (defining classes or
database tables) for detail.

1.5 Monolithic vs Distributed Systems

A monolithic system is built as one unified codebase or deployable artifact. All functionality (UI,
business logic, data access) runs in a single process or group of tightly-coupled processes. For example,
an early-stage startup might ship all features in one web app and one database. Monoliths are easy to
develop and deploy at first: there’s one project, one deployment. Advantages include simple testing and
no cross-service communication overhead. However, as the codebase grows, a monolith becomes hard
to maintain and scale. Scaling a monolith means duplicating the whole app (even parts that don’t need
it). Also, a bug in one module can crash the entire application.

In contrast, a distributed (often microservices) architecture decomposes functionality into many small,
independent services. Each service owns its code and usually its data store. For example: Auth Service,
User Service, Payment Service each as separate processes with their own database. Clients or
gateways route requests to the appropriate service. The advantages: each service can be developed,
deployed, and scaled independently. Teams can work in parallel, using different languages or
frameworks. Netflix famously moved from a monolith to ~1000 microservices on AWS, enabling much
faster deployments. Microservices also improve fault isolation: a failure in one service doesn’t
necessarily bring down others.

However, distributed systems come with trade-offs. They introduce network latency and complex
failure modes (partial failures, data consistency issues). Service-to-service communication
(REST/gRPC calls, messaging) needs robust retry and timeout logic. Distributed deployments require
orchestration (e.g. Kubernetes), service discovery, and end-to-end monitoring. Microservices can also
lead to operational overhead (“communication overhead”): each service needs its own CI/CD pipeline,
logging, and tracing. Debugging across services can be harder. The cost (servers, databases, infra)
often grows.

Monolith vs Microservices – Trade-offs:

Development Speed: Monoliths allow fast initial development (one codebase); microservices
enable parallel work by teams.
Scalability: Monoliths can only scale as a whole (hard to scale one function); microservices can
scale individual services (if the User service is hot, scale only that).
Reliability: Monolith crashes can take the whole app down; microservices failure is isolated (with
good design, one service down doesn’t kill the site).
 Complexity: Monoliths are simpler to deploy initially; microservices add distributed systems
complexity (data consistency, network issues).

When to use: Monoliths work well for small teams or projects with limited scope. If an app is simple and
unlikely to grow much, a monolith is pragmatic. Distributed systems shine for large-scale applications
with many users (e.g. Amazon, Netflix, Twitter) or for products with clearly separable domains (e.g.
Payments vs Social Feed). In interviews, you might be asked when to break a monolith: common triggers
are performance bottlenecks or organizational scaling needs. For instance, Twitter moved to a
distributed architecture to serve 6,000 tweets/sec, using strategies like precomputed timelines and
caching for scale. Uber uses a microservices/event-driven design to handle real-time ride matching and
tracking.

Monolith vs Distributed (ASCII Diagram):

Monolith:
+----------------------------+ +----------+
| Entire App Code | ----> | Database |
+----------------------------+ +----------+

Microservices:
+--------+ +--------+ +--------+ +----------+
| Auth | | User | | Order |--> | DB_Auth |
|Service | |Service | |Service | +----------+
+--------+ +--------+ +--------+ +----------+
| | | -> | DB_User |
v v v +----------+
[Service Discovery/Load Balancer] -> | DB_Order |
+----------+

Pitfalls & Considerations: In distributed systems, watch out for network latency and partial failures:
implement timeouts, retries, and fallbacks (circuit breakers). Data consistency can become tricky
(consider CAP theorem). Also plan for monitoring and tracing from the start, since many small moving
parts need observability. Monoliths risk becoming a “big ball of mud” if not modularized; microservices
risk being an “integration nightmare” if there’s no clear API and data strategy.

Interview Perspective

System design interviews often probe these foundational topics. You may be asked to explain or
compare them, or design a system under these constraints. For example:

What is system design? (Expect to define it as planning the architecture for requirements,
focusing on modules, data flow, performance.)
HLD vs LLD: Interviewers frequently ask you to contrast high-level and low-level design (as part of
a design problem). Be ready to draw a component diagram (HLD) and then pick one component to
design in detail (LLD).
Goals: You may be asked “how would you ensure reliability/scalability?” or “what are your design
priorities if the system must always be up 99.999% of the time?” Cite goals like scalability,
availability, etc., and talk about techniques (e.g. “To scale, I would add caching and partition data”;
“To increase availability, use multiple AZs and health checks”). Use examples: e.g. “Netflix uses
CDNs and caches to minimize latency and maximize availability.”
Layers: For a typical web application design, interviewers expect mention of presentation,
application, data layers and why separation matters (e.g. “separate concerns for flexibility and
security”). You might sketch a 3-tier (UI → App → DB) or microservices API gateway pattern.
Monolith vs Distributed: Common questions include “When would you break a monolith?” or
“Design an e‑commerce site: monolith or microservices?” Be prepared to list pros/cons of
monolithic vs microservices architectures (as above). Real-world cases: Netflix’s migration to
microservices, Twitter’s need for distributed caching, Uber’s event-driven design. Show
awareness of CAP theorem (consistency vs availability) if designing databases or caches.
Failure Scenarios: Interviewers often test failure handling: “What if a server/datacenter goes
down?” For instance, mention how Uber uses a backup datacenter and even driver phone state to
recover from a DC failure. Or how to handle a cache/database failure (fallback logic, retries).
Trade-offs & Scaling: Always discuss trade-offs (e.g. SQL vs NoSQL, consistency vs throughput,
latency optimizations). For example, “We could denormalize data for faster reads but that means
writes become more complex.” Show you understand latency and throughput (e.g. introduce
asynchronous queues to absorb bursts). Use examples like Twitter’s fan-out vs fan-in for timelines
to illustrate read vs write scaling choices.
Diagrams: Interviewers expect you to draw simple block diagrams (ASCII art on a whiteboard). For
layered architecture, you might draw a 3-tier stack; for microservices, a set of boxes behind a load
balancer. Diagrams should match your explanation of layers and data flow.