Code Level
Asynchronous:
Referring to a web app this means doing the time-consuming work in advance and serving the finished
work with a low request [Link] often this paradigm is used to turn dynamic content into static content.
Pages of a website, maybe built with a massive framework or CMS, are pre-rendered and locally stored
as static HTML files on every change. Often these computing tasks are done on a regular basis, maybe by
a script which is called every hour by a cronjob.
A user comes to your website and starts a very computing intensive task which would take several
minutes to finish. So the frontend of your website sends a job onto a job queue and immediately signals
back to the user: your job is in work, please continue to the browse the page. The job queue is constantly
checked by a bunch of workers for new jobs. If there is a new job then the worker does the job and after
some minutes sends a signal that the job was done. The frontend, which constantly checks for new “job is
done” - signals, sees that the job was done and informs the user about it. I know, that was a very
simplified example.
Key Scalability & Hosting Concepts
Web Hosting & Features: Minimum essential features should be evaluated when choosing a web host. Certain
companies or IP ranges (such as GoDaddy, YouTube, or Facebook) might be blocked by specific networks or
countries, requiring initial testing.
SFTP vs. FTP: Secure File Transfer Protocol (SFTP) is crucial over standard FTP because standard FTP
transmits usernames and passwords in cleartext across the network. SFTP encrypts all traffic.
Shared Hosting Constraints: Ultra-cheap "unlimited" hosting plans (like those from DreamHost) rely on virtual
hosting where hundreds of customers share the same physical server resources. If a business needs guaranteed,
growing resources, shared hosting is not ideal.
Virtual Private Servers (VPS):
A VPS provides its own isolated virtual machine/operating system running on top of a hypervisor (e.g.,
VMware, Citrix).
Resources are sliced per virtual machine, meaning no other customers hold user accounts on that specific
VM.
Privacy Note: System administrators/owners of the VPS provider can still access files or reboot the VM in
single-user mode to bypass root passwords.
Cloud Infrastructure (AWS EC2): Services like Amazon Web Services (AWS) EC2 allow self-service spawning
of virtual machines paid by the minute, allowing automated scaling up during unexpected traffic spikes (e.g.,
being posted on Reddit) and scaling down when traffic subsides.
Vertical vs. Horizontal Scaling
Vertical Scaling:
Involves upgrading a single machine by throwing more RAM, CPUs/cores, or disk space at it.
Multi-core/multi-CPU servers allow true parallel processing of multiple web requests.
Limitation: Hits a hardware technology ceiling or financial limit, as a single machine can only be
upgraded so far.
Horizontal Scaling:
Involves distributing the architecture across multiple cheaper or commodity servers instead of relying on
one expensive server.
Requires a way to distribute inbound HTTP requests across all backend servers.
The cloning the code is a difficult task but there are automation tools like Capistrano that help in
maintaining the updates and making sure each instance is running the exact same code.
Load Balancing & Sticky Sessions
DNS Round-Robin:
Configures DNS servers (such as BIND) to return different IP addresses sequentially for a given domain
name request.
Drawbacks:
1. Dumb distribution: Cannot detect server load, leading to potential hot-spotting if one server
receives compute-heavy users.
2. Caching & TTL: Browsers and operating systems cache DNS responses based on Time-To-Live
(TTL) settings, causing repeated requests from the same user to stick to one IP rather than
spreading round-robin.
Load Balancers:
Placed between client traffic and backend servers to distribute requests based on various heuristics (e.g.,
server load, round-robin, randomness).
Allows backend servers to use private IP addresses (e.g., 10.x.x.x or 192.168.x.x), keeping them
hidden and protected from direct internet access.
Implementations include software options (AWS ELB, HAProxy, Linux Virtual Server) and enterprise
hardware (Barracuda, Cisco, Citrix, F5).
Sticky Sessions Problem & Solutions:
Traditional PHP sessions store data locally in text files (e.g., /tmp), which breaks when a load balancer
routes a user to a different server between requests.
Solution 1 (Shared Storage/Database): Storing session state in a centralized file server (NFS, Fiber
Channel) or database (MySQL) shared across all web servers.
Solution 2 (Cookie-based Sticky Sessions): The load balancer inserts a cookie containing a unique
identifier mapping the client to a specific backend server on subsequent requests.
Performance Optimizations & Caching
Storage Options:
SATA / Parallel ATA: standard desktop/laptop drives.
SAS (Serial Attached SCSI): Spins faster (10,000–15,000 RPM vs. 7,200 RPM), ideal for database disk
writes.
SSD (Solid State Drives): Fastest performance due to no moving parts, but higher cost and smaller
capacities.
RAID (Redundant Array of Independent Disks):
RAID 0: Striping data across multiple drives for maximum performance (no redundancy).
RAID 1: Mirroring data across 2 drives for instant failover (50% storage overhead).
RAID 5: Requires 3+ drives; uses 1 drive's worth of capacity for parity, allowing 1 drive failure.
RAID 6: Dual parity, allowing any 2 drives to fail without data loss.
RAID 10: Combines striping (RAID 0) and mirroring (RAID 1) across 4+ drives.
PHP Acceleration: Tools (like APC or eAccelerator) cache compiled PHP opcodes in memory so PHP files do
not need to be re-parsed and re-interpreted on every request.
Caching Strategies:
Static HTML Caching: Generating dynamic content into static .html files on disk (Craigslist model)
allows high-speed serving by web servers, though it complicates sitewide design changes.
MySQL Query Cache: Caches identical SELECT query results in memory until underlying tables are
modified.
Memcached: An in-memory key-value store used to cache database query results or objects in RAM to
avoid expensive disk/database lookups. Uses Least Recently Used (LRU) principles/timestamps to evict
old items when RAM fills up.
what to chache: -The Database query: it is ok to cache it relatively easy also. -The objects: This are
been created in the server after doing multiple querys to the DB. It is like the final response that will be
going to user. So caching this is far more better and prevents going to db also.
Database Scaling & High Availability
MySQL Storage Engines:
InnoDB: Default engine supporting transactions and row-level locking.
MyISAM: Legacy engine using full-table locks without transaction support.
Memory (Heap): Table stored purely in RAM for fast, temporary cache-like access.
Archive: Automatically compresses data; optimized for log files where writes are frequent and reads are
rare.
Replication Configurations:
Master-Slave: Reads can be offloaded to multiple slave databases to scale read-heavy sites, while all
writes go to the Master. Slaves also serve as backups if the Master fails.
Master-Master: Two database servers replicate bidirectionally, allowing writes to either node and
eliminating the single point of write failure.
Partitioning: Dividing database content across distinct servers based on logical criteria (e.g., initial letter of user's
last name or specific school/network affiliations, as Facebook did early on).
High Availability (HA): Redundant paired systems (e.g., Active-Active or Active-Passive load balancers) that
monitor each other via network "heartbeats" to automatically take over IP addresses and operations if one node
fails.
Performance vs scalability
A service is scalable if it results in increased performance in a manner proportional to resources added. Generally,
increasing performance means serving more units of work, but it can also be to handle larger units of work, such as when
datasets grow.1 ([Link]
Another way to look at performance vs scalability:
If you have a performance problem, your system is slow for a single user.
If you have a scalability problem, your system is fast for a single user but slow under heavy load.
Latency vs throughput
Latency is the time to perform some action or to produce some result.
Throughput is the number of such actions or results per unit of time.
Generally, you should aim for maximal throughput with acceptable latency.
Availability vs consistency
CAP theorem
([Link]
primer/blob/master/images/[Link])
In a distributed computer system, you can only support two of the following guarantees:
Consistency - Every read receives the most recent write or an error
Availability - Every request receives a response, without guarantee that it contains the most recent version of the
information
Partition Tolerance - The system continues to operate despite arbitrary partitioning due to network failures
Networks aren't reliable, so you'll need to support partition tolerance. You'll need to make a software tradeoff between
consistency and availability.
CP - consistency and partition tolerance
Waiting for a response from the partitioned node might result in a timeout error. CP is a good choice if your business
needs require atomic reads and writes.
AP - availability and partition tolerance
Responses return the most readily available version of the data available on any node, which might not be the latest.
Writes might take some time to propagate when the partition is resolved.
AP is a good choice if the business needs to allow for eventual consistency or when the system needs to continue
working despite external errors.
Consistency patterns
With multiple copies of the same data, we are faced with options on how to synchronize them so clients have a
consistent view of the data. Recall the definition of consistency from the CAP theorem - Every read receives the most
recent write or an error.
Weak consistency
After a write, reads may or may not see it. A best effort approach is taken.
This approach is seen in systems such as memcached. Weak consistency works well in real time use cases such as
VoIP, video chat, and realtime multiplayer games. For example, if you are on a phone call and lose reception for a few
seconds, when you regain connection you do not hear what was spoken during connection loss.
Eventual consistency
After a write, reads will eventually see it (typically within milliseconds). Data is replicated asynchronously.
This approach is seen in systems such as DNS and email. Eventual consistency works well in highly available systems.
Strong consistency
After a write, reads will see it. Data is replicated synchronously.
This approach is seen in file systems and RDBMSes. Strong consistency works well in systems that need transactions.
Source(s) and further reading
Availability patterns
There are two complementary patterns to support high availability: fail-over and replication.
Fail-over
Active-passive
With active-passive fail-over, heartbeats are sent between the active and the passive server on standby. If the heartbeat
is interrupted, the passive server takes over the active's IP address and resumes service.
The length of downtime is determined by whether the passive server is already running in 'hot' standby or whether it
needs to start up from 'cold' standby. Only the active server handles traffic.
Active-passive failover can also be referred to as master-slave failover.
Active-active
In active-active, both servers are managing traffic, spreading the load between them.
If the servers are public-facing, the DNS would need to know about the public IPs of both servers. If the servers are
internal-facing, application logic would need to know about both servers.
Active-active failover can also be referred to as master-master failover.
Disadvantage(s): failover
Fail-over adds more hardware and additional complexity.
There is a potential for loss of data if the active system fails before any newly written data can be replicated to the
passive.
Replication
Master-slave and master-master
This topic is further discussed in the Database section:
Availability in numbers
Availability is often quantified by uptime (or downtime) as a percentage of time the service is available. Availability is
generally measured in number of 9s--a service with 99.99% availability is described as having four 9s.
99.9% availability - three 9s
Duration Acceptable downtime
Downtime per year 8h 45min 57s
Downtime per month 43m 49.7s
Downtime per week 10m 4.8s
Downtime per day 1m 26.4s
99.99% availability - four 9s
Duration Acceptable downtime
Downtime per year 52min 35.7s
Downtime per month 4m 23s
Downtime per week 1m 5s
Downtime per day 8.6s
Availability in parallel vs in sequence
If a service consists of multiple components prone to failure, the service's overall availability depends on whether the
components are in sequence or in parallel.
In sequence
Overall availability decreases when two components with availability < 100% are in sequence:
Availability (Total) = Availability (Foo) * Availability (Bar)
If both Foo and Bar each had 99.9% availability, their total availability in sequence would be 99.8%.
In parallel
Overall availability increases when two components with availability < 100% are in parallel:
Availability (Total) = 1 - (1 - Availability (Foo)) * (1 - Availability (Bar))
If both Foo and Bar each had 99.9% availability, their total availability in parallel would be 99.9999%.
Domain name system
A Domain Name System (DNS) translates a domain name such as [Link] ([Link] to an
IP address.
DNS is hierarchical, with a few authoritative servers at the top level. Your router or ISP provides information about which
DNS server(s) to contact when doing a lookup. Lower level DNS servers cache mappings, which could become stale due
to DNS propagation [Link] propagation delay is the time it takes for a change made to a DNS record (like
updating a domain's IP address) to spread across the entire internet and take effect globally. DNS results can also be
cached by your browser or OS for a certain period of time, determined by the time to live (TTL).
Services such as CloudFlare and Route 53 provide managed DNS services. Some DNS services can route traffic
through various methods:
Weighted round robin
Prevent traffic from going to servers under maintenance
Balance between varying cluster sizes
A/B testing
Latency-based
Geolocation-based
Disadvantage(s): DNS
DNS Spoofing / Cache Poisoning: Attackers can inject false IP mappings into a DNS resolver's cache,
redirecting users to malicious phishing websites without altering the actual URL in the address bar.
Amplification Attacks: Because DNS primarily uses connectionless UDP, attackers spoof a victim's IP address
and send small requests that return large DNS responses, overwhelming the victim with amplified traffic (DDoS).
Propagation Downtime: Changes do not take effect globally at once, leading to an inconsistent user experience
across different regions and ISPs.
Content delivery network
([Link]
primer/blob/master/images/[Link])
A content delivery network (CDN) is a globally distributed network of proxy servers, serving content from locations closer
to the user. Generally, static files such as HTML/CSS/JS, photos, and videos are served from CDN, although some CDNs
such as Amazon's CloudFront support dynamic content. The site's DNS resolution will tell clients which server to contact.
Serving content from CDNs can significantly improve performance in two ways:
Users receive content from data centers close to them
Your servers do not have to serve requests that the CDN fulfills
Push CDNs
Push CDNs receive new content whenever changes occur on your server. You take full responsibility for providing
content, uploading directly to the CDN and rewriting URLs to point to the CDN. You can configure when content expires
and when it is updated. Content is uploaded only when it is new or changed, minimizing traffic, but maximizing storage.
Sites with a small amount of traffic or sites with content that isn't often updated work well with push CDNs. Content is
placed on the CDNs once, instead of being re-pulled at regular intervals.
Pull CDNs
Pull CDNs grab new content from your server when the first user requests the content. You leave the content on your
server and rewrite URLs to point to the CDN.
Concrete Example
1. Without CDN URL Rewriting Your web application serves HTML that points directly to your origin server:
HTML
<!-- The browser asks YOUR origin server for the image -->
<img src="[Link] />
<script src="[Link]
2. With CDN URL Rewriting When using a Push CDN, you upload [Link] directly to the CDN storage (e.g., S3 /
CloudFront). Then, you update your application so it renders HTML pointing to the CDN's domain:
HTML
<!-- The browser asks the CDN directly for the image -->
<img src="[Link] />
<script src="[Link]
Why You Must Do This for Push CDNs
Push CDN vs. Pull CDN: In a Pull CDN, the CDN automatically proxies requests back to your origin server when
an item is missing. In a Push CDN, the CDN has no link back to your server. It only serves files you explicitly
pushed to it.
Direct Offloading: If you do not update the URL paths in your frontend code, user browsers will continue hitting
your origin server for images and static assets, rendering the Push CDN unused.
Automated Rewriting: Developers rarely change URLs manually. Modern build tools (Webpack, Vite, [Link]) or
framework view helpers automatically prefix static assets with an environment variable like ASSET_HOST=
[[Link] during application deployment or build pipelines.
This results in a slower request until the content is cached on the CDN.
A time-to-live (TTL) determines how long content is cached. Pull CDNs minimize storage space on the CDN, but can
create redundant traffic if files expire and are pulled before they have actually changed.
Sites with heavy traffic work well with pull CDNs, as traffic is spread out more evenly with only recently-requested content
remaining on the CDN.
Disadvantages of cdn
Data Transfer Spikes: CDN pricing is typically based on bandwidth usage. An unexpected viral traffic surge, a
DDoS attack (if not properly mitigated by high-tier protection), or scraped assets can cause massive, unbudgeted
cloud bills.
Purge Delays: When you release a hotfix or update a critical asset (like a JavaScript file or stylesheet), clearing it
across hundreds of global edge nodes takes time.
Stale Asset Bugs: If cache headers (like Cache-Control) or TTLs are improperly configured, users may
continue to receive outdated or broken versions of your frontend code.
Cascading System Failures: Because full-site proxy CDNs sit directly in front of your domain, you are
completely reliant on their uptime.
Geo-Blocking Restrictions: Certain CDNs or edge locations may be blocked or throttled by regional internet
firewalls (e.g., in China or specific national ISPs), restricting accessibility for localized user bases.
Load balancer
([Link]
primer/blob/master/images/[Link])
Load balancers distribute incoming client requests to computing resources such as application servers and databases. In
each case, the load balancer returns the response from the computing resource to the appropriate client. Load balancers
are effective at:
Preventing requests from going to unhealthy servers
Preventing overloading resources
Helping to eliminate a single point of failure
Load balancers can be implemented with hardware (expensive) or with software such as HAProxy.
Additional benefits include:
SSL termination - Decrypt incoming requests and encrypt server responses so backend servers do not have to
perform these potentially expensive operations
Removes the need to install X.509 certificates on each server
Session persistence - Issue cookies and route a specific client's requests to same instance if the web apps do
not keep track of sessions
To protect against failures, it's common to set up multiple load balancers, either in active-passive or active-active mode.
Load balancers can route traffic based on various metrics, including:
Random
Least loaded
Session/cookies
Round robin or weighted round robin
Layer 4
Layer 7]
Layer 4 load balancing
Layer 4 load balancers look at info at the transport layer ([Link]
primer#communication) to decide how to distribute requests. Generally, this involves the source, destination IP
addresses, and ports in the header, but not the contents of the packet. Layer 4 load balancers forward network packets to
and from the upstream server, performing Network Address Translation (NAT)
([Link]
Layer 7 load balancing
([Link]
Layer 7 load balancers look at the application layer ([Link]
primer#communication) to decide how to distribute requests. This can involve contents of the header, message, and
cookies. Layer 7 load balancers terminate network traffic, reads the message, makes a load-balancing decision, then
opens a connection to the selected server. For example, a layer 7 load balancer can direct video traffic to servers that
host videos while directing more sensitive user billing traffic to security-hardened servers.
At the cost of flexibility, layer 4 load balancing requires less time and computing resources than Layer 7, although the
performance impact can be minimal on modern commodity hardware.
Horizontal scaling
([Link]
Load balancers can also help with horizontal scaling, improving performance and availability. Scaling out using
commodity machines is more cost efficient and results in higher availability than scaling up a single server on more
expensive hardware, called Vertical Scaling. It is also easier to hire for talent working on commodity hardware than it is
for specialized enterprise systems.
Disadvantage(s): horizontal scaling
([Link]
Scaling horizontally introduces complexity and involves cloning servers
Servers should be stateless: they should not contain any user-related data like sessions or profile pictures
Sessions can be stored in a centralized data store such as a database
([Link] (SQL, NoSQL) or a persistent cache
([Link] (Redis, Memcached)
Downstream servers such as caches and databases need to handle more simultaneous connections as upstream
servers scale out
Disadvantage(s): load balancer
([Link]
The load balancer can become a performance bottleneck if it does not have enough resources or if it is not
configured properly.
Introducing a load balancer to help eliminate a single point of failure results in increased complexity.
A single load balancer is a single point of failure, configuring multiple load balancers further increases complexity.
Reverse proxy (web server)
([Link]
primer/blob/master/images/[Link])
A reverse proxy is a web server that centralizes internal services and provides unified interfaces to the public. Requests
from clients are forwarded to a server that can fulfill it before the reverse proxy returns the server's response to the client.
Additional benefits include:
Increased security - Hide information about backend servers, blacklist IPs, limit number of connections per client
Increased scalability and flexibility - Clients only see the reverse proxy's IP, allowing you to scale servers or
change their configuration
SSL termination - Decrypt incoming requests and encrypt server responses so backend servers do not have to
perform these potentially expensive operations
Removes the need to install X.509 certificates on each server
Compression - Compress server responses
Caching - Return the response for cached requests
Static content - Serve static content directly
HTML/CSS/JS
Photos
Videos
Etc
|Feature|Load Balancer|Reverse Proxy| |---|---|---| |Primary Goal|Distribute traffic load across a pool of servers|Inspect,
route, and optimize incoming traffic| |Core Functions|Traffic distribution, health monitoring, auto-scaling support|SSL
termination, caching, request rewriting, rate limiting| |Server Target|Requires multiple backend servers to balance
across|Can front a single server or multiple servers| |OSI Layer|Operates at Layer 4 (TCP/UDP) or Layer 7
(HTTP/S)|Operates primarily at Layer 7 (Application)| |Focus Area|Availability, redundancy, and horizontal
scaling|Security, abstraction, and response speed| Disadvantages of a Reverse Proxy:
Single Point of Failure: If the reverse proxy crashes or becomes misconfigured, all incoming traffic to the
underlying backend servers is blocked—unless redundant proxy nodes and high-availability clustering are
implemented.
Increased Latency: Adding an intermediate node introduces an extra network hop for every incoming request
and outgoing response, slightly increasing latency (though caching often counteracts this).
Security & Traffic Bottleneck: Because all encrypted traffic terminates at the reverse proxy (SSL termination),
unencrypted internal traffic or a compromised proxy node exposes sensitive data, request headers, and routing
architecture to potential inspection.
Troubleshooting Difficulty: Investigating issues becomes harder because client IP addresses are masked
(requiring X-Forwarded-For header tracing), and logs must be correlated across both proxy layers and
backend services.
Application layer
([Link]
primer/blob/master/images/[Link]) The web layer acts as the front door that handles client connections and
delivers static files, while the application layer runs the underlying business logic, handles API endpoints, and interacts
with databases.
Web Layer (Traffic & Static Delivery)
Responsibilities: Serves static assets (HTML, CSS, JavaScript, images), manages incoming HTTP/HTTPS
connections, handles SSL termination, and routes traffic to backend services.
Characteristics: Extremely lightweight on CPU and memory because it mostly handles Network I/O and file
delivery rather than complex computation.
Typical Tech: NGINX, Apache, Cloudflare, or edge reverse proxies.
Application Layer / Platform Layer (Business Logic & APIs)
Responsibilities: Executes backend application code, validates user data, processes core business rules,
queries databases, and generates dynamic JSON/data responses.
Characteristics: High CPU and memory consumption due to data processing, algorithmic computations, and
database communication.
Typical Tech: [Link]/Express, Python/FastAPI, Go, Java Spring, or microservice containers.
Why Separate Them?
Imagine a web dashboard:
1. Initial Page Load: The client requests the page once. The web layer serves the static HTML and JS bundle
immediately and handles the request in milliseconds.
2. Dashboard Usage: Once loaded, the web page makes dozens of complex API calls (fetching database records,
calculating analytics, processing payments).
Because computation happens entirely in the application layer, those backend servers get overloaded while the web
layer remains barely stressed. By separating them, you can spin up 10 extra application servers to handle the heavy API
computation without wasting money scaling the web servers that only route requests or serve static files.
Disadvantage(s): application layer
Adding an application layer with loosely coupled services requires a different approach from an architectural,
operations, and process viewpoint (vs a monolithic system).
Microservices can add complexity in terms of deployments and operations.