ROUTING IN BACKEND

ENGINEERING
How Requests Find Their Way Home
Complete Class Notes to Crack High-Paying Backend Offers

Topics: IP Routing | DNS | Load Balancers | Reverse Proxies | API Gateways | Service Mesh |
Path Matching | Microservices Routing
1. What is Routing? — The Big Picture
Routing is the process of determining WHERE a request should go and HOW to get it there. It
happens at every layer of your stack — from DNS all the way to your application code.

1.1 Why Every Backend Engineer Must Understand Routing

• Every HTTP request travels through multiple routing layers before your code ever runs.
• Misunderstanding routing causes: wrong endpoints being hit, auth bypasses, infinite
redirect loops, poor performance.
• System design interviews constantly ask: 'How would you route traffic to 100
microservices?'
• Debugging production issues almost always involves tracing a request's routing path.

1.2 The Journey of a Request — Overview

From browser to your function, a request passes through roughly 7 routing layers:

Lay Routing Component What It Does

er

1 DNS Resolves domain name → IP address

2 Network Router Routes IP packets hop-by-hop toward destination
(ISP/Internet)
3 Load Balancer Distributes traffic across multiple servers
4 Reverse Proxy / CDN Edge Terminates TLS, caches, forwards to origin
5 API Gateway Auth, rate limiting, routing to microservices
6 Service Router / Service Routes between microservices internally
Mesh
7 Application Router Matches URL path → your handler function

Mental Model: Think of routing like mailing a letter: DNS = address lookup, Load Balancer = post
office sorter, API Gateway = building reception, App Router = office floor map — each layer
narrows down WHERE the request goes.

2. DNS Routing — Where It All Begins

DNS (Domain Name System) is the internet's phone book. It translates human-readable names into
machine-readable IP addresses.
2.1 DNS Resolution — Step by Step
• Browser checks its local DNS cache first (TTL-controlled).
• If not cached, OS resolver is queried (checks /etc/hosts on Linux/Mac too).
• Query goes to Recursive Resolver (usually your ISP or [Link] / [Link]).
• Recursive Resolver asks Root Name Server — 'Where is .com?'
• Root NS replies with TLD Name Server address for .com.
• Recursive Resolver asks TLD NS — 'Where is [Link]?'
• TLD NS replies with Authoritative Name Server for [Link].
• Authoritative NS returns the actual IP address (A record for IPv4, AAAA for IPv6).
• Recursive Resolver caches the answer (per TTL) and returns IP to client.

2.2 Key DNS Record Types

Reco Purpose Example
rd

A Domain → IPv4 address [Link] → [Link]

AAAA Domain → IPv6 address [Link] → 2606:2800::1
CNA Domain → Another domain [Link] → [Link]
ME (alias)
MX Mail server for a domain [Link] → [Link] (priority 10)
TXT Arbitrary text (SPF, DKIM, v=spf1 include:_spf.[Link] ~all
verification)
NS Which name servers are [Link] → [Link]
authoritative
SRV Service location (host + port) _https._tcp.[Link] 443 [Link]
PTR Reverse DNS (IP → domain) [Link].[Link] → [Link]

2.3 DNS-Based Load Balancing & Routing Tricks

• Round-Robin DNS: one domain returns multiple IPs in rotating order — primitive load
balancing.
• Geo-DNS / GeoDNS: returns different IPs based on the requester's geographic location.
○ User in India → gets Singapore datacenter IP
○ User in Germany → gets Frankfurt datacenter IP
• Weighted DNS: return IP-A 80% of the time, IP-B 20% — useful for blue/green
deployments.
• Health-Check DNS (Route 53 Failover): if primary IP goes down, DNS automatically
returns backup IP.
• TTL Strategy: Low TTL (60s) = fast failover, High TTL (86400s) = faster resolution
(cached longer).
 Interview Tip: Low TTL during deployments allows fast rollback. Raise TTL back after
deployment is stable. Always warn that DNS changes take TTL-seconds to propagate globally.

3. Network Layer Routing — IP & Packets

Before your server even sees the request, IP packets are routed across the internet through dozens
of hops. Understanding this is critical for debugging latency and system design.

3.1 How IP Routing Works

• The internet is a network of networks (Autonomous Systems — AS).
• Routers maintain routing tables: 'To reach 93.184.x.x, send packets to next-hop
[Link]'.
• Routing decisions are made hop-by-hop — each router only knows the NEXT hop, not
the full path.
• BGP (Border Gateway Protocol) is the protocol routers use to advertise and learn routes
between ASes.

3.2 IP Addresses & Subnets (Backend Context)

• IPv4: 32-bit address, written as 4 octets — e.g., [Link]
• IPv6: 128-bit address — e.g., 2001:0db8:85a3::8a2e:0370:7334
• CIDR notation: [Link]/8 means first 8 bits are fixed → 16M addresses
○ [Link]/8 → 16,777,216 addresses (large VPC)
○ [Link]/24 → 256 addresses (typical subnet)
○ [Link]/25 → 128 addresses (small subnet)
• Private IP ranges (RFC 1918) — not routable on public internet:
○ [Link]/8, [Link]/12, [Link]/16

3.3 NAT (Network Address Translation)

• NAT maps private IPs to a public IP so multiple servers can share one external IP.
• Cloud VPCs use NAT Gateways so private subnets can reach the internet without being
publicly exposed.
• Load balancers perform DNAT (Destination NAT) — change the destination IP of
incoming packets to a backend server's IP.

Cloud Context: In AWS/GCP/Azure: public subnet = has internet gateway route. Private subnet
= goes through NAT Gateway. Backend servers should ALWAYS be in private subnets.
4. Load Balancers — Distributing the Load
A load balancer is the entry point to your backend fleet. It distributes incoming requests across
multiple servers for availability, scalability, and reliability.

4.1 Why Load Balancers Are Essential

• Single server = single point of failure. Load balancer routes around failures
automatically.
• Horizontal scaling: add more servers behind the LB without changing client
configuration.
• Health checks: LB removes unhealthy servers from rotation automatically.
• TLS Termination: LB handles encryption so backend servers deal with plain HTTP
(simpler + faster).

4.2 Types of Load Balancers

Type OSI Layer Routing Based On Examples

L4 (Transport) Layer 4 IP + Port only (TCP/UDP) AWS NLB, HAProxy (TCP mode)
L7 Layer 7 HTTP headers, URL path, AWS ALB, NGINX, Traefik
(Application) cookies
Global Layer 3/7 Geolocation, latency Cloudflare, AWS Global
(Anycast) Accelerator
DNS Load DNS Round-robin / weighted DNS AWS Route 53, Cloudflare
Balancer

L4 Load Balancers
• Operate at TCP/UDP level. They do NOT inspect HTTP content.
• Extremely fast and low-latency — just forward packets.
• Cannot make routing decisions based on URL path or HTTP headers.
• Used for: raw TCP connections, databases, gaming, low-latency services.

L7 Load Balancers
• Inspect full HTTP request — headers, URL path, body.
• Can route /api/* to backend cluster A and /static/* to CDN or cluster B.
• Support sticky sessions (session affinity) — route same user to same server via cookie.
• Can perform SSL termination, request/response modification, auth validation.
4.3 Load Balancing Algorithms
Algorithm How It Works Best For

Round Robin Requests go to servers Equal servers, similar request weights

1,2,3,1,2,3... in order
Weighted Round Server A gets 70%, Server B Different server capacities
Robin gets 30%
Least Connections Route to server with fewest Long-lived connections (WebSockets)
active connections
Least Response Time Route to fastest-responding Heterogeneous server performance
server
IP Hash Hash client IP → same server Stateful apps needing affinity
every time
Random Pick a server at random Simple, surprisingly effective
Resource-Based Route based on server Variable workloads
CPU/memory

4.4 Health Checks

• LBs continuously probe backend servers to detect failures.
• TCP health check: just checks if port is open (Layer 4).
• HTTP health check: hits GET /health and expects 200 OK (Layer 7).
• Custom health: checks if DB connection is alive, cache is warm, etc.
• Thresholds: 'Remove server after 3 consecutive failures. Re-add after 2 consecutive
successes.'

Best Practice: Always implement a /health endpoint that checks ALL dependencies (DB, cache,
external APIs). Return 200 if healthy, 503 if degraded. This is the LB's signal to stop routing
traffic.

4.5 Session Affinity (Sticky Sessions)

• Problem: If a user's session is stored in server memory, routing them to a different
server loses their session.
• Solution: LB sets a cookie (e.g., AWSALB=...) and always routes that user to the same
server.
• Better solution: Store sessions in Redis/DB — then any server can handle any user.
Stateless backends scale better.

Interview Tip: Sticky sessions are a smell — they prevent true horizontal scaling. The preferred
pattern is stateless backends with shared session stores (Redis).
5. Reverse Proxies — The Traffic Controllers
A reverse proxy sits in front of your servers, intercepting all incoming requests. It's different from a
forward proxy (which acts on behalf of clients).

5.1 Forward Proxy vs Reverse Proxy

Aspect Forward Proxy Reverse Proxy

Sits between Client and internet Internet and backend servers

Acts on behalf of Client (hides client) Server (hides server)
Client knows Yes (configured in browser) No (transparent to client)
about it?
Common use Bypass restrictions, anonymity, Load balancing, caching, TLS
corporate filtering termination, security
Examples Squid, corporate VPN proxies NGINX, HAProxy, Cloudflare, AWS ALB

5.2 What a Reverse Proxy Does

• TLS Termination: Handles HTTPS so backend servers only deal with HTTP internally.
• Request Buffering: Accepts full request before forwarding — protects slow backends
from slow clients.
• Response Caching: Cache static assets/API responses at the proxy layer.
• Compression: Gzip/Brotli compress responses before sending to client.
• Request Routing: Route by path, header, host to different backends.
• Security: Hide backend server IPs, filter malicious requests, add rate limiting.
• Logging & Observability: Central place to log all requests with timing.

5.3 NGINX as a Reverse Proxy — Key Config

Basic NGINX reverse proxy config:
server {
listen 443 ssl;
server_name [Link];

# Route /api/v1 to backend cluster

location /api/v1/ {
proxy_pass [Link]
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
 # Route /static to CDN / file server
location /static/ {
proxy_pass [Link]
expires 1y;
add_header Cache-Control public;
}
}

Critical Header: Always pass X-Forwarded-For header so your backend knows the REAL client
IP. Without it, your app sees the load balancer's IP instead.

6. API Gateways — The Smart Front Door

An API Gateway is a specialized reverse proxy for API traffic. It's the single entry point for all clients
and handles cross-cutting concerns so your services don't have to.

6.1 What an API Gateway Does

Concern How API Gateway Handles It

Authentication Validates JWT/API keys before request reaches any service

Authorization Checks if caller has permission for the specific route
Rate Limiting Throttle per user/IP/API key — returns 429 Too Many Requests
Request Routing Routes /users → user-service, /orders → order-service
Request Transformation Adds headers, rewrites URLs, converts XML↔JSON
Response Aggregation Calls multiple services, merges responses (Backend-for-Frontend
pattern)
Circuit Breaking Stops routing to a failing service, returns fallback
Logging & Tracing Injects trace IDs, logs all request metadata centrally
SSL Termination Single TLS endpoint for all services
Caching Cache responses for identical requests

6.2 API Gateway Routing Patterns

Path-Based Routing
GET /api/users/* → user-service:8001
GET /api/orders/* → order-service:8002
GET /api/products/* → product-service:8003
POST /api/payments/* → payment-service:8004
Header-Based Routing
X-API-Version: v2 → route to v2 backend
X-Client: mobile → route to mobile-optimized backend
Accept-Language: de → route to German region service

Method-Based Routing
GET /api/users → read-replica cluster (no DB write locks)
POST /api/users → primary write cluster (full DB access)

6.3 Popular API Gateways

Gateway Type Best For

Kong Open-source / Complex plugin-based routing, on-prem

Enterprise
AWS API Gateway Managed Cloud Serverless (Lambda) + AWS ecosystem
Nginx + Lua Self-managed High-perf, custom logic via scripts
Traefik Open-source Kubernetes-native, auto-discovery
Envoy Open-source proxy Service mesh data plane (Istio uses it)
Apigee (Google) Enterprise managed API monetization, enterprise scale
Azure API Managed Cloud Azure ecosystem, enterprise
Management

Interview Tip: When asked 'how do you secure microservices?' — answer: API Gateway
handles auth/rate-limiting at the edge. Internal service-to-service calls use mutual TLS (mTLS) or
service mesh policies.

7. Application-Level Routing — Path Matching

Once a request reaches your application server, the framework's router matches the URL path to a
handler function. This is the routing you write every day.

7.1 How Application Routers Work

• The router maintains a table of patterns → handler functions.
• When a request comes in, the router iterates patterns in priority order.
• First match wins (in most frameworks). Specific routes before wildcard routes.
• Extracted path parameters are injected into the handler context.
7.2 Route Matching Types
Type Pattern Matches Does Not Match

Exact match GET /users /users /users/123 or /users/

Path GET /users/:id /users/123, /users/abc /users/123/orders
parameter
Wildcard GET /files/*path /files/a/b/[Link] /users/anything
segment
Optional GET /users/:id? /users, /users/123 /users/123/orders
param
Regex route GET /items/[0-9]+ /items/42 /items/abc
Query params GET /search?q=... /search?q=hello (in N/A — not in route def
handler, not route)

7.3 Route Priority & Ordering

Routes are matched in this priority order in most frameworks:
• Static routes first: GET /users/me (matches before :id)
• Parameterized routes: GET /users/:id
• Wildcard routes last: GET /users/*

Example: [Link] routing order matters!

// CORRECT order — specific before generic
[Link]('/users/me', getMeHandler); // matches /users/me
[Link]('/users/:id', getUserHandler); // matches /users/123
[Link]('/users/*', catchAllHandler); // matches anything else

// WRONG order — :id matches 'me' first!

[Link]('/users/:id', getUserHandler); // matches /users/me TOO
[Link]('/users/me', getMeHandler); // NEVER reached

Common Bug: '/users/me' being treated as '/users/:id' where id='me' is a classic ordering
mistake. Always register specific routes before parameterized ones.

7.4 Route Groups & Middleware Chains

Modern frameworks let you group routes and apply middleware at the group level:
// [Link] example
const apiRouter = [Link]();
 // Apply auth middleware to all /api/v1 routes
[Link]('/api/v1', authMiddleware, apiRouter);

// These routes are protected by authMiddleware

[Link]('/users', getUsers);
[Link]('/users', createUser);
[Link]('/orders', getOrders);

// Go (Gin) example — route groups

v1 := [Link]('/api/v1')
[Link](AuthMiddleware())
{
[Link]('/users', GetUsers)
[Link]('/users', CreateUser)
}

7.5 Middleware in the Routing Pipeline

Middleware functions execute in sequence BEFORE your handler. They can modify the request,
short-circuit the response, or pass to next.

Middleware What It Does Short-circuits on

Authentication Validates token, attaches user to Invalid/missing token → 401

context
Authorization Checks user has permission for route Insufficient perms → 403
Rate Limiter Checks request count per user/IP Limit exceeded → 429
Request Validator Validates body/params against Invalid data → 400/422
schema
Logger Logs method, path, timing Never (always passes through)
CORS Adds CORS headers, handles OPTIONS preflight → 204
OPTIONS preflight
Body Parser Parses JSON/form body into Malformed body → 400
structured object
Error Handler Catches errors from all handlers Returns 4xx/5xx to client

Best Practice: Order middleware carefully: Logger → CORS → Rate Limiter → Body Parser →
Auth → Authorization → Your Handler → Error Handler. Auth before authorization, always.

8. Microservices Routing — Service Discovery

In a microservices architecture, services need to find and communicate with each other dynamically.
Service discovery solves the problem of 'how does service A know the address of service B?'
8.1 The Problem with Hardcoded Addresses
• Microservices run as containers/pods — their IPs change when they restart.
• Services scale up and down — number of instances varies.
• Hardcoding IPs means updating config every time a service moves. Impossible at scale.

8.2 Client-Side Discovery

• Service instances register themselves with a Service Registry (e.g., Consul, Eureka) on
startup.
• When Service A needs to call Service B, A queries the registry for B's current addresses.
• Service A then picks one instance (using its own load balancing logic) and calls it
directly.

// Pseudo-code: client-side discovery

const instances = [Link]('user-service');
const target = loadBalance(instances); // round-robin / random
const response = await
[Link](`[Link]

8.3 Server-Side Discovery

• Service A sends request to a load balancer or API gateway.
• The load balancer queries the registry and routes to an available instance.
• Service A knows NOTHING about the registry or individual instances.
• Simpler for services, but adds a network hop.

8.4 Service Registry Options

Tool Type Key Features

Consul (HashiCorp) Distributed registry Health checks, KV store, DNS interface, multi-
datacenter
Eureka (Netflix) Java-based registry Built for Spring Boot / Netflix OSS stack
etcd Distributed KV Used by Kubernetes internally for service state
store
Kubernetes DNS Built-in K8s Automatic DNS for Services: user-
[Link]
AWS Cloud Map Managed cloud Integrates with Route 53, ECS, EKS
ZooKeeper (Apache) Distributed coord. Older, complex. Used in Kafka, HBase
8.5 Kubernetes Service Routing
Kubernetes has built-in service routing that every backend engineer should understand.
• Service: A stable virtual IP (ClusterIP) in front of a set of Pods. Even if pods restart and
get new IPs, the Service IP stays the same.
• kube-proxy: Runs on every node, maintains iptables/IPVS rules to route Service IPs to
Pod IPs.
• CoreDNS: Kubernetes DNS server. Assigns DNS names: [Link]-
[Link]

Kubernetes Service Types:

Service Type What It Does Use Case

ClusterIP (default) Internal IP only — no external Service-to-service within cluster

access
NodePort Opens a port on every cluster Dev/testing external access
node
LoadBalancer Provisions cloud LB (AWS ELB, Production external access
GCP LB)
ExternalName DNS alias to external service Route to external DB/API by name
Headless No ClusterIP — returns Pod IPs StatefulSets, custom discovery
directly

Interview Tip: In Kubernetes, Ingress is different from Service. Ingress is an L7 routing rule (host
+ path → Service). An Ingress Controller (like NGINX or Traefik) is the actual load balancer
implementing those rules.

9. Service Mesh — Advanced Microservices Routing

A service mesh adds a dedicated infrastructure layer for service-to-service communication —
handling routing, retries, circuit breaking, mTLS, and observability automatically.

9.1 The Problem Service Meshes Solve

• Dozens of microservices each implementing their own: retry logic, timeouts, circuit
breaking, mTLS, tracing.
• This logic is duplicated across every service in every language.
• Service mesh moves all this logic OUT of application code into a sidecar proxy.

9.2 Sidecar Proxy Pattern

• Every service Pod gets a sidecar container (Envoy proxy) injected automatically.
 • ALL inbound and outbound traffic goes through the sidecar — the service never talks
directly to the network.
• The sidecar handles: mTLS, retries, circuit breaking, load balancing, tracing.
• The control plane (Istiod) configures all sidecars centrally.

Without service mesh:

ServiceA ──────────────────────────► ServiceB

With service mesh (Istio):

ServiceA → [Envoy sidecar] ──mTLS──► [Envoy sidecar] → ServiceB
| |
Telemetry/Logs Telemetry/Logs
└──────────────────────────┘
Istiod Control Plane

9.3 Service Mesh Routing Capabilities

• Traffic Splitting: Send 90% to v1, 10% to v2 — for canary deployments.
• Fault Injection: Inject artificial delays or errors to test resilience.
• Circuit Breaking: Stop routing to a service after X consecutive failures.
• Retries: Automatically retry failed requests (with jitter to avoid thundering herd).
• mTLS: Mutual TLS authentication between every service pair automatically.
• Distributed Tracing: Inject trace IDs for full request flow visibility (Jaeger/Zipkin).

Feature Istio Linkerd Consul Connect

Sidecar Proxy Envoy Linkerd2-proxy Envoy

(Rust)
Performance overhead Medium Very Low Low
Complexity High Medium Medium
mTLS Yes Yes (automatic) Yes
Traffic splitting Yes Yes (TrafficSplit) Yes
(VirtualService)
Best for Feature-rich Simplicity + Multi-DC / non-K8s
enterprise performance

10. CDN Routing — Edge Networking

A CDN (Content Delivery Network) routes requests to the closest server on earth. It's not just for
static files — modern CDNs route all API traffic.
10.1 How CDN Routing Works
• CDN has Points of Presence (PoPs) — servers in 50-300+ cities worldwide.
• User's DNS query is answered with the IP of the nearest PoP (via Anycast or GeoDNS).
• Request hits the PoP first. If content is cached → served instantly.
• If not cached (cache miss) → PoP fetches from origin server, caches, serves to user.

10.2 Anycast Routing

• Multiple CDN PoPs advertise the SAME IP address via BGP.
• Internet routing automatically sends packets to the topologically closest PoP.
• Used by Cloudflare, Google, and all major CDNs.
• Benefit: DDoS mitigation — attack traffic is absorbed by many PoPs, not concentrated at
origin.

10.3 CDN Cache Routing Logic

Cache What Happens Headers to Use
Status

HIT Served from CDN cache. Cache-Control: public, max-age=86400

Origin not called.
MISS CDN fetches from origin, Cache-Control: public, s-maxage=3600
caches, serves.
BYPASS CDN skips cache, always goes Cache-Control: private, no-store
to origin.
STALE Cached but expired. CDN Cache-Control: stale-while-revalidate=60
revalidates in background.
EXPIRED Cache expired. CDN fetches ETag / Last-Modified for 304 optimization
fresh copy.

10.4 Edge Computing (Routing Logic at CDN Edge)

• Modern CDNs (Cloudflare Workers, AWS Lambda@Edge) let you run code at PoPs.
• Use cases: A/B testing routing, auth token validation, geo-based redirects, bot blocking.
• Happens BEFORE request reaches origin — dramatically reduces latency.

System Design: For global apps, use CDN edge routing to: (1) serve static assets from cache,
(2) route API requests to nearest datacenter, (3) handle auth and rate limiting at edge — origin
servers only handle unique requests.
11. Advanced Routing Patterns
These patterns appear in senior-level interviews and system design discussions at top companies.

11.1 Blue-Green Deployment Routing

• Run two identical production environments: Blue (current) and Green (new version).
• Router sends 100% traffic to Blue. Deploy new version to Green. Test Green.
• Switch router to send 100% traffic to Green. Blue becomes the standby.
• Rollback = just switch router back to Blue. Zero-downtime deployments.

11.2 Canary Deployment Routing

• Gradually shift traffic to new version: 1% → 5% → 25% → 100%.
• Monitor error rates, latency, business metrics at each stage.
• If metrics degrade, automatically roll back. If good, increase traffic.
• Implementation: Weighted routing at LB/API Gateway/Service Mesh.

# Kubernetes Argo Rollouts canary example

steps:
- setWeight: 5 # 5% to new version
- pause: {duration: 10m}
- setWeight: 25
- pause: {duration: 30m}
- setWeight: 100 # full rollout

11.3 Circuit Breaker Pattern

• When a downstream service fails repeatedly, 'open the circuit' — stop routing to it
immediately.
• Three states: Closed (normal) → Open (all requests fail fast) → Half-Open (let a few
through to test).
• Prevents cascading failures: one bad service shouldn't take down your whole system.

State Behavior Transition

Closed All requests route to service normally → Open after N failures in window
Open All requests fail immediately (no → Half-Open after timeout (e.g., 30s)
network call). Return fallback.
Half-Open Let N test requests through to probe → Closed if success, → Open if fail
service health.
11.4 Retry with Exponential Backoff
• On transient failures (503, 504, timeout), retry automatically.
• Exponential backoff: wait 1s, then 2s, then 4s, then 8s... with jitter (random offset).
• Jitter prevents 'thundering herd': if 1000 clients all retry at exactly the same time, the
server gets hammered.
• Always limit retries for non-idempotent operations (POST) — retrying creates duplicates.

// Retry with exponential backoff + jitter

for attempt in 0..maxRetries:
try:
response = await callService(request)
return response
except TransientError:
wait = min(baseDelay * 2^attempt + random(0, jitter), maxDelay)
sleep(wait)

11.5 Rate Limiting Algorithms

Algorithm How It Works Pros / Cons

Token Bucket Bucket holds N tokens. Each Allows bursts up to bucket size. Smooth
request uses 1. Tokens replenish overall rate.
at fixed rate.
Leaky Bucket Requests enter queue, Very smooth output. Doesn't handle
processed at fixed rate. Overflow bursts well.
= dropped.
Fixed Window Count requests in a fixed time Simple. Boundary burst problem (200 req
window (e.g., 100/minute). at window edges).
Sliding Window Log Track timestamp of every Most accurate. High memory use.
request. Count in last N seconds.
Sliding Window Hybrid: two fixed windows Accurate + memory efficient. Used by
Counter weighted by time elapsed. Redis rate limiting.

12. Debugging Routing Issues — Production Skills

Knowing how to trace a request through routing layers is what separates senior engineers. Here are
the essential debugging tools and techniques.

12.1 Tracing a Request's Path

• Start from DNS: dig [Link] — shows what IP the domain resolves to.
• Trace network hops: traceroute [Link] — shows each router hop and latency.
 • Check TLS: curl -vI [Link] — shows certificate chain and TLS
negotiation.
• Full request debug: curl -v [Link] — shows all headers in and
out.

# Useful debugging commands

dig [Link] # DNS lookup
dig +trace [Link] # Full DNS resolution trace
traceroute [Link] # Network hop trace
curl -v [Link] # Full HTTP debug output
curl -H 'X-Debug: true' [Link] # Send custom debug header
openssl s_client -connect [Link] # TLS debug

12.2 Common Routing Bugs and Fixes

Symptom Likely Cause Fix

404 on correct URL Route registered in wrong Check route registration order, print route
order or prefix mismatch table
Request hits wrong Specific route registered after Register specific routes BEFORE
handler wildcard parameterized/wildcard
App sees LB IP not X-Forwarded-For header not Add proxy_set_header X-Forwarded-For
client IP passed in NGINX/LB config
CORS errors in browser Missing Access-Control-Allow- Add CORS middleware, ensure
Origin or wrong origin OPTIONS preflight handled
Auth works direct, fails Authorization header stripped Check proxy config — some strip auth
via LB by proxy headers by default
Infinite redirect loop HTTPS redirect on LB + app Detect X-Forwarded-Proto: https in app,
also redirects skip redirect
Service unreachable in Service selector doesn't match kubectl get endpoints my-service —
K8s pod labels check if endpoints populated
Slow first request Cold start / TCP connection Enable keep-alive, connection pooling,
not reused warm-up requests

12.3 Distributed Tracing

• In microservices, a single request may touch 10+ services. Distributed tracing tracks the
full path.
• Trace ID is injected at the entry point (API Gateway) and propagated in every
downstream call.
• Each service records a Span (start time, end time, service name, operation).
• Tools: Jaeger, Zipkin, AWS X-Ray, Datadog APM.
• Standard headers: traceparent (W3C standard), X-B3-TraceId (Zipkin), X-Amzn-Trace-Id
(AWS).
 Best Practice: Always propagate trace headers between services. Inject them in your HTTP
client middleware. This is the single most valuable debugging investment in a microservices
system.

13. Interview Questions & Model Answers

Q1: Walk me through how a request reaches your backend in a production

system.
Answer: DNS resolves domain to LB IP → TLS terminates at load balancer → LB health-checks
and distributes to API Gateway → Gateway validates auth, rate-limits, routes to correct
microservice → Service mesh handles service-to-service routing with mTLS → Application router
matches path to handler → Response follows reverse path.

Q2: What is the difference between a load balancer and an API gateway?
Answer: A load balancer distributes traffic across identical servers (L4 or L7). An API gateway is
a specialized L7 proxy that handles cross-cutting API concerns: auth, rate limiting, routing to
different services, request transformation. They often work together: LB in front of multiple API
gateway instances.

Q3: How would you design zero-downtime deployments?

Answer: Use either blue-green (switch 100% traffic at once, instant rollback) or canary (gradual
traffic shift, monitor metrics). Requires: health check endpoints, LB/service mesh traffic splitting
capability, and feature flags for risky changes. Always keep old version running until new version
is proven stable.

Q4: How does Kubernetes route external traffic to a Pod?

Answer: External traffic → Cloud Load Balancer → NodePort on worker nodes → kube-proxy
iptables/IPVS rules route to Service ClusterIP → Service selects healthy Pod endpoints by label
selector → traffic reaches Pod. Ingress Controller adds L7 host/path routing on top of this flow.

Q5: What is a circuit breaker and when would you use it?
Answer: A circuit breaker monitors calls to a service. After N failures in a window, it 'opens' —
failing fast without making network calls (preventing cascade failures). After a timeout, it 'half-
opens' to test if the service recovered. Use it whenever calling any external service, DB, or
downstream microservice that might fail.
Q6: How do you prevent routing a request to a slow/failing server?
Answer: Four layers: (1) LB health checks remove unhealthy instances, (2) Circuit breaker stops
calling failing services fast, (3) Timeouts prevent requests from hanging indefinitely, (4) Retries
with backoff handle transient failures. Also use least-connections load balancing so slow servers
get fewer requests.

Q7: How would you implement geo-based routing?

Answer: Option 1: GeoDNS — return region-specific IPs based on requester's location
(Cloudflare, Route 53). Option 2: Anycast — advertise same IP from all regions, BGP routes to
nearest. Option 3: CDN Edge — route at CDN PoP based on CF-IPCountry header. For
compliance (GDPR), ensure EU users' data stays in EU region.

14. Quick Reference Cheat Sheet

Topic Key Points to Remember

DNS Flow Browser cache → OS → Recursive Resolver → Root NS → TLD NS →

Authoritative NS
DNS Records A=IPv4, AAAA=IPv6, CNAME=alias, MX=mail, TXT=verification,
SRV=service+port
L4 vs L7 LB L4=TCP/IP only (fast). L7=HTTP-aware (path/header routing, TLS
termination)
LB Algorithms Round Robin, Least Connections, IP Hash, Weighted, Least Response
Time
Health Check GET /health → 200 = healthy, 503 = unhealthy. LB removes failed
instances.
Sticky Sessions Cookie-based affinity. Anti-pattern — prefer stateless + Redis for
sessions.
Reverse Proxy Hides backend IPs, TLS termination, caching, compression, request
buffering
X-Forwarded-For Always pass this header through proxies so app sees real client IP
API Gateway Auth + Rate limit + Routing + Transformation + Circuit Breaking at
edge
Service Discovery Client-side (query registry yourself) vs Server-side (LB does it for you)
K8s Service Types ClusterIP=internal, NodePort=dev, LoadBalancer=production,
Ingress=L7 routing
Service Mesh Sidecar proxies (Envoy) for mTLS, retries, tracing. Istio / Linkerd.
Circuit Breaker Closed→Open (fail fast)→Half-Open (probe). Prevents cascading
failures.
Topic Key Points to Remember

Canary Deploy 1%→5%→25%→100% traffic shift with metric monitoring and auto-
rollback
Rate Limiting Token Bucket (bursts OK), Sliding Window (accurate). Redis for
distributed.
Distributed Tracing Inject trace ID at gateway, propagate in every service call.
Jaeger/Zipkin.
CDN/Anycast Same IP advertised globally, BGP routes to nearest PoP. Cache-
Control for CDN.
Debug Tools dig (DNS), traceroute (hops), curl -v (HTTP), openssl s_client (TLS)

Every request has a home. Now you know how it gets there — and how to route it
better than anyone in the room.