Microservices Complete Course

From Monolith to Production: Architecture, Patterns, Kafka, Resilience, Security, Kubernetes & Interview
Preparation

For a Java / Spring Boot Developer preparing for Senior Microservices interviews

PART 1 — FOUNDATIONS
1. What is a Monolith?
2. Why Microservices?
3. What are Microservices?
4. Microservices vs Monolith — Comparison
PART 2 — MICROSERVICE ARCHITECTURE
5. How to Identify Microservices — Bounded Context & DDD
6. Bounded Context
7. Database per Service
PART 3 — COMMUNICATION BETWEEN MICROSERVICES
8. Synchronous Communication (REST)
9. Asynchronous Communication (Events)
10. REST vs Kafka — When to Use Which
PART 4 — KAFKA IN MICROSERVICES
PART 5 — API GATEWAY
PART 6 — SERVICE DISCOVERY
PART 7 — LOAD BALANCING
PART 8 — FAULT TOLERANCE
PART 9 — DISTRIBUTED TRANSACTIONS & SAGA
PART 10 — OUTBOX PATTERN
PART 11 — DISTRIBUTED DATA CONSISTENCY & CAP
PART 12 — SECURITY
PART 13 — OBSERVABILITY
PART 14 — CONFIGURATION MANAGEMENT
PART 15 — DEPLOYMENT: DOCKER & KUBERNETES
PART 16 — CI/CD
PART 17 — TESTING MICROSERVICES
PART 18 — ADVANCED PATTERNS
PART 19 — COMPLETE REAL-WORLD PROJECT: E-COMMERCE MICROSERVICES SYSTEM
PART 20 — COMPLETE REQUEST FLOW SCENARIOS
PART 21 — PRODUCTION ARCHITECTURE
PART 22 — INTERVIEW PREPARATION
Beginner
Intermediate
Advanced
System Design Prompts (practice these end-to-end)
PART 1 — FOUNDATIONS

1. What is a Monolith?
Analogy. A single restaurant kitchen where one team of chefs shares one stove, one fridge, one prep
counter, cooking everything — starters, mains, desserts, drinks. If the stove breaks, the whole kitchen
stops. If only drinks orders spike, you can’t add stoves just for drinks — you’d have to duplicate the whole
kitchen.

Definition. A monolith is an application built and deployed as one unit: one codebase, one running
process (one JVM), one shared database. Modules talk to each other via plain in-memory Java method calls.

Monolithic Application (one Spring Boot JAR)

┌─────────────────────────────────────┐
│ Order Module ──▶ Payment Module │
│ │ │ │
│ ▼ ▼ │
│ Inventory Module Notification Module│
└───────────────┬───────────────────────┘
▼
Single Shared Database

Spring Boot example:

@Service
public class OrderService {
private final PaymentService paymentService; // in-memory call
private final InventoryService inventoryService; // in-memory call

@Transactional
public Order placeOrder(OrderRequest request) {
[Link]([Link](), [Link]());
[Link]([Link](), [Link]());
return [Link](new Order(request));
}
}

@Transactional gives free ACID guarantees across “modules” because they share one DataSource . This is
trivial in a monolith and becomes one of the hardest problems in microservices (→ motivates the Saga
pattern, Part 9).

Advantages: simple to build/test/deploy/debug, trivial transactions, no network latency.

Disadvantages: codebase complexity at scale, team merge conflicts, any change forces a full redeploy,
can’t scale one hot module independently, tech lock-in, poor failure isolation (one module’s crash kills the
whole JVM), database coupling, slow release cycles.

2. Why Microservices?
Monolith problem What happens
Codebase complexity Thousands of engineers can’t safely share one repo
Teams block each other; “release trains” slow everyone
Team dependencies
down
Deployment dependency A 1-line fix requires redeploying and re-testing everything
Scaling problems Must scale the whole app even if only Payment is hot
Technology limits Whole app stuck on one stack/version
Failure isolation One bad module crashes the entire process

Database coupling One shared schema breaks under many teams’ changes
 Businesses want to ship many times a day, not once a
Release cycles
quarter

MONOLITH MICROSERVICES
┌─────────────────┐ ┌────────┐ ┌────────┐ ┌───────────┐
│ Order/Payment/ │ ──▶ │ Order │ │Payment │ │ Inventory │
│ Inventory │ │Service │ │Service │ │ Service │
│ (1 process, 1 DB) │ └───┬────┘ └───┬────┘ └─────┬─────┘
└─────────────────┘ ▼ ▼ ▼
Order DB Payment DB Inventory DB

Each service now deploys, scales, and fails independently — that independence is the point of everything
below.

3. What are Microservices?

Definition: an architectural style where an application is built as a collection of small, autonomous
services, each modeling a specific business capability, each independently deployable, each owning
its own data.

Characteristics: loose coupling (services interact only through well-defined APIs/events, not shared
internals), high cohesion (each service owns one business capability end-to-end), independent scaling,
independent deployment, fault isolation, decentralized data ownership, technology independence (each
team can pick its own stack).

What microservices are NOT: not simply “small services” chopped up arbitrarily by technical layer (e.g.,
a “validation service,” a “DB access service”) — that reintroduces tight coupling through chatty calls. Not a
free way to get better code quality — bad boundaries in microservices are worse than bad boundaries in a
monolith, because now they cross the network. Not something every project needs — see interview Q4
below.

4. Microservices vs Monolith — Comparison

Aspect Monolith Microservices
Development Single codebase, simple locally Multiple repos/services, more setup
Deployment One unit, all-or-nothing Independent, per-service
Scaling Whole app scales together Per-service, matched to actual load
Database One shared DB Database per service
Testing Easy in-process integration tests Needs contract/component/E2E testing
Distributed tracing/log aggregation
Monitoring One process to watch
needed
Failure handling One failure can cascade to all Isolated, but partial failures are common
Autonomous teams per service
Team structure Centralized, coordination heavy
(Conway’s Law)
DevOps Simple pipeline CI/CD, containers, orchestration required
Lower infra, higher coordination cost at Higher infra/ops cost, lower coordination
Cost scale cost

High operationally, lower in-code per

Complexity Low operationally, high in-code at scale
service
PART 2 — MICROSERVICE ARCHITECTURE

5. How to Identify Microservices — Bounded Context & DDD

Use Domain-Driven Design: find business capabilities (Customer, Order, Payment, Inventory,
Shipping, Notification) and draw boundaries around things that change together and are owned by one
team (high cohesion), while minimizing cross-service chatter (low coupling).

E-commerce domain → Customer | Order | Payment | Inventory | Shipping | Notification

Ask: does this capability have its own lifecycle, its own data, its own reason to change? If yes, it’s a strong
candidate for its own service. Don’t split by technical layer (e.g., a separate “validation service”) — that
creates chatty, tightly-coupled services with none of the benefits.

6. Bounded Context
Bounded Context (from DDD) says the same real-world entity can mean different things in different parts
of the system, and each service should model only the fields it actually needs — never a giant shared
“Customer” object.

Customer Service: Customer { id, name, email, address, preferences }

Order Service: Customer { id, name, shippingAddress } ← minimal view
Payment Service: Customer { id, billingAddress, paymentMethodId } ← minimal view

A shared “God” Customer model recreates monolith-style coupling: any field change forces every service to
redeploy. Each service keeps its own local, purpose-built representation and references others only by ID.

7. Database per Service

Each service owns its schema exclusively; no other service may query it directly.

Order Service → Order DB Payment Service → Payment DB Inventory Service → Inventory DB

Problem: what if Order Service needs Payment info? It cannot do a SQL join across databases. Options: -
Synchronous call — Order Service calls Payment Service’s API (Part 3) - Event-driven replication —
Payment Service publishes PaymentCompleted events; Order Service keeps a local read-optimized copy of just
the fields it needs (this is the seed of CQRS, Part 18)

This trade-off — no more free SQL joins, no more free cross-table @Transactional — is the central cost of
microservices, and it’s why Parts 8–10 (resilience, Saga, Outbox) exist.
PART 3 — COMMUNICATION BETWEEN
MICROSERVICES

8. Synchronous Communication (REST)

Order Service --HTTP request/response--> Payment Service

Request lifecycle: Order Service serializes a request → sends HTTP → Payment Service deserializes,
processes, responds → Order Service deserializes response and continues. The caller blocks/waits.

Options in Spring: OpenFeign (declarative HTTP clients), WebClient (reactive, non-blocking, the modern
default), and RestTemplate (blocking, now in maintenance mode — avoid for new code).

@FeignClient(name = "payment-service", url = "${[Link]}")

public interface PaymentClient {
@PostMapping("/payments")
PaymentResponse charge(@RequestBody PaymentRequest request);
}

9. Asynchronous Communication (Events)

Problem: synchronous calls couple services at runtime — if Payment Service is down, Order Service’s
request fails immediately. For workflows that don’t need an instant answer (e.g., “notify the customer”),
this tight coupling is unnecessary and fragile.

Solution — events via a broker (Kafka): a producer publishes an event; any number of consumers
react independently, without the producer knowing or caring who’s listening.

Order Service --publishes--> [OrderCreated] --> Kafka --> Payment Service

\--> Inventory Service

Internally: Order Service writes the event to a Kafka topic and moves on immediately (no waiting). Kafka
durably stores it. Payment Service and Inventory Service each consume it independently, at their own
pace, even if one of them is temporarily down (it catches up later).

10. REST vs Kafka — When to Use Which

Use REST when Use Kafka when
You need an immediate response (e.g., “is this in stock right
The caller doesn’t need to wait for the result
now?”)
Strong request/response semantics You want to decouple producer from consumer(s)
Simple point-to-point calls Multiple services need to react to the same event
Low fan-out Need durability/replay, buffering during downstream outages

Real production pattern: Order placement uses REST for the inventory check (customer waits for
immediate confirmation) but publishes an event for downstream notification/analytics (no one
needs to wait for those).
PART 4 — KAFKA IN MICROSERVICES
Core concepts: a Broker is a Kafka server; a Topic is a named event stream, split into Partitions for
parallelism; a Producer writes messages; a Consumer reads them; a Consumer Group lets multiple
consumer instances share the work of one topic (each partition is read by exactly one consumer within a
group); an Offset tracks each consumer’s read position per partition; Replication copies each partition
across brokers for durability, with one Leader handling reads/writes and Followers replicating; the Group
Coordinator manages group membership and triggers Rebalancing when consumers join/leave.

Delivery semantics: - At-most-once — message may be lost, never redelivered (commit offset before
processing) - At-least-once — message never lost, may be redelivered (commit offset after processing) —
the common default - Exactly-once — requires idempotent producers + transactional writes; Kafka
supports this via its transactional API, but most teams achieve the effect of exactly-once by combining at-
least-once delivery with idempotent consumers

@KafkaListener(topics = "order-events", groupId = "payment-service")

public void handleOrderCreated(OrderCreatedEvent event) {
if ([Link]([Link]())) {
return; // idempotency check — duplicate message, already processed
}
[Link](event);
}

What happens when things fail: - Payment Service crashes → its consumer stops committing offsets; on
restart, it resumes from the last committed offset (at-least-once) — no messages lost. - A Kafka broker
crashes → a replica (follower) on another broker is promoted to leader; producers/consumers reconnect
transparently. - A consumer crashes mid-processing → the group coordinator detects the missed heartbeat,
triggers a rebalance, and reassigns that partition to another consumer in the group, which resumes from
the last committed offset. - Message processing fails → retry (often with exponential backoff); after
repeated failures, route to a Dead Letter Topic so a poison message doesn’t block the whole partition. - A
duplicate message arrives → the idempotency check above (dedupe by a business key, e.g., orderId )
makes reprocessing safe. - Consumer group rebalances → partitions are redistributed; briefly no messages
are processed for the moving partitions.
PART 5 — API GATEWAY
Problem without a gateway: every client must know the network location of every service, implement
auth/rate-limiting itself, and handle N different service URLs. Cross-cutting concerns (auth, logging, SSL)
get duplicated in every service.

API Gateway is a single entry point that handles: routing, authentication/authorization, rate limiting,
request filtering/logging, load balancing, SSL termination.

Client → API Gateway → { Order Service | Payment Service | Customer Service }

@Bean
public RouteLocator routes(RouteLocatorBuilder builder) {
return [Link]()
.route("order-service", r -> [Link]("/orders/**")
.filters(f -> [Link](c -> [Link]("orderCB")))
.uri("lb://ORDER-SERVICE"))
.build();
}

Request flow: client hits the Gateway → Gateway validates the JWT (Part 12) → applies rate limiting →
resolves the target service via service discovery (Part 6) → load-balances across instances (Part 7) →
forwards the request → streams the response back.
PART 6 — SERVICE DISCOVERY
Problem: with static URLs, every consumer must be reconfigured whenever an instance’s IP/port changes
(routine in cloud/container environments where instances come and go). A service registry solves this:
services register themselves on startup (with periodic health checks), and consumers discover healthy
instances dynamically.

Eureka (Netflix/Spring Cloud) — services register with a Eureka server; clients query it for healthy
instances
Kubernetes Service Discovery — a K8s Service gives a stable DNS name/IP that load-balances
across pod instances automatically
DNS-based discovery — resolving a service name via DNS to a set of IPs

In Kubernetes-native shops, Kubernetes’ built-in discovery has largely replaced Eureka; Eureka remains
common in traditional Spring Cloud stacks not yet on K8s.
PART 7 — LOAD BALANCING
Client-side load balancing: the calling service (or its library, e.g. Spring Cloud LoadBalancer) knows
about all healthy instances and picks one itself before calling. Server-side load balancing: a dedicated
component (e.g., the Gateway, a cloud load balancer, or K8s Service ) sits in front and distributes traffic.

┌── Order Service Instance 1

Load Balancer ───┼── Order Service Instance 2
└── Order Service Instance 3

Common strategies: round robin (rotate evenly), weighted routing (send more traffic to bigger
instances), always combined with health checks so traffic never goes to a dead instance. In Kubernetes,
the Service object performs this transparently across pod replicas.
PART 8 — FAULT TOLERANCE
Why microservices fail: network failures, timeouts, a dependency being unavailable, database/Kafka
outages — and critically, partial failure: unlike a monolith’s all-or-nothing crash, in microservices some
calls succeed while others fail, which is a fundamentally new failure mode to design for.

Resilience patterns (Resilience4j + Spring Boot):

@CircuitBreaker(name = "paymentService", fallbackMethod = "fallbackCharge")

@Retry(name = "paymentService")
@TimeLimiter(name = "paymentService")
public CompletableFuture<PaymentResponse> charge(PaymentRequest request) {
return [Link](() -> [Link](request));
}

public CompletableFuture<PaymentResponse> fallbackCharge(PaymentRequest request, Throwable t) {

return [Link]([Link]());
}

Timeout — never wait forever for a dependency

Retry with exponential backoff — retry transient failures, with increasing delay to avoid
hammering a struggling service
Circuit Breaker — stop calling a failing dependency for a cooldown period instead of piling up failing
requests
Bulkhead — isolate thread pools per dependency so one slow dependency can’t exhaust all threads
and starve unrelated calls
Rate Limiting — cap outgoing/incoming call rates to protect both sides
Fallback — return a degraded-but-useful response (cached data, a “pending” state) instead of a hard
failure

Circuit Breaker state machine:

CLOSED --(failure rate exceeds threshold)--> OPEN --(wait duration elapses)--> HALF-OPEN
^ |
|------------------------------(trial calls succeed)-------------------------- |
|<--------------------------------(trial calls fail)---------------------------OPEN

CLOSED — calls flow normally, failures are counted. OPEN — calls fail fast immediately (no network call at
all) for a cooldown window, protecting the struggling downstream service and freeing the caller’s threads.
HALF-OPEN — a limited number of trial calls are allowed through; if they succeed, back to CLOSED; if they
fail, back to OPEN.
PART 9 — DISTRIBUTED TRANSACTIONS & SAGA
@Transactional guarantees ACID within one database. Across Order DB , Payment DB , Inventory DB there is no

single transaction manager spanning all three — a classic two-phase commit is possible in theory but rarely
used in practice (it’s slow, blocking, and doesn’t play well with availability goals).

Saga Pattern — break the business transaction into a sequence of local transactions, each with a
compensating transaction to undo it if a later step fails.

Create Order → Reserve Inventory → Process Payment → Confirm Order (happy path)

Payment fails → Release Inventory (compensate) → Cancel Order (compensate)

Choreography — each service listens for events and reacts, publishing its own event in turn (no
central coordinator; simple for short sagas, harder to trace as they grow)
Orchestration — a central Saga Orchestrator explicitly calls each step and issues compensations
on failure (easier to reason about and monitor; adds a coordinating component)

Choose choreography for a small number of steps and loosely-coupled reactions; choose orchestration
when the workflow is long, has complex branching, or needs centralized visibility/monitoring — a very
common interview follow-up.
PART 10 — OUTBOX PATTERN
The dual-write problem: a service wants to update its database and publish a Kafka event for the same
business action. What if the DB commit succeeds but the Kafka publish fails (or vice versa)? You get
inconsistent state — data changed but no one downstream was told, or an event fired for a change that
never actually committed.

Transactional Outbox: write the business change and the event to be published into the same local
database transaction, in an outbox table. A separate Event Publisher process (polling the table, or
using Change Data Capture like Debezium) reliably reads the outbox and publishes to Kafka, then marks
rows as sent.

Application → [Business DB write + Outbox table write] (one local ACID transaction)
│
▼
Event Publisher (poller/CDC)
│
▼
Kafka

@Transactional
public void placeOrder(OrderRequest request) {
Order order = [Link](new Order(request));
[Link](new OutboxEvent("OrderCreated", [Link]())); // same DB, same transaction
}

Because both writes are in the same local database transaction, they succeed or fail together — the event
can never be “lost” relative to the DB change.
PART 11 — DISTRIBUTED DATA CONSISTENCY &
CAP
Strong consistency — every read sees the latest write immediately (hard to achieve across services
without heavy coordination/latency cost)
Eventual consistency — after an update, all replicas/services will converge, but not instantly (the
normal state in event-driven microservices)
Read-your-own-writes — a specific, weaker guarantee: the user who just wrote sees their own
change immediately, even if the rest of the system hasn’t converged yet

CAP theorem in this context: during a network partition, a service must choose between Consistency
(refuse the request until it can confirm the latest state) and Availability (serve a possibly-stale response).
Example: Inventory Service can’t reach Payment Service to confirm stock reservation status — does it
reject the order (favoring consistency) or accept it optimistically and reconcile later (favoring availability)?
Most e-commerce systems favor availability for browsing/reads and consistency for the actual
payment/stock-decrement step — a nuanced, “it depends” answer interviewers want to hear rather than a
flat CAP definition.
PART 12 — SECURITY

Client → Identity Provider (login, issues JWT) → API Gateway (validates JWT) → Service A / Service B

OAuth2 — authorization framework; OpenID Connect — an identity layer on top of OAuth2 for
authentication
JWT (JSON Web Token) — [Link] , base64-encoded; the payload carries claims (user
id, roles, expiry); the signature lets any service verify authenticity without calling the Identity Provider
on every request
Access Token — short-lived, used on every request; Refresh Token — long-lived, used to obtain a
new access token without re-authenticating
Resource Server — a microservice that validates the token and enforces authorization (often via
Spring Security’s @PreAuthorize )
Service-to-service auth — internal calls typically also carry a token (client-credentials OAuth2 flow
or mTLS) so Service B can trust that a call really came from Service A/the Gateway, not an attacker
inside the network

@PreAuthorize("hasRole('CUSTOMER')")
@PostMapping("/orders")
public ResponseEntity<Order> placeOrder(@RequestBody OrderRequest request) { ... }

Gateway-level: the Gateway validates the JWT signature/expiry once at the edge; downstream services can
re-validate independently (defense in depth) without needing a network call back to the Identity Provider,
because JWT verification is just a local signature check.
PART 13 — OBSERVABILITY
Why plain logs stop working: one request now crosses many processes; grepping isolated log files per
service can’t reconstruct the full picture.

Client → Gateway → Order Service → Payment Service → Kafka → Notification Service

Correlation ID / Trace ID — generated at the Gateway, propagated on every downstream call (HTTP
header or Kafka message header), so every log line across every service for one request can be
grepped together
Span ID — identifies one specific hop/operation within the overall trace
Structured logging — JSON logs (not free text) so log aggregators (e.g., ELK) can query by field,
including the trace ID
Distributed tracing — tools like OpenTelemetry (instrumentation standard) feeding Jaeger or
Zipkin (trace visualization) show the full waterfall of a request across every service, with per-hop
latency
Metrics — throughput, latency (p50/p95/p99), error rate, CPU/memory/JVM metrics, usually scraped
by Prometheus and visualized in Grafana
PART 14 — CONFIGURATION MANAGEMENT
Hardcoded config means rebuilding a Docker image just to change a URL or feature flag — unacceptable at
microservice scale, especially across dev/staging/prod. Spring Cloud Config centralizes configuration in a
Git-backed config server; services fetch their config at startup (and can refresh at runtime via @RefreshScope
+ a /actuator/refresh trigger, often wired to a message bus for broadcast refresh). Secrets (DB passwords,
API keys) are kept out of Git and injected via a secrets manager (Vault, AWS Secrets Manager, or
Kubernetes Secret s) rather than plain config files. Spring Profiles ( dev , staging , prod ) let the same artifact
behave differently per environment without code changes.
PART 15 — DEPLOYMENT: DOCKER &
KUBERNETES
Docker: an Image is an immutable snapshot of your app + runtime; a Container is a running instance of
an image; a Dockerfile defines how to build the image; a Registry (Docker Hub, ECR) stores images for
deployment.

FROM eclipse-temurin:21-jre
COPY target/[Link] [Link]
ENTRYPOINT ["java", "-jar", "/[Link]"]

Kubernetes — because running (and healing, scaling, updating) hundreds of container instances by hand
is impossible, Kubernetes automates it: a Pod runs one or more containers; a Deployment manages a set
of Pod replicas and rolling updates; a Service gives Pods a stable network identity and load-balances
across them; a ConfigMap/Secret injects configuration/secrets; an Ingress routes external HTTP traffic
in; a Namespace isolates groups of resources; the Horizontal Pod Autoscaler adds/removes Pod
replicas automatically based on load.

Internet → Ingress → Service → { Pod | Pod | Pod }

PART 16 — CI/CD
A typical pipeline: Git push triggers the pipeline → build the app → run unit tests → run integration
tests (often against Testcontainers-managed real dependencies) → build the Docker image → push to the
registry → deploy to the cluster (rolling update) → automated rollback if health checks fail post-deploy.
This lets teams release many times a day safely, directly solving the “slow release cycle” pain point from
Part 2.
PART 17 — TESTING MICROSERVICES
Testing is harder than in a monolith because a request now spans multiple independently-deployed
processes you can’t just call in-process.

Unit testing — same as always, test one class/method in isolation

Integration testing — test a service against its real infrastructure (DB, Kafka) using
Testcontainers (spins up real Docker containers for tests, avoiding “works with H2, breaks on
Postgres” bugs)
Contract testing (e.g., Spring Cloud Contract, Pact) — the consumer and provider of an API agree on
a contract; each side tests against that contract independently, catching breaking changes without
needing to run the other service
Component testing — test one service fully, with its real dependencies stubbed/virtualized at the
boundary
End-to-end testing — the full system running together; valuable but slow/flaky, used sparingly for
critical flows
PART 18 — ADVANCED PATTERNS
For each: problem → solution → when to use.

API Composition / Aggregator — a service (or the Gateway) calls several services and combines
results for the client, since the client shouldn’t have to call N services itself
Strangler Fig — incrementally replace pieces of a legacy monolith by routing specific routes to new
services while the rest still goes to the monolith, until nothing is left of the old system
Sidecar — deploy a helper process (e.g., a service mesh proxy like Envoy) alongside your service in
the same Pod to handle cross-cutting concerns (mTLS, retries, metrics) without touching application
code
Ambassador — a specialized sidecar that proxies outbound calls, adding retry/circuit-breaking
transparently
Anti-Corruption Layer — a translation layer that isolates your clean domain model from a legacy or
external system’s messy model
Backend for Frontend (BFF) — a dedicated aggregation service per client type (mobile BFF, web
BFF) tailoring responses to each client’s needs
CQRS — separate the write model (normalized, transactional) from the read model (denormalized,
fast, often event-driven-updated); use when read and write patterns/scaling needs diverge sharply
Event Sourcing — store state as an append-only sequence of events rather than current-state rows;
current state is derived by replaying events; enables full audit history and time-travel debugging, at
the cost of query complexity
Idempotency — design consumers/APIs so repeating the same operation has no additional effect
(essential for at-least-once delivery, Part 4)
Distributed locking / Leader election (e.g., via ZooKeeper, etcd, or Redis) — coordinate exclusive
access or a single active coordinator across multiple service instances
Dead Letter Queue / poison messages — route messages that repeatedly fail processing to a
separate topic/queue so they don’t block the rest of the partition, for manual inspection later
PART 19 — COMPLETE REAL-WORLD PROJECT: E-
COMMERCE MICROSERVICES SYSTEM
Services: API Gateway · Customer Service · Product Service · Order Service · Payment Service · Inventory
Service · Notification Service. Stack: Spring Boot, Spring Security, Spring Cloud Gateway,
OpenFeign/WebClient, Kafka, PostgreSQL, Redis, Docker, Kubernetes, Resilience4j, OpenTelemetry,
Prometheus, Grafana.

Order Service — representative slice (each service follows this same layered shape: Entity →
Repository → Service → Controller → Kafka producer → Resilience-wrapped Feign client → Dockerfile → K8s
manifest):

// Entity
@Entity
public class Order {
@Id @GeneratedValue private Long id;
private Long customerId;
private Long productId;
private int quantity;
@Enumerated([Link]) private OrderStatus status;
}

// Repository
public interface OrderRepository extends JpaRepository<Order, Long> { }

// Service — orchestrates inventory (sync), payment (sync), and publishes an event (async)
@Service
public class OrderService {
private final InventoryClient inventoryClient; // Feign, wrapped with @CircuitBreaker
private final PaymentClient paymentClient; // Feign, wrapped with @CircuitBreaker
private final KafkaTemplate<String, OrderCreatedEvent> kafkaTemplate;
private final OrderRepository orderRepository;
private final OutboxRepository outboxRepository;

@Transactional
public Order placeOrder(OrderRequest request) {
[Link]([Link](), [Link]());
Order order = [Link](new Order(request, [Link]));
[Link]([Link]("OrderCreated", order)); // Outbox pattern, Part 10
return order;
}
}

// Controller
@RestController
@RequestMapping("/orders")
public class OrderController {
@PostMapping
@PreAuthorize("hasRole('CUSTOMER')")
public ResponseEntity<Order> placeOrder(@RequestBody OrderRequest request) {
return [Link]([Link](request));
}
}

FROM eclipse-temurin:21-jre
COPY target/[Link] [Link]
ENTRYPOINT ["java","-jar","/[Link]"]

apiVersion: apps/v1
kind: Deployment
metadata: { name: order-service }
spec:
replicas: 3
selector: { matchLabels: { app: order-service } }
template:
metadata: { labels: { app: order-service } }
 spec:
containers:
- name: order-service
image: registry/order-service:latest
ports: [{ containerPort: 8080 }]
---
apiVersion: v1
kind: Service
metadata: { name: order-service }
spec:
selector: { app: order-service }
ports: [{ port: 80, targetPort: 8080 }]

The other six services follow the same shape, swapping the domain entity and adding what’s specific to
them (Payment Service adds idempotent Kafka consumption from Part 4; Notification Service is a pure
Kafka consumer with no REST API; Inventory Service exposes the reserveStock / releaseStock endpoints called
above).
PART 20 — COMPLETE REQUEST FLOW
SCENARIOS
Scenario 1 — Place Order (happy path): Client → Gateway (JWT validated, rate-limited) → Order Service
→ Inventory Service (REST, stock reserved) → Payment Service (REST, charge succeeds) → Order saved as
CONFIRMED + Outbox event written, same local transaction → Event Publisher pushes OrderCreated to Kafka →
Notification Service consumes it and emails the customer. A trace ID generated at the Gateway threads
through every hop’s logs and spans.

Scenario 2 — Payment Failure: Inventory reservation succeeds, but the Payment Service call fails
(declined card). Order Service catches this, triggers the Saga compensation: release the reserved
inventory, mark the order CANCELLED , and (via Outbox) publish an OrderCancelled event so Notification Service
can inform the customer.

Scenario 3 — Inventory Service Down: the Feign call times out; the Circuit Breaker trips after
repeated failures, causing subsequent calls to fail fast (no more waiting on a dead service); Order Service
returns a clear “temporarily unavailable” response instead of hanging, and can offer a fallback (e.g.,
accept the order as PENDING_STOCK_CHECK for later reconciliation, if the business allows it).

Scenario 4 — Kafka Consumer Crash: Notification Service’s consumer process dies mid-read. The Group
Coordinator detects the missed heartbeat and rebalances, handing that partition to another Notification
Service instance, which resumes from the last committed offset — no message is lost, though there’s a
brief processing pause during the rebalance.

Scenario 5 — Duplicate Kafka Message: network retries or a rebalance cause the same OrderCreated
event to be delivered twice. Payment Service’s consumer checks [Link](...) before
processing (the idempotency pattern from Part 4) — the second delivery is a safe no-op.

Scenario 6 — Database Failure: if Order DB is briefly unreachable, the @Transactional write fails fast and
the client sees an error rather than a hang; with a replicated/HA database setup, a standby is promoted
and the service reconnects automatically after a short outage window — this is why each service owning its
own small database (rather than one giant shared one) also limits the blast radius of a DB outage to a
single capability.
PART 21 — PRODUCTION ARCHITECTURE

Internet
│
Load Balancer
│
Ingress / API Gateway ── Identity Provider (OAuth2/OIDC)
│
┌──┴─────────────────────────────────────────────┐
│ Order Svc Payment Svc Inventory Svc ... │ (Kubernetes cluster, autoscaled)
└──┬──────────────┬──────────────┬────────────────┘
│ │ │
Order DB Payment DB Inventory DB Redis (caching/session)
│ │ │
└──────────────┴──────┬───────┘
▼
Kafka Cluster
│
Prometheus/Grafana (metrics) · OpenTelemetry → Jaeger (tracing)
Centralized logging (ELK) · Alerting · CI/CD pipeline
Secrets Manager (Vault) · Disaster recovery (multi-AZ/region backups)

Every arrow above corresponds to a Part in this course: the Gateway and Identity Provider (Parts 5, 12), the
service mesh of independently-deployed, independently-scaled services (Parts 1–3, 6, 7), the database-per-
service layer (Part 7 domain, Part 20 failure modes), Kafka for async workflows (Parts 4, 9, 10), and the
observability/CI/CD/secrets layer wrapping the whole system (Parts 13, 14, 16).
PART 22 — INTERVIEW PREPARATION

Beginner

What are Microservices? Small, independently deployable services, each owning a business
capability and its own data, communicating over the network.
Monolith vs Microservices? See the Part 1 comparison table — deployment, scaling, DB, team
structure are the core axes interviewers probe.
What is loose coupling? Services depend only on each other’s public contracts (APIs/events), never
on internal implementation details, so one service can change internally without breaking others.
What is service discovery? The mechanism (registry + health checks) by which services find each
other’s current network location dynamically, instead of hardcoded URLs.

Intermediate

REST vs Kafka? Sync/blocking request-response vs async/decoupled event-driven — pick based on

whether the caller needs an immediate answer and how many consumers should react.
API Gateway? Single entry point centralizing routing, auth, rate limiting, and other cross-cutting
concerns.
Circuit breaker? Stops calling a failing dependency for a cooldown period to prevent cascading
failure and free up caller resources; states are CLOSED → OPEN → HALF-OPEN.
Database per service — why? Avoids coupling teams/services through a shared schema; trade-off
is losing cross-service joins/transactions, addressed via events and Sagas.
Eventual consistency? After a write, the system converges to a consistent state over time rather
than instantly — the normal state in distributed, event-driven systems.
Saga? A sequence of local transactions with compensating actions for failure, replacing a single
distributed ACID transaction; choreography (event-reactive) vs orchestration (central coordinator).

Advanced

Distributed transactions? Two-phase commit exists but is rarely used in practice (blocking, poor
availability); Sagas are the practical answer.
Outbox pattern? Solves the dual-write problem by writing the DB change and the event to publish in
one local transaction, then relaying reliably to Kafka.
Idempotency? Designing operations so repeated execution has the same effect as executing once —
essential under at-least-once delivery.
Exactly-once processing? Practically achieved via at-least-once delivery + idempotent consumers,
or Kafka’s transactional producer/consumer APIs for tighter guarantees.
Kafka rebalancing? Redistribution of partitions among consumers in a group when membership
changes (crash, scale up/down); briefly pauses processing for affected partitions.
CAP theorem? During a network partition, choose Consistency (reject until certain) or Availability
(serve possibly-stale data) — explain with a concrete scenario, not just the textbook definition.
CQRS? Separate read and write models when their access patterns/scaling needs diverge
significantly.
Event sourcing? Persist state as an event log rather than current-state rows; enables audit/replay at
the cost of query complexity.
Kubernetes scaling? Horizontal Pod Autoscaler adds/removes replicas based on metrics (CPU,
custom metrics like queue depth).
Distributed tracing? Trace ID + Span ID propagated across every hop, visualized in Jaeger/Zipkin
via OpenTelemetry instrumentation, to reconstruct one request’s full journey.

System Design Prompts (practice these end-to-end)

1. Design an E-commerce system (services, data ownership, order flow, failure handling)
2. Design a Payment system (idempotency, reconciliation, fraud checks, PCI concerns)
 3. Design an Order processing system (Saga steps, compensation, status tracking)
4. Design a Food delivery system (real-time location, matching, ETA, notifications)
5. Design a UPI/payment transaction system (exactly-once transfer guarantees, ledger consistency,
timeout/reversal handling)

For each, structure your answer as: clarify requirements/scale → identify services & bounded contexts
→ data ownership per service → sync vs async communication choices → failure modes and how each is
handled → security → observability → then invite follow-up questions. Interviewers are grading your
reasoning process and trade-off awareness far more than a “correct” diagram.

Common mistakes to avoid: proposing microservices for a project with no team-scaling or deployment-
cadence problem; drawing a shared database “for simplicity”; ignoring partial failure and network
unreliability; treating Saga/eventual consistency as an afterthought instead of a first-class design decision;
not mentioning observability until asked.