ENGINEERING HANDBOOK

Volume 5

Infrastructure,
Deployment &
Debugging

How the system is built and run: the multi-stage Dockerfile and the Compose
stack line by line, networking, volumes, configuration precedence, health-
gated startup, how to run it locally and in containers, and a component-by-
component debugging playbook.

Docker Compose Healthchecks Alembic Ops

Distributed URL Shortener · FastAPI · PostgreSQL 16 · Redis 7 · SQLAlchemy 2.0 (async) · Alembic · Docker
Internal engineering onboarding & interview-preparation document
Table of Contents
1. The Dockerfile, Line by Line 3
1.1 A multi-stage build 3
1.2 The techniques worth knowing 3

2. [Link], Service by Service 4

2.1 The api service 4
2.2 The postgres service 4
2.3 The redis service 5

3. Networking, Volumes, Config & Startup 6

3.1 Service discovery on the bridge network 6
3.2 Persistence via named volumes 6
3.3 Configuration precedence 6
3.4 Startup order and health gating 7

4. Running It, Two Ways 8

4.1 Docker Compose (recommended) 8
4.2 Manual setup 8
4.3 Running the tests 8
4.4 The full execution flow, one more time 9

5. Debugging Playbook 10
5.1 The app won’t start 10
5.2 Database problems 10
5.3 Redis problems 10
5.4 Auth problems 10
5.5 Redirect / URL problems 11
5.6 The tools you will use 11

toc ii
1. The Dockerfile, Line by Line
This volume is about running and operating the system. We start with how the application image is
built, because everything else (Compose, networking, startup order) builds on it.

1.1 A multi-stage build

The Dockerfile uses two stages. The builder stage installs a C compiler and Postgres headers to compile
dependencies (asyncpg and psycopg2 have native extensions). The runtime stage starts fresh from a slim
base and copies only the installed packages — not the compilers — so the final image is smaller and has a
reduced attack surface.

Dockerfile

# --- Builder stage ---

FROM python:3.12-slim AS builder
RUN apt-get update && apt-get install -y --no-install-recommends gcc libpq-dev ...
WORKDIR /app
COPY [Link] . # copied first for layer caching
RUN pip install --no-cache-dir -r [Link]

# --- Runtime stage ---

FROM python:3.12-slim AS runtime
RUN apt-get install -y --no-install-recommends libpq5 # runtime lib only
RUN useradd --create-home --shell /bin/bash appuser # non-root user
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-
packages
COPY --chown=appuser:appuser . .
USER appuser
EXPOSE 8000
HEALTHCHECK ... CMD python -c "... urlopen('[Link] || exit 1
CMD ["uvicorn","[Link]:app","--host","[Link]","--port","8000","--workers","4"]

1.2 The techniques worth knowing

Technique Line Why it matters

Multi-stage build two FROM s Ship runtime artefacts without build tools → smaller, safer
image

Layer caching COPY [Link] Dependencies re-install only when requirements change,
before code not on every code edit

--no-cache-dir pip install Don’t bloat the image with pip’s download cache

Non-root user useradd + USER A compromised process isn’t root inside the container

HEALTHCHECK curl /health Docker/K8s can detect and restart an unhealthy container

4 workers CMD Rule of thumb ~2×CPU cores for I/O-bound async

workloads

NOTE — Dev vs prod command

The Dockerfile’s default CMD runs 4 workers (production). The Compose file overrides it with --reload and a
mounted source volume for hot-reload during development. Same image, different command.

Distributed URL Shortener - Engineering Handbook Page 3 of 11

2. [Link], Service by Service
Compose defines three services on a shared network with two named volumes. It is the one-command way to
bring up the whole stack locally.

Three containers on one bridge network; the app reaches the others by service name; data persists in named volumes.

2.1 The api service

• Builds from the local Dockerfile (runtime target) and overrides the command with uvicorn --reload .
• Mounts the project directory into the container ( .:/app ) so code edits reload instantly.
• Publishes 8000:8000 to the host.
• Injects all configuration as environment variables, crucially pointing DATABASE_HOSTNAME=postgres and
REDIS_URL=redis://redis:6379/0 at the service names, not localhost.

• Declares depends_on with condition: service_healthy for both dependencies, so it won’t start until
they are ready.

2.2 The postgres service

Runs postgres:16-alpine , persists data to the postgres_data volume, and defines a health check using
pg_isready . The health check is what lets the api service wait for a genuinely ready database rather than
merely a started container.

Distributed URL Shortener - Engineering Handbook Page 4 of 11

 [Link] (postgres)

healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DATABASE_USERNAME:-postgres} -d ${DATABASE_NAME:-
urlshortener}"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s

2.3 The redis service

Runs redis:7-alpine with append-only persistence and, importantly, a bounded memory with LRU eviction
— correct for a cache:

[Link] (redis)

command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru

KEY IDEA — Why allkeys-lru is the right policy for a cache

When Redis hits its memory cap it evicts the least-recently-used keys. Because every cached value can be rebuilt
from PostgreSQL, evicting a cold URL is harmless — the next request for it simply repopulates the cache. You
would not use this policy for data that only lives in Redis.

Distributed URL Shortener - Engineering Handbook Page 5 of 11

3. Networking, Volumes, Config & Startup

3.1 Service discovery on the bridge network

All three containers join the urlshortener_net bridge network. Docker’s embedded DNS resolves each
service name to its container IP, so the app connects to postgres and redis by name. This is why the app
config uses DATABASE_HOSTNAME=postgres inside Compose but localhost when running bare on your
machine.

3.2 Persistence via named volumes

Volume Mounted at Holds

postgres_data /var/lib/postgresql/data All relational data — survives container recreation

redis_data /data Redis append-only file (cache/rate-limit/blocklist)

NOTE — Volumes outlive containers

docker compose down removes containers but keeps named volumes; your data persists. docker compose
down -v also deletes the volumes — a clean slate, and how you reset a corrupted local database.

3.3 Configuration precedence

Config resolves in this order: explicit environment variables (set by Compose) override the .env file, which
overrides the defaults baked into the Pydantic Settings class. This is what lets one image run unchanged in
dev, staging, and prod — only the injected variables differ.

Distributed URL Shortener - Engineering Handbook Page 6 of 11

 3.4 Startup order and health gating

The api container is blocked until both Postgres and Redis report healthy.

WARNING — Migrations are a manual step

Bringing the stack up does not create the tables. After docker compose up , run docker compose exec api
alembic upgrade head once. Until you do, any DB query will fail because the tables don’t exist — a very
common first-run confusion.

Distributed URL Shortener - Engineering Handbook Page 7 of 11

4. Running It, Two Ways

4.1 Docker Compose (recommended)

one-command stack

cp .[Link] .env
# set a strong SECRET_KEY: openssl rand -hex 32
docker compose up --build
docker compose exec api alembic upgrade head # once, in another terminal
# open [Link]

4.2 Manual setup

bare-metal run

python3.12 -m venv venv && source venv/bin/activate

pip install -r [Link]
createdb urlshortener
cp .[Link] .env # set DB creds + SECRET_KEY, hostnames = localhost
alembic upgrade head
uvicorn [Link]:app --reload --host [Link] --port 8000

4.3 Running the tests

Tests need no running services — they use in-memory SQLite and a mocked Redis:

pytest

pytest # all tests + coverage (fails under 70%)

pytest tests/integration/test_auth.py -v # one file
pytest -k "test_login" -v # by name

Distributed URL Shortener - Engineering Handbook Page 8 of 11

 4.4 The full execution flow, one more time

From build command to first served redirect, with migrations as the manual gate before traffic.

Distributed URL Shortener - Engineering Handbook Page 9 of 11

5. Debugging Playbook
A component-by-component guide to the failures you are most likely to hit, what causes them, and how to
confirm and fix them.

5.1 The app won’t start

Symptom Likely cause Check / fix

Exits immediately on boot Bad config type (e.g. non-numeric Pydantic prints the offending field; fix .env
port)

Hangs then crashes connecting to Started before Postgres was ready Ensure
DB depends_on: service_healthy ; check
docker compose ps

ModuleNotFoundError: app Wrong working dir / path Run from project root; in Docker the code is
under /app

5.2 Database problems

Symptom Likely cause Check / fix

relation "urls" does not Migrations not run alembic upgrade head
exist

Auth/connection refused Wrong host/creds In Compose host = postgres ; bare =

localhost

TimeoutError getting a connection Pool exhausted Raise pool size, or add PgBouncer; look for
un-closed sessions

Alembic can’t connect Alembic uses the sync URL Confirm SYNC_DATABASE_URL / psycopg2
is installed

5.3 Redis problems

Symptom Likely cause Check / fix

Redirects slow / all hit the DB Redis down → cache fails open as redis-cli ping ; check the api log’s
miss startup Redis message

Rate limiting not enforced Redis unavailable (fails open) Same — restore Redis; add alerting on
this state

Readiness probe returns 503 App can’t reach Redis GET /health/ready reports the redis
field; check network/URL

5.4 Auth problems

Symptom Likely cause Check / fix

401 on every protected call Missing/expired/wrong-type token Send Authorization: Bearer ; re-login;
don’t send a refresh token as access

Distributed URL Shortener - Engineering Handbook Page 10 of 11

 Symptom Likely cause Check / fix

Still logged in after logout Blocklist not hit (Redis down) Confirm Redis; blocklist check is O(1)
EXISTS

Token invalid across restarts SECRET_KEY changed Keep the key stable and identical on every
replica

5.5 Redirect / URL problems

Symptom Likely cause Check / fix

404 for a slug you created Inactive/soft-deleted, or wrong slug Check is_active ; slugs are case-
sensitive

Link expires unexpectedly after edit PATCH expiry side-effect (Vol 2, Send expires_at explicitly, or apply the
7.6) fix

Expiry never triggers Create stores NULL expiry (Vol 2, Apply the create + cache-hit expiry fixes
7.1)

Noisy LONG URL = lines in logs Debug print in [Link] Delete the print statement

5.6 The tools you will use

• docker compose ps / logs -f api — container status and live logs.
• docker compose exec postgres psql -U postgres urlshortener — a SQL shell.
• docker compose exec redis redis-cli — inspect keys ( KEYS url:slug:* , TTL ... ).
• The X-Request-ID response header — grep all logs for one request across replicas.
• /docs — try any endpoint interactively.

TIP — Correlation IDs are your best friend

Every response carries an X-Request-ID , and every log line during that request carries the same ID (bound by
RequestIDMiddleware ). When a user reports an error, ask for that ID and grep it across all instances to
reconstruct the entire request.

NOTE — Where to go next

Volume 6 ties it together: end-to-end feature walkthroughs, the missing frontend (and how to add one), practical
guides for extending the system, and a full bank of interview questions with answers.

Distributed URL Shortener - Engineering Handbook Page 11 of 11