Edge Caching for API Gateways in

Microservice Architectures:
A Practical, RFC-9111-Aligned Evaluation
Framework
A practical evaluation framework for correctness-first caching at the edge and gateway

Sky Chen (Digital Channel Team) — Technical Note

Date: 2026-02-05

Keywords: HTTP caching, RFC 9111, API gateway, CDN, microservices, cache key design, validation, purge

1
Abstract
This paper proposes a practical framework for deciding when and how to cache API responses at the
edge and/or at an API gateway in microservice architectures. Rather than treating caching as a single
toggle, we separate the problem into (1) cacheability and correctness constraints, (2) cache key design
and variance control, (3) freshness and validation mechanisms (TTL, revalidation, and surrogate
controls), (4) operational safety (purge, observability, and incident response), and (5) measurable
outcomes (latency, origin load, and cost). We provide a minimal experimental design with synthetic but
reproducible workloads and show how cache hit ratio drives end-to-end latency and origin cost. Finally,
we present a decision matrix and deployment checklist intended for teams operating CDNs, gateways,
and Kubernetes-based services.

2
Contents
1. Introduction

2. Problem statement and failure modes

3. Cacheability tiers and correctness constraints

4. Cache key design and variance control

5. Freshness models: TTL, revalidation, and purge

6. Operational safety: purge, observability, and incident response

7. Evaluation methodology

8. Implementation patterns in common stacks

9. Synthetic experiment results

10. Decision matrix and checklist

11. Conclusion

References

3
1. Introduction
Edge caching for APIs is often discussed as a performance optimization, but in production systems it is
primarily a correctness decision. A cache that serves the wrong representation (for the wrong user,
locale, authorization scope, or feature flag state) can create silent data leakage and hard-to-debug
inconsistencies. For this reason, teams operating a CDN, an API gateway, and a microservice backend
need a shared, explicit model of what is safe to cache, how freshness is maintained, and how cache
behavior is observed and controlled during incidents.

This technical note focuses on shared caches placed "in front" of an API gateway (a CDN or edge
cache) and reverse-proxy caches placed "inside" the gateway layer. We assume a microservice
architecture where requests traverse (1) a client (browser or mobile), (2) an edge/CDN, (3) an API
gateway (authentication, routing, policy enforcement), and (4) one or more services and data stores. The
contribution is a practical evaluation framework that can be used to: (i) classify endpoints by cacheability
and risk, (ii) design cache keys and variance control, (iii) choose freshness and validation strategies
aligned with HTTP caching semantics, (iv) define safe operational controls such as purge and
observability, and (v) quantify outcomes in latency, load, and cost under realistic workload patterns.

The framework follows modern HTTP caching semantics as specified by RFC 9111 [1]. We also
reference practical CDN and gateway implementations to ground the discussion, including Cloudflare's
documented default cache behavior [2] and Kong Gateway's reverse-proxy cache plugins [3].

Figure 1. Reference placement of edge/CDN cache and gateway cache in a microservice architecture.

4
2. Problem statement and failure modes
Why API caching is harder than static asset caching
Static asset caching is usually safe because assets are immutable or content-addressed (e.g., hashed
filenames), and they are not personalized. API responses, in contrast, frequently vary by authentication
context, localization, device, experiment cohort, or business-state transitions. A naive "cache GET
responses for 60 seconds" rule can cause multiple classes of production issues:

1) Personalization leakage: user A receives a response intended for user B because the cache key does
not include an authorization scope, cookie, or relevant header. 2) Staleness bugs: clients see outdated
business-state (e.g., limits, budgets, inventory, mission status) beyond acceptable windows. 3)
Thundering herds and stampedes: when a popular key expires, many concurrent clients trigger origin
requests simultaneously. 4) Debuggability gaps: without explicit telemetry on hit/miss, cache age, and
revalidation, teams attribute failures to upstream services and chase the wrong root cause.

The remainder of the paper turns these risks into a decision process with concrete controls.

Table 1. Core HTTP caching controls used in practice.

Directive / Header Where it applies What it controls Operational note

Cache-Control: max-age Client & shared caches Freshness lifetime for all caches Use for client-side caching when safe

Cache-Control: s-maxage Shared caches Freshness lifetime for shared cachesPrefer for edge caching of APIs

Cache-Control: private Shared caches Prevents shared caching Use for personalized responses

Cache-Control: no-store All caches Do not store response Use for highly sensitive data

ETag / If-None-Match Origin + caches Validation (revalidation) Enables low-cost 304 responses

Last-Modified / If-Modified-Since
Origin + caches Validation (coarser) Useful when ETag is unavailable

Vary Shared caches Declares header variance Document and keep minimal

5
3. Cacheability tiers and correctness constraints
Cacheability and correctness constraints
We recommend classifying each endpoint into one of four cacheability tiers. The tier drives whether edge
caching is permitted, whether the gateway may cache, and which controls are mandatory.

Tier A - Public, non-personalized representations. Examples: public content catalog, static reference
data, public health endpoints. Typical policy: shared caching allowed; explicit Cache-Control with
s-maxage; Vary limited and documented.

Tier B - Shared but partitionable representations. Examples: data shared within a tenant, company, or
role group; responses stable within a small set of authorization scopes. Typical policy: shared caching
allowed only if the cache key includes a stable partition key (e.g., tenant id, role id) and privacy review
passes.

Tier C - Private representations. Examples: user profile, personalized recommendations, transaction

history. Typical policy: allow only private (browser) caching if desired; shared caches must be bypassed;
enforce Cache-Control: private or no-store as appropriate.

Tier D - Non-idempotent or state-transition sensitive. Examples: POST/PUT operations, or GET

endpoints that are read-after-write sensitive (inventory reservations, budgets). Typical policy: do not
cache responses in shared caches; optionally allow short-lived gateway-side memoization only when
guarded by strong validation and explicit business approval.

In RFC 9111 terms, the default stance for sensitive content is "do not store" in shared caches, using
directives like no-store, private, and explicit Vary usage. ■cite■turn0search0■

4. Cache key design and variance control

Cache key design and variance control
A cache key is the function that maps a request to a stored representation. If the key is too coarse, you
risk serving the wrong representation. If it is too fine, hit ratio collapses and the cache provides little
value. We recommend designing keys by starting from "minimal correctness" and then tightening
variance only where necessary.

Key components that are usually required: - HTTP method (GET vs HEAD) and normalized path. -
Selected query parameters that change the representation. - Selected request headers that change the
representation, as captured by Vary. - A stable partition key if Tier B caching is used (e.g., tenant or
role).

Key components that are usually harmful: - Volatile analytics parameters (utm_*, tracking ids). -
Non-semantic ordering differences (unordered query parameters). - Device-specific noise if the response
does not actually vary.

The Vary header is the standardized way for an origin to declare which request header fields affect the
representation. Caches that ignore or incorrectly implement Vary can serve incorrect responses. RFC
9111 specifies how Vary interacts with stored responses and validation. [1]

Practical guidance: 1) Prefer representation-stable headers (Accept-Language, Accept-Encoding) only

when truly needed. 2) Avoid varying on Authorization or Cookie in shared caches unless you first map

6
those to a stable partition key (Tier B). 3) When in doubt, bypass shared caches until you have telemetry
for the candidate Vary set.

Many CDNs and gateways provide a configurable cache key; for example, Azure Front Door exposes
query string caching behavior options, which influence whether query strings are ignored or included in
the cache key. [4].

5. Freshness models: TTL, revalidation, and surrogate controls

Freshness models: TTL, revalidation, and surrogate controls
Caching is not a binary choice; it is a choice of freshness model. We present three canonical models,
ordered by increasing correctness and operational complexity:

Model 1 - TTL-only freshness. The cache serves a representation until its max-age (or s-maxage for
shared caches) expires, after which a new origin fetch is required. This model is simple but can amplify
stampedes at expiry boundaries.

Model 2 - TTL plus validation (ETag or Last-Modified). The cache may serve a stored response and then
revalidate it with a conditional request (If-None-Match or If-Modified-Since). If the origin replies 304 Not
Modified, the cache can extend freshness without transferring the full payload. Validators are defined by
RFC 9111 and are the standard mechanism for safe reuse of responses. [1].

Model 3 - Surrogate controls and explicit purge. Some deployments treat the edge as an
application-level cache by using surrogate directives (e.g., surrogate keys) and explicit purge hooks. This
enables aggressive caching with precise invalidation when business events occur (catalog change, price
update, mission state transition). However, it requires robust purge pipelines and incident playbooks,
because purge failures manifest as user-visible staleness.

In real systems, a hybrid approach is common: Tier A content uses Model 1 or 2 at the edge; Tier B uses
Model 2 with partitioned keys; Tier C bypasses shared caches but may allow private browser caching;
Tier D is typically uncached at the edge.

Note that some providers do not cache certain MIME types by default; for example, Cloudflare states
that it does not cache HTML or JSON by default, and instead caches based on file extension unless
rules override behavior. [2]. This matters for APIs that return JSON: a team might believe an endpoint is
cached when it is not, or vice versa if cache rules are changed later.

7
Figure 2. Conceptual trade-off between TTL and staleness risk, and the effect of validation.

8
6. Operational safety: purge, observability, and incident
response
Operational safety: purge, observability, and incident response
Caching changes failure modes. A healthy cache can hide origin instability; a misconfigured cache can
spread incorrect behavior quickly. Therefore, caching must be operated with explicit controls:

Purge mechanisms: - Targeted purge by cache key or surrogate key (preferred). - Time-based expiry
(fallback). - Emergency global bypass: a feature flag that forces Cache-Control: no-store or injects a
cache-bypass header.

Observability: - Metrics: hit ratio, miss ratio, revalidation ratio, age distribution, and "served stale" counts.
- Traces: annotate spans with cache status (HIT, MISS, REVALIDATED) and cache age. - Logs: include
cache key hash, Vary set, and upstream latency.

Incident playbooks: - If incorrect responses are served, first enable bypass to stop spread, then purge
affected keys, then correct key/Vary configuration. - If origin is overloaded, enable temporary
stale-while-revalidate behavior if supported, and increase TTL conservatively for Tier A/B. - If stampedes
occur at expiry boundaries, introduce jitter, request coalescing, or shielding (collapsing concurrent
misses).

At the gateway layer, reverse-proxy caching can be added selectively. Kong's Proxy Cache plugin family
is one example of an implementation that caches based on response codes, content types, and request
methods, and an advanced variant supports Redis-backed storage. [3]. The benefit is fine-grained
control close to the application, but the risk is that gateway-side caches can become a hidden state
unless monitored and purged systematically.

6.1 Cache stampede mitigation

Cache stampede mitigation: jitter, coalescing, and shielding
Even correctly configured caches can overload origins when many clients request the same key at the
moment it expires. This is the cache stampede problem. Three mitigation techniques are common:

1) TTL jitter. Rather than a fixed TTL for all keys, add small randomization so expirations are spread over
time. This reduces synchronized refresh spikes.

2) Request coalescing (collapsed forwarding). When multiple concurrent requests miss for the same key,
the cache or gateway allows only one upstream fetch and makes others wait for the result. Some
platforms provide this as "request collapsing" or "cache lock" behavior; when absent, a gateway-side
singleflight mechanism can help.

3) Shielding. Introduce an intermediate cache layer (for example, a gateway cache behind an edge
cache) so that edge misses do not all hit the origin directly. Shielding also enables different TTL and
validation strategies by layer (longer at the edge, shorter at the gateway).

The recommended practice is to implement at least one mitigation for any key that is known to be high
popularity (top 1%).

9
6.2 Security considerations
Security considerations: poisoning, normalization, and privacy
Shared caches sit on the trust boundary between untrusted clients and protected origins. If an attacker
can influence the cache key or stored representation, they may cause cache poisoning (serving
malicious or incorrect content to other clients) or privacy leakage. A security-oriented caching review
should include:

Cache key normalization: - Normalize query parameter ordering and casing to avoid multiple keys for
equivalent requests. - Remove non-semantic parameters and enforce allow-lists for cache key
parameters. - Reject ambiguous encodings (double-encoding, mixed UTF-8/percent-encoding) at the
gateway layer before caching.

Header variance controls: - Avoid varying on arbitrary headers supplied by clients unless the origin
explicitly declares them via Vary. - Prefer a small, explicit set of variance headers; each additional
header multiplies the cache key space and reduces hit ratio.

Authorization and identity: - Do not cache responses containing identity-bound information in shared
caches unless you have a stable partition key and have performed privacy review. - When in doubt, use
Cache-Control: private or no-store to prevent shared storage. RFC 9111 and practical guidance
emphasize the difference between private and shared caches. [1,5].

Response splitting and injection: - Ensure that upstream services cannot inject unexpected cache
directives through untrusted input. - At the gateway, sanitize and override Cache-Control for endpoints
that should never be stored, regardless of upstream behavior.

These controls make caching safer, but they also make it more predictable and measurable.

7. Evaluation methodology
Evaluation methodology: what to measure and how
To decide whether caching is worth enabling for an endpoint (and at which layer), we recommend
measuring:

1) Latency distribution: p50, p95, and p99 end-to-end, split by cache status (HIT/MISS/REVALIDATED).
2) Origin load: requests per second at the gateway and at each upstream service, including database
queries and cache reads. 3) Error budget impact: changes in 4xx/5xx rates, timeouts, and tail latency
during spikes. 4) Correctness indicators: staleness incidents, cache poisoning attempts blocked, and
authorization leakage tests. 5) Cost proxies: compute utilization at origin, egress volume, and cache
storage/operation costs.

Workload design: - Include both steady-state and burst traffic. - Include key skew (Zipf-like popularity)
because caches benefit from reuse. - Include invalidation events (e.g., catalog updates) to test purge
correctness.

A minimal reproducible benchmark can be synthetic: generate requests to N endpoints with controlled
popularity, response sizes, and update rates. The figures below use synthetic numbers to demonstrate
relationships, not to claim universal magnitudes.

10
 Figure 3. Effect of cache hit ratio on p50 and p95 latency (synthetic example).

Figure 4. Illustrative origin-cost reduction as cache hit ratio increases (synthetic).

11
8. Implementation patterns in common stacks
Implementation patterns in common stacks
CDN layer: - Default behavior is not uniform across providers. Cloudflare documents that it caches
based on file extension and does not cache HTML or JSON by default, which can surprise API teams if
the API is served without a cacheable extension. [2]. - Providers typically support cache rules that
override defaults and define query string behaviors, cache keys, and TTL. - Always couple cache
enablement with response headers so that behavior is visible to clients and to intermediaries.

Edge-to-origin query string handling: - Query strings can represent either semantic variance (e.g.,
?page=2) or cache-busting tokens (e.g., ?v=hash). - Azure Front Door provides configuration for query
string caching behavior, including options that ignore query strings or include them in the cache key, and
documentation distinguishes these behaviors. [4]. - A recommended practice is to "include only known
semantic parameters" and ignore the rest, when the platform supports it.

API gateway layer: - Reverse-proxy caching can be enabled per-route, which is useful when only a small
set of endpoints are Tier A/B. - Kong Gateway's Proxy Cache plugin caches responses based on
configurable response codes, content types, and request methods; the advanced variant adds
Redis-based storage for better scalability. [3].

In all cases, the cache should be treated as a first-class component with versioned configuration, change
review, and rollback.

9. Synthetic experiment results

Synthetic experiment: results and interpretation
We simulate an API system with a mix of Tier A and Tier B endpoints and a Zipf-like popularity
distribution. For the purpose of illustration, assume: - Median origin response time is 220 ms (including
service and database time). - Median edge response time is 30 ms (cache hit path). - p95 values are
higher due to queueing effects and cold starts.

Figure 2 shows how increasing cache hit ratio reduces both p50 and p95 latency. The relationship is
non-linear: small improvements in hit ratio at already-high levels can still reduce tail latency, because the
tail is dominated by origin trips and queueing.

Figure 3 illustrates a conceptual trade-off between TTL and staleness risk. Longer TTL increases the
chance that a client observes outdated business state, but validation (ETag revalidation) can reduce the
risk by allowing the cache to confirm freshness with a low-cost 304 response.

Figure 4 shows an illustrative reduction in origin cost as hit ratio increases. Although absolute numbers
depend on the platform and workload, the direction is robust: origin costs scale primarily with misses.

The most important practical message is that teams should not chase hit ratio blindly. Hit ratio must be
improved only through safe key design and correct freshness models; otherwise, the "performance win"
becomes a latent correctness incident.

10. Decision matrix and checklist

Decision matrix and checklist

12
The following decision matrix helps teams apply the framework quickly:

Step 1: Determine tier (A/B/C/D) based on personalization and state-transition sensitivity. Step 2: Identify
representation variance: path, query params, and candidate Vary headers. Step 3: Choose freshness
model (TTL-only, validation, or purge-based) and define SLAs for staleness. Step 4: Decide caching
layer: - Edge only (simple, globally distributed, best for Tier A). - Gateway only (fine control, best for Tier
B or edge-disabled environments). - Both (tiered caching): edge handles long-lived public content;
gateway handles short-lived partitioned content. Step 5: Define operational controls: purge APIs, bypass
switch, dashboards, alerts, and incident runbooks. Step 6: Run a staged rollout: shadow cache (record
keys, do not serve), then serve to 1%, 10%, 50%, 100% with correctness checks.

We strongly recommend an explicit "cache contract" per endpoint: documented cache key components,
Vary headers, Cache-Control directives, TTL/validation, and a named owner.

11. Conclusion
Conclusion
Edge caching for APIs can deliver large improvements in latency and origin scalability, but only when it is
treated as a correctness-first design. The proposed framework provides a repeatable way to classify
endpoints, design cache keys, choose freshness models, and operate caches safely. Most importantly, it
encourages teams to treat caching behavior as observable and controllable, with explicit purge and
bypass mechanisms. By aligning configuration with HTTP caching semantics (RFC 9111) and by
measuring outcomes with realistic workloads, teams can capture performance benefits without
introducing subtle security or consistency failures. [1].

References
1. Fielding, R., Nottingham, M., & Reschke, J. (2022). RFC 9111: HTTP Caching. RFC Editor / IETF.

2. Cloudflare. (2025). Default cache behavior. Cloudflare Developers documentation.

3. Kong Inc. (n.d.). Proxy Cache plugin documentation. Kong Gateway plugin docs.

4. Microsoft. (n.d.). Caching with Azure Front Door and query string behavior. Microsoft Learn.

5. Mozilla. (n.d.). HTTP caching guide. MDN Web Docs.

13