Edge Caching: Varnish vs Gateway-Native Cache

Every response your gateway can serve from memory is a response your origin never sees — but the moment you introduce a cache you also introduce the two hardest problems in the field: getting the cache key right so no user ever sees another user’s data, and invalidating entries fast enough that nobody sees stale data past its useful life. The architectural fork in this decision is whether to run a dedicated Varnish tier as a purpose-built HTTP cache, or to lean on the caching that your gateway already ships — Kong 3.x’s proxy-cache plugin or the Envoy 1.32+ response cache filter. This page compares the two along the dimensions that actually break in production: cache-key construction, Vary handling, invalidation reach, stale-while-revalidate semantics, and where each sits in the request topology.

The scenario and what is at stake

You have an API tier where a meaningful fraction of traffic is cacheable — product catalogs, pricing tables, configuration documents, reference data that changes on the order of minutes, not milliseconds. Origin latency is 80–300 ms and origin capacity is the scaling ceiling. A cache in front of the origin can absorb 70–95% of that read traffic, but only if it can distinguish requests that are genuinely equivalent from requests that merely look similar. Get the equivalence wrong and you leak data across tenants; get invalidation wrong and you serve a deleted price for an hour. Both failure modes are worse than the origin load you were trying to shed.

Prerequisite concepts

This comparison assumes you understand the mechanics covered in caching and response optimization — cacheability rules, Cache-Control directives, and the difference between private and shared caches — and how a cache sits inside the broader middleware chain relative to authentication and transformation. Because a cache lookup must happen after the gateway has authenticated the consumer but before it burns an origin round-trip, its position in the plugin ordering matters as much as its configuration; the authentication proxying and token validation layer must run first so the cache key can incorporate a verified identity rather than a raw, untrusted token.

Topology: where the cache sits

The single most consequential decision is not which cache you pick but where it sits relative to the gateway. The diagram below contrasts the two dominant layouts.

In layout A, the gateway does the security work — terminates TLS, validates the token, strips credentials, and rewrites the request into a canonical form — and only then hands a clean, cache-friendly request to Varnish. Varnish becomes a shared cache pool that every gateway node consults over the network. In layout B, the cache lives inside the gateway process itself, so a hit never leaves the box, but each gateway node keeps its own independent store and there is no shared hit ratio across the fleet.

Cache-key construction

A cache key is the fingerprint that decides whether two requests are the same request. The default in every system is method plus scheme plus host plus path plus query string, and that default is wrong the moment authentication or content negotiation enters the picture.

Varnish builds the key in VCL. You have full imperative control, which is both the strength and the hazard:

# Varnish 7.x — vcl_hash: identity-scoped cache key for an authenticated API
sub vcl_hash {
    hash_data(req.url);
    hash_data(req.http.host);

    # The gateway has already validated the token and injected a
    # normalized, trusted consumer id header. Key on that, never on
    # the raw Authorization header.
    if (req.http.X-Consumer-Id) {
        hash_data(req.http.X-Consumer-Id);
    }

    # Normalize Accept-Encoding to gzip|identity so we do not fragment
    # the cache across a dozen browser encoding permutations.
    if (req.http.Accept-Encoding ~ "gzip") {
        hash_data("gzip");
    } else {
        hash_data("identity");
    }
    return (lookup);
}

Kong 3.x builds the key declaratively. You cannot write arbitrary logic; you choose which request dimensions feed the key, which is safer by default but less flexible:

# Kong 3.x — proxy-cache plugin scoped to a consumer, keyed on selected headers
plugins:
  - name: proxy-cache
    config:
      response_code: [200, 301, 404]
      request_method: ["GET", "HEAD"]
      content_type: ["application/json"]
      cache_ttl: 120
      cache_control: true          # honor upstream Cache-Control over cache_ttl
      vary_headers: ["Accept-Encoding"]
      vary_query_params: ["page", "limit"]
      strategy: memory

Kong’s vary_headers and vary_query_params are an allowlist: only the query parameters you name participate in the key, so a tracking parameter like utm_source cannot fragment the cache. Attaching the plugin to a consumer rather than globally scopes each cache namespace to that consumer, which is Kong’s answer to per-identity keying. The critical rule for both systems is the same one covered under caching and response optimization: a response that depends on the caller’s identity must carry that identity in the key, or the cache becomes a data-leak vector.

Vary handling

Vary is the origin’s way of telling the cache “this response was negotiated on these request headers.” A shared cache that ignores Vary will happily serve a gzip-encoded body to a client that cannot decode it, or a French response to an English speaker.

Varnish, by default, respects beresp.http.Vary but caches a separate object per distinct value of each varying header — which means a header with high cardinality (a raw User-Agent, for example) explodes into thousands of near-duplicate objects and destroys your hit ratio. The standard remedy is to normalize the varying header in vcl_recv down to a handful of canonical buckets before the object is ever stored:

# Varnish 7.x — normalize Accept-Language to two buckets before caching
sub vcl_recv {
    if (req.http.Accept-Language ~ "^\s*fr") {
        set req.http.Accept-Language = "fr";
    } else {
        set req.http.Accept-Language = "en";
    }
}

Envoy’s response cache filter honors Vary from the upstream and includes the named request headers in its internal lookup key automatically; it does not give you a normalization hook, so if the origin emits Vary: Accept-Encoding you must ensure the gateway upstream sends a normalized Accept-Encoding before the filter runs. Kong’s proxy-cache derives its varying dimensions from the explicit vary_headers list rather than trusting the response Vary header, which trades correctness-by-default for predictability — you decide the cache dimensions rather than the origin.

Stale-while-revalidate

Serving a slightly stale object while asynchronously refreshing it is the difference between a cache that shields the origin during a traffic spike and one that stampedes the origin the instant a popular key expires.

Varnish implements this through grace mode. An object past its TTL but within its grace window is served immediately to the client while Varnish fetches a fresh copy in the background:

# Varnish 7.x — serve stale up to 30s past TTL during async refresh
sub vcl_backend_response {
    set beresp.ttl = 120s;
    set beresp.grace = 30s;   # stale-while-revalidate window
}

Envoy 1.32+ honors the stale-while-revalidate response directive natively in the response cache filter, so an origin that emits Cache-Control: max-age=120, stale-while-revalidate=30 gets the same behavior without any Envoy-side configuration:

# Envoy 1.32+ — response cache filter in the HTTP filter chain
http_filters:
  - name: envoy.filters.http.cache
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.cache.v3.CacheConfig
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.http.cache.simple_http_cache.v3.SimpleHttpCache
  - name: envoy.filters.http.router
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

Kong’s proxy-cache has no asynchronous background revalidation. When an entry’s cache_ttl expires, the next request is a synchronous miss that blocks on the origin, and under a hot key that means a brief thundering herd against the upstream. If your workload has hot keys and you are running Kong, you either accept that herd, front Kong with a cache that does support grace, or pair it with the retry and concurrency controls described in circuit breaking and retry budgets so a synchronous refresh spike cannot cascade.

Invalidation

Every cache instance holds an independent store, so invalidation is fundamentally a fan-out problem: a purge must reach every node that might hold the entry.

Varnish has the richest toolkit. A PURGE request removes a single exact object; a BAN invalidates every object matching an expression (for example, all objects whose URL starts with /products/); and a soft purge marks objects stale rather than removing them, so grace mode can still serve them during the refetch. Each cache node must receive the purge independently, typically through a small fan-out service or a message bus:

# Varnish — targeted purge and pattern ban, issued per cache node
curl -X PURGE -H "Host: api.example.com" http://varnish-node-1:6081/products/42
curl -X BAN  -H "X-Ban-Url: ^/products/" http://varnish-node-1:6081/

Kong exposes cache management on its Admin API: you can delete a specific cached entity by its key or purge the entire proxy-cache store, and because the Admin API is per-node in traditional deployments you issue the delete against each data-plane node. Envoy’s simple HTTP cache has no push-based purge mechanism at all — the only way to evict is to let the TTL expire or to force revalidation with a no-cache request. In practice that makes short TTLs plus stale-while-revalidate the entire Envoy invalidation strategy, which is fine for time-bounded staleness but unusable when you need immediate, event-driven eviction.

Decision matrix

Choose a dedicated Varnish tier when you have a high volume of cacheable content, need pattern-based or event-driven invalidation, and can justify running and monitoring a separate cache fleet. Choose the gateway-native cache when the cacheable fraction is modest, staleness is acceptable within a TTL, and the operational simplicity of one fewer moving part outweighs Varnish’s richer feature set.

Gotchas and failure signals

Caching authenticated responses on the raw token. If the cache key includes Authorization verbatim, every token rotation misses the cache and — far worse — a shared cache that strips Authorization from the key serves one user’s private response to another. Always key on a gateway-validated, normalized consumer identity, never the credential itself.

Silent Vary fragmentation. A response that varies on an unnormalized high-cardinality header (User-Agent, a full Accept-Language) produces a hit ratio near zero even though the cache is “working.” The failure signal is a cache-hit-rate metric that stays low while cache memory fills — normalize the varying header into buckets.

Kong hot-key herd on expiry. Because Kong has no background revalidation, a popular key expiring under load sends a burst of concurrent misses to the origin. The signal is a periodic origin latency spike synchronized to your cache_ttl interval.

Envoy purge expectations. Teams migrating from Varnish assume they can purge Envoy’s cache on a content-change event; they cannot. Design for TTL-bounded staleness from the start rather than discovering the missing purge path during an incident.

Caching error responses. Accidentally caching a 404 or 500 with a long TTL turns a transient origin blip into a sustained outage. Restrict cacheable status codes explicitly (Kong’s response_code allowlist) and keep negative-cache TTLs short.

Validation

• Cache key includes a gateway-validated consumer identity for any response that varies per user — never the raw Authorization header.
• Every header the origin negotiates on (Accept-Encoding, Accept-Language) is either in the cache key or normalized to canonical buckets before lookup.
• Cacheable status codes are an explicit allowlist; error responses have short or zero TTL.
• Vary behavior verified end-to-end by requesting the same URL with two encodings and confirming distinct cached objects.
• Invalidation path tested: a purge (Varnish PURGE/BAN, Kong Admin delete) reaches every cache node, verified by cache-miss after purge.
• Stale-while-revalidate window sized so a hot key never blocks on a synchronous origin fetch (or Kong’s synchronous-miss herd is accepted and bounded).
• Cache-hit-ratio and origin-request-rate metrics dashboarded; low hit ratio with rising memory flags Vary fragmentation.

FAQ

Should I put Varnish in front of or behind my API gateway?

Put Varnish behind the gateway when caching decisions depend on authentication or per-consumer policy that only the gateway can evaluate, because the gateway must strip credentials and normalize the cache key before Varnish sees the request. Put Varnish in front only for fully anonymous, public content where TLS termination and coarse routing happen at Varnish and the gateway handles the bulk of dynamic traffic. For most authenticated APIs the gateway’s native cache is simpler and avoids a second hop entirely.

Why does my cache serve the wrong response to different users?

This is almost always a Vary or cache-key problem. If the origin returns content negotiated on Accept-Encoding, Accept-Language, or Authorization but the cache key does not include those dimensions, one user’s response is served to another. Either add the varying header to the cache key explicitly, or normalize the header to a small set of canonical values before the lookup. Never cache a response that varies on Authorization without including the consumer identity in the key.

How do I invalidate a cached response immediately across all edge nodes?

Varnish supports targeted invalidation through PURGE and BAN commands issued to each cache node, and soft purge lets you serve stale content while a fresh copy is fetched. Kong’s proxy-cache plugin exposes a delete endpoint on the Admin API keyed by cache entity, and Envoy’s response cache honors no-cache revalidation but has no push-based purge, so short TTLs with stale-while-revalidate are the practical invalidation strategy. In all cases, invalidation must fan out to every cache instance because each holds an independent store.

Does stale-while-revalidate work the same in Varnish and gateway caches?

The intent is the same but the mechanism differs. Varnish implements it through beresp.grace and stale-while-revalidate semantics that serve an expired object while an asynchronous backend fetch refreshes it. Envoy’s response cache filter honors the stale-while-revalidate Cache-Control directive directly. Kong’s proxy-cache does not implement asynchronous background revalidation, so a stale window in Kong means serving from cache until TTL expiry and then blocking the first request on a synchronous origin fetch.

Related

• Caching & Response Optimization — cacheability rules, Cache-Control semantics, and shared-cache safety that underpin every choice on this page.
• Circuit Breaking & Retry Budgets — bounding the origin herd when a hot cache key expires under load.
• Authentication Proxying & Token Validation — validating identity before the cache key is built so per-consumer caching is safe.

← Caching & Response Optimization