HTTP Caching, Cookies, and Proxies

Between a browser and your application sit layers that change what each request and response look like: the browser's own cache, corporate proxies, CDNs, load balancers, and reverse proxies. Most of the time they make the web fast. When something goes wrong — a user sees another user's dashboard, a deploy is invisible for an hour, a login cookie is silently dropped, the application logs every client as the same IP — the cause is usually a misunderstanding of how HTTP caching, cookies, and intermediaries interact.

This page explains the headers that control each, how they combine, and how to trace a request through the layers to find out which one is responsible.

TL;DR

Quick Example

Response headers for three kinds of resources:

A cheap revalidation of the HTML page:

A session cookie that is only sent over HTTPS, hidden from JavaScript, and withheld from most cross-site requests:

Seeing what each layer did:

Core Concepts

Freshness and Cache-Control

Responses without explicit directives may be cached heuristically based on Last-Modified — a common cause of mysterious staleness. Always set Cache-Control explicitly.

Validation

When a cached response is stale (or marked no-cache), the cache sends a conditional request: If-None-Match with the stored ETag, or If-Modified-Since with Last-Modified. If nothing changed, the server replies 304 Not Modified with no body, saving bandwidth. Strong ETags change whenever bytes change; weak ETags (W/"...") indicate semantic equivalence.

Vary

Vary lists request headers that select between response variants — Accept-Encoding for compressed versus uncompressed bodies, Accept-Language for translations. A missing Vary serves one variant to everyone; Vary: Cookie or Vary: User-Agent fragments caches so badly they rarely hit. Prefer separate URLs or private for per-user variation.

Shared and Private Caches

Anything personalized must not reach a shared cache. See CDN and Nginx Caching.

Cookies

Attributes

Browsers limit cookie size (about 4 KB each) and count, and cookies are sent on every matching request, so keep them small. See Session Management and Cookie Security and CSRF.

Cookies and Caching Together

A response that sets a cookie, or whose content depends on one, must not be served from a shared cache to other users. Mark such responses private or no-store, keep session cookies off static asset paths (or serve assets from a separate cookieless domain), and configure CDNs to bypass caching when session cookies are present.

Proxies and Intermediaries

Proxies add headers describing the original request: X-Forwarded-For (client IP chain), X-Forwarded-Proto (original scheme), X-Forwarded-Host, or the standard Forwarded header, plus Via. Applications must trust these only from known proxy addresses — otherwise any client can spoof its IP address or claim HTTPS. See Nginx Reverse Proxy.

Tracing Through Layers

Best Practices

Fingerprint Static Assets

Put a content hash in asset file names and serve them with a one-year max-age and immutable; deploy new names instead of invalidating caches.

Make HTML and APIs Explicit

Give HTML no-cache with validators (or short max-age with stale-while-revalidate), and give authenticated API responses private or no-store.

Use Strict Cookie Defaults

Secure, HttpOnly, SameSite=Lax, and the __Host- prefix for session cookies; loosen only with a specific reason.

Configure Trusted Proxies Explicitly

Tell frameworks which proxy addresses to trust for forwarded headers, and strip incoming forwarded headers at the edge.

Common Mistakes

Caching Personalized Pages at the CDN

Misreading no-cache

Trusting Any X-Forwarded-For

FAQ

What is the difference between no-cache and no-store?

no-cache allows caches to store a response but requires revalidation with the server before every reuse. no-store forbids storing the response at all, which is appropriate for sensitive data.

How do ETags work?

The server sends an ETag identifying the version of a resource. On later requests the cache sends If-None-Match with that value, and the server replies 304 Not Modified if the resource is unchanged, avoiding resending the body.

What does SameSite do on cookies?

It controls whether cookies are sent with cross-site requests. Lax (the default in modern browsers) withholds cookies on cross-site subrequests but sends them on top-level navigation; Strict never sends them cross-site; None always sends them, and requires Secure.

Why is my site serving old content after a deploy?

Common causes are long max-age on HTML, heuristic caching due to missing Cache-Control, CDN caches that were not purged, or assets without fingerprinted names. Check Cache-Control, Age, and cache status headers at each layer.

What is X-Forwarded-For?

A header added by proxies listing the client IP address and intermediate proxies. Applications behind proxies use it to find the original client IP but must only trust values added by their own proxies.

Related Topics

References