HTTP Caching, Cookies, and Proxies
Between a browser and your application sit layers that change what each request and response look like: the browser's own cache, corporate proxies, CDNs, load balancers, and reverse proxies. Most of the time they make the web fast. When something goes wrong — a user sees another user's dashboard, a deploy is invisible for an hour, a login cookie is silently dropped, the application logs every client as the same IP — the cause is usually a misunderstanding of how HTTP caching, cookies, and intermediaries interact.
This page explains the headers that control each, how they combine, and how to trace a request through the layers to find out which one is responsible.
TL;DR
Cache-Controldecides who may cache a response and for how long:max-age,s-maxage,no-cache,no-store,private,public,immutable.no-cachemeans "revalidate", not "do not cache";no-storemeans do not store at all.- Validators (
ETag,Last-Modified) let caches revalidate cheaply with304 Not Modified. Varytells caches which request headers change the response; get it wrong and caches serve the wrong variant.- Cookies need
Secure,HttpOnly, and an appropriateSameSite; responses that set or depend on cookies should beprivateor uncached in shared caches. - Proxies add
X-Forwarded-Forand related headers; trust them only from known proxies.
Quick Example
Response headers for three kinds of resources:
A cheap revalidation of the HTML page:
A session cookie that is only sent over HTTPS, hidden from JavaScript, and withheld from most cross-site requests:
Seeing what each layer did:
Core Concepts
Freshness and Cache-Control
Responses without explicit directives may be cached heuristically based on Last-Modified — a common cause of mysterious staleness. Always set Cache-Control explicitly.
Validation
When a cached response is stale (or marked no-cache), the cache sends a conditional request: If-None-Match with the stored ETag, or If-Modified-Since with Last-Modified. If nothing changed, the server replies 304 Not Modified with no body, saving bandwidth. Strong ETags change whenever bytes change; weak ETags (W/"...") indicate semantic equivalence.
Vary
Vary lists request headers that select between response variants — Accept-Encoding for compressed versus uncompressed bodies, Accept-Language for translations. A missing Vary serves one variant to everyone; Vary: Cookie or Vary: User-Agent fragments caches so badly they rarely hit. Prefer separate URLs or private for per-user variation.
Shared and Private Caches
Anything personalized must not reach a shared cache. See CDN and Nginx Caching.
Cookies
Attributes
Browsers limit cookie size (about 4 KB each) and count, and cookies are sent on every matching request, so keep them small. See Session Management and Cookie Security and CSRF.
Cookies and Caching Together
A response that sets a cookie, or whose content depends on one, must not be served from a shared cache to other users. Mark such responses private or no-store, keep session cookies off static asset paths (or serve assets from a separate cookieless domain), and configure CDNs to bypass caching when session cookies are present.
Proxies and Intermediaries
Proxies add headers describing the original request: X-Forwarded-For (client IP chain), X-Forwarded-Proto (original scheme), X-Forwarded-Host, or the standard Forwarded header, plus Via. Applications must trust these only from known proxy addresses — otherwise any client can spoof its IP address or claim HTTPS. See Nginx Reverse Proxy.
Tracing Through Layers
- Send a request ID or
traceparentfrom the edge and log it at every layer. - Read cache status headers:
Age(seconds in cache),Cache-Status(standardized), and vendor headers such asX-Cache. - Compare responses fetched directly from the origin and through the CDN to locate the layer that changed them.
- Use browser developer tools' Network panel to see whether a response came from memory, disk, or the network.
Best Practices
Fingerprint Static Assets
Put a content hash in asset file names and serve them with a one-year max-age and immutable; deploy new names instead of invalidating caches.
Make HTML and APIs Explicit
Give HTML no-cache with validators (or short max-age with stale-while-revalidate), and give authenticated API responses private or no-store.
Use Strict Cookie Defaults
Secure, HttpOnly, SameSite=Lax, and the __Host- prefix for session cookies; loosen only with a specific reason.
Configure Trusted Proxies Explicitly
Tell frameworks which proxy addresses to trust for forwarded headers, and strip incoming forwarded headers at the edge.
Common Mistakes
Caching Personalized Pages at the CDN
Misreading no-cache
Trusting Any X-Forwarded-For
FAQ
What is the difference between no-cache and no-store?
no-cache allows caches to store a response but requires revalidation with the server before every reuse. no-store forbids storing the response at all, which is appropriate for sensitive data.
How do ETags work?
The server sends an ETag identifying the version of a resource. On later requests the cache sends If-None-Match with that value, and the server replies 304 Not Modified if the resource is unchanged, avoiding resending the body.
What does SameSite do on cookies?
It controls whether cookies are sent with cross-site requests. Lax (the default in modern browsers) withholds cookies on cross-site subrequests but sends them on top-level navigation; Strict never sends them cross-site; None always sends them, and requires Secure.
Why is my site serving old content after a deploy?
Common causes are long max-age on HTML, heuristic caching due to missing Cache-Control, CDN caches that were not purged, or assets without fingerprinted names. Check Cache-Control, Age, and cache status headers at each layer.
What is X-Forwarded-For?
A header added by proxies listing the client IP address and intermediate proxies. Applications behind proxies use it to find the original client IP but must only trust values added by their own proxies.
Related Topics
- HTTP — Protocol fundamentals
- CDN — Edge caching
- Session Management and Cookie Security — Secure sessions
- Nginx Caching — Reverse proxy caching
- CSRF — Cross-site request forgery and SameSite
- Cache Invalidation and Consistency — Keeping cached content correct