Garbage Collection: Tracing, Generational GC & Pause Times

Garbage Collection

Garbage collection removes an entire class of bug — use-after-free, double-free, and most memory leaks — by making the runtime responsible for reclaiming memory nothing can reach. The price is paid in throughput (the collector uses CPU), latency (collection pauses), and memory footprint (collectors need headroom to work efficiently). Every GC design is a different position on those three axes, and you can't optimize all of them.

The idea that makes modern collectors work is the generational hypothesis: most objects die young. It holds so consistently across real programs — typically 80–95% of allocations become garbage almost immediately — that essentially every production collector is built around it. Collect the young generation frequently and cheaply, promote the survivors, and touch the old generation rarely.

For most engineers the practical relevance isn't building a collector but diagnosing one: understanding why a service has a 400ms pause every few minutes, why raising the heap size made latency worse, and why the allocation rate matters more than the live set.

TL;DR

GC reclaims memory that is unreachable, not memory that is "unused" — reachability is the definition.
Tracing collectors walk from roots and reclaim what they don't reach; reference counting frees at zero.
Reference counting has immediate reclamation and cannot free cycles without a backup tracer.
The generational hypothesis — most objects die young — is why nursery collections are cheap.
Write barriers track old→young pointers so a young collection doesn't have to scan the old generation.
Throughput, latency, footprint: pick two. Every collector is a position on this triangle.
Modern low-latency collectors (ZGC, Shenandoah, Go's) hold sub-millisecond pauses at a throughput cost.
The most effective GC tuning is usually allocating less.

Core Concepts

Reachability

"Garbage" means unreachable from any root, and that's the whole definition. Note the consequence: an object you'll never use again but that is still referenced from a live collection is not garbage. That's why a cache with no eviction, a growing static list, or a forgotten event listener is a memory leak in a garbage-collected language — the collector is working correctly and you told it the object was still needed.

Tracing vs. reference counting

CPython uses reference counting for immediate reclamation plus a generational tracing collector specifically to break cycles. Swift uses reference counting with no cycle collector at all, which is why weak and unowned are part of the language you must actively use — a strong reference cycle in Swift is a genuine leak.

Mark-and-sweep

Simple and non-moving, which means pointers stay valid — important when native code holds references. The problem is fragmentation: free space exists but is scattered, so a large allocation can fail on a heap with plenty of total free memory. Mark-compact adds a phase that slides live objects together, fixing fragmentation at the cost of a longer pause and updating every pointer.

Copying collection

This is the key insight behind nursery collection. Because copying cost scales with survivors rather than with garbage, and because most young objects die, a young-generation collection is extremely cheap. Allocation also becomes a pointer increment plus a bounds check — often faster than malloc.

The cost is memory: a classic semi-space collector needs twice the space for the region it manages.

Generational collection

The complication: an old object may point to a young one, so young collection can't just scan the roots — it would miss that reference and free a live object. Scanning the whole old generation would defeat the purpose.

Write barriers

Write barriers are why generational GC works and why it isn't free: every pointer assignment in the program pays a small cost. They're also the mechanism behind concurrent collectors, which need to know about mutations happening while marking is in progress (via a snapshot-at-the-beginning or incremental-update barrier).

Concurrent and incremental collection

Modern low-latency collectors do the bulk of marking and even relocation concurrently with the running program, pausing only for short "safepoints" to snapshot roots and flip references.

Go's design is a deliberate outlier: no generational collection and no compaction, in exchange for simplicity and very short pauses, with the tradeoff appearing as higher allocation cost and heap fragmentation. Its GOGC knob is unusually blunt and unusually easy to reason about.

Tuning and Diagnosis

The tradeoff triangle

The counterintuitive one: increasing the heap improves throughput and can make latency worse, because collections are rarer but each one has more to do. For a latency-sensitive service, a larger nursery with a modest old generation often beats simply raising the total.

What to measure

A high promotion rate is the strongest signal that something is wrong: it means objects are surviving the nursery, which usually indicates either a too-small nursery or genuinely long-lived allocation. Both are fixable, and both cause expensive old-generation collections downstream.

Reducing GC pressure

Object pooling is the technique most often reached for and most often counterproductive — pooled objects survive the nursery, get promoted, and turn cheap young collections into expensive old-generation work. Pool only large, genuinely expensive objects (buffers, connections), and measure.

Diagnosing a leak

A "leak" in a managed language is always unintended retention — something reachable that shouldn't be:

Tools: Eclipse MAT or VisualVM for the JVM, pprof for Go, Chrome DevTools heap snapshots for JavaScript, tracemalloc and objgraph for Python.

Best Practices

Allocate less before tuning anything

The most effective GC intervention is reducing allocation rate. Fewer objects means fewer collections, less promotion, and shorter pauses — and it usually improves cache behavior too. Profile allocation before touching a single flag.

Set an explicit heap size in production

Defaults derived from container memory are frequently wrong, and an unbounded heap on a container with a hard memory limit gets OOM-killed rather than collecting. Set maximum heap explicitly and leave headroom for non-heap memory (thread stacks, metaspace, native buffers).

Choose the collector to match the workload

Batch and throughput work wants a throughput-oriented collector and a large heap. A latency-sensitive service wants a concurrent, low-pause collector even at a throughput cost. Using the default for a workload it wasn't designed for is a common and easily fixed problem.

Bound every cache

An unbounded cache is a memory leak with a nicer name. Use an LRU with a size or weight limit, or a TTL. Weak and soft references are sometimes proposed as a solution and behave unpredictably under memory pressure — an explicit bound is more reliable.

Watch promotion rate, not just pause time

Rising promotion is the leading indicator of degrading GC behavior. It's visible before pauses get bad and it's usually cheaper to fix at that stage — often just a larger nursery or one allocation site.

Be careful with finalizers

Java finalizers are deprecated for good reason: they delay reclamation by at least one collection cycle, run on an unpredictable thread at an unpredictable time, and can resurrect objects. Use explicit close methods with try-with-resources, Cleaner, or the equivalent in your language.

Load-test long enough to reach steady state

GC behavior in the first minute is unrepresentative — the heap hasn't filled, the old generation hasn't been collected, and no compaction has occurred. Run for long enough to see several old-generation cycles before drawing conclusions.

Understand what the numbers mean before changing flags

GC tuning by trying flags is a well-known way to spend a week and make things worse. Read the collection logs, identify whether the problem is allocation rate, live set size, or promotion, and change one thing at a time.

Common Mistakes

Assuming GC prevents memory leaks

Object pooling as a reflex

Calling the collector manually

Tuning by flag roulette

Ignoring the write barrier cost

Not testing at production heap size

FAQ

Is garbage collection slower than manual memory management?

For throughput, often comparable and sometimes better — bump allocation in a nursery is faster than malloc, and batch reclamation amortizes well. Where GC loses is predictability: pauses, and a memory footprint typically 2–5× the live set for good performance. Manual management (C, C++) and compile-time management (Rust's ownership) give deterministic behavior and tighter memory, at the cost of a bug class GC eliminates entirely.

Why does my service pause for 400ms?

Almost always a full or old-generation collection with a large live set, and the usual root cause is a high promotion rate — objects surviving the nursery when they shouldn't. Check GC logs for the collection type and the live set size. Fixes in order of typical effectiveness: reduce allocation, enlarge the nursery, bound your caches, and switch to a concurrent collector.

Should I use a low-pause collector like ZGC?

If p99 latency matters more than throughput, and your heap is large. ZGC and Shenandoah hold sub-millisecond pauses on very large heaps at a throughput cost of roughly 10–15%. For batch processing, a throughput collector with longer pauses does more total work. Measure your actual pause distribution before switching — many services assume they have a GC latency problem and don't.

How does Rust avoid needing GC?

Ownership and borrowing determine object lifetimes at compile time, so the compiler inserts deallocation at the point where a value goes out of scope. No runtime collector, no pauses, and deterministic memory use. The cost is a substantially steeper learning curve and some data structures (graphs, doubly-linked lists) needing explicit Rc/RefCell — which is reference counting, with its cycle problem. See Rust.

Why is Go's collector not generational?

A deliberate design choice. Go's escape analysis puts many short-lived objects on the stack rather than the heap, reducing the nursery's benefit, and its value types mean less pointer-heavy allocation than typical Java. Combined with a preference for implementation simplicity and very short pauses, a concurrent non-generational mark-sweep was judged the better tradeoff. Generational Go has been explored repeatedly and hasn't shipped.

What is a write barrier costing me?

A few instructions on every pointer store — typically a card-table write or a check-and-record. For most programs it's a small single-digit percentage. For code doing enormous numbers of pointer writes in a tight loop it can be significant, which is one reason primitive arrays and value types outperform arrays of object references by more than the memory saving alone explains.