JIT Compilation

A just-in-time compiler translates code to machine code while the program is running. Stated that way it sounds strictly worse than compiling ahead of time — you pay the compilation cost during execution, and you have less time to spend optimizing. The reason JITs win anyway, often decisively, on dynamic languages is that they know things a static compiler cannot: which branches actually get taken, what types actually flow through a function, which call sites are monomorphic, and which code is hot enough to be worth optimizing at all.

That knowledge licenses speculation. If a function has been called ten thousand times and x has been an integer every time, the JIT compiles a version that assumes x is an integer — no type check, no boxing, just an add instruction. It guards the assumption cheaply, and if the guard ever fails it deoptimizes: discards the specialized code, reconstructs the interpreter's state, and continues. Being able to bail out safely is what makes aggressive speculation possible.

The architecture that follows is tiering: interpret first (fast startup, no compilation cost), profile, compile hot code at a low optimization level, and recompile the hottest code aggressively.

TL;DR

Core Concepts

Tiered execution

Compilation is expensive, so you only want to pay it where it returns. Most code in a program runs a handful of times and never justifies optimization; a small fraction runs constantly and justifies a great deal. Tiering is how a runtime spends its compilation budget where it matters.

V8 uses Ignition (interpreter) → Sparkplug (baseline) → Maglev (mid-tier) → TurboFan (optimizing). HotSpot uses the interpreter → C1 → C2. The shape is consistent across engines.

Profiling

This is exactly the information static compilation lacks. A C compiler can be told about likely branches with __builtin_expect or profile-guided optimization from a training run; a JIT gets it for free from the actual workload, and it updates as the workload changes.

Speculation and guards

The guards are cheap — a tag check is a compare-and-branch the predictor learns immediately. The payoff is that the body collapses from a dozen branches and a possible allocation into a single arithmetic instruction.

Speculation is only safe because deoptimization exists. Without a way to bail out, the compiler could only emit code valid for every possible input, which is precisely the generic slow path it's trying to avoid.

Deoptimization

The deopt map is metadata the optimizing compiler must maintain through every transformation — which constrains what optimizations are legal and is a large part of why optimizing JITs are complex. The reward is that the compiler may assume anything it can guard.

Deoptimization loops are the pathological case: code deoptimizes, gets recompiled with the same bad assumption, deoptimizes again. Runtimes detect repeated deopts at a site and either widen the assumption or give up and stay in the baseline tier.

Inlining

The most valuable optimization in a JIT, because it's an enabler for all the others.

JITs inline far more aggressively than static compilers because profile data tells them which call sites are monomorphic — always calling the same target. A JavaScript call through a polymorphic reference is undecidable statically and trivially decidable after ten thousand observations.

The limits are code size (inlining explodes the compiled footprint and hurts instruction cache), compile time, and megamorphic call sites where no single target dominates.

Inline caches and hidden classes

This is why property access order matters in JavaScript: creating objects with fields in different orders produces different shapes, turning a monomorphic site megamorphic and costing an order of magnitude.

On-stack replacement

Tiering normally promotes a function on its next invocation — useless for a long-running loop that will never be called again. OSR compiles the loop body and transfers execution into the compiled version mid-iteration, mapping the interpreter's live values into the compiled frame. It's what makes a single hot loop benefit from the JIT at all.

JIT vs. AOT

This is why serverless and CLI tools favor AOT — a function invoked for 200ms never reaches peak performance, and warmup is pure cost. It's why long-running servers favor JIT. And it's why the JVM ecosystem produced GraalVM Native Image and the .NET ecosystem produced ReadyToRun and Native AOT: the same runtime, compiled ahead of time, for workloads where startup dominates.

Profile-guided optimization narrows the gap from the AOT side: run a training workload, record a profile, compile with it. It's a real improvement and it's a static snapshot — it can't adapt when production behavior differs from training, which is a recurring practical problem.

Best Practices

Write monomorphic code

Keep object shapes consistent — same fields, same order, initialized at construction. Pass consistent types to hot functions. A megamorphic call site or property access is often 10× slower than a monomorphic one, and it's usually a small refactor to fix.

Don't fight the JIT with micro-optimizations

Manual loop unrolling, avoiding function calls, and clever bit tricks in a JIT-compiled language usually make things worse — they defeat pattern recognition the optimizer relies on and confuse inlining heuristics. Write clear, consistent code and let the compiler work.

Warm up before measuring

A benchmark that measures the first thousand iterations is measuring the interpreter and the baseline tier. Use a proper harness (JMH for the JVM, tinybench or similar for Node.js) that discards warmup iterations and detects when performance has stabilized.

Watch for deoptimization in production

Runtimes expose this (--trace-deopt in V8, -XX:+PrintDeoptimization on HotSpot). Repeated deopts at a hot site mean the JIT keeps making an assumption your data violates — often a rare null, an occasional string where numbers are expected, or an inconsistent object shape.

Choose AOT for short-lived processes

CLI tools, serverless functions, and anything with a lifetime measured in hundreds of milliseconds never amortize warmup. GraalVM Native Image, .NET Native AOT, and shipping a compiled binary are the right answer there, and the throughput loss is irrelevant at that timescale.

Keep hot methods small enough to inline

Inlining budgets are size-based. A 500-line method won't be inlined into its callers, and everything downstream of that boundary loses optimization opportunities. Extracting the hot path into a small method is sometimes a large win.

Give the runtime enough memory headroom

Compiled code, profile data, and the compiler itself all consume memory. A container sized tightly against the live heap will thrash the code cache and force recompilation. Check code cache metrics if throughput degrades over time.

Understand tiering before tuning flags

Compilation thresholds, tier counts, and inlining budgets are all tunable and rarely should be. The defaults encode a great deal of measurement across many workloads; changing them without understanding which tier is actually the problem reliably makes things worse.

Common Mistakes

Benchmarking without warmup

Creating inconsistent object shapes

Passing mixed types to a hot function

Fighting the optimizer

Ignoring warmup in serverless

Assuming JIT means "always fast"

FAQ

Why is a JIT faster than AOT on dynamic languages?

Because the interesting facts aren't knowable statically. In JavaScript, a + b could be numeric addition, string concatenation, or an arbitrary valueOf call — a static compiler must emit code handling all of it. A JIT observes ten thousand executions where both were integers and emits one add instruction with a guard. That kind of specialization is unavailable ahead of time, and it's worth an order of magnitude on hot code.

What actually is warmup?

The period before the optimizing tier has compiled the hot paths: interpreting, collecting profiles, compiling at the baseline tier, and eventually recompiling. It's typically seconds for a server workload and can be tens of seconds for a large application. It's why JVM services are often "warmed" with synthetic traffic before entering a load balancer rotation.

Should I use GraalVM Native Image?

If startup and memory matter more than peak throughput — serverless, CLI tools, containers scaled frequently, or anything where a 100MB resident set is a problem. The costs are a closed-world assumption that breaks reflection-heavy frameworks without configuration, longer builds, and roughly 10–30% lower peak throughput. Spring Boot, Quarkus, and Micronaut have invested heavily in making this work, so framework support is much better than it once was.

How do I know if the JIT is optimizing my code?

Use the runtime's diagnostics. V8 offers --trace-opt, --trace-deopt, and --print-opt-code; HotSpot offers -XX:+PrintCompilation and -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining. Tools like JITWatch (JVM) and --prof with the built-in profiler (Node.js) present this more readably. The question worth asking first is usually "is this deoptimizing repeatedly?"

Do interpreted languages without a JIT have a path forward?

CPython's answer has been the specializing adaptive interpreter (PEP 659), which rewrites hot bytecode into type-specialized variants — a technique borrowed from JIT design applied inside an interpreter — plus an experimental JIT tier. Ruby ships YJIT, which has produced substantial real-world speedups. The trend is that "interpreted language" increasingly means "has a JIT you don't think about." See Interpreters.

What is a tracing JIT?

An alternative design that compiles hot execution traces — the actual sequence of operations through a loop, across function boundaries — rather than whole methods. Traces are linear, so optimization is simple, and inlining is automatic. LuaJIT and PyPy use this approach to considerable effect. It struggles when control flow is highly variable, since each distinct path needs its own trace, which is part of why method-based JITs dominate in JavaScript engines.

Related Topics

References