Interpreting bytecode instruction by instruction is portable but slow. Mechanically, where does the slowness come from, and why does a runtime that interprets first still keep an interpreter around after it can compile code to native instructions?

## What an interpreter does per instruction Conceptually the execution loop is: ``` while (true) { opcode = code[pc]; switch (opcode) { ... } // or: jump to handler[opcode] } ``` For every bytecode the machine must: load the opcode byte, decode any inline operands, compute the address of the handler, take an **indirect branch** to it, perform the actual semantic work, update the program counter, and branch back. The semantic work for `iadd` is a single machine add. Everything around it is overhead — and it is overhead that recurs on *every* execution of that instruction, including on the millionth iteration of a loop. Three distinct costs stack up: **1. Dispatch overhead.** The indirect branch at the end of each handler is hard for branch predictors: the next bytecode is data-dependent. A mispredict costs on the order of a dozen-plus cycles on a modern out-of-order core — vastly more than the instruction being interpreted. Interpreter designs fight this with threaded dispatch (each handler ends with its own indirect jump, giving the predictor per-opcode history) rather than a single shared switch. **2. Data movement through the operand stack.** Bytecode semantics say values live on the operand stack. A faithful interpreter therefore stores intermediate results to memory and reloads them, where compiled code would keep them in registers. `x = x + 1` becomes load-local, push constant, add, store-local: four dispatches and several memory touches for what compiled code does in one increment. (Real interpreters cache the top of stack in a register to claw some of this back.) **3. Absence of optimization across instructions.** This is the big one and it is structural, not an implementation weakness. An interpreter sees one instruction at a time, so it cannot: - inline a call — every `invokevirtual` performs a real dispatch through the receiver's method table, and the callee's body can never be merged with the caller's; - fold constants or eliminate redundant loads; - hoist loop-invariant computation out of a loop; - eliminate a bounds check it could prove redundant; - keep a hot object's fields in registers, or scalar-replace an object that never escapes. Compiled code gets all of these because the compiler sees a whole method (and, after inlining, effectively several). ## Why not compile everything up front, then? Because compilation costs time and memory, and most code does not repay it: - **Startup latency.** Compiling every method before running it delays the first useful work; interpreting begins immediately. For a short-lived process, or the first seconds of a server's life, the interpreter is genuinely the faster choice. - **Cold code dominates.** In a large application the overwhelming majority of loaded methods execute a handful of times — configuration parsing, one-shot initialization, error paths. Native code for them consumes CPU to produce and code cache to hold, and buys nothing. Runtimes therefore compile only what proves hot, using counters the interpreter maintains. - **The profile has to come from somewhere.** Aggressive optimizations are speculative: "this call site has only ever seen one receiver type, so inline it", "this branch is never taken, so do not emit it", "this exception path is dead". Those facts are observations about the running program. The interpreter is the cheap instrument that gathers them — invocation counts, loop back-edge counts, per-call-site receiver types, branch taken/not-taken data. - **Speculation needs a safety net.** When a speculative assumption is invalidated — a new subclass is loaded and the monomorphic call site becomes polymorphic, or a "never taken" branch is finally taken — the runtime must abandon the compiled code mid-execution and continue somewhere correct. That somewhere is the interpreter: the runtime reconstructs an interpreter frame (locals and operand stack) at the corresponding bytecode index and resumes interpreting. Without an interpreter, deoptimization would have no destination, and speculative optimization would be unsafe. - **Some code is not compilable in practice.** Very large methods, and certain rarely executed constructs, are simply left to the interpreter. ## How to observe the effect Running with interpretation only (`-Xint`) versus the default typically slows CPU-bound benchmarks by roughly an order of magnitude — a crude but convincing demonstration of the gap. Conversely, this is why benchmark harnesses insist on **warmup**: measurements taken while code is still interpreted describe the interpreter, not the program. ## The takeaway framing Interpretation and compilation are not competitors; they are a two-speed system. Interpretation gives portability and instant start with no speculation; compilation gives speed but needs evidence, time, and a way back. A production JVM runs both simultaneously, on different methods, all the time.