Compilers & Interpreters

Three routes from source code to silicon — and why the route decides the speed
Part 7 of 9
• Translation
• Part 7 of 9
The Premise
The Translation Problem
The CPU executes only machine code — the ISA from Part 6. Every language must get there somehow; when and how the translation happens is the whole game.
›
Ahead-of-Time · C / C++
Compile Once, Run Bare
Translate the whole program to native machine code once, before running. At runtime there is nothing between your code and the CPU.
›
Interpreted · Python
Read It, Do It, Repeat
A program (the interpreter) reads your code and performs it, instruction by instruction, every single run. Maximum flexibility, maximum overhead.
›
Hybrid · Java (JIT)
The Middle Path
Compile to portable bytecode, then a virtual machine compiles the hot parts to native code at runtime. Slow start, near-native cruise.
Section A

Three pipelines, one source line

Follow the same loop down three different roads. Watch the phase chips: everything marked before run is work done once, by a tool, while you wait at your desk. Everything marked at runtime is work your CPU pays for while the program is running — sometimes on every single iteration. That single distinction explains the entire performance gap.
Shared source line for i in 0..N: sum += i N = 100,000,000 · same algorithm, same machine, same answer — three translation strategies
C / C++
Ahead-of-time compiled
All translation happens before you press Run. The shipped artifact is machine code.
before run · build time
main.cpp — source
Human text. Static types declared: long sum; long i;
Preprocessor
Textual pass: #include files pasted in, macros expanded, #ifdef branches resolved. Output: one big translation unit.
Front-end: parse & type-check
Tokens → abstract syntax tree → typed intermediate representation. Every type is known now, so no type check is needed later.
Optimizer -O2
Inlining, constant folding, loop unrolling, register allocation, auto-vectorization — the loop may become 4-wide SIMD, or vanish entirely into a closed-form sum. The compiler gets minutes to think; the CPU gets nanoseconds.
Assembly → object code
ISA-specific text (x86-64 / ARM64) assembled into relocatable machine code: main.o
Linker
Stitches object files + libraries, resolves every symbol to a real address.
Native binary ./a.out
A file of raw ISA instructions. The translation is finished forever. Not portable — one binary per OS + CPU family.
at runtime
CPU executes directly
Machine code on silicon — nothing else. One thin block, because there is nothing else to draw.
1–4 machine instr / iteration And with SIMD one instruction does 4–8 additions at once, so the effective cost drops below one instruction per element.
Java
Bytecode + JIT (hybrid)
Half the translation up front, half deferred until the VM learns which code is hot.
before run · build time
Main.java — source
Static types, like C++: long sum; long i;
javac
Parses and type-checks, then emits portable stack-machine bytecode. Deliberately does almost no optimization — that job is the VM's.
Bytecode Main.class
Instructions for an imaginary CPU — iload, ladd, lstore. Runs anywhere a JVM exists: "write once, run anywhere." No real silicon speaks this.
at runtime · inside the JVM
1 · JVM loads & verifies
Class loading, bytecode verification, heap and GC set-up. This is the JVM's famous startup cost — tens of milliseconds before your first line runs.
2 · Interpreter runs bytecode
The first thousands of iterations are interpreted — slow, roughly Python-like territory. But typed and stack-based, so still much leaner than CPython.
3 · Profiler spots the hot loop
The VM counts invocations and back-edges. Cross the threshold (~10k) and this loop is officially hot.
warm-up
4 · JIT compiles it to native
C2 / Graal compiles the hot method to real machine code — inlining, unrolling, even vectorizing, using runtime facts a static compiler never sees (actual types, actual branch odds). Then execution jumps into the native version.
Native execution (cruise)
Now nearly C-speed — but a GC pause or a deoptimization can interrupt at any moment.
~1–5 machine instr / iteration · after JIT Before the JIT kicks in, closer to 20–40. The whole Java performance story is: pay warm-up once, cruise fast forever.
Python
Interpreted (CPython)
Almost nothing happens before run. Everything — parsing aside — happens again, and again, and again.
before run · barely anything
main.py — source
No type declarations at all. Nobody knows what i or sum is until the line actually runs.
Transparent compile → .pyc
CPython does compile — to its own bytecode, cached in __pycache__. It happens invisibly at import/startup, not as a build step, and it performs no real optimization: no inlining, no unrolling, no type specialization worth the name.
at runtime · the eval loop
The interpreter is a C program
python3 is itself a compiled C binary. Your script is just data it chews on. The CPU never sees your code — it sees the interpreter's code.
per iteration · repeats 100,000,000×
1
Fetch next bytecode
Read opcode + oparg from the code object, bump the instruction pointer.
2
Dispatch — the giant switch
An indirect jump into a ~200-case table. Hard for the CPU's branch predictor: a mispredict costs ~15 cycles (Part 4).
3
Type-check both operands
Is it int? float? A class with __add__? Follow each object's type pointer and look up the numeric slot. Every time.
4
Unbox two heap objects
A Python int is a PyLongObject on the heap: refcount + type ptr + size + digits. Dereference pointers to reach the actual number — likely a cache miss (Part 3).
5
Add
The real work: one machine instruction. Everything else on this list is ceremony around it.
6
Box the result
Allocate a brand-new PyLongObject for the sum (ints are immutable), initialize its header, hand back a pointer. Allocation, in a loop, 100M times.
7
Update reference counts
Incref the new object, decref the old one, and free it when it hits zero — extra memory writes that also serialize things (the GIL, Part 5).
CPU executes the interpreter
Fast silicon, faithfully running hundreds of instructions of bookkeeping per one instruction of your arithmetic.
100+ machine instr / iteration Same addition. Same CPU. Two orders of magnitude more work, paid on every one of the 100 million iterations.
Read the lanes vertically and the moral is obvious: the C++ lane is tall at the top (build time) and almost empty at the bottom (runtime); the Python lane is the exact mirror image. Java splits the difference — and pays for it with a warm-up. Nothing here is about "language speed" in the abstract; it is about where the translation work is scheduled.
Section B · Interactive

The race: same loop, three languages

One core, one algorithm, three translation strategies. Press Start and watch in scaled real time (the whole race is stretched to about 8 seconds of wall clock, so Python's grind is actually watchable). Times are typical orders of magnitude for a single core — not a benchmark of your machine.
Task sum = 0; for i in 0..100,000,000: sum += i single core · no libraries · answer: 4,999,999,950,000,000
C++
g++ -O2 · native
idle — ready to run
0 ms
finished
Java
JVM · C2 JIT
not started
jit warm-up
idle — ready to run
0 ms
finished
Python
CPython · pure loop
idle — ready to run
0 ms
finished
The first 200 milliseconds, magnified 25× — because at true scale C++ and Java are already done before you can blink
C++ done · 50 ms
JIT compiles · 45 ms
Java done · 150 ms
Python at 200 ms: 4% done — 4.8 s to go
0 ms50100150200 ms
Same algorithm. Same CPU. 100× spread — the route from source to silicon is the difference.
1×C++ baseline
3×Java (with warm-up)
100×CPython
Wall clock 0.0 s · simulated 0 ms
Fairness footnote. This is the worst case for Python, chosen because it isolates interpreter overhead. Real Python code escapes to C for heavy math — a NumPy np.arange(N).sum() hands the loop to compiled C/Fortran and lands within a small factor of C++, because the interpreter runs once instead of 100 million times. JIT'd Java on long-running servers sits within roughly 1–2× of C++. The gap is largest for pure interpreted loops exactly like this one — and it collapses the moment the loop moves into compiled code.
Section C · Anatomy

Why exactly is C/C++ faster?

Zoom all the way in on sum += i — one iteration — and count what the silicon actually does. Left: the compiled version, which is essentially the addition itself. Right: the interpreted version, where the addition is a rounding error inside the bookkeeping.

C++ — the actual machine code

x86-64, g++ -O2, scalar form
This is the whole loop body. Not a representation of it, not bytecode about it — these four instructions are what the decoder feeds the execution units.
1
add  rax, rcx ; sum += i
One ALU micro-op. Both operands are already in registers — zero memory traffic. Throughput: up to 4 of these per cycle on a modern core.
2
inc  rcx ; i++
The loop counter is a register, not an object. No allocation, no refcount, no indirection.
3
cmp  rcx, rbx ; i < N ?
Compare against the loop bound, also in a register. Sets flags; costs a fraction of a cycle.
4
jne  .Lloop ; branch back
Taken 99,999,999 times out of 100,000,000 — the branch predictor gets it right essentially always, so it is effectively free (Part 4).
4 instructions · ~1 ns
Roughly one cycle of real work per iteration. sum and i live in registers — Part 3's fastest storage that exists, ~0.3 ns, no cache involved at all. Nothing is allocated, nothing is dereferenced, nothing is checked.
⚡ and then the optimizer gets greedy
vpaddq  ymm0, ymm0, [rsi+rax*8]   ; 4 adds in ONE instruction
add     rax, 4                    ; stride 4 elements
; ...horizontal reduce ymm0 into rax at the end
With AVX2 the loop becomes 4-wide (AVX-512: 8-wide), so the effective cost drops
below one instruction per element. -O3 may even recognise the closed form
N(N-1)/2 and delete the loop entirely — 100M iterations, zero runtime.

Python — the CPython bytecode

dis.dis() of the same iteration
Each line below is not a machine instruction — it is a request to the interpreter, and every request expands into dozens of real instructions. What is written under each one is what it truly costs.
1
FOR_ITER ; next value from the range
Interpreter dispatch — indirect jump through the ~200-case opcode switch, an often-mispredicted branch (~15 cycles). Then call the iterator's __next__ and box a fresh int object for i.
~25 machine instr
2
LOAD_FAST  sum ; push a pointer
Dispatch again, then read a pointer out of the frame's local array and push it on the value stack. You loaded an address, not a number.
~10 machine instr
3
LOAD_FAST  i ; push a pointer
Same ceremony a second time. Two dispatches so far, and not one arithmetic operation has happened yet.
~10 machine instr
4
BINARY_OP  += ; the "add"
Type-check both operands (follow two type pointers, look up the numeric slot) → unbox two heap PyObjects (dereference into the digit array — pointer-chasing, so likely cache misses, Part 3) → do the one real add → allocate and box a brand-new int object for the result.
~50–70 machine instr
5
STORE_FAST  sum ; rebind the name
Write the new pointer into the frame, then update reference counts: incref the new object, decref the old one, and free it if it reached zero.
~12 machine instr
6
JUMP_BACKWARD ; loop again
Back to the top of the eval loop — and pay all of the above again, 100 million times over.
~6 machine instr
≈ 100+ instructions · ~50 ns
Several memory dereferences per iteration, at least one heap allocation, and a handful of hard-to-predict branches. Pointer-chasing = cache misses (Part 3), and a DRAM miss alone costs more than C++'s entire iteration. The arithmetic is <1% of the work.
REASON 01
No interpreter in the way
Your instructions go straight to the decoder. Nothing fetches, dispatches or re-decides anything at runtime.
REASON 02
Static types → registers, not objects
A known long lives in a 64-bit register. A Python int is a heap object with a header and a refcount.
REASON 03
Contiguous data, no boxing
An array of 8-byte values is cache-friendly and prefetchable. A Python list is a pointer array to objects scattered across the heap (Part 3).
REASON 04
A whole optimizer, for free
Inlining, unrolling, strength reduction, SIMD vectorization — minutes of machine reasoning baked in once, before you ship.
REASON 05
No GC pauses, deterministic memory
You decide when memory is freed, so there is no collector deciding to stop the world mid-loop. Predictability, not just speed — the point of Part 9.
Speed isn't free
C++ shifts the burden to the programmer: memory bugs (use-after-free, buffer overruns, whole classes of security vulnerabilities), longer builds, manual lifetimes, and far more code to say the same thing. Python trades machine time for human time — and since machine time is cheap and engineers are not, that is very often the right trade. Part 9 (tick-to-trade) is the world where it never is: when a microsecond is money, you pay the human cost and take the registers.
Instruction counts and nanosecond figures are order-of-magnitude, rounded for intuition; exact values depend on CPython version (3.11+ specializing adaptive interpreter narrowed this meaningfully), compiler flags, and microarchitecture. The shape of the comparison — four instructions versus a hundred — is what matters.