Part 9 of 9 · Finale The finale: everything Parts 1–8 built — and how HFT bypasses half of it for speed.

Tick-to-Trade: The Path of Market Data

From the exchange matching engine to the trading algorithm and back — why a microsecond is an eternity
• HFT
• Low Latency
← Series index
The Problem
The Race
Every co-located firm sees the same market event at the same instant. The first executable order to reach the gateway wins the trade; second place gets nothing.
›
The Hard Floor
Speed of Light Budget
Fiber ≈ 5 ns/m, microwave ≈ 3.3 ns/m. Physics sets the floor, so you buy a rack in the exchange building — co-location minimizes distance.
›
The Software Tax
The Kernel is Too Slow
The Linux network stack costs 1–10 µs per packet and jitters badly. Bypass it: user-space networking (Onload, DPDK, ef_vi) or an FPGA on the NIC.
›
The Craft
Hot Path Discipline
Keep the critical path in L1 cache, branch-free, allocation-free, lock-free and syscall-free. Do the work before the packet arrives.
▪ The Geography of Latency
fiber · light at ⅔c ≈ 200,000 km/s microwave · ≈ c in air — beats fiber on long routes
EXCHANGE matching engine · t = 0 your rack CO-LOCATION your servers, metres away microwave · ≈ c in air — beats fiber on long routes fiber · light at ⅔c ≈ 200,000 km/s — ≈ 5 ns per metre
Why firms pay for the rack CLOSEST to the matching engine: distance is latency physics can’t refund.
Exchange
Network (fiber / switch)
NIC — kernel path
NIC — bypass path
PCIe DMA / memory
CPU — trading algorithm
All latencies are order of magnitude — they vary by venue, hardware and configuration.
▪ Tick-to-Trade Round Trip
t = 0
1 · Exchange
Matching Engine
A resting order is hit: a trade prints and the order book changes. The engine publishes the update as a multicast UDP packet (~100 bytes of ITCH-style binary) to every subscriber at once.
~250 ns
2 · Physical Transport
Fiber in the Colo Hall
Cross-connect ~50 m ≈ 250 ns. Light in glass travels ~5 ns per metre and no software trick reduces it. Exchanges equalize cable lengths between tenants for fairness.
~300 ns
3 · Colo Switch
Cut-Through vs Store-and-Forward
Cut-through forwards after reading only the header: ~300 ns port-to-port. Store-and-forward buffers the whole frame and validates the CRC first: microseconds, plus queueing jitter.
4 · Network Interface Card
Two Ways In — Pick One
Kernel path
interrupt → driver → kernel UDP stack → socket copy to user space
~2–10 µs
Naive path
Bypass path
user-space polling of the NIC ring buffer (Onload / DPDK / ef_vi)
~300 ns
FPGA on NIC: ~100 ns
~200–300 ns
5 · PCIe DMA → Memory
Packet Lands in the Machine
The NIC DMAs the frame across PCIe. With Data Direct I/O (DDIO) it is written straight into L3 cache instead of DRAM, so the trading thread's first read is an L3 hit (~40 ns), not an ~80 ns DRAM trip.
6 · CPU
Trading Algorithm (Hot Path)
Feed handler
parse ITCH/FIX
100–300 ns
Order book
update levels
~100 ns
Pricing / signal
decide
50–200 ns
Risk checks
limits
~50 ns
Order gen
serialize
~100 ns
The entire working set must stay in L1/L2. One DRAM miss (~80 ns) blows a visible hole in the budget — the most expensive instruction is a cache miss.
~300 ns
7 · Order Out
NIC → Switch → Exchange Gateway
The same wire, walked backwards. Bypass send pushes the message straight onto the transmit ring; a kernel send() drags it back down through the whole stack it came up.
—
8 · Finish Line
Exchange Receives the Order
The gateway timestamps the order and the matching engine queues it. Whoever arrived first takes the fill; everybody else is trading against a price that no longer exists.
0.00 µs
Wire-to-wire elapsed
04.5 µs9 µs13.5 µs18 µs
⚑ The race — same event, same shared 18 µs scale FINISH · 18 µs
Kernel path
—
Bypass path
—
Step or play a scenario: the bright lane runs, the dim lane shows where the other path finishes.
Stage This path Naive path
Total wire-to-wire — 18.00 µs

Simulation Log

Step 0 / 0