Before any Python: a real-time system is not a fast system, it is a system with a deadline — and an answer that arrives after the deadline is not a slow answer, it is a wrong answer. Everything else in this course is one loop, built ten weeks deep, with intelligence pushed inside that deadline.
Two words, and both of them are usually misread. Real-time gets read as “fast”; intelligent gets read as “machine learning”. Neither is right, and the cost of the misreading is a system that is quick on a laptop and useless in production.
A system is real-time when its correctness depends on when the answer arrives, not only on what the answer is. The deadline is set by the world — by how fast the price moves, how fast the car travels, how fast the furnace heats — and no amount of engineering negotiates it down. Your only choice is whether you meet it.
So “is it fast enough?” is the wrong question. The right one is: what is the deadline, how often do we miss it, and what happens on the miss? A system that answers in 2 ms with a deadline of 1 ms fails. A system that answers in 4 hours with a deadline of a day is real-time. Speed is relative to a clock that somebody else owns.
The three classes below are engineering choices, and you have to pick one per decision, not per system. Most real systems are a hard core wrapped in firm middle layers wrapped in a soft shell — and confusing which layer you are in is how a dashboard ends up blocking a brake command.
Steering, braking, an airbag, a motor commutation loop, a pacemaker. The deadline is part of the safety case, so the design is driven by the worst case, never the typical one.
A quote, a bid, a video frame, a card authorisation. Nothing breaks, but the late answer has zero value and is thrown away — which is why lateness shows up as lost revenue rather than as an error log.
A dashboard, a nightly risk report, a recommendation. Value decays with lateness instead of falling off a cliff. Almost all “real-time analytics” products are soft real-time, and that is fine — as long as nobody puts one in the hot path of something firm.
| name | what it actually measures |
|---|---|
| latency | how old your input is when you act on it |
| throughput | how many events per second you can absorb |
| jitter | how much latency varies cycle to cycle |
| deadline | the instant after which the answer is worthless |
Report a mean latency and you have hidden exactly the events that matter. If one request in a hundred takes forty times the median, the mean barely moves — and one per cent of your decisions are wrong. At a thousand decisions a second that is ten wrong decisions every second, all invisible in the average.
So the metrics in this course are p50, p99, p99.9 and a deadline-miss count. The tail is the system. Part 2 makes that concrete with two systems that have the same mean and completely different fates.
Not a model. A decision. The system observes the world as it is right now, chooses an action, and takes it — with consequences. The intelligence can be four lines of arithmetic or a neural network; what makes it interesting is that it sits inside the loop, under the deadline, where being clever costs milliseconds and milliseconds cost correctness. An offline model that is 4% more accurate and 300 ms slower can be strictly worse than the rule it replaced. That trade is the subject of this whole course, and Part 2 lets you make it with a slider.
Every system on this page is these six stages. Click one to see what it is, what it costs, and how it fails. The ruler underneath stacks the six stage budgets — at the median and at the 99th percentile — against the deadline you set. Watch the p99 bar, not the p50 bar: a loop can sit comfortably inside its budget on a typical cycle and blow through it on one cycle in a hundred.
—
The first experiment separates “fast” from “real-time” using two systems with an identical mean latency. The second buys accuracy with latency and watches it stop being worth the money. Together they are the two decisions you will make every week for the next ten weeks.
Set them up honestly. System A is the one everybody builds: a fast path with an occasional stall — a garbage collection, a lock, a cache miss, a log write that touched the disk, a retry, a Python allocation at the wrong moment. System B is boring: slower on a typical event, but bounded. Both report the same average latency to management. Only one of them is real-time.
Drag the tail weight. As more of A's events fall into the stall, A's fast path gets faster to keep the mean pinned at the same value — the histogram body slides left while the tail stays where it is. A looks better on every average you can compute, and misses more deadlines on every drag. Then drag the deadline and watch which system you would rather own.
There are four ways to compute a decision from live data, and they form a ladder: each rung is more accurate, more expensive, and fails in a nastier way. Nothing forces you to the top. What decides the rung is the deadline — and the fact that a decision which arrives late does not merely score zero, it usually scores negative, because something downstream already acted on the stale answer.
Each rung gets a thousand decision opportunities. A decision that lands inside the deadline scores its edge; one that lands late either falls back to the cheap rule or acts on stale state, depending on the switch. The capacity slider makes every rung bigger: more accurate, and slower. Find the deadline at which the learned model beats the rule — and the deadline at which it stops.
—
The reason this is one course and not six is that the loop does not change. Swap the sensor, swap the actuator, change the deadline by four orders of magnitude, and the engineering stays recognisably the same: bound the tail, keep the decision inside the budget, degrade on purpose, measure everything, be able to replay it. Learn it once in a market, and the fraud detector and the car are re-parameterisations.
Every point of accuracy is bought with time: more features, a longer window, a bigger model, another confirmation. Past the deadline the purchase is worthless. Widget 3 is this trade.
Fresher state means polling harder, holding more in memory, and duplicating work per node. You can always be fresher; you cannot always afford it. Ask what a millisecond of staleness actually costs before you buy it.
The safe move is to do nothing when unsure — but a system that abstains constantly is not doing its job. Every real system encodes where that line is, usually as a threshold nobody wrote down. Write it down.
Pick an industry. The diagram is the same six stages every time; only the labels, the numbers and the consequences change. The deadlines are real order-of-magnitude figures — read them next to each other, because the spread from a motor loop to a fraud rule is five orders of magnitude and the code looks almost identical.
—
—
You run a market-making desk in the class arena. Your loop: read the top of the order book, decide a bid and an ask around your estimate of fair value, send both, and watch what fills. Your deadline is set by how long your quote survives before the fair value has moved underneath it — call it 50 ms in this arena.
Now put intelligence in it: a model that predicts the next move and skews your quotes accordingly. Offline it is clearly better. Online it takes 70 ms at p99. On the one-cycle-in-a-hundred where it is late, your old quote is still resting on the book at a price the world has left behind — and the only counterparty who wants it is the one who already knows. You do not lose the spread on that trade; you pay someone else's.
That is the whole discipline in one paragraph. The model did not fail. The model was late, which in a real-time system is the same thing as being wrong, except that it does not show up as an error — it shows up as a slow bleed in the P&L that no unit test will ever catch. Weeks 3 and 4 have you build exactly this and measure exactly that.
These are conceptual failures, not bugs. Every one of them survives code review, passes its tests, and shows up in production as something nobody can explain. Open them.
The benchmark says 4 ms. Production says 4 ms. The deadline is 20 ms. And once every few hundred events something stalls for 300 ms — the garbage collector, a lock, a log flush that hit the disk, a DNS lookup, a socket buffer filling. Averages absorb all of it.
Worse, the usual optimisations make it worse: batching, caching, adding a queue to “smooth the load”. Each one improves the mean and lengthens the tail. A queue in front of a deadline is a machine for converting a capacity problem into a correctness problem.
Fix: state the deadline first, then measure p99 and p99.9 and a miss count, and treat the worst observed case as the number that describes your system. If a change improves the mean and worsens p99, it is a regression.
“We process two million events a second” answers a different question from “we decide within 10 ms”. High-throughput architectures win by batching and buffering, which is exactly the wrong instinct under a deadline: the system is busy and healthy and every individual answer is late.
The tell is a design that scales by adding partitions and a design goal expressed in events per second. Both are fine — for the ingest stage. Neither says anything about the decide stage, where the deadline lives.
Fix: separate the two budgets explicitly. Ingest is a throughput problem and may batch; decide is a latency problem and may not. Put the boundary between them in your diagram, and put a bounded queue with a drop policy on it — not an unbounded one.
Every value your system holds has an age, and almost no codebase stores it. So a price from 400 ms ago, a position from before the last three fills, a camera frame from two frames back and a fresh tick all arrive at the decision function looking identical — and the decision is computed on a world that does not exist.
This is the most common way an intelligent real-time system produces confident nonsense. The model is fine. The inputs disagree about what time it is.
Fix: timestamp at the source, carry the timestamp with the value, and make the decision function check the age of what it is about to use. An input older than its budget is not an input; it is a reason to degrade. “No answer, on time” beats “a good answer, computed from last week”.
Two machines never agree on the time — ordinary clock offsets are milliseconds and can be far worse — so “event X happened before event Y” is a claim you cannot make from two timestamps taken in two places. Meanwhile failure is not binary: the interesting state is the one where a peer is alive, reachable, and answering 800 ms late, or answering with stale data, or answering wrongly. Partial failure is the normal case.
Every distributed real-time system you will meet has this shape, and the reflex — retry, wait, block until the peer responds — converts one slow peer into a system-wide deadline miss.
Fix: order events with sequence numbers from a single writer rather than with wall clocks; put a timeout with a defined fallback on every remote call; and decide in advance what your system does with a peer that is late rather than dead. Week 5 makes you cut a link and live with it.
The decision rule was fitted to a world, and the world drifts: volatility regime, traffic mix, lighting, seasonality, a new product, a new attacker. Accuracy decays quietly, and because the system keeps answering on time, every dashboard stays green. Add an adversary — someone who profits from your predictable response — and the drift stops being accidental: your own behaviour becomes the thing being exploited.
The second half of this pitfall is that a system with no monitor cannot degrade, because nothing knows anything is wrong. It just keeps acting.
Fix: make the measure stage a first-class part of the loop, not an afterthought. Track the input distribution as well as the output, alarm on divergence between what you predicted and what happened, and give the system a defined safe state it can enter on its own. Week 9 makes you write that down as a one-page safety case.
Three questions, instant feedback, nothing recorded.
Q1Two systems, one deadline of 40 ms. A: mean 9 ms, p50 6 ms, p99 240 ms. B: mean 19 ms, p50 19 ms, p99 26 ms. Which one would you ship?
Q2Your decision arrives 15 ms after a 50 ms deadline, and it is correct. What is its value?
Q3In a real-time intelligent system, where does the “intelligence” actually live?
Ten weeks, one system, four domains. Every week adds a stage to the loop you met in Part 1, or hardens one that is already there — and every week ends with a measurement rather than an opinion.
Click a week. The chips show which of the six stages it touches; the panel says what you build and what you measure. Use the arrow keys once a week is focused.
—
make install, and run the smoke test before week 1.