A real-time system does a dozen things at once on a single thread, and speaks to the world through a byte stream that has never heard of your messages. Exactly one blocking call, or one missing length prefix, is enough to break either half.
Concurrency is structure: many tasks in progress, interleaved.
Parallelism is execution: many tasks running in the same instant. One chef juggling six
pans is concurrent; six chefs at six stoves are parallel. Python’s asyncio gives you the
first and never the second — and for a real-time pipeline that spends 95% of its life waiting on
sockets, the first is what you actually needed.
A socket read. A log line. Waiting for an OrderAck. The process is parked
doing nothing while bytes are in flight. Interleaving wins here because there is idle time to fill:
while one task waits, another runs, on the same core.
Scoring an anomaly model. A forward pass. Sweeping parameters. Nobody is waiting on anything — the core is genuinely busy. Interleaving buys nothing, because there is no idle time to reclaim.
CPython holds a Global Interpreter Lock: only one thread executes Python bytecode at a
time. Four threads crunching numbers therefore take as long as one thread doing all four jobs. But the
GIL is released around blocking calls — a socket read, a disk read, an HTTP
request — which is why threads still help for I/O, and why the one remaining thread in the arena
is the blocking yfinance poller.
# 4 threads, CPU-bound — NO speedup (the GIL serialises them) from threading import Thread def crunch(n): while n: n -= 1 # pure Python bytecode ts = [Thread(target=crunch, args=(10**7,)) for _ in range(4)] [t.start() for t in ts]; [t.join() for t in ts]
An async def function is a coroutine. Calling it does not
run it — you get a coroutine object. Only await, or the loop via a Task, drives it,
and every await is a point where it may hand control back so something else can run. No
await, no hand-back: that is the whole rule, and the whole danger.
async def read_feed(ws): while True: raw = await ws.recv() # pause here; loop runs others await q.put(parse(raw)) await asyncio.gather( # concurrent, not parallel read_feed(ws), strategy(q), heartbeat(ws))
Four coroutines from a real bot: a feed reader, your strategy, a fill handler and a heartbeat — plus one thread, off the loop, doing the blocking price poll. Press Step to advance one tick. Watch tasks move waiting → ready → running and back, and watch that exactly one box is ever running.
select() waiting for the kernel to say a socket is
readable. Busy near 100% does not mean productive; it means the loop has no slack, and the next tick
will queue behind whatever is running.Everything below is one workload with two knobs. Move the I/O fraction and watch the ranking of the four concurrency models invert — there is no “fastest” model, only a model that matches the workload.
| Model | Wall clock | Speed-up | Why |
|---|
The feed below is a coroutine appending ticks to a chart. Now do a slow thing — a model fit, a big CSV, a synchronous HTTP call — three different ways, and watch what happens to everything else. The counter that matters is ticks missed: those are messages that arrived while your loop could not read the socket.
The producer is the market; you do not get to slow it down. The only thing you choose is what happens when your consumer cannot keep up. Three choices, three very different failure modes — and only one of them is honest about it.
This is the part every student is sure they understand until their recorder eats a
message. TCP is a stream of bytes with no boundaries. recv() returns
“whatever has arrived” — half a message, one and a half messages, seven messages glued
together. Your framing is what turns that back into messages, and your schema is what turns messages back
into meaning.
Below, three book_snapshot messages are put on a
socket and arrive in chunks the sender did not choose. Pick a framing and deliver the chunks one at a
time. Watch which framings recover exactly three messages and which recover two, four, or none.
You have recovered a message. It is still just text. Edit the
order below and watch a Pydantic-style validator check it against PlaceOrder — field by
field, with the same errors you will see in your terminal. Every error caught here is an error that
does not become a wrong position two hops downstream.
# shared/messages.py — the single source of truth class PlaceOrder(BaseModel): type: Literal['place_order'] = 'place_order' team_id: str symbol: str side: Literal['buy', 'sell'] order_type: Literal['limit', 'market', 'ioc', 'post_only'] price: float quantity: int
Four of these five will cost you messages in a live session. Open them.
time.sleep() or requests.get() inside a coroutineThe canonical mistake, and it is silent: no exception, no warning, just a system that is
inexplicably worse than it was on your laptop. Any blocking call inside async def stops
the entire loop — feed, orders, heartbeat, telemetry, everything.
It hides in innocent places: a pandas.read_csv on a big file, a print() to
a slow terminal, a synchronous yfinance call you added “just to check
something”, an onnxruntime forward pass you moved into on_tick.
Fix: await asyncio.sleep() for delays, an async client for HTTP, and
await loop.run_in_executor(None, fn, *args) for anything blocking you cannot replace. Run
with PYTHONASYNCIODEBUG=1 in development; it warns when a callback takes too long.
recv() is one messagemsg = json.loads(await reader.read(4096)) works on localhost with small messages and
fails the first time the network is real. You will see two symptoms and misdiagnose both: a
JSONDecodeError on a truncated message, and — worse — a
successful parse of the first of two glued messages, silently discarding the
second.
The second symptom is the dangerous one. Nothing raises, your recorder is short by 3% of messages, and you find out in week 6 when your detector cannot reproduce a known event.
Fix: frame it. A 4-byte big-endian length prefix
(struct.pack('!I', len(payload))) is 10 lines and always correct; a newline delimiter
works only if the payload can never contain one. Or use websockets, which frames for
you — and understand that this is the actual reason to use it.
asyncio.Queue() with no maxsize never applies back-pressure. A fast feed
and a slow consumer means the queue grows until the process is killed — and long before that,
your logic is acting on data that is ten seconds old.
Stale is worse than missing. A book snapshot from 8 seconds ago is not “some information”; it is actively wrong information you are about to send an order against.
Fix: bound it (maxsize=100) so put() waits and the pressure
travels upstream where you can see it, or coalesce — keep only the newest snapshot per symbol and
throw the rest away. Then export the stall or drop counter as a metric; it is the earliest warning you
will get.
Four threads fitting four models is exactly as slow as one thread fitting four models, because the GIL lets only one execute bytecode at a time. You paid for locks, races and debugging and got nothing.
The mirror-image mistake is reaching for processes to do I/O: you pay 50 ms of spawn cost and a pickling round trip to wait on a socket you could have awaited for free.
Fix: CPU-bound → ProcessPoolExecutor. I/O-bound →
asyncio. Threads are for blocking I/O that has no async API — which in this platform
is exactly one thing, the yfinance poller.
ConnectionClosed end the sessionNetworks blink. Wi-Fi hiccups, a peer restarts, a laptop lid closes. An uncaught
websockets.exceptions.ConnectionClosed ends your process and your telemetry stops at
whatever it was — often with a position still open.
Reconnecting badly is its own trap: a tight retry loop hammers a recovering server, and resuming without re-syncing means your local view of the world is a fiction you are now trading against.
Fix: catch it, reconnect with exponential backoff and jitter, re-handshake, and request a fresh snapshot rather than assuming continuity. Then re-assert your risk state before you send anything. Week 5 makes this a graded property, not a nicety.
Three questions, instant feedback, nothing recorded.
Q1Your on_tick calls a library function that blocks for 800 ms. What happens to the feed-reader coroutine on the same loop?
Q2You are hand-rolling a TCP market-data feed. Messages are JSON of varying length, and any string field may contain arbitrary text. What goes on the wire?
Q3Your feed produces 400 messages/s; your consumer handles 90/s. Which change keeps the system both alive and useful?
websockets and the real
message schemas — and say what the library bought you.ConnectionClosed with backoff and a fresh snapshot on reconnect.AI-USE note on the deliverable.make install.