A model that is right after the deadline is wrong — this week you put a signal inside a live hot path, pay for it in milliseconds, and wrap it in controls that fire before you do.
Week 3 gave you a loop: ingest → decide → act → observe. The decide stage was a placeholder. This week it becomes intelligence — and the moment it does, it inherits the loop's constraint. Intelligence in the loop is not a model; it is a model that has to answer before the next event arrives.
That single constraint separates this course from a machine-learning course. Offline, accuracy is the only currency and you may spend as long as you like. Online, accuracy is one of two currencies, and the other is time. A detector that is 3% better and 40 ms slower may be strictly worse. Naming that trade — out loud, with numbers — is the skill.
Batch, retryable, reproducible. Sees the whole dataset, including rows that arrived after the moment it is reasoning about. Fit it, cross-validate it, publish a confusion matrix. Wall clock is a convenience.
One event at a time, in order, with no second look. Sees only the past, decides under a deadline, and its wrong answers are actions in the world — orders, positions, exposure. Wall clock is the specification.
In the arena SDK the decision stage is one method. It receives the world as of now and returns an intent or nothing. It is a pure function of the past, which is exactly what makes it testable on a tape and safe on the wire.
# arena/trader.py — the whole surface of "intelligence" in this system class MyBot(Trader): def on_tick(self, market: MarketData, portfolio: Portfolio) -> Signal | None: # market: last prices, price history, book snapshot — up to 'now', never past it # portfolio: cash, positions, average entry — your own state # return a Signal, or None to do nothing. Must return before the next tick. ... def on_fill(self, side, symbol, quantity, price, fee): ... # the world answered def on_event(self, event, message, data): ... # halt, shock, session close
A signal is the narrow, testable core inside that method: a function that maps observable state to a number, positive for “buy pressure”, negative for “sell pressure”. Three families cover almost everything you will build this quarter, and they differ in which observable they trust.
Trust the recent trend: sign(p₀ − SMA(p, L)). Cheap, robust, and it pays
when a move has an information cause that has not finished propagating. Dies in chop, where every
crossing is noise you pay a spread to trade.
Trust the level: −z(p, L), short the stretch, buy the dip. Pays when moves are
liquidity events with no information in them. Its failure mode is a real regime change, where
“cheap” keeps getting cheaper.
Trust the book, not the tape: (B − A) / (B + A) over the top levels —
OrderBook.order_book_imbalance(). It is the only one of the three that sees intent
before it becomes a print, so it is the shortest-lived and the most latency-sensitive.
on_tick, name the
instant it became knowable. If any input is knowable only after the decision timestamp, your
backtest is fiction and your live P&L will be the correction. Most look-ahead bugs are not maths
errors — they are a column read from the wrong row.A signal is a number. P&L is what happens after that number becomes an order, the order crosses a spread, the venue charges a fee, and the price moves while all of that is in flight. Everything between the signal and the fill is friction, and friction is the reason most signals that look real are not tradeable.
A buy lifts the ask, a sell hits the bid. scripts/tca_report.py scores every fill in
signed bps against the prevailing mid: negative means you paid up. Volume-weight it and you have
your real execution cost, per strategy, per session.
Maker/taker is the default: taker pays 0.0015 of notional, maker earns a
0.0010 rebate. A 25 bps round-trip swing between crossing and posting is often larger
than the entire signal.
exchange/limits.py meters you with a token bucket:
ORDER_QUOTA_PER_TICK = 20, burst ×3. Over quota the venue answers
RATE_LIMITED — a signal that fires every tick will be throttled precisely when
the market is moving.
1.4 × sqrt(qty/100) bps of impact plus the 15 bps taker fee. Hit
rate counts closed round trips, not ticks — the same convention tca_report.py uses.Now add the clock. Your decision function has a compute time C; events arrive every
T; and your edge has a half-life — the time after which half of it has
already been arbitraged away by someone faster. Those three numbers, and nothing else, decide whether
your intelligence is worth running.
Keep only the newest event; skip whatever you could not process. You miss ticks, but every
decision you do make is fresh. Age stays bounded at roughly C. This is what a real-time
loop should do — and what asyncio.Queue(maxsize=1) plus a
put_nowait/discard gives you.
Process every event in order. Nothing is dropped, so it feels safer — but if
C > T the queue grows without bound and your decision age grows linearly with
session length. By minute ten you are trading a market that no longer exists. This is the failure
every student ships once.
exp(−ln2 × age /
half_life), where age is the decision's own staleness — queue delay plus compute plus half
a tick of sampling. The sweep on the right holds everything fixed except C, so the cliff
you see at C = T is the whole argument for measuring p99 instead of the mean.Season rank in this arena is deliberately not final net worth. exchange/scoring.py
computes a risk-adjusted score from your equity snapshots, and it punishes exactly the behaviour a
freshly-tuned signal encourages.
# exchange/scoring.py — the formula you are actually optimising r[i] = e[i] / e[i-1] - 1 excess[i] = r[i] - RISK_FREE_PER_SNAPSHOT # idle cash must score ~0 risk_adjusted = (mean(excess) / max(vol, VOL_FLOOR)) \ * max(0, 1 - 2.0 * max_drawdown) # SEASON_DRAWDOWN_PENALTY # clamped to +/- 10. A 25% drawdown halves your score; a 50% drawdown zeroes it.
Read that as an engineering requirement, not a grading rule. It says: volatility is a cost and drawdown is a cliff. A bot that doubles and then gives half back scores worse than one that grinds out a third of the return with a shallow curve. Which is why risk controls are part of the system, not a politeness — and why they belong inside the loop, on the same deadline as the signal.
RATE_LIMITED. It protects the exchange from you; plan for it or
it becomes your outage.venue-sideYour signal has parameters, and you will be tempted to pick them by scoring the whole tape. That number is not an estimate of anything: it is the maximum of a set of random draws. Walk-forward fixes the procedure rather than the parameter — fit on a window, trade the next window blind, roll forward, and report the concatenation of the blind segments. What you learn is not the best parameter but whether your method survives being re-fitted, which is what will happen every week of the season.
| Fold | Train Sharpe | Chosen L | Blind Sharpe | Gap | Blind return |
|---|
Every one of these produces a system that runs, logs cheerfully, and is wrong. They are the five we see every year, in roughly this order.
You call a fitted estimator, a pandas rolling window, or a joblib load directly
inside async def on_tick. It takes 180 ms. Nothing else in the process runs for
180 ms: not the websocket reader, not the heartbeat, not the kill switch. The queue behind you
fills, every subsequent decision is stale, and your “drop-latest” policy never gets a
chance to drop anything because you never yielded to the loop.
The fix. Measure the decision path first — a
perf_counter around the call, reported in p99, not mean. Then keep the hot path pure
arithmetic on pre-computed state, and push anything heavy to a
run_in_executor/process worker whose result the loop polls. If the model cannot
meet the budget, the correct answer is a cheaper model, not a faster laptop.
The obvious version — a signal built from the bar it trades — you will catch with widget 1. The version that survives review is subtler: a recorded tape whose rows carry fields stamped by the writer, not the reader. A trade record that already contains the settlement mid. A book snapshot logged after the event that changed it. A feature column back-filled by a later batch job. Replay that file and your bot is quietly reading the answer.
The fix. Record with receipt timestamps and replay strictly by them, never by the venue's event time. Then do the knowability audit input by input: for each field, at what instant did this process learn it? Anything you cannot answer, drop.
The tape version is a vectorised notebook; the live version is a rewrite. Now every difference in P&L has two candidate causes — the market, or the rewrite — and you can never attribute the gap. This is the single most common reason a student's week-4 signal “stops working” on the wire.
The fix. One on_tick, two drivers. The offline driver feeds
it a recorded tape; the live driver feeds it the websocket. The strategy object cannot tell which,
because it only ever sees (market, portfolio). Then the gap has exactly one cause left
— fills, fees and latency — and you can measure each with
tca_report.py.
A stop loss is a promise to sell at a price. In a flash crash there is no bid at that price; your
stop becomes a market order into a vacuum and you learn what “slippage” means. Worse is
the control implemented as a resting order on the venue: during a halt, a disconnect or a
RATE_LIMITED burst it does not exist at all, and your bot is unhedged and blind.
The fix. Layer the stack (widget 4) and make the outermost layer refuse to act rather than try to act. A position limit is arithmetic. A kill switch that stops sending needs no counterparty and cannot slip. Test both against a disconnect, not just against a price move.
You grid-search a lookback on 720 ticks, pick the best, report its Sharpe, and hold out the last 20% — after having already looked at it once to sanity-check the pipeline. The held-out number is now contaminated, and the best-of-N maximum is biased upward by construction: try 40 variants on noise and the winner still scores well above zero.
The fix. Fix the procedure, not the parameter, and evaluate the procedure walk-forward (widget 5). Report the number of configurations you tried — it is part of the result. And treat parameter instability across folds as a finding: a lookback that jumps from 8 to 54 between folds is telling you there is no parameter to find.
Instant feedback, nothing recorded. Pick the best answer.
async def on_tick calls a fitted
estimator that takes 180 ms synchronously. What actually happens?Build one signal, backtest it on a recorded tape, run the same code live and account for the gap. Then add a stop loss and a throttle, and a latency histogram of your own decision path. Phase 1 goes live, and season scoring starts here.
make install. Labs and deliverables live in your course's own system.