CI/CD, Testing & UI — then the Season Finale

A test is an executable claim about your code. pytest discovers every function named test_*, runs it, and reports which claims still hold. That is all. The engineering judgement is not in writing asserts — it is in choosing which claim to make at which layer.

The pyramid is that judgement made visible. Many unit tests at the base, each pinning one function in milliseconds with no I/O. Fewer integration tests, wiring two or three components together with fakes instead of the network. A handful of end-to-end runs that drive the whole path — bot to exchange to fill to P&L — and catch the wiring bugs nothing else can see, slowly and a little flakily.

Push logic down. A bug caught by a unit test costs you seconds; the same bug caught end-to-end costs a minute of CI; the same bug caught live costs you a session, in public.

# tests/test_signal.py
from strategy import momentum_signal

def test_momentum_goes_long_on_uptrend():
    prices = [100, 101, 102, 103, 104]
    sig = momentum_signal(prices, lookback=3)
    assert sig.side == 'buy'
    assert sig.qty > 0

def test_momentum_flat_when_no_trend():
    assert momentum_signal([100, 100, 100], 3) is None

A GitHub Actions workflow is YAML in .github/workflows/. An event (on: [push]) triggers jobs; each job runs ordered steps on a fresh runner; a matrix fans the same job out across Python versions. Nothing in it is clever — and that is the point. The runner has no internet by design, no cached state, and none of your local environment. If your suite passes there, it passes anywhere.

on: [push, pull_request]        # trigger
jobs:
  test:                         # a job
    runs-on: ubuntu-latest      # the runner
    strategy:
      matrix:
        python: ['3.11', '3.12']  # fan out
    steps:                      # ordered
      - uses: actions/checkout@v4
      - run: make install
      - run: pytest -q

Reproducibility is the other half. Same code plus same dependencies plus same OS equals same behaviour, so pin exact versions (==, never >=), isolate in a venv, and freeze the whole environment in an image. Order the Docker layers stable-to-volatile so the pip install layer stays cached and only your code layer rebuilds.

# requirements.txt — pin exact versions
websockets==12.0
pydantic==2.7.1

# Dockerfile — freeze the environment
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .         # cached layer
RUN pip install -r requirements.txt
COPY . .                        # code layer
CMD ["python", "-m", "team.trader"]

Shipping is not the end of the job. A running bot needs configuration you can change without a rebuild (ports, poll intervals — plain env vars), secrets that never enter git or the image (tokens injected at runtime), and observability: structured logs for what happened, metrics for what is happening now.

A dashboard is metrics made glanceable, and a trading dashboard has four non-negotiable panes. Positions — what do I own, and is it what I think I own. P&L — realized, unrealized, and the shape of the curve. Risk — how close am I to a limit, and what is my worst case right now. Connectivity — is the socket up, is the feed fresh, am I being throttled. Miss the fourth pane and you will confidently quote on a price that stopped updating four minutes ago.

The arena's own UI is exactly this: teacher/web_dashboard.py with MARKET (live books), FLOW (who trades with whom), STATS and SEASON (per-team analytics and standings). The deploy kit behind it — systemd units, Caddy for TLS, Docker for the environment — is how the hosted arena becomes reproducible from bare metal up.

Unit correctness means each object honours its own contract in isolation — roughly 1,280 tests do that for the platform. Integration correctness means the objects honour each other's contracts when wired together, live, under load. The second does not follow from the first.

The classic seam bugs all have the same shape: two correct components disagree about timing or units. The signal fires faster than risk can veto it. A fill arrives before its ack. Two messages race on the same book. One side sends a price as a float, the other rounds to ticks — and both are right by their own spec.

In AlgoArena the seams are explicit, which is the whole reason the wire protocol is Pydantic. Three processes speak only models from shared/messages.py; if a message round-trips, the seam holds. So test the conversation, not just the speakers.

A tearsheet is one page that answers two questions: did this make money, and how much risk did it take to do it? The equity curve tells the story before any number does — a smooth 20% beats a jagged 40%, because the second is one bad tick from a blow-up. Then the numbers: Sharpe (return per unit of risk), max drawdown (the worst pain an investor had to sit through), Calmar, turnover, exposure.

# exchange/scoring.py  (score_equity)
r      = returns(equity)        # per-snapshot returns
vol    = pstdev(r)              # volatility of the book
mdd    = max_drawdown(equity)   # worst peak-to-trough
excess = mean(r) - RISK_FREE    # above idle cash
ratio  = excess / max(vol, VOL_FLOOR)
score  = ratio * max(0, 1 - PENALTY*mdd)   # PENALTY = 2.0
# ranked PER ROLE: traders vs traders, brokers vs brokers