The fastest instruction is the one you never execute. Last week you gave the compiler the types; this week you give it the tables, the dispatch and the configuration too — and then you test the zero-overhead claim instead of repeating it, because compile-time design has a real bill: build time, debuggability, and the flag you can no longer flip at 09:31.
constexpr and consteval, and prove it with static_assert — then explain why a constexpr function is not a promise.std::variant and a compile-time visitor, and with tag dispatch, and price all four designs against each other.constexpr table, a policy, a visitor
HW6: static vs dynamic dispatch, shown in -O2 -S
Project Phase 1: no virtual on the receive path
Your compiler is a computer, and it is idle while your bot is trading. Anything it can
finish during the build — a conversion table, a tick ladder, a decision about which handler to call —
is work your hot path never does. The three keywords that let you say so are
constexpr, consteval and static_assert, and the difference
between the first two is the difference between a permission and a
guarantee.
constexpr — mayA function that can be evaluated at compile time when its arguments are constant expressions, and that compiles to an ordinary runtime function otherwise. Nothing forces the compile-time path. Call it with a variable read from a socket and it runs, at run time, exactly as slowly as you feared.
consteval — mustAn immediate function (C++20). It must be evaluated at compile time; calling it with a runtime value is a compile error, and no runtime copy of it is ever emitted. Use it when a runtime evaluation would be a bug rather than a slowdown — parsing a symbol table, building a fee ladder, checking a wire layout.
static_assert — proveThe cheapest test you will ever write, because it runs during the build and costs
nothing afterwards. Assert tick sizes, sizeof, alignof, wire offsets and
table contents. A violated assumption becomes a failed build instead of a failed session.
The same tick-size table, four ways. Watch the marker move
along the program's lifetime — build, link, load, main, first tick — and read what each
choice costs where.
mainstatic_assertableconstexpr int t = ticks_from_price(px); is compile-time because the
variable is constexpr. int t =
ticks_from_price(px); with the same function is a normal call. If you want the guarantee,
either declare the destination constexpr, or declare the function
consteval.Every design in this section answers one question — given a message,
which code runs? — and they differ only in when the question is answered.
Virtual answers it per call, from memory. A switch answers it per call, from a tag.
std::visit and CRTP answer it while compiling.
Press Step to walk a single call through both
designs. The virtual lane has six stages and two of them are dependent memory loads; the CRTP lane
has one, and the static_cast emits no instruction at all.
// what we are replacing: a vtable per object struct IStrategy { virtual double signal(const Book&) const = 0; virtual ~IStrategy() = default; }; struct VMomentum : IStrategy { double signal(const Book& b) const override { return b.obi * 0.5; } };
// CRTP: the base is templated on its own derived type template <class Derived> struct Strategy { double signal(const Book& b) const { return static_cast<const Derived*>(this)->signal_impl(b); } // no vtable, no load }; struct Momentum : Strategy<Momentum> { double signal_impl(const Book& b) const { return b.obi * 0.5; } }; struct MeanRevert : Strategy<MeanRevert> { double signal_impl(const Book& b) const { return -b.obi * 0.25; } }; // generic over strategies — and NO common base pointer template <class S> double tick(const Strategy<S>& s, const Book& b) { return s.signal(b); // resolves & inlines }
std::vector<Strategy*>, because Strategy<Momentum> and
Strategy<MeanRevert> are unrelated types. If you need a heterogeneous
container chosen from a config file, you need type erasure — and that is a legitimate reason to
keep a virtual interface at the edge.Four designs, the same closed set of messages, the same work per message. p50 is where the abstraction shows; p99 is where the branch predictor does. Raise the number of alternatives and shuffle the mix to see which designs care.
switch on a tag: +0.25 bounds test, +0.55 jump table (an indirect jump — it
mispredicts too), +0.90 lost inlining across the arms. std::visit: +0.35 for the
variant's own index dispatch, arms inlined. CRTP: 0. Mispredicts cost 15 cycles at
4.0 GHz and land almost entirely in p99. Directional — measure your own.| design | ns/msg p50 | ns/msg p99 | ns/tick p50 | mispredict | zero-overhead? |
|---|
std::variant + compile-time visitorusing Msg = std::variant<BookUpdate, Fill, SessionEvt>; struct Router { // one visitor, N arms template <class T> double operator()(const T& m) const { if constexpr (std::is_same_v<T, BookUpdate>) return decide(m); else if constexpr (std::is_same_v<T, Fill>) return track(m); else if constexpr (std::is_same_v<T, SessionEvt>) return flag(m); else static_assert(always_false<T>, "unhandled message"); } }; double on_msg(const Msg& m) { return std::visit(Router{}, m); } // or, with the classic lambda overload set: template <class... Fs> struct overload : Fs... { using Fs::operator()...; }; template <class... Fs> overload(Fs...) -> overload<Fs...>;
static_assert arm matters. Add
CancelOrder to the variant and this fails to build with your message. A
[](auto&){} catch-all would have compiled and silently dropped it — which is the
same class of bug as a default: that does nothing.Policy-based design is Alexandrescu's answer to the deep virtual hierarchy: a host class takes its behaviour as template parameters, one per axis of variation, and the compiler stitches them together and inlines through the joins. Turning a feature off does not set a flag — it changes a type, and the code for that feature stops existing.
Compose MdHandler from five policies and watch
the emitted hot path shrink line by line. The struck-out lines are not skipped at run time — they are
not there.
Now the other side of the ledger. Policies multiply: every axis you add multiplies the set of types your build system, your error messages and your debugger have to cope with. And a type is not a runtime knob.
-Og and [[gnu::noinline]] exist for exactly this afternoon.Compile-time on the hot path, run time at the edges. The receive
path, the codec and the signal are a closed set you know at build time — template them, compose
them, inline them, and let static_assert guard the layout. Configuration loading,
the reporting tab, the shutdown sequence and anything an operator touches can be as virtual and
as dynamic as you like, because none of it runs per tick.
When you need both — a strategy chosen from a config file and an
inlined hot path — put the type erasure at the boundary: pick the concrete
Bot<Policies…> once at startup behind one virtual run(), and pay
that indirection exactly once per session instead of once per message.
shared/messages.py is a pydantic discriminated union keyed on a
type field. Mirror the closed set in C++ as a std::variant, dispatch
with the if constexpr visitor above, and the hand-rolled
switch (msg.type) disappears from the receive path — along with the
default: that used to swallow the message you forgot.
Then compose the handler: NoLog and NoChecksum in the
session build, StderrLog and Crc32 in the replay build, from the same
source. Re-run scripts/latency_replay.py and
scripts/latency_report.py before and after, and write down what happened to p99.9 —
including “nothing”, if that is the honest answer.
These are the five that cost marks on HW6, every time. Open them.
constexpr means “at compile time”constexpr on a function is a permission, not an instruction. It
says “this may be evaluated at compile time if the arguments allow it”. Call
ticks_from_price(px) where px came off a socket and you get an ordinary
function call, every tick, with the floating multiply and the rounding you thought you had
pre-computed. Nothing warns you; the code is correct, it is just running in the wrong era.
The same illusion with tables: const std::array<double,256> T = make_ticks();
where make_ticks is constexpr may still be a
dynamic initialiser that runs before main — and if two
translation units do that, you also inherit the static-initialisation-order problem.
Fix: force it. Declare the destination constexpr auto TICKS =
make_ticks(); — now it must be a constant expression or the build fails. Where a runtime
evaluation would be a bug, use consteval. And prove it:
static_assert(TICKS[0] == 1); only compiles if the value really exists at build time.
The classic. Your base is template <class D> struct Strategy { double signal(const
Book& b) const { return static_cast<const D*>(this)->signal(b); } }; — base
and derived hook share the same name. A derived class that overrides
signal works. A derived class that forgets — or that misspells it, or gets the
const qualification wrong so it does not hide the base — resolves the call back to
Strategy::signal, which calls it again. It compiles cleanly and blows the stack on
the first tick.
The neighbouring mistake is forgetting that the base is templated on the derived type at all:
struct Momentum : Strategy<MeanRevert> compiles, and the
static_cast then reinterprets your object as an unrelated type. That is undefined
behaviour with no diagnostic.
Fix: give the hook a different name — signal in the base,
signal_impl in the derived — so a missing override is a compile error rather than a
loop. Then assert the relationship in the base:
static_assert(std::is_base_of_v<Strategy<D>, D>), in a member function
body so D is complete by then. Better still, constrain the derived type with a
concept that requires signal_impl.
std::visit(overload{ …, [](auto&){} }, m) looks tidy and it disables the one
guarantee you switched to a variant for. The whole point of a closed set is that adding
CancelOrder to the union should break the build until you handle
it. A generic lambda matches it, does nothing, and the message vanishes — the exact failure mode
of a default: label that silently returns.
Two more edges worth knowing. std::visit on a variant whose alternatives are not
all handled with the same return type will not compile at all, which is good. And a variant can
become valueless by exception if an alternative's move constructor throws
mid-assignment; std::visit then throws std::bad_variant_access. Keep
hot-path alternatives trivially copyable and that state is unreachable.
Fix: no auto arm on the hot path. Either write one lambda per
alternative, or use the if constexpr chain with
else static_assert(always_false<T>, "unhandled message") — the dependent-false
idiom, because a bare static_assert(false) fires even in discarded arms.
Five policy axes with three options each is 243 distinct instantiations of your handler. Your CI builds four of them. The other 239 are code that has never been compiled, which means they have never been type-checked either — a policy combination nobody tried can fail to build months later, at the moment somebody needs it.
The build cost is the visible half: each combination is a fresh compile of the whole host
class, and template-heavy translation units are where minutes go. The invisible half is the
mangled names, which get long enough to make a stack trace unreadable and a
nm listing useless.
Fix: keep the axes genuinely orthogonal and few — if two policies are always
changed together, they are one policy. Give every axis a default so the common instantiation is
one word. Constrain each axis with a concept so a wrong policy fails at the parameter with its own
name. Then static_assert-instantiate the combinations you actually ship, in one small
translation unit, so CI type-checks them all.
A fully inlined, policy-composed hot path has no stack frames. There is nothing to step into,
no symbol to breakpoint, and the profiler attributes every cycle to one enormous
on_packet. Meanwhile a deep template error prints the instantiation stack instead of
the mistake, and a constexpr loop that got too big fails with
constexpr evaluation hit maximum step limit, which reads like a compiler bug
and is not.
The operational version of this is worse than an afternoon. Logging chosen by a policy cannot be turned on during the session it would have explained. If an on-call engineer might need it, compile-time was the wrong place for it.
Fix: keep a -Og build with [[gnu::noinline]] on the
handlers for the afternoons you need frames, and check its behaviour matches. Raise
-fconstexpr-steps / -fconstexpr-ops-limit when a table legitimately
needs it. And draw the line explicitly: anything an operator flips is a runtime flag, read once
per session, off the hot path — never a template parameter.
Three questions, instant feedback, nothing recorded.
Q1constexpr std::int64_t ticks_from_price(double px); called as auto t = ticks_from_price(msg.px);, where msg came off a socket. What happens?
Q2Your CRTP base calls static_cast<const D*>(this)->signal(b) from a method also named signal. A new derived class forgets to define signal. What happens?
Q3You composed your handler from policies, including NoLog. Mid-session, an on-call engineer needs verbose logging to diagnose a stuck order. What is the honest answer?
-O2 -S: the CRTP call becomes a plain multiply, the virtual one keeps its indirect
call.constexpr table with a constexpr constructor, proven by
static_assert(kTicks.px[50] == 100.5).Quoter<FeePolicy>, instantiated two ways, and the observation that the fee
is folded into the arithmetic rather than loaded.std::variant receive path with the overload helper — then add an
alternative and confirm the build breaks.constexpr lookup table guarded by static_assert, that
the program actually uses.make test runs the grader over everything you have built so far.