How it was run, and what it does not show

DeMo_Fixed_Band_Experiment implements DeMo (Peng et al.) with its top-k selection replaced by fixed bands of transform coefficients, and a band computed by an emulated optical transform, because a photonic link module (Kundu et al., arXiv:2608.21536) lists transforms and arithmetic but no comparisons. From its results.md:

Hardware: x86_64, 2 threads, PyTorch 2.11.0+cu130, Python 3.13.15. Model: 15,833,472 parameters, 8 layers, width 384, context 256. 8 simulated workers, 8 sequences each per step, 3000 steps. Data: byte-level BPE, 4096 tokens.

Bytes are per worker per step, values in BF16 (2 bytes). Payload = what a worker originates; top-k indices are counted at the bytes needed for a position within a chunk (the reference code sends int64, shown separately). Wire = bytes a worker sends in a ring collective (all-reduce for A, all-gather for the DeMo variants); wire vs dense compares it with a ring all-reduce of the dense gradient. An all-gather's wire bytes grow with the number of workers, an all-reduce's do not, so the wire ratio is smaller than the payload ratio.

Every configuration ran 3,000 steps on 8 simulated workers, evaluated every 200 steps. 5 configurations have three seeds; the rest one. Learning rates came from a short sweep on AdamW (0.003) and top-k (0.001), the latter reused for every DeMo variant.

Training_Transit_Sim is a SimPy discrete-event model with closed forms it must agree with; devices and links come from Disaggregated_Inference_Sim (pinned at 69ffd2b). 15,833,472 parameters, 8 workers, 8 sequences of 256 tokens per worker per step (2,048 tokens), an H100 per worker at an MFU of 0.40 (illustrative). Target loss for the headline tables: 3.158 (the 2-D fixed band's final loss), the one ladder loss that the dense, top-k, exact-band and optics variants all reach.

This site runs the closed forms only, ported to TypeScript. At zero background load the simulation's discrete-event engine reproduces them to rounding, and the port reproduces the engine's recorded results (results.md sections 1 to 8, and the extrapolations of section 10). The simulation's queueing on a shared WAN (section 9) needs the engine, so the transit page shows its recorded runs instead, and says so.

• One model size (15,833,472 parameters), 8 workers and 3,000 steps. Steps to a loss were measured only there; the simulation's larger models and DiLoCo-style syncing are extrapolations, and the transit page labels them so (extrapolations).
• The learning rate was tuned only on top-k and AdamW, and reused for every fixed-band variant. AdamW's rate (0.003) is the higher of the two tried, so AdamW might do slightly better still.
• Single seeds for E, F and the 1-D 64-point band. That the repaired optics band beats the exact band (2.878 at ENOB 10 against 3.124) is a hypothesis, so every in-transit result is also given on the exact band's steps.
• The optics noise model is the series' own, not measured on the module.
• Steps to a loss are resolved to the evaluation interval (200 steps).
• In the simulation, compute and communication do not overlap, and a pluggable's processing and its collective run one after the other: both conservative for the variants they touch.
• C′ and E′ were not run: they assume that summing the band gives the same update as gathering it.
• GPU runs are close but not bit-identical between runs, and they ran on several GPU types.

The module's figures and their sources are listed in the simulation's hardware.py; claims taken from the developer's website are cited as such there.

• Every vendored file is byte for byte its source at a pinned commit (the table below); CI re-fetches them and fails on any difference, and re-runs the simulation's Python package to regenerate the port's constants and parity fixtures.
• The TypeScript port equals the simulation's closed forms exactly, and its engine to rounding, on hundreds of configurations; formatted as results.py formats them, its numbers equal every cell of results.md it can reproduce, character for character (step times, times to the target, hot spots, energy, stage times, every break-even).
• The loss curves, the ladder, the energy averages and the optics table are recomputed from the run logs and checked against the experiment's results.json and results.md.
• A test greps the source and the build output for the module developer's name, which must not appear.

The source, the tests and the scripts are in compute-in-transit-results (RUNBOOK.md has the update procedure).