Compute in Transit · results
Can an optical link compress gradients as they cross the network?
A photonic link module (arXiv:2608.21536) can transform data as it leaves a server. Could it compress training gradients on their way across the network? This study built the experiment and ran it: at this scale, no.
Loading the loss curves…
The answer, in one picture
Training across data centres is slowed by sending gradients over the wide-area network, so compressing them helps. The compression the optics can do (keep a fixed band of the transform) trains worse than the compression it cannot do (keep the largest values), and the simulation finds the case where computing in the link pays is narrow. Below is the evidence, and what the hardware would need instead. In the jargon: can a fixed band replace DeMo's top-k?
No. Top-k ends at 1.785 ± 0.007; the fixed low band at 3.124 ± 0.008, though it sends half top-k's bytes (13.9 MB against 27.7 MB a step on the wire). Even a random fixed set of positions (2.700, one seed) beats the low band, and the emulated optics, once repaired, does no better than 2.878. AdamW, the dense baseline, ends at 1.646 ± 0.005.
How this is checkedEvery vendored file is its source byte for byte at a pinned commit, the TypeScript port reproduces every cell of the simulation's results.md that it can, character for character, and the loss curves are recomputed from the run logs. How it is tested →
The question
DeMo compresses each worker's momentum by keeping the top-k coefficients of a chunked DCT. A photonic link module (arXiv:2608.21536) can run the transform but not the comparisons top-k needs; a fixed band of positions needs none. An experiment on a 15.8M-parameter model measured what the substitution costs, and a simulation asked when computing the band in the network could pay.
Why this exists
The goal: compress gradient traffic on the wire without sending indices, with the compression done for free in the link. A fixed frequency band needs no comparisons, so the optics can compute it in transit, and both ends already know which coefficients are sent.
What was found: a negative result at this scale. The measure is the model's validation loss after 3,000 training steps: its cross-entropy on held-out text, in nats per token (lower is better). Synced through the fixed low band, the model ends at 3.124 to 3.158, against 1.785 with top-k and 1.646 with plain AdamW, and a pluggable module computing the band never breaks even with AdamW in the simulation. The findings and the proposed system, end to end are below.
I'd read online about the possibility of computing on data in transit, doing useful work on it as it crosses the network, for LLM processing. I wanted to run a serious investigation in this area, and go through the real process in detail.
It's an appealing idea, and an easy one to sketch on a slide. I didn't want a sketch. I wanted one real question taken through the whole process: could a photonic link module (arXiv:2608.21536) take over the compression step of distributed training, and if so, when would it pay? That meant every stage, including the one that went wrong, with every number traceable to a recorded result and every limit stated.
The process, step by step
- Read the sources, and check themThe photonic module's paper and DeMo, claim by claim, including what the paper leaves open: whether the optical transform counts against the compute budget.Deck 01 →
- Form a falsifiable hypothesisA fixed band of DCT positions, which needs no comparisons, converges as well as DeMo's top-k.Deck 04 →
- Build the experiment, with testsDeMo, the fixed bands and an emulated optics path, written from scratch, with their own tests.The repository →
- Run it on real GPUs33 runs on Colab GPUs, from one notebook that resumes after a disconnect./notebook →
- Find the failureEvery one of the first optics design's 6 runs diverged.went wrong · /optics →
- Diagnose it, fix it, re-runError feedback was removing the exact band, not what the optics sent. Feeding back the values sent fixed it.put right · /optics →
- Turn steps into time and energyA discrete-event simulation of training across data centres, from the measured steps and bytes./transit →
- Publish every number, with the limitsEach figure read from recorded results; what is illustrative, hypothetical or extrapolated says so./method →
What it found
3.124–3.158
final validation loss; top-k 1.785, AdamW 1.646
A fixed band is no substitute for top-k at this scale
The fixed low band's final validation losses are 3.124 to 3.158 across three chunk shapes, against 1.785 for top-k and 1.646 for AdamW.
0.013 vs 0.643
share of momentum energy kept: low band vs top-k
Why: the band misses where the energy is
On real momentum, in 2-D 64x64 chunks, the low band keeps 0.013 of the energy, against top-k's 0.643; a random set of positions keeps 0.065. In 1-D 256-point chunks it does better (0.146 against 0.429), and still loses.
11.153–558.834
final validation loss of the first optics runs; repaired 2.878–3.423
The emulated optics first diverged, then trained
All 6 runs of the first design diverged, clipping 22.59% to 57.50% of values. Feeding back the values actually sent fixed it: the repaired runs ended at 2.878 to 3.423. That the best of them beat the exact band (3.141) is a hypothesis: one seed per run.
hypothesis, not a resultThe detail →
ENOB 8
the break-even
ENOB 6 never reaches the target; 8 is the break-even
At ENOB 6 the repaired optics never reaches 3.158. Where an in-transit variant wins at all, the break-even ENOB is 8, the lowest measured that trains.
1.57 GB/s
below which switch summing might beat AdamW
A pluggable module never breaks even; a switch might
Computing the band in a pluggable module never beats AdamW at any WAN bandwidth tested (0.1 to 1,000 GB/s). Summing it at a switch might, below 1.57 GB/s (0.83 on the exact band's steps). But a GPU ring all-reduce of the band, with no module, beats AdamW below 6.58 GB/s, and on the exact band's steps the switch never beats it.
4.0x, not 16.0x
fewer bytes on the wire than AdamW
An all-gather gives back most of the saving
The band's payload is 16.0x smaller than AdamW's, but an all-gather's wire bytes grow with the workers: on 8 workers each sends 13,854,288 bytes a step, only 4.0x fewer than AdamW's all-reduce (55,417,152).
A negative result is still a result
The main hypothesis mostly failed. A fixed band did not replace top-k, and computing it in a pluggable module did not pay at any WAN bandwidth tested. That is worth knowing, and worth knowing early. It rules out a direction cheaply, with a 15.8M-parameter model on rented GPUs, before anyone builds hardware for it. It says what the hardware would need instead: comparisons, so it can select top-k, or summing at the switch, and an optical transform that is not charged against its compute budget. It turns a vague promise into specific questions, such as whether the repaired optics' advantage survives more seeds, and whether the measured steps hold at a larger scale. And the experiment, the simulator and the notebook are there to test the next idea.
Final validation loss
- A · AdamW55.4 MB a step on the wire (the dense baseline)1.646 ± 0.005
- B · top-k27.7 MB a step on the wire, 2.0x less than A1.785 ± 0.007
- C · low band, 1-D 25613.9 MB a step on the wire, 4.0x less than A3.124 ± 0.008
- C · low band, 2-D 64×6413.9 MB a step on the wire, 4.0x less than A3.158 ± 0.009
- D · calibrated band13.9 MB a step on the wire, 4.0x less than A3.087 ± 0.027
- F · random band13.9 MB a step on the wire, 4.0x less than A2.700
- E · ENOB 8, 8 planes13.9 MB a step on the wire, 4.0x less than A3.017
- E · ENOB 10, 8 planes13.9 MB a step on the wire, 4.0x less than A2.878
The proposed system, end to end
Variant E, from a worker's gradient to the weight update on the receiving GPU: the delta leaves the TX GPU in 1-D chunks of 256, the module writes each chunk onto light as bit planes, the optics transforms it, and only the band's 16 outputs are read and sent: values, no indices. The receivers place them at the fixed positions, average, invert the DCT and apply the sign; each sender feeds back what it actually sent. Switch to E′ to see the band computed and summed at the switch instead, the simulation's hypothetical variant. Each block is tagged by where it comes from: the module's paper, the experiment's model, the simulation, or an assumption.
Loading the chain animation…
Why optics rather than digital logic in the NIC?
It is the first question a hardware architect asks, and this study does not settle it in either direction. What it can say:
- The arithmetic is not large. At the module's port rate (12.5 billion BF16 values a second), computing the band directly takes 16 multiply-adds per value (400 GFLOP/s), and a fast transform of the whole 256-value chunk 4 (100 GFLOP/s): the same order as the 10 to 99 GFLOP/s budget the simulation gives the module. A NIC, DPU or switch with a dedicated block of that rate is a plausible design; its area and energy are not in the sources used, so they are not costed here.
- Digital logic can do what the optics cannot. A digital block can compare, so it could keep the largest values (top-k), which this experiment found trains better than any fixed band (1.785 ± 0.007 against 3.124 ± 0.008).
- Even without new hardware the GPUs can do it. The band computed on the GPUs and summed by an ordinary ring all-reduce (C′ in the simulation) beats AdamW below 6.58 GB/s (52.7 Gb/s) of WAN, and on the exact band's steps the switch-summed module case (E′) never beats it.
Where optics could still win is outside what this study tested: energy per operation for much larger transforms, or doing the work in a link that is already there. Testing those cases would need measured figures for both sides.
What the module could and couldn't do
The simulation (Training_Transit_Sim) takes the measured steps to a loss of 3.158 and asks how long each way of syncing takes on 8 workers in two data centres joined by a 100 GbE WAN. A pluggable module computing the band as the delta leaves each worker (E) is slower than doing it on the GPU; summing the band at the switch (E′) is the case that can pay, below a few GB/s of WAN.
- A · AdamW, dense ring all-reduce400 measured steps1.63 s
- B · DeMo top-k, all-gather400 measured steps0.93 s
- C · fixed band on the GPU, all-gather2,667 measured steps3.85 s
- C′ · fixed band on the GPU, ring all-reducehypothetical: not run2.16 s
- E · band in transit (pluggable), all-gather1,800 measured steps7.10 s
- E′ · band in transit, summed at the switchhypothetical: not run6.71 s
- The pluggable all-gather (E) never breaks even with AdamW from 0.1 to 1,000 GB/s of WAN (none in range): its all-gather still sends a quarter of AdamW's wire bytes, and it needs more steps.
- E′ beats AdamW below 1.57 GB/s (12.5 Gb/s) with E's steps; with the exact band's steps, below 0.83 GB/s (6.64 Gb/s).
- The same summation without a module (C′, the band ring all-reduced by the GPUs) beats AdamW below 6.58 GB/s (52.7 Gb/s), and on the exact band's steps E′ never beats C′ (none in range).
The series
The Compute in Transit hub has the glossary and the code. Its five decks: