- qwen3-8b
- per-document digest, then the reconcile into one briefing
- fanned across the fleet's qwen replicas
healthcare · distributed inference
Distributed chart synthesis
A real referral arrives as a packet: a letter, an ED summary, a prior consult, labs, imaging, a med list. Each document can be digested independently, so the work is embarrassingly parallel. We run the same packet two ways at the same time: one node working through it document by document, and the mesh digesting every document at once across its qwen replicas; then both reconcile the digests into one briefing. The two lanes share the one fleet, so when the gateway lands a call from each on the same node, the frontend detects it and subtracts the wait. The speedup you see is the fan-out, not the two lanes colliding. Watch the clocks. Every token is served on-prem.
pre-warming the mesh (qwen + 2-stage chain)…
The packet
The race
Loading packets…