qwen3-8b
per-document digest, then the reconcile into one briefing
fanned across the fleet's qwen replicas

healthcare · distributed inference

Distributed chart synthesis

A real referral arrives as a packet: a letter, an ED summary, a prior consult, labs, imaging, a med list. Each document can be digested independently, so the work is embarrassingly parallel. We run the same packet two ways at the same time: one node working through it document by document, and the mesh digesting every document at once across its qwen replicas; then both reconcile the digests into one briefing. The two lanes share the one fleet, so when the gateway lands a call from each on the same node, the frontend detects it and subtracts the wait. The speedup you see is the fan-out, not the two lanes colliding. Watch the clocks. Every token is served on-prem.
pre-warming the mesh (qwen + 2-stage chain)…

The packet

The race

Loading packets…