qwen3-8b
each specialist's read, then the chair's reconciliation
fanned across the fleet's qwen replicas

healthcare · distributed inference

Tumor-board panel

A real tumor board puts six specialists in a room (radiology, pathology, medical and surgical oncology, radiation oncology, and molecular), each reading the same case from their own lens before a chair pulls it together. Those reads are independent, so we run them two ways at the same time: one node consulting the panel one specialist at a time, and the mesh consulting every specialist at once across its qwen replicas; then the chair reconciles them into one plan. The two lanes share the one fleet, so when the gateway puts a call from each on the same node, the frontend detects it and subtracts the wait. The speedup is the fan-out, not the two lanes colliding. Watch the clocks. Every token is served on-prem.
pre-warming the mesh (qwen + 2-stage chain)…

The case

The race

Loading cases…