NeuroTrainer
VR tool for neurosurgeons and radiologists at HUG to explore MRI and CT volumes in real time. Custom HLSL shader for 3D texture rendering; streaming architecture to handle 100 GB+ datasets without VR latency.
Challenge
Two problems defined this project.
Rendering MRI volumes in VR. MRI and CT data isn't a stack of images; it's a 3D texture that needs to be sampled along arbitrary ray paths in real time. No standard Unity shader handles this. I wrote a custom HLSL shader that performs direct volume rendering: ray-casting through the 3D texture, compositing density values into colour, and mapping transfer functions for contrast. Getting it to run at 90 fps in VR, on hardware a hospital could actually buy, required careful optimisation of the sampling step and early-exit conditions.
Streaming datasets without lag. A single MRI scan can exceed 100 GB. Loading it into memory isn't an option in VR; a frame drop at the wrong moment causes immediate nausea. I built a streaming architecture that loads volumetric slabs on demand, pre-fetching the next region by spatial proximity before it's needed. The result: continuous navigation through the full dataset with no perceptible load stutter.
Building for VR in a clinical context taught me something about constraints: the hardware ceiling is low, the tolerance for error is zero, and the people using it have no patience for UX that gets in the way of their work.
In use
Built with HUG (Hôpitaux Universitaires de Genève), where clinicians use it to explore MRI and CT volumes in three dimensions rather than as flat 2D slices.