GPU image reconstruction for a 3D ultrasound scanner

The Problem

A 3D ultrasound imaging company’s CUDA-based image reconstruction pipeline was the critical path for turning raw transducer data into a usable scan. The transmission reconstruction stage — a large iterative CUDA codebase built around 2D FFTs — was running well below its theoretical potential, but the team had no reliable way to tell whether a given code change made performance better or worse, and no automated way to catch a correctness regression before it shipped.

Our Approach

We started with a code and algorithm review of the reconstruction pipeline, then instrumented it to produce detailed per-stage performance data and added verification stages that check every computation against “golden” reference output. We stood up a Jenkins CI server so that performance and correctness were tracked automatically on every commit, turning what had been a black box into a system the client’s own engineers could iterate against with confidence.

With that instrumentation in place, profiling showed the pipeline was memory-bandwidth bound and dominated by 2D FFTs, with CUFFT running at roughly half of NVIDIA’s published throughput because of an undersized batch size, plus sub-linear scaling across multiple GPUs. We shipped a multi-GPU scaling fix, along with benchmarking data and a prioritized list of further software and hardware optimizations (larger FFT batches via parallel view processing, folding element-wise updates into the FFT via CUFFT callbacks, and eliminating CPU overhead).

The client brought us back years later, after the codebase had grown substantially on newer GPU hardware, for a second round of profiling. Using Nsight-level kernel timing, we confirmed the pipeline was now hitting close to optimal FFT throughput and 40-60% of peak memory bandwidth, found and fixed a memory-allocation bug that freed several GB of GPU memory, and traced a drop-off in performance at high parallelism to GPU L2 cache size — giving the client a formula for the optimal batch size on new GPU architectures. We also flagged that their hand-rolled GPU memory manager was adding overhead without benefit, and recommended replacing it with direct allocation calls.

Results

Both engagements left the client with more than a one-time speedup: a CI pipeline that tracks performance and correctness on every commit, a standalone FFT benchmarking tool for evaluating new GPU hardware before buying it, and a clear, quantified picture of where the remaining headroom is — and isn’t — in their reconstruction code.