NVIDIA has published details on three cuTile kernels it built for the forward pass of Mamba-2, the state-space model architecture that has drawn attention as an alternative to transformer-based attention. The company says its optimized forward-pass code ran between 1.14x and 1.82x faster than a Triton reference implementation.
The disclosure is narrow — three kernels, one pass through the model — but it lands at a moment when teams running Mamba-2 at scale are hunting for any efficiency they can get inside GPU kernels rather than at the model level.
What the three kernels actually cover
NVIDIA's writeup describes the kernel set as covering the Mamba-2 forward pass, which is the inference-time path through the model. The comparison baseline is a Triton reference, the kind of implementation many research groups start from because Triton makes it relatively quick to express tiled GPU computations in Python rather than hand-written CUDA.
That choice of baseline matters. Triton references are widely used and reasonably optimized, so a speedup against one is a more meaningful claim than a speedup against naive code. NVIDIA put the range at 1.14x on the low end and 1.82x on the high end, meaning the gain varies by kernel and presumably by the shape of the workload.
The company didn't disclose which hardware the numbers were measured on, what sequence lengths or batch sizes were used, or how the three kernels divide the forward-pass work among themselves. Those details would determine how much of the speedup carries over to someone else's deployment.
Why the forward pass is where the argument gets settled
Mamba-2's pitch has always been about cost at inference. The architecture replaces the quadratic attention mechanism of transformers with a structured state-space recurrence, which in principle means the compute and memory footprint grow more gently with sequence length. Whether that promise holds depends heavily on how well the math maps onto GPU hardware.
Kernel-level work is where that mapping either succeeds or doesn't. A model that looks cheap on paper can lose its advantage if the implementation leaves the GPU underutilized or spends too much time shuttling data between memory tiers. The 1.14x-to-1.82x spread NVIDIA is reporting is the kind of range that usually reflects exactly those implementation details rather than the architecture itself.
NVIDIA has a clear commercial interest in making state-space models run well on its hardware. The more architectures that run efficiently on NVIDIA GPUs, the less pressure there is on customers to look elsewhere for inference capacity. That doesn't make the benchmark wrong, but it's context for reading a vendor's own performance numbers.
The cuTile angle
CuTile is the vehicle NVIDIA chose for this work. The naming signals a tile-based programming approach, which is a common way to organize GPU kernels so that data reuse happens in fast on-chip memory rather than through slower global memory reads. Tiling is not new, but how a vendor exposes it in its tooling shapes what kinds of kernels ordinary developers can realistically write.
NVIDIA framed the result as evidence that cuTile can express the Mamba-2 forward pass at competitive speed. For developers already working in NVIDIA's ecosystem, that's the practical takeaway: the tooling can handle this class of model without falling back to hand-tuned CUDA.
What's still missing before anyone can replicate it
The obvious next step is a reproducible benchmark. NVIDIA has given a speedup range and a baseline, but not the test conditions. Without the hardware configuration, the input shapes, and the exact Triton version used for comparison, independent teams can't check the numbers against their own workloads.
The other open question is whether the kernels will be released as usable code or remain a described result. A 1.82x speedup on a kernel nobody outside NVIDIA can run is a data point, not a tool. Developers evaluating Mamba-2 for production will want the implementation in hand — and will want to see how it holds up on sequence lengths and batch sizes that match their own traffic.
Until then, the result stands as a vendor-reported benchmark on one forward pass, with the low end of the range — 1.14x — the more conservative figure to plan around.
