NVIDIA CUDA Rust: SIMT cuda-oxide and Safe Tile-Based cutile-rs

On September 8, 2026, NVIDIA announced CUDA Rust, an official toolchain for writing and compiling GPU kernels directly in Rust. At RustConf 2026 held on the same day, NVIDIA also joined the Rust Foundation as a Platinum Member.

NVIDIA has previously expanded its Rust adoption in the Nova driver for Linux distributions and the core implementation of the NVIDIA Dynamo inference infrastructure. With this release, NVIDIA provides an official environment to author GPU compute kernels natively in Rust, treating Rust as a first-class language within the CUDA platform alongside C++ and Python.

The toolchain consists of two tracks: cuda-oxide for low-level thread control, and cutile-rs for memory-safe parallel execution over data blocks (tiles). This post breaks down how they differ from earlier third-party crates, their compiler architectures, and how their type systems enforce memory safety.

Differences from Earlier Rust GPU Approaches

Prior to this announcement, using NVIDIA GPUs from Rust typically relied on three approaches.

The first approach uses host-side API wrappers such as cudarc. The GPU kernel itself is written in CUDA C++ and precompiled into PTX or cubin binaries using NVIDIA’s nvcc compiler. The Rust host program loads these binaries via the CUDA Driver API and launches kernels by passing pointers. While this handles host-side memory in Rust, writing kernel logic still requires C++, splitting the development pipeline across two languages.

The second approach relies on community-driven experimental compilers like rust-cuda. These projects attempted to generate PTX directly from Rust using LLVM’s NVPTX backend. However, keeping up with breaking internal API changes in nightly Rust and supporting newer GPU architectures required heavy maintenance, making long-term production use difficult.

The third approach uses FFI bindings to call CUDA C++ functions via extern "C". Projects automate the build process using the cc crate or CMake inside build.rs, but this approach cannot guarantee type consistency or memory safety across the host-device boundary.

flowchart TD
    A["Legacy Setup: CUDA C++ (.cu)"] -->|Precompile with nvcc| B["PTX / cubin binary"]
    C["Rust Host Code (.rs)"] -->|Load via Driver API| B
    B --> D["GPU Execution (Split across two languages)"]

    E["CUDA Rust: Single Rust Codebase (.rs)"] -->|cuda-oxide / cutile-rs| F["Direct PTX Codegen / JIT Execution"]
    F --> G["GPU Execution (Pure Rust)"]

CUDA Rust resolves this split under official NVIDIA support. Instead of requiring C++ wrappers or external nvcc build steps, it generates PTX code directly from Rust source. Developers get a single-source workflow where host code and device kernels live in the same crate, fully integrated with Cargo.

Low-Level Thread Control: The cuda-oxide SIMT Track

cuda-oxide is a low-level track that adopts the same SIMT (Single Instruction, Multiple Threads) execution model as CUDA C++. It gives developers direct control over individual GPU threads, making it suitable for explicit shared memory management and thread synchronization.

Compilation Pipeline and Architecture

cuda-oxide operates as a custom code generator (codegen backend) for the Rust compiler (rustc). The compilation pipeline emits PTX through the following stages:

1. rustc parses Rust source code and generates Mid-level Intermediate Representation (MIR).
2. cuda-oxide intercepts the MIR for kernel functions marked with #[kernel].
3. It lowers the MIR to a GPU-targeted intermediate representation through Pliron, an MLIR-based intermediate representation framework.
4. It generates native PTX instructions directly via LLVM’s NVPTX backend.

Because cuda-oxide emits PTX directly from MIR without invoking an external C++ compiler, Rust language features—including generics, traits, and closures—can expand directly inside kernel code.

Kernel Syntax and Code Structure

In cuda-oxide, kernels are defined inside a module annotated with #[cuda_module], and each kernel function is marked with #[kernel].

use cuda_core::simt::LaunchConfig;
use cuda_device::{cuda_module, kernel};

#[cuda_module]
mod kernels {
    use super::*;

    #[kernel]
    pub fn vector_add<T: Copy + std::ops::Add<Output = T>>(
        a: &[T],
        b: &[T],
        c: &mut [T],
    ) {
        let idx = cuda_core::thread::index();
        if idx < c.len() {
            c[idx] = a[idx] + b[idx];
        }
    }
}

Developers retrieve thread indices via cuda_core::thread::index(), and array bounds checks and element additions use standard Rust syntax. Kernel definitions can reside in the same file as host-side logic, and type parameter T monomorphizes into concrete types at compile time.

Development Status and Requirements

As of September 2026, cuda-oxide is in early alpha. Because it hooks directly into rustc internal data structures as a custom codegen backend, it requires a pinned nightly Rust toolchain and a dedicated LLVM build. It is not yet merged into the stable Rust toolchain, meaning breaking changes from upstream compiler internals remain common.

cutile-rs: Safe Data-Chunk Processing via the Tile Track

cutile-rs is a high-level track built on the Tile programming model, expressing parallelism over blocks of data (tiles) rather than individual threads. It abstracts low-level thread scheduling and hardware-specific layout arithmetic, enforcing memory safety at the language level.

Extending the Ownership Model with Tensor Partitioning

In standard CUDA, data races occur easily when multiple threads access an array through shared raw pointers and write to overlapping regions. cutile-rs extends Rust’s borrow checker to the GPU execution boundary.

When allocating a tensor on the host, developers use .partition() to split the tensor into non-overlapping, disjoint sub-tensors. The mutable reference &mut Tensor passed into the kernel is treated by the type system as an isolated block with guaranteed exclusive access. This eliminates data races from concurrent writes across GPU workers at compile time, enabling memory-safe kernels without unsafe blocks.

Kernel Syntax and Host Integration

cutile-rs relies on declarative macros and just-in-time (JIT) compilation using CUDA Tile IR.

use cutile::api;
use cutile::prelude::*;

#[cutile::module]
mod kernels {
    use super::*;

    #[cutile::entry()]
    fn vector_add<const B: i32>(
        z: &mut Tensor<f32, {[B]}>,
        x: &Tensor<f32, {[-1]}>,
        y: &Tensor<f32, {[-1]}>,
    ) {
        let tx = x.load_like(z);
        let ty = y.load_like(z);
        z.store(tx + ty);
    }
}

async fn run() -> Result<(), Box<dyn std::error::Error>> {
    let size = 1024;
    let x = api::ones::<f32>(&[size]).await;
    let y = api::ones::<f32>(&[size]).await;

    // Partition the destination tensor into independent 128-element chunks
    let z = api::zeros::<f32>(&[size]).partition([128]);

    // Launch the kernel and synchronize
    let (_z, _x, _y) = kernels::vector_add(z, x, y).sync()?;

    Ok(())
}

The kernel receives the tile size via const generics (const B: i32). High-level methods like load_like and store automatically schedule shared-memory staging and register transfers. The host API supports async/await natively, letting developers treat CUDA stream queueing and CUDA Graph replay as standard asynchronous tasks.

Environment and Production Adoption

cutile-rs runs on Stable Rust 1.89+ and CUDA 13.3+. It does not require a compiler fork or custom LLVM build—developers can add the crate to standard Cargo dependencies. It is already used in production workloads, including Hugging Face’s distributed inference engine Grout and the Rust LLM inference framework mistral.rs as their backend for GEMM (general matrix multiply) and attention operations. In benchmarks, it delivers throughput comparable to vendor-optimized cuBLAS.

Feature and Specification Comparison

cuda-oxide and cutile-rs serve different design goals and abstraction levels.

Feature	cuda-oxide (SIMT Track)	cutile-rs (Tile Track)
Programming granularity	Thread	Tile (Tensor Block)
Abstraction level	Low-level (CUDA C++ equivalent)	High-level (DSL / Tile operations)
Memory safety	Managed via DisjointSlice and launch contracts	Compile-time race prevention via partitioned ownership
unsafe requirement	Partially required for low-level control	Not required (pure Safe Rust)
Compilation model	Direct PTX codegen via rustc backend	JIT compilation via macro expansion and CUDA Tile IR
Rust toolchain	Pinned nightly + custom LLVM	Stable Rust (1.89+)
Required CUDA version	Aligned with development environment	CUDA 13.3+
Development status	Early alpha (experimental)	Early adoption (used in production inference engines)
Primary use cases	Fine-grained thread scheduling, custom assembly optimization	Deep learning operations, GEMM, tensor processing

cuda-oxide fits projects that require low-level hardware control and custom assembly instructions. cutile-rs is better suited for matrix math and memory-safe high-throughput inference pipelines.

Safety Guarantees via the Type System

In traditional CUDA C++, pointer bugs and thread synchronization errors often stay hidden until runtime. CUDA Rust detects these issues at compile time by bringing Rust’s static analysis and type system to GPU programming.

The safety model relies on three key mechanisms:

• Aliasing Prevention: In C++, passing pointers to the same buffer across multiple threads can easily introduce unintended overlapping writes. In CUDA Rust, types like DisjointSlice and partitioned tensors statically guarantee that slices assigned to different workers never overlap.
• Static Data Race Elimination: Rust’s exclusive mutable borrow rule (&mut cannot alias) applies to device-side code. While multiple threads can share immutable references (&), mutable references are confined to a single tile, preventing simultaneous write conflicts.
• Array Bounds Checking: Dynamic memory accesses retain Rust’s standard slice boundary checks, preventing GPU kernel panics and out-of-bounds memory corruption before bad addresses can be accessed.

Roadmap and Future Outlook

NVIDIA plans to mature CUDA Rust as a core component of the CUDA platform rather than keeping it as a standalone experiment. Key milestones planned across late 2026 and 2027 include:

1. Toolchain Stabilization for cuda-oxide: Eliminate the current dependency on nightly Rust and custom LLVM builds, moving toward integration with upstream rustc and official target triples.
2. Multi-Language Interoperability: Broaden runtime ABI and memory layout interfaces so CUDA Rust can run alongside CUDA C++ and CUDA Python within the same process. This gives teams an incremental migration path to replace specific kernel layers with Rust without discarding existing C++ assets.
3. Inclusion in the CUDA Toolkit: Package the CUDA Rust toolchain as a default component in upcoming major releases of the CUDA Toolkit, integrating it into installers and official documentation.