Candle for Rust: A Minimalist ML Framework from Hugging Face

Candle is a modern machine learning framework written in Rust and developed by the Hugging Face team. Unlike heavyweight solutions like PyTorch or TensorFlow, Candle focuses on performance, memory safety, and minimal resource consumption. It lets you run neural network models on CPU and GPU (via CUDA and Vulkan) without installing massive runtimes.

Why use Candle? First, it's perfect for embedded systems, WebAssembly (WASM), and edge devices where binary size and load speed are critical. Second, Rust guarantees thread safety and no garbage collector — giving you predictable inference performance. Third, Candle supports loading models from PyTorch formats (safetensors, bin) and the Hugging Face Hub, making it an excellent choice for production.

As of this writing, Candle is under active development: it supports transformers (BERT, GPT, LLaMA, Whisper), convolutional networks (ResNet, ViT), and even generative models (Stable Diffusion). In this guide, we'll cover installation, key methods, and build a complete text classification example.

Installation

Candle is a Rust library, so installation is done through Cargo. Add the dependency to your Cargo.toml:

[dependencies]
candle-core = "0.7"
candle-nn = "0.7"
candle-transformers = "0.7"
tokenizers = "0.21"  # for working with tokenizers
anyhow = "1.0"       # for error handling

If you need CUDA support, use the --features cuda flag:

cargo add candle-core --features cuda
cargo add candle-nn --features cuda
cargo add candle-transformers --features cuda

For WebAssembly (WASM) support, add --features wasm.

Quick Start: Minimal Working Example

Let's create a tensor and perform a simple operation — matrix multiplication:

use candle_core::{Device, Tensor};

fn main() -> Result<(), Box> {
    // Create tensors on CPU
    let a = Tensor::new(&[[1.0f32, 2.0], [3.0, 4.0]], &Device::Cpu)?;
    let b = Tensor::new(&[[5.0f32, 6.0], [7.0, 8.0]], &Device::Cpu)?;

    // Matrix multiplication
    let c = a.matmul(&b)?;
    println!("Result: {:?}", c.to_vec2::()?);

    Ok(())
}

This code will output:

Result: [[19.0, 22.0], [43.0, 50.0]]

Notice: all operations return Result, so we use ? for error handling.

Key Methods and Functions

Let's look at 10 essential methods from candle-core and candle-nn that you'll use in 90% of projects.

1. Tensor::new — Creating a Tensor

Signature: pub fn new, D: Into>(data: impl Into>, device: D) -> Result<Tensor>

Description: Creates a tensor from a flat vector of floating-point numbers. The shape is specified via Shape (e.g., Shape::from((2, 3))).

use candle_core::{Device, Tensor, Shape};

fn main() -> anyhow::Result<()> {
    // Create a 2x3 tensor from a vector
    let t = Tensor::new(&[1.0, 2.0, 3.0, 4.0, 5.0, 6.0], &Device::Cpu)?;
    // Reshape it
    let t_reshaped = t.reshape(&Shape::from((2, 3)))?;
    println!("{:?}", t_reshaped.to_vec2::()?);
    Ok(())
}

2. Tensor::matmul — Matrix Multiplication

Signature: pub fn matmul(&self, other: &Tensor) -> Result<Tensor>

Description: Performs matrix multiplication of two tensors. Supports broadcasting for batches.

use candle_core::{D