rig — A Rust Library for LLM Integration

In the world of Rust development, working with large language models (LLMs) has long been a challenge. Most SDKs are written in Python or JavaScript, while the Rust ecosystem offered only scattered HTTP clients. The rig library (Rust Integration with GGUF/LLM) solves this by providing a unified, ergonomic, and type-safe interface for interacting with dozens of LLM providers: OpenAI, Anthropic, Cohere, Google Gemini, Llama.cpp, Ollama, and more.

Why use rig? First, it offers a type-safe API: all requests and responses are strictly typed, eliminating a whole class of runtime errors. Second, rig implements pipelines — composable chains of LLM calls with support for streaming, context windows, and tool calling. Third, the library is designed for performance: async out of the box, minimal allocations, and batching support.

This article is your ultimate guide to rig. We'll cover installation, core methods, walk through a real-world example, and share optimization tips. By the end, you'll be able to build production-ready Rust applications powered by LLMs.

Installation

Add rig to your Cargo project:

cargo add rig-core --features openai,anthropic,cohere,gemini,ollama,llama-cpp

If you only need one provider, like OpenAI:

cargo add rig-core --features openai

For local models via llama.cpp, you'll need to install llama.cpp itself (download from GitHub) and compile it with CUDA/Metal support. Then add the llama-cpp feature.

Minimum Rust version: 1.75.0. Check yours:

rustc --version

Quick Start

Create a src/main.rs file and write a simple request to OpenAI:

use rig::providers::openai;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize client using the OPENAI_API_KEY environment variable
    let client = openai::Client::from_env();
    
    // Create the model
    let model = client.model("gpt-4o-mini").build();
    
    // Send a request
    let response = model.prompt("Tell me a programmer joke").await?;
    
    println!("{}", response);
    Ok(())
}

Run it:

export OPENAI_API_KEY="sk-..."
cargo run

You'll see a generated joke. Congratulations — you've just integrated an LLM into Rust!

Core Methods and Functions

Let's dive into rig's key methods. For each, we'll show the signature, description, and a working example.

1. Client::from_env()

Signature: pub fn from_env() -> Self

What it does: Creates a client for a specific provider by reading the API key from an environment variable (e.g., OPENAI_API_KEY).

use rig::providers::openai;
let client = openai::Client::from_env();

2. Client::model(name)

Signature: pub fn model(&self, model_name: &str) -> ModelBuilder

What it does: Returns a builder for configuring the model. Allows you to set temperature, max_tokens, stop words, and other parameters.

let model = client.model("gpt-4o")
    .temperature(0.7)
    .max_tokens(200)
    .build();

3. Model::prompt(text)

Signature: pub async fn prompt(&self, prompt: impl Into<String>) -> Result<String, ModelError>

What it does: Sends a prompt to the model and returns the generated text response.