Providers & Clients

A provider client is the entry point to an LLM vendor. It holds the vendor’s configuration (credential, base URL, options) and a transport, and builds the models you call: completion, embedding, transcription, image, and audio models, depending on what the provider offers.

use rig::providers::openai::{self, OpenAI};

// Read OPENAI_API_KEY (and the optional OPENAI_BASE_URL).
let client = OpenAI::from_env()?;

// Or pass the key yourself.
let client = OpenAI::new("sk-...");

// Build models from the client.
let gpt = client.completion(openai::GPT_5_5);
let embedder = client.embedding(openai::TEXT_EMBEDDING_3_SMALL, None);

Clients are cheap to clone and so are the models they build; create them once and share them.

Provider clients

Providers with their own API format have their own client type in rig::providers::<name>:

Client	Environment	Builds
`openai::OpenAI`	`OPENAI_API_KEY`, optional `OPENAI_BASE_URL`	completion, chat, responses, embedding, rerank, transcription, image, audio
`anthropic::Anthropic`	`ANTHROPIC_API_KEY`, optional `ANTHROPIC_BASE_URL`	completion
`gemini::Gemini`	`GEMINI_API_KEY`	completion, embedding, transcription, image
`cohere::Cohere`	`COHERE_API_KEY`	completion, embedding
`ollama::Ollama`	optional `OLLAMA_API_BASE_URL`, `OLLAMA_API_KEY`	completion, native completion, embedding
`voyageai::VoyageAi`	`VOYAGE_API_KEY`	embedding, rerank
`copilot::Copilot`	`GITHUB_COPILOT_API_KEY` (or `COPILOT_API_KEY`)	completion, embedding

Each has from_env() and new(api_key) (Ollama’s new() takes no key and talks to the local daemon). Image and audio models need the image and audio cargo features.

use rig::providers::anthropic::{self, Anthropic};
use rig::providers::ollama::Ollama;

let claude = Anthropic::from_env()?.completion(anthropic::CLAUDE_SONNET_5_5);

// Ollama on localhost:11434, no key needed.
let llama = Ollama::new().completion("llama3.2");

Amazon Bedrock, Google Vertex AI, Gemini over gRPC, and local Candle models live in companion crates, enabled by the bedrock, vertexai, gemini-grpc, and candle features (see Integrations).

OpenAI-compatible vendors

Many vendors serve OpenAI’s API format. They don’t have a client type of their own: their module has from_env() and new(api_key) functions that return an OpenAI client preconfigured with the vendor’s base URL, credential variable, and quirks.

use rig::providers::{deepseek, groq, openrouter};

let deepseek = deepseek::from_env()?.completion(deepseek::DEEPSEEK_V4_FLASH);
let groq = groq::from_env()?.completion(groq::LLAMA_3_3_70B_VERSATILE);
let router = openrouter::from_env()?.completion("anthropic/claude-sonnet-5-5");

Module	Credential variable
`azure`	`AZURE_API_KEY` (or `AZURE_TOKEN`), plus `AZURE_ENDPOINT` and `AZURE_API_VERSION`
`chatgpt`	`CHATGPT_ACCESS_TOKEN` (a ChatGPT sign-in token, not an API key)
`deepseek`	`DEEPSEEK_API_KEY`
`doubleword`	`DOUBLEWORD_API_KEY`
`groq`	`GROQ_API_KEY`
`huggingface`	`HUGGINGFACE_API_KEY`
`hyperbolic`	`HYPERBOLIC_API_KEY`
`llamacpp`	`LLAMACPP_API_KEY`, `LLAMACPP_API_BASE_URL`
`minimax`	`MINIMAX_API_KEY`
`mira`	`MIRA_API_KEY`
`mistral`	`MISTRAL_API_KEY`
`moonshot`	`MOONSHOT_API_KEY`
`openrouter`	`OPENROUTER_API_KEY`
`perplexity`	`PERPLEXITY_API_KEY`
`together`	`TOGETHER_API_KEY`
`venice`	`VENICE_API_KEY`
`xai`	`XAI_API_KEY`
`xiaomimimo`	`XIAOMI_MIMO_API_KEY`
`zai`	`ZAI_API_KEY`

MiniMax, Moonshot, Xiaomi MiMo, and Z.AI also serve Anthropic’s format; their anthropic_from_env() and anthropic_new(key) functions return an Anthropic client instead.

Any other OpenAI-compatible server

For a server Rig doesn’t know (vLLM, LM Studio, a gateway), configure OpenAI’s format with your own base URL and turn the configuration into a client:

use rig::providers::openai::OpenAIConfig;

let local = OpenAIConfig::new("not-needed")
    .with_base_url("http://localhost:8000/v1")
    .client();

let model = local.chat("my-model");

For OpenAI, xAI, and ChatGPT, completion(..) targets the Responses API; for most other vendors it targets Chat Completions. Use chat(..) or responses(..) to pick the endpoint explicitly. Every client also exposes its configuration through config(), and each *Config type (OpenAIConfig, AnthropicConfig, …) has the same from_env(), new(..), and with_* setters if you want to adjust it before building the client.

Choosing a transport

With the default reqwest feature, clients from from_env() and new(..) send through a shared reqwest client. To set timeouts or proxies, add middleware, or use a test double, pass any HTTP client to with_http:

use rig::http_client::ReqwestClient;
use rig::providers::openai::OpenAI;
use rig::rig_reqwest::reqwest;

let http = reqwest::Client::builder()
    .timeout(std::time::Duration::from_secs(60))
    .build()?;

let client = OpenAI::from_env()?.with_http(ReqwestClient::from(http));

with_http accepts anything implementing rig::http_client::HttpClientExt. DynHttpClient::new(..).with_middleware(..) wraps a client with request and response hooks (injecting headers, logging bodies, reading rate-limit headers). A configuration can also be connected straight to a transport with OpenAIConfig::from_env()?.connect(http).

Models a client builds

Method	Returns
`completion(model)`	a completion model for Completions and Agents
`embedding(model, ndims)`	an embedding model for Embeddings; `ndims` is `None` for the model’s default width
`transcription(model)`	a speech-to-text model
`image_generation(model)`	an image model (`image` feature)
`audio_generation(model)`	a text-to-speech model (`audio` feature)

Not every client has every method: a missing capability is a compile error, not a runtime surprise. See Media for the last three.

Every model is a Model. When you need to store models from different providers in one place, erase the provider type with .into() or .erase() to get a DynModel:

use rig::operation::Completion;
use rig::providers::anthropic::{self, Anthropic};
use rig::providers::openai::{self, OpenAI};

let models: Vec<DynModel<Completion>> = vec![
    OpenAI::from_env()?.completion(openai::GPT_5_5).into(),
    Anthropic::from_env()?.completion(anthropic::CLAUDE_SONNET_5_5).into(),
];

Choosing a provider at runtime

When the provider comes from configuration rather than code, parse a vendor:model reference and build the model from it. Credentials are read from the provider’s usual environment variable:

use rig::providers::registry::ProviderRef;

let reference = ProviderRef::parse("deepseek:deepseek-v4-flash")?;
let model = reference.completion_model()?; // a DynModel<Completion>

A ProviderRef serializes without credentials, so it’s safe to store in config files. rig::providers::registry::connect("anthropic/claude-opus-5-5", api_key) does the same with an explicit key.

Listing models and checking credentials

Clients whose provider has a model-listing endpoint (OpenAI and its compatible vendors, Anthropic, Gemini, Ollama, Copilot) have list_models(). OpenAI, Anthropic, and Gemini clients can also verify() that the credential is accepted:

use rig::providers::anthropic::Anthropic;

let client = Anthropic::from_env()?;
client.verify().await?;

for model in client.list_models().await?.iter() {
    println!("{} ({})", model.id, model.display_name());
}

For facts about a model without a network call (context window, output limit, reasoning and caching support, prices), look it up in the built-in catalog:

use rig::catalog::Catalog;

if let Some(spec) = Catalog::builtin().resolve("anthropic/claude-haiku-4-5") {
    println!("context window: {:?}", spec.context_window);
}

ModelSpec::validate checks a request’s generation options against what the model supports, and ModelSpec::cost prices a response’s Usage.

Writing your own provider

A provider is a wire (how to encode a CompletionRequest and decode the reply for one endpoint) paired with a transport. The Write Your Own Provider guide walks through it.

Next steps

CompletionsCall a model directly with a CompletionRequest.

AgentsWrap a model with a preamble, tools, and context.

Model ProvidersPer-vendor setup, auth, and supported capabilities.

Previous
Overview Next
Completions