FAQ

Short answers to common questions. For anything conceptual, the linked pages go deeper.

What is Rig?

Rig is an open-source Rust library for building applications powered by large language models: one API across providers, plus higher-level building blocks like agents, tools, RAG, and structured extraction. Start with the Introduction and the Quickstart.

Should I depend on rig, rig-core, or rig-agent?

Depend on rig. It re-exports rig-core (provider, model, tool, and vector-store contracts) and rig-agent (the agent runtime, on by default), and exposes integrations behind cargo features. Depend on rig-core directly only if you want the provider contracts without the agent runtime or the facade’s integrations. See Architecture.

Which model providers does Rig support?

More than 20 behind one API, including OpenAI, Anthropic, Google Gemini, Cohere, Mistral, xAI, DeepSeek, Groq, OpenRouter, Perplexity, Together, Azure OpenAI, Hugging Face, and Ollama, plus Amazon Bedrock and Google Vertex AI through feature-gated companion crates. See Providers & Clients and Model Providers.

Can I run local or self-hosted models?

Yes. Use the Ollama or llama.cpp clients for a local server, point OpenAIConfig::with_base_url at any OpenAI-compatible endpoint, or enable the candle feature to run supported models in-process. See Providers & Clients.

Do I have to use Tokio?

The examples use #[tokio::main], but Rig doesn’t require Tokio. The bundled HTTP transport brings what it needs, and an agent run is a plain future you can drive on any executor.

Does Rig work in the browser?

Yes. Rig builds for wasm32-unknown-unknown with no extra features. MCP tools and the bundled websocket backend are native-only, and WASI targets are not supported. See Installation.

Does Rig support fine-tuning?

No. Rig focuses on inference and orchestration, not training. Fine-tune with your provider’s tooling, then use the resulting model id with Rig like any other model.

How do I continue a conversation?

Pass the earlier messages with .history(&history) on a prompt, then append the run’s new messages: history.extend(response.messages). For conversations that load and save themselves, attach a memory backend. See Agents and Memory.

How do I set reasoning effort, prompt caching, or other provider settings?

Common settings are typed and portable: .reasoning(Effort::High), .cache(CacheRetention::Long), .service_tier(..), .seed(..), and others on a CompletionRequest, an AgentBuilder, or a single run. Provider-only settings go through typed options from rig::providers::<name>::extension with .provider_option(..), and additional_params still takes raw JSON. See Completions.

How does Rig handle rate limits and retries?

Rig doesn’t retry provider calls on its own. ProviderError::is_retryable() tells you whether a failure is transient, so you can retry with backoff around a call; you can also retry at the HTTP layer with a reqwest-middleware client passed to with_http, or retry a model turn from an agent hook. Typed extraction has its own .retries(n) for answers that fail to parse. See Error Handling.

Can I use Rig in a web service?

Yes. Rig is async, so it fits web frameworks like Axum and Actix as well as serverless targets. See the Guides, including the AWS Lambda deploy walkthroughs.

Can I test without calling a real model?

Yes. The test-utils feature provides mock models you can script turn by turn, and the cassette feature records real runs and replays them offline. See Testing.

How does Rig compare to other LLM frameworks?

Rig gives you the building blocks you’d expect (agents, tools, RAG, structured extraction) with Rust’s performance and type safety and a single API across providers. See Architecture for how the pieces fit together.