In-Memory Vector Store

InMemoryVectorStore is built into rig. It keeps documents and their embeddings in RAM and ranks them by cosine similarity, with no database and no extra feature. Use it for development, tests, and small datasets.

Quick start

use rig::prelude::*;
use serde::{Deserialize, Serialize};
use rig::providers::openai::{self, OpenAI};

#[derive(Embed, Clone, Serialize, Deserialize, Debug)]
struct WordDefinition {
    id: String,
    #[embed]
    definition: String,
}

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let model = OpenAI::from_env()?.embedding(openai::TEXT_EMBEDDING_3_SMALL, None);

    let documents = vec![
        WordDefinition { id: "doc0".into(), definition: "First document content".into() },
        WordDefinition { id: "doc1".into(), definition: "Second document content".into() },
    ];

    let embeddings = EmbeddingsBuilder::new(model.clone())
        .documents(documents)?
        .build()
        .await?;

    // Build the store, then an index that embeds queries with the same model.
    let index = InMemoryVectorStore::from_documents(embeddings).index(model);

    let req = VectorSearchRequest::builder()
        .query("search query")
        .samples(5)
        .build();

    for result in index.top_n::<WordDefinition>(req).await? {
        println!("{:.3} {} {:?}", result.score, result.id, result.document);
    }

    Ok(())
}

InMemoryVectorStore<D> holds the data; .index(model) turns it into an InMemoryVectorIndex<D>, which implements VectorStoreIndex and embeds each query with model. Use the model the documents were embedded with.

Adding documents

The store maps a string id to a document and its embeddings. Three constructors fill it, differing in how ids are chosen:

use rig::embeddings::Embedding;

#[derive(Serialize, Clone)]
struct WordDefinition { id: String }

let doc = |id: &str| (WordDefinition { id: id.into() }, vec![Embedding::default()]);

// 1. Generated ids: "doc0", "doc1", ...
let store = InMemoryVectorStore::from_documents(vec![doc("a"), doc("b")]);

// 2. Explicit ids
let store = InMemoryVectorStore::from_documents_with_ids(vec![
    ("custom_id_1", doc("a").0, doc("a").1),
    ("custom_id_2", doc("b").0, doc("b").1),
]);

// 3. Ids derived from each document
let store = InMemoryVectorStore::from_documents_with_id_f(
    vec![doc("a"), doc("b")],
    |d| format!("word:{}", d.id),
);

Each pair is (document, Vec<Embedding>), exactly what EmbeddingsBuilder::build() returns. To grow an existing store, use the matching methods add_documents, add_documents_with_ids and add_documents_with_id_f. add_documents never overwrites an existing id; the other two replace a document whose id already exists.

Search strategies

By default a query scans every document (brute force). For larger collections, approximate search with locality-sensitive hashing (LSH) narrows the scan to likely candidates before scoring them exactly. Choose it with the builder:

use rig::embeddings::Embedding;
use rig::vector_store::IndexStrategy;

let store = InMemoryVectorStore::builder()
    .index_strategy(IndexStrategy::LSH { num_tables: 8, num_hyperplanes: 16 })
    .documents(embeddings)
    .build();

A document with several embeddings is ranked by its best-matching one. Filters and thresholds are applied during the scan, before the top results are picked.

Filtering

The in-memory store uses the backend-neutral Filter, matched against each document’s JSON form:

use rig::vector_store::request::{Filter, SearchFilter};

let req = VectorSearchRequest::builder()
    .query("search query")
    .samples(5)
    .filter(Filter::eq("category", json!("science")))
    .build();

Using it from an agent

An in-memory index can be an agent tool (.tool(index)) or retrieved on every prompt (.dynamic_context(n, index)):

use rig::prelude::*;
use rig::providers::openai::{self, OpenAI};

let client = OpenAI::from_env()?;
let model = client.embedding(openai::TEXT_EMBEDDING_3_SMALL, None);
let index = InMemoryVectorStore::<String>::default().index(model);

let agent = AgentBuilder::new(client.completion(openai::GPT_5_5))
    .preamble("Answer using the retrieved documents.")
    .dynamic_context(2, index)
    .build();

Considerations

Memory: every document and embedding lives in RAM, so usage grows with document count and vector width.
Persistence: none. Rebuild the store on each run, or move to a database-backed store.
Sharing: InMemoryVectorStore is Clone, Send and Sync. .index(model) consumes the store, so add documents before you build the index.

For persistence, larger datasets, or server-side filtering, switch to one of the database stores; the embedding and query code stays the same.