This week, Liquid AI released two new retrieval models. They are LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M. Both hold 350M parameters. Both are the first bidirectional members of the LFM family. They build on LFM2.5-350M-Base, released in March. The pair targets fast multilingual and cross-lingual search across 11 languages. Their footprint is small enough to run almost anywhere. Both are available now on Hugging Face under the LFM Open License v1.0.

LFM2.5 Retrievers

The two models share one backbone but represent text differently. LFM2.5-Embedding-350M is a dense bi-encoder. It turns each document into a single vector. Pick it when you want the fastest search and the smallest, cheapest index.

LFM2.5-ColBERT-350M is a late-interaction model. It converts each token into a vector rather than one vector per document. This lets it match queries word-by-word for higher accuracy and better generalization. The trade-off is a larger index. Pick it when accuracy matters more than storage. Its query length is capped at 32 tokens. It can also rerank a first-stage retriever’s results without building an index.

Both target short-context search. Good fits include product catalogs, FAQ knowledge bases, and support docs. Liquid AI positions both as a drop-in replacement for an existing RAG pipeline.

The Architecture Change: Causal to Bidirectional

Both models start from LFM2.5-350M-Base, a mid-trained general-purpose checkpoint. Liquid AI applies a small set of bidirectional patches to the LFM2 architecture. These adapt it from a causal decoder to a bidirectional encoder.

In a causal setup, each token uses only itself and previous tokens. That suits left-to-right generation but is less natural for retrieval. The team replaces the causal attention mask with a bidirectional one. Now every token can attend to both left and right context. They also make the LFM2 short convolutions non-causal. These mix local information symmetrically around each token, not only from the past.

This preserves the LFM2 backbone’s efficiency while producing the full-context representations retrieval needs. Each model has 17 layers: 10 convolution, 6 attention, and 1 pooling or dense. Context length reaches 32,768 tokens, though documents are tuned to 512 tokens. From the shared encoder, the two models differ only in output. Embedding uses CLS-style pooling for one 1024-dim vector. ColBERT keeps 128-dim per-token embeddings for MaxSim late interaction.

Training and Data

Both models follow the same three-stage recipe:

• Stage one is large-scale contrastive pretraining in English.
• Stage two is multilingual and cross-lingual distillation from a strong teacher across all 11 languages.
• Stage three is final fine-tuning on hard-mined negatives.

The Embedding model receives slightly more cross-lingual data than ColBERT. Cross-lingual retrieval emerges more naturally in the late-interaction setup. Training data combines curated internal data with open-source English retrieval datasets. LLM-based translation expands the multilingual and cross-lingual pairs.

Benchmark

Liquid AI evaluated two capabilities. The first is multilingual retrieval with NanoBEIR. The second is cross-lingual open-domain QA with MKQA-11. Both report results across all 11 languages: Arabic, German, English, Spanish, French, Italian, Japanese, Korean, Norwegian, Portuguese, and Swedish.

On average, both models lead their class. Here are the comparison details:

ColBERT leads on both averages. Embedding is close behind on MKQA-11 at 0.691. Both beat Qwen3-Embedding-0.6B, a larger model. The new ColBERT also improves on the earlier LFM2-ColBERT-350M, from 0.540 to 0.605 on NanoBEIR. Liquid AI also notes that NanoBEIR English tracks the more expensive full BEIR. The two stay highly correlated, with NanoBEIR scoring a near-constant ~15% higher. The research team therefore uses NanoBEIR as a practical proxy during training runs.

Latency and Edge Deployment

Liquid AI released GGUF variants for llama.cpp. These let both models run on CPUs, laptops, and edge devices. The figures below use a MacBook Pro M4 Max at FP16. Queries are 32 tokens; documents are 256 tokens.

When document embeddings are pre-computed, median (p50) query latency stays under 10 ms. Encoding documents at query time pushes ColBERT to 34.3 ms. For enterprise scale, Liquid AI also built an internal GPU stack. On an H100 at FP16, it observes latencies as low as 1 ms. Embedding query latency there is 1.5 ms p50.

Use Cases With Examples

• E-commerce: Search a product catalog across many languages with one index. A shopper types a Korean query and the system surfaces an English product listing. Cross-lingual retrieval makes this work without per-language indexes.
• FAQ and support knowledge bases: Retrieve the right answer reliably across customer-facing surfaces. A French support question maps to an English help article.
• On-device semantic search: Search files, emails, and notes locally on consumer hardware. The GGUF build keeps data on the device at near-zero cost.
• Enterprise knowledge assistants: Retrieve internal legal, financial, and technical documents across languages. ColBERT suits this when answer accuracy outranks index size.

Code: Getting Started

The Embedding model runs through sentence-transformers. Always pass the asymmetric prompts, query: and document:. Omitting them silently degrades retrieval quality.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "LiquidAI/LFM2.5-Embedding-350M",
    trust_remote_code=True,
)

queries = ["What is the capital of France?"]
documents = ["Paris is the capital and largest city of France."]

q_emb = model.encode(queries,   prompt_name="query",    normalize_embeddings=True)
d_emb = model.encode(documents, prompt_name="document", normalize_embeddings=True)

scores = q_emb @ d_emb.T  # shape: (n_queries, n_documents)

The ColBERT model runs through PyLate. Its PLAID index uses FastPLAID for efficient similarity search.

from pylate import indexes, models, retrieve

model = models.ColBERT(
    model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M",
    trust_remote_code=True,
)
model.tokenizer.pad_token = model.tokenizer.eos_token

index = indexes.PLAID(index_folder="pylate-index", index_name="index", override=True)

docs_emb = model.encode(["document 1 text", "document 2 text"], is_query=False)
index.add_documents(documents_ids=["1", "2"], documents_embeddings=docs_emb)

retriever = retrieve.ColBERT(index=index)
q_emb = model.encode(["a search query"], is_query=True)
scores = retriever.retrieve(queries_embeddings=q_emb, k=10)

To rerank an existing first-stage pipeline instead, skip the index and use rank.rerank.

from pylate import models, rank

model = models.ColBERT(model_name_or_path="LiquidAI/LFM2.5-ColBERT-350M", trust_remote_code=True)

queries = ["query A"]
documents = [["candidate doc 1", "candidate doc 2"]]
documents_ids = [[1, 2]]

q_emb = model.encode(queries, is_query=True)
d_emb = model.encode(documents, is_query=False)

reranked = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=q_emb,
    documents_embeddings=d_emb,
)

You can also fine-tune either model on your own data. The Embedding card provides snippets using sentence-transformers and MultipleNegativesRankingLoss.

Key Takeaways

• Liquid AI’s LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M are the first bidirectional LFMs, built for multilingual search across 11 languages.
• Both 350M models lead their class on NanoBEIR and MKQA-11, beating the larger Qwen3-Embedding-0.6B.
• Embedding gives the smallest, cheapest index; ColBERT trades a larger index for higher per-token accuracy.
• GGUF builds run on CPUs, laptops, and edge via llama.cpp, with cached p50 query latency under 10 ms.
• They drop into existing RAG pipelines through sentence-transformers and PyLate, under the LFM Open License v1.0.

Interactive Explainer