What is BM25?
Also called Okapi BM25
Definition
BM25 is a lexical relevance scoring method that uses term frequency, inverse document frequency, and document-length normalization. It ranks documents by statistical relationships between query terms and document text rather than by generated answers.
The operation of BM25 is explained.
Elastic uses BM25 as its default full-text scoring algorithm. A term’s frequency, rarity across the collection, and the document’s length influence its contribution. This is not the same process as embedding-based similarity.
The relevance of BM25 is explained.
Exact product names, API parameters, and error messages matter in SaaS documentation searches. A lexical retriever can identify those strings even when a semantic retriever favors a broader conceptual match.
The evaluation of BM25 is explained.
Relevance evaluation includes exact-match questions and paraphrases. Established parameter defaults provide a baseline for comparisons against relevance judgments. Keyword repetition does not replace accurate, useful documentation.
Frequently asked questions about BM25
Does BM25 generate an answer?
BM25 scores candidate documents; a separate component can generate an answer from them.
Does BM25 use embeddings?
BM25 itself is a lexical scoring method rather than an embedding model.
Can BM25 support RAG?
A RAG system can use BM25 to retrieve source documents before generation.