Brief
Build a BM25 inverted index from scratch: a tokeniser, posting lists (for each word, the list of documents that contain it) and the k1/b parameters (k1 controls how much repeated words count, b controls the penalty for long documents). Add dense retrieval with a small pretrained bi-encoder (a model that turns queries and documents into vectors separately, so you compare them by dot product), run through your own pooling and your PL-B09 index. Combine the two rankings with reciprocal-rank fusion (give each document a score based on its rank in each list, then add the scores), and rerank the top 100. The reranker is a cross-encoder (a model that reads the query and the document together; slower but more accurate). Evaluate on BEIR SciFact, NFCorpus and FiQA with your own nDCG@10.
VICTORY CONDITIONS
- your own BM25 within 0.01 nDCG@10 of Pyserini; hybrid + rerank ≥ +0.05 average nDCG@10 over BM25 at p95 ≤ 300 ms (nDCG@10 (a ranking score that rewards putting relevant documents near the top of the first 10 results) per dataset and on average; p95 query latency (95% of queries finish faster than this)).
- A public repository with a README that shows your result and how to reproduce it.
Terms in plain words
- BM25
- A classic keyword-matching score that weighs how often query words appear in a document and how rare they are overall.
- Bi-encoder
- A model that embeds the query and each document separately, so documents can be embedded once and searched fast.
- Cross-encoder
- A model that reads the query and a document together and outputs one relevance score; accurate but too slow for the whole corpus.
- Reciprocal-rank fusion
- Merging several ranked lists by giving each document points that shrink with its rank in each list.
- nDCG@10
- A 0-to-1 score for a top-10 ranking that gives more credit for relevant results placed higher.
Why it rates ★★☆☆☆
Retrieval for RAG is an in-demand practical skill. Measured against BEIR, it beats every vibes-based RAG demo.