GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #65
A team uses Mosaic AI Vector Search to retrieve documents for their Retrieval-Augmented Generation (RAG) pipeline. The search query returns five relevant documents, and the first three are added to…
The correct answer is A. Use a reranker to order the documents based on the relevance scores. The root problem is a ranking mismatch: the initial vector-similarity retrieval returns documents in an order that doesn't match their actual relevance to the query. A reranker (typically a cross-encoder model) takes the query and each candidate document together and produces a…
Question
A team uses Mosaic AI Vector Search to retrieve documents for their Retrieval-Augmented Generation (RAG) pipeline. The search query returns five relevant documents, and the first three are added to the prompt as context. Performance evaluation with Agent Evaluation shows that some lower-ranked retrieved documents have higher context relevancy scores than higher- ranked documents. Which option should the team consider to optimize this workflow?
Options
- AUse a reranker to order the documents based on the relevance scores.
- BModify the prompt to instruct the LLM to order the documents based on the relevance scores.
- CUse a different embedding model for computing document embeddings.
- DIncrease the number of documents added to the prompt to improve context relevance.
How the community answered
(41 responses)- A73% (30)
- B7% (3)
- C5% (2)
- D15% (6)
Explanation
The root problem is a ranking mismatch: the initial vector-similarity retrieval returns documents in an order that doesn't match their actual relevance to the query. A reranker (typically a cross-encoder model) takes the query and each candidate document together and produces a fine-grained relevance score, then reorders the documents before the top-k are added to the prompt. This directly fixes the ordering issue. Option B (prompt instruction to reorder) asks the LLM to do something it cannot do reliably without seeing all documents simultaneously. Option C (different embedding model) might change retrieval quality overall but won't solve an ordering problem that reranking is designed for. Option D (add more documents) increases noise and token cost without correcting the ranking.
Topics
Community Discussion
No community discussion yet for this question.