A company uses Amazon Bedrock to implement a Retrieval Augmented Generation (RAG)-based system to serve medical information to users. The company needs to compare multiple chunking strategies…

Question

Options

• ACreate a retrieve-only evaluation job that uses a supported version of Anthropic Claude Sonnet
• BCreate a retrieve-and-generate evaluation job that uses custom precision-at-k metrics and an
• CCreate a separate evaluation job for each chunking strategy and FM combination. Use Amazon
• DSet up a pipeline that uses multiple retrieve-only evaluation jobs to assess retrieval quality.

• A
  3% (2)
• B
  80% (56)
• C
  6% (4)
• D
  11% (8)

Explanation

A retrieve-and-generate evaluation job in Amazon Bedrock Knowledge Bases tests the complete RAG pipeline end-to-end, covering both the retrieval stage (affected by chunking strategy) and the generation stage (the FM). Using custom metrics such as precision-at-k alongside built-in quality metrics allows evaluation of retrieval accuracy across different chunking configurations and direct comparison of two FM outputs. Quality thresholds can be applied programmatically to gate deployment. Option A (retrieve-only) only evaluates retrieval and cannot compare FM generation quality. Option C creates separate jobs per combination but lacks a unified automated comparison with threshold enforcement. Option D uses only retrieve-only jobs and misses FM evaluation entirely.

No community discussion yet for this question.