AIP-C01 · Question #85
A company has a customer service application that uses Amazon Bedrock to generate personalized responses to customer inquiries. The company needs to establish a quality assurance process to evaluate…
The correct answer is B. Use Amazon Bedrock evaluation jobs to compare model outputs by using custom prompt. Amazon Bedrock evaluation jobs natively support running multiple prompt templates against the same set of test inputs and automatically computing quantitative quality metrics (such as ROUGE, BERTScore, or custom metrics). Bedrock evaluation also supports a human review workflow…
Question
A company has a customer service application that uses Amazon Bedrock to generate personalized responses to customer inquiries. The company needs to establish a quality assurance process to evaluate prompt effectiveness and model configurations across updates. The process must automatically compare outputs from multiple prompt templates, detect response quality issues, provide quantitative metrics, and allow human reviewers to give feedback on responses. The process must prevent configurations that do not meet a predefined quality threshold from being deployed. Which solution will meet these requirements?
Options
- ACreate an AWS Lambda function that sends sample customer inquiries to multiple Amazon
- BUse Amazon Bedrock evaluation jobs to compare model outputs by using custom prompt
- CSet up Amazon CloudWatch alarms to monitor response latency and error rates from Amazon
- DUse AWS Lambda functions to create an automated testing framework that samples production
How the community answered
(35 responses)- A3% (1)
- B74% (26)
- C9% (3)
- D14% (5)
Explanation
Amazon Bedrock evaluation jobs natively support running multiple prompt templates against the same set of test inputs and automatically computing quantitative quality metrics (such as ROUGE, BERTScore, or custom metrics). Bedrock evaluation also supports a human review workflow where reviewers can rate and provide feedback on model responses directly within the service. Evaluation results include quality scores that can serve as automated deployment gates, preventing configurations below a predefined threshold from being promoted. Option A (custom Lambda) requires significant custom code to replicate this built-in functionality. Option C (CloudWatch latency and error alarms) monitors operational metrics but cannot evaluate response quality or enable human review. Option D samples production traffic but lacks structured human-feedback integration and automated quality gates.
Topics
Community Discussion
No community discussion yet for this question.