AIP-C01 · Question #78
A company is developing a generative AI (GenAI) application that analyzes customer service calls in real time and generates suggested responses for human customer service agents. The application…
The correct answer is B. Deploy a low-latency, real-time optimized model on Amazon Bedrock. Purchase provisioned. A low-latency, real-time optimized model on Amazon Bedrock with provisioned throughput satisfies all three constraints. A low-latency model (such as a compact, distillation-optimized model) can consistently deliver responses under 200 ms. Provisioned throughput on Bedrock…
Question
A company is developing a generative AI (GenAI) application that analyzes customer service calls in real time and generates suggested responses for human customer service agents. The application must process 500,000 concurrent calls during peak hours with less than 200 ms end- to-end latency for each suggestion. The company uses existing architecture to transcribe customer call audio streams. The application must not exceed a predefined monthly compute budget and must maintain auto scaling capabilities. Which solution will meet these requirements?
Options
- ADeploy a large, complex reasoning model on Amazon Bedrock. Purchase provisioned throughput
- BDeploy a low-latency, real-time optimized model on Amazon Bedrock. Purchase provisioned
- CDeploy a large language model (LLM) on an Amazon SageMaker real-time endpoint that uses
- DDeploy a mid-sized language model on an Amazon SageMaker serverless endpoint that is
How the community answered
(58 responses)- A5% (3)
- B67% (39)
- C17% (10)
- D10% (6)
Explanation
A low-latency, real-time optimized model on Amazon Bedrock with provisioned throughput satisfies all three constraints. A low-latency model (such as a compact, distillation-optimized model) can consistently deliver responses under 200 ms. Provisioned throughput on Bedrock pre-allocates dedicated model units, eliminating the queuing delays that cause throttling at 500,000 concurrent calls, and its fixed per-unit pricing is predictable and budget-controllable. Option A (large reasoning model) is designed for accuracy, not speed; reasoning chains routinely exceed 200 ms. Option C (SageMaker real-time endpoint with auto scaling) introduces scaling lag during peak ramp-up and is harder to tune for sub-200 ms latency at this concurrency. Option D (SageMaker serverless endpoint) has cold-start overhead that makes consistent sub-200 ms latency impossible.
Topics
Community Discussion
No community discussion yet for this question.