A company is using Amazon Bedrock and Anthropic Claude 3 Haiku to develop an AI assistant. The AI assistant normally processes 10,000 requests each hour but experiences surges of up to 30,000…

Question

Options

• APurchase provisioned throughput and sufficient model units (MUs) in a single Region. Configure
• BImplement token batching to reduce API overhead. Use cross-Region inference profiles to
• CSet up auto scaling AWS Lambda functions in each Region. Implement client-side round-robin
• DImplement batch inference for all requests by using Amazon S3 buckets across multiple Regions.

• A
  5% (3)
• B
  66% (37)
• C
  11% (6)
• D
  18% (10)

Explanation

Option B is the correct solution because it directly addresses both throughput bottlenecks and latency requirements using native Amazon Bedrock performance optimization features that are designed for real-time, high-volume generative AI workloads. Amazon Bedrock supports cross-Region inference profiles, which allow applications to transparently route inference requests across multiple AWS Regions. During peak usage periods, traffic is automatically distributed to Regions with available capacity, reducing throttling, request queuing, and timeout risks. This approach aligns with AWS guidance for building highly available, low-latency GenAI applications that must scale elastically across geographic boundaries. Token batching further improves efficiency by combining multiple inference requests into a single model invocation where applicable. AWS Generative AI documentation highlights batching as a key optimization technique to reduce per-request overhead, improve throughput, and better utilize model capacity. This is especially effective for lightweight, low-latency models such as Claude 3 Haiku, which are designed for fast responses and high request volumes.

No community discussion yet for this question.