A credit card company has a fraud detection model in production on an Amazon SageMaker endpoint. The company develops a new version of the model. The company needs to assess the new model's…

Question

Options

• ASet up SageMaker Debugger and create a custom rule.
• BSet up blue/green deployments with all-at-once traffic shifting.
• CSet up blue/green deployments with canary traffic shifting.
• DSet up shadow testing with a shadow variant of the new model.

• A
  4% (1)
• B
  9% (2)
• C
  4% (1)
• D
  83% (19)

Explanation

Shadow testing (D) lets you route a copy of live production traffic to the new model variant in parallel, capture its inferences, and compare results against the production model - all without exposing end users to the new model's predictions. This is purpose-built for the exact scenario described: evaluating a challenger model on real data without any risk to production.

Why the distractors are wrong:

• A (SageMaker Debugger) is for monitoring training jobs and detecting issues like vanishing gradients - it doesn't evaluate a new model variant against live production traffic.
• B (blue/green, all-at-once) immediately shifts 100% of traffic to the new model, directly affecting production end users - the opposite of what's required.
• C (blue/green, canary) shifts a small percentage of real user traffic to the new model, meaning some production end users are affected by the untested model.

Memory tip: Think of shadow testing as a "ghost" - the new model silently processes every request alongside the production model, but its responses are never returned to users. If canary = some users see the new model, shadow = no users see it, but it still gets full live traffic to learn from.

No community discussion yet for this question.