MLS-C01 Question #325: Real Exam Question with Answer & Explanation

Question

An insurance company developed a new experimental machine learning (ML) model to replace an existing model that is in production. The company must validate the quality of predictions from the new experimental model in a production environment before the company uses the new experimental model to serve general user requests. New one model can serve user requests at a time. The company must measure the performance of the new experimental model without affecting the current live traffic. Which solution will meet these requirements?

Options

• AA/B testing
• BCanary release
• CShadow deployment
• DBlue/green deployment

Explanation

{"question_number": 1, "question_summary": "Validate a new ML model in production without affecting live traffic", "correct_answer": "C", "explanation": "Shadow deployment (C) is the correct choice because it mirrors a copy of live production traffic to the new experimental model in parallel, but the new model's responses are discarded - they are never served to actual users. This allows engineers to compare the new model's predictions against the existing production model using real traffic without any user impact. The key requirement here is 'measure performance without affecting current live traffic,' which shadow deployment uniquely satisfies. A/B testing (A) and canary releases (B) both route a portion of real user requests to the new model, directly impacting some users. Blue/green deployment (D) cuts all traffic over to the new environment at once, replacing the old model entirely. None of those options meet the constraint that only one model serves user requests while the other is validated invisibly.", "generated_by": "claude-sonnet", "llm_judge_score": 4}

No community discussion yet for this question.