PROFESSIONAL-MACHINE-LEARNING-ENGINEER Question #125: Real Exam Question with Answer & Explanation

Question

You work for a gaming company that manages a popular online multiplayer game where teams with 6 players play against each other in 5-minute battles. There are many new players every day. You need to build a model that automatically assigns available players to teams in real time. User research indicates that the game is more enjoyable when battles have players with similar skill levels. Which business metrics should you track to measure your model's performance?

Options

• AAverage time players wait before being assigned to a team
• BPrecision and recall of assigning players to teams based on their predicted versus actual ability
• CUser engagement as measured by the number of battles played daily per user
• DRate of return as measured by additional revenue generated minus the cost of developing a new

Explanation

For a team assignment model where user enjoyment is linked to skill-level matching, the most relevant business metric to track is user engagement, as it directly reflects whether players find the game more enjoyable and continue playing.

Common mistakes.

• A. Average time players wait before being assigned to a team is an operational metric measuring system efficiency, not the quality of the team assignments based on skill level or the resulting user enjoyment.
• B. Precision and recall are technical model evaluation metrics that assess the accuracy of the model's predictions, but they do not directly measure the business impact of improved user enjoyment or engagement.
• D. While revenue is a crucial business outcome, 'rate of return' is too high-level and indirectly linked; user engagement is a more immediate and measurable proxy for the success of a model specifically designed to improve player enjoyment through team assignments.

Concept tested. Business metrics for ML model success

No community discussion yet for this question.