You work for a large social network service provider whose users post articles and discuss news. Millions of comments are posted online each day, and more than 200 human moderators constantly review…

Question

Options

• ANumber of messages flagged by the model per minute
• BNumber of messages flagged by the model per minute confirmed as being inappropriate by
• CPrecision and recall estimates based on a random sample of 0.1% of raw messages each minute
• DPrecision and recall estimates based on a sample of messages flagged by the model as

• A
  14% (3)
• B
  5% (1)
• C
  9% (2)
• D
  73% (16)

Explanation

In a human-in-the-loop content moderation system, the model's job is to surface suspicious comments for human review. Option D - computing precision and recall on a sample of messages the model flagged - directly measures how well the model performs that job. Precision tells you what fraction of flagged comments are truly inappropriate (i.e., is the model wasting reviewers' time?); recall on the flagged set tells you about the model's confidence distribution. Option C (random 0.1% of all raw messages) is misleading because the vast majority of raw messages are clean - the resulting metrics would reflect the class imbalance, not the model's flagging quality. Option A measures only volume. Option B is a raw count of true positives, not a normalized metric - it doesn't account for the total number of flags or misses.

No community discussion yet for this question.