PROFESSIONAL-MACHINE-LEARNING-ENGINEER Question #170: Real Exam Question with Answer & Explanation

Question

Your organization manages an online message board. A few months ago, you discovered an increase in toxic language and bullying on the message board. You deployed an automated text classifier that flags certain comments as toxic or harmful. Now some users are reporting that benign comments referencing their religion are being misclassified as abusive. Upon further inspection, you find that your classifier's false positive rate is higher for comments that reference certain underrepresented religious groups. Your team has a limited budget and is already overextended. What should you do?

Options

• AAdd synthetic training data where those phrases are used in non-toxic ways.
• BRemove the model and replace it with human moderation.
• CReplace your model with a different text classifier.
• DRaise the threshold for comments to be considered toxic or harmful.

Explanation

To address a high false positive rate for benign comments from underrepresented religious groups with limited resources, you should raise the classification threshold for comments to be considered toxic.

Common mistakes.

• A. Adding synthetic training data requires development effort for data generation and integration, which may not be feasible for an overextended team with a limited budget.
• B. Replacing the model with human moderation would incur significant operational costs and discard the existing ML investment, which is contrary to having a limited budget.
• C. Replacing the model with a different text classifier demands substantial research, development, and deployment efforts, which are not suitable for a team that is already overextended and has a limited budget.

Concept tested. Model bias mitigation with classification thresholds

Reference. https://developers.google.com/machine-learning/crash-course/classification/precision-and-recall

No community discussion yet for this question.