DP-100 Question #530: Real Exam Question with Answer & Explanation

Question

Drag and Drop Question You manage an Azure Machine Learning workspace. You build automated machine learning training experiments for computer vision models. You need to use a primary metric for model optimization and hyperparameter tuning for each model. Which primary metrics should you use for the models? To answer, move the appropriate primary metrics to the correct computer vision models. You may use each primary metric once, more than once, or not at all. You may need to move the split bar between panes or scroll to view content. NOTE: Each correct selection is worth one point. Answer:

Explanation

The correct approach involves matching 'Intersection over union' to 'Multilabel image classification' and 'Mean average precision' to 'Object detection', aligning with specific evaluation metrics for these computer vision tasks.

Approach. The correct interaction, as depicted in the second image, is to drag 'Intersection over union' to 'Multilabel image classification' and 'Mean average precision' to 'Object detection'.

• For Multilabel image classification: The metric 'Intersection over union' (IoU) is selected. While common metrics for pure multilabel image classification tasks (assigning multiple labels to an entire image) typically include F1-score or Average Precision, IoU becomes relevant in scenarios where the multilabel task extends to identifying and localizing multiple distinct objects or regions within an image, akin to instance segmentation or semantic segmentation. In such contexts, where each label corresponds to a specific spatial region, IoU evaluates the overlap between the predicted and ground-truth regions for each labeled instance or segment, assessing the spatial accuracy of these predictions.
• For Object detection: The metric 'Mean average precision' (mAP) is the standard and widely accepted primary metric. Object detection models aim to identify objects within an image and draw bounding boxes around them. mAP provides a comprehensive measure of a detector's performance by averaging the average precision across all object classes and typically across various Intersection over Union (IoU) thresholds, accounting for both classification accuracy and localization precision of bounding boxes.

Common mistakes.

• common_mistake. Common mistakes include selecting metrics inappropriate for computer vision tasks or misapplying metrics between tasks.
• Absolute percentage error and Mean absolute error are metrics primarily used for regression tasks, where the model predicts a continuous numerical value. They are not suitable for classification, object detection, or segmentation tasks, which involve categorical predictions or spatial localization.
• Confusing 'Intersection over union' with 'Mean average precision' for object detection is another mistake. While IoU is a component of mAP (used for thresholding and evaluating individual detections), mAP is the more comprehensive and standard overall primary metric for object detection, as it considers precision-recall curves across multiple IoU thresholds and multiple classes. Using IoU directly as the primary metric for object detection would be less complete than mAP, although IoU is a valid metric for evaluating individual detections or for segmentation tasks.
• Incorrectly applying classification metrics (if they were options) to object detection, or vice-versa, indicates a misunderstanding of the output and goals of each computer vision task.

Concept tested. Understanding and applying appropriate evaluation metrics for different computer vision tasks, specifically multilabel image classification and object detection, in the context of machine learning model optimization and hyperparameter tuning.

No community discussion yet for this question.