Which performance metric is BEST suited to assess the quality of trained models detecting fraudulent credit card transactions?

Question

Options

• ASensitivity
• BAccuracy
• CF1 value

• A
  15% (4)
• B
  7% (2)
• C
  78% (21)

Explanation

The ISTQB CT-AI syllabus explains in Section3.2 - Functional Performance Criteria of ML Modelsthat accuracy becomes unreliable when class imbalance exists. In fraud detection, more than 99% of transactions are non-fraudulent, meaning the dataset is extremely imbalanced. Because accuracy counts all correct non-fraudulent classifications, it will appear artificially high, even if the fraud detection performance is poor. Therefore, accuracy is not suitable for evaluating fraud detection systems. The syllabus further explains that sensitivity (recall)captures the proportion of correctly identified fraudulent cases. This metric is important, as missing fraudulent events can cause high financial loss. However, the client also stresses thatlegitimate transactions must be correctly identified, meaning false positives must be minimized to maintain customer satisfaction. The F1 score, defined as the harmonic mean of precision and recall, balances both: Precision protects legitimate customers by minimizing false alarms. Recall ensures fraudulent transactions are detected. Section 3.2 emphasizes that when both false positives and false negatives have significant consequences, and the data is highly imbalanced, F1 is the most appropriate metricbecause it reflects the combined importance of detecting fraud while avoiding unnecessary alerts. Thus, Option C is the correct choice.

No community discussion yet for this question.