An IS auditor is testing an AI-based fraud detection system that flags suspicious transactions and finds that the system has a high false positive rate. Which of the following testing methods should b

The correct answer is B. Cross-validation testing. Cross-validation (e.g., k-fold) partitions training data into multiple subsets and repeatedly trains and evaluates the model on different splits. This technique directly measures generalization performance, exposes overfitting or underfitting, and allows tuning of the decision th

AI Audit Planning and Execution

Question

An IS auditor is testing an AI-based fraud detection system that flags suspicious transactions and finds that the system has a high false positive rate. Which of the following testing methods should be prioritized to BEST optimize the detection rate?

Options

ARegression testing
BCross-validation testing
CSubstantive testing
DBenford's Law analysis

How the community answered

(39 responses)

A
15% (6)
B
72% (28)
C
8% (3)
D
5% (2)

Explanation

Cross-validation (e.g., k-fold) partitions training data into multiple subsets and repeatedly trains and evaluates the model on different splits. This technique directly measures generalization performance, exposes overfitting or underfitting, and allows tuning of the decision threshold and model parameters to balance precision against recall-directly optimizing the false positive/negative trade-off. Regression testing (A) verifies that existing functionality hasn't broken and is unrelated to ML performance metrics. Substantive testing (C) is a financial audit technique. Benford's Law analysis (D) detects anomalies in naturally occurring numerical datasets, not ML model calibration issues.

Topics

#AI model testing#Cross-validation#False positives#AI audit execution

Community Discussion

No community discussion yet for this question.

Full AAIA Practice