An IS auditor is interviewing management about implemented controls around machine learning (ML) models deployed in the production environment. Which of the following schedules for reviewing the perfo

The correct answer is C. One time prior to migrating to production. Reviewing model performance only once - prior to production migration - is the greatest concern because it treats a dynamic system as static. ML models are subject to concept drift (the relationship between inputs and outputs changes over time) and data drift (the statistical dis

AI Risk Management and Controls

Question

An IS auditor is interviewing management about implemented controls around machine learning (ML) models deployed in the production environment. Which of the following schedules for reviewing the performance of a deployed model would be of GREATEST concern to the auditor?

Options

AAfter changes to hardware and software platforms
BAfter functionality changes
COne time prior to migrating to production
DOn an annual recurring basis

How the community answered

(21 responses)

A
5% (1)
B
5% (1)
C
76% (16)
D
14% (3)

Explanation

Reviewing model performance only once - prior to production migration - is the greatest concern because it treats a dynamic system as static. ML models are subject to concept drift (the relationship between inputs and outputs changes over time) and data drift (the statistical distribution of input data shifts), both of which degrade prediction accuracy after deployment. A single pre-production review provides no assurance about ongoing performance. Post-change reviews (A, B) are appropriate event-driven triggers. Annual reviews (D) are infrequent but at least establish a recurring cadence. A one-time review with no subsequent monitoring leaves the organization with no mechanism to detect or respond to model degradation.

Topics

#ML model monitoring#Post-deployment review#AI audit#Model performance review

Community Discussion

No community discussion yet for this question.

Full AAIA Practice