An IS auditor is reviewing a machine learning model that predicts the likelihood that a user will watch a certain movie. Which of the following would be of GREATEST concern to the auditor?

Question

Options

• AThe data set for training the model was obtained from an unreliable source
• BWhen the model was tested with data drawn from a different population, the accuracy decreased
• CAn open source programming language was used to develop the model
• DThe model was tested with data drawn from the same population as the training data

• A
  42% (19)
• B
  29% (13)
• C
  9% (4)
• D
  20% (9)

Explanation

Unreliable training data is the greatest concern because ML models are only as trustworthy as the data they learn from - if the source is compromised, biased, or fabricated, every prediction the model makes is built on a flawed foundation, and no amount of testing can salvage it ("garbage in, garbage out").

Why the distractors are wrong:

• B describes expected ML behavior - a model trained on one population performing worse on a different population is a known limitation, not a red flag, and can be addressed through re-training or scope adjustments.
• C is irrelevant to model reliability; open source languages like Python and R are industry-standard tools for ML development and carry no inherent risk.
• D is actually acceptable practice - testing on data drawn from the same population (but a different sample) is normal; the concern would be testing on the exact same records used for training, which D does not say.

Memory tip: In IS audit questions, always trace the problem to its root. For ML models, data quality is the root - if the soil is poisoned, the tree will be too, regardless of how well you tend the branches.

No community discussion yet for this question.