nerdexam
IAPP

AIGP · Question #46

CASE STUDY Please use the following to answer the next question: A leading insurance provider that offers a range of coverage options to individuals has decided to utilize AI to streamline and improve

The correct answer is A. Continue disparity testing.. Continuing disparity testing during monitoring helps identify ongoing bias issues in model outputs, allowing for timely mitigation and improvement.

AI Governance Program

Question

CASE STUDY Please use the following to answer the next question:

A leading insurance provider that offers a range of coverage options to individuals has decided to utilize AI to streamline and improve its customer acquisition and underwriting process, including the accuracy and efficiency of pricing policies. The company has engaged a cloud provider to utilize and fine-tune its pre-trained, general purpose large language model (“LLM”). The company intends to use its historical customer data - including applications, policies and claims - and proprietary pricing and risk strategies to provide an initial qualification assessment of potential customers, which would then be routed to a human underwriter for final review. The company and the cloud provider have completed training and testing the LLM, performed a readiness assessment, and made the decision to deploy the LLM into production. They have designated an internal compliance team to monitor the model during the first month, specifically to evaluate the accuracy, fairness and reliability of its output. After the first month in production, the company realizes that the LLM declines a higher percentage of women’s applications. During the first month when the company monitors the model for bias, it is most important to:

Options

  • AContinue disparity testing.
  • BAnalyze the quality of the training and testing data.
  • CCompare the results to human decisions prior to deployment.
  • DSeek approval from management for any changes to the model.

How the community answered

(25 responses)
  • A
    64% (16)
  • B
    20% (5)
  • C
    12% (3)
  • D
    4% (1)

Explanation

Continuing disparity testing during monitoring helps identify ongoing bias issues in model outputs, allowing for timely mitigation and improvement.

Topics

#disparity testing#bias monitoring#fairness#ongoing model governance

Community Discussion

No community discussion yet for this question.

Full AIGP Practice