AIGP · Question #191
CASE STUDY Please use the following to answer the next question: A leading insurance provider that offers a range of coverage options to individuals has decided to utilize AI to streamline and…
The correct answer is A. Continue disparity testing. During initial production monitoring, the most important action is to continue disparity testing to detect and quantify whether the model is producing biased outcomes across protected groups. This directly aligns with the goal of evaluating fairness, accuracy, and reliability…
Question
CASE STUDY Please use the following to answer the next question:
A leading insurance provider that offers a range of coverage options to individuals has decided to utilize AI to streamline and improve its customer acquisition and underwriting process, including the accuracy and efficiency of pricing policies. The company has engaged a cloud provider to utilize and fine-tune its pre-trained, general purpose large language model (“LLM”). The company intends to use its historical customer data - including applications, policies and claims - and proprietary pricing and risk strategies to provide an initial qualification assessment of potential customers, which would then be routed to a human underwriter for final review. The company and the cloud provider have completed training and testing the LLM, performed a readiness assessment, and made the decision to deploy the LLM into production. They have designated an internal compliance team to monitor the model during the first month, specifically to evaluate the accuracy, fairness and reliability of its output. After the first month in production, the company realizes that the LLM declines a higher percentage of women’s applications. During the first month when the company monitors the model for bias, it is most important to:
Options
- AContinue disparity testing.
- BProvide regular awareness training.
- CAnalyze the quality of the training and testing data.
- DDocument the results of final decisions made by the human underwriter.
How the community answered
(55 responses)- A58% (32)
- B15% (8)
- C7% (4)
- D20% (11)
Explanation
During initial production monitoring, the most important action is to continue disparity testing to detect and quantify whether the model is producing biased outcomes across protected groups. This directly aligns with the goal of evaluating fairness, accuracy, and reliability, enabling the organization to identify and respond to discriminatory patterns early.
Topics
Community Discussion
No community discussion yet for this question.