CT-AI · Question #31
Consider a natural language processing (NLP) algorithm that attempts to predict the next word that you would like to type in a text message. An update to the algorithm has been created that should…
The correct answer is B. A/B testing. The syllabus states: "A/B testing can be used to test updates to an AI-based system where there are agreed acceptance criteria, such as ML functional performance metrics, as described in Chapter 5. A/B testing is used to compare the updated variant with the previous variant."
Question
Consider a natural language processing (NLP) algorithm that attempts to predict the next word that you would like to type in a text message. An update to the algorithm has been created that should increase the accuracy of the predictions based on user typing patterns. The old algorithm was rated for accuracy by the users. Then, after the new update was released, the users rated the updated algorithm. A statistical test was used to compare between the two versions of the algorithm to see whether or not the update should remain in place. This is an example of what type of testing?
Options
- AMetamorphic testing
- BA/B testing
- CExploratory testing
- DPairwise testing
How the community answered
(24 responses)- A13% (3)
- B79% (19)
- C4% (1)
- D4% (1)
Explanation
The syllabus states: "A/B testing can be used to test updates to an AI-based system where there are agreed acceptance criteria, such as ML functional performance metrics, as described in Chapter 5. A/B testing is used to compare the updated variant with the previous variant."
Topics
Community Discussion
No community discussion yet for this question.