GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #102
GENERATIVE-AI-ENGINEER-ASSOCIATE Question #102: Real Exam Question with Answer & Explanation
The correct answer is A: Massive Multi-task Language Understanding (MMLU) score. The MMLU score is a benchmark used to evaluate an LLM's performance on a variety of tasks, but it is not directly relevant for monitoring the performance of a customer service LLM application in production. MMLU is typically used for assessing general model capabilities during de
Question
A Generative AI Engineer has just deployed an LLM application at a manufacturing company that assists with answering customer service inquiries. They need to identity the key enterprise metrics to monitor the application in production. Which is NOT a metric they will implement for their customer service LLM application in production?
Options
- AMassive Multi-task Language Understanding (MMLU) score
- BNumber of customer inquiries processed per unit of time
- CFactual accuracy of the response
- DTime taken for LLM to generate a response
Explanation
The MMLU score is a benchmark used to evaluate an LLM's performance on a variety of tasks, but it is not directly relevant for monitoring the performance of a customer service LLM application in production. MMLU is typically used for assessing general model capabilities during development and benchmarking, not for real-time operational monitoring. Exam Questions, Study Guides, Practice Tests. Lead the way to help you pass any IT Certification exams, 100% Pass Guaranteed or Full Refund. Especially Cisco, Microsoft, CompTIA, Citrix, EMC, HP, Oracle, VMware, Juniper, Check Point, LPI, Nortel, EXIN and so on. Our Slogan: First Test, First Pass. Help you to pass any IT Certification exams at the first try. You can reach us at any of the email addresses listed below. Any problems about IT certification or our products, you could rely upon us, we will give you satisfactory answers in 24 hours.
Topics
Community Discussion
No community discussion yet for this question.