GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #64
GENERATIVE-AI-ENGINEER-ASSOCIATE Question #64: Real Exam Question with Answer & Explanation
The correct answer is C: Curate a dataset that can test the retrieval and generation components of the system separately.. To evaluate the system, it is important to break it down into its core components - retrieval and generation - and test them independently to understand their individual performance. Curating a dataset that tests these two components separately helps to identify specific areas of
Question
A Generative AI Engineer has created a RAG application which can help employees interpret HR documentation. The prototype application is now working with some positive feedback from internal company testers. Now the Generative AI Engineer wants to formally evaluate the system's performance and understand where to focus their efforts to further improve the system How should the Generative AI Engineer evaluate the system?
Options
- AUse ROUGE score to comprehensively evaluate the quality of the final generated answers.
- BUse an LLM-as-a-judge to evaluate the quality of the final answers generated.
- CCurate a dataset that can test the retrieval and generation components of the system separately.
- DBenchmark multiple LLMs with the same data and pick the best LLM for the job.
Explanation
To evaluate the system, it is important to break it down into its core components - retrieval and generation - and test them independently to understand their individual performance. Curating a dataset that tests these two components separately helps to identify specific areas of improvement. MLflow’s evaluation metrics can then be used to assess the performance of these components, providing a comprehensive evaluation of the system's strengths and weaknesses.
Topics
Community Discussion
No community discussion yet for this question.