nerdexam
ExamsGENERATIVE-AI-ENGINEER-ASSOCIATEQuestions#60
DatabricksDatabricks

GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #60

GENERATIVE-AI-ENGINEER-ASSOCIATE Question #60: Real Exam Question with Answer & Explanation

The correct answer is B: Curate a dataset that can test the retrieval and generation components of the system separately.. See the full explanation below for the reasoning.

Evaluating RAG Systems for Improvement

Question

A Generative AI Engineer has created a RAG application which can help employees retrieve answers from an internal knowledge base, such as Confluence pages or Google Drive. The prototype application is now working with some positive feedback from internal company testers. Now the Generative AI Engineer wants to formally evaluate the system's performance and understand where to focus their efforts to further improve the system. How should the Generative AI Engineer evaluate the system?

Options

  • AUse cosine similarity score to comprehensively evaluate the quality of the final generated
  • BCurate a dataset that can test the retrieval and generation components of the system separately.
  • CBenchmark multiple LLMs with the same data and pick the best LLM for the job.
  • DUse an LLM-as-a-judge to evaluate the quality of the final answers generated.

Topics

#RAG System Evaluation#Component-wise Evaluation#System Diagnostics#Generative AI Engineering

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full GENERATIVE-AI-ENGINEER-ASSOCIATE PracticeBrowse All GENERATIVE-AI-ENGINEER-ASSOCIATE Questions