nerdexam
Databricks

GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #6

A Generative AI Engineer is developing a RAG system for their company to perform internal document Q&A for structured HR policies, but the answers returned are frequently incomplete and…

The correct answer is A. Add the section header as a prefix to chunks E. Fine tune the response generation model. The problem is twofold: incomplete answers (retriever misses relevant chunks) and unstructured output (generation model formats poorly). (A) Adding the section header as a prefix to each chunk gives the embedding model richer semantic context about what a chunk covers - for…

RAG System Design and Optimization

Question

A Generative AI Engineer is developing a RAG system for their company to perform internal document Q&A for structured HR policies, but the answers returned are frequently incomplete and unstructured. It seems that the retriever is not returning all relevant context. The Generative AI Engineer has experimented with different embedding and response generating LLMs but that did not improve results. Which TWO options could be used to improve the response quality? (Choose two.)

Options

  • AAdd the section header as a prefix to chunks
  • BSplit the document by sentence
  • CUse a larger embedding model
  • DIncrease the document chunk size
  • EFine tune the response generation model

How the community answered

(28 responses)
  • A
    46% (13)
  • B
    32% (9)
  • C
    7% (2)
  • D
    14% (4)

Explanation

The problem is twofold: incomplete answers (retriever misses relevant chunks) and unstructured output (generation model formats poorly). (A) Adding the section header as a prefix to each chunk gives the embedding model richer semantic context about what a chunk covers - for structured HR documents organized by section, this significantly improves retrieval recall. (E) Fine-tuning the response generation model on structured HR policy Q&A teaches the model to produce complete, well-formatted answers consistent with the document style. Option B (splitting by sentence) would make chunks too small, worsening completeness. Option C (larger embedding model) was already explored without improvement. Option D (larger chunks) might reduce precision and introduce noise without addressing the structural output quality problem.

Topics

#RAG Optimization#Context Retrieval#Text Chunking#LLM Fine-tuning

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full GENERATIVE-AI-ENGINEER-ASSOCIATE Practice