nerdexam
Databricks

GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #87

Generative AI Engineer at an electronics company just deployed a RAG application for customers to ask questions about products that the company carries. However, they received feedback that the RAG…

The correct answer is A. Assess the quality of the retrieved context. When a RAG system returns irrelevant product information, the root cause is poor retrieval quality, so assessing and improving the retrieved context is the correct first step.

RAG Application Optimization

Question

Generative AI Engineer at an electronics company just deployed a RAG application for customers to ask questions about products that the company carries. However, they received feedback that the RAG response often returns information about an irrelevant product. What can the engineer do to improve the relevance of the RAG’s response?

Options

  • AAssess the quality of the retrieved context
  • BImplement caching for frequently asked questions
  • CUse a different LLM to improve the generated response
  • DUse a different semantic similarity search algorithm

How the community answered

(40 responses)
  • A
    45% (18)
  • B
    15% (6)
  • C
    33% (13)
  • D
    8% (3)

Why each option

When a RAG system returns irrelevant product information, the root cause is poor retrieval quality, so assessing and improving the retrieved context is the correct first step.

AAssess the quality of the retrieved contextCorrect

Assessing the quality of retrieved context identifies whether the vector search is surfacing semantically similar but topically incorrect chunks, allowing the engineer to tune embedding models, similarity thresholds, or chunking strategies to fix the retrieval stage - where the actual problem lies.

BImplement caching for frequently asked questions

Caching addresses latency for repeated queries but does nothing to fix the underlying relevance problem in the retrieval pipeline.

CUse a different LLM to improve the generated response

Switching the LLM improves generation quality but cannot compensate for the retrieval stage returning the wrong product context in the first place.

DUse a different semantic similarity search algorithm

Changing the similarity search algorithm is a potential downstream fix, but without first assessing retrieved context quality you cannot determine whether the algorithm is even the cause of the irrelevance.

Concept tested: RAG retrieval quality assessment and debugging

Source: https://docs.databricks.com/en/generative-ai/retrieval-augmented-generation.html

Topics

#RAG#Retrieval Quality#Relevance#Troubleshooting

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full GENERATIVE-AI-ENGINEER-ASSOCIATE Practice