nerdexam
Databricks

GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #90

A Generative AI Engineer is building a RAG application that answers questions about internal documents for the company SnoPen AI. The source documents may contain a significant amount of irrelevant…

The correct answer is C. Include in the system prompt that the application is not supposed to answer any questions. The most reliable way to prevent a RAG application from answering questions about irrelevant topics is to explicitly instruct the model in the system prompt to refuse out-of-scope queries.

Building and Optimizing RAG Applications

Question

A Generative AI Engineer is building a RAG application that answers questions about internal documents for the company SnoPen AI. The source documents may contain a significant amount of irrelevant content, such as advertisements, sports news, or entertainment news, or content about other companies. Which approach is advisable when building a RAG application to achieve this goal of filtering irrelevant information?

Options

  • AKeep all articles because the RAG application needs to understand non-company content to
  • BInclude in the system prompt that any information it sees will be about SnoPenAI, even if no data
  • CInclude in the system prompt that the application is not supposed to answer any questions
  • DConsolidate all SnoPen AI related documents into a single chunk in the vector database.

How the community answered

(26 responses)
  • A
    15% (4)
  • B
    8% (2)
  • C
    73% (19)
  • D
    4% (1)

Why each option

The most reliable way to prevent a RAG application from answering questions about irrelevant topics is to explicitly instruct the model in the system prompt to refuse out-of-scope queries.

AKeep all articles because the RAG application needs to understand non-company content to

Retaining all off-topic documents in the vector index guarantees that irrelevant content will be retrieved and surfaced to users, directly undermining the goal of focused, relevant responses.

BInclude in the system prompt that any information it sees will be about SnoPenAI, even if no data

Instructing the model that all information it sees is about SnoPen AI when it may not be is a false premise that can cause the model to hallucinate company-specific details from unrelated content.

CInclude in the system prompt that the application is not supposed to answer any questionsCorrect

Including an explicit instruction in the system prompt that the application should not answer questions outside its defined scope - SnoPen AI content - establishes a hard behavioral boundary enforced at inference time, preventing the model from using retrieved or parametric knowledge about irrelevant topics.

DConsolidate all SnoPen AI related documents into a single chunk in the vector database.

Consolidating all SnoPen AI documents into a single chunk destroys granularity in the vector index, making semantic retrieval ineffective and preventing meaningful similarity matching.

Concept tested: System prompt design for RAG scope restriction

Source: https://docs.databricks.com/en/generative-ai/prompt-engineering.html

Topics

#RAG Applications#Prompt Engineering#Information Filtering#LLM Behavior Control

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full GENERATIVE-AI-ENGINEER-ASSOCIATE Practice