A company wants to implement a large language model (LLM) based chatbot to provide customer service agents with real-time contextual responses to customers' inquiries. The company will use the…

Question

Options

• ARetrain the LLM on the company policy data.
• BFine-tune the LLM on the company policy data.
• CImplement Retrieval Augmented Generation (RAG) for in-context responses.
• DUse pre-training and data augmentation on the company policy data.

• A
  4% (1)
• B
  13% (3)
• C
  75% (18)
• D
  8% (2)

Explanation

Explanation

RAG (Retrieval Augmented Generation) is the most cost-effective solution because it allows the LLM to dynamically retrieve relevant company policy documents at query time and inject them into the prompt as context - requiring no model training whatsoever, which eliminates the enormous computational and financial costs of modifying model weights.

Why the distractors are wrong:

• A (Retraining) is the most expensive option possible, requiring training the entire LLM from scratch on new data - completely unnecessary here.
• B (Fine-tuning) adjusts model weights using company data and is cheaper than retraining, but still requires significant GPU compute costs and ongoing retraining whenever policies change.
• D (Pre-training + data augmentation) is essentially retraining with extra steps - even more costly and complex than necessary.

Key insight: Options A, B, and D all bake knowledge permanently into the model weights, making policy updates extremely expensive. RAG instead keeps knowledge in an external, easily-updated document store, making it both cost-effective and flexible.

Memory Tip: Think of RAG as giving the LLM a "cheat sheet" at test time rather than forcing it to memorize everything - fast, cheap, and easy to update when policies change.

No community discussion yet for this question.