nerdexam
Google

PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #340

You deployed a conversational application that uses a large language model (LLM). The application has 1,000 users. You collect user feedback about the verbosity and accuracy of the model 's…

The correct answer is C. Ask users to indicate all scenarios where they expect concise responses versus verbose. The responses are already factually correct - only verbosity is mismatched. The scalable fix is to gather user-defined scenarios specifying when concise versus verbose responses are expected, then encode those rules into the model's system prompt or prompt templates that are…

Submitted by yasin.bd· Apr 18, 2026Monitoring, optimizing, and maintaining ML solutions

Question

You deployed a conversational application that uses a large language model (LLM). The application has 1,000 users. You collect user feedback about the verbosity and accuracy of the model 's responses. The user feedback indicates that the responses are factually correct but users want different levels of verbosity depending on the type of question. You want the model to return responses that are more consistent with users' expectations, and you want to use a scalable solution. What should you do?

Options

  • AImplement a keyword-based routing layer. If the user's input contains the words "detailed" or
  • BAsk users to provide examples of responses with the appropriate verbosity as a list of question
  • CAsk users to indicate all scenarios where they expect concise responses versus verbose
  • DExperiment with other proprietary and open-source LLMs. Perform A/B testing by setting each

How the community answered

(32 responses)
  • A
    13% (4)
  • B
    6% (2)
  • C
    78% (25)
  • D
    3% (1)

Explanation

The responses are already factually correct - only verbosity is mismatched. The scalable fix is to gather user-defined scenarios specifying when concise versus verbose responses are expected, then encode those rules into the model's system prompt or prompt templates that are selected dynamically based on question type. This leverages prompt engineering rather than retraining or replacing the model, and scales to 1,000+ users without per-user customization. Option A (keyword routing on 'detailed'/'concise') is brittle - it requires users to always include specific keywords. Option B (few-shot examples from users) is operationally heavy to collect and maintain at scale. Option D (A/B test other LLMs) replaces the model entirely, which does not address the root cause - the verbosity issue is about prompting strategy, not model capability.

Topics

#Large Language Models (LLM)#Prompt Engineering#User Feedback#Model Optimization

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice