GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #59
Which indicator should be considered to evaluate the safety of the LLM outputs when qualitatively assessing LLM responses for a translation use case?
The correct answer is D. The accuracy and relevance of the responses. For a translation use case, evaluating the accuracy and relevance of responses is the primary qualitative indicator of both output quality and safety.
Question
Which indicator should be considered to evaluate the safety of the LLM outputs when qualitatively assessing LLM responses for a translation use case?
Options
- AThe ability to generate responses in code
- BThe similarity to the previous language
- CThe latency of the response and the length of text generated
- DThe accuracy and relevance of the responses
How the community answered
(14 responses)- A7% (1)
- C7% (1)
- D86% (12)
Why each option
For a translation use case, evaluating the accuracy and relevance of responses is the primary qualitative indicator of both output quality and safety.
The ability to generate code is a capability metric unrelated to the safety or quality of translation outputs.
Similarity to the previous language is not a meaningful or standard safety metric for translation evaluation.
Latency and text length are performance and efficiency metrics, not qualitative indicators of translation safety or accuracy.
Accuracy ensures the translated output faithfully represents the source content without introducing misinformation or distortion. Relevance ensures the response stays on-topic and contextually appropriate. Together these two dimensions are the core qualitative safety signals for a translation system, covering both correctness and appropriateness of the output.
Concept tested: Qualitative safety evaluation of LLM translation outputs
Source: https://docs.databricks.com/en/generative-ai/llm-evaluation.html
Topics
Community Discussion
No community discussion yet for this question.