DP-100 · Question #560
You manage an Azure AI Foundry project. You build a multi-turn chatbot application. You plan to filter your traces to identify issues while observing how the application is responding. The solution mu
The correct answer is C. CoherenceEvaluator. CoherenceEvaluator measures the logical consistency and flow of chatbot responses across turns without requiring an external knowledge base or reference answers.
Question
Options
- ASimilarityEvaluator
- BRelevanceEvaluator
- CCoherenceEvaluator
- DIndirectAttackEvaluator
How the community answered
(29 responses)- A3% (1)
- B14% (4)
- C76% (22)
- D7% (2)
Why each option
CoherenceEvaluator measures the logical consistency and flow of chatbot responses across turns without requiring an external knowledge base or reference answers.
SimilarityEvaluator computes semantic similarity between a generated response and a reference ground-truth answer, requiring known correct outputs that depend on an external knowledge base or dataset.
RelevanceEvaluator measures how relevant a response is to retrieved context from an external knowledge base, making it inapplicable when no external knowledge base is used.
CoherenceEvaluator assesses how logically well-structured and consistent chatbot responses are within a conversation, which is directly applicable to a multi-turn chatbot. It evaluates internal quality of responses rather than comparing against external data, satisfying the constraint of no external knowledge base, and is useful for identifying conversational breakdown issues in traces.
IndirectAttackEvaluator is a safety metric designed to detect indirect prompt injection vulnerabilities, not to assess response quality or conversational coherence.
Concept tested: Azure AI Foundry built-in evaluator selection for multi-turn chatbot
Source: https://learn.microsoft.com/en-us/azure/ai-studio/concepts/evaluation-metrics-built-in
Topics
Community Discussion
No community discussion yet for this question.