AIF-C01 · Question #263
An education company is building a chatbot whose target audience is teenagers. The company is training a custom large language model (LLM). The company wants the chatbot to speak in the target…
The correct answer is D. Bilingual Evaluation Understudy (BLEU) score. BLEU evaluates surface-level n-gram overlap between the LLM’s outputs and reference examples-in this case, targets written in creative spelling and shorthand, making it well suited to measure how closely the model’s style matches the teenager-oriented language.
Question
An education company is building a chatbot whose target audience is teenagers. The company is training a custom large language model (LLM). The company wants the chatbot to speak in the target audience's language style by using creative spelling and shortened words. Which metric will assess the LLM's performance?
Options
- AF1 score
- BBERTScore
- CRecall-Oriented Understudy for Gisting Evaluation (ROUGE)
- DBilingual Evaluation Understudy (BLEU) score
How the community answered
(60 responses)- A13% (8)
- B3% (2)
- C5% (3)
- D78% (47)
Explanation
BLEU evaluates surface-level n-gram overlap between the LLM’s outputs and reference examples-in this case, targets written in creative spelling and shorthand, making it well suited to measure how closely the model’s style matches the teenager-oriented language.
Topics
Community Discussion
No community discussion yet for this question.