What best describes tokenization?

Question

Options

• AAdding lexical relations to the raw text
• BConverting text into the list of terms
• CConverting text into a list of unique terms
• DReducing variant forms of tokens to their base forms

• A
  4% (1)
• B
  84% (21)
• C
  4% (1)
• D
  8% (2)

No community discussion yet for this question.