MLS-C01 · Question #301
A medical device company is building a machine learning (ML) model to predict the likelihood of device recall based on customer data that the company collects from a plain text survey. One of the…
The correct answer is A. Spell check the column. Use Amazon SageMaker one-hot encoding on the column to transform a. The task is to effectively encode a high-cardinality, redundant, and misspelled categorical feature (medication names) into a numeric feature.
Question
A medical device company is building a machine learning (ML) model to predict the likelihood of device recall based on customer data that the company collects from a plain text survey. One of the survey questions asks which medications the customer is taking. The data for this field contains the names of medications that customers enter manually. Customers misspell some of the medication names. The column that contains the medication name data gives a categorical feature with high cardinality but redundancy. What is the MOST effective way to encode this categorical feature into a numeric feature?
Options
- ASpell check the column. Use Amazon SageMaker one-hot encoding on the column to transform a
- BFix the spelling in the column by using char-RNN. Use Amazon SageMaker Data Wrangler one-
- CUse Amazon SageMaker Data Wrangler similarity encoding on the column to create embeddings
- DUse Amazon SageMaker Data Wrangler ordinal encoding on the column to encode categories
How the community answered
(47 responses)- A72% (34)
- B17% (8)
- C2% (1)
- D9% (4)
Why each option
The task is to effectively encode a high-cardinality, redundant, and misspelled categorical feature (medication names) into a numeric feature.
Spell checking the column first is crucial to consolidate misspellings into canonical medication names, reducing redundancy and effective cardinality. Once cleaned, one-hot encoding is a standard and effective method for converting categorical variables into a numeric format without implying ordinal relationships, creating a robust feature for the ML model.
Using a char-RNN for spelling correction is a complex deep learning approach that is overkill and less efficient than standard spell-checking for this specific problem and less automated within SageMaker Data Wrangler.
While similarity encoding can handle some variations, it might not effectively consolidate misspelled words into canonical forms as precisely as a dedicated spell check, potentially leading to less distinct embeddings if the initial data is very messy.
Ordinal encoding assigns an arbitrary integer rank to categories, which is inappropriate for medication names that lack inherent order or magnitude, potentially misleading the machine learning model.
Concept tested: Feature engineering for messy categorical data
Source: https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler-transformations.html
Topics
Community Discussion
No community discussion yet for this question.