nerdexam
Amazon

MLS-C01 · Question #184

A data scientist is training a text classification model by using the Amazon SageMaker built-in BlazingText algorithm. There are 5 classes in the dataset, with 300 samples for category A, 292…

The correct answer is A. Classes C and D are too similar. Based on the context of evaluating a classification model with confusion matrices, if the matrices show a high rate of misclassification specifically between classes C and D, it indicates that these classes are too similar or have overlapping features, making them inherently…

Modeling

Question

A data scientist is training a text classification model by using the Amazon SageMaker built-in BlazingText algorithm. There are 5 classes in the dataset, with 300 samples for category A, 292 samples for category B, 240 samples for category C, 258 samples for category D, and 310 samples for category E. The data scientist shuffles the data and splits off 10% for testing. After training the model, the data scientist generates confusion matrices for the training and test sets. What could the data scientist conclude form these results?

Exhibits

MLS-C01 question #184 exhibit 1
MLS-C01 question #184 exhibit 2

Options

  • AClasses C and D are too similar.
  • BThe dataset is too small for holdout cross-validation.
  • CThe data distribution is skewed.
  • DThe model is overfitting for classes B and E.

How the community answered

(23 responses)
  • A
    70% (16)
  • B
    9% (2)
  • C
    4% (1)
  • D
    17% (4)

Why each option

Based on the context of evaluating a classification model with confusion matrices, if the matrices show a high rate of misclassification specifically between classes C and D, it indicates that these classes are too similar or have overlapping features, making them inherently difficult for the model to distinguish.

AClasses C and D are too similar.Correct

If the confusion matrices, especially on the test set, show that a significant number of samples from class C are misclassified as class D, and vice-versa, it is a strong indication that the model struggles to differentiate between these two classes because their underlying features are too similar or overlap considerably. This is a direct insight derived from analyzing confusion matrix patterns.

BThe dataset is too small for holdout cross-validation.

While the overall dataset size and split (10% for testing) are factors in model training, concluding that the dataset is "too small for holdout cross-validation" is a general statement about the experimental setup and not a specific conclusion that directly arises from analyzing confusion matrices for specific class performance or similarity.

CThe data distribution is skewed.

The provided class distribution (A: 300, B: 292, C: 240, D: 258, E: 310) is relatively balanced, with no single class being overwhelmingly larger or smaller than others, so it is incorrect to conclude that the data distribution is skewed.

DThe model is overfitting for classes B and E.

Without specific performance metrics (e.g., higher training accuracy than test accuracy) or details from the confusion matrices, one cannot definitively conclude that the model is overfitting specifically for classes B and E; overfitting is a general model behavior, not typically localized to just two classes unless there's an unusual bias.

Concept tested: Interpreting confusion matrices for class similarity

Source: https://scikit-learn.org/stable/modules/model_evaluation.html#confusion-matrix

Topics

#Text Classification#Model Evaluation#Confusion Matrix#Class Similarity

Community Discussion

No community discussion yet for this question.

Full MLS-C01 Practice