MLS-C01 · Question #367
A tourism company uses a machine learning (ML) model to make recommendations to customers. The company uses an Amazon SageMaker environment and set hyperparameter tuning completion criteria to…
The correct answer is C. CompleteOnConvergence. To stop a SageMaker hyperparameter tuning job when an internal algorithm determines that further improvement over the objective metric is unlikely (e.g., less than 1%), the ML specialist should set the completion criteria to CompleteOnConvergence.
Question
A tourism company uses a machine learning (ML) model to make recommendations to customers. The company uses an Amazon SageMaker environment and set hyperparameter tuning completion criteria to MaxNumberOfTrainingJobs. An ML specialist wants to change the hyperparameter tuning completion criteria. The ML specialist wants to stop tuning immediately after an internal algorithm determines that tuning job is unlikely to improve more than 1% over the objective metric from the best training job. Which completion criteria will meet this requirement?
Options
- AMaxRuntimeInSeconds
- BTargetObjectiveMetricValue
- CCompleteOnConvergence
- DMaxNumberOfTrainingJobsNotImproving
How the community answered
(56 responses)- A4% (2)
- B7% (4)
- C79% (44)
- D11% (6)
Why each option
To stop a SageMaker hyperparameter tuning job when an internal algorithm determines that further improvement over the objective metric is unlikely (e.g., less than 1%), the ML specialist should set the completion criteria to `CompleteOnConvergence`.
`MaxRuntimeInSeconds` stops the tuning job after a specified maximum time duration, irrespective of the improvement or convergence of the objective metric.
`TargetObjectiveMetricValue` stops the tuning job when a predefined absolute target value for the objective metric is reached, not when the rate of improvement becomes negligible.
The `CompleteOnConvergence` completion criterion for SageMaker hyperparameter tuning jobs enables SageMaker's internal algorithms to monitor the objective metric and automatically stop the tuning process when it is determined that further training jobs are unlikely to yield significant improvements, such as improving by more than 1% over the best objective metric found so far. This precisely matches the described requirement.
`MaxNumberOfTrainingJobsNotImproving` stops the tuning job after a specified number of consecutive training jobs fail to improve the objective metric, which is a simpler rule than the sophisticated convergence detection described.
Concept tested: SageMaker Hyperparameter Tuning completion criteria
Source: https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-stopping-criteria.html#automatic-model-tuning-complete-on-convergence
Topics
Community Discussion
No community discussion yet for this question.