MLS-C01 · Question #359
A data scientist is implementing a deep learning neural network model for an object detection task on images. The data scientist wants to experiment with a large number of parallel hyperparameter…
The correct answer is D. Hyperband. To efficiently tune hyperparameters for a deep learning model with many parallel jobs, stopping underperforming ones early and allocating resources to promising configurations to minimize computational time, Hyperband is the most effective technique.
Question
A data scientist is implementing a deep learning neural network model for an object detection task on images. The data scientist wants to experiment with a large number of parallel hyperparameter tuning jobs to find hyperparameters that optimize compute time. The data scientist must ensure that jobs that underperform are stopped. The data scientist must allocate computational resources to well-performing hyperparameter configurations. The data scientist is using the hyperparameter tuning job to tune the stochastic gradient descent (SGD) learning rate, momentum, epoch, and mini-batch size. Which technique will meet these requirements with LEAST computational time?
Options
- AGrid search
- BRandom search
- CBayesian optimization
- DHyperband
How the community answered
(40 responses)- A13% (5)
- B25% (10)
- C5% (2)
- D57% (23)
Why each option
To efficiently tune hyperparameters for a deep learning model with many parallel jobs, stopping underperforming ones early and allocating resources to promising configurations to minimize computational time, Hyperband is the most effective technique.
Grid search exhaustively evaluates every combination of hyperparameters, which is computationally expensive and does not offer mechanisms to stop underperforming jobs early or intelligently allocate resources.
Random search randomly samples hyperparameter combinations, which is more efficient than grid search but lacks the adaptive resource allocation and early stopping capabilities of Hyperband.
Bayesian optimization intelligently selects hyperparameters based on past performance, but it typically focuses on finding the global optimum with fewer trials rather than explicitly stopping underperforming jobs early or reallocating resources to minimize computational time like Hyperband.
Hyperband is a hyperparameter optimization algorithm specifically designed to minimize computational time by efficiently exploring a large search space, actively stopping underperforming jobs early, and dedicating more resources to the most promising hyperparameter configurations.
Concept tested: Efficient hyperparameter optimization techniques (early stopping)
Source: https://docs.aws.amazon.com/sagemaker/latest/dg/hyperparameter-tuning-how-it-works.html
Topics
Community Discussion
No community discussion yet for this question.