Which of the following does not belong to automatic hyperparameter optimization algorithm?

Question

Options

• AGrid search
• BRandom gradient descent
• CRandom search
• DModel-based hyper parameter optimization

• A
  7% (1)
• B
  87% (13)
• C
  7% (1)

Explanation

Random gradient descent (B) is an optimization algorithm used to minimize a model's loss function during training - it tunes model parameters (weights), not hyperparameters. Hyperparameters, by definition, must be set before training begins and cannot be learned by gradient descent.

Grid search (A) is a classic automatic method that exhaustively evaluates every combination in a predefined hyperparameter grid. Random search (C) is also automatic, sampling combinations randomly from the search space and is often more efficient than grid search. Model-based hyperparameter optimization (D) - typically Bayesian optimization - automatically builds a surrogate model of the objective function to intelligently select the next hyperparameter configuration to try.

Memory tip: If an algorithm uses gradients and updates weights during training, it belongs to model optimization, not hyperparameter optimization. Hyperparameter tuners work around training runs, not inside them.

No community discussion yet for this question.