MLS-C01 · Question #46
A Data Scientist is working on optimizing a model during the training process by varying multiple parameters. The Data Scientist observes that, during multiple runs with identical parameters, the…
The correct answer is B. Reduce the batch size. Decrease the learning rate. It is most likely that the loss function is very curvy and has multiple local minima where the training is getting stuck. Decreasing the batch size would help the Data Scientist stochastically get out of the local minima saddles. Decreasing the learning rate would prevent…
Question
A Data Scientist is working on optimizing a model during the training process by varying multiple parameters. The Data Scientist observes that, during multiple runs with identical parameters, the loss function converges to different, yet stable, values. What should the Data Scientist do to improve the training process?
Options
- AIncrease the learning rate. Keep the batch size the same.
- BReduce the batch size. Decrease the learning rate.
- CKeep the batch size the same. Decrease the learning rate.
- DDo not change the learning rate. Increase the batch size.
How the community answered
(42 responses)- A14% (6)
- B76% (32)
- C5% (2)
- D5% (2)
Explanation
It is most likely that the loss function is very curvy and has multiple local minima where the training is getting stuck. Decreasing the batch size would help the Data Scientist stochastically get out of the local minima saddles. Decreasing the learning rate would prevent overshooting the global loss function minimum.
Topics
Community Discussion
No community discussion yet for this question.