An ML engineer has trained a neural network by using stochastic gradient descent (SGD). The neural network performs poorly on the test set. The values for training loss and validation loss remain high

Question

Options

• AIntroduce early stopping.
• BIncrease the size of the test set.
• CIncrease the learning rate.
• DDecrease the learning rate.

• A
  5% (1)
• B
  5% (1)
• C
  9% (2)
• D
  82% (18)

Explanation

D is correct because an oscillating loss pattern - where loss decreases then increases repeatedly - is the classic symptom of a learning rate that is too large. SGD is "overshooting" the loss minimum on each step, bouncing back and forth across it rather than converging. Decreasing the learning rate shrinks each update step, allowing the optimizer to settle into a minimum rather than oscillate around it.

Why the distractors are wrong:

• A (Early stopping): Early stopping halts training when validation loss stops improving, but it treats the symptom, not the cause. It would simply cut off an already unstable training run without fixing the oscillation.
• B (Increase test set size): The problem is training instability, not a data quantity issue. A larger test set changes nothing about how the model trains.
• C (Increase the learning rate): This would make oscillation worse, causing even larger overshooting and potentially diverging loss.

Memory tip: Think of the learning rate as the size of your footsteps while hiking to a valley floor. Oscillating loss means your steps are so large you keep stepping over the valley to the other side - you need smaller steps (lower LR) to walk down into it precisely.

No community discussion yet for this question.