In the gradient descent algorithm, which of the following algorithms is the most confusing algorithm for the trajectory on the loss function surface?

Question

Options

• ASGD
• BBGD
• CMGD
• DMBGD

• A
  82% (32)
• B
  5% (2)
• C
  3% (1)
• D
  10% (4)

Explanation

SGD (Stochastic Gradient Descent) produces the most erratic, "confusing" trajectory because it computes gradients from a single randomly chosen training sample per update, causing high variance - the loss surface path zigzags wildly as each step reacts to a noisy, unrepresentative gradient estimate. BGD (Batch Gradient Descent) is wrong because it uses the entire dataset per update, yielding a smooth, stable descent toward the minimum. MBGD (Mini-Batch Gradient Descent) is wrong because it strikes a middle ground - using a subset of samples - producing moderate noise but far less chaos than SGD. MGD (Momentum Gradient Descent) is wrong because the momentum term dampens oscillations, actually smoothing the trajectory compared to plain SGD.

Memory tip: Think "S for Single sample, S for Scattered path" - SGD's one-at-a-time updates make it the lone wolf of gradient descent, wandering erratically rather than marching steadily downhill.

No community discussion yet for this question.