What is the most important difference between batch gradient descent, mini-batch gradient descent, and stochastic gradient descent?

Question

Options

• AGradient size
• BGradient direction
• CLearning rate
• DNumber of samples used

• A
  16% (8)
• B
  4% (2)
• C
  6% (3)
• D
  75% (38)

Explanation

Option D is correct because the defining characteristic that separates these three algorithms is how many training samples are used to compute each gradient update: batch gradient descent uses the entire dataset, mini-batch uses a fixed subset (e.g., 32–256 samples), and stochastic gradient descent uses exactly one sample per update.

Why the distractors are wrong:

• A (Gradient size) and B (Gradient direction) are consequences of the sample choice, not the defining difference - a single sample produces a noisier gradient, but that noise is a side effect, not the definition.
• C (Learning rate) is an independent hyperparameter; all three variants can use any learning rate, and varying it doesn't change which variant you're using.

Memory tip: Think of "batch" like cooking - full batch cooks the whole pot before tasting, mini-batch tastes a spoonful, and stochastic tastes one single noodle. The amount you taste (samples used) is what defines each approach.

No community discussion yet for this question.