The activation function plays an important role in the neural network model learning and understanding of very complex problems. The following statement about the activation function is correct.

Question

Options

• AActivation functions are linear functions
• BActivation functions are non-linear functions
• CThe activation function is partly a nonlinear function, partly a linear function
• DMost of the activation functions are nonlinear functions, and a few are linear functions

• A
  1% (1)
• B
  84% (58)
• C
  10% (7)
• D
  4% (3)

Explanation

Activation functions must be non-linear to give neural networks their power - without non-linearity, stacking multiple layers would collapse into a single linear transformation, making deep networks mathematically equivalent to a single-layer model and incapable of learning complex patterns like images, speech, or language. Option A is wrong because purely linear activation functions (e.g., f(x) = x) eliminate the benefit of depth entirely. Option C is wrong because "partly nonlinear, partly linear" mischaracterizes the definition - an activation function is either one or the other, not a blend of both categories. Option D is a tempting near-miss, but it's incorrect because there is no meaningful class of activation functions in standard use that are truly linear (even ReLU, which is linear in each piece, is piecewise non-linear overall and is classified as non-linear).

Memory tip: Think of it this way - "activation" implies doing something interesting; a linear function passes data through unchanged (boring), while a non-linear function activates the network's ability to model complexity. All real activation functions (Sigmoid, Tanh, ReLU, Leaky ReLU, Softmax) are non-linear - no exceptions in practice.

No community discussion yet for this question.