Which factor MOST increases variance in reinforcement learning policy gradient methods?

The correct answer is B. Sparse or delayed rewards. Policy gradient methods estimate the gradient of expected return using sampled trajectories. When rewards are sparse or delayed, the agent must complete long action sequences before receiving any signal, and many different trajectories may produce the same sparse reward for entir

NVIDIA Certified Associate (NCA) Core AI Concepts

Question

Options

ADeterministic rewards
BSparse or delayed rewards
CSmall state space
DTabular representations

How the community answered

(27 responses)

B
89% (24)
C
7% (2)
D
4% (1)

Explanation

Policy gradient methods estimate the gradient of expected return using sampled trajectories. When rewards are sparse or delayed, the agent must complete long action sequences before receiving any signal, and many different trajectories may produce the same sparse reward for entirely different reasons. This makes the return estimate noisy and highly variable across episodes, inflating gradient variance. Techniques like baselines and advantage functions are specifically designed to combat this. Deterministic rewards (A) actually reduce variance. A small state space (C) and tabular representations (D) simplify the problem and tend to reduce variance, not increase it.

Topics

#Reinforcement Learning#Policy Gradient#Variance#Sparse Rewards

Community Discussion

No community discussion yet for this question.

Full NCA-AIIO Practice