Why do attention-based models (Transformers) scale better than RNNs for long sequences?

The correct answer is B. They enable full parallel computation. RNNs process tokens sequentially - each hidden state depends on the previous one, so training cannot be parallelized across time steps. This bottleneck becomes severe for long sequences. Transformers compute attention scores between all token pairs simultaneously in a single matr

NVIDIA Certified Associate (NCA) Core AI Concepts

Question

Options

AThey eliminate training
BThey enable full parallel computation
CThey remove embeddings
DThey reduce dataset size

How the community answered

(53 responses)

B
94% (50)
C
2% (1)
D
4% (2)

Explanation

RNNs process tokens sequentially - each hidden state depends on the previous one, so training cannot be parallelized across time steps. This bottleneck becomes severe for long sequences. Transformers compute attention scores between all token pairs simultaneously in a single matrix operation, allowing the entire sequence to be processed in parallel on modern GPU/TPU hardware. This is why Transformers train dramatically faster at scale. They do not eliminate training (A), they still use embeddings (C), and they do not reduce dataset size (D). The parallelism is the core architectural advantage.

Topics

#Transformers#RNNs#Parallel Computing#Deep Learning Architectures

Community Discussion

No community discussion yet for this question.

Full NCA-AIIO Practice