What is the purpose of splitting documents into smaller overlapping chunks in a RAG system?

Question

Options

• ATo simplify the process of training the embedding model
• BTo enable the matching of different relevant passages to user queries
• CTo improve the efficiency of encoding queries into vector representations
• DTo reduce the storage space required for the vector database

• A
  10% (2)
• B
  81% (17)
• C
  5% (1)
• D
  5% (1)

Explanation

Splitting documents into overlapping chunks allows a retrieval system to surface different, targeted passages that each address different aspects of a user's query. A single long document might contain many relevant sections, and chunking lets the retrieval step match the most relevant portion rather than forcing the entire document to compete as one unit. Overlap between chunks ensures that context isn't lost at boundaries - a key sentence split across two chunks won't be missed.

Why the distractors are wrong:

• A is wrong because chunking happens at inference/indexing time, not during embedding model training - the model is trained separately on large corpora.
• C is wrong because query encoding is independent of how documents are chunked; the query vector is computed the same way regardless of chunk size.
• D is wrong because chunking actually increases storage - one document becomes many vectors - so it trades storage efficiency for retrieval precision.

Memory tip: Think of chunking like cutting a long book into index cards. You don't do it to save shelf space (not D) or to help someone write better summaries (not A/C) - you do it so you can pull out exactly the right card when someone asks a specific question (B).

No community discussion yet for this question.