GENERATIVE-AI-ENGINEER-ASSOCIATE · Question #67
Which steps are essential when writing chunked text into Delta Lake tables in Unity Catalog? (Choose two.)
The correct answer is B. Partitioning data based on categories D. Structuring chunks to support efficient querying. Writing chunked text to Delta Lake in Unity Catalog requires partitioning by category and structuring chunks for efficient querying to ensure performant and well-organized data storage.
Question
Which steps are essential when writing chunked text into Delta Lake tables in Unity Catalog? (Choose two.)
Options
- AWriting all chunks as a single file
- BPartitioning data based on categories
- CAvoiding partitions for simplicity
- DStructuring chunks to support efficient querying
How the community answered
(53 responses)- A8% (4)
- B89% (47)
- C4% (2)
Why each option
Writing chunked text to Delta Lake in Unity Catalog requires partitioning by category and structuring chunks for efficient querying to ensure performant and well-organized data storage.
Writing all chunks as a single file creates a massive, unpartitioned file that is slow to read and cannot leverage Delta Lake's parallel processing or partition pruning.
Partitioning by category distributes data into separate storage directories, allowing Delta Lake to skip irrelevant partitions during queries and dramatically reducing scan costs.
Avoiding partitions forces every query to scan the entire dataset, which is inefficient and costly at scale in a large Delta Lake table.
Structuring chunks with consistent schemas and meaningful metadata fields ensures that downstream queries can filter, join, and retrieve chunks efficiently without full table scans.
Concept tested: Delta Lake chunked text storage with Unity Catalog partitioning
Source: https://docs.databricks.com/en/delta/best-practices.html
Topics
Community Discussion
No community discussion yet for this question.