DEA-C01 · Question #123
A company uses Amazon S3 as a data lake. The company sets up a data warehouse by using a multi-node Amazon Redshift cluster. The company organizes the data files in the data lake based on the data sou
The correct answer is D. Create a manifest file that contains the data file locations. Use a COPY command to load the data. Amazon Redshift's COPY command is optimized for parallel loading-it distributes file loading across all compute nodes simultaneously. When multiple separate COPY commands are issued (one per folder), each command runs sequentially, preventing parallelism. A manifest file lists al
Question
A company uses Amazon S3 as a data lake. The company sets up a data warehouse by using a multi-node Amazon Redshift cluster. The company organizes the data files in the data lake based on the data source of each data file. The company loads all the data files into one table in the Redshift cluster by using a separate COPY command for each data file location. This approach takes a long time to load all the data files into the table. The company must increase the speed of the data ingestion. The company does not want to increase the cost of the process. Which solution will meet these requirements?
Options
- AUse a provisioned Amazon EMR cluster to copy all the data files into one folder. Use a COPY
- BLoad all the data files in parallel into Amazon Aurora. Run an AWS Glue job to load the data into
- CUse an AWS Give job to copy all the data files into one folder. Use a COPY command to load the
- DCreate a manifest file that contains the data file locations. Use a COPY command to load the data
How the community answered
(36 responses)- A6% (2)
- B3% (1)
- C11% (4)
- D81% (29)
Explanation
Amazon Redshift's COPY command is optimized for parallel loading-it distributes file loading across all compute nodes simultaneously. When multiple separate COPY commands are issued (one per folder), each command runs sequentially, preventing parallelism. A manifest file lists all S3 file locations in a single JSON document, allowing one COPY command to load all files in parallel across all Redshift nodes-dramatically reducing load time at no additional cost. Options A and C (EMR or Glue to consolidate files first) add unnecessary compute costs. Option B (loading into Aurora first) introduces both cost and latency.
Topics
Community Discussion
No community discussion yet for this question.