DAS-C01 · Question #70
A financial company uses Amazon S3 as its data lake and has set up a data warehouse using a multi- node Amazon Redshift cluster. The data files in the data lake are organized in folders based on the…
The correct answer is D. Create a manifest file that contains the data file locations and issue a COPY command to load the. A Redshift COPY manifest file is a JSON document that lists multiple S3 file paths (including paths in different prefixes/folders). A single COPY command with a manifest file loads all listed files in parallel using Redshift's massively parallel processing, which is far faster…
Question
A financial company uses Amazon S3 as its data lake and has set up a data warehouse using a multi- node Amazon Redshift cluster. The data files in the data lake are organized in folders based on the data source of each data file. All the data files are loaded to one table in the Amazon Redshift cluster using a separate COPY command for each data file location. With this approach, loading all the data files into Amazon Redshift takes a long time to complete. Users want a faster solution with little or no increase in cost while maintaining the segregation of the data files in the S3 data lake. Which solution meets these requirements?
Options
- AUse Amazon EMR to copy all the data files into one folder and issue a COPY command to load
- BLoad all the data files in parallel to Amazon Aurora, and run an AWS Glue job to load the data
- CUse an AWS Glue job to copy all the data files into one folder and issue a COPY command to
- DCreate a manifest file that contains the data file locations and issue a COPY command to load the
How the community answered
(30 responses)- A3% (1)
- B10% (3)
- C13% (4)
- D73% (22)
Explanation
A Redshift COPY manifest file is a JSON document that lists multiple S3 file paths (including paths in different prefixes/folders). A single COPY command with a manifest file loads all listed files in parallel using Redshift's massively parallel processing, which is far faster than issuing sequential COPY commands. The files remain in their original S3 locations, preserving data segregation. Options A and C require copying data to a new location (extra cost, extra latency). Option B adds Aurora as an unnecessary intermediary, increasing both cost and complexity.
Topics
Community Discussion
No community discussion yet for this question.