DEA-C01 · Question #51
A data engineer needs to build an extract, transform, and load (ETL) job. The ETL job will process daily incoming .csv files that users upload to an Amazon S3 bucket. The size of each S3 object is…
The correct answer is C. Write an AWS Glue PySpark job. Use Apache Spark to transform the data. AWS Glue Python Shell Job is billed $0.44 per DPU-Hour for each job. AWS Glue PySpark is billed $0.29 per DPU-Hour for each job with flexible execution and $0.44 per DPU-Hour for each job with standard execution. https://aws.amazon.com/glue/pricing/
Question
A data engineer needs to build an extract, transform, and load (ETL) job. The ETL job will process daily incoming .csv files that users upload to an Amazon S3 bucket. The size of each S3 object is less than 100 MB. Which solution will meet these requirements MOST cost-effectively?
Options
- AWrite a custom Python application. Host the application on an Amazon Elastic Kubernetes
- BWrite a PySpark ETL script. Host the script on an Amazon EMR cluster.
- CWrite an AWS Glue PySpark job. Use Apache Spark to transform the data.
- DWrite an AWS Glue Python shell job. Use pandas to transform the data.
How the community answered
(50 responses)- A2% (1)
- B6% (3)
- C78% (39)
- D14% (7)
Explanation
AWS Glue Python Shell Job is billed $0.44 per DPU-Hour for each job. AWS Glue PySpark is billed $0.29 per DPU-Hour for each job with flexible execution and $0.44 per DPU-Hour for each job with standard execution. https://aws.amazon.com/glue/pricing/
Topics
Community Discussion
No community discussion yet for this question.