A company wants to ingest streaming data into an Amazon Redshift data warehouse from an Amazon Managed Streaming for Apache Kafka (Amazon MSK) cluster. A data engineer needs to develop a solution…

Question

Options

• ACreate an external schema that maps to the MSK cluster. Create a materialized view that
• BDevelop an AWS Glue streaming extract, transform, and load (ETL) job to process the incoming
• CCreate an external schema that maps to the streaming data source. Create a new Amazon
• DCreate an Amazon S3 bucket. Ingest the data from Amazon MSK. Create an event-driven AWS

• A
  3% (1)
• B
  76% (25)
• C
  15% (5)
• D
  6% (2)

Explanation

By using a serverless Glue streaming job to continuously pull your MSK records, transform them as needed, and land them in Parquet (or another columnar) files in S3, you: 1. Optimize storage costs ?your data sits in S3, where you pay pennies per GB-month and can tier it further with lifecycle rules. 2. Get low-latency access ?Redshift Spectrum lets you query S3-backed tables with millisecond planning time, so freshly landed data becomes queryable almost immediately. 3. Minimize ops overhead ?you don't have to stand up or manage any EC2-based brokers, Lambda polling loops, or custom connector infrastructure. Glue's managed streaming runtime handles checkpointing, autoscaling, and fault tolerance for Once the data lands in S3, you simply define an external schema in Redshift that points at the Glue Data Catalog database where your streaming job writes tables. Analysts can then query the "live" dataset via Spectrum as if it were inside Redshift, meeting both your performance and cost goals with minimal operational effort.

No community discussion yet for this question.