nerdexam
Amazon

MLS-C01 · Question #300

A network security vendor needs to ingest telemetry data from thousands of endpoints that run all over the world. The data is transmitted every 30 seconds in the form of records that contain 50…

The correct answer is C. Use Amazon Kinesis Data Analytics to read and aggregate the data hourly. Transform the data. The vendor needs to ingest real-time telemetry data from Kinesis Data Streams, perform hourly aggregations and transformations, and store it in Parquet for Athena, minimizing customization.

Data Engineering

Question

A network security vendor needs to ingest telemetry data from thousands of endpoints that run all over the world. The data is transmitted every 30 seconds in the form of records that contain 50 fields. Each record is up to 1 KB in size. The security vendor uses Amazon Kinesis Data Streams to ingest the data. The vendor requires hourly summaries of the records that Kinesis Data Streams ingests. The vendor will use Amazon Athena to query the records and to generate the summaries. The Athena queries will target 7 to 12 of the available data fields. Which solution will meet these requirements with the LEAST amount of customization to transform and store the ingested data?

Options

  • AUse AWS Lambda to read and aggregate the data hourly. Transform the data and store it in
  • BUse Amazon Kinesis Data Firehose to read and aggregate the data hourly. Transform the data
  • CUse Amazon Kinesis Data Analytics to read and aggregate the data hourly. Transform the data
  • DUse Amazon Kinesis Data Firehose to read and aggregate the data hourly. Transform the data

How the community answered

(34 responses)
  • A
    9% (3)
  • B
    3% (1)
  • C
    82% (28)
  • D
    6% (2)

Why each option

The vendor needs to ingest real-time telemetry data from Kinesis Data Streams, perform hourly aggregations and transformations, and store it in Parquet for Athena, minimizing customization.

AUse AWS Lambda to read and aggregate the data hourly. Transform the data and store it in

Using AWS Lambda for hourly aggregation and transformation would require significant custom code development and orchestration, which increases customization effort compared to Kinesis Data Analytics.

BUse Amazon Kinesis Data Firehose to read and aggregate the data hourly. Transform the data

Amazon Kinesis Data Firehose can deliver data to S3 and convert formats, but its built-in transformation capabilities are more limited for complex hourly aggregations across multiple fields than Kinesis Data Analytics, likely requiring more manual pre- or post-processing.

CUse Amazon Kinesis Data Analytics to read and aggregate the data hourly. Transform the dataCorrect

Amazon Kinesis Data Analytics (KDA) is purpose-built for real-time processing of streaming data using SQL or Flink, making it ideal for performing hourly aggregations and transformations directly from Kinesis Data Streams. KDA can then output the processed data to Amazon S3 in Apache Parquet format, which is optimized for Athena queries, all with minimal custom coding.

DUse Amazon Kinesis Data Firehose to read and aggregate the data hourly. Transform the data

This choice is identical to B; Kinesis Data Firehose's transformation capabilities are generally less suitable for complex real-time aggregations than Kinesis Data Analytics, requiring more customization.

Concept tested: Real-time streaming ETL with Kinesis services

Source: https://aws.amazon.com/kinesis/data-analytics/

Topics

#Kinesis Data Analytics#Streaming Data Processing#Data Aggregation#Serverless Querying

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full MLS-C01 Practice