SAP-C02 · Question #558
A company ingests and processes streaming market data. The data rate is constant. A nightly process that calculates aggregate statistics takes 4 hours to complete. The statistical analysis is not…
The correct answer is B. Update the ingestion process to use Amazon Kinesis Data Firehose to save data to Amazon S3. To optimize costs and architecture for constant streaming data ingestion and non-critical nightly processing, switch to Kinesis Data Firehose for S3 ingestion and utilize EC2 Spot Instances for the flexible, fault-tolerant batch processing.
Question
A company ingests and processes streaming market data. The data rate is constant. A nightly process that calculates aggregate statistics takes 4 hours to complete. The statistical analysis is not critical to the business, and data points are processed during the next iteration if a particular run fails. The current architecture uses a pool of Amazon EC2 Reserved Instances with 1-year reservations. These EC2 instances run full time to ingest and store the streaming data in attached Amazon Elastic Block Store (Amazon EBS) volumes. A scheduled script launches EC2 On- Demand Instances each night to perform the nightly processing. The instances access the stored data from NFS shares on the ingestion servers. The script terminates the instances when the processing is complete. The Reserved Instance reservations are expiring. The company needs to determine whether to purchase new reservations or implement a new design. Which solution will meet these requirements MOST cost-effectively?
Options
- AUpdate the ingestion process to use Amazon Kinesis Data Firehose to save data to Amazon S3.
- BUpdate the ingestion process to use Amazon Kinesis Data Firehose to save data to Amazon S3.
- CUpdate the ingestion process to use a fleet of EC2 Reserved Instances with 3-year reservations
- DUpdate the ingestion process to use Amazon Kinesis Data Firehose to save data to Amazon
How the community answered
(31 responses)- A3% (1)
- B84% (26)
- C10% (3)
- D3% (1)
Why each option
To optimize costs and architecture for constant streaming data ingestion and non-critical nightly processing, switch to Kinesis Data Firehose for S3 ingestion and utilize EC2 Spot Instances for the flexible, fault-tolerant batch processing.
This option suggests using EC2 On-Demand instances for processing, which is more expensive than Spot Instances, especially for a non-critical, fault-tolerant batch workload.
Amazon Kinesis Data Firehose is a fully managed service that simplifies ingesting and delivering streaming data to destinations like Amazon S3, eliminating the need for EC2 instances for ingestion and EBS volumes for storage, thus reducing costs and operational overhead. For the nightly statistical analysis, which is not critical and can tolerate failures, EC2 Spot Instances are a highly cost-effective choice. They offer significant discounts compared to On-Demand instances and are suitable for fault-tolerant, flexible workloads that can be interrupted, making them ideal for the batch processing component.
Using 3-year Reserved Instances for ingestion still retains the overhead of managing EC2 instances and EBS, which is less cost-effective and managed than Kinesis Data Firehose, and it also doesn't optimize the nightly processing.
While Kinesis Data Firehose and S3 are good for ingestion, using AWS Batch with EC2 On-Demand instances for the nightly processing is not the most cost-optimized choice compared to using Spot Instances for a fault-tolerant workload.
Concept tested: Cost optimization for streaming data and batch processing
Source: https://aws.amazon.com/kinesis/data-firehose/
Community Discussion
No community discussion yet for this question.