Which method will drop undesired logs and reduce the amount of data being ingested?

Question

Options

• A[COLLECT:vendor="vendor", product="product", target_brokers="", no_hit=drop] * drop _raw_log
• B[INGEST:vendor="vendor", product="product", target_dataset="vendor_product_raw",no_hit=drop]
• C[COLLECT:vendor="vendor", product="product", target_dataset="", no_hit=drop] * drop _raw_log
• D[INGEST:vendor="vendor", product="product", target_brokers="vendor_product_raw",

• A
  15% (8)
• B
  8% (4)
• C
  73% (38)
• D
  4% (2)

Explanation

Option C correctly combines two mechanisms that work together to reduce ingested data: the no_hit=drop parameter silently discards logs that don't match any parsing rules, and the * drop _raw_log action removes the raw log field from logs that do match - preventing the full raw text from being stored and billed. Using COLLECT (not INGEST) is critical here because COLLECT operates before data enters the ingestion pipeline, giving you the earliest and cheapest place to filter.

Why the others fail:

• A uses target_brokers instead of target_dataset in a COLLECT block - target_brokers is not the correct parameter here, making the directive invalid.
• B uses INGEST, which processes data after it's already entering the pipeline; it also lacks drop _raw_log, so raw log volume is not reduced.
• D also misuses INGEST with target_brokers (a mismatched parameter combination) and is syntactically incomplete.

Memory tip: Think "COLLECT = Cut early, cut cheap." COLLECT filters before ingestion, no_hit=drop cuts unmatched logs, and drop _raw_log trims the payload - three layers of reduction, all before data costs accumulate. If you see INGEST or target_brokers in a COLLECT block, that's a red flag.

No community discussion yet for this question.