A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table. The code block used by the data engineer is…

The correct answer is B. trigger(availableNow=True). trigger(availableNow=True) was introduced in Spark 3.3 as a more flexible replacement for trigger(once=True). It instructs Structured Streaming to process all data that is currently available at job start across as many micro-batches as needed, then stop - unlike…

Submitted by renata2k· Apr 18, 2026ELT with Spark SQL and Python

Question

A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table. The code block used by the data engineer is below: If the data engineer only wants the query to process all of the available data in as many batches as required, which of the following lines of code should the data engineer use to fill in the blank?

Options

AprocessingTime(1)
Btrigger(availableNow=True)
Ctrigger(parallelBatch=True)
Dtrigger(processingTime="once")
Etrigger(continuous="once")

How the community answered

(23 responses)

A
4% (1)
B
91% (21)
E
4% (1)

Explanation

trigger(availableNow=True) was introduced in Spark 3.3 as a more flexible replacement for trigger(once=True). It instructs Structured Streaming to process all data that is currently available at job start across as many micro-batches as needed, then stop - unlike trigger(once=True) which forces everything into a single batch. This is ideal for incremental batch-style workloads without a live continuous stream. processingTime sets a fixed interval for continuous streaming. trigger(continuous=...) enables experimental millisecond-latency continuous processing. trigger(parallelBatch=True) is not a valid Spark Streaming trigger option.

Topics

#Structured Streaming#Streaming Triggers#availableNow Trigger#Batch Processing

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-DATA-ENGINEER-ASSOCIATE Practice