CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #83
The business intelligence team has a dashboard configured to track various summary metrics for retail stories. This includes total sales for the previous day alongside totals and averages for a…
The correct answer is B. Populate the dashboard by configuring a nightly batch job to save the required to quickly update. The requirements are: data refreshed once daily, fast interactive queries throughout the day, and minimal total compute. Option B - a nightly batch job that pre-aggregates required fields and saves them to a small summary table - satisfies all three. Dashboard queries hit a…
Question
The business intelligence team has a dashboard configured to track various summary metrics for retail stories. This includes total sales for the previous day alongside totals and averages for a variety of time periods. The fields required to populate this dashboard have the following schema:
For Demand forecasting, the Lakehouse contains a validated table of all itemized sales updated incrementally in near real-time. This table named products_per_order, includes the following fields:
Because reporting on long-term sales trends is less volatile, analysts using the new dashboard only require data to be refreshed once daily. Because the dashboard will be queried interactively by many users throughout a normal business day, it should return results quickly and reduce total compute associated with each materialization. Which solution meets the expectations of the end users while controlling and limiting possible costs?
Options
- AUse the Delta Cache to persists the products_per_order table in memory to quickly the dashboard
- BPopulate the dashboard by configuring a nightly batch job to save the required to quickly update
- CUse Structure Streaming to configure a live dashboard against the products_per_order table
- DDefine a view against the products_per_order table and define the dashboard against this view.
- EConfigure a webhook to execute an incremental read against products_per_order each time the
How the community answered
(40 responses)- A3% (1)
- B70% (28)
- C3% (1)
- D18% (7)
- E8% (3)
Explanation
The requirements are: data refreshed once daily, fast interactive queries throughout the day, and minimal total compute. Option B - a nightly batch job that pre-aggregates required fields and saves them to a small summary table - satisfies all three. Dashboard queries hit a compact, pre-computed table rather than the large products_per_order table, ensuring fast response times. Compute runs once nightly rather than on every dashboard query. Option A (Delta Cache) caches the full raw table in memory - expensive and still requires aggregation at query time. Option C (Structured Streaming) is overkill for once-daily refresh. Option D (view) recomputes aggregations on every query, which is slow and costly at interactive scale. Pre-aggregation is the standard pattern for this scenario.
Topics
Community Discussion
No community discussion yet for this question.