DP-700 · Question #61
You need to create a simple ETL pipeline that extracts data from a SQL Server database, performs basic data cleaning, and loads the data into a data warehouse. Which of the following options is the…
The correct answer is D. Use a combination of dataflows and T-SQL scripts, with notebooks for exploratory analysis. For a simple ETL pipeline involving SQL Server data extraction, basic cleaning, and loading into a data warehouse, a combination of dataflows and T-SQL scripts provides efficiency and cost-effectiveness, with notebooks supporting exploratory analysis.
Question
Options
- AUse a dataflow with T-SQL scripts for data extraction and transformation.
- BUse a series of T-SQL scripts to perform the ETL process.
- CUse a dataflow with a Python notebook for data extraction and transformation.
- DUse a combination of dataflows and T-SQL scripts, with notebooks for exploratory analysis.
How the community answered
(33 responses)- A6% (2)
- B12% (4)
- C15% (5)
- D67% (22)
Why each option
For a simple ETL pipeline involving SQL Server data extraction, basic cleaning, and loading into a data warehouse, a combination of dataflows and T-SQL scripts provides efficiency and cost-effectiveness, with notebooks supporting exploratory analysis.
While dataflows are good, relying solely on T-SQL scripts within dataflows for all transformation might miss out on the visual transformation benefits and potentially overcomplicate the dataflow design for "basic" cleaning.
Using only T-SQL scripts can be inefficient for orchestrating cross-system data movement and lacks the built-in managed services, scalability, and visual development experience of modern ETL tools like dataflows.
Using a Python notebook for all data extraction and transformation within a dataflow might be overkill for "simple ETL" and could add unnecessary complexity and cost compared to leveraging the dataflow's native capabilities for basic cleaning.
This option leverages the strengths of different tools: dataflows are efficient for visual ETL and basic transformations, T-SQL scripts can be highly effective for direct database operations (extraction, bulk loading), and notebooks offer flexibility for advanced cleaning, validation, or exploratory analysis. This combination provides a robust, adaptable, and often cost-effective solution by using the right tool for each specific task within the pipeline.
Concept tested: ETL pipeline design-tool selection
Source: https://learn.microsoft.com/en-us/fabric/data-factory/data-factory-overview
Topics
Community Discussion
No community discussion yet for this question.