nerdexam
Microsoft

DP-700 · Question #61

You need to create a simple ETL pipeline that extracts data from a SQL Server database, performs basic data cleaning, and loads the data into a data warehouse. Which of the following options is the…

The correct answer is D. Use a combination of dataflows and T-SQL scripts, with notebooks for exploratory analysis. For a simple ETL pipeline involving SQL Server data extraction, basic cleaning, and loading into a data warehouse, a combination of dataflows and T-SQL scripts provides efficiency and cost-effectiveness, with notebooks supporting exploratory analysis.

Design and implement data ingestion and transformation

Question

You need to create a simple ETL pipeline that extracts data from a SQL Server database, performs basic data cleaning, and loads the data into a data warehouse. Which of the following options is the most efficient and cost-effective choice for this scenario ?

Options

  • AUse a dataflow with T-SQL scripts for data extraction and transformation.
  • BUse a series of T-SQL scripts to perform the ETL process.
  • CUse a dataflow with a Python notebook for data extraction and transformation.
  • DUse a combination of dataflows and T-SQL scripts, with notebooks for exploratory analysis.

How the community answered

(33 responses)
  • A
    6% (2)
  • B
    12% (4)
  • C
    15% (5)
  • D
    67% (22)

Why each option

For a simple ETL pipeline involving SQL Server data extraction, basic cleaning, and loading into a data warehouse, a combination of dataflows and T-SQL scripts provides efficiency and cost-effectiveness, with notebooks supporting exploratory analysis.

AUse a dataflow with T-SQL scripts for data extraction and transformation.

While dataflows are good, relying solely on T-SQL scripts within dataflows for all transformation might miss out on the visual transformation benefits and potentially overcomplicate the dataflow design for "basic" cleaning.

BUse a series of T-SQL scripts to perform the ETL process.

Using only T-SQL scripts can be inefficient for orchestrating cross-system data movement and lacks the built-in managed services, scalability, and visual development experience of modern ETL tools like dataflows.

CUse a dataflow with a Python notebook for data extraction and transformation.

Using a Python notebook for all data extraction and transformation within a dataflow might be overkill for "simple ETL" and could add unnecessary complexity and cost compared to leveraging the dataflow's native capabilities for basic cleaning.

DUse a combination of dataflows and T-SQL scripts, with notebooks for exploratory analysis.Correct

This option leverages the strengths of different tools: dataflows are efficient for visual ETL and basic transformations, T-SQL scripts can be highly effective for direct database operations (extraction, bulk loading), and notebooks offer flexibility for advanced cleaning, validation, or exploratory analysis. This combination provides a robust, adaptable, and often cost-effective solution by using the right tool for each specific task within the pipeline.

Concept tested: ETL pipeline design-tool selection

Source: https://learn.microsoft.com/en-us/fabric/data-factory/data-factory-overview

Topics

#ETL pipeline#Dataflows Gen2#T-SQL#Data Warehouse

Community Discussion

No community discussion yet for this question.

Full DP-700 Practice