DP-600 · Question #119
You have a Fabric tenant that contains a lakehouse named Lakehouse1. You have an external Snowflake database that contains a table with 200 million rows. You need to use a data pipeline to migrate the
The correct answer is A. Data Pipeline (Copy datA.. For the most performant ingestion of a large 200 million-row table from Snowflake to a Fabric Lakehouse using a data pipeline, the Copy data activity is the fastest method.
Question
You have a Fabric tenant that contains a lakehouse named Lakehouse1. You have an external Snowflake database that contains a table with 200 million rows. You need to use a data pipeline to migrate the database to Lakehouse1. What is the most performant (fastest) method for ingesting data this large (200 million rows) by using a data pipeline?
Options
- AData Pipeline (Copy datA.
- BData Pipeline (Dataflow Gen2)
- CData Pipeline (Lookup)
- DData Pipeline Spark (Notebook)
How the community answered
(54 responses)- A80% (43)
- B4% (2)
- C9% (5)
- D7% (4)
Why each option
For the most performant ingestion of a large 200 million-row table from Snowflake to a Fabric Lakehouse using a data pipeline, the Copy data activity is the fastest method.
The Copy data activity within a Fabric data pipeline is purpose-built for high-performance, large-scale data ingestion. It leverages optimized connectors and parallel processing capabilities to efficiently transfer massive datasets from various sources, including Snowflake, to a Lakehouse, making it the fastest option for bulk migration.
Dataflow Gen2 is designed for data ingestion and transformation, but for pure high-volume, high-speed data transfer without complex transformations, the Copy data activity typically offers superior performance.
The Lookup activity is designed to retrieve a single row or a small set of configuration data, not to perform bulk ingestion of 200 million rows.
A Spark notebook activity requires writing custom code and introduces Spark cluster overhead, which might be less performant than the highly optimized, native Copy data activity for a straightforward bulk data migration.
Concept tested: High-performance data ingestion (Copy activity)
Source: https://learn.microsoft.com/en-us/fabric/data-factory/copy-data-performance-tuning-guide
Topics
Community Discussion
No community discussion yet for this question.