In Azure Data Factory, when do you need a Mapping Data Flow instead of a Copy activity?

## The two things you are choosing between Azure Data Factory ships two quite different ways of getting data somewhere useful, and interviewers ask this because picking the wrong one is the most common and most expensive ADF mistake. The **Copy activity** is the data-movement primitive. You give it a source dataset and a sink dataset and it streams rows from one store to the other. It has the widest connector surface in the service, it converts formats (delimited text to Parquet, JSON to tabular), it maps and renames columns and casts types in its mapping tab, and for many relational sources it lets you supply a query or a stored procedure so filtering and joining happen inside the source system before anything moves. What it will not do is take two independent inputs and combine them, group and aggregate, pivot, or mark a row as an update rather than an insert. A **Mapping Data Flow** is ADF's visual transformation designer. You build a directed graph - Source, Select, Derived Column, Filter, Conditional Split, Join, Lookup, Exists, Aggregate, Window, Pivot, Surrogate Key, Alter Row, Sink - and configure each node with expressions rather than code. At runtime the service compiles that graph into Apache Spark work and executes it on a Spark cluster that it provisions and manages. The whole point of the feature is Spark-scale transformation for people who do not want to write Spark. ## How the execution models differ A Copy activity runs as a data-movement job on an integration runtime. Its parallelism is expressed in Data Integration Units, the runtime is already available, and the activity begins moving rows almost immediately. A Data Flow activity does not do the work in the pipeline itself. It requests a Spark cluster whose shape comes from the data flow compute settings on the Azure integration runtime the activity runs on - compute type (general purpose, memory optimized, compute optimized) and core count. If no warm cluster is available for that integration runtime, acquiring one takes minutes before the first row is ever read, and you are paying for cluster time in core-hours while it runs. That asymmetry is the whole decision. Moving a 200-row lookup table with a data flow can spend minutes of cluster acquisition to do a few seconds of work, while a Copy activity would have finished before the cluster finished starting. ## A second, quieter difference: connector coverage The set of stores that a Mapping Data Flow can use as a source or sink is smaller than the set the Copy activity supports, and data flows run only on Azure integration runtimes. When your data lives somewhere a data flow cannot read - or behind a network boundary that needs a different runtime - the answer is not to abandon the transformation. It is to land the data first with a Copy activity into Blob Storage or ADLS Gen2, and point the data flow at the landed copy. This is the standard two-step shape and a good candidate says it without prompting. ## The third option nobody should forget There is always a third placement: let something else do the transformation and let ADF orchestrate it. A Stored Procedure or Script activity pushes SQL into a database or warehouse you are already paying for; a Databricks or Synapse notebook activity runs code your engineers can test. Mapping Data Flows sit between those and Copy: more capable than Copy, less capable and less testable than code, and worth their managed cluster only when the transformation genuinely needs a distributed engine and the team wants a visual surface. ## Choosing in practice - Pure movement, format conversion, column mapping, one source to one sink: **Copy activity**. - Many small tables to move on a schedule: a **ForEach over a parameterized Copy activity**, not one data flow per table. - Joining streams, deduplicating, aggregating, generating surrogate keys, driving upserts and deletes into a target table, and the team wants no code: **Mapping Data Flow**. - The transformation is already SQL and the target is a warehouse: push it down and let ADF orchestrate. - The logic needs libraries, unit tests and reviewable diffs: external compute. ## Cost shape, stated honestly Copy is billed on movement (Data Integration Unit time) plus activity runs. Data flows are billed on the cluster's core time. Neither is universally cheaper - a very large transformation that a data flow does in one pass can beat repeated Copy-and-stored-procedure round trips - but the fixed startup cost means a data flow's economics get worse the smaller and more frequent the job is. Consolidate: one data flow with several branches and several sinks costs one cluster acquisition, while five data flow activities cost up to five.