skip to content

In Azure Data Factory, why can a pipeline report Succeeded after its Copy activity failed?

level: seniorimportance: nice to knowfreq 35%

answer

  1. the handler ran, and the handler worked
  2. status comes from how the run ended
  3. an unmet condition skips, it does not fail
  4. one activity exists purely to end red

basics

~20 s

Because the pipeline's status is derived from the final activities that ran. If a failure-path activity handled the error and itself succeeded, the run ends with no failed leaf activity and Azure Data Factory reports the pipeline as Succeeded.

solid answer

~40 s

Each activity's `dependsOn` entry carries a dependency condition — `Succeeded`, `Failed`, `Skipped` or `Completed` — and a failure path is just an activity that depends on `Failed`. ADF computes the pipeline outcome from how the run ended, not from whether any activity ever failed along the way, so the classic try/catch shape (Copy → on Failed → a Web activity that logs the error) finishes with a successful leaf and the run shows **Succeeded**. Monitoring, alerts on pipeline failure, and any parent `Execute Pipeline` therefore see green. If the failure must be visible, end the failure path with a **Fail** activity, which sets a custom `message` and `errorCode` and forces the run to end Failed. To repair, use rerun from the failed activity in Monitor so the already-successful activities are not repeated.

code

json · 19 lines
json
{
  "activities": [
    { "name": "CopyOrders", "type": "Copy" },
    {
      "name": "LogError",
      "type": "WebActivity",
      "dependsOn": [{ "activity": "CopyOrders", "dependencyConditions": ["Failed"] }]
    },
    {
      "name": "FailRun",
      "type": "Fail",
      "dependsOn": [{ "activity": "LogError", "dependencyConditions": ["Succeeded"] }],
      "typeProperties": {
        "message": "@concat('Copy failed: ', activity('CopyOrders').Error.message)",
        "errorCode": "COPY_ORDERS_FAILED"
      }
    }
  ]
}

go deeper

for a junior

Know that arrows between activities carry conditions — success, failure, skipped, completed — and that a failure arrow is how errors are routed.

for a middle

Explain that run status is derived from terminal activity states, so a successful handler on the failure path leaves the run green, and name the Fail activity as the way to force red.

for a senior

Connect it to operations: alerts that never fired, parent pipelines that carried on, cleanup on the Completed condition, and rerun-from-failed with its idempotency and parameter-reuse caveats.

for a principal

Own the failure-handling convention across the estate — when a handled error is a business outcome versus an incident, what gets paged, and how run status maps to the freshness SLA the consumers actually care about.

## Dependency conditions are the control flow ADF has no separate error-handling block. Ordering and error handling are the same mechanism: every activity's `dependsOn` array names prior activities and a condition on each. - `Succeeded` — run only if the predecessor succeeded (the default green arrow). - `Failed` — run only if the predecessor failed. - `Skipped` — run only if the predecessor was skipped, which happens when *its* condition was not met. - `Completed` — run regardless of outcome. Multiple entries on one activity are ANDed: every listed predecessor must satisfy its condition. Combined with the fact that an unmet condition makes an activity *skipped* rather than failed, this gives ADF a small algebra that surprises people. ## How the run status is computed The pipeline run's status comes from how the graph terminated — specifically from the final activities on the paths that actually executed. Take the common try/catch shape: ```text CopyOrders --(Succeeded)--> TransformOrders CopyOrders --(Failed)-----> LogError (Web activity) ``` If `CopyOrders` fails, `TransformOrders` is skipped, `LogError` runs and succeeds — and the run terminates with its executed leaf in a succeeded state. ADF reports **Succeeded**. The activity-level view still shows the Copy in red, but the run row in Monitor is green, the failure alert never fires, and a parent pipeline that invoked this one with `waitOnCompletion: true` continues down its own success path. This is the single most surprising behaviour in ADF error handling, and it is where the interview question comes from: a team "has alerting on pipeline failure" and still misses outages, because their error handler is doing its job too well. ## Making failure visible again: the Fail activity The supported fix is the **Fail** activity. Put it at the end of the failure path, after whatever logging or cleanup you want, and the run terminates Failed with your own message and error code: ```json { "name": "FailRun", "type": "Fail", "dependsOn": [{ "activity": "LogError", "dependencyConditions": ["Succeeded"] }], "typeProperties": { "message": "@concat('Copy failed: ', activity('CopyOrders').Error.message)", "errorCode": "COPY_ORDERS_FAILED" } } ``` Note `@activity('CopyOrders').Error.message`, which is how a downstream activity reads the upstream error text — useful for the log payload as well as for the failure message. The result is the best of both: the error is recorded somewhere queryable *and* the run is red, so alerting and parent pipelines behave. The alternative shapes are worth knowing by name. A "do-if-else" arrangement routes both outcomes and deliberately lets the run end green because the error is a business outcome, not an incident. A cleanup activity depending on `Completed` runs either way and is the ADF equivalent of a finally block. ## Skipped is not failed The second half of the surprise is `Skipped`. When `CopyOrders` fails, `TransformOrders` is *skipped*, and anything depending on `TransformOrders` with a `Succeeded` condition is skipped in turn, cascading quietly to the end of the branch. Skipped activities are not errors, which is why a partially executed pipeline can end green even without an explicit handler if the failing branch had no downstream leaf in a failed state. Reading a run's activity list and understanding which activities were skipped versus failed is a core Monitor skill. ## Repairing a run ADF's Monitor blade offers a rerun from a chosen point — commonly "rerun from the failed activity". This creates a new run that reuses the original run's parameter values and resumes at that activity, so the successful upstream work is not repeated. That matters when the upstream step was a two-hour copy and the failure was in a five-second stored procedure. Two caveats. First, rerun is only safe if the activities downstream of the restart point are idempotent — a rerun that re-appends the same rows creates duplicates. Second, reusing the original parameters is what you want for a date-parameterised load and is exactly *not* what you want if the parameters themselves were the bug; in that case start a fresh run with corrected values. For tumbling-window-driven pipelines there is a second layer: the window itself can be rerun from the trigger runs view, which reissues the whole pipeline with the same window boundaries. ## Interview signals Strong answers state that status is derived from terminal activity states, demonstrate the Fail activity, distinguish skipped from failed, and connect all of it to alerting — the real cost of the behaviour is not the green row but the page that never went out.

  • What happens to activities downstream of a failed activity that had no failure path?
    They are skipped, not failed, and anything depending on them with a Succeeded condition is skipped in turn, cascading to the end of that branch. Skipped is a distinct state in Monitor, so reading a run means checking which activities were skipped as well as which were red.
  • How do you read the upstream error message inside a failure-path activity?
    Reference the failed activity's error: @activity('CopyOrders').Error.message, usually wrapped in concat to build a log payload or a Fail activity message. That gives the operator the actual cause in your own alert rather than sending them into the portal to hunt for it.
  • When is rerun from the failed activity the wrong repair?
    When the downstream activities are not idempotent, since resuming can re-append rows already written, and when the run's parameters were themselves wrong — the rerun reuses the original values. In both cases start a fresh run with corrected parameters instead of resuming.
  • How would you run cleanup regardless of whether the previous activity succeeded?
    Give the cleanup activity a dependency condition of Completed on the predecessor, which fires on either outcome. That is ADF's equivalent of a finally block, and it composes with a separate Failed path that logs and, if needed, ends with a Fail activity.

saying these in an interview costs you the question

  • Assumes any failed activity always fails the whole pipeline run
  • Treats skipped activities as failures in Monitor
  • Adds an error handler and keeps alerting on pipeline failure
  • Thinks rerun from failed re-executes the entire pipeline
  • Believes a failed child pipeline always fails the parent run

context