skip to content

How do you test that your plugin fails the build correctly, and how do you assert on failure output?

level: middleimportance: should knowfreq 40%

answer

  1. buildAndFail() expects failure
  2. build() throws UnexpectedBuildFailure
  3. buildAndFail() throws UnexpectedBuildSuccess
  4. assert task outcome == FAILED
  5. assert output contains the message

basics

~10 s

Call buildAndFail() instead of build(). It returns a BuildResult only when the build fails, otherwise it throws. Then assert the failing task's outcome is FAILED and that result.output contains your error message.

solid answer

~40 s

TestKit has two terminal methods on `GradleRunner`: `build()` expects success and throws `UnexpectedBuildFailure` if the build fails; `buildAndFail()` expects failure and throws `UnexpectedBuildSuccess` if it unexpectedly passes. To test that your plugin rejects bad configuration or input, configure the test project to trigger the error, call `buildAndFail()`, then assert: the failing task's `outcome` is `TaskOutcome.FAILED`, and `result.output` contains your specific, actionable error message. This is how you lock in good error UX — verifying the user actually sees a helpful message, not just that *something* failed. It is the symmetric counterpart to success-path testing and pairs naturally with input-validation logic in tasks.

code

kotlin · 8 lines
kotlin
val result = GradleRunner.create()
    .withProjectDir(testProjectDir)
    .withArguments("deploy")
    .withPluginClasspath()
    .buildAndFail()

assertEquals(TaskOutcome.FAILED, result.task(":deploy")!!.outcome)
assertTrue(result.output.contains("Missing required 'targetEnv'"))

go deeper

for a junior

Know buildAndFail() is used to test that the build fails.

for a middle

Use FAILED outcome plus output-content assertions; know both Unexpected* exceptions.

for a senior

Design negative tests that lock in error messages as part of the plugin contract.

for a principal

Establish team conventions for negative-path coverage and error-message quality across plugins.

## Two terminal methods `GradleRunner` ends a test in one of two ways: - **`build()`** — asserts the build *succeeds*. If it fails, TestKit throws `UnexpectedBuildFailure` (carrying the `BuildResult` so you can inspect it). - **`buildAndFail()`** — asserts the build *fails*. If it unexpectedly succeeds, TestKit throws `UnexpectedBuildSuccess`. Picking the right one makes the test self-documenting: the method name states the expectation. ## Testing the failure path Good plugins validate inputs and fail with clear messages. To prove that: 1. Set up the test project to violate a precondition (missing required property, invalid value, etc.). 2. Call `buildAndFail()`. 3. Assert the relevant task `outcome == FAILED`. 4. Assert `result.output` contains the human-readable message you promised users. ```kotlin val result = GradleRunner.create() .withProjectDir(dir) .withArguments("validateConfig") .withPluginClasspath() .buildAndFail() assertEquals(TaskOutcome.FAILED, result.task(":validateConfig")!!.outcome) assertTrue(result.output.contains("property 'apiUrl' must be set")) ``` ## Why assert on the message, not just failure A build can fail for many reasons. Asserting only "it failed" lets a misleading or wrong error pass the test. Asserting on the *content* of the message verifies the error UX — that a real user would understand what to fix. This treats error messages as part of the plugin's contract. ## Note on output capture By default failure output (stderr/stack traces) is included in `result.output`. You can also forward the runner's output to the test's own console for debugging with `forwardOutput()` or `forwardStdOutput(...)`.

  • What happens if you call buildAndFail() but the build actually succeeds?
    TestKit throws UnexpectedBuildSuccess, failing your test — which is correct, because the expectation (failure) was not met.
  • Why assert on output content rather than just that the build failed?
    Because a build can fail for the wrong reason. Asserting the specific message verifies your plugin's error UX and turns the error text into a tested contract.

saying these in an interview costs you the question

  • Using build() and try/catching the failure instead of buildAndFail().
  • Asserting only that it failed without checking the message — weak test.
  • Confusing UnexpectedBuildFailure (from build()) with UnexpectedBuildSuccess (from buildAndFail()).

context

resolving links…