skip to content

How do you write a TestKit test that asserts a build fails as expected, and what does buildAndFail() return?

level: middleimportance: should knowfreq 35%

answer

  1. build() expects success, buildAndFail() expects failure
  2. assert output.contains(error) + task FAILED
  3. test negative/validation paths
  4. --stacktrace to surface cause
  5. stable message = good diagnostics

basics

~20 s

Use buildAndFail() instead of build() when you expect failure. It runs the build, expects a non-zero result, and returns a BuildResult whose output contains the error so you can assert on the failure message and the failing task's FAILED outcome.

solid answer

~40 s

TestKit distinguishes success and failure expectations: **`build()`** throws if the build fails, while **`buildAndFail()`** throws if the build *succeeds*. So when you want to verify that your plugin rejects bad input or that a validation fails, call `buildAndFail()` and assert on the returned `BuildResult` — typically `result.output.contains("<your error message>")` and `result.task(":validate")?.outcome == TaskOutcome.FAILED`. This is how you test the **negative paths**: misconfiguration throwing a `GradleException`/`InvalidUserDataException`, a missing required property, or a failing verification task. Pair it with `--stacktrace` in `withArguments` when you need the cause in output. Always make the error message you assert on stable and meaningful — that doubles as a test of your plugin's diagnostics quality.

code

kotlin · 8 lines
kotlin
val result = GradleRunner.create()
    .withProjectDir(dir)
    .withArguments("validate", "--stacktrace")
    .withPluginClasspath()
    .buildAndFail()

assertEquals(TaskOutcome.FAILED, result.task(":validate")?.outcome)
assertTrue(result.output.contains("version must be set"))

go deeper

for a junior

Know that buildAndFail() is the method for expected-failure tests and returns a BuildResult.

for a middle

Assert both the FAILED task outcome and a stable error message; add --stacktrace when debugging.

for a senior

Treat negative tests as a contract for plugin diagnostics; keep assertions on user-facing messages, not internals.

for a principal

Mandate negative-path coverage for misconfiguration and define standards for plugin error messages that tests enforce.

## Two terminal methods, opposite expectations GradleRunner has two ways to finish a run: - **`build()`** — runs and **expects success**. If the underlying build fails, `build()` raises an `UnexpectedBuildFailure` containing the `BuildResult`. - **`buildAndFail()`** — runs and **expects failure**. If the build *succeeds*, it raises an `UnexpectedBuildSuccess`. This lets each test declare its intent. Negative tests — the ones proving your plugin correctly rejects invalid state — use `buildAndFail()`. ## What to assert on a failure The returned `BuildResult` still gives you `output` and task outcomes. For failures you usually assert: - **`result.output.contains("Expected message")`** — that the user-facing error is present and clear. - **`result.task(":task")?.outcome == TaskOutcome.FAILED`** — that the specific task failed (not some unrelated one). ## Example: rejecting invalid configuration Suppose your plugin validates that a required `apiKey` extension property is set: ```kotlin @Test fun `fails when apiKey missing`(@TempDir dir: File) { dir.resolve("settings.gradle.kts").writeText("") dir.resolve("build.gradle.kts").writeText( """ plugins { id("com.example.api") } // apiKey intentionally not set """.trimIndent() ) val result = GradleRunner.create() .withProjectDir(dir) .withArguments("callApi", "--stacktrace") .withPluginClasspath() .buildAndFail() assertEquals(TaskOutcome.FAILED, result.task(":callApi")?.outcome) assertTrue(result.output.contains("apiKey must be configured")) } ``` ## Tips - Add `--stacktrace` (or `--info`) to `withArguments` to surface the cause when debugging, but keep production assertions on the stable, user-facing message rather than stack frames. - Throw deliberate, descriptive exceptions in the plugin (`throw GradleException("apiKey must be configured")`) so the message you assert on is also good UX. - Don't assert on exact stack-trace text — it's brittle across Gradle versions. - Use `buildAndFail()` only when failure is the *contract*; a flaky/unexpected failure should make `build()` throw loudly so you notice.

  • What happens if you call buildAndFail() but the build actually succeeds?
    TestKit raises an UnexpectedBuildSuccess (a build exception), failing the test — symmetric to build() raising UnexpectedBuildFailure when a build it expected to succeed instead fails.
  • Why assert on the user-facing message rather than the exception class or stack trace?
    The console output is the stable, public contract a user sees; stack frames and internal exception types vary across Gradle versions and refactors, making such assertions brittle. Asserting the message also enforces good diagnostics.

saying these in an interview costs you the question

  • Using build() for a test that expects failure (it will throw on the expected failure).
  • Asserting on exact stack-trace text instead of the user-facing message.
  • Throwing generic exceptions with no message, then having nothing meaningful to assert.

context

resolving links…