How do you write a TestKit test that asserts a build fails as expected, and what does buildAndFail() return?
answer
- build() expects success, buildAndFail() expects failure
- assert output.contains(error) + task FAILED
- test negative/validation paths
- --stacktrace to surface cause
- stable message = good diagnostics
basics
~20 sUse buildAndFail() instead of build() when you expect failure. It runs the build, expects a non-zero result, and returns a BuildResult whose output contains the error so you can assert on the failure message and the failing task's FAILED outcome.
solid answer
~40 sTestKit distinguishes success and failure expectations: **`build()`** throws if the build fails, while **`buildAndFail()`** throws if the build *succeeds*. So when you want to verify that your plugin rejects bad input or that a validation fails, call `buildAndFail()` and assert on the returned `BuildResult` — typically `result.output.contains("<your error message>")` and `result.task(":validate")?.outcome == TaskOutcome.FAILED`. This is how you test the **negative paths**: misconfiguration throwing a `GradleException`/`InvalidUserDataException`, a missing required property, or a failing verification task. Pair it with `--stacktrace` in `withArguments` when you need the cause in output. Always make the error message you assert on stable and meaningful — that doubles as a test of your plugin's diagnostics quality.
code
kotlin · 8 linesval result = GradleRunner.create()
.withProjectDir(dir)
.withArguments("validate", "--stacktrace")
.withPluginClasspath()
.buildAndFail()
assertEquals(TaskOutcome.FAILED, result.task(":validate")?.outcome)
assertTrue(result.output.contains("version must be set"))go deeper
Know that buildAndFail() is the method for expected-failure tests and returns a BuildResult.
Assert both the FAILED task outcome and a stable error message; add --stacktrace when debugging.
Treat negative tests as a contract for plugin diagnostics; keep assertions on user-facing messages, not internals.
Mandate negative-path coverage for misconfiguration and define standards for plugin error messages that tests enforce.
## Two terminal methods, opposite expectations GradleRunner has two ways to finish a run: - **`build()`** — runs and **expects success**. If the underlying build fails, `build()` raises an `UnexpectedBuildFailure` containing the `BuildResult`. - **`buildAndFail()`** — runs and **expects failure**. If the build *succeeds*, it raises an `UnexpectedBuildSuccess`. This lets each test declare its intent. Negative tests — the ones proving your plugin correctly rejects invalid state — use `buildAndFail()`. ## What to assert on a failure The returned `BuildResult` still gives you `output` and task outcomes. For failures you usually assert: - **`result.output.contains("Expected message")`** — that the user-facing error is present and clear. - **`result.task(":task")?.outcome == TaskOutcome.FAILED`** — that the specific task failed (not some unrelated one). ## Example: rejecting invalid configuration Suppose your plugin validates that a required `apiKey` extension property is set: ```kotlin @Test fun `fails when apiKey missing`(@TempDir dir: File) { dir.resolve("settings.gradle.kts").writeText("") dir.resolve("build.gradle.kts").writeText( """ plugins { id("com.example.api") } // apiKey intentionally not set """.trimIndent() ) val result = GradleRunner.create() .withProjectDir(dir) .withArguments("callApi", "--stacktrace") .withPluginClasspath() .buildAndFail() assertEquals(TaskOutcome.FAILED, result.task(":callApi")?.outcome) assertTrue(result.output.contains("apiKey must be configured")) } ``` ## Tips - Add `--stacktrace` (or `--info`) to `withArguments` to surface the cause when debugging, but keep production assertions on the stable, user-facing message rather than stack frames. - Throw deliberate, descriptive exceptions in the plugin (`throw GradleException("apiKey must be configured")`) so the message you assert on is also good UX. - Don't assert on exact stack-trace text — it's brittle across Gradle versions. - Use `buildAndFail()` only when failure is the *contract*; a flaky/unexpected failure should make `build()` throw loudly so you notice.
- What happens if you call buildAndFail() but the build actually succeeds?TestKit raises an UnexpectedBuildSuccess (a build exception), failing the test — symmetric to build() raising UnexpectedBuildFailure when a build it expected to succeed instead fails.
- Why assert on the user-facing message rather than the exception class or stack trace?The console output is the stable, public contract a user sees; stack frames and internal exception types vary across Gradle versions and refactors, making such assertions brittle. Asserting the message also enforces good diagnostics.
- Using build() for a test that expects failure (it will throw on the expected failure).
- Asserting on exact stack-trace text instead of the user-facing message.
- Throwing generic exceptions with no message, then having nothing meaningful to assert.