Your codebase has a large data-driven test suite built on Kotest's `withData`, and generated test names have become a maintenance problem. How would you set a naming and reporting convention for it, and what would you enforce automatically?
answer
- names are identifiers, not documentation
- stable / unique-in-scope / legible
- data class default, nameFn for binary or third-party
- always wrap withData in a named container
- enforce with DuplicateTestNameMode.Error, cap cross-products
basics
~20 sTreat generated names as identifiers: require inputs to be data classes or implement WithDataTestName, use nameFn for third-party or binary payloads, wrap each withData in a named container, and enforce DuplicateTestNameMode.Error in project config so bad names fail the build.
solid answer
~50 sI'd start from what the names are *for*: they are the addresses the IDE re-runs, name-based filters match, and CI history and flaky-test tooling key on. That gives three requirements — stable across runs, unique within scope, short enough to read in a report column. The convention I'd write down: inputs to `withData` are data classes, or implement `WithDataTestName` when the natural label is shorter than `toString()`; third-party types and any input carrying a byte array, blob or long string use the `nameFn` overload; curated scenario lists use the map overload so the name is a sentence; every `withData` sits inside a named container that states the behaviour, so case names can stay short. Enforcement: set `duplicateTestNameMode = DuplicateTestNameMode.Error` project-wide so collisions are a red build, and cap cross-products from nested `withData` — a matrix that multiplies to thousands of leaves is a signal to switch dimensions to property-based testing.
code
kotlin · 12 linesobject ProjectConfig : AbstractProjectConfig() {
override val duplicateTestNameMode = DuplicateTestNameMode.Error
}
class ShippingTest : FunSpec({
context("free shipping above the threshold") {
withData(
nameFn = { "${it.itemCount} items totalling ${it.totalCents}c" },
cases,
) { case -> shipping(case) shouldBe case.expected }
}
})go deeper
Say that data classes give readable names and that each withData should sit inside a container that names the behaviour.
Add the override mechanisms (WithDataTestName, @IsStableType, nameFn, map overload) and when each fits.
Reason from the consumers of test names — re-runs, filtering, CI history — and propose DuplicateTestNameMode.Error as the enforcement point.
Present it as a reporting contract with a written convention, an automatic enforcement point, a migration plan, and an honest limit on cross-product size.
## Frame the problem correctly The instinct is to treat test names as documentation. In a data-driven suite they are better understood as **identifiers in a reporting contract**. Concretely, the generated name of a `withData` case is: - what the IDE sends back when someone clicks re-run on a single case; - what Kotest's name/path-based test filtering matches; - what CI stores per run, so "has this test been failing for a week?" is a name lookup; - what flaky-test detection, quarantine lists and ownership mappings key on. Everything downstream therefore needs the same three properties: **stability** (same input ⇒ same name, every run, on every machine), **uniqueness within scope** (or Kotest disambiguates with positional indexes that move), and **legibility** (a human triaging a red build reads it in a narrow column). ## The mechanics you are choosing between Kotest derives a name from each element: `WithDataTestName.dataTestName()` first, then an `@IsStableType`-annotated class's `toString()`, then a data class's generated `toString()`, otherwise a type-derived fallback. At the call site you can override with the `nameFn` overload or the map-of-names overload. Collisions are governed by `DuplicateTestNameMode` (`Warn` by default in Kotest 5.x, plus `Silent` and `Error`), configured on `AbstractProjectConfig`. A convention is just a set of defaults over those knobs. ## A convention that holds up **1. Inputs are data classes.** Free readable names, and the type is a value object anyway. This alone removes the most common failure — the type-derived fallback producing N identical names. **2. `WithDataTestName` when `toString()` is not the label.** A scenario type with ten fields where only the id identifies the case should say so once, on the type, rather than in every call site. **3. `nameFn` for anything binary, large, or third-party.** Byte arrays print their identity hash inside a data class `toString()`, which reintroduces instability; long JSON bodies produce a name nobody can read. Rule of thumb: if the input has a field you would not want printed in a CI report, the call site names the case explicitly. **4. The map overload for curated scenarios.** When the cases are a hand-written list of business situations, the name should be the sentence a reviewer wants ("cart over free-shipping threshold"), not a dump of the input. Caveat to teach alongside it: duplicate map keys collapse *before* Kotest sees them, so a case can silently vanish — worse than a collision, because nothing warns. **5. Always wrap in a named container.** `context("free shipping threshold") { withData(...) }` gives one heading and N short case names, keeps the report tree self-describing, and widens the space of acceptable names because uniqueness is per scope. **6. Cap the cross-product.** Nested `withData` multiplies. Two dimensions of fifty is 2,500 leaves, each a node with a computed name in the report tree. Beyond a threshold the suite is slow to display, the XML report is enormous, and — more importantly — an enumerated matrix that large is usually the wrong tool; the dimensions that do not genuinely interact belong in separate tests, and the ones with huge domains belong in property-based tests. ## What to enforce mechanically Conventions that rely on review decay. Two enforcement points are cheap: - **`duplicateTestNameMode = DuplicateTestNameMode.Error`** in `AbstractProjectConfig`. This converts the whole class of naming defects — fallback names, a `nameFn` that dropped the distinguishing field, genuinely duplicated cases — into a failing build at the moment it is introduced, when the fix costs a minute. The default `Warn` keeps the build green and is precisely why these defects accumulate. - **A review checklist item, or a lint rule if your tooling supports it**, for `withData` over a non-data class with no `nameFn`. Even as a checklist line it catches most of the remainder. ## Trade-offs to name honestly - `Error` mode will break existing builds on adoption. Land it behind a cleanup pass, not as a surprise. - Type-side naming (`WithDataTestName`) puts a test concern on a production type. Acceptable for test fixtures; questionable for domain models — for those prefer `nameFn` at the call site. - Very descriptive names age: a case named after the expected outcome becomes a lie when the expectation changes. Prefer naming the *input situation*, not the assertion. - Long names bloat report payloads and are truncated in most UIs. Aim for names that survive an 80-character column. ## The judgment being tested There is no single right answer here; the interviewer is checking whether you reason from the *consumers* of test names (IDE, filters, CI history, humans on a red build) back to the mechanics, and whether you can pick an enforcement point rather than relying on everyone remembering. "Data classes plus a named container, `nameFn` for anything ugly, and `DuplicateTestNameMode.Error` so the build enforces it" is a complete answer.
- How do you migrate an existing suite to `DuplicateTestNameMode.Error` without blocking everyone?Flip it locally first and collect the failures — they cluster, usually on a handful of input types that are not data classes. Fix those in one cleanup change (data class, `WithDataTestName`, or a `nameFn`), then land the project-config change in a separate commit so the enforcement lands on an already-clean tree. Anything genuinely unfixable in the short term can be isolated by wrapping it in its own named container, since uniqueness is per scope.
- When would you stop adding rows to a `withData` matrix and reach for property-based testing instead?When the dimensions have large or open domains and you are enumerating a sample of them rather than the cases that matter. An enumerated matrix is right for a closed set of business situations you can name; once the list exists only to cover a range, a property with a generator states the invariant directly and explores far more inputs. The report cost is also a signal: a cross-product of thousands of nodes is a lot of reporting machinery for a claim one property would express.
saying these in an interview costs you the question
- "Test names are just documentation" — they are identifiers used by IDE re-runs, filters and CI history.
- "Longer, more descriptive names are always better" — long names are truncated in reports and age badly when they encode expected outcomes.
- "The default duplicate-name handling is fine because the build stays green" — that is exactly why the defect accumulates unnoticed.
- "Put WithDataTestName on every domain model" — that pushes a test concern into production types; prefer `nameFn` at the call site there.
- "Bigger matrices mean better coverage" — a large enumerated cross-product is usually a signal to switch those dimensions to property-based testing.