Kotest's TestCaseOrder has the values Sequential, Random and Lexicographic. What does each do, what is the default, where do you configure it, and why would a team deliberately run tests in random order?
answer
- Sequential = declaration order = default
- Lexicographic = sorted by name
- Random = shuffle siblings at each level
- per spec: override testCaseOrder(); project: AbstractProjectConfig
- random order finds state leaks, not flakiness
basics
~20 sTestCaseOrder controls the order of sibling tests within a spec: Sequential is declaration order and is the default, Lexicographic sorts by name, Random shuffles. Set it per spec by overriding testCaseOrder(), or globally in the project config. Random is used to expose tests that secretly depend on each other's leftover state.
solid answer
~60 s`TestCaseOrder` governs the order in which **sibling tests inside a spec** execute — root tests, and the children of each container. - `Sequential` — declaration order, the default. Predictable, and the order a reader sees in the file. - `Lexicographic` — sorted by test name. Stable across refactors that move code around. - `Random` — shuffled. You set it per spec by overriding `testCaseOrder()` to return a value, or as a default for the whole run through `AbstractProjectConfig`; the spec-level setting wins for that spec. Random exists to break order dependence. A suite that only passes in declaration order has hidden coupling: test B relies on a row test A inserted, or on a mock configured earlier. Sequential hides that until the day someone reorders or deletes a test. Random surfaces it as a failure you can fix. The cost is reproducibility — a random-order failure needs the state leak fixed rather than replayed — which is why teams treat such a failure as a real defect in test isolation, not as noise to retry away.
code
kotlin · 9 linesclass AccountSpec : FunSpec() {
override fun testCaseOrder() = TestCaseOrder.Random
init {
test("creates an account") { }
test("closes an account") { }
}
}go deeper
Name the three values, know Sequential/declaration order is the default, and that the setting applies within a spec.
Add where it is configured at spec and project scope, that it orders siblings at each level of the tree, and why Random exposes order dependence.
Discuss random order as a detector for shared state, how to debug a non-reproducible failure, and how it pairs with isolation settings.
Decide whether the org adopts random ordering, and how to roll it out on new modules first so the legacy failures get triaged rather than ignored.
## What it orders A Kotest spec is a tree: root tests and containers, containers holding further tests. `TestCaseOrder` applies to **siblings at each level** — the root tests among themselves, and each container's children among themselves. It does not reorder across specs (that is a separate setting) and it does not flatten the tree. ## The three values **Sequential** is the default and means declaration order: the order the registration builders were called in the spec body. This is what almost everyone assumes is happening, and it is what makes a spec read top to bottom like a document. **Lexicographic** sorts sibling tests by name. Its appeal is stability: the execution order does not change when someone moves a test up the file, so a report diff stays meaningful. It also makes the order independent of source churn, which some teams prefer for reproducibility. **Random** shuffles siblings. This is the interesting one. ## Where you configure it Per spec, override `testCaseOrder()` in the spec class to return the value you want. For the whole run, set the equivalent property in your `AbstractProjectConfig` implementation, which supplies the default that specs without an override inherit. The usual pattern is a project-wide default with a handful of specs overriding — though a spec that *needs* an override is usually a spec with an isolation problem. ## Why random order earns its keep Order dependence is one of the classic silent defects in a test suite. Test A inserts a row, updates a singleton, sets a system property, or trains a shared mock; test B passes because that state is still there. Under `Sequential` the suite is green and stays green — until someone reorders the file, deletes test A, or runs test B alone from an IDE, at which point B fails for reasons that appear unrelated to the change that exposed it. `Random` converts that latent defect into an intermittent failure that shows up early, while the person who introduced the coupling is still nearby. That is the trade: you accept some non-reproducible failures in exchange for finding real coupling. The honest cost is debugging. A random-order failure is not replayed by rerunning — the order differs again. The right response is almost never "rerun until green"; it is to find the shared state. Common culprits are objects and companion objects holding mutable state, a database not rolled back between tests, static mocks, and caches. Kotest's isolation settings address the same problem from the other end by controlling how spec instances are created, and the two are usually reasoned about together: strong isolation plus random order gives high confidence that tests are genuinely independent. ## Interview framing A good answer names the three values and the default, says where the knob lives at both spec and project scope, and then makes the judgment explicit: random ordering is not a style preference, it is a detector for state leaks, and its output should be treated as a bug report against test isolation rather than as flakiness. Teams that adopt it usually do so on a new module first, fix what falls out, then widen it — turning it on across a legacy suite in one go produces a wall of failures nobody triages.
- A spec passes under Sequential and fails intermittently under Random. How do you investigate?Treat it as a state-leak bug rather than flakiness. Look for mutable state shared across tests: properties on the spec instance, objects or companion objects, unrolled-back database writes, static or global mocks, caches. Reproducing by rerunning is unreliable, so bisect by state instead of by order — move the suspect state into per-test setup and see whether the failure disappears. Kotest's isolation settings are the companion lever when the shared state is the spec instance itself.
saying these in an interview costs you the question
- Believing TestCaseOrder also controls the order specs run in
- Treating a random-order failure as flakiness to be retried rather than as real coupling
- Claiming Sequential means alphabetical rather than declaration order
- Thinking the setting only exists at project level with no per-spec override