skip to content

Your CI splits a Cypress suite across four jobs with hand-written `--spec` globs. What does that cost?

level: seniorimportance: should knowfreq 48%

answer

  1. Written once, the suite keeps changing
  2. Which job runs a brand-new spec
  3. Green does not mean everything ran
  4. --spec does not replace specPattern
  5. Four jobs, four exit codes

basics

~10 s

It costs maintenance and silent coverage loss. The partition never rebalances as spec durations drift, and a new spec matched by no job's glob is simply never run while every job still exits green.

solid answer

~40 s

Hand-splitting is the free alternative to recorded parallelisation, and its costs are deferred rather than absent. The partition is static while spec durations are not, so one job becomes the long pole and nothing rebalances. Worse, a spec matched by no job's glob runs nowhere and every job still exits zero — coverage disappears into a green build. Cypress also computes the **intersection** of `--spec` and the configured `specPattern`, so a spec outside `specPattern` is not found even by exact path, which makes moving a package's specs fail confusingly. And four jobs mean four exit codes and four artefact sets that someone must assemble. The mitigation is to derive the split at job time from the filesystem rather than hard-coding globs, and to make an unrun spec fail loudly.

code

bash · 24 lines
bash
#!/usr/bin/env bash
set -euo pipefail

# shard.sh <index> <total> - derive this job's specs from the filesystem
INDEX="$1"
TOTAL="$2"

mapfile -t ALL < <(find packages -path '*/cypress/e2e/*' -name '*.cy.js' | sort)

if [ "${#ALL[@]}" -eq 0 ]; then
  echo "no storefront specs discovered - refusing to report green" >&2
  exit 1
fi

MINE=()
for i in "${!ALL[@]}"; do
  if [ $(( i % TOTAL )) -eq $(( INDEX - 1 )) ]; then
    MINE+=("${ALL[$i]}")
  fi
done

printf 'shard %s/%s running %s specs\n' "$INDEX" "$TOTAL" "${#MINE[@]}"
IFS=,
npx cypress run --spec "${MINE[*]}"

go deeper

for a junior

Know that cypress run --spec accepts a glob or comma-separated list, and that a spec must also match specPattern to be found at all.

for a middle

Explain why a static partition drifts out of balance, and what the intersection of --spec and specPattern means when a package's specs move to a new folder.

for a senior

Name silent coverage loss as the failure that matters, and describe how you would engineer against it — a derived split, a discovered-count assertion, or a scheduled unsplit run.

for a principal

Decide whether the maintenance burden of a hand-split is cheaper than a hosted coordinator for this team, and set the standard so every repository in the monorepo splits the same way.

## What the hand-split actually is The free way to fan a Cypress suite out across machines is to stop asking for one run and start running several. Each CI job invokes `cypress run` with its own `--spec` value, no record key is involved, and the CI provider's own parallelism gives you the machines. In a multi-package storefront monorepo it usually appears as one glob per package: - job 1 — `cypress run --spec "packages/checkout/cypress/e2e/**/*.cy.js"` - job 2 — `cypress run --spec "packages/catalog/cypress/e2e/**/*.cy.js"` - job 3 — `cypress run --spec "packages/cart/cypress/e2e/**/*.cy.js"` - job 4 — `cypress run --spec "packages/account/cypress/e2e/**/*.cy.js"` Nothing here is wrong, and on day one it works. Wall-clock time drops, the runner phones nobody, nothing is metered. The costs are all deferred, and they are all maintenance costs. ## The four ways it decays **1. The partition is static and the durations are not.** Balancing that is written once assumes the shape of the suite on the day it was written. Checkout grows, catalog shrinks, and one job becomes the long pole while three sit idle. Nothing rebalances, because nothing is measuring; you find out from the pipeline's wall-clock, if anyone is watching it. **2. A new spec can belong to no job at all.** This is the dangerous one, because it is silent. Add `packages/wishlist/cypress/e2e/gift-list.cy.js` and no glob matches it. No job errors — each one runs the specs it was pointed at and exits zero — so the pipeline is green, the coverage is gone, and nobody is told. Hand-split globs turn "we forgot to update CI" into a **passing build**. **3. `--spec` is intersected with `specPattern`, not substituted for it.** Cypress computes the intersection of the `--spec` value and the configured `specPattern`, so a spec outside `specPattern` is not found even when you pass its exact path. Moving a package's specs to a new directory, or adding a package whose layout differs, fails in the most confusing way available: the glob looks right, the file exists, and the job finds nothing. **4. There is no single view of the run.** Four jobs produce four independent results, four exit codes and four sets of artefacts. Answering "did the suite pass, and which tests failed?" becomes work someone has to do — in the CI UI, or by aggregating reporter output yourself. ## What to say you would do instead The fix is not necessarily to buy orchestration. It is to stop hand-writing the partition: 1. **Derive the split at job time.** Have the job enumerate spec files from the filesystem, sort them deterministically, take every Nth file for shard N, and pass that concrete list to `--spec` as a comma-separated value. Now a new spec is picked up by exactly one job the moment it is committed, with no workflow edit. 2. **Make the orphan case loud.** Assert the number of discovered spec files against a committed count, or run one unsplit job on a schedule, so a spec nobody runs is a failure rather than a silence. 3. **Keep the split out of the workflow file.** A shell or Node script in the repository can be tested and reviewed; four literal globs pasted into YAML across two repositories cannot. 4. **Rebalance on evidence.** If you must keep directory globs, record each job's duration over time and move packages between jobs when the spread grows, rather than when someone complains. ## The trade, stated plainly | | Hand-split `--spec` globs | Recorded `--parallel` | |---|---|---| | Cost | CI compute only | CI compute plus a metered hosted service | | Balancing | Static, by whoever wrote the globs | By measured spec duration | | New spec appears | May be run by no job, silently | Included automatically | | Result view | One per job, assembled by you | One run | | Failure mode | Green build, missing coverage | Network or key failure stops the run | The senior answer is not "hand-splitting is bad". It is that hand-splitting moves the cost from a line item to your team's attention, and that the specific thing you must engineer against is **silent coverage loss** — because that is the only failure on the list that does not announce itself.

  • Why does `cypress run --spec packages/wishlist/cypress/e2e/gift-list.cy.js` find nothing?
    Because Cypress intersects `--spec` with the configured `specPattern` rather than replacing it. If `specPattern` still points only at `packages/checkout` and `packages/catalog`, the wishlist file is outside the configured set and will not be found even though you named its exact path. Extend `specPattern` in `cypress.config.js` to cover the new package's location.
  • How would you detect that a Cypress spec is being run by none of your CI jobs?
    Make the discovered set the source of truth rather than the globs. Derive each shard from a filesystem listing so every spec lands in exactly one job, and add a check that the total discovered count matches what the shards collectively ran. A scheduled unsplit run of the whole suite is a cheap backstop that catches orphans within a day.

saying these in an interview costs you the question

  • Assumes an unmatched spec makes some job fail loudly
  • Thinks --spec overrides specPattern rather than intersecting it
  • Balances the split by package count instead of duration
  • Treats four green jobs as one passing suite result
  • Hard-codes globs in two workflow files and never revisits them