skip to content

A Detox suite for a hotel-booking app passes locally but fails on the CI runner — how do you diagnose it?

level: seniorimportance: must knowfreq 45%

answer

  1. evidence before retries
  2. record-logs, screenshots, videos: failing
  3. trailing slash on artifacts location
  4. slower, headless, parallel, fresh runner
  5. reproduce with CI's configuration

basics

~20 s

Turn on Detox artifacts for failing tests (logs, screenshots, videos) and trace logging, upload them from CI, then compare the runner's conditions — speed, headless device, workers, data, binary — and reproduce locally with the same configuration before touching retries.

solid answer

~40 s

I start with evidence: `--record-logs failing`, `--take-screenshots failing`, `--record-videos failing` and `--loglevel trace`, with the artifacts folder uploaded even on failure. Then I compare the runner with my laptop: a slower headless machine hits setup or step timeouts, parallel workers can book the same room with the same account, a fresh or reused runner has different state, CI usually tests the release build while I ran debug, and the backend may be slower from CI's network. I reproduce with the same configuration, worker count and `--headless`, looping the failing file. Then I fix the cause — timeouts, test data or a real race — and only use `--retries` as a tracked stopgap.

code

bash · 8 lines
bash
detox test -c ios.sim.release \
  --headless \
  --maxWorkers 2 \
  --record-logs failing \
  --take-screenshots failing \
  --record-videos failing \
  --artifacts-location artifacts/ \
  --loglevel trace

go deeper

for a junior

Recall the three artifact flags and that failing keeps artifacts only for failed tests.

for a middle

Explain which laptop-versus-runner differences cause CI-only failures and how artifacts show each one.

for a senior

Run the diagnosis end to end: collect evidence, reproduce with CI's configuration, workers and headless mode, then fix the cause.

for a principal

Decide what the pipeline stores by default, how long flaky tests may stay retried, and who owns fixing them.

## The situation The hotel-booking app's **Detox** suite is green on a developer's laptop and red on the CI runner: the "book a room" test times out on the payment screen, and a different test fails the next night. Interviewers ask this to see whether a candidate **collects evidence first** and knows the differences between a laptop and a runner, rather than adding sleeps and retries. ## Step 1: make the runner tell you what happened Detox keeps no automatic artifacts by default, so the first change is to the CI command: ```bash detox test -c ios.sim.release \ --record-logs failing \ --take-screenshots failing \ --record-videos failing \ --artifacts-location artifacts/ \ --loglevel trace ``` - **`--record-logs failing`** keeps device and app logs for failed tests only — crashes, red-box errors and native warnings live here. - **`--take-screenshots failing`** keeps the before-and-after screenshots of failed tests. - **`--record-videos failing`** keeps a screen recording, which usually shows the stuck spinner or the unexpected dialog at once. - **`--artifacts-location artifacts/`** — the trailing slash puts files straight into that folder; without it Detox appends a sub-folder named after the configuration and a timestamp. - **`--loglevel trace`** prints every command Detox runs and its traffic with the app. Upload the artifacts folder from the CI job even when it fails, or all of this is lost with the runner. If you cannot edit the CI script, the `DETOX_ARGV_OVERRIDE` environment variable appends extra arguments to the `detox test` invocation for an ad-hoc run. ## Step 2: check the usual laptop-versus-runner differences | Difference | How it shows up | What to check | |---|---|---| | Slower CPU, no GPU | timeouts in setup or long steps | `testRunner.jest.setupTimeout`, Jest `testTimeout`, emulator `gpuMode` | | Headless device | layout or rendering differences | run the same headless flags locally | | Fresh machine | first-boot dialogs, missing simulator runtime | video and screenshots of the first test | | Parallel workers | two tests book the same room or share an account | worker count and test data per file | | Real backend or network | slow or rate-limited API from the runner's network | logs for request failures; test environment stubs | | Different binary | CI builds release, laptop runs debug | build the same configuration locally | | Leftover state | a reused runner keeps simulators and data | reinstall per file, `--cleanup` at the end | The Detox flakiness guide frames the same idea: a test that always passes on your machine and fails on a slower one, such as CI, is the classic flaky test, and more data is the first step. ## Step 3: reproduce locally with the runner's conditions 1. Build and run the **same configuration** CI uses — usually the release one. 2. Match the **worker count** (`--maxWorkers 2`) and **headless** mode (`--headless`). 3. Loop the failing file several times; a failure in one run out of ten is still a failure. 4. If the video shows the app waiting on something that never settles, the next stop is Detox's synchronization diagnostics — the app-busy report that `--debug-synchronization` prints after a step takes too long. ## Step 4: fix the cause, then decide about retries - A **timeout on a slow runner** is fixed by the right timeout for the environment or a faster emulator setup (quick-boot snapshots, hardware acceleration), not by `waitFor` everywhere. - **Shared data between parallel files** is fixed by giving each file its own account, hotel and dates. - **A real app bug** that only shows under slow conditions — a race between the price fetch and the "Book" tap — is a product fix. `--retries` can keep the pipeline moving while this happens, but it should be a tracked exception: a retried pass is still a flaky test. ## A known CI trap with videos On iOS, video recording on a runner without hardware graphics acceleration fails with an error that recording requires Metal. The Detox docs' answer is to enable acceleration on the machine or disable video recording on that CI job — so video may be the one artifact you cannot have everywhere.

  • Why does the trailing slash on --artifacts-location matter on CI?
    Without it, Detox appends a sub-folder named after the configuration and a timestamp, which is handy locally to avoid overwriting runs. On CI you usually want a fixed path the upload step knows, so `artifacts/` with the slash puts files directly in that folder.
  • How do you add debug flags to a CI run you cannot easily edit?
    Set `DETOX_ARGV_OVERRIDE` in the job's environment, for example `--loglevel trace` plus a test-name pattern and one worker. Detox appends it to the `detox test` arguments. The docs present it as an escape hatch for ad-hoc troubleshooting, not a replacement for the config file.

saying these in an interview costs you the question

  • Add sleeps until the CI run turns green.
  • Detox keeps screenshots and logs of failures by default.
  • A test that passes locally cannot have a real app bug.
  • Turning on retries is the diagnosis.
  • CI and the laptop run the same binary, so builds are irrelevant.