In a Jetpack Macrobenchmark, how do StartupMode.COLD, WARM and HOT change each iteration, and what do StartupTimingMetric's two values measure?
answer
- process alive or not
- killed between setupBlock and measureBlock
- first frame vs fully drawn
- reportFullyDrawn, ReportDrawnWhen
- compare the median
basics
~20 sCOLD kills the process before every measured launch, WARM keeps the process but recreates the activity, HOT brings the existing activity forward. StartupTimingMetric reports timeToInitialDisplayMs to the first frame and timeToFullDisplayMs to the app's fully-drawn signal.
solid answer
~30 s`startupMode` sets the state `measureBlock` starts from. With `StartupMode.COLD` the library kills the app process between `setupBlock` and `measureBlock` every iteration, so process start and `Application` init are measured; `WARM` keeps the process alive and recreates the activity; `HOT` brings the existing activity back to the foreground; `null`, the default, forces nothing. `StartupTimingMetric` reports `timeToInitialDisplayMs`, from the launch intent to the first frame of the destination activity, and `timeToFullDisplayMs`, to the frame that follows `reportFullyDrawn()` or Compose's `ReportDrawnWhen`. It gives min, median and max; compare medians before and after a change.
code
kotlin · 17 lines@Test
fun coldStartup() = benchmarkRule.measureRepeated(
packageName = "com.example.feed",
metrics = listOf(StartupTimingMetric()),
iterations = 10,
startupMode = StartupMode.COLD,
setupBlock = { pressHome() },
) {
startActivityAndWait()
}
// In the app: signal fully drawn once the first feed page is shown
@Composable
fun FeedScreen(state: FeedUiState) {
ReportDrawnWhen { state.items.isNotEmpty() }
FeedList(state.items)
}go deeper
Recall the three modes by what survives: nothing for COLD, the process for WARM, the process and the activity for HOT.
Explain where the library kills the process in COLD, why setupBlock is not traced, and how the two startup values differ in their end point.
Show you can localize a regression with the modes and both values, and that you make full display meaningful by signalling it after real content loads.
Decide which launch the team guards: cold start for first impressions, warm for returning users, and whether full display is defined consistently across screens.
## The scenario An app's cold start regressed after a dependency upgrade. Before anyone changes code, the team needs a number it can trust and repeat: a startup **Macrobenchmark**. Two parameters decide what that number means: the `startupMode` passed to `MacrobenchmarkRule.measureRepeated`, and the `StartupTimingMetric` listed in `metrics`. ## How a measureRepeated iteration runs `measureRepeated` takes a `setupBlock` and a `measureBlock`. Conceptually it compiles the app according to `compilationMode`, then repeats `iterations` times: run `setupBlock` untraced, then run `measureBlock` while a system trace is recorded. Only what happens inside `measureBlock` is measured. `startupMode` (type `StartupMode`, default `null`) tells the library what state the app's process and activity must be in when `measureBlock` begins: - **`StartupMode.COLD`** is a start from scratch: the process is not alive and must be created along with the activity. The library **kills the app process between `setupBlock` and `measureBlock`** in every iteration. That is convenient, but it also means you cannot prepare state in `setupBlock` that must survive into the measured launch; if you need that, use `WARM`, or pass `null` and call `killProcess()` yourself in `setupBlock`. - **`StartupMode.WARM`** creates and displays a new activity in a process that is already running. The library keeps the process alive and recreates the activity, so application-level initialization is not re-measured. - **`StartupMode.HOT`** brings an existing activity to the foreground; both the process and the activity survive from the previous launch. - **`null`** forces nothing. Use it when `measureBlock` is not a launch at all, or when you control the process yourself. For `WARM` and `HOT`, the library may run an extra unmeasured launch before the first measured iteration, simply to create the process or activity that the mode expects to exist. A typical cold-start test passes `setupBlock = { pressHome() }` and calls `startActivityAndWait()` in `measureBlock`. That method launches the app's launcher intent and waits until the launch completes, so the trace contains the whole startup. | `StartupMode` | Process at measure time | Activity at measure time | What it isolates | |---|---|---|---| | `COLD` | killed, started fresh | created | everything: process start, `Application`, first activity | | `WARM` | alive | recreated | activity creation, layout and first frame | | `HOT` | alive | alive, brought to front | resume and redraw | | `null` | whatever `setupBlock` left | whatever `setupBlock` left | nothing forced | ## What StartupTimingMetric reports `StartupTimingMetric` reads the trace of each iteration and reports two values in milliseconds: 1. **`timeToInitialDisplayMs`** runs from the moment the system receives the launch intent to the rendering of the **first frame of the destination activity**. It is always available for a launch. 2. **`timeToFullDisplayMs`** runs from the same launch intent until the app **reports fully drawn**, and stops when the first frame after, or containing, that signal finishes rendering. It only appears when the app sends the signal, and may be unavailable before API 29. The signal is `reportFullyDrawn()` on the activity. In a Compose screen you do not call it by hand; you use `ReportDrawn`, `ReportDrawnWhen { feedLoaded }` or `ReportDrawnAfter { ... }`, which signal once asynchronous content such as the first page of a feed is on screen. Without that signal you can only measure the first frame, which may be an empty skeleton. The metric outputs the **minimum, median and maximum** across iterations. Track the **median**: it is the best estimate of a typical launch, while the extremes mostly show noise such as a background job running at the wrong moment. ## Reading the regression For the dependency-upgrade regression, run the same `COLD` benchmark on the same device model before and after the upgrade, with the same `compilationMode` and iteration count, and compare medians. Then use the modes to narrow it down: - if `COLD` regressed but `WARM` did not, the extra cost sits in process or `Application` start, which is where a library's auto-initialization usually runs; - if `timeToInitialDisplayMs` held steady but `timeToFullDisplayMs` grew, the first frame is fine and the delay is in loading content after it; - each iteration's trace file, written next to the JSON results, shows which work grew. Startup budgets and which work belongs on the critical path are a design discussion of their own; this benchmark is the instrument that says whether a change moved the number.
- Your COLD benchmark needs a logged-in user, but setting it up in setupBlock is lost before the measured launch; why, and what do you do?With `StartupMode.COLD` the library kills the process between `setupBlock` and `measureBlock`, so anything held only in memory disappears. Persist the state to disk in `setupBlock` (it survives the kill), or set `startupMode = null`, prepare the app, then call `killProcess()` yourself at the end of `setupBlock` so the measured launch is still cold.
- Why might timeToFullDisplayMs be missing from the results while timeToInitialDisplayMs is present?`timeToFullDisplayMs` only exists when the app reports fully drawn, via `reportFullyDrawn()` or Compose's `ReportDrawn`, `ReportDrawnWhen` or `ReportDrawnAfter`, and it may be unavailable before API 29. If the screen never sends the signal, only the first-frame value can be computed.
- StartupMode.WARM kills the process but keeps the activity in memory.
- timeToInitialDisplayMs stops when content from the network appears.
- Compare the minimum launch time, since it is the cleanest run.
- setupBlock state always survives into a COLD measured launch.
- Macrobenchmark finds fully drawn on its own without any app signal.