In Jetpack Macrobenchmark, how do CompilationMode.None, Partial and Full differ, and which mode does measureRepeated use when you pass none?
answer
- how much is compiled before measuring
- worst case vs fresh install vs unrealistic
- BaselineProfileMode Require, UseIfAvailable, Disable
- DEFAULT is Partial, UseIfAvailable
basics
~20 sNone resets compilation so code runs JIT-only, the worst case; Partial pre-compiles with the Baseline Profile or warmup runs, like a store install; Full compiles everything ahead of time. The default is Partial with BaselineProfileMode.UseIfAvailable on API 24+.
solid answer
~40 s`CompilationMode` sets how much of the app is compiled before measuring. `None()` resets the profile and leaves everything to the JIT, the worst case without a Baseline Profile. `Partial()` installs the app's Baseline Profile through ProfileInstaller and compiles with `speed-profile`; its `baselineProfileMode` defaults to `Require`, which throws if the profile is missing, and `warmupIterations` can instead build a profile from extra runs. `Full()` compiles every method: stable but unrealistic, and often slower at startup because of code size. With no argument, `measureRepeated` uses `CompilationMode.DEFAULT`, which is `Partial(BaselineProfileMode.UseIfAvailable)` on API 24+ and `Full` below. Compare `None()` against `Partial(Require)` to prove a profile helps.
code
kotlin · 25 lines@Test
fun startupNoCompilation() = startup(CompilationMode.None())
@Test
fun startupBaselineProfile() =
startup(CompilationMode.Partial(baselineProfileMode = BaselineProfileMode.Require))
@Test
fun startupAfterUse() = startup(
CompilationMode.Partial(
baselineProfileMode = BaselineProfileMode.Disable,
warmupIterations = 3,
)
)
private fun startup(mode: CompilationMode) = benchmarkRule.measureRepeated(
packageName = "com.example.feed",
metrics = listOf(StartupTimingMetric()),
compilationMode = mode,
iterations = 10,
startupMode = StartupMode.COLD,
setupBlock = { pressHome() },
) {
startActivityAndWait()
}go deeper
Recall the three modes as worst case, fresh install with a profile, and everything compiled, and that the library resets compilation before measuring.
Explain the default, what BaselineProfileMode.Require does when the profile is missing, and why Full can be slower than Partial at startup.
Show you keep the mode fixed when measuring code changes, use None against Partial(Require) to price a profile, and know the API 34 reset behaviour.
Decide which compilation state the team's numbers represent and enforce it, so dashboards never mix fresh-install results with fully compiled ones.
## Why compilation state matters to a benchmark Android apps ship as DEX bytecode. On a user's device the Android Runtime runs that code through a mix of an interpreter, a just-in-time (JIT) compiler and ahead-of-time (AOT) compilation guided by **profiles**: lists of classes and methods worth compiling in advance. How much of your app is compiled before a launch changes startup and frame times a great deal. A benchmark that ignores this measures some accidental state, so `MacrobenchmarkRule.measureRepeated` takes a `compilationMode` and **resets and recompiles the app before measuring**. ## The modes `CompilationMode` is a sealed class with these public options: - **`CompilationMode.None()`** resets the compilation profile and does no pre-compilation: everything runs interpreted and JIT-compiled as it goes. It shows the worst case, how the app performs with no Baseline Profile. Since Benchmark 1.4.0 the library also clears ART's runtime image on API 34+ each time it kills the app, because on those versions the runtime otherwise partly compiles the app after its first launch and `None` would stop being the worst case. - **`CompilationMode.Partial(...)`** compiles part of the app ahead of time, the way an installer such as Google Play does. Two knobs: - `baselineProfileMode`: `Require` (the constructor's default), `UseIfAvailable` or `Disable`. With `Require` or `UseIfAvailable` the library asks the app's ProfileInstaller to install its Baseline Profile and then compiles with ART's `speed-profile` filter. With `Require`, a missing profile or a missing ProfileInstaller **throws**; with `UseIfAvailable` it is only logged. - `warmupIterations`: run the measured journey that many extra times first, save the runtime profile it produces, then compile with it. This mimics an app a user has run for a day or more, after the device's idle-time optimization. At least one must be active: `Partial(baselineProfileMode = BaselineProfileMode.Disable)` with zero warmups is rejected. - **`CompilationMode.Full()`** compiles every method ahead of time (`cmd package compile -f -m speed`). No real device does this for apps on API 24+, so it is unrealistic, but it removes JIT noise. It is often **slower** at startup than `Partial`, because the larger compiled code costs more to load from disk and puts more pressure on the instruction cache. - **`CompilationMode.Ignore()`** (experimental) leaves the compilation state alone, for when you prepared it yourself. ## The default If you pass nothing, `measureRepeated` uses **`CompilationMode.DEFAULT`**: on API 24+ that is `Partial(baselineProfileMode = BaselineProfileMode.UseIfAvailable, warmupIterations = 0)`, a fresh store install that uses the Baseline Profile *if one is there*. On API 23 and lower, apps are always fully compiled, so the default is `Full`. The library's own documentation recommends passing a mode explicitly, such as `Partial(BaselineProfileMode.Require)`, to avoid ambiguity and to fail loudly when the profile is not actually being used. | Mode | What runs before measuring | Represents | |---|---|---| | `None()` | profile reset, no AOT | worst case, no Baseline Profile | | `Partial()` | Baseline Profile installed, `speed-profile` | fresh install with your profile | | `Partial(Disable, warmupIterations = 3)` | profile from warmup runs | an app used for a while | | `Full()` | everything compiled, `speed` | unrealistic, low JIT noise | | `DEFAULT` | `Partial(UseIfAvailable)` on API 24+ | install, profile if present | ## Using the modes to prove a change The standard comparison runs the same journey twice: 1. `CompilationMode.None()`: the baseline without any profile. 2. `CompilationMode.Partial(BaselineProfileMode.Require)`: the same journey with your Baseline Profile. The gap between their medians is what the profile buys a fresh install. For a code change rather than a profile change, keep one mode fixed for both the before and after runs; comparing a `Partial` run against a `None` run measures compilation, not your code. ## Practical notes - On Android versions before 14, fully resetting compilation state requires reinstalling the APK between modes, which is slower; on API 34 and later the state can be reset in place. - Partial compilation with a profile needs the target app to include ProfileInstaller and a Baseline Profile packaged by AGP. - `Full` is tempting for its stable numbers, but a result that only holds when everything is pre-compiled may not hold for users. How ART decides what to compile in the background, and its compiler filters in depth, are outside this library; what matters here is choosing the mode that matches the claim you want to prove.
- Why pass Partial(BaselineProfileMode.Require) explicitly instead of relying on CompilationMode.DEFAULT?`DEFAULT` uses `UseIfAvailable`, which only logs when the profile or ProfileInstaller is missing, so a broken profile setup still produces numbers that look fine. `Require` throws in that case, so the benchmark proves the profile is really being installed and compiled.
- What does warmupIterations add to CompilationMode.Partial, and when is it useful?The library runs the measured journey that many extra times, saves the runtime profile those runs produce, then compiles with `speed-profile`. That approximates an app after some real use and background optimization. It is useful for steady-state numbers, but it measures a profile built from your test journey, not the Baseline Profile you ship.
- CompilationMode.Full is the most realistic picture of user devices.
- Partial() silently falls back to JIT when no profile exists.
- Compare a None run before against a Partial run after a code change.
- Full compilation is always faster at startup than Partial.
- The default mode leaves the app's compilation state untouched.
Assumes Benchmark 1.5; since 1.4.0 CompilationMode.None clears ART's runtime image on API 34+ so it keeps measuring the worst case.