skip to content

How do you run Flutter integration_test tests on Firebase Test Lab for Android, and what must the uploaded builds contain?

level: seniorimportance: should knowfreq 30%

answer

  1. two APKs: app and androidTest
  2. FlutterTestRunner on an androidTest class
  3. assembleAndroidTest
  4. assembleDebug -Ptarget=integration_test/...
  5. a release build omits integration_test

basics

~20 s

Add an androidTest class run by FlutterTestRunner, build a debug app APK with -Ptarget pointing at the integration test plus a test APK with assembleAndroidTest, then upload both to Test Lab as an Instrumentation test.

solid answer

~40 s

Test Lab runs Android **instrumentation** tests, so the Flutter tests need a native entry point: a class under `android/app/src/androidTest/...` annotated `@RunWith(FlutterTestRunner::class)` with an `ActivityTestRule` for `MainActivity`. The integration binding then reports each `testWidgets` result to that runner. From `android/` I run `flutter build apk --debug` once to generate the Gradle files, `./gradlew app:assembleAndroidTest` for the test APK, and `./gradlew app:assembleDebug -Ptarget=integration_test/checkout_test.dart` so the app APK's Dart entry point is the test rather than `lib/main.dart`. Both APKs go to Test Lab as an Instrumentation test, from the console or with `gcloud firebase test android run --type instrumentation --app ... --test ...`, choosing devices there. The Flutter docs note a standard `--release` build does not include `package:integration_test`, so the app APK is a debug build.

code

bash · 10 lines
bash
flutter build apk --debug
cd android
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=integration_test/checkout_test.dart
cd ..

gcloud firebase test android run --type instrumentation \
  --app build/app/outputs/apk/debug/app-debug.apk \
  --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
  --timeout 2m

go deeper

for a junior

Recall that Test Lab needs two APKs - the app built with the integration test as its target, and an androidTest APK - uploaded as an Instrumentation test.

for a middle

Explain the bridge: a FlutterTestRunner androidTest class, the integration binding reporting over a channel, and the Gradle tasks with -Ptarget that build each APK.

for a senior

Automate the build and gcloud run in CI with debug APKs, dart-defines for environments, per-test timeouts and reset device state, and diagnose zero-test or wrong-entry-point runs.

for a principal

Weigh a cloud device matrix against a single CI emulator, based on the device fragmentation that actually causes the app's production issues.

## Why Test Lab needs extra wiring **Firebase Test Lab** runs tests on a range of Android devices in the cloud. For Android it runs **instrumentation tests**: an app APK plus a test APK whose runner the system starts. A Flutter integration test is Dart code inside the app, so it needs a bridge: - a native **androidTest** class whose runner is `FlutterTestRunner` from the `integration_test` plugin, - an app APK whose Dart entry point is the **integration test file**, not `lib/main.dart`, - the **`IntegrationTestWidgetsFlutterBinding`** in that test, which reports each `testWidgets` result over a channel so `FlutterTestRunner` can surface it as an instrumentation result. ## Step 1: the androidTest class Create a file under `android/app/src/androidTest/kotlin/<your/package/path>/`, for example `MainActivityTest.kt`: ```kotlin @RunWith(FlutterTestRunner::class) class MainActivityTest { @Rule @JvmField val rule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java, initialTouchMode = true, launchActivity = false) } ``` Imports come from `androidx.test.rule.ActivityTestRule`, `dev.flutter.plugins.integration_test.FlutterTestRunner`, `org.junit.Rule` and `org.junit.runner.RunWith`. The Android Gradle configuration also needs the AndroidX test runner and its dependencies, as described in the `integration_test` README. ## Step 2: build the two APKs From the project, run the documented sequence: 1. `flutter build apk --debug` - generates the Flutter pieces the Gradle build needs. 2. `cd android` 3. `./gradlew app:assembleAndroidTest` - builds the **test APK** (`build/app/outputs/apk/androidTest/debug/*.apk`). 4. `./gradlew app:assembleDebug -Ptarget=integration_test/checkout_test.dart` - builds the **app APK** with the test as its Dart entry point (`build/app/outputs/apk/debug/*.apk`). Dart defines can be passed to Gradle as base64-encoded `-Pdart-defines=...` values when the test needs configuration such as a staging API URL. ## Step 3: upload and run - **Console**: Test Lab, "Run a test", choose **Instrumentation**, put the app APK in the app box and the test APK in the test box, pick devices. - **Command line** (what CI uses): `gcloud firebase test android run --type instrumentation --app <app.apk> --test <androidTest.apk> --timeout 2m --results-bucket=... --results-dir=...`, adding device options as needed. Test Lab installs both, starts the instrumentation, and `FlutterTestRunner` reports one result per `testWidgets`. Failures come back with the runner's per-test results and device logs. ## Common mistakes | Symptom | Cause | |---|---| | Test Lab reports zero tests | the androidTest class is missing or not run by `FlutterTestRunner` | | The app opens its normal home screen and nothing is tested | the app APK was built without `-Ptarget=integration_test/...` | | Build or runtime errors about `integration_test` | a release build was uploaded; the Flutter docs note a standard `--release` build does not include `package:integration_test` | | Checkout test hangs on some devices | a native dialog or slow network; pre-grant permissions, fake services, set `testWidgets` timeouts | ## How it relates to running locally Locally and on a CI emulator, `flutter test integration_test/checkout_test.dart -d <device>` runs the same Dart file without any Gradle steps; the tool reports results itself and native reporting is switched off. Test Lab is the step up when you need a **device matrix** - several OS versions and manufacturers - rather than one emulator. For iOS, Test Lab takes a zipped XCTest bundle built with `flutter build ios` and `xcodebuild build-for-testing`, following the same idea of a native runner around the Dart test.

  • Why must the app APK be built with -Ptarget=integration_test/checkout_test.dart?
    The instrumentation runner only starts the activity; which Dart `main` runs is fixed at build time. Without `-Ptarget` the APK runs `lib/main.dart`, the normal app, and no `testWidgets` ever registers, so the runner has no results to report.
  • How do results from testWidgets reach Test Lab's report?
    `IntegrationTestWidgetsFlutterBinding` collects each test's outcome and sends them over a platform channel when the tests finish. `FlutterTestRunner` on the Android side receives them and reports each as an instrumentation test result, which Test Lab displays per test.

saying these in an interview costs you the question

  • Upload only the app APK; Test Lab discovers Dart tests itself
  • Use a --release build so the tests run at full speed
  • Test Lab can run Flutter integration tests from the app APK alone
  • The androidTest class needs one JUnit method per Dart test
  • The app APK can be the normal build without a -Ptarget override