Mobile Testing & Device Matrices: What to Test

Q: How do I test process death?

On Android, background the app and kill its process with adb shell am kill, enable "Don't keep activities", or use Compose's state restoration tester; on iOS, test state restoration by terminating the app from Xcode while it is in the background.

Mobile Testing and Device Matrices

Mobile apps run on thousands of device models, several OS versions at once, screen sizes from small phones to foldables and tablets, flaky networks, and users who rotate, background, and abandon the app mid-task. A bug that appears only on one manufacturer's devices, only after the process is killed, or only with large text enabled is still a bug for everyone who hits it — and fixing it takes a new store release.

Effective mobile testing is not about testing every device. It combines fast unit and UI tests in CI, a device matrix chosen from real usage data, targeted tests for the conditions that break mobile apps — lifecycle changes, networks, accessibility, upgrades, and release-build optimizations — and gradual rollouts with monitoring to catch what testing misses.

TL;DR

Build a pyramid: many unit tests, focused UI tests (Espresso, Compose, XCUITest), screenshot tests, and a few end-to-end flows (Maestro, Detox, Appium).
Choose a device matrix from analytics: top OS versions, screen sizes, manufacturers, and low-end devices.
Use emulators and simulators in CI and real devices (in-house or device farms) for hardware, performance, and OEM behavior.
Test rotation, backgrounding, and process death; weak and changing networks; and permission grants and denials.
Test accessibility (screen readers, font scaling, contrast) and localization (pseudo-locales, right-to-left).
Test upgrades from old versions and release builds (minification, signing), and quarantine flaky UI tests.

Quick Example

A Jetpack Compose UI test using a fake repository:

A cross-platform end-to-end flow with Maestro that also checks data survives an app restart:

Simulating process death on Android while the app is in the background:

Core Concepts

The Mobile Test Pyramid

Keep most logic testable without a device by separating view models and repositories from platform UI. See Unit Testing and Visual Regression Testing.

Choosing a Device Matrix

Use production analytics to pick devices and OS versions:

Revisit the matrix every quarter as the user base changes.

Emulators, Simulators, and Real Devices

Emulators and simulators are fast, scriptable, and ideal for CI. Real devices are required for camera, sensors, Bluetooth, push notifications, performance, battery behavior, and manufacturer-specific bugs. Device farms — Firebase Test Lab, AWS Device Farm, BrowserStack, and others — run tests across many real devices without maintaining a lab.

Conditions That Break Mobile Apps

Accessibility

Test with TalkBack and VoiceOver, the largest font sizes, sufficient contrast, and touch target sizes. Automated checks — Android Accessibility Scanner, Espresso accessibility checks, Xcode's Accessibility Inspector and audits — catch common issues; manual screen-reader passes catch the rest. See Accessibility Testing.

Localization

Pseudo-locales (en-XA for long text, ar-XB for right-to-left on Android) reveal truncation and layout bugs before translations arrive.

Flaky UI Tests

UI tests flake because of timing, animations, shared state, and device variability. Reduce flakiness by disabling animations on test devices, waiting on conditions instead of sleeping, resetting app state between tests, using fake backends, and running tests in isolation (for example with Android Test Orchestrator). Track flaky tests, quarantine them quickly, and fix or delete them — a suite people ignore protects nothing.

Best Practices

Drive the Matrix From Data

Choose devices and OS versions from analytics and crash reports, and update the matrix regularly.

Test Lifecycle and Process Death Routinely

Include rotation, backgrounding, and process death in automated tests for important flows.

Test the Build You Ship

Run end-to-end tests against signed, minified release builds before rollout.

Combine Testing With Staged Rollouts

Release gradually and watch crash-free rates by device and OS version to catch what the matrix missed.

Common Mistakes

Testing Only on Flagship Devices

Ignoring Upgrade Paths

Sleeping in UI Tests

FAQ

How many devices should we test on?

Enough to cover your users' main OS versions, screen sizes, manufacturers, and performance tiers — often 10 to 20 configurations for automated runs — chosen from analytics and revised as usage changes.

Are emulators good enough for testing?

For most automated UI and unit tests, yes. Real devices are needed for hardware features, performance, battery and background behavior, and manufacturer-specific issues.

How do I test process death?

On Android, background the app and kill its process with adb shell am kill, enable "Don't keep activities", or use Compose's state restoration tester; on iOS, test state restoration by terminating the app from Xcode while it is in the background.

What is Maestro?

A cross-platform mobile UI testing framework that describes end-to-end flows in YAML and runs them on Android and iOS emulators, simulators, and devices.

How do I reduce flaky mobile UI tests?

Disable animations, wait on conditions instead of fixed delays, isolate tests with fresh state, use fake backends, and track and quarantine flaky tests promptly.