AI exploratory testing for Android apps: charters, a worked session, and the report to ask for.
The charter template is Elisabeth Hendrickson's, from Explore It!; timeboxed sessions with a debrief come from session-based test management. Phonebox facts are from its docs, checked November 16, 2026.
Why exploration suits an agent
A scripted test checks what someone already thought of. Exploration looks for what nobody did, which is the part of testing a script can't do and a model can: decide the next step from the screen in front of it.
| Scripted UI test | Human exploratory session | Agent exploratory session | |
|---|---|---|---|
| Decides each step | In advance, in code | From what the tester sees | From the screen elements it reads |
| Finds | Regressions in known journeys | New bugs, confusing flows, visual problems | New bugs, broken edge cases, text that doesn't fit |
| Misses | Everything not written down | What the tester didn't think to try | Subtle visual and timing problems, anything needing real hardware |
| Repeatable | Exactly | Through notes | Through the report and a recording |
| Runs | On every commit | A few times a release | On every branch, while you review |
Write a charter, not a script
A charter names a target, the resources to use on it and the information you want: Explore <target> with <resources> to discover <information>. It keeps the agent on one area without telling it every tap. These four cover the bugs agents find most reliably on Android:
Explore checkout with a declined test card and the Back button
to discover where an interrupted payment leaves the order.
Explore sign-up with empty, 500-character and emoji input
to discover which fields lose what was typed, or crash the app.
Explore the app in German (de-DE)
to discover cut-off labels, untranslated text and wrong date formats.
Explore the app after closing it in the middle of a task
to discover what state survives a restart.Phonebox can change what the phone reports, which makes some charters possible that are awkward on an emulator: set the locale to de-DE or ar-EG for long words and right-to-left layouts, or the location for anything regional. Locale, location and timezone changes need a plan.
Heuristics to put in the task
Name the moves you want tried. Each one maps to an action the agent can take on the phone:
| Heuristic | What it catches | Action |
|---|---|---|
| Back in the middle of a flow | Lost input, duplicate submits, wrong back stack | back |
| Leave and come back | State that doesn't survive the app going to the background | home, then open_app |
| Close the app mid-task | State that doesn't survive a restart | close_app, then open_app |
| First run again | Onboarding that only works once | close_app with clear_data |
| Empty, very long and emoji input | Validation gaps and crashes | type with clear |
| Tap twice, fast | Double orders and double sends | double_tap |
| Arrive from outside | Deep links that skip a required step | open_url |
| Check the notification shade | Notifications with wrong or missing text | notifications |
Run a timeboxed session
Set the timebox on the phone itself. --max parks the phone when time is up, so a session that wanders off can't keep billing:
export PHONEBOX_PHONE=ph_7kx2m6q4v3ta
phonebox start --max 20m # the timebox: the phone parks itself after 20 minutes
phonebox apps install app/build/outputs/apk/debug/app-debug.apk
phonebox record start --name "explore checkout" # optional: a video of the sessionThen give the agent the charter and the rules: read the screen before every action, note each finding with its steps as it goes, and park the phone at the end. Here is part of a session on the first charter. The agent reads the payment screen, fills it with a test card that's always declined, and checks where Back leads:
observe
snapshot snp_q3k7z2m4x6ta
app com.example.shop (.CheckoutActivity)
keyboard hidden
screen 1080x2400
[1] TextView "Payment" (540,180)
[2] EditText "Card number" #card_number clickable editable (540,520)
[3] EditText "MM/YY" #expiry clickable editable (300,700)
[4] EditText "CVC" #cvc clickable editable (780,700)
[5] Button "Pay $36.00" #pay clickable (540,2200)
act tap #card_number, type "4000 0000 0000 0002", tap #expiry, type "12/30",
tap #cvc, type "123", tap "Pay $36.00", wait_for "declined"
→ 8 of 8 ok; "Your card was declined" appeared after 2.4 s
act back
observe
app com.example.shop (.BasketActivity)
[1] TextView "Your basket" (540,180)
[4] TextView "Order #1043 · Pending" #order_status (540,640)The last line is the finding: the order exists although the payment failed. The agent saw it because it read the screen after Back, not because a test asserted it. If the app crashes, the next observation shows app com.android.launcher3 or a "keeps stopping" dialog; Phonebox phones don't expose ADB, so there's no logcat, and the recording is the evidence.
The report to ask for
Ask for findings ranked by severity, each with steps and the screen text, then what the session covered and what it didn't. The coverage lines matter as much as the bugs: they tell you what to explore next.
Charter: checkout with a declined card and the Back button (18 min)
1. Serious: a declined payment still creates an order
Steps: Basket → Checkout → card 4000 0000 0000 0002 → "Pay $36.00"
→ "Your card was declined" → Back
Saw: [4] TextView "Order #1043 · Pending" #order_status on "Your basket"
2. Minor: "MM/YY" accepts 13/30 and only fails after "Pay"
3. Confusing: Back from "Payment" skips "Delivery" and lands on "Your basket"
Covered: card payment, decline, Back from each checkout step
Not covered: PayPal and saved cards (no test account for them)
Recording: explore-checkout.mp4Because each step quotes an element from the screen, a developer can follow it exactly. For the ones that matter, have the agent write the steps as a Maestro flow or an Espresso test, so they run on every commit from then on.
What a session costs
| Timebox | Phone time | Ten sessions a week |
|---|---|---|
| 10 minutes | $0.60 | $6 |
| 20 minutes | $1.20 | $12 |
| 45 minutes | $2.70 | $27 |
A recording costs nothing extra; Starter keeps 20 recordings for 30 days and Pro keeps 100 for 90 days. Your agent's model tokens are billed by your model provider. Between sessions, park the phone: it keeps the build and the signed-in test account and costs nothing.
Questions
How do I do AI exploratory testing on an Android app?
Install your build on a cloud Android phone, connect the phone to a coding agent such as Claude Code, Codex or Cursor through Phonebox's MCP server, and give it a charter: what to explore, with what, to discover what. Set a timebox, ask it to read the screen before every action, and ask for findings with steps and the screen text it saw.
Can AI replace manual exploratory testing?
Not entirely. An agent is good at broad, repetitive exploration: edge-case input, interrupted flows, restarts, other locales. It misses subtle visual and timing problems and anything that needs real hardware such as a SIM. Use it to cover more ground on every branch, and keep human sessions for judgment calls.
What is a test charter?
A short mission for an exploratory session, often written as 'Explore <target> with <resources> to discover <information>', a template from Elisabeth Hendrickson's book Explore It!. It focuses the session on one area without scripting each step.
Get started with Phonebox. New accounts start with a free 7-day trial, card required.
The Phonebox team