AI exploratory testing for Android apps: charters, a worked session, and the report to ask for.

The charter template is Elisabeth Hendrickson's, from Explore It!; timeboxed sessions with a debrief come from session-based test management. Phonebox facts are from its docs, checked November 16, 2026.

Why exploration suits an agent

A scripted test checks what someone already thought of. Exploration looks for what nobody did, which is the part of testing a script can't do and a model can: decide the next step from the screen in front of it.

Three ways to look for bugs in an Android build
Scripted UI testHuman exploratory sessionAgent exploratory session
Decides each stepIn advance, in codeFrom what the tester seesFrom the screen elements it reads
FindsRegressions in known journeysNew bugs, confusing flows, visual problemsNew bugs, broken edge cases, text that doesn't fit
MissesEverything not written downWhat the tester didn't think to trySubtle visual and timing problems, anything needing real hardware
RepeatableExactlyThrough notesThrough the report and a recording
RunsOn every commitA few times a releaseOn every branch, while you review

Write a charter, not a script

A charter names a target, the resources to use on it and the information you want: Explore <target> with <resources> to discover <information>. It keeps the agent on one area without telling it every tap. These four cover the bugs agents find most reliably on Android:

Charters to paste
Explore checkout with a declined test card and the Back button
to discover where an interrupted payment leaves the order.

Explore sign-up with empty, 500-character and emoji input
to discover which fields lose what was typed, or crash the app.

Explore the app in German (de-DE)
to discover cut-off labels, untranslated text and wrong date formats.

Explore the app after closing it in the middle of a task
to discover what state survives a restart.

Phonebox can change what the phone reports, which makes some charters possible that are awkward on an emulator: set the locale to de-DE or ar-EG for long words and right-to-left layouts, or the location for anything regional. Locale, location and timezone changes need a plan.

Heuristics to put in the task

Name the moves you want tried. Each one maps to an action the agent can take on the phone:

Android exploratory heuristics and the action behind each
HeuristicWhat it catchesAction
Back in the middle of a flowLost input, duplicate submits, wrong back stackback
Leave and come backState that doesn't survive the app going to the backgroundhome, then open_app
Close the app mid-taskState that doesn't survive a restartclose_app, then open_app
First run againOnboarding that only works onceclose_app with clear_data
Empty, very long and emoji inputValidation gaps and crashestype with clear
Tap twice, fastDouble orders and double sendsdouble_tap
Arrive from outsideDeep links that skip a required stepopen_url
Check the notification shadeNotifications with wrong or missing textnotifications

Run a timeboxed session

Set the timebox on the phone itself. --max parks the phone when time is up, so a session that wanders off can't keep billing:

Terminal
export PHONEBOX_PHONE=ph_7kx2m6q4v3ta
phonebox start --max 20m     # the timebox: the phone parks itself after 20 minutes
phonebox apps install app/build/outputs/apk/debug/app-debug.apk
phonebox record start --name "explore checkout"   # optional: a video of the session

Then give the agent the charter and the rules: read the screen before every action, note each finding with its steps as it goes, and park the phone at the end. Here is part of a session on the first charter. The agent reads the payment screen, fills it with a test card that's always declined, and checks where Back leads:

What the agent sees and does (abbreviated)
observe
  snapshot snp_q3k7z2m4x6ta
  app com.example.shop (.CheckoutActivity)
  keyboard hidden
  screen 1080x2400
  [1] TextView "Payment" (540,180)
  [2] EditText "Card number" #card_number clickable editable (540,520)
  [3] EditText "MM/YY" #expiry clickable editable (300,700)
  [4] EditText "CVC" #cvc clickable editable (780,700)
  [5] Button "Pay $36.00" #pay clickable (540,2200)

act   tap #card_number, type "4000 0000 0000 0002", tap #expiry, type "12/30",
      tap #cvc, type "123", tap "Pay $36.00", wait_for "declined"
      → 8 of 8 ok; "Your card was declined" appeared after 2.4 s

act   back
observe
  app com.example.shop (.BasketActivity)
  [1] TextView "Your basket" (540,180)
  [4] TextView "Order #1043 · Pending" #order_status (540,640)

The last line is the finding: the order exists although the payment failed. The agent saw it because it read the screen after Back, not because a test asserted it. If the app crashes, the next observation shows app com.android.launcher3 or a "keeps stopping" dialog; Phonebox phones don't expose ADB, so there's no logcat, and the recording is the evidence.

The report to ask for

Ask for findings ranked by severity, each with steps and the screen text, then what the session covered and what it didn't. The coverage lines matter as much as the bugs: they tell you what to explore next.

The report it hands back
Charter: checkout with a declined card and the Back button (18 min)

1. Serious: a declined payment still creates an order
   Steps: Basket → Checkout → card 4000 0000 0000 0002 → "Pay $36.00"
          → "Your card was declined" → Back
   Saw: [4] TextView "Order #1043 · Pending" #order_status on "Your basket"
2. Minor: "MM/YY" accepts 13/30 and only fails after "Pay"
3. Confusing: Back from "Payment" skips "Delivery" and lands on "Your basket"

Covered: card payment, decline, Back from each checkout step
Not covered: PayPal and saved cards (no test account for them)
Recording: explore-checkout.mp4

Because each step quotes an element from the screen, a developer can follow it exactly. For the ones that matter, have the agent write the steps as a Maestro flow or an Espresso test, so they run on every commit from then on.

What a session costs

Phone time per exploratory session at $0.06 a minute, billed per second
TimeboxPhone timeTen sessions a week
10 minutes$0.60$6
20 minutes$1.20$12
45 minutes$2.70$27

A recording costs nothing extra; Starter keeps 20 recordings for 30 days and Pro keeps 100 for 90 days. Your agent's model tokens are billed by your model provider. Between sessions, park the phone: it keeps the build and the signed-in test account and costs nothing.

Questions

How do I do AI exploratory testing on an Android app?

Install your build on a cloud Android phone, connect the phone to a coding agent such as Claude Code, Codex or Cursor through Phonebox's MCP server, and give it a charter: what to explore, with what, to discover what. Set a timebox, ask it to read the screen before every action, and ask for findings with steps and the screen text it saw.

Can AI replace manual exploratory testing?

Not entirely. An agent is good at broad, repetitive exploration: edge-case input, interrupted flows, restarts, other locales. It misses subtle visual and timing problems and anything that needs real hardware such as a SIM. Use it to cover more ground on every branch, and keep human sessions for judgment calls.

What is a test charter?

A short mission for an exploratory session, often written as 'Explore <target> with <resources> to discover <information>', a template from Elisabeth Hendrickson's book Explore It!. It focuses the session on one area without scripting each step.

Get started with Phonebox. New accounts start with a free 7-day trial, card required.

The Phonebox team

← All guides