What is agentic testing? A practical definition for mobile app teams.
Definitions compared across Tricentis, Momentic, Katalon, Autify and Forrester, checked October 13, 2026.
Three ways to automate a UI test
| Scripted | AI-assisted | Agentic | |
|---|---|---|---|
| Who decides the steps | The engineer, in code | AI writes the code; it runs like any script | The agent, at run time |
| When the UI changes | The test breaks | AI repairs the locator | The agent finds another way |
| Repeatability | Exact | Exact | Different paths to the same goal |
| Mobile examples | Espresso, UI Automator, Appium, Maestro | Copilot writing Espresso, self-healing locators | Claude Code on a cloud phone, Journeys in Android Studio |
Where it falls short
- Non-determinism: two runs can take different paths, so a failure can't always be replayed step for step.
- Cost and speed: every step is a model call, so it's slower and dearer than a script.
- Gestures: Google lists long-press, double tap, multi-finger gestures and rotation among the gaps in its own Journeys preview.
- Judgment: the agent only knows the business rules you write down.
Try it on an Android app
Give your coding agent a Phonebox phone and a goal. It reads the screen as numbered text elements, so its report quotes what the app showed:
claude mcp add --transport http phonebox https://phonebox.dev/mcp \
--header "Authorization: Bearer $PHONEBOX_API_KEY"Install the debug APK on my Phonebox phone and open it.
Sign in with the test account from TEST_ACCOUNTS.md, open each tab
in the bottom bar once, then sign out. Read the screen before every
action. Report any crash, error message or empty screen, quoting the
screen text. Park the phone when you finish.Keep scripted tests as the gate for every commit, and use the agent for new screens, exploratory passes and bug repros.
Questions
What is agentic testing?
Agentic testing is software testing where an AI agent plans and executes the steps itself from a stated goal, deciding each action from what it observes and adapting until the goal passes or fails, instead of replaying a fixed script.
What is the difference between agentic testing and AI-assisted testing?
In AI-assisted testing, AI helps write or repair test scripts, which then run deterministically. In agentic testing, the AI model is in the execution loop and chooses each step at run time.
Is agentic testing reliable enough for CI?
Not as the only gate. Agent runs aren't exactly repeatable and cost a model call per step, so most teams keep scripted suites for merges and use agents for exploratory checks, new screens and bug reproduction.