Phonebox / Docs
Troubleshooting

Actions fail

What to do when a target isn't found, matches several elements or has gone stale, when there is nowhere to type, and when a wait runs out.

View as Markdown

A batch stops at the first action that fails, and that action's result carries the error. The response then includes the element list, so you can see where the batch stopped, unless reading the screen after the batch failed or ran out of time. Then observation is missing, and you observe the screen yourself. Read the code, look at the screen, and change the action before you send it again. The actions before it already ran, so send only the changed action and the ones after it, never the whole batch. Action errors lists every code.

validation_failed

An HTTP 400 validation_failed means no action ran. Read error.details.issues and correct the named fields before sending another request. Do not retry the same invalid request in a loop. Each schema issue includes its path, code and message, with expected types or bounds when available. A target with an invalid shape also includes the accepted formats and branch alternatives. details.validation_stage distinguishes schema validation, JSON syntax, content type and header checks.

Send an object containing actions, not a bare array. Coordinates are integers from 0 to 10000, and a coordinate target is {"x":5000,"y":5000}, not {"point":{"x":5000,"y":5000}}. Check action names against the actions reference. The SDK and CLI validate batches locally before sending a write; a local validation error has no request ID.

target_not_found

No element matches the target. details.target repeats it, and details.candidates lists up to 10 elements that share a word with it, or else clickable elements with a label.

  • Look at the screen. The app may be on another screen than you expected, a dialog may cover it, or the keyboard may hide the element. Observe with a screenshot when the element list doesn't explain it.
  • Scroll. Only elements on the screen are listed. Scroll toward the element and try again.
  • Match the label. A text target tries an exact match first, then the same ignoring case, then text that contains the target. Use a shorter part of the label, or a candidate's exact text.
  • Target something else. Try the element's resource ID, {"id": "sign_in"}, or its description, {"desc": "Search"}, which icons often have instead of text. When the element has no label at all, tap it by coordinates.
  • Check nth. When nth is larger than the number of matches, details.matches says how many there were.

ambiguous_target

Several elements match the target equally well, such as three buttons that read "Add to cart". details.candidates lists them in reading order.

  • For a text target, add nth to pick one, counting from 1 in reading order: {"text": "Add to cart", "nth": 2}. Only text targets take nth, so narrow an ID or description target instead, for example with the full resource ID.
  • Or target a candidate by its ref: {"ref": 5, "snapshot_id": "snp_4f2k7m3q6z3a"}, with the snapshot_id from details.snapshot_id.
  • When details names no snapshot_id, the screen moved before the batch read it again, so observe before you act by ref.

stale_snapshot

The element behind a ref changed or went away since the observation that gave you the ref. A ref target holds only while the element is still on the screen with the same type, resource ID, text and description, overlapping its old position by at least half. A snapshot also expires after 15 minutes.

Observe again and use the new refs. Always take a ref and its snapshot_id from the same observation, and prefer text or ID targets on screens that change on their own, such as lists that load more items.

no_focused_field

A type action found no text field with focus, so there was nowhere to type. Tap the field first, in the same batch:

POST /v1/phones/{id}/actions
{
  "actions": [
    { "type": "tap", "target": { "id": "email" } },
    { "type": "type", "text": "alex@example.com" }
  ],
  "observe": "ui"
}

An observation's keyboard.focused_editable says whether a text field has focus. Some screens take a moment to focus the field after a tap, so a short wait between the tap and the typing can help.

Waits run out

wait_timeout means a wait_for didn't see its target appear, or disappear with gone, before timeout_ms. It is retryable, and details repeats the timeout_ms and the target.

  • Observe to see what the screen shows instead. A slow network, a dialog or an error message are common causes. On the first launch, Chrome and other apps may show setup, sign-in or permission screens instead of opening the requested page. Handle the visible screen, using its actual labels and language, before waiting for page content.
  • If the screen is still loading, wait again, with a longer timeout_ms. One wait_for can wait up to 30 seconds, and the waits in one batch up to 60 seconds in all, so wait longer by sending the wait_for again in a new request. Wait a few times at most: when the screen still hasn't changed, decide from what it shows, or park the phone and tell your user.
  • Wait for what you expect to see, not for a fixed time. A wait of a few seconds is right only when nothing on the screen tells you the change is done.

The action succeeded but nothing changed

  • The screen hadn't caught up. Many actions start an animation or a load. Follow them with a wait_for on the next thing you expect before you observe.
  • Another agent uses the phone. Phonebox doesn't lock a phone, so two agents on one phone act on screens they didn't read. Give each agent its own phone.
  • The app went to the background. In the first seconds after a phone becomes ready, the phone can still be finishing its own start-up. If the app isn't in front when you observe, open it again.

The whole request fails

A request that fails with one of Phonebox's errors, in its error envelope with an error.code, ran no actions at all, except when the error is 500 internal_error. The usual causes are a phone that isn't ready, such as 409 phone_not_running for a parked phone, and a phone that was briefly out of reach, 503 phone_unavailable. Both say what to do in next, and a batch refused with a retryable error other than internal_error is safe to send again. After 500 internal_error, observe first, because the batch may have run. When nothing ran lists these errors.

A failure without that envelope is different: a 502 or 504 that a proxy or the hosting platform sends, a timeout on your side, or a dropped connection. The batch may have run, in part or in full, so observe before you send it again, as Unknown outcomes describes.

When an action's outcome is unknown, its result has action_outcome_unknown, and the rules are different: read Unknown outcomes.

On this page