# Actions Source: https://phonebox.dev/docs/api-reference/actions > Run a batch of actions, with every action type, target form and limit, the result format, and the errors of a batch and of each action. [Actions and targets](/docs/using-phones/actions) explains how batches, targets and failures work, with worked examples. This page is the exact reference. ## Run actions [#run-actions] `POST /v1/phones/{id}/actions` needs the `phones:control` scope. Runs up to 20 actions on a `ready` phone, in order, and stops at the first one that fails. The body is required and may be up to 200 KB. | Field | Type | Default | Rules | | --------- | ------ | ------- | --------------------------------------------------------------------------------------- | | `actions` | array | None | From 1 to 20 [actions](/docs/api-reference/actions#action-types). | | `observe` | string | `ui` | `none`, `ui`, `screenshot` or `both`: what the response shows of the screen afterwards. | The `ms` of every `wait` and the `timeout_ms` of every `wait_for` in a batch may add up to 60,000 at most. A `wait_for` without `timeout_ms` counts as 10,000. ```json title="POST /v1/phones/{id}/actions" { "actions": [ { "type": "tap", "target": { "id": "email" } }, { "type": "type", "text": "alex@example.com", "clear": true }, { "type": "tap", "target": { "text": "Continue" } }, { "type": "wait_for", "target": { "text": "Enter your password" }, "timeout_ms": 15000 } ], "observe": "none" } ``` ```bash curl https://phonebox.dev/v1/phones/ph_7kx2m6q4v3ta/actions \ -H "Authorization: Bearer $PHONEBOX_API_KEY" \ -H "Content-Type: application/json" \ -d '{"actions": [{"type": "tap", "target": {"id": "email"}}, {"type": "type", "text": "alex@example.com", "clear": true}, {"type": "tap", "target": {"text": "Continue"}}, {"type": "wait_for", "target": {"text": "Enter your password"}, "timeout_ms": 15000}], "observe": "none"}' ``` A batch that ran answers `200 OK`, whether or not every action succeeded: ```json title="Response: batch" { "results": [ { "index": 0, "type": "tap", "ok": true, "ms": 231, "resolved": { "ref": 2, "center": [540, 700] } }, { "index": 1, "type": "type", "ok": true, "ms": 1406 }, { "index": 2, "type": "tap", "ok": true, "ms": 198, "resolved": { "ref": 3, "center": [540, 950] } }, { "index": 3, "type": "wait_for", "ok": true, "ms": 2517 } ], "completed": 4 } ``` | Field | Meaning | | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `results` | One entry for each action that ran, in order, ending with the one that failed if one did. Actions after a failure don't run and have no entry. | | `results[].index`, `results[].type` | The action's position in the batch, from 0, and its type. | | `results[].ok` | Whether the action succeeded. | | `results[].ms` | How long it took, in milliseconds. | | `results[].resolved` | For `tap`, `double_tap` and `long_press`: the `center` acted on and, when the target was an element, its `ref` on the screen as it was read for the action. | | `results[].error` | For a failed action: `code`, `message`, `retryable`, `next` and `details`, as in the [action errors](/docs/api-reference/actions#action-errors). | | `completed` | How many actions succeeded. | | `observation` | The screen afterwards, as `observe` asked. | `observe` decides what `observation` holds: | `observe` | After a batch that completed | After a batch that stopped early | | ------------ | -------------------------------------------------------- | ------------------------------------ | | `none` | No `observation`. | The element list. | | `ui` | The element list. | The element list. | | `screenshot` | Only `taken_at` and a JPEG `screenshot` 720 pixels wide. | The element list and the screenshot. | | `both` | The element list and the screenshot. | The element list and the screenshot. | The element list is a full [observation](/docs/api-reference/observe#observe-the-screen), with a new `snapshot_id` whose refs you can target. A batch that stops early includes it, so you can see where it stopped, as long as the screen can be read. Reading the screen afterwards gets only the time the request has left. If it fails or doesn't finish in time, the response leaves `observation` out, and the results still come back, so observe the screen before you continue. ### When a batch stops [#when-a-batch-stops] This batch opened Settings, then found nothing labeled Bluetooth: ```json title="POST /v1/phones/{id}/actions" { "actions": [ { "type": "open_app", "package": "com.android.settings" }, { "type": "tap", "target": { "text": "Bluetooth" } } ] } ``` ```json title="Response: batch" { "results": [ { "index": 0, "type": "open_app", "ok": true, "ms": 1284 }, { "index": 1, "type": "tap", "ok": false, "ms": 367, "error": { "code": "target_not_found", "message": "No element on the screen matches the target.", "retryable": false, "next": null, "details": { "target": { "text": "Bluetooth" }, "candidates": [ { "ref": 1, "type": "TextView", "text": "Search settings", "desc": null, "id": "com.android.settings:id/search", "center": [540, 210] }, { "ref": 2, "type": "TextView", "text": "Network & internet", "desc": null, "id": null, "center": [540, 520] }, { "ref": 3, "type": "TextView", "text": "Connected devices", "desc": null, "id": null, "center": [540, 760] }, { "ref": 4, "type": "TextView", "text": "Apps", "desc": null, "id": null, "center": [540, 1000] } ], "snapshot_id": "snp_4f2k7m3q6z3a" } } } ], "completed": 1, "observation": { "snapshot_id": "snp_4f2k7m3q6z3a", "taken_at": "2026-09-29T14:36:02.337Z", "app": { "package": "com.android.settings", "activity": null }, "keyboard": { "visible": false, "focused_editable": false }, "screen": { "width": 1080, "height": 2400 }, "elements": [ { "ref": 1, "type": "TextView", "text": "Search settings", "desc": null, "id": "com.android.settings:id/search", "bounds": [60, 160, 1020, 260], "center": [540, 210], "state": ["clickable"] }, { "ref": 2, "type": "TextView", "text": "Network & internet", "desc": null, "id": null, "bounds": [0, 460, 1080, 580], "center": [540, 520], "state": ["clickable"] }, { "ref": 3, "type": "TextView", "text": "Connected devices", "desc": null, "id": null, "bounds": [0, 700, 1080, 820], "center": [540, 760], "state": ["clickable"] }, { "ref": 4, "type": "TextView", "text": "Apps", "desc": null, "id": null, "bounds": [0, 940, 1080, 1060], "center": [540, 1000], "state": ["clickable"] } ], "truncated": false, "text": "app com.android.settings\nkeyboard hidden\nscreen 1080x2400\n[1] TextView \"Search settings\" #search clickable (540,210)\n[2] TextView \"Network & internet\" clickable (540,520)\n[3] TextView \"Connected devices\" clickable (540,760)\n[4] TextView \"Apps\" clickable (540,1000)" } } ``` The action read the screen before the batch's observation did, and the screen can move in between. So Phonebox looks up each candidate of a `target_not_found` or `ambiguous_target` in the returned observation, gives it that element's `ref` and `center`, and names the observation's `snapshot_id` in `details.snapshot_id`. When any candidate isn't there anymore, `details` keeps the refs as the action saw them and names no snapshot, so observe again before you act by ref. Here, the next step is to tap one of the candidates, such as `{"ref": 3, "snapshot_id": "snp_4f2k7m3q6z3a"}`, and look again. The actions before the one that failed have already run, so the next batch starts at the failed action, changed as needed, followed by the actions that came after it. Never send the whole batch again: here, a new batch would not open Settings a second time. ### When nothing ran [#when-nothing-ran] Every error that Phonebox sends from this endpoint, in the [error envelope](/docs/api-reference/errors) with its `error.code`, means that no action ran, except `500 internal_error`. That covers the request's own problems, such as a bad body or a phone that isn't ready, and a phone service that failed before the first action reached the phone: ```json title="Response: error" { "error": { "type": "provider_error", "code": "phone_unavailable", "message": "The phone is temporarily unavailable.", "retryable": true, "next": "Nothing was done on the phone; retry this action in a few seconds.", "request_id": "req_t3v5x2z6b4d7", "details": { "retry_after_seconds": 2 } } } ``` A batch refused this way is safe to send again after the wait. Any other failure leaves the outcome unknown: a `500 internal_error`, a `502` or `504` without the envelope, which a proxy or the hosting platform sends, a timeout on your side, a dropped connection, or no answer at all. The batch may have run, in part or in full, so observe before you send anything again. [Failures without the envelope](/docs/api-reference/errors#failures-without-the-envelope) explains why. | Error | When | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `400 validation_failed` | The body breaks a rule: an unknown action type or field, a value out of range, more than 20 actions, or more than 60 seconds of waits. | | `413 payload_too_large` | The body is larger than 200 KB. | | `503 phone_unavailable` | The first action couldn't reach the phone, so nothing ran. This error is retryable. When the phone's control service was out of reach, `Retry-After` and `details.retry_after_seconds` are 2 seconds. Otherwise the response has no `Retry-After`, so wait a few seconds. | | `503 capacity_unavailable` | The phone service had no capacity for the first action, so nothing ran. Try again after `Retry-After`. | | `429 rate_limited` | The phone service asked Phonebox to slow down before the first action, so nothing ran. `Retry-After` and `details.retry_after_seconds` say how long to wait. | | `502 provider_error` | The phone service failed before the first action reached the phone, for example while reading the screen to find the target. Nothing ran, and this error is retryable. | The readiness errors in [Common errors](/docs/api-reference#common-errors) apply too. ## Action types [#action-types] | `type` | Fields | Rules and defaults | | ------------------------------------------ | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `tap` | `target` | Any [target](/docs/api-reference/actions#targets). | | `double_tap` | `target` | Two taps 80 ms apart. | | `long_press` | `target`, `duration_ms` | `duration_ms` from 300 to 5000, 800 by default. | | `swipe` | `from`, `to`, `duration_ms` | `from` and `to` are points, `{"x": …, "y": …}`. `duration_ms` from 50 to 5000, 300 by default. | | `scroll` | `direction`, `target`, `amount` | `direction` is `up`, `down`, `left` or `right`. `target` is optional: an element to scroll inside, or the whole screen when you leave it out or give a point. `amount` from 0.1 to 1, 0.6 by default. | | `type` | `text`, `clear`, `submit` | `text` has 1 to 5000 characters and goes into the focused field. `clear` empties the field first and `submit` presses Enter afterwards, both `false` by default. | | `key` | `key` | A key name, or an Android key code from 0 to 400. | | `back`, `home`, `recents`, `notifications` | None | Press Back, go home, open the recent apps, or open the notifications. | | `open_app` | `package`, `activity` | `package` is an Android package name. `activity` is optional, up to 300 characters. | | `close_app` | `package`, `clear_data` | `clear_data`, `false` by default, also erases the app's data and its sign-ins. | | `open_url` | `url`, `package` | `url` is a web address or a deep link, 3 to 2000 characters. `package` is optional: the app to open it in. | | `wait` | `ms` | From 1 to 10000 milliseconds. | | `wait_for` | `target`, `gone`, `timeout_ms` | `target` is a text, ID or description target. `gone`, `false` by default, waits for the target to disappear instead. `timeout_ms` from 100 to 30000, 10000 by default. It checks the screen every 500 ms. | The key names are `enter`, `delete`, `tab`, `escape`, `space`, `up`, `down`, `left`, `right`, `page_up`, `page_down`, `move_home`, `move_end`, `menu`, `search`, `volume_up` and `volume_down`. A package name has at least two parts separated by dots, such as `org.wikipedia`, and at most 255 characters. ## Targets [#targets] A target takes exactly one of these forms. [Targets](/docs/using-phones/actions#targets) explains how each one finds its element. | Form | Rules | | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | `{"x": 540, "y": 950}` | A point in device pixels. Each coordinate is an integer from 0 to 10000. | | `{"ref": 3, "snapshot_id": "snp_4f2k7m3q6z3a"}` | `ref` from 1 to 1000, and the `snapshot_id` of an observation from the last 15 minutes. The element must still be on the screen, unchanged. | | `{"text": "Sign in", "nth": 2}` | `text` from 1 to 500 characters, matched against text and descriptions. `nth` is optional, from 1 to 50. | | `{"id": "sign_in"}` | 1 to 300 characters: a full resource ID, or the part after `:id/`. | | `{"desc": "Back"}` | 1 to 500 characters, matched against content descriptions. | Text, ID and description targets see the same elements that an observation lists, which are at most 250. ## Action errors [#action-errors] A failed action's `error` has the same fields as an [error envelope](/docs/api-reference/errors), without `type` and `request_id`. These are the codes it can carry: | Code | Retryable | Details | What to do | | ------------------------ | --------- | ---------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `target_not_found` | `false` | `target`, `candidates` (up to 10), `snapshot_id`, and `matches` when `nth` was larger than the number of matches | Use one of the candidates, scroll to bring the element into view, or observe again. | | `ambiguous_target` | `false` | `candidates` (up to 10), `snapshot_id` | Target a candidate by `ref` with `details.snapshot_id`, or add `nth` to a text target. Only text targets take `nth`, so narrow an ID or description target instead, for example with the full resource ID. | | `stale_snapshot` | `false` | `snapshot_id` | The element behind the ref changed or went away, or the snapshot is older than 15 minutes. Observe again. | | `no_focused_field` | `false` | None | A `type` action found no focused text field. Tap the field, then type. | | `wait_timeout` | `true` | `timeout_ms`, `target` | A `wait_for` ran out of time. Observe, and wait again with a longer `timeout_ms` if the screen is still loading. | | `app_not_found` | `false` | `package` | The app isn't installed, or its install hasn't finished. Install it, or check the package name. | | `action_outcome_unknown` | `false` | None | The phone didn't confirm the action, which may or may not have happened. Observe before anything else. | | `phone_unavailable` | `true` | `retry_after_seconds`, when the phone's control service was out of reach | This action didn't run. After a few seconds, send it and the actions after it again. | | `rate_limited` | `true` | `retry_after_seconds` | This action didn't run. Wait, then send it and the actions after it again. | | `capacity_unavailable` | `true` | None | This action didn't run. Try it and the actions after it again later. | | `provider_error` | `true` | None | The phone service refused or failed the action. Observe, then decide whether to try again. | | `provider_timeout` | `true` | `not_run` | The batch had already run for 90 seconds, so this action and the ones after it didn't run. `not_run` counts them. Observe, then send them in a new batch. | | `internal_error` | `true` | None | Something failed on our side. Observe before you decide. | ### Partial actions [#partial-actions] A `type` with `submit` types and then presses Enter, and a `double_tap` taps twice. When the second step fails after the first reached the phone, the error has `details.partial` set to `true` and `retryable` set to `false`, whatever its code, and `next` says which step was done. Observe before you do anything else. The second step may itself have happened, so send it only when the screen shows that it didn't, for example a form still waiting with your text in it. Never repeat the whole action. [Unknown outcomes](/docs/troubleshooting/unknown-outcomes) shows how to recover from these cases.