Sign in to apps and hand off to a human

Many apps have to be signed in to before your agent can do anything useful in them. Your agent can drive most of a sign-in itself. When a step needs the person who owns the account, such as a code sent to their own phone or a check that only a person should pass, your agent hands them the phone through a live view link, waits for them to finish, and carries on. The account then stays signed in, parked or not.

Sign in only to accounts that your user owns or may use, and only as the app's own terms allow.

1. Drive the sign-inCopy Anchor Link

Open the app, wait for its sign-in screen, and fill in what your agent knows. This batch taps the email field, types the address and submits it, then waits for the next screen:

{
  "actions": [
    { "type": "open_app", "package": "com.example.app" },
    { "type": "wait_for", "target": { "text": "Sign in" }, "timeout_ms": 15000 },
    { "type": "tap", "target": { "text": "Sign in" } },
    { "type": "wait_for", "target": { "id": "email" }, "timeout_ms": 10000 },
    { "type": "tap", "target": { "id": "email" } },
    { "type": "type", "text": "alex@example.com", "submit": true },
    { "type": "wait_for", "target": { "text": "Enter the code" }, "timeout_ms": 20000 }
  ],
  "observe": "ui"
}

Look at the result's observation before the next step, since sign-in screens change from one attempt to the next: a consent screen, a cookie banner or an "is this you?" prompt can appear in between. Phonebox never stores the text your agent types. Activity records it only as [redacted]. Even so, when your agent shouldn't know a password at all, leave that field to the person too.

2. Hand the phone to a personCopy Anchor Link

When the screen asks for something only your user can give, such as a code sent to their phone or email, create a live view link and send it to them:

{
  "expires_in": 900
}

{
  "url": "https://phonebox.dev/live/Pz7mK2vQ9xR4tW8nB3cL6hJ5gF1dS0aYeUoIiTqNrVb",
  "expires_at": "2026-09-29T16:20:00.512Z"
}

From the CLI, phonebox live --expires 15m does the same, and over MCP the tool is live_view_url. Tell your user what to do in words, such as "Enter the code we sent you, then tap Verify", along with the link.

The link lets anyone who opens it control the phone, and every account signed in on it, until it expires. Send it only to the person who needs it, over a private channel, and give it the shortest expiry that works. Live view and human handoff describes what the person sees.

3. Wait for them to finishCopy Anchor Link

Your agent waits for the screen that comes after the person's step, with a wait_for of up to 30 seconds at a time:

{
  "actions": [
    { "type": "wait_for", "target": { "text": "Inbox" }, "timeout_ms": 30000 }
  ],
  "observe": "ui"
}

wait_timeout means the person isn't done yet, so your agent sends the same request again, for as long as the link lasts. When the link expires and the person still hasn't finished, your agent stops waiting, parks the phone and tells your user. A wait_for only reads the screen, so repeating it is always safe. Each request counts as activity, so the phone doesn't park for being idle while your agent waits, and an open live view page keeps it awake too. The session's max_duration still applies, so start the phone again to renew it if a handoff runs long.

When the expected screen appears, your agent observes and carries on. If a different screen appears, such as an error or another check, it observes and decides again, and hands the phone back to the person when it needs them once more.

4. Park the phoneCopy Anchor Link

Park the phone when the task is done. A parked phone keeps its apps and signed-in accounts, so the next session starts signed in, and the person doesn't have to repeat the sign-in.

phonebox park

Some apps sign out after a while of their own accord, or ask for a code again on a new session. When your agent finds the sign-in screen on a later start, it goes through the same steps.

Checks that keep coming backCopy Anchor Link

If an app keeps asking for a check, don't try to get around it. Circumventing the security measures of other services breaks Phonebox's terms, and apps treat such attempts as abuse. Hand the phone to the person, and if the checks continue, slow your agent down or ask the app's owner for another way in, such as an API.