Computer-Use tools
The verbs a run can call on a Computer-Use binding: a browser for a product with no MCP server, limited to the sites the binding allows.
Generated from the built-in tools registry. Do not edit by hand.
Some products have no MCP server. A Computer-Use binding gives a run a browser instead: a prefix the enterprise chooses, the hosts it may open, and a start URL. A task names each verb it may use under that prefix, <prefix>.<verb>, and gets no browser for a bare prefix.
The browser only opens the hosts the binding allows. Credentials the run uses are taken out of everything the model reads back.
Each verb is gated by name, the way it is granted. A gate the task declares on <prefix>.<verb>, or on the bare prefix, holds that call for a person before it runs, and a gate on one verb holds no other: a gate on <prefix>.click does not hold a <prefix>.press of Enter that submits the same form before it runs.
What this gives up: on a site its binding allows, a run submits, pays, sends and deletes without asking anyone, unless the task declares a gate on the verb. The task’s granted verbs and the binding’s sites are the whole grant. Whatever a verb sets off goes to the site as the browser sends it: a form that posts, a fetch that writes, a beacon. So does what the page sends on its own between actions. Nothing is held, gated or aborted.
The sites are a rule about where the browser goes, not about what a page sends. A landing, a redirect or a tab on a host outside the binding’s sites is refused, and the browser is taken back to the last allowed page. A page on an allowed host can still send requests to other hosts, such as an embedded payment form posting to its processor, or analytics, and those go out as any browser sends them.
A person is asked only where the task declares a gate, and for two verbs: upload, which waits for a person to approve the file before it is attached, and sign_in, which stops for a person to sign in. Rejecting an upload cancels the run, as rejecting any gate does, and the file is never attached. A declared gate waits on Gates for the person it names as its owner, like any gated call, and nothing happens on the site until they approve it. Approving runs the verb, and what the verb sends goes with it: no second gate is opened. A gate the task tags with an outcome holds its verb the same way, and nothing else.
A task that wants a person to see a submission first should gate every verb that can submit it: click, type, press and select, or the bare prefix. For example, with the prefix crm, a gate on each of crm.click, crm.type, crm.press and crm.select asks before any of those verbs runs, while a gate on crm itself asks before every call the browser makes, reading ones included.
A person who has taken over a run’s browser acts on the site directly, as in their own browser, and hands it back when they are done.
| Tool | What it does | Effect |
|---|---|---|
<prefix>.navigate |
Open url in the run's browser. |
read |
<prefix>.click |
Click the element ref or selector names. |
write |
<prefix>.type |
Type text into the element ref or selector names. |
write |
<prefix>.screenshot |
Take a screenshot of the page. | read |
<prefix>.extract |
Read the text of the element ref or selector names. |
read |
<prefix>.download |
Download the file the site offers at url, or behind the control selector names, and keep it as a run file a person can take from the run. |
read |
<prefix>.upload |
Attach a run file, named by the handle (file) a download or the task gave it, to the file input selector names. Waits for a person to approve it first. |
write |
<prefix>.sign_in |
Stop at the sign-in page the browser is on and wait for a person to sign in. Their sign-in is saved for the run's member on that site. | read |
<prefix>.snapshot |
Read the page as its accessibility tree, with a reference such as [ref=e12] on each element click, type and extract can take as ref. |
read |
<prefix>.wait |
Wait until the page reaches a state: its address contains url_contains, text is visible, or the element ref or selector names is state, within timeout. |
read |
<prefix>.select |
Choose the option labelled label in the dropdown ref or selector names. |
write |
<prefix>.press |
Press the key key, on the element ref or selector names or on whatever has focus. |
write |
<prefix>.switch_tab |
Make the open tab whose URL or title contains match the one every later verb acts on: a popup or a page the site opened in a new tab. |
read |
<prefix>.hover |
Move the pointer over the element ref or selector names, so a menu that opens on hover opens. |
read |
<prefix>.scroll |
Bring the element ref or selector names into view, or scroll the page amount pixels in direction. |
read |
<prefix>.navigate
Open url in the run's browser.
What the model reads:
Open a page in this run's browser. Pass url, the full address, on one of the sites the <prefix> binding allows: any other host is refused. Returns the address the browser reached, after any redirect.
| Field | Type | Required | Description |
|---|---|---|---|
url |
string | yes | The full address to open. Its host must be one of the sites the binding allows. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The page reached.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
<prefix>.click
Click the element ref or selector names.
What the model reads:
Click one element on the page open in this run's browser. Pass ref, the reference the latest snapshot gave it, or selector, a CSS selector for it; ref is used when both are given. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave the element to click: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to click. |
- Effect: write.
- Retry: Not retry-safe: a repeat acts again.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.type
Type text into the element ref or selector names.
What the model reads:
Type into one field on the page open in this run's browser. Pass ref, the reference the latest snapshot gave the field, or selector, a CSS selector for it; ref is used when both are given. Pass text, which replaces what the field holds. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave the element to type into: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to type into. |
text |
string | yes | What to put in the element. It replaces what the element holds. |
- Effect: write.
- Retry: Not retry-safe: a repeat acts again.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.screenshot
Take a screenshot of the page.
What the model reads:
Take a screenshot of the page open in this run's browser. Takes no arguments. The screenshot is reported by its size, not pasted into the conversation.
Takes no arguments.
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The screenshot, reported by size rather than pasted into the conversation.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
<prefix>.extract
Read the text of the element ref or selector names.
What the model reads:
Read the text of one element on the page open in this run's browser. Pass ref, the reference the latest snapshot gave it, or selector, a CSS selector for it; ref is used when both are given. Every credential the run used is taken out of the text before you read it.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave the element whose text to read: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element whose text to read. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The text, with every credential the run used taken out.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.download
Download the file the site offers at url, or behind the control selector names, and keep it as a run file a person can take from the run.
What the model reads:
Download a file from the site open in this run's browser and keep it as a run file. Pass url, the file's address on one of the sites the <prefix> binding allows, or selector, a CSS selector for the control whose click starts the download. url is used when both are given, and every redirect is checked against the allowed sites. Returns the run file's handle (id), name, size and content type, and the URL it came from, never its bytes.
| Field | Type | Required | Description |
|---|---|---|---|
url |
string | no | The file's address. Its host, and every redirect's, must be one of the sites the binding allows. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element whose click starts the download. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The run file's handle (
id), name, size and content type, and the URL it came from. Never its bytes.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_file_not_saved |
The downloaded file could not be kept, for example because the artifact store refused its size. | Tool error |
<prefix>.upload
Attach a run file, named by the handle (file) a download or the task gave it, to the file input selector names. Waits for a person to approve it first.
What the model reads:
Attach a file this run holds to a file input on the page open in this run's browser. Pass selector, a CSS selector for the file input, and file, the handle of a run file: the id a download returned, or the handle of a file the task was given. A path or a URL is refused. A person approves every upload before the file is attached. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
selector |
string | yes | A CSS selector for the element to attach the file to. |
file |
string | yes | The handle of a file this run holds: the id a download returned, or the handle of a file the task was given. A path or a URL is refused. |
- Effect: write.
- Retry: Not retry-safe: a repeat acts again.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The page the file was attached on.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_upload_refused |
The upload named something that is not a file this run holds: a path, a URL, or a handle no file has. | Tool error |
<prefix>.sign_in
Stop at the sign-in page the browser is on and wait for a person to sign in. Their sign-in is saved for the run's member on that site.
What the model reads:
Call this when the page open in this run's browser asks you to sign in. Never type a username or password yourself. The run stops and waits for a person to take the browser, sign in and hand it back. Then you carry on from the page they left it on. What they typed is never shown to you. Pass reason, what on the page says a sign-in is needed. Returns the address the browser is on once the person has handed it back.
| Field | Type | Required | Description |
|---|---|---|---|
reason |
string | no | What on the page says a sign-in is needed, for the person who is asked to sign in. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The page the browser is on after the person signed in.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_sign_in_not_completed |
Nobody signed in before the enterprise's gate timeout. The run is cancelled, and its record names the site. | Run fails |
<prefix>.snapshot
Read the page as its accessibility tree, with a reference such as [ref=e12] on each element click, type and extract can take as ref.
What the model reads:
Read the page open in this run's browser as its accessibility tree: one line per element with its role and name, and a reference such as [ref=e12] on each element you can act on, iframe content included under its iframe. Pass that reference as ref to click, type or extract instead of a selector. Only the latest snapshot's references work, and once the page has changed you snapshot again. Pass ref or selector to read only that element and what it contains, ref is used when both are given, and depth to keep only that many levels. A snapshot too large for one tool result is refused with its size, never cut short: scope it or make it shallower. Every credential the run used is taken out of the text. Pass since as true to get only what changed since the latest snapshot of this page, read with the same ref, selector and depth: the elements added, changed and removed, each anchored under its parent's reference, or a line saying nothing changed. After it, every reference to an element still on the page works, listed or not. When there is no earlier snapshot of this page to compare with, because there is none yet, the page has moved, you switched tabs or you asked for another part of the page, it returns a full snapshot and says so. Returns the page's title and address, then the tree.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave an element: e12 for [ref=e12]. Reads only that element and what it contains. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element whose part of the page to read. |
depth |
integer, at least 1 | no | How many levels of the tree to keep, counted from the top, or from the element ref or selector names. Every level when it is left out. |
frame |
boolean | no | true to keep a frame (a screenshot) of the page for this snapshot in the run's step record, for a person watching the run. Left out, the snapshot is recorded with no frame. What you are told back does not change. |
since |
boolean | no | true to get only what changed since the latest snapshot of this page, read with the same ref, selector and depth: the elements added, changed and removed, each anchored under its parent's reference. Every reference to an element still on the page keeps working, listed or not. With no earlier snapshot of this page to compare with (none yet, the page has moved, another tab, or another part of the page), it returns a full snapshot and says so. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The page's title and address, then the accessibility tree with a reference on each element you can act on, with every credential the run used taken out. With since, only what changed since the latest snapshot of this page, or a full snapshot that says why when there is none to compare with.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_snapshot_too_large |
The tree is too large for one tool result. It is refused with its size, never cut short: scope it to one element with ref or selector, or make it shallower with depth. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.wait
Wait until the page reaches a state: its address contains url_contains, text is visible, or the element ref or selector names is state, within timeout.
What the model reads:
Wait until the page open in this run's browser reaches a state, then return. Pass url_contains, text the page's address must contain; text, text that must be visible on the page, iframes included, ignoring case and runs of whitespace; or ref or selector with state (visible, hidden or enabled), an element that must reach that state, visible when state is left out; ref is used when both are given. When you pass more than one, all of them must hold before it returns, and a call naming none is refused. Pass timeout, in milliseconds, to wait less than the operator cap, COMPUTER_USE_WAIT_MAX_MS: 180000 ms (three minutes) unless the operator set it lower: left out it is that cap, and a longer one is clamped to it, never refused. A wait that runs out is the tool error computer_use_wait_timeout, naming what it waited for, the timeout it used and the page it ended on; the browser stays open and you carry on. It does not wait for network idle or for a download. If the page moved while you waited, references from your last snapshot no longer work: snapshot again. Returns the address the browser is on and the conditions that held.
| Field | Type | Required | Description |
|---|---|---|---|
url_contains |
string | no | Text the address of the browser's page must contain, for a redirect to land. |
text |
string | no | Text that must be visible on the page, iframes included. Case and runs of whitespace are ignored. |
ref |
string | no | A reference the latest snapshot gave the element to wait on: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to wait on. |
state |
string, one of visible, hidden, enabled |
no | The state the element ref or selector names must reach. visible when it is left out. |
timeout |
integer | no | How long to wait, in milliseconds. Left out, it is the operator cap, COMPUTER_USE_WAIT_MAX_MS: 180000 ms (three minutes) unless the operator set it lower; a longer one is clamped to that. |
frame |
boolean | no | true to keep a frame (a screenshot) of the page for this wait in the run's step record, for a person watching the run. Left out, the wait is recorded with no frame. A failed wait always keeps a frame, whatever this says. What you are told back does not change. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The address the browser is on, and the conditions that held with the timeout used. No accessibility tree.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_wait_timeout |
The page did not reach every condition asked for within the timeout, which is clamped to the operator cap, COMPUTER_USE_WAIT_MAX_MS: 180000 ms (three minutes) unless the operator set it lower. The error names the conditions, the timeout used and the page the browser is on. The browser stays open. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.select
Choose the option labelled label in the dropdown ref or selector names.
What the model reads:
Choose an option in one native dropdown (a select element) on the page open in this run's browser. Pass ref, the reference the latest snapshot gave the dropdown, or selector, a CSS selector for it; ref is used when both are given. Pass label, the option's visible label exactly as the dropdown shows it: the label only, there is no value argument. A label the dropdown does not offer is the tool error computer_use_unknown_option, which lists the labels it does offer, and the dropdown keeps the value it had. Choosing an option that moves the browser off the sites the <prefix> binding allows is refused, as a click would be. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave the element to choose an option in: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to choose an option in. |
label |
string | yes | The visible label of the option to choose, exactly as the dropdown shows it. Where two options share it, the first is chosen. |
- Effect: write.
- Retry: Idempotent: a repeat of the same call acts once, or leaves the same state.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The address the browser is on afterwards.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_unknown_option |
The dropdown offers no option whose visible label is exactly label. The error lists the labels it does offer, in the order it offers them. Nothing was chosen: the dropdown keeps the value it had. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.press
Press the key key, on the element ref or selector names or on whatever has focus.
What the model reads:
Press one key on the page open in this run's browser. Pass key, the name of the key: Escape, Enter, Tab, ArrowDown, ArrowUp, Backspace, Space and the like. It goes to whatever has focus; pass ref, the reference the latest snapshot gave an element, or selector, a CSS selector for it, to focus that element first; ref is used when both are given. A key name the browser does not know is refused. A key that moves the browser off the sites the <prefix> binding allows is refused, as a click would be, and once the page has moved, snapshot again before using a reference. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
key |
string | yes | The name of the key to press: Escape, Enter, Tab, ArrowDown, ArrowUp, Backspace, Space and the like. |
ref |
string | no | A reference the latest snapshot gave the element to focus before the key is pressed: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to focus before the key is pressed. |
- Effect: write.
- Retry: Not retry-safe: a repeat acts again.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The address the browser is on afterwards.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.switch_tab
Make the open tab whose URL or title contains match the one every later verb acts on: a popup or a page the site opened in a new tab.
What the model reads:
Switch to another of the tabs open in this run's browser: a page the site opened in a new tab or a popup, such as a payment provider's sign-in or a 3-D Secure challenge, or back to the page you came from. Pass match, a substring of the tab's URL or title, case ignored; where more than one tab has it, the one opened first is chosen. Every later verb acts on that tab until you switch again. A tab on a host the <prefix> binding does not allow is refused, and you stay on the tab you were on. A match no tab has, or no match at all, is the tool error computer_use_tab_not_found, which lists every open tab by its title and host. References from an earlier snapshot do not work on the tab you switch to: snapshot it first. If the tab you switched to closes itself, the next verb acts on the tab you switched from. Returns the URL of the tab you are now on, and its title.
| Field | Type | Required | Description |
|---|---|---|---|
match |
string | no | A piece of the tab's URL or title, case ignored: pay.example.org, or PayPal. Where more than one tab has it, the one opened first is chosen. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The URL of the tab the run is now on, and its title.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_tab_not_found |
No open tab's URL or title contains match, or no match was given. The error lists every open tab by its title and host, the tab the run is on marked. Nothing changed: the run stays on that tab. | Tool error |
<prefix>.hover
Move the pointer over the element ref or selector names, so a menu that opens on hover opens.
What the model reads:
Move the pointer over one element on the page open in this run's browser, the way a person hovers it, so a menu or a tooltip that only appears on hover opens. Pass ref, the reference the latest snapshot gave it, or selector, a CSS selector for it; ref is used when both are given. Snapshot afterwards to see what appeared. A hover that moves the browser off the sites the <prefix> binding allows is refused, as a click would be. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave the element to move the pointer over: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to move the pointer over. |
- Effect: read.
- Retry: Safe: repeating the call changes nothing.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The address the browser is on afterwards.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |
<prefix>.scroll
Bring the element ref or selector names into view, or scroll the page amount pixels in direction.
What the model reads:
Scroll the page open in this run's browser, so content it renders only once it is scrolled to appears. Pass ref, the reference the latest snapshot gave an element, or selector, a CSS selector for it, to bring that element into the viewport; ref is used when both are given, and direction and amount are then ignored. Otherwise pass direction (up, down, left or right; down when left out) and amount, how far in CSS pixels (one viewport when left out), to scroll the page itself: a bare scroll is one viewport down. Any other direction, or an amount that is not a positive number, is refused and nothing scrolls. A scroll that moves the browser off the sites the <prefix> binding allows is refused, as a click would be. Snapshot afterwards to see what appeared. Returns the address the browser is on afterwards.
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | no | A reference the latest snapshot gave the element to bring into view: e12 for [ref=e12]. Used instead of selector when both are given. |
selector |
string | no | A CSS selector for the element to bring into view. |
direction |
string, one of up, down, left, right |
no | Which way to scroll the page: up, down, left or right. down when it is left out. Ignored when ref or selector names an element. |
amount |
integer, at least 1 | no | How far to scroll the page, in CSS pixels. One viewport when it is left out. Ignored when ref or selector names an element. |
- Effect: read.
- Retry: Not retry-safe: a repeat acts again. A page scroll (direction and amount) moves the page again. Bringing an element into view (ref or selector) leaves one already in view where it is.
- Called by: a run.
- Needs:
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
<prefix>.<verb>by name: a bare prefix grants no browser.
- A Computer-Use binding on this prefix (host-only allowed sites and a start URL), a model that can drive a browser, and a task that lists
- Returns: The address the browser is on afterwards.
| Error | When | What happens |
|---|---|---|
computer_use_model_unsupported |
The run's model is not one trusted to drive a browser. The run is blocked before any model call. | Run fails |
computer_use_browser_unavailable |
No browser could be started for the run. | Run fails |
computer_use_site_blocked |
The page is on a host outside the binding's allowed sites. | Tool error |
computer_use_failed |
The browser action itself failed. | Tool error |
computer_use_stale_ref |
The ref is not from the latest snapshot, or the page has changed since that snapshot. Nothing was done: snapshot again and use a reference from the new one. | Tool error |