Agent Browser

> **Install the belt CLI skill:** `npx skills add belt-sh/cli`

# Agentic Browser

Browser automation for AI agents via [inference.sh](https://inference.sh). Uses Playwright under the hood with a simple `@e` ref system for element interaction.


## Quick Start

> Requires inference.sh CLI (`belt`). [Install instructions](https://raw.githubusercontent.com/inference-sh/skills/refs/heads/main/cli-install.md)

```bash belt login

# Open a page and get interactive elements belt app run agent-browser --function open --input '{"url": "https://example.com"}' --session new ```

## Core Workflow

Every browser automation follows this pattern:

1. **Open** - Navigate to URL, get `@e` refs for elements 2. **Interact** - Use refs to click, fill, drag, etc. 3. **Re-snapshot** - After navigation/changes, get fresh refs 4. **Close** - End session (returns video if recording)

```bash # 1. Start session RESULT=$(belt app run agent-browser --function open --session new --input '{ "url": "https://example.com/login" }') SESSION_ID=$(echo $RESULT | jq -r '.session_id') # Elements: @e1 [input] "Email", @e2 [input] "Password", @e3 [button] "Sign In"

# 2. Fill and submit belt app run agent-browser --function interact --session $SESSION_ID --input '{ "action": "fill", "ref": "@e1", "text": "user@example.com" }' belt app run agent-browser --function interact --session $SESSION_ID --input '{ "action": "fill", "ref": "@e2", "text": "password123" }' belt app run agent-browser --function interact --session $SESSION_ID --input '{ "action": "click", "ref": "@e3" }'

# 3. Re-snapshot after navigation belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'

# 4. Close when done belt app run agent-browser --function close --session $SESSION_ID --input '{}' ```

## Functions

| Function | Description | |----------|-------------| | `open` | Navigate to URL, configure browser (viewport, proxy, video recording) | | `snapshot` | Re-fetch page state with `@e` refs after DOM changes | | `interact` | Perform actions using `@e` refs (click, fill, drag, upload, etc.) | | `screenshot` | Take page screenshot (viewport or full page) | | `execute` | Run JavaScript code on the page | | `close` | Close session, returns video if recording was enabled |

## Interact Actions

| Action | Description | Required Fields | |--------|-------------|-----------------| | `click` | Click element | `ref` | | `dblclick` | Double-click element | `ref` | | `fill` | Clear and type text | `ref`, `text` | | `type` | Type text (no clear) | `text` | | `press` | Press key (Enter, Tab, etc.) | `text` | | `select` | Select dropdown option | `ref`, `text` | | `hover` | Hover over element | `ref` | | `check` | Check checkbox | `ref` | | `uncheck` | Uncheck checkbox | `ref` | | `drag` | Drag and drop | `ref`, `target_ref` | | `upload` | Upload file(s) | `ref`, `file_paths` | | `scroll` | Scroll page | `direction` (up/down/left/right), `scroll_amount` | | `back` | Go back in history | - | | `wait` | Wait milliseconds | `wait_ms` | | `goto` | Navigate to URL | `url` |

## Element Refs

Elements are returned with `@e` refs:

``` @e1 [a] "Home" href="/" @e2 [input type="text"] placeholder="Search" @e3 [button] "Submit" @e4 [select] "Choose option" @e5 [input type="checkbox"] name="agree" ```

**Important:** Refs are invalidated after navigation. Always re-snapshot after: - Clicking links/buttons that navigate - Form submissions - Dynamic content loading

## Features

### Video Recording

Record browser sessions for debugging or documentation:

```bash # Start with recording enabled (optionally show cursor indicator) SESSION=$(belt app run agent-browser --function open --session new --input '{ "url": "https://example.com", "record_video": true, "show_cursor": true }' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file belt app run agent-browser --function close --session $SESSION --input '{}' # Returns: {"success": true, "video": <File>} ```

### Cursor Indicator

Show a visible cursor in screenshots and video (useful for demos):

```bash belt app run agent-browser --function open --session new --input '{ "url": "https://example.com", "show_cursor": true, "record_video": true }' ```

The cursor appears as a red dot that follows mouse movements and shows click feedback.

### Proxy Support

Route traffic through a proxy server:

```bash belt app run agent-browser --function open --session new --input '{ "url": "https://example.com", "proxy_url": "http://proxy.example.com:8080", "proxy_username": "user", "proxy_password": "pass" }' ```

### File Upload

Upload files to file inputs:

```bash belt app run agent-browser --function interact --session $SESSION --input '{ "action": "upload", "ref": "@e5", "file_paths": ["/path/to/file.pdf"] }' ```

### Drag and Drop

Drag elements to targets: