Metadata-Version: 2.4
Name: X5Browser
Version: 1.6.0
Summary: Browser library for AI agents: real Chrome over CDP (every action returns result + stable map + screenshot), ad-defense shield, live view.
Author: X5Coder
License: MIT
Keywords: browser,chrome,cdp,ai-agent,automation,live-view,vnc,adblock
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp>=3.9
Requires-Dist: websockets>=15
Provides-Extra: annotate
Requires-Dist: Pillow>=10; extra == "annotate"
Dynamic: license-file

# X5Browser

Real **Google Chrome** for AI agents: drive the original browser over raw CDP (no Playwright/Puppeteer/Selenium, no Chromium substitutes), read pages as numbered text maps, search Google and fetch pages as clean data, arm page-monitor triggers on their own cadence, survive ad-infested sites with a built-in shield, and hand the **same browser** to a human and back.

```bash
pip install X5Browser
```

```python
import asyncio
from x5browser import X5Browser, BrowserConfig

async def main():
    async with X5Browser(BrowserConfig.default()) as browser:
        await browser.navigate("https://example.com")
        print(await browser.page_snapshot())          # numbered element map
        refs = await browser.find_elements("Learn more")
        # ... pick a ref, e.g. 3
        result = await browser.click(ref=3)           # real mouse click
        print(result.message)
        print(result.observation)                     # fresh stable map, attached

asyncio.run(main())
```

Every **acting** tool waits for full page stability, then replies with
`message` + fresh stable element map + one screenshot — the agent always sees
the outcome at once, no extra snapshot call needed.

---

## 0. Setup check — the doctor

One command answers: *can this machine run the browser right now?*

```bash
python -m x5browser.doctor            # inspect only
python -m x5browser.doctor --fix     # also install Chrome when missing
python -m x5browser.doctor --smoke   # also launch + close Chrome once
```

```python
from x5browser import run_doctor
report = run_doctor()                # {"ok": ..., "checks": [...]}
```

Nothing installs at `pip install` time and nothing runs before you ask: the
first Chrome process appears at the first `start()` / `open_browser` (lazy).
If the host skips the doctor, the first start runs the environment gate
automatically (unsupported OS / unwritable profile fail with a fix hint).

---

## 1. Chrome, the original

- **Google Chrome only — Chromium is never accepted.** The user's own system
  Chrome (with their login and UI) always wins when present.
- **Missing Chrome is installed officially:** the genuine installer from
  `dl.google.com` (the only allowed host). Windows shows one normal UAC
  prompt then continues silently; Linux tries passwordless `sudo`, then an
  interactive prompt. If everything fails you get exact copy-paste manual
  commands — and only then. Opt out with `X5_NO_AUTO_INSTALL_CHROME=1`.
- **Servers:** Ubuntu without a GUI runs `BrowserConfig(headless=True)` —
  every agent tool works fully headless (only live view needs Xvfb). Running
  as root adds `--no-sandbox` automatically (Chrome refuses its sandbox as
  root).

---

## 2. Two facades

| Class | Use it when | Import |
|---|---|---|
| `X5Browser` | You wire your own agent loop / UI | `from x5browser import X5Browser, BrowserConfig` |
| `LiveBrowser` | You want hidden Chrome + live link + timeline in one object | `from x5browser import LiveBrowser` |

```python
from x5browser import LiveBrowser

live = LiveBrowser()              # library profile data, headed Chrome
await live.start()
print(live.links())               # {"control": ..., "preview": ..., "ready": true}
print(live.live_url)              # raw noVNC link (open in any browser / <iframe>)
await live.navigate("https://example.com")
for step in live.steps():         # timeline rows with screenshots
    ...
await live.take_control()         # user drives the SAME browser
await live.agent_control()        # agent resumes with a fresh map
print(live.links())               # same links until close/restart
await live.stop_browser()         # quiet close, data kept
```

### Live view — one gated link

`live.links()` (or the `human_control` / `live_preview` tool replies) gives the
two session links: a **control** link (interactive) and a **preview** link
(watch-only). They are random, distinct, bound to the same session, and stay
identical until the browser closes or restarts; a restart returns a new pair.

The link points at the token gate:

- valid token + live browser -> `200` viewer HTML;
- anything else -> `410` `{"status": "Not Connect"}`;
- both answer `Access-Control-Allow-Origin: *`, so one plain `fetch` tells
  "live" (HTML) from "closed" (JSON) — the consumer's single check.

The viewer's mode is forced from the token (never the query string), so a
preview link can never send input. Session state is pushed over a single SSE
stream (`GET /api/events`), so a UI needs no polling; the pusher only runs
while a client is connected.

Run the gate (brings up the hidden stack; the browser stays off until a tool
needs it):

```bash
python scripts/run_ui.py            # gate: http://127.0.0.1:8790/view
```

On Windows the hidden stack runs in WSL: the library keeps the distro alive
(no idle shutdown) while it is in use, and the systemd units self-heal
(`Restart=always`), so a dropped broadcast returns on the same link.

### Live view on a native Linux server

On a VPS the same flow applies — units are installed automatically on first
start (`scripts/setup_local.py` does it explicitly), and the ports are fixed:
viewer `6080`, VNC `5900`, stack CDP relay `9334` (unit Chrome on `9333`),
library Chrome `9222`. `LiveStack.endpoints()` probes each one, and the
doctor reports units + ports (`python -m x5browser.doctor`). Separate helper
processes on the same host (e.g. `agent_step.py`) must see the same display:
`export DISPLAY=:99` in their shell (the fallback sets it only in-process).

### Remote viewing (not on the same machine)

Links say `127.0.0.1` because the gate binds loopback. From anywhere else,
the official path is an **SSH tunnel** (nothing is exposed):

```bash
ssh -L 8790:127.0.0.1:8790 -L 6080:127.0.0.1:6080 user@your-vps
# then open the printed control/preview link locally, unchanged
```

When the viewer really must be reached directly (reverse proxy / public
host), mint the links with that address so they open as printed:

```bash
X5_LIVE_PUBLIC_URL=http://203.0.113.7:8790/view python scripts/run_ui.py \
    --host 0.0.0.0 --public-base http://203.0.113.7:8790/view
```

Security note: binding the gate to `0.0.0.0` also publishes the
unauthenticated state endpoints (`/api/state`, `/api/events` carry the live
links). Only the token gate (`/view?token=…`) is authenticated — so prefer
the SSH tunnel, or put your own auth (and TLS) in front of a public gate.

---

## 3. Tool catalog (42 tools, 9 groups)

`X5Browser.tools()` returns the default 25-tool surface; `tools(full=True)`
returns all 42. Model descriptions are concise professional English.
All interaction is genuine input events (`Input.*`); there is no JS-drive tool.

### Perception (read-only)

| Tool | What it does | Example |
|---|---|---|
| `page_snapshot` | Numbered map of interactive elements (modals, iframes). Start here | `page_snapshot(scope="viewport")` |
| `read_text` | Visible page/element text | `read_text(ref=5)` |
| `read_markdown` | Full page/element text as organized Markdown (headings, lists, tables, links, form labels); uncapped by default | `read_markdown()` |
| `quick_search` | Live Google search as JSON `{title, url, snippet, page}` (up to 20 pages, hidden tabs, closed after) | `quick_search(query="...", pages=3)` |
| `fetch_pages` | 1–5 URLs as clean Markdown (scrolls to bottom, filters nav/cookie noise), failures per URL | `fetch_pages(urls=[...])` |
| `find_elements` | Find by name/role, returns refs directly | `find_elements(query="checkout")` |
| `inspect_element` | Deep diagnosis: attributes, state, box, covering element, HTML | `inspect_element(ref=12)` |
| `screenshot` | Viewport/element/full-page capture; `annotate` draws ref numbers | `screenshot(annotate=true)` |
| `extract_data` | Structured JSON: table, list, links, forms, images, meta | `extract_data(kind="table")` |
| `extract_image` | Original image bytes by ref or URL (QR codes, photos), returned without saving; find URLs via `extract_data(kind="images")` or `image_search` | `extract_image(src="/qr.png")` |
| `image_search` | Google Images from a query: **random, link-verified** images as JSON (default 5, max 20) with `url` (original), `title`, `width`, `height`, `type`, `bytes`, `source_page`, `thumbnail`; nothing is written to disk | `image_search(query="red panda", count=5)` |
| `get_page_info` | Tab state: URL, title, control, viewport, scroll, frames, focus | `get_page_info()` |

### Navigation

| Tool | What it does | Example |
|---|---|---|
| `navigate` | Open URL / back / forward / reload; replies with fresh map | `navigate(to="https://example.com")` |
| `wait_for` | Wait for text, element, URL, network idle, download, seconds | `wait_for(condition="text", value="Done")` |

### Interaction (real input + fresh stable map in every reply)

| Tool | What it does | Example |
|---|---|---|
| `click` | Real click; follows real hrefs directly; absorbs ad hijacks (auto-clean + one re-click) | `click(ref=7)` |
| `click_verify` | Click plus checks in one call | `click_verify(ref=7, checks=[...])` |
| `fill` | Type into a field; optional Enter | `fill(ref=8, text="hi", submit=true)` |
| `fill_form` | Fill typed fields + optional submit (needs approval) | `fill_form(fields=[{ref:8,value:"hi"}])` |
| `press_keys` | Key/shortcut, optional focus + repeat | `press_keys(keys="Ctrl+A")` |
| `scroll` | Wheel / to element / in container / to edge | `scroll(direction="down", amount=600)` |
| `hover` | Gradual hover; replies with appeared elements | `hover(ref=6)` |
| `hover_click` | Hover menu open + click item, one call | `hover_click(menu_ref=6, target_ref=9)` |
| `choose_option` | Dropdown value (native/custom), verified | `choose_option(ref=13, label="Egypt")` |
| `drag_drop` | Gradual drag between refs/coordinates | `drag_drop(from_ref=3, to_ref=9)` |
| `upload_files` | Attach workspace files (needs approval) | `upload_files(ref=20, paths=["id.png"])` |

### Tabs / events / downloads

| Tool | What it does | Example |
|---|---|---|
| `manage_tabs` | List, open, switch, close tabs | `manage_tabs(action="list")` |
| `get_events` | New tabs, dialogs, permissions, downloads, crashes | `get_events()` |
| `answer_popup` | Answer dialog/permission/file-chooser (do first) | `answer_popup(kind="dialog", accept=true)` |
| `manage_downloads` | List / wait / cancel downloads | `manage_downloads(action="list")` |

### DevTools / task

| Tool | What it does | Example |
|---|---|---|
| `read_console` | Last 500 console messages, filterable | `read_console(level="error")` |
| `console_command` | One JS expression, read value (diagnosis only; needs `allow_unsafe_js=True`) | `console_command(script="document.title")` |
| `list_requests` | Always-on network log; `id` gives full detail | `list_requests(url_contains="api")` |
| `export_page` | Save PDF/MHTML/HTML (needs approval) | `export_page(format="pdf", path="p.pdf")` |
| `verify_state` | Pass/fail checks, cheaper than a snapshot | `verify_state(checks=[...])` |
| `batch_actions` | Several tools in one call, refs remap | `batch_actions(actions=[...])` |
| `manage_triggers` | List / start / update / stop page-monitor triggers | `manage_triggers(action="list")` |

### Lifecycle / handoff

| Tool | What it does | Example |
|---|---|---|
| `browser_status` | Report running/closed + uptime + links; never starts the browser | `browser_status()` |
| `open_browser` | Start explicitly; **refuses** if it already runs | `open_browser()` |
| `close_browser` | Quiet close, data kept; **refuses** if already closed | `close_browser()` |
| `restart_browser` | Restart on the same profile; new link pair, old links die | `restart_browser()` |
| `human_control` | Hand the live browser to the user (non-blocking) | `human_control(reason="pay")` |
| `take_control` / `agent_control` | Hand control over / resume agent | `take_control()` |
| `live_preview` | Watch-only link + JSON | `live_preview()` |

**Lifecycle contract.** The browser is never closed automatically by age:
only `close_browser` (or the library's own `stop`/`shutdown`) closes it. Any
tool auto-opens the browser when it is closed, and every session mints one
control link and one preview link that stay identical until the browser
closes or restarts. Nothing starts the browser on its own at boot.

**Ref rules:** refs come from `page_snapshot`/`find_elements` and die on page
change (`ref_not_found` → re-snapshot). Optional `pre_wait`/`post_wait`
(0–30s) on every tool, and an optional `reason` (≤500 chars) where you can note
why you called the tool; it is echoed back in the reply.

---

## 4. Search, fetch, triggers

**`quick_search`** reads live Google result pages and returns organized JSON
only — no map, no picture, your tabs and refs untouched. Pages spread over up
to 5 hidden tabs at once; inside one tab the first page opens directly and
every next page is reached like a user: scroll to the bottom, then a real
click on Google's Next button (genuine `Input.*` events, no injected JS).

**`fetch_pages`** really visits 1–5 URLs (hidden tab each, concurrently),
scrolls each page to its true bottom for lazy content, then returns clean
Markdown: nav bars, cookie banners, footers and duplicate chrome are
filtered, failures come back per URL.

**`image_search`** turns a text query into pictures. The
search reads live Google Images pages in one hidden tab that is closed right
after (your tab, refs and map stay untouched) and answers a small JSON set:
**5 by default, 20 at the most per call** — ask for less or more with
`count`, and a bigger request is refused with a clear message. The pick is
**random**, so the same query asked twice returns different pictures, and
every image is **link-checked first**: the server must really serve image
bytes, otherwise the candidate is skipped, so a dead link or a non-image page
never reaches the model. Each record describes the picture: `url` (the
original full-size image), `title`, `width`,
`height`, `type`, `bytes`, `source_page`, `source_host`, `thumbnail`. When
Google has nothing, the answer is a clear `no_images` message telling the
model to simplify or fix the query. **Nothing is ever saved**: the answer is
JSON of links and facts; when the bytes themselves are needed, `extract_image`
returns them without writing a file.

**Triggers** watch a site on their own cadence while the agent works. Each
trigger owns a **private tab** (invisible to the agent: hidden from tab
lists, counts and events) on its URL, checks every `check_interval_s`
(default 5s, 1–30), optionally reloading before each check, and fires once on
the first matching condition — then dies and its tab closes:

```python
await browser.manage_triggers("start", url="https://example.com/",
    conditions=[{"kind": "text_present", "value": "sold out"}],
    check_interval_s=5.0, refresh_each_cycle=True, timeout_s=3600)
await browser.manage_triggers("list")     # every running trigger + recent
await browser.manage_triggers("update", trigger_id="t1",
    add_conditions=[{"kind": "text_gone", "value": "loading"}])
await browser.manage_triggers("stop", trigger_id="t1")
```

One trigger fires one webhook to the configured URL (`trigger_webhook_url`).

---

## 5. Ad-defense shield (automatic, no page JS)

A layer between Chrome and the agent. Ads are received normally on the wire
and cancelled at network/tab/map/display layers, so sites see ordinary HTTP.

| Line | What it does | Env |
|---|---|---|
| `NET` | Block/stub ad requests via `Fetch.*` (incl. empty-VAST for pre-roll) | `DEFENSE_NET=1` |
| `TABS` | Close ad popups + settle blank ones safely, restore hijacked tabs (≤5), same-tab href bypass + sacrifice re-click in `click` | `DEFENSE_TABS=1` |
| `TREE` | Hide ad iframes/labels from the agent map | on with `NET` |
| `HEURISTIC` | Ad paths, affiliate redirects, rotating CDNs, brand creatives, tracking-param strip | `DEFENSE_HEURISTIC=1` |
| `SELFHEAL` | One calm reload with the harshest layer off per broken site | `DEFENSE_SELFHEAL=1` |
| `SHADOW` | Inspector stylesheet (CDP CSS, no script): ads render invisibly, clicks pass through | `DEFENSE_SHADOW=1` |

`COSMETIC/STUBS/OVERLAY/MEDIA` stay **off by design** (they need injected page
script). `DNS/EXT` are deployment-time (see `x5browser/defense/policy.json`).

```python
print(browser.defense_report())   # per-line counters: the impact meter
```

**Allowlist** (`x5browser/defense/allowlist.yaml`, editable): OAuth/SSO,
payments + bank 3DS, captchas, embeds, fonts, CDNs are never touched — with
built-in fallbacks even if the file is corrupt. Google services
(`google.com`, `gstatic.com`, `googleapis.com`, `googleusercontent.com`) are
listed so Google's own windows and sign-in open normally. Extra hosts:
```yaml
extra_hosts:
  - my-sso.example.com
```

**Popups:** a popup that opens blank (sign-in windows commit to their host a
moment later) is settled instead of closed at creation: an allowlisted landing
or a landing on the opener's own site — popup flows finish on their own domain
(Firebase `/__/auth/handler`, SSO callbacks) — stays, anything else closes
exactly like a gesture-less popup would.

**Blocklists:** built-in core + `defense/lists/custom.txt` (hand-kept) +
optional full lists (231k hosts):
```bash
python scripts/update_adlists.py   # refresh lists/*.txt at build time
python scripts/measure_defense.py  # blocked counts per line
```

## 6. Configuration

```python
BrowserConfig(
    profile_dir="...",      # default: inside the package (persistent)
    downloads_dir="...",     # default: x5browser/downloads/ (in-package)
    debug_port=9222,         # loopback only, never public
    window_width=1280, window_height=800,
    headless=False,          # headed so live shows everything (True on servers)
    confirm_dangerous=True,  # approval gates for pay/delete/send/…
    allow_unsafe_js=False,   # enable console_command explicitly
    extra_chrome_args=(),    # appended verbatim
)
```

## 7. Same-browser handoff cycle

`agent → human_control → user → agent_control → agent`, same Chrome, same
profile, same tabs and cookies. No new session, no Done button. While the user
holds control, agent tools refuse with `control_not_agent`.

## 8. Security notes

- CDP binds `127.0.0.1` only; uploads/downloads confined to the workspace.
- Passwords and card numbers are masked (`••••`) in maps, logs, screenshots.
- No Playwright/Puppeteer/Selenium; no JS-driven interaction (`Input.*` only).
- Secrets: copy `.env.example` → `.env` (never committed).

## 9. Changelog

**1.6.0** — the release cut of everything below: `1.5.3` and `1.5.4` were
development versions that never reached PyPI, so this version ships them
together — image search that can be trusted, bounded self-heal overrides,
sign-in popups that survive, closing the active tab, a close that really
closes, and the browser-lifecycle fixes (a refused tool call no longer starts
Chrome, the launcher never adopts a browser on someone else's profile, and a
repeated `image_search` query keeps answering fresh pictures). No code change
beyond the version bump: full detail is in the 1.5.4 entry below.

**1.5.4** — image search that can be trusted: `image_search(query, count=5)`
returns a small **random, link-verified** set of Google Images as JSON
(5 by default, 20 maximum, bigger requests refused with a message; every
candidate is checked the way the bytes are later read — one plain bounded GET
that must answer 200 with real image magic **and** readable dimensions of at
least 64 px — so dead links, hotlink blocks, non-image pages and tracking
placeholders are dropped and each record carries real metadata: `url`,
`title`, `width`, `height`, `type`, `bytes`, `source_page`, `source_host`,
`thumbnail`; `width`/`height` are the measured ones whenever the header was
read, and `bytes` is what the server declared at verification time) and a
clear `no_images` answer when a query finds nothing.
**Fetched images are never written to disk** — the answer is JSON only, and
`extract_image` returns bytes when they are needed (the earlier saving tool
was removed). Self-heal overrides are written once per host/step (the file
used to grow a duplicate key on every run; it was also merged back to 11
lines). Sign-in popups fixed on both paths: a blank popup gets a bounded
settle window, a same-site or allowlisted popup is kept, and a kept popup
window that had opened as its own browser window is no longer closed by the
legacy window step (it used to die ~1.5s after the guard said keep, see
`TabGuard.window_survives`), so `signInWithPopup` flows (Google/Firebase)
survive while popunder defence is unchanged. Also fixed: closing the active
tab used to close it and then raise `not_started` instead of returning a
result (destroy-event race). Closing the browser now really closes it: Chrome
left running by an earlier session has no process handle in this process, so
the quiet close did nothing while answering success (measured: 11 Chrome
processes and the CDP endpoint still alive after `close_browser`, and the next
open showed the old page). The close path now attaches once and asks Chrome
itself (`Browser.close`) before the process kill, checks that the endpoint is
gone, and answers `browser_close_failed` instead of a false success — so a
close/restart really restarts and lands on `chrome://newtab/`. Two more holes
were found while verifying the same report ("the profile browser keeps
reopening when I close it"): any tool call started Chrome *before* validating
its arguments, so even a refused call — and the test that guards bad
arguments — launched a real browser (the guard test leaked a 10-process Chrome
tree holding port 9222, which is why a new window appeared on every later
run); input checks now run before the browser is ensured (`_early_refusal`),
and a refused call leaves no profile and no process behind. And the launcher
used to adopt whatever answered on the debug port, so the library could drive
a browser on someone else's profile: `start()` / `read_debugger_url()` now
refuse a foreign endpoint with `port_in_use`, and `is_alive()` means *this*
profile's browser (the command line is matched against the profile directory
and port). `image_search` additionally keeps a bounded memory of the URLs it
already answered (`HISTORY_PER_QUERY=200`, `HISTORY_QUERIES=20`), so the same
query keeps returning fresh pictures (measured: `cat` three times → 15
different images, zero overlap) and only falls back to already-answered ones —
reported as `repeated` — once the pool for that query runs dry.

**1.5.2** — WSL fix: `LiveStack.units_installed()` probes `/etc/systemd/system`
inside the distro via `_run` instead of the Windows disk (no more false
`units_not_installed` + visible-Chrome fallback); handoff cleanup:
`take_control`/`agent_control` removed from the agent catalog (internal
UI/recovery only), `human_control` is the single agent handoff tool.

**1.5.1** — Linux hardening round: live units self-install on every start
path + no-systemd process fallback (same ports); public link base
(`X5_LIVE_PUBLIC_URL`) with SSH-tunnel remote-viewing docs; CDP session
self-heal (`repair_session`, honest `attached|detached|closed` state);
broken-install vs missing Chrome diagnosis; probed live endpoints
(+ doctor); Python floor lowered to 3.10. Refactor: single
`_attach_browser()` owner, failed trial opens close after themselves.

**1.5.0** — official Chrome assurance (original only, silent official install
with one prompt, manual steps last, `--no-sandbox` for root); setup doctor
(`python -m x5browser.doctor`) + automatic first-start environment gate;
`quick_search` (live Google JSON, real Next clicks, up to 20 pages over 5
tabs); `fetch_pages` (real visit + scroll + noise filter); triggers on
private hidden tabs with OR conditions, per-trigger cadence and the
`manage_triggers` agent tool.

## License

MIT — see `LICENSE`.
