Metadata-Version: 2.4
Name: je_web_runner_dev
Version: 0.0.101
Summary: WEB Automation Framework
Author-email: JE-Chen <jechenmailman@gmail.com>
Project-URL: Homepage, https://github.com/Integration-Automation/WebRunner
Project-URL: Documentation, https://webrunner.readthedocs.io/en/latest/
Project-URL: Code, https://github.com/Integration-Automation/WebRunner
Classifier: Programming Language :: Python :: 3.10
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Environment :: Win32 (MS Windows)
Classifier: Environment :: MacOS X
Classifier: Environment :: X11 Applications
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: selenium>=4.0.0
Requires-Dist: requests>=2.33.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: webdriver-manager>=4.0.0
Requires-Dist: defusedxml>=0.7.1
Requires-Dist: Pillow>=12.3.0
Requires-Dist: je_action_core>=0.0.3
Provides-Extra: autocontrol
Requires-Dist: je_auto_control>=0.0.225; extra == "autocontrol"
Dynamic: license-file

# WebRunner

<p align="center">
  <strong>Cross-platform web automation: Selenium + Playwright, plus a JSON-driven action executor with batteries included.</strong>
</p>

<p align="center">
  <a href="https://pypi.org/project/je-web-runner/"><img src="https://img.shields.io/pypi/v/je_web_runner" alt="PyPI Version"></a>
  <a href="https://pypi.org/project/je-web-runner/"><img src="https://img.shields.io/pypi/pyversions/je_web_runner" alt="Python Version"></a>
  <a href="https://github.com/Integration-Automation/WebRunner/blob/main/LICENSE"><img src="https://img.shields.io/github/license/Integration-Automation/WebRunner" alt="License"></a>
  <a href="https://webrunner.readthedocs.io/en/latest/"><img src="https://readthedocs.org/projects/webrunner/badge/?version=latest" alt="Documentation Status"></a>
</p>

<p align="center">
  <a href="README/README_zh-TW.md">繁體中文</a> |
  <a href="README/README_zh-CN.md">简体中文</a>
</p>

---

WebRunner (`je_web_runner`) started as a Selenium wrapper and grew into a full automation platform: a Selenium and a Playwright backend behind one JSON-driven action executor, plus modules for reporting, observability, orchestration, security, and AI assistance. Every executor command has a deterministic name (`WR_*`) and a single dispatch point, so an action JSON can mix browser, HTTP, database, and webhook calls in the same script.

> **Auto-generated reference** — every registered `WR_*` command (signature + summary) is exported under [`docs/reference/command_reference.md`](docs/reference/command_reference.md), and a JSON Schema for action JSON files lives at [`docs/reference/webrunner-action-schema.json`](docs/reference/webrunner-action-schema.json).

## Table of Contents

- [Highlights](#highlights)
- [Installation](#installation)
- [Architecture](#architecture)
  - [System overview](#system-overview)
  - [Action lifecycle](#action-lifecycle)
  - [Backend dispatch](#backend-dispatch)
  - [Module map](#module-map)
- [Quick Start](#quick-start)
- [Public API & Deprecation Policy](#public-api--deprecation-policy)
- [Core API](#core-api)
- [Action Executor](#action-executor)
- [Backends](#backends)
  - [Selenium (default)](#selenium-default)
  - [Playwright](#playwright)
  - [Cloud Grid](#cloud-grid)
  - [Appium (mobile)](#appium-mobile)
- [Reports](#reports)
- [Observability](#observability)
- [Test Orchestration](#test-orchestration)
- [Quality & Security](#quality--security)
- [More Capabilities](#more-capabilities)
- [Specialized Modules](#specialized-modules)
- [Advanced WebDriverWrapper](#advanced-webdriverwrapper)
- [Browser Internals](#browser-internals)
- [Test Data](#test-data)
- [Auth & APIs](#auth--apis)
- [Recorder](#recorder)
- [CI / Integrations](#ci--integrations)
- [Desktop control (AutoControl)](#desktop-control-autocontrol)
- [AI Assistance](#ai-assistance)
- [CLI Usage](#cli-usage)
- [Test Record](#test-record)
- [Exception Handling](#exception-handling)
- [Logging](#logging)
- [Supported Browsers](#supported-browsers)
- [Supported Platforms](#supported-platforms)
- [License](#license)

## Highlights

- **Two backends, one executor.** Selenium is the default; the Playwright backend covers most of the same operations under `WR_pw_*` and is fully opt-in. A comparison table lists the equivalent commands and what exists on one backend only.
- **Action JSON as a contract.** Every command resolves through `Executor.event_dict`; legacy aliases stay alongside snake_case names for back-compat, and a JSON Schema is exported for IDE autocomplete.
- **Reports in five formats.** HTML, JSON, XML, JUnit XML (CI-native), and Allure result files; a single manifest binds every output for downstream globs.
- **Orchestration built in.** Tag filters, dependency declarations with topological order, ledger-backed re-run-only-failed, flaky detection, A/B run mode, multi-user matrix, deterministic sharding, watch mode, and a stdlib scheduler.
- **Observability without extra plumbing.** Auto-screenshot on failure, retry policy, OpenTelemetry hook, live HTTP dashboard, replay studio (HTML timeline), HAR capture + diff.
- **Quality & security guards.** Action linter, migration helper, hard-coded secrets scanner, HTTP security headers audit, axe-core accessibility audit, Lighthouse runner, perf metrics (FCP/LCP/CLS), visual regression, snapshot testing, network throttling, arbitrary-script gate.
- **Browser internals.** Raw CDP, console + network event capture, localStorage / sessionStorage / IndexedDB, service worker / cache control, Shadow DOM piercing, multi-iframe, file upload / download, browser extension loaders.
- **Advanced WebDriverWrapper surface.** `set_driver(experimental_options=, extension_paths=, enable_bidi=, install_driver=)`, `attach_to_existing_browser`, native CDP shortcuts (`set_timezone` / `set_locale` / `set_device_metrics` / `set_user_agent` / `set_extra_http_headers` / `set_geolocation` / `set_network_conditions` / `block_urls` / `set_cache_disabled` / `set_download_directory`), Fetch interception primitives (`enable_fetch_interception` / `continue_request` / `fulfill_request` / `fail_request`), W3C BiDi listeners (`add_console_listener` / `add_js_error_listener`), `save_cookies` / `load_cookies` for session reuse, `save_full_page_screenshot`, `print_page` (PDF), `reload(ignore_cache)`, `bring_to_front`, `switch_to_window_by_url|title`, page metadata getters (`get_current_url` / `get_title` / `get_page_source` / `get_window_handles` / `new_window` / `close_window`). All exposed via `WR_*` aliases too.
- **Standalone CDP / BiDi modules.** Background `CDPEventListener` (WebSocket loop + sync `send` / `on` / context manager), `record_trace(driver, path)` for Chrome DevTools-loadable performance traces, and a `bidi_network` module wrapping `driver.network.add_request_handler` / `add_response_handler` / `add_auth_handler` for cross-browser request interception.
- **Test data & fixtures.** Faker integration, factory pattern, testcontainers (Postgres / Redis / generic), per-environment `.env` loader with `${ENV.X}` placeholder expansion, CSV/JSON data-driven runner with `${ROW.x}`.
- **Auth, API, DB.** OAuth2 / OIDC client-credentials / password / refresh-token flows with token cache, HTTP API testing commands with JSON assertions, SQLAlchemy-backed database validation.
- **Integrations.** TCP socket server with token + TLS, BrowserStack / Sauce Labs / LambdaTest cloud Grid, Appium mobile, JIRA + TestRail, Slack / generic webhook notifier, GitHub Actions inline annotations, Locust load testing.
- **AI hooks.** Pluggable LLM callable powers self-healing locators and natural-language → action JSON drafts.
- **Cross-platform & multi-browser.** Windows, macOS, Linux, Raspberry Pi · Chrome, Chromium, Firefox, Edge, IE, Safari · Chromium, Firefox, WebKit (Playwright).

## Installation

**Stable:**

```bash
pip install je_web_runner
```

**Development:**

```bash
pip install je_web_runner_dev
```

`je_web_runner_dev` follows the `dev` branch: CI publishes a new version each time a push to `dev`
passes the tests and changes what the package ships.

**Optional dependencies** (each enables a slice of features; install only what you use):

```bash
pip install playwright           # Playwright backend
python -m playwright install     # Browser binaries for Playwright
pip install Pillow               # Visual regression
pip install faker                # Random test data (WR_faker_*)
pip install sqlalchemy           # Database validation (WR_db_*)
pip install opentelemetry-sdk    # Distributed traces (WR_set_action_span_factory)
pip install Appium-Python-Client # Mobile native (WR_appium_*)
pip install testcontainers       # Spin up Postgres / Redis (WR_tc_*)
pip install locust               # Load testing (WR_locust_*)
```

Hard requirements: Python **3.10+**, `selenium>=4.0.0`, `requests`, `python-dotenv`, `webdriver-manager`, `defusedxml`, `Pillow`, `je_action_core` (the action executor's shared core).

## Architecture

### System overview

```mermaid
flowchart LR
  subgraph Authoring
    A1["Action JSON files"]
    A2["Programmatic Python API"]
    A3["Browser recorder<br/>(JS injection)"]
    A4["LLM NL → action draft"]
  end

  subgraph Core
    EXE["Executor<br/>event_dict"]
    REC["Test record<br/>singleton"]
    LDG["Run ledger /<br/>flaky detection"]
  end

  subgraph Backends
    SEL["Selenium<br/>WebDriverWrapper"]
    PW["Playwright<br/>PlaywrightWrapper"]
    APM["Appium<br/>Mobile"]
    HTTP["HTTP API<br/>requests"]
    DB["Database<br/>SQLAlchemy"]
  end

  subgraph Outputs
    REP["Reports<br/>HTML/JSON/XML/JUnit/Allure"]
    OBS["Observability<br/>OTel · dashboard · replay"]
    NOT["Notifiers<br/>Slack · webhook · GH · JIRA · TestRail"]
  end

  A1 --> EXE
  A2 --> EXE
  A3 --> A1
  A4 --> A1
  EXE --> SEL
  EXE --> PW
  EXE --> APM
  EXE --> HTTP
  EXE --> DB
  SEL --> REC
  PW --> REC
  APM --> REC
  HTTP --> REC
  DB --> REC
  REC --> LDG
  REC --> REP
  REC --> OBS
  REC --> NOT
```

### Action lifecycle

```mermaid
flowchart LR
  IN["Action<br/>[cmd, args, kwargs]"] --> VAL["JSON validator<br/>(WR_validate_*)"]
  VAL --> ENV["${ENV.X} / ${ROW.x}<br/>placeholder expansion"]
  ENV --> SPAN["OTel span factory<br/>(optional)"]
  SPAN --> RETRY["Retry policy<br/>retries × backoff"]
  RETRY --> GATE["Arbitrary-script<br/>gate"]
  GATE --> DISP["event_dict[cmd](*args, **kwargs)"]
  DISP --> RECORD["test_record_instance<br/>append()"]
  DISP -- failure --> SHOT["Auto-screenshot<br/>(failure dir)"]
  RECORD --> DONE["Result dict"]
  SHOT --> DONE
```

### Backend dispatch

```mermaid
flowchart TB
  CMD["Action command name"] --> ROUTE{"prefix?"}
  ROUTE -- "WR_pw_*" --> PW["Playwright backend<br/>(PlaywrightWrapper)"]
  ROUTE -- "WR_pw_element_*" --> PWE["Playwright element<br/>(PlaywrightElementWrapper)"]
  ROUTE -- "WR_appium_*" --> APM["Appium driver"]
  ROUTE -- "WR_http_*" --> HTTP["requests wrapper"]
  ROUTE -- "WR_db_*" --> DB["SQLAlchemy validator"]
  ROUTE -- "WR_pw_a11y_* / WR_a11y_*" --> AXE["axe-core audit"]
  ROUTE -- "WR_pw_throttle / WR_throttle" --> THR["Network throttling<br/>(CDP)"]
  ROUTE -- "WR_pw_route_*" --> ROUTE_MOCK["Playwright route mock"]
  ROUTE -- "WR_*<br/>(default)" --> SEL["Selenium backend<br/>(WebDriverWrapper)"]
  ROUTE -- "WR_element_*<br/>(default)" --> SE["Selenium element<br/>(WebElementWrapper)"]
```

### Module map

Every `utils/` subpackage with a one-line summary, grouped into the core engine, the
`je_web_runner.api` facade themes and the rest: [`docs/reference/utils_index.md`](docs/reference/utils_index.md)
(generated by `scripts/gen_utils_index.py`).

```
je_web_runner/
├── __init__.py
├── __main__.py                    # CLI: --execute_dir / --watch / --tag / --shard / --migrate ...
├── element/web_element_wrapper.py
├── manager/webrunner_manager.py
├── webdriver/
│   ├── webdriver_wrapper.py             # Selenium core
│   ├── webdriver_with_options.py
│   ├── playwright_wrapper.py            # Playwright sync backend (full)
│   ├── playwright_element_wrapper.py
│   └── playwright_locator.py            # TestObject ↔ Playwright selector
└── utils/
    ├── ab_run/                  # A/B run mode (run_ab + diff_records)
    ├── accessibility/           # axe-core audit
    ├── ai_assist/               # Pluggable LLM scaffold
    ├── api/                     # HTTP API testing commands
    ├── appium_integration/      # Mobile native via Appium
    ├── auth/                    # OAuth2 / OIDC token helpers
    ├── callback/                # Callback executor
    ├── cdp/                     # Raw CDP passthrough
    ├── ci_annotations/          # GitHub Actions ::error::
    ├── cli/                     # CLI parser, watch mode, dispatch
    ├── cloud_grid/              # BrowserStack / Sauce Labs / LambdaTest
    ├── dashboard/               # Live progress HTTP server
    ├── database/                # SQL validation (SQLAlchemy)
    ├── data_driven/             # CSV/JSON dataset + ${ROW.x}
    ├── docs/                    # Auto-generated command reference
    ├── dom_traversal/           # Shadow DOM / iframe helpers
    ├── env_config/              # .env loader + ${ENV.X}
    ├── exception/               # Exception hierarchy
    ├── executor/                # Action executor + retry/screenshot/gate
    ├── extensions/              # Browser extension loaders
    ├── factories/               # Factory pattern helpers
    ├── file_process/            # File utilities
    ├── file_transfer/           # Upload / download helpers
    ├── generate_report/         # HTML/JSON/XML/JUnit/Allure + manifest
    ├── har_diff/                # HAR file diff
    ├── json/                    # JSON I/O + validator (length 1/2/3)
    ├── lighthouse/              # Lighthouse CLI runner
    ├── linter/                  # action_linter + migration
    ├── load_test/               # Locust wrapper
    ├── logging/                 # Rotating file handler
    ├── multi_user/              # Multi-user matrix runner
    ├── network_emulation/       # Throttling presets via CDP
    ├── notifier/                # Slack / generic webhooks
    ├── observability/           # Console+network capture · OTel
    ├── package_manager/         # Dynamic plugin loader
    ├── perf_metrics/            # FCP / LCP / CLS / TTFB
    ├── pom_generator/           # POM skeleton from URL/HTML
    ├── project/                 # Project template generator
    ├── recorder/                # JS-injection recorder + PII mask
    ├── replay_studio/           # HTML timeline studio
    ├── run_ledger/              # ledger · flaky · classifier
    ├── schema/                  # Action JSON Schema export
    ├── scheduler/               # stdlib-sched scheduled runner
    ├── secrets_scanner/         # Hard-coded credential scanner
    ├── security_headers/        # HTTP headers audit
    ├── selenium_utils_wrapper/  # Keys / Capabilities
    ├── self_healing/            # Fallback locator registry
    ├── service_worker/          # SW unregister + cache clear
    ├── sharding/                # Deterministic test sharding
    ├── snapshot/                # Text/DOM snapshot testing
    ├── socket_server/           # TCP server with token + TLS
    ├── storage/                 # localStorage / session / IDB
    ├── test_data/               # Faker integration
    ├── test_filter/             # Tag filter + dependency graph
    ├── test_management/         # JIRA + TestRail
    ├── test_object/             # TestObject + record
    ├── test_record/             # Action recording
    ├── testcontainers_integration/   # Postgres / Redis / generic
    ├── visual_regression/       # Pillow-based image diff
    └── xml/                     # XML utilities
```

## Cookbook

The `examples/` directory ships runnable recipes that exercise the new
helpers against real Chrome / network. Each is invoked from the repo root:

| Example | Demonstrates |
|---|---|
| `counting_stars.{py,json}` | `WR_sleep`, `WR_set_driver` with Chrome flags, autoplay-policy override, JS-driven `video.play()`, skip-ad polling. |
| `google_search.py` | Consent dismissal, search-box typing, ENTER submit, result heading scrape. |
| `form_submit.py` | `form_autofill.plan_fill_actions` + `state_diff.capture_state` round trip against `httpbin/forms/post`. |
| `smart_wait_demo.py` | `wait_for_fetch_idle` + `wait_for_spa_route_stable` + `memory_leak.detect_growth` against a real page. |
| `fanout_demo.py` | `fanout.run_fan_out` parallel HTTP preflights. |
| `pii_redact_demo.py` | `pii_scanner.scan_text` + `redact_text` + `assert_no_pii` (pure logic). |
| `quick_smoke.json` | Minimal `WR_set_driver` → `WR_sleep` → `WR_execute_script` → `WR_quit_all` smoke via the executor CLI. |

Run a Python example directly:

```bash
python examples/google_search.py
```

Run an action JSON example through the executor:

```bash
python -m je_web_runner -e examples/quick_smoke.json
```

## Test Tiers

```
test/
├── unit_test/         # mock-based unit tests
├── integration_test/  # wired-modules tests with real I/O
└── e2e_test/          # real-browser tests; skips without Selenium Grid
```

- **Unit** (`test/unit_test/test_*.py`) — runs everywhere; pulled in by both
  `test_dev.yml` and `test_stable.yml`.
- **Integration** (`test/integration_test/`) — wires 2+ modules together
  with real SQLite, in-process HTTP servers, and real subprocesses for
  the MCP / LSP. Same workflows as unit, second step.
- **E2E** (`test/e2e_test/`) — talks to a Selenium Grid via
  `WEBRUNNER_E2E_HUB`. Locally: `cd docker && docker compose up -d`.
  CI: `.github/workflows/e2e_browser.yml` boots `selenium/hub:4.20.0` +
  `selenium/node-chrome` daily / on demand.

## Thematic API Façade

The utility helpers live under `je_web_runner.utils.<area>`; every subpackage outside the core
engine is also re-exported by exactly one theme under `je_web_runner.api` (a name two packages of
one theme share is prefixed with its package, e.g. `cors_matrix_classify`):

```python
from je_web_runner.api import (
    authoring,      # action_formatter, md_authoring, templates, sel_to_pw, bootstrap
    debugging,      # cross_browser, pr_comment, extension_harness
    frontend,       # device emulation, geo/locale, multi-tab, shadow pierce, …
    infra,          # driver pin, k8s runner, pipeline, lock, watch_mode, …
    mobile,         # Appium gestures
    networking,     # api_mock, contract_testing, GraphQL, mock services, har_replay
    observability,  # timeline, failure bundle, trace recorder, OTLP, BiDi, cdp_tap
    quality,        # a11y_diff, a11y_trend, perf budgets/drift, trend, failure cluster
    reliability,    # adaptive retry, browser pool, smart wait, throttler, supervisor
    security,       # PII, license, CSP, cookie consent, header tampering
    test_data,      # DB fixtures, fixture record/replay, form auto-fill
    accessibility_i18n,  # screen reader, forced colours, pseudo-localisation, RTL, OCR, visual AI
    ai,             # LLM assist, auto-repair, edge cases, triage, RAG / tool-call / hallucination checks
    api_testing,    # HTTP, auth, GraphQL N+1, gRPC, OpenAPI drift, rate limits, HAR tools
    audit,          # clickjacking, CORS, cookies, CSP, XSS taint, open redirect, TLS, SRI, secrets, SBOM
    diagnostics,    # OpenTelemetry bridge, backend log correlation, console error budget
    governance,     # flakes, quarantine, owners, test debt, ROI, cost, PR risk, merge gates
    messaging,      # email render / deliverability, push, OTP, webhooks, message queues, Slack digest
    mobile_pwa,     # touch gestures, viewport, virtual keyboard, pull-to-refresh, real-device clouds
    orchestration,  # A/B runs, personas, multi-user, flag matrix, chaos, data-driven, DB snapshots
    performance,    # web vitals, INP, Lighthouse, bundle / third-party budgets, load tests
    platform,       # CDP, BiDi, Chrome profiles, extensions, cloud grid, AutoControl bridge, CLI
    tooling,        # recorder, action schema, command docs, POM generator, replay studio, dashboards
    web_platform,   # WebSocket / SSE / WebRTC, storage, service workers, WebAuthn, WebUSB, payments…
)
```

The original Selenium-flavoured top-level surface (`webdriver_wrapper_instance`,
`execute_action`, `TestObject`, …) is unchanged.

## Quick Start

### Direct API

```python
from je_web_runner import TestObject, get_webdriver_manager, web_element_wrapper

manager = get_webdriver_manager("chrome")
manager.webdriver_wrapper.to_url("https://www.google.com")
manager.webdriver_wrapper.implicitly_wait(2)

search_box = TestObject("q", "name")
manager.webdriver_wrapper.find_element(search_box)
web_element_wrapper.click_element()
web_element_wrapper.input_to_element("WebRunner automation")

manager.quit()
```

### JSON action list (modern aliases)

```python
from je_web_runner import execute_action

actions = [
    ["WR_new_driver", {"webdriver_name": "chrome"}],
    ["WR_to_url", {"url": "https://www.google.com"}],
    ["WR_implicitly_wait", {"time_to_wait": 2}],
    ["WR_save_test_object", {"test_object_name": "q", "object_type": "NAME"}],
    ["WR_find_recorded_element", {"element_name": "q"}],
    ["WR_element_click"],
    ["WR_element_input", {"input_value": "WebRunner automation"}],
    ["WR_quit_all"],
]
execute_action(actions)
```

`execute_action` returns a dict keyed `execute: <action>`, one entry per step (a repeated identical action gets `#2`, `#3` … so every outcome is kept); it records a failed step and moves on. To run one action and get its value back, use `execute_one(["WR_get_current_url"])`: it goes through the same command gates, retry policy and failure screenshots, prints nothing, and raises `WebRunnerExecuteException` (with the original error as its cause) when the action fails.

The legacy names (`WR_get_webdriver_manager`, `WR_SaveTestObject`, `WR_quit`, `WR_input_to_element`, …) still work — see [Quality & Security](#quality--security) for the one-shot migration helper.

### Mixed positional + keyword arguments

```python
[
    ["WR_to_url", ["https://example.com"], {"timeout": 30}],
]
```

The validator accepts length-1, length-2 (`[cmd, dict_or_list]`), and length-3 (`[cmd, [positional], {kwargs}]`) actions.

## Public API & Deprecation Policy

**What is public** — these are the only things other code should depend on, and the only things this
policy protects:

| Surface | What it covers |
| --- | --- |
| Top-level names | Everything listed in `je_web_runner.__all__` (`from je_web_runner import …`) |
| CLI | `python -m je_web_runner` flags, including the original `-e/--execute_file`, `-d/--execute_dir` and `--execute_str` (Windows double-encoded JSON included) |
| Action JSON | The `WR_*` command names registered in `executor.event_dict`, and the `webdriver_wrapper` / `meta` top-level keys of an action file |
| Socket server | `start_web_runner_socket_server`, `send_command`, `read_frame`, `encode_frame` and the length-prefixed framing |
| Supported module paths | `je_web_runner.utils.executor.action_executor` (`executor`, `execute_one`), `je_web_runner.utils.logging.loggin_instance` (`web_runner_logger`, `WebRunnerLoggingHandler`), `je_web_runner.webdriver.webdriver_wrapper` (`WebDriverWrapper`, `webdriver_wrapper_instance`, the `_options_dict` / `_webdriver_dict` / `_webdriver_manager_dict` patch targets) |

The three module paths are supported because other repositories already import them: AutoControlGUI's
WebRunner bridge takes `executor` and `execute_one`, Jeffrey_RPA hooks `loggin_instance` by that exact (misspelled)
name and patches the wrapper dictionaries. They are treated as public rather than asked to move.
`test/unit_test/test_public_api.py` fails if any of them disappears.

**What is not public**: every other module under `je_web_runner.utils.`, anything named with a
leading underscore other than the patch targets above, the shape of report files, and the log
format. They can move in any release.

**Retiring something public**

1. The replacement ships first, and the old name keeps working as an alias.
2. The alias raises `DeprecationWarning` naming the replacement, and the release notes say so.
3. The alias stays for at least two further releases before it can be removed, and only a release
   that names the removal may drop it.
4. A name listed in `architecture.md` §6 as a cross-project contract is different: it never changes
   without the consuming repository changing in the same round, and §6 is updated on both sides.

## Core API

The original Selenium-flavoured API remains the canonical entry point for programmatic use. Sections preserved from the original README:

- **WebDriver Manager** — `get_webdriver_manager`, `new_driver`, `change_webdriver`, `close_choose_webdriver`, `quit`.
- **WebDriver Wrapper** — `to_url`, `forward`, `back`, `refresh`, `find_element`, `find_elements`, `implicitly_wait`, `explict_wait` (alias `WR_explicit_wait`), `set_script_timeout`, `set_page_load_timeout`, the full ActionChains-backed mouse/keyboard surface, cookies, `execute_script`, window management, screenshots, frame/window/alert switching, `get_log`.
- **Web Element Wrapper** — `click_element`, `input_to_element`, `clear`, `submit`, `get_attribute`, `get_property`, `get_dom_attribute`, `is_displayed`, `is_enabled`, `is_selected`, `value_of_css_property`, `screenshot`, `change_web_element`, `check_current_web_element`, plus the new `select_by_value` / `select_by_index` / `select_by_visible_text`.
- **TestObject** — `TestObject(name, type)`, `create_test_object`, `get_test_object_type_list` (returns `['ID', 'NAME', 'XPATH', 'CSS_SELECTOR', 'CLASS_NAME', 'TAG_NAME', 'LINK_TEXT', 'PARTIAL_LINK_TEXT']`).

Programmatic examples for each surface are kept identical to the previous edition; see the relevant Sphinx pages under `docs/source/Eng/doc/` for full code snippets.

## Action Executor

The executor maps a string command name to a Python callable. Every backend, integration, and helper registers under `event_dict`.

### Action shapes

```python
["command"]                                    # no args
["command", {"key": "value"}]                  # kwargs
["command", [arg1, arg2]]                      # positional
["command", [arg1], {"key": "value"}]          # positional + kwargs (length 3)
```

### Length-3 example

```python
[
    ["WR_pw_evaluate", ["() => document.title"], {"arg": None}],
]
```

### Pacing actions

`WR_sleep` blocks the executor thread for a given number of seconds — useful when the page needs settle time, when a JS animation needs to finish, or when an example wants to hold the browser open for the user to watch:

```python
[
    ["WR_to_url", {"url": "https://example.com"}],
    ["WR_sleep", {"seconds": 2.5}],
    ["WR_get_screenshot_as_png"],
]
```

Negative or non-numeric `seconds` raise `ValueError`. For pacing inside JavaScript (e.g. waiting on a custom event from the page) use `WR_execute_async_script` with a `setTimeout`-driven callback.

### Top-level shapes

```python
[ ...actions... ]                                                  # bare list

{
  "webdriver_wrapper": [ ...actions... ],
  "meta": {"tags": ["smoke", "fast"], "depends_on": ["login"]}     # optional
}
```

`meta.tags` and `meta.depends_on` are picked up by the CLI for filtering and topological execution.

### Adding custom commands

```python
from je_web_runner import add_command_to_executor

def my_step(name: str) -> None:
    print(f"hello {name}")

add_command_to_executor({"my_command": my_step})
```

### Retry, screenshots, scripts

```python
from je_web_runner.utils.executor.action_executor import executor

executor.set_retry_policy(retries=2, backoff=0.5)             # global retry
executor.set_failure_screenshot_dir("./failures")              # auto PNG on raise
executor.set_allow_arbitrary_script(False)                     # gate WR_execute_script / WR_pw_evaluate / WR_cdp
executor.allow_packages("my_helpers")                         # WR_add_package_to_executor may load these
executor.set_allow_arbitrary_packages(False)                   # and refuses every other package
```

## Backends

### Selenium (default)

Selenium is the original backend. Every legacy command (and its modern alias) routes here unless an explicit `WR_pw_*` / `WR_appium_*` prefix is used.

The Selenium backend has the twins of the Playwright element and page helpers: JSON-usable waits (`WR_wait_for_element` with `present` / `visible` / `clickable` / `hidden`, `WR_wait_for_url`, `WR_wait_for_title`, `WR_wait_for_ready_state`), `WR_find_element_by` / `WR_find_elements_by` for a raw selector, `WR_element_check` / `_uncheck` / `_hover` / `_inner_text` / `_inner_html`, `WR_alert_accept` (with `prompt_text`) / `WR_alert_dismiss` / `WR_alert_text`, and on Chromium `WR_emulate_device` / `WR_list_devices`, `WR_grant_permissions` / `WR_clear_permissions` and `WR_clock_freeze`. Unlike the older Selenium commands these fail their action when they fail.

**The older Selenium commands fail their action too.** Every Selenium wrapper call (`WR_to_url`, `WR_element_click`, `WR_element_assert`, `WR_check_current_webdriver` …) records its error in the test record and then raises it, so a wrong `WR_element_assert` fails. They used to let the action pass; `executor.set_raise_wrapper_errors(False)`, or `["WR_set_raise_wrapper_errors", [false]]` first in an action file, brings that back.

With a driver started with `enable_bidi=True` (Chrome or Firefox) the Selenium backend also has the Playwright event capture, HAR and response mocks over W3C BiDi: `WR_event_capture_start` (`dom_mutations=True` also records DOM changes) / `_stop` / `_clear`, `WR_console_messages`, `WR_network_responses`, `WR_dom_mutations`, `WR_assert_no_console_errors` / `WR_assert_no_5xx` / `WR_assert_no_4xx_or_5xx`, `WR_start_har_recording` / `WR_stop_har_recording(har_path)` (headers, status and timings, no bodies), and `WR_route_mock` / `WR_route_mock_json` / `WR_route_unmock` / `WR_route_clear`. A mock pattern is an exact URL, `*/path` (that path on any host) or a dict of URL parts; mock what the page fetches, not the page itself, whose navigation a mock would block.

### Playwright

The Playwright backend covers most everyday operations of the Selenium wrapper under `WR_pw_*`, but it is not a one-to-one copy. The comparison table in `docs/source/Eng/doc/backends/backends_doc.rst` lists the equivalent commands and how they differ (for example, timeouts in milliseconds instead of seconds, tabs switched by index only), what cannot exist on the other backend (Internet Explorer, Selenium Grid, Appium and OS window geometry on Selenium only; WebKit, the trace viewer and automatic actionability waits on Playwright only), and what is not on the other backend yet.

The page-level shortcuts pass extra Playwright options through (`button`, `modifiers`, `timeout`, `force`, `wait_until` …), page-level actions are added to the test record like Selenium steps, and pages the site opens itself (popups, `window.open`) are tracked so `WR_pw_switch_to_page` can reach them. The commands:

- **Lifecycle / pages / navigation** — `WR_pw_launch`, `WR_pw_quit`, `WR_pw_new_page`, `WR_pw_switch_to_page`, `WR_pw_close_page`, `WR_pw_to_url`, `WR_pw_forward`, `WR_pw_back`, `WR_pw_refresh`, `WR_pw_url`, `WR_pw_title`, `WR_pw_content`.
- **Find** — `WR_pw_find_element`, `WR_pw_find_elements`, `WR_pw_find_element_with_test_object_record`, `WR_pw_find_with_healing`.
- **Page-level shortcuts** — `WR_pw_click`, `WR_pw_dblclick`, `WR_pw_hover`, `WR_pw_fill`, `WR_pw_type_text`, `WR_pw_press`, `WR_pw_check`, `WR_pw_uncheck`, `WR_pw_select_option`, `WR_pw_drag_and_drop`.
- **Element-level (after `WR_pw_find_element_with_test_object_record`)** — `WR_pw_element_click`, `WR_pw_element_dblclick`, `WR_pw_element_fill`, `WR_pw_element_type_text`, `WR_pw_element_press`, `WR_pw_element_check`, `WR_pw_element_uncheck`, `WR_pw_element_select_option`, `WR_pw_element_get_attribute`, `WR_pw_element_inner_text`, `WR_pw_element_inner_html`, `WR_pw_element_is_visible`, `WR_pw_element_is_enabled`, `WR_pw_element_is_checked`, `WR_pw_element_scroll_into_view`, `WR_pw_element_screenshot`, `WR_pw_element_change`.
- **Script / cookies / waits / viewport / mouse / keyboard / frames** — `WR_pw_evaluate`, `WR_pw_get_cookies`, `WR_pw_add_cookies`, `WR_pw_clear_cookies`, `WR_pw_screenshot`, `WR_pw_wait_for_selector`, `WR_pw_wait_for_load_state`, `WR_pw_wait_for_timeout`, `WR_pw_wait_for_url`, `WR_pw_set_viewport_size`, `WR_pw_mouse_*`, `WR_pw_keyboard_*`.
- **Mobile emulation / locale / clock** — `WR_pw_emulate("iPhone 13")`, `WR_pw_set_locale`, `WR_pw_set_timezone`, `WR_pw_clock_install` / `_set_time` / `_run_for`, `WR_pw_set_geolocation`, `WR_pw_grant_permissions`.
- **Context options** — `WR_pw_launch(context_options={...})`, `WR_pw_set_context_options` (any `browser.new_context` option), `WR_pw_set_user_agent`, `WR_pw_set_extra_http_headers`, `WR_pw_save_storage_state`. The wrapper keeps one merged set of context options, so emulation, timezone, locale and HAR recording no longer replace each other; a setting that needs a new context carries cookies, localStorage and the current page's URL over.
- **HAR + route mock** — `WR_pw_start_har_recording`, `WR_pw_stop_har_recording`, `WR_pw_route_mock`, `WR_pw_route_mock_json`, `WR_pw_route_unmock`, `WR_pw_route_clear`.
- **Trace viewer / video** — `WR_pw_tracing_start`, `WR_pw_tracing_save_chunk(path)`, `WR_pw_tracing_stop(path)` (open the zip with `playwright show-trace`); `WR_pw_video_start(video_dir, width, height)` and `WR_pw_video_stop` (returns the video files). While a trace is recording and `executor.set_failure_screenshot_dir(...)` is set, each failed action also saves `<timestamp>_<command>.trace.zip` beside its screenshot.
- **Selenium twins** — `WR_pw_add_init_script`, `WR_pw_block_urls` / `WR_pw_unblock_urls` (`*` wildcards; both survive context rebuilds), `WR_pw_clear_geolocation`, `WR_pw_bring_to_front`, `WR_pw_switch_to_page_by_url` / `_by_title`, `WR_pw_screenshot_base64`, `WR_pw_print_page` (headless Chromium), `WR_pw_scroll` / `_to_top` / `_to_bottom`, `WR_pw_drag_and_drop_offset`. Also `WR_pw_get_cookie` / `WR_pw_delete_cookie`, `WR_pw_element_submit`, `WR_pw_element_value_of_css_property`, and the assertions `WR_pw_check_current_page` / `WR_pw_element_assert`, which fail their action on a mismatch.
- **Frames, user-facing lookup, dialogs** — `WR_pw_switch_to_frame` (nested with a list) / `_to_parent_frame` / `_to_main_frame`: selector commands act in the selected iframe until you switch back or navigate. `WR_pw_find_by(by, value)` finds by `role` (with `name`), `text`, `label`, `placeholder`, `alt_text`, `title` or `test_id` and makes the first match the current element. `WR_pw_set_dialog_policy("accept" | "dismiss", prompt_text)` answers alerts, confirms and prompts on every page; `WR_pw_last_dialog` returns the last one.
- **Attach, profile, downloads, cache** — `WR_pw_connect_over_cdp(endpoint_url)` attaches to a Chromium started with a debugging port (quitting only disconnects); `WR_pw_connect(ws_endpoint)` reaches a Playwright browser server; `WR_pw_launch_persistent(user_data_dir, extension_paths=[...])` opens a profile and loads unpacked extensions. `WR_pw_download(selector, save_to)` clicks and saves the download (to a file, or into a directory under the suggested name). `WR_pw_set_cache_disabled` switches the HTTP cache off and on in every browser.
- **Visual regression, recorder, callbacks** — `WR_pw_visual_capture_baseline` / `WR_pw_visual_compare` and `WR_pw_recorder_start` / `_stop` / `_pull_events` / `_save` do on the Playwright page what their `WR_*` twins do on Selenium, and every `WR_pw_*` command can trigger a callback.
- **Several browsers** — `WR_pw_new_browser(browser, ...)` opens another browser and switches to it (returns its index), `WR_pw_switch_browser(index)`, `WR_pw_close_browser`, `WR_pw_browser_count`; each keeps its own pages and settings, and `WR_pw_quit` closes them all.

Existing scripts can move to Playwright incrementally; `TestObject` records are translated to Playwright selectors automatically (`CSS_SELECTOR` → as-is, `XPATH` → `xpath=…`, `ID` → `#…`, `NAME` → `[name="…"]`, `LINK_TEXT` → `text=…`, `PARTIAL_LINK_TEXT` → `:has-text("…")`).

### Cloud Grid

```python
from je_web_runner.utils.cloud_grid.cloud_drivers import (
    connect_browserstack,
    build_browserstack_capabilities,
)

connect_browserstack(
    username="...",
    access_key="...",
    capabilities=build_browserstack_capabilities(
        browser_name="chrome",
        browser_version="latest",
        os_name="Windows",
        os_version="11",
        project="WebRunner",
        build="ci-2026-04-26",
    ),
)
# All existing WR_* commands now run against the cloud session.
```

`connect_saucelabs` and `connect_lambdatest` follow the same shape.

### Appium (mobile)

```python
from je_web_runner.utils.appium_integration.appium_driver import (
    start_appium_session,
    build_android_caps,
    build_ios_caps,
)

start_appium_session(
    "https://appium.example/wd/hub",
    capabilities=build_android_caps(app="/path/to/app.apk"),
)
# WR_* commands now drive the mobile session.
```

## Reports

```python
from je_web_runner import (
    generate_html_report,
    generate_json_report,
    generate_xml_report,
    generate_junit_xml_report,
    generate_allure_report,
)
from je_web_runner.utils.generate_report.report_manifest import generate_all_reports

# Run every generator + write a manifest binding all outputs:
result = generate_all_reports("run_2026_04_26", allure_dir="allure-results")
print(result["manifest_path"])  # → run_2026_04_26.manifest.json
```

| Format    | Output shape                                             | Spec-driven? |
|-----------|----------------------------------------------------------|--------------|
| JSON      | `<base>_success.json` + `<base>_failure.json`            | split        |
| HTML      | `<base>.html`                                            | single       |
| XML       | `<base>_success.xml` + `<base>_failure.xml`              | split        |
| JUnit XML | `<base>_junit.xml`                                       | single       |
| Allure    | `<allure_dir>/<uuid>-result.json` (× N)                  | directory    |

The manifest captures the actual paths produced — CI globs no longer need to know the per-format conventions.

**Interactive HTML report.** `generate_interactive_html_report(html_name, screenshot_dir=None, har_path=None, a11y_results=None)` (`WR_generate_interactive_html_report`) writes one self-contained `<html_name>.html`, with no external resources and no script; each section expands with `<details>`:

- a timeline of every recorded step, failures open, with the screenshots from `screenshot_dir` placed where they were taken and embedded (the executor's failure screenshots are named by time, other images go by their modification time);
- each failure's full error, plus a unified diff when an assertion compares two values (`a != b`, `expected X but got Y`);
- a network waterfall from a HAR 1.2 file (`WR_pw_start_har_recording` / `WR_start_har_recording` write one), with method, status, duration and URL per request;
- axe-core violation badges by impact and the violations themselves, from the result dict or its JSON file.

Every recorded value is escaped.

## Observability

```python
from je_web_runner import (
    test_record_instance,
    summarise_run,
    notify_run_summary,
)
from je_web_runner.utils.executor.action_executor import executor
from je_web_runner.utils.observability.otel_tracing import install_executor_tracing
from je_web_runner.utils.dashboard.live_dashboard import start_dashboard
from je_web_runner.utils.replay_studio.replay_studio import export_replay_studio

executor.set_failure_screenshot_dir("./failures")
install_executor_tracing("webrunner")                 # one OTel span per action
start_dashboard("127.0.0.1", 8080)                    # browser-friendly progress UI
test_record_instance.set_record_enable(True)

# … run actions …

export_replay_studio("./run.html", screenshot_dir="./failures")
notify_run_summary("https://hooks.slack.com/services/...")
```

Failure screenshot, OpenTelemetry tracing, retry policy, and the live dashboard all hook into the same `Executor.event_dict` so they compose without coupling.

There are two dashboards. `start_dashboard` above follows the current run live. `DashboardServer` (`utils/live_dashboard`) aggregates history from files other modules write:

```python
from je_web_runner import DashboardConfig, DashboardServer

server = DashboardServer(DashboardConfig(
    ledger_path="ledger.json",                 # runs, pass-rate chart, flake scores
    quarantine_path="quarantine.json",
    locator_findings_path="locator_health.json",
    schedule_path="schedule.json",             # test_scheduler output
    triage_report_path="triage.json",          # failure_triage output
    bind_port=8765,                            # 127.0.0.1 unless bind_host says otherwise
))
print(server.start())                          # http://127.0.0.1:8765, served from a daemon thread
```

- Pages: Overview (summary cards coloured by status, a daily pass-rate chart with failure bars, the latest runs), Runs (`?limit=`, "Show more"), Flake, Quarantine (with links to triage tickets), Locators (weakest locators and self-healing fallback offenders), Schedule and Triage. Any source left as `None` shows an empty state saying which `DashboardConfig` field to set.
- Tables sort by clicking a header and filter as you type (Runs also by result); times show in the browser's time zone. The page refreshes its data every 15 seconds, keeping your filter, and the Pause button stops it. Light and dark follow the system setting.
- The same data is JSON under `/api/summary`, `/api/runs`, `/api/flake`, `/api/quarantine`, `/api/locators`, `/api/schedule`, `/api/triage` and `/api/trend`.
- Standard library only. Every page works without JavaScript; the script and stylesheet are served from the same origin under a strict Content-Security-Policy (`default-src 'self'`), and every value is HTML-escaped.

## Test Orchestration

```bash
# Filter by tag, run in parallel processes, persist a ledger, fail fast on dep breaks.
python -m je_web_runner \
    --execute_dir ./actions \
    --tag smoke,fast \
    --exclude-tag slow \
    --parallel 4 \
    --parallel-mode process \
    --ledger ./.run_ledger.json

# Re-run only the files that failed last time:
python -m je_web_runner --execute_dir ./actions --rerun-failed ./.run_ledger.json

# Watch a directory and re-run on file change:
python -m je_web_runner --execute_dir ./actions --watch ./actions

# Distribute across 4 runners deterministically (per machine):
python -m je_web_runner --execute_dir ./actions --shard 1/4
python -m je_web_runner --execute_dir ./actions --shard 2/4
python -m je_web_runner --execute_dir ./actions --shard 3/4
python -m je_web_runner --execute_dir ./actions --shard 4/4
```

Companion APIs — `WR_run_for_users` (multi-user matrix), `WR_run_ab` (A/B mode), `WR_flakiness_stats`, `WR_classify_failure`, `WR_schedule` + `WR_run_scheduler_for`.

### Async execution

`--parallel-mode async` (or `AsyncExecutor` from Python) runs action files on asyncio: one headless Chromium, and for each file a fresh browser context, so concurrent files act as separate users with their own cookies, storage and tabs. `--parallel N` caps how many run at once. It needs Playwright.

```bash
python -m je_web_runner -d ./user_journeys --parallel 8 --parallel-mode async --ledger ledger.json
```

```python
import asyncio
from je_web_runner.utils.async_executor import AsyncExecutor

async def main():
    async with AsyncExecutor(browser="chromium", headless=True) as runner:
        results = await runner.run_many([alice_actions, bob_actions], concurrency=2)  # [(record, failed), ...]

asyncio.run(main())
```

The `WR_apw_*` commands act in the file's own context: `WR_apw_goto`, `WR_apw_click`, `WR_apw_fill`, `WR_apw_press`, `WR_apw_text`, `WR_apw_title`, `WR_apw_url`, `WR_apw_wait_for_selector`, `WR_apw_assert_text`, `WR_apw_assert_title`, `WR_apw_new_page` / `WR_apw_switch_to_page` / `WR_apw_close_page` (tabs of the same user), `WR_apw_screenshot`, `WR_apw_evaluate` (behind the arbitrary-script gate), `WR_apw_sleep` (pauses that file only), and `WR_apw_websocket_messages` / `WR_apw_wait_for_websocket_message` for streaming pages. Any other `WR_*` command runs in a worker thread with the usual gates, retries and failure screenshots, but it shares the process-wide Selenium and Playwright singletons, as in `--parallel-mode thread`. Outside the async executor a `WR_apw_*` command fails with a message saying where it runs.

## Quality & Security

- **Action linter** — `WR_lint_action` / `WR_lint_action_file` flag legacy command names, hard-coded URLs, dangerous scripts, missing tags, duplicate consecutive actions.
- **Migration helper** — `python -m je_web_runner --migrate ./actions` rewrites the eleven legacy aliases to their preferred names (`--migrate-dry-run` reports without writing).
- **Hard-coded secrets scanner** — `WR_scan_secrets_file` / `WR_assert_no_secrets` catch AWS / GitHub / Slack / JWT / Google / private-key strings before they land in commits.
- **Security headers audit** — `WR_audit_security_headers_url` checks HSTS / CSP / X-Frame-Options / X-Content-Type-Options / Referrer-Policy / Permissions-Policy.
- **Accessibility audit** — `WR_a11y_run_audit` injects user-supplied axe-core (`load_axe_source`) and runs against the active session; Playwright variant `WR_pw_a11y_run_audit`.
- **Lighthouse** — `WR_lighthouse_run` shells out to the official `lighthouse` Node CLI; `WR_lighthouse_assert_scores` enforces budgets.
- **Page perf metrics** — `WR_perf_collect` / `WR_pw_perf_collect` snapshot FCP / LCP / CLS / TTFB / domContentLoaded / load via `PerformanceObserver`, with the LCP element (`lcp_element`, `lcp_url`) and the top layout-shift sources (`cls_sources`); `WR_perf_assert_within` checks thresholds.
- **Core Web Vitals budget** — `WR_assert_web_vitals(budgets=None, observe_ms=1000, source="page", url=None, json_path=None, html_path=None)` asserts LCP, CLS, INP and FCP in one step. `source="page"` measures the running backend's current page (Selenium, else Playwright), with INP from the event-timing log (interactions of 104 ms or more before the call still count; INP is not measured when there were none). `source="lighthouse"` runs Lighthouse on `url` and reads its lab audits, with Total Blocking Time (`tbt`) instead of INP. Budgets default to Google's "good" limits (LCP 2500 ms, CLS 0.1, INP 200 ms, FCP 1800 ms); each metric is rated good / needs-improvement / poor. A breach fails the action naming the metric, its rating and the diagnostics (LCP element, shifting elements, slowest interaction); `json_path` / `html_path` write the breakdown, pass or fail.
- **Visual regression** — `WR_visual_capture_baseline` + `WR_visual_compare` (Pillow soft-dep).
- **Snapshot testing** — `WR_match_snapshot` / `WR_update_snapshot` (text/DOM, unified diff on mismatch).
- **Network throttling** — `WR_throttle("slow_3g")` / `WR_pw_throttle("offline")`; presets cover Slow 3G, Fast 3G, Regular 4G, Wi-Fi, Offline, no-throttling.
- **HAR diff** — `WR_diff_har` / `WR_diff_har_files` show added / removed / status-changed requests between two runs.
- **Arbitrary-script gate** — `executor.set_allow_arbitrary_script(False)` blocks `WR_execute_script` / `WR_execute_async_script` / `WR_pw_evaluate` / `WR_cdp` / `WR_pw_cdp` for untrusted action JSON.
- **Package gate** — `WR_add_package_to_executor` imports a Python package and registers its functions as commands, so an action file that can name `os` or `subprocess` can run anything. `executor.allow_packages("name", …)` lists the packages it may load (submodules included) and every other package is refused before it is imported, unless the host calls `executor.set_allow_arbitrary_packages(True)`; neither is an action command, so an action file cannot open its own gate.

## Extended Capabilities

Reliability & flake reduction:

- **Adaptive retry** — `je_web_runner.utils.adaptive_retry.run_with_retry(fn, policy=...)` replays only failures the classifier marks transient / flaky / environment; real bugs short-circuit.
- **Locator strength scorer** — `linter.locator_strength.score_locator(strategy, value)` ranks locators 0–100; `assert_strength` fails CI on fragile XPath / TAG_NAME picks.
- **Smart wait** — `smart_wait.wait_for_fetch_idle` and `wait_for_spa_route_stable` patch `window.fetch` and `history.pushState` to detect SPA quiescence — no more `time.sleep`.
- **Service throttler** — `throttler.throttle("payments-api")` is a file-semaphore that caps cross-shard concurrency on a shared service.

Debugging & observability:

- **Timeline merger** — `observability.timeline.build(spans=, console=, responses=)` merges OTel spans, console messages, and network responses into one chronologically-sorted event list.
- **Failure bundle** — `failure_bundle.FailureBundle("login_test", error_repr).add_screenshot(...).write("bundle.zip")` packages screenshots / DOM / network / console / trace into a single replayable zip with manifest.
- **Memory leak detector** — `memory_leak.detect_growth(driver, action, iterations=10, growth_bytes_per_iter_budget=...)` polls `performance.memory.usedJSHeapSize` and fails on linear-fit growth above budget.
- **Playwright trace recorder** — `trace_recorder.TraceRecorder(output_dir="trace-out").start(context, name); …; .stop(context)` always writes a `.zip` viewable with `playwright show-trace`.
- **CSP reporter** — `csp_reporter.CspViolationCollector` injects a `securitypolicyviolation` listener and exposes `assert_none()` / `assert_no_directive("script-src")`.

Test data & determinism:

- **Record/replay fixture** — `snapshot.fixture_record.FixtureRecorder("fx.json", mode="auto")` saves the producer's output the first time, replays it forever after.
- **DB fixture loader** — `database.fixtures.load_fixture_file("seed.json")` + `load_into_connection(conn, fixture)` seeds testcontainers Postgres / MySQL / SQLite from a `{table: [rows]}` JSON.

API & contract testing:

- **API mocking** — `api_mock.MockRouter().add("GET", "/api/users/*", body={"id": 1}).attach_to_page(page)` intercepts Playwright routes; URL globs and `re:` regex patterns supported.
- **Contract testing** — `contract_testing.validate_response(body, schema)` runs a JSON-Schema subset; `validate_against_openapi(body, doc, "/users/{id}", "GET", 200)` resolves `$ref` and checks the right schema for the response status.
- **GraphQL helper** — `graphql.GraphQLClient("https://api/graphql").execute("{ me { id } }")`; `extract_field(payload, "me.id")` plucks values via dotted path.
- **In-process mock services** — `mock_services.MockOAuthServer().start()` issues fake bearer tokens, `MockSmtpServer` captures sent mails, `MockS3Storage` is a memory KV.

Security probes:

- **Header tampering** — `header_tampering.HeaderTampering().set_header("X-Forwarded-For", "192.0.2.1").attach_to_page(page)` mutates outbound requests so testers can probe missing-CSRF / wrong-origin / stripped-auth handling.
- **License scanner** — `license_scanner.scan_text(bundle_text)` finds SPDX identifiers and known license phrases (AGPL/GPL/MIT/Apache-2.0/MPL/ISC/BSD) so SBOM gates can `assert_allowed_licenses`.

Browser & locale:

- **Device emulation presets** — `device_emulation.playwright_kwargs("iPhone 15 Pro")` and `apply_to_chrome_options(opts, "Desktop 1080p")`; viewport + DPR + UA + touch in one call.
- **Geo / TZ / locale** — `geo_locale.GeoOverride(latitude=51.5, longitude=-0.13, timezone="Europe/London", locale="en-GB")` produces both CDP commands and Playwright `new_context` kwargs.
- **Multi-tab choreographer** — `multi_tab.TabChoreographer().open_new(driver, "side", url=...)` registers tabs by alias so action JSON can `WR_switch_tab("side")`.
- **WebAuthn virtual authenticator** — `webauthn.enable_virtual_authenticator(driver)` uses CDP `WebAuthn.*` to simulate passkey / FIDO2 sign-in flows.
- **Cookie consent dismisser** — `cookie_consent.ConsentDismisser().dismiss(driver)` clicks the first matching OneTrust / TrustArc / Cookiebot / Didomi / Quantcast button; selector list extensible via `register_selector`.

Reporting & CI:

- **PR comment poster** — `pr_comment.post_or_update_comment("owner/repo", 42, body, token=...)` is idempotent via a hidden HTML marker so retried CI runs don't pile up.
- **Trend dashboard** — `trend_dashboard.compute_trend("ledger.json")` buckets the ledger by day; `render_html(trend)` produces a self-contained SVG line chart + table.

Orchestration & developer experience:

- **Action template library** — `action_templates.render_template("login_basic", {...})` substitutes `{{placeholders}}` in built-in flows (login, accept-cookies, switch-locale, close-modal).
- **Diff-aware shard** — `sharding.diff_shard.select_for_changed(candidates, base_ref="main")` filters candidates to those touched by the current branch's `git diff`.
- **Watch mode** — `watch_mode.watch_loop(directory, on_change=callback, interval=0.5)` re-runs a callback whenever JSON files change.
- **Kubernetes runner** — `k8s_runner.render_job_manifests(ShardJobConfig(name_prefix="run", image=..., total_shards=8, actions_dir="/actions"))` produces one `batch/v1 Job` per shard.
- **Per-route perf budgets** — `perf_metrics.budgets.evaluate_metrics("/checkout", {"lcp_ms": 2300}, budgets)` plus `assert_within_budget(result)` enforce route-specific thresholds.

AI assistance:

- **Failure RCA** — `ai_assist.llm_assist.explain_failure(test_name, error_repr, console=, network=, steps=)` asks the registered LLM for `{likely_cause, evidence, next_steps, confidence}`.

## MCP Server

WebRunner ships a [Model Context Protocol](https://modelcontextprotocol.io/) server so any MCP-aware client (Claude, IDE plugins, etc.) can drive WebRunner over JSON-RPC stdio.

```bash
python -m je_web_runner.mcp_server
```

The default tool list (22 tools) exposes:

Live browser execution:
- `webrunner_run_actions` — execute any `WR_*` action list. Covers all 556 `WR_*` commands, including the advanced WebDriverWrapper additions: `WR_attach_to_existing_browser`, `WR_execute_cdp_cmd`, `WR_set_timezone` / `_locale` / `_device_metrics` / `_user_agent` / `_extra_http_headers` / `_geolocation` / `_network_conditions`, `WR_block_urls` / `_set_cache_disabled` / `_set_download_directory`, `WR_save_cookies` / `_load_cookies` / `_clear_origin_storage`, `WR_save_full_page_screenshot` / `_print_page`, `WR_reload(ignore_cache=True)`, `WR_bring_to_front`, `WR_switch_to_window_by_url|title`, `WR_new_window` / `_close_window`, page metadata getters, Fetch interception primitives, `WR_add_script_to_evaluate_on_new_document`, …
- `webrunner_run_action_files` — batch-run JSON files on disk
- `webrunner_list_commands` — discover the full `WR_*` surface

Plus the utility tools below (no live browser):

Action JSON authoring & linting:
- `webrunner_lint_action`, `webrunner_score_action_locators`, `webrunner_locator_strength`
- `webrunner_format_actions`, `webrunner_parse_markdown`, `webrunner_render_template`
- `webrunner_translate_actions_to_playwright`, `webrunner_translate_python_to_playwright`

Code generation:
- `webrunner_pom_from_html`

Quality / triage:
- `webrunner_a11y_diff`, `webrunner_cluster_failures`, `webrunner_compute_trend`

Security & privacy:
- `webrunner_scan_pii`, `webrunner_redact_pii`

Reporting & contract:
- `webrunner_summary_markdown`, `webrunner_validate_response`

Sharding & infra:
- `webrunner_diff_shard`, `webrunner_render_k8s`, `webrunner_partition_shard`

```python
from je_web_runner.mcp_server import McpServer, Tool, build_default_tools, serve_stdio

# Or build a custom server
server = McpServer()
for tool in build_default_tools():
    server.register(tool)
server.register(Tool(
    name="my_custom_tool",
    description="…",
    input_schema={"type": "object", "properties": {"x": {"type": "string"}}},
    handler=lambda args: f"hello {args['x']}",
))
serve_stdio(server=server)
```

The server speaks both MCP eras, decided per request:

- `2026-07-28` (stateless): a request whose `params._meta` names `io.modelcontextprotocol/protocolVersion` (and carries `io.modelcontextprotocol/clientCapabilities`) is served on its own, with no `initialize`. Methods: `server/discover` (supported versions, capabilities, `instructions`), `tools/list`, `tools/call`. Results carry `resultType`, the server's identity in `_meta`, and `ttlMs` / `cacheScope` where cacheable; another version in `_meta` gets error `-32022` listing the supported ones.
- `2025-11-25`, `2025-06-18`, `2025-03-26` and `2024-11-05` (handshake): `initialize` answers with the client's version when it is one of these, otherwise with `2025-11-25`, and also returns the package version and short `instructions` for the model. Methods: `initialize`, `notifications/initialized`, `tools/list`, `tools/call`, `ping`.

- stdio is UTF-8 with `\n` line ends whatever the console code page. Anything else the process writes to stdout (`print`, child processes) goes to stderr, so only protocol messages reach the client.
- A tool that fails returns a result with `isError: true` and the error text, so the client can correct its call. `webrunner_run_actions` and `webrunner_run_action_files` also set `isError` when an action failed, and list the failed record keys under `failed`.
- An unknown tool or non-object `arguments` is a JSON-RPC `-32602` error. Notifications are never answered, and batch requests are rejected with `-32600`.
- The run tools refuse `WR_add_package_to_executor`, `WR_add_package_to_callback_executor`, `WR_set_allow_arbitrary_script` and the `WR_ac_*` commands that drive or read the desktop (all but `WR_ac_available` and `WR_ac_list_commands`), including inside nested action lists: a model must not load `os`, re-open a script gate the operator closed, or reach the desktop through AutoControl. `WEBRUNNER_MCP_ALLOW_UNSAFE_COMMANDS=1` lifts this. When `WEBRUNNER_MCP_ROOT` is set, `webrunner_run_action_files` and `webrunner_compute_trend` only read files inside that directory.
- Every tool has a `title` and `annotations`: the offline tools and `webrunner_list_commands` are `readOnlyHint: true`, the two run tools `destructiveHint: true` and `openWorldHint: true`. Input schemas describe each argument and reject unknown ones; arguments that do not match come back as an `isError` result naming the problem. A result that is a JSON object is also sent as `structuredContent`, and the two run tools declare an `outputSchema`.

## Action JSON LSP

A standard Language Server Protocol implementation for action JSON files:

```bash
python -m je_web_runner.action_lsp
```

`textDocument/completion` returns every registered `WR_*` command; `textDocument/publishDiagnostics` runs the action linter on `didOpen` / `didChange`. Pair with VS Code's *Configure JSON Language Servers* or the JetBrains LSP plugin.

## More Capabilities

Smaller modules, grouped by what they help with. The larger areas have their own sections above and in
[Specialized Modules](#specialized-modules).

### CLI & orchestration polish

- **Regex test selector** — `test_filter.name_filter.filter_paths(paths, include=["smoke.*"], exclude=["slow"])` keeps only matching candidate paths; orthogonal to the existing tag filter.
- **Process supervisor** — `process_supervisor.ProcessSupervisor().kill_orphans()` walks the OS process table for `chromedriver` / `geckodriver` / `msedgedriver` and kills stragglers (skips `os.getpid()` automatically). `with_watchdog(callable, timeout_seconds=300)` wraps a long callable with a hard wall-clock raise.
- **Pipeline DSL** — `pipeline.load_pipeline({"stages": [...]})` + `run_pipeline(pipeline, runner)` execute multi-stage gates: `continue_on_failure=True` makes a stage non-blocking (linters / scanners), otherwise downstream stages skip.

### Frontend / mobile / coverage

- **Storybook visual snapshots** — `storybook.visual_snapshots.capture_story_snapshots(stories, base_url, take_screenshot, navigate, baseline_dir=...)` walks every story, persists deterministic filenames (`components-button--primary.png`), and diffs against an optional baseline. `assert_no_visual_regressions(report)` is the gate.
- **Appium gestures** — `appium_integration.gestures` ships `swipe`, `scroll`, `long_press`, `pinch`, `double_tap` that prefer Appium's `mobile:` named-gesture extension and fall back to W3C Actions on older drivers.
- **Coverage map** — `coverage_map.build_coverage_map("./actions")` walks every action JSON file, normalises `WR_to_url` paths (`/users/42` → `/users/:id`) and produces a route → files reverse index. `coverage.uncovered(declared_routes)` answers "which routes have no test?".

### Debugging & reproducibility

- **CDP message tap** — `cdp_tap.CdpRecorder("cdp.ndjson").attach(driver)` wraps `execute_cdp_cmd` so every command + return value is appended to an ndjson log; `CdpReplayer(load_recording(...))` plays it back against a stub for offline debugging.
- **Cross-browser parity** — `cross_browser.diff_runs([chromium_run, firefox_run, webkit_run])` diffs title / DOM hash / console / network status / screenshot hash, classifying each finding as `major` (5xx, title, DOM mismatch) or `minor`. `assert_parity(report, only_major=True)` is the gate.
- **Browser state diff** — `state_diff.capture_state(driver)` snapshots cookies + localStorage + sessionStorage; `diff_states(before, after)` lists added / removed / changed keys per section so cart / auth flows stay traceable.

### Authoring / scaffolding

- **Page Object codegen** — `pom_codegen.discover_elements_from_html(html)` walks every element with `data-testid` / `id` / form `name`; `render_pom_module(elements, class_name="LoginPage")` returns a Python module with one `TestObject` property per element.

### CI reproducibility

- **Workspace lock file** — `workspace_lock.build_lock(drivers=..., playwright_versions={"chromium": "127.0.0.0"})` snapshots every Python distribution + driver version + Playwright browser version; `write_lock(lock, ".webrunner/lock.json")` and `diff_locks(before, after)` complete the pipeline.

### Long-running observability

- **A11y trend dashboard** — `a11y_trend.aggregate_history(history)` buckets axe runs by day and impact; `render_html(points)` produces a self-contained SVG line chart so regressions are visible at a glance.
- **Perf drift detector** — `perf_drift.detect_drift({"lcp_ms": samples}, baseline_window=20, recent_window=5)` compares the recent P95 against a rolling baseline P95 and flags drift outside `tolerance`. `assert_no_regression(report)` is the strict path; `higher_is_better={"frame_rate"}` for inverted metrics.

### Authoring / formatting

- **Action JSON formatter** — `action_formatter.format_actions(actions)` writes a canonical multi-line array with kwargs in a stable preferred-then-alphabetical order; `format_file(path)` reformats in place and reports `(text, changed)`.
- **Markdown → action JSON** — `md_authoring.parse_markdown(text)` understands `- open <url>`, `- click #id`, `- type "x" into <selector>`, `- wait 3s`, `- assert title "..."`, `- press Enter`, `- screenshot`, `- run template <name>`, `- quit`. Lines that don't match are preserved as `WR__note` so the round-trip is loss-less.

### Triage / production observability

- **Failure clustering** — `failure_cluster.cluster_failures(failures, top_n=5)` reduces each error message to a stable signature (strips timestamps, hex addresses, line numbers, paths, large numerics, quoted substrings) so the same root cause across runs lands in one bucket.
- **Synthetic monitoring** — `synthetic_monitoring.SyntheticMonitor(alert_sink).register("homepage", check)` reruns checks; the sink only fires on edge transitions (`green → red` / `red → green`) with `failure_threshold` / `recovery_threshold` to silence flapping.
- **OTLP exporter** — `observability.otlp_exporter.configure_otlp_export(provider, OtlpExportConfig(endpoint="https://otlp:4317"))` ships the existing OTel spans to Jaeger / Tempo / any OTLP backend (gRPC by default, HTTP fallback).

### Frontend / component

- **Storybook integration** — `storybook.discover_stories(index_path)` reads Storybook 7+ `index.json` (or legacy `stories.json`); `plan_actions_for_stories(stories, base_url, run_a11y=True)` builds a flat action list visiting each story in iframe mode and running axe + screenshot.
- **Shadow DOM auto-pierce** — `dom_traversal.shadow_pierce.find_first(driver, "button.primary")` recursively walks open shadow roots (Selenium `execute_script` or Playwright `evaluate`) so a single CSS selector can match across shadow boundaries.

### Onboarding / migration

- **Workspace bootstrapper** — `bootstrapper.init_workspace("my-tests")` drops `actions/sample.json`, `.webrunner/ledger.json`, pinned-driver template, JSON schema, pre-commit hook, and a starter GitHub Actions workflow.
- **Driver pinner** — `driver_pin.install_for_browser(".webrunner/drivers.json", "firefox")` reads a JSON pin file (`name` / `version` / `url` / `archive_format` / `binary_inside`), downloads + extracts once, then serves from cache. Bypasses the GitHub API rate limit that webdriver-manager hits in CI.
- **Selenium → Playwright translator** — `sel_to_pw.translate_python_source(text)` rewrites `driver.find_element(By.ID, "x")` → `page.locator("#x")` and similar; `translate_action_list(actions)` rewrites `WR_*` action JSON to its `WR_pw_*` equivalent (drops `WR_implicitly_wait` since Playwright auto-waits).

### Test authoring

- **Form auto-fill** — `form_autofill.plan_fill_actions(fields, fixture, submit_locator=...)` infers each field from `data-testid` / `id` / `name` / `placeholder` / `label` / `type` and emits a ready-to-run `WR_save_test_object` + `WR_element_input` sequence.

### Quality

- **A11y diff** — `accessibility.a11y_diff.diff_violations(baseline, current)` buckets axe-core findings into `added` / `resolved` / `persisting` keyed on `(rule_id, target)`; `assert_no_regressions(diff, allow_rules=...)` is the CI gate.

### Performance / orchestration

- **Fan-out** — `fanout.run_fan_out([("preflight-a", task_a), task_b, ...], max_workers=4)` runs read-only callables concurrently inside one test, returning per-task duration + outcome with `raise_for_failures()` for the strict path.
- **Event bus** — `event_bus.EventBus(".webrunner/events.log").publish("setup-done", {"shard": 1})`; subscribers `poll()` from a remembered offset or `wait_for(topic, predicate=..., timeout=30)`. File-backed ndjson — no Redis dependency.

### Browser internals

- **Extension test harness** — `extension_harness.parse_manifest("./ext")` reads MV2 / MV3 manifests; `apply_to_chrome_options(options, [ext_dir])` adds `--load-extension` flags; `playwright_persistent_context_args(...)` returns the kwargs needed for `launch_persistent_context`.

### Reliability & dev-loop

- **Browser pool** — `browser_pool.BrowserPool(factory, size=4, max_uses=50).warm()`; `with pool.session() as ses: …` removes browser cold-start from local dev. Health check + recycle policy built in.
- **WebDriver BiDi bridge** — `bidi_backend.BidiBridge().subscribe(target, "console", callback)` works against either Selenium 4 BiDi (`driver.script.add_console_message_handler`) or Playwright `page.on(...)`. `register_translator` lets you wire custom event names.

### Determinism & offline runs

- **HAR replay server** — `har_replay.HarReplayServer(load_har("recorded.har")).start()` boots a local HTTP server that serves recorded responses; supports literal / glob / `re:` URL matching with rotation across duplicates. Drop-in for staging-API outages.

### Quality / privacy

- **PII scanner** — `pii_scanner.scan_text(text)` finds emails, E.164 phones, Luhn-validated credit cards, US SSN, ROC ID, and IPv4. `assert_no_pii(text, allow_categories=...)` for CI gates; `redact_text(text)` returns a sanitised copy.
- **Visual diff review UI** — `visual_review.VisualReviewServer(baseline_dir, current_dir).start()` opens a local web UI showing each baseline / current pair side-by-side with an *Accept current as baseline* button (idempotent file copy with path-traversal guard).

### Test orchestration

- **Test impact analysis** — `impact_analysis.build_index("./actions")` walks every action JSON file and projects locator names, URLs, template names, and `WR_*` commands into a reverse index; `affected_action_files(index, locators=["primary_cta"])` answers "which tests touch this?" so diff-aware shards can go beyond filename matching. `build_index("./actions", cache_path=".webrunner_impact_cache.json")` keeps each file's names in a cache and parses only what changed: a file with the same modification time and size is reused, and one whose time changed (a fresh checkout) is reused when its SHA-256 matches. Measured on 3000 files of 30 actions: 510 ms uncached, 138 ms cached.

## Specialized Modules

A second wave of utility modules, each in its own subpackage under
`je_web_runner/utils/`, organised by capability area. Each module is
fully unit-tested and ships independent of the core executor (import
only what you use).

### Web Platform APIs

- **`webtransport_assert`** — HTTP/3 WebTransport datagram + stream
  frame recorder with count / payload / JSON-shape / stream-complete
  assertions (mirror of `websocket_assert` and `sse_assert`).
- **`indexed_db_explorer`** — Browser-side harvest JS + typed
  `IdbSnapshot`; assertions cover store existence, record count, key
  presence, index presence, plus per-store diff.
- **`file_system_access`** — JS shim mocking `showOpenFilePicker` /
  `showSaveFilePicker` / `showDirectoryPicker`; records every write
  performed against the fake handle for later assertion.
- **`notifications_audit`** — Tracks `Notification.requestPermission`
  call timing (user-gesture check, min page age) and policy violations
  (re-prompt after deny, notification spam after deny, tag reuse).
- **`sse_assert`** — Server-Sent Events stream recorder + chunk-buffer
  feed + count / data-contains / JSON-shape / strictly-increasing-id
  assertions.
- **`websocket_assert`** — WebSocket frame recorder + count / payload /
  pubsub-pattern / JSON-shape assertions.
- **`webrtc_assert`** — `PeerSnapshot.from_dict`, `aggregate_stats`
  (getStats), and connected / track-present / SDP-codec / packet-loss /
  min-bytes assertions.
- **`view_transitions`** — Instrumentation snippet for the View
  Transitions API + duration budget / CLS budget / group-name asserts.

### Security & Headers

- **`mixed_content_audit`** — HAR + console-message scan for HTTP
  resources on HTTPS pages (active vs passive vs HSTS-upgrade).
- **`clickjacking_audit`** — X-Frame-Options + `frame-ancestors` parser +
  iframe-probe page generator; STRICT / SAMEORIGIN / ALLOWED / MISSING
  verdict.
- **`open_redirect_detector`** — Eight-payload probe set (`//evil`,
  `@userinfo`, `javascript:`, `data:`, mixed-case bypass…) +
  classifier (BLOCKED / ALLOWED / AMBIGUOUS).
- **`sri_verify`** — Parse `<script>` / `<link rel=stylesheet>` tags →
  validate `integrity=` strength + crossorigin requirement +
  recompute hash from a caller-supplied payload provider.
- **`coop_coep_audit`** — `crossOriginIsolated` page-header check (COOP
  `same-origin` + COEP `require-corp` / `credentialless`) + per-resource
  CORP / CORS validator.
- **`token_leak_detector`** — Scan response bodies / HAR / log lines for
  leaked JWTs (with header validation), AWS / GitHub / Slack / Stripe /
  Google / generic-bearer tokens. Deduped by token suffix.
- **`consent_audit`** — Cookie catalogue (GA / FB pixel / Hotjar /
  LinkedIn / Mixpanel / Stripe / Intercom / CSRF / session) + pre-consent +
  post-reject reintroduction detector.
- **`pii_in_screenshot`** — OCR + PII regex (Luhn-validated card, SSN,
  ROC ID, IBAN, IPv4, phone, email) over screenshots; reuses
  `ocr_assert` for the OCR layer.

### Performance Budgets

- **`inp_tracker`** — Interaction-to-Next-Paint instrumentation +
  p98-INP + good/needs-work/poor rating per Google's thresholds.
- **`hydration_check`** — SSR hydration mismatch detection (DOM diff
  with framework-attr/comment stripping + console-marker scan covering
  React / Vue / Svelte / Astro / Nuxt).
- **`bundle_budget`** — HAR → per-AssetKind transfer totals (script /
  stylesheet / image / font / media) + breach detail + biggest-asset
  ranking.
- **`third_party_budget`** — Vendor catalogue (GA / FB Pixel / Hotjar /
  Intercom / Stripe / Segment / Mixpanel / Amplitude / Sentry / etc.) +
  req / byte / blocking-ms / vendor-count budgets.
- **`long_animation_frame`** — `long-animation-frame` PerformanceObserver
  listener + per-script attribution (forced reflow time, pause time).
- **`console_error_budget`** — JS console / unhandled-rejection budget
  with regex ignore patterns; Selenium and CDP adapters.

### Backend Integration

- **`grpc_tester`** — gRPC stub method wrapper + gRPC-Web framing
  (length-prefix encode/decode + trailer parser) + status assertions.
- **`webhook_receiver`** — Stdlib threaded HTTP server (random port) +
  `wait_for(predicate)` polling + path / header / JSON-predicate
  assertion helpers. Drop-in for "did the app POST a webhook?" tests.
- **`idempotency_check`** — Run a request twice + compare
  status / body / state / side-effect count. `ignore_body_keys`,
  `allow_status_change_to` for legitimate 409-on-second.
- **`pagination_audit`** — Walk all pages via caller-supplied fetcher;
  detects duplicates across pages, cursor-loop, off-by-one totals, and
  sort-order violations.
- **`backend_log_correlator`** — W3C traceparent → fetch matching log
  lines from Loki / Elasticsearch / JSON-lines file → attach to a
  failure bundle.
- **`email_render`** — MailHog / Mailpit / `.eml` capture →
  cross-viewport screenshot via pluggable render driver.

### AI / Workflow

- **`failure_narrator`** — Load failure-bundle directory → LLM-driven
  natural-language "why this failed" summary → strict JSON envelope →
  markdown report. LLM client pluggable.
- **`repro_minimizer`** — Classic delta-debugging (ddmin) shrinking a
  failing action list to its smallest still-failing subsequence.
- **`locator_hardener`** — Heuristic fragility score (nth-of-type /
  text-xpath / hashed-class / deep-descendant) → LLM-suggested stable
  selectors with safety filter on the response.
- **`test_categorizer`** — Regex rules over action-name patterns → auto
  tag: smoke / regression / perf / a11y / security / payment /
  data_driven / visual / api.
- **`exploratory_ai`** — Agentic exploratory tester with `PageObserver` +
  `ActionPlanner` protocols; ships a deterministic `RandomPlanner` as
  fuzz fallback, collects `BugSignal`s from observed errors.
- **`story_to_actions`** — LLM-driven translation of a user story +
  optional Figma frame hints into validated WR action JSON; validator
  rejects unsafe action names and bad locator strategies.
- **`session_to_test`** — rrweb / generic-event-stream → WR action JSON;
  auto-detects input format.
- **`test_auto_repair`** — LLM-driven test rewrite from a failure bundle +
  git diff context.
- **`edge_case_generator`** — LLM edge-case variant generator
  (complement to `mutation_testing`).
- **`multimodal_qa`** — Send screenshot + question to a vision LLM,
  parse pass/fail/uncertain verdict with confidence floor; useful for
  UI "is this correct?" checks beyond pixel diff.
- **`prompt_drift_monitor`** — Track an app-internal LLM feature's
  output drift via baseline embeddings + must_include / must_exclude
  lexical anchors.
- **`test_dedup_ai`** — Structural (canonical fingerprint) + semantic
  (cosine clustering with pluggable embedder) dedupe of action JSON
  files.
- **`walkthrough_docs`** — Generate step-by-step SOP / Confluence-style
  docs from recorded runs.

### a11y / i18n / Visual

- **`ocr_assert`** — OCR-based text assertion (`contains` / `fuzzy` /
  `any`) for canvas / WebGL / image content; whitespace + accent
  normalisation built in.
- **`screen_reader_runner`** — Walk an accessibility tree to simulate
  NVDA / VoiceOver reading order + flag unnamed interactive elements,
  heading-level skips, missing alt, generic link text.
- **`pseudo_localization`** — Pseudo-localise strings
  (`__éxámplé strîng__`) + scan rendered page for hard-coded text
  leaks. Preserves `{name}` / `%d` / `<tag>` placeholders.
- **`forced_colors_mode`** — CDP-features builder for the four CSS
  media queries (color-scheme / reduced-motion / forced-colors /
  contrast) + computed-style diff with "became invisible" detection.
- **`visual_ai`** — aHash / dHash / pHash + SSIM-proxy for canvas /
  chart visual diff.

### Governance & Reporting

- **`pr_risk_score`** — Fuses flake / impact-analysis / locator-health /
  coverage signals into a 0-100 PR risk score with markdown report and
  is_blocking gate.
- **`flag_matrix`** — Feature-flag combination matrix with
  forbid / require constraints, pinned baselines, deterministic
  sampling, and greedy smallest-failing-subset cover.
- **`chaos_hooks`** — Seeded chaos injection (offline / throttle /
  mid-flow reload / tab-background) with deterministic schedule per
  action list.
- **`db_snapshot`** — Per-test DB savepoint/rollback isolation with
  pluggable backend protocol; ships an `InMemoryBackend` for
  unit-testing the workflow itself.
- **`time_freezer`** — CDP injection script that overrides
  `Date` / `Date.now` / `performance.now`; freeze or slow-motion modes
  for deterministic time-bound tests.
- **`persona_runner`** — Same suite × N personas (admin / free /
  enterprise / guest) matrix; summary flags persona-specific vs
  file-specific regressions.
- **`git_bisect_flake`** — Ledger-only or probe-driven bisect for the
  regression commit that caused a test to start failing.
- **`test_cost_estimator`** — Per-runner rate-card (Sauce / BrowserStack /
  LambdaTest / GitHub Actions) × ledger minutes → USD + CO₂ estimate
  per suite / runner / test.
- **`slack_digest`** — Render Slack Block-Kit + Teams Adaptive Card +
  plain-text test digest with quarantine activity, top-risk PRs, cost
  trend, and pass-rate delta.
- **`quarantine_age_report`** — Add fresh / lingering / stale /
  abandoned tier per quarantined test + escalation alerts.
- **`test_debt_dashboard`** — Scan pytest skip / xfail / TODO + JSON
  `_skip` markers + age + CODEOWNERS-derived owner mapping.
- **`sla_tracker`** — Weekly / daily bucketed % of suites finishing
  under an SLA duration threshold + trend.
- **`bug_repro_stability`** — Repeat a failing probe N times → classify
  deterministic / flaky / non-reproducible + error-signature grouping +
  longest pass / fail streak.
- **`test_owners_map`** — CODEOWNERS parser (last-match-wins glob
  semantics) + per-test override layer + unowned-test audit.
- **`failure_triage`** — AI failure root-cause analysis on failure
  bundles.
- **`flake_detector`** — Time-decayed flake scoring + quarantine
  registry.
- **`locator_health`** — Project-wide locator audit + upgrade
  suggestions.
- **`mutation_testing`** — Action JSON mutation testing (kill-rate /
  score).
- **`live_dashboard`** — Aggregated web UI: runs + flake + quarantine +
  locators + schedule + triage (see Observability).
- **`test_scheduler`** — Value-density scheduler under time + cloud
  budget.

### Other Specialised Modules

- **`chrome_profile`** — Persistent Chrome profile + stealth +
  snapshot / sync-back.
- **`device_cloud`** — Real-device cloud (BrowserStack / Sauce /
  LambdaTest) connector.
- **`otel_bridge`** — W3C traceparent injection for distributed
  tracing.
- **`otp_interceptor`** — MailHog / Mailpit / IMAP / SMS OTP polling
  for 2FA flows.
- **`download_verify`** — PDF / CSV / Excel / JSON / SHA256 download
  assertions.
- **`openapi_to_e2e`** — OpenAPI / Swagger spec → `WR_http_*` action
  JSON generator.
- **`cross_tab_sync`** — Multi-page BroadcastChannel / storage
  propagation asserts.

### Modern web platform & runtime APIs

Modules covering newer browser surfaces that are awkward to drive
through plain WebDriver:

- **`popover_assert`** — `<dialog>` / popover open / close / invoker
  / "only one modal" assertions.
- **`cookie_store_api`** — Async `cookieStore` API harvest +
  change-event assertions + secure-only enforcement.
- **`speculation_rules`** — Speculation Rules (`prerender` /
  `prefetch`) verification, prerendering activation, no-double-fire.
- **`web_locks`** — Multi-tab Web Locks contention harness with
  deadlock + serialisation + acquired-count assertions.
- **`storage_buckets`** — Storage Buckets API isolation, durability
  hint, and IDB-per-bucket isolation checks.
- **`hydration_streaming`** — Streaming SSR per-boundary timing
  (arrival, interactive) + order assertions.
- **`web_push_assert`** — Push subscription VAPID key match,
  endpoint allowlist, `userVisibleOnly`, `showNotification` payload.
- **`background_sync_assert`** — Background Sync register / fire /
  retry / `lastChance` (quota-exhaustion) assertions.
- **`wake_lock_assert`** — Screen wake lock acquire / release / leak
  / re-acquire-on-visibility detection.
- **`pip_assert`** — Picture-in-Picture (video + Document PiP)
  enter / exit / size assertions.
- **`web_share_assert`** — `navigator.share` payload recording +
  fallback-UI assertions.
- **`compression_streams`** — `CompressionStream` gzip / deflate /
  brotli round-trip + compression ratio budget.
- **`compute_pressure`** — Compute Pressure API fake observer + app
  throttle-reaction assertions.

### Modern auth, payments, identity

- **`webauthn_mock`** — Deterministic `navigator.credentials` shim
  for Passkey / FIDO2 / WebAuthn flows; build canned credentials by
  user.
- **`credential_management`** — Password / Federated Credential
  Management API mock + autofill / `preventSilentAccess` assertions.
- **`payment_request_assert`** — Payment Request API shim + Apple
  Pay / Google Pay sheet validation (currency, shipping, `complete()`).
- **`three_d_secure_flow`** — 3-D Secure 2.x branch model
  (frictionless / challenge / fallback / reject) + silent-finalize
  detection.

### Mobile-web specific

- **`touch_gesture`** — `tap` / `swipe` / `pinch` / `long_press`
  CDP-frame builder + event assertions.
- **`viewport_audit`** — Viewport meta + safe-area-inset audit +
  WCAG 1.4.4 user-scalable audit.
- **`virtual_keyboard`** — `visualViewport` before / after + keyboard
  inset CSS variable + focused-element visibility.
- **`pull_to_refresh`** — `overscroll-behavior` + threshold + refresh
  handler + network-refetch assertions for PWAs.

### LLM / AI feature testing

- **`rag_grounding_assert`** — RAG citation in retrieved set,
  lexical overlap, unsupported-claim phrase scan.
- **`llm_token_cost_tracker`** — Per-test token / $ ledger with
  per-model rate card + budget assertion.
- **`streaming_chat_assert`** — TTFT / inter-token gap / UTF-8
  cleanliness / duplicate-or-OOS chunk assertions for streaming chat.
- **`tool_call_assert`** — LLM tool / function-call name + ordering +
  JSON Schema argument validation.
- **`hallucination_probe`** — Ground-truth probe runner + refusal
  detection + hallucination-rate budget.

### Email & notification delivery

- **`email_deliverability`** — SPF / DKIM / DMARC headers +
  `List-Unsubscribe` (Gmail/Yahoo bulk rules) + BCC-leak audit.
- **`inbox_render_outlook`** — Outlook (Word renderer) / Gmail /
  Apple Mail render-compatibility pre-flight findings.
- **`push_delivery`** — FCM / APNs payload size + required fields +
  PII scan + collapse key + TTL validation.

### Performance budgets (cont.)

- **`memory_pressure_emulate`** — CDP memory / CPU pressure
  emulation profiles + run-under-profile assertions.
- **`third_party_block_test`** — Vendor-by-vendor block-resilience
  matrix (no-vendor / blocked / passed).
- **`bundle_diff_pr`** — PR bundle delta (added / removed / grew) +
  growth-gate + markdown report.
- **`lcp_image_audit`** — LCP image preloaded + no `loading="lazy"` +
  `fetchpriority="high"` assertions.
- **`font_loading_strategy`** — `@font-face` `font-display` strategy +
  `size-adjust` fallback for FOUT / FOIT / FOFT verification.
- **`resource_hints_audit`** — `preload` / `prefetch` / `preconnect`
  used vs declared + `preload as=` validation.
- **`critical_css_audit`** — Inline-CSS-in-`<head>` budget + render-
  blocking external stylesheet preload audit.
- **`lighthouse_regression`** — Lighthouse score regression vs
  baseline + Core Web Vitals metric budgets.

### Security & headers (cont.)

- **`prompt_injection_scanner`** — LLM jailbreak payload library +
  canary-leak detection.
- **`cors_matrix`** — CORS preflight matrix probe + credentials /
  origin policy assertions.
- **`oauth_pkce_replay`** — Confirm authorization server rejects
  replayed OAuth state / PKCE verifier.
- **`cookie_chips_audit`** — CHIPS Partitioned cookie compliance
  (third-party requires Partitioned + Secure + SameSite=None).
- **`sbom_diff`** — CycloneDX SBOM diff (added / removed / upgrade
  / license / vulnerability gates).
- **`webhook_signature_verify`** — GitHub / Stripe / Slack / generic
  HMAC webhook signature verifier.
- **`dom_xss_taint`** — Lightweight DOM-XSS taint tracking via JS
  instrumentation + canary detection.
- **`csp_violation_parser`** — CSP `report-uri` / `report-to`
  payload parser + recon-attempt heuristic.
- **`hsts_preload_audit`** — HSTS preload-list compliance
  (`max-age` ≥ 1y + `includeSubDomains` + `preload`).
- **`tls_cipher_audit`** — Live TLS handshake + version + cipher
  allowlist + certificate subject check.
- **`cookie_scope_abuse`** — Session-like cookie scope (apex domain
  / `Path=/`) + `HttpOnly` / `Secure` / `SameSite` audit.

### Backend integration (cont.)

- **`graphql_n_plus_1`** — N+1 query detector with per-field SQL
  template repetition + cartesian-fanout heuristic.
- **`mq_assert`** — Kafka / RabbitMQ / SQS-style message-queue
  publish assertions (drain + matcher + idempotency + ordering).
- **`grpc_streaming_assert`** — gRPC streaming (unary / server /
  client / bidi) frame count + size + order + half-close assertions.
- **`openapi_drift`** — Live API vs OpenAPI spec drift (undocumented
  endpoint / method / status, zombie endpoints).
- **`api_version_compat`** — Old-client vs new-server backward-compat
  matrix on response shape + required request fields.
- **`rate_limit_assert`** — 429 + `Retry-After` + `X-RateLimit-*`
  monotonic + recovery-after-wait assertions.
- **`har_to_openapi`** — HAR → OpenAPI 3.1 reverse engineering
  (path templates, query params, response schemas).

### QA governance & DevX (cont.)

- **`failure_auto_tag`** — Heuristic + LLM failure auto-tagger
  (`flaky-locator` / `timeout` / `js-error` / `network-5xx` …).
- **`test_self_describe`** — Reverse-engineer Gherkin
  `Given / When / Then` paragraph from action JSON.
- **`pr_title_generator`** — Conventional-Commits PR title from
  diff + commit history.
- **`action_refactor_suggester`** — Action-JSON refactor smells
  (hard sleep, positional XPath, duplicated locator, click-wait-click).
- **`test_roi_scorer`** — Find-rate × cost × coverage × recency
  weighted ROI score per test.
- **`pre_merge_gate_dsl`** — Declarative `when` / `require` pre-merge
  gate rules over a `PrFacts` snapshot.
- **`commit_msg_trigger`** — Parse `[skip ci]` / `[ci e2e]` /
  `[ci shard=3/8]` / `Closes #123` from commit message.
- **`flakiness_graveyard`** — Quarantine / revive / bury ledger with
  TTL for stale flaky tests.
- **`test_blame_owner`** — CODEOWNERS + git-blame + HEAD + default
  → test-owner resolution chain.
- **`test_dup_dry`** — Structural action-JSON duplicate + prefix-
  overlap detection (extract-helper opportunity).
- **`snapshot_diff_approval`** — Baseline / pending / rejected
  snapshot register + approval workflow.
- **`failure_cluster_dbscan`** — Failure-message tokeniser + DBSCAN
  root-cause clustering (pure-Python, no sklearn).
- **`test_naming_lint`** — `should_when` / `given_when_then` /
  `camel_subject` naming convention linter.

### i18n / a11y (cont.)

- **`rtl_layout_verify`** — RTL direction + logical-property
  (`margin-inline-start`) + bidi-isolation audit.
- **`dst_boundary_test`** — DST spring-forward / fall-back gap &
  overlap detection + scheduled-fire model.
- **`number_currency_locale`** — Number / currency / date locale-
  format assertion helpers (incl. Indian lakh grouping).
- **`wcag22_touch_target`** — WCAG 2.2 SC 2.5.8 target-size auditor
  with spacing-circle exception.

### Emerging-tech device APIs

- **`webgpu_pixel_verify`** — WebGPU canvas pixel readback + mean /
  solid-colour / tile-diff assertions.
- **`webhid_mock`** — WebHID device shim with input / output report
  capture harness.
- **`webusb_mock`** — WebUSB device shim with control / bulk
  transfer capture.
- **`webserial_mock`** — Web Serial UART shim + line-write capture.
- **`webcodecs_assert`** — WebCodecs chunk codec / resolution /
  keyframe-interval / framerate assertions.
- **`speech_api_assert`** — `SpeechSynthesis` / `SpeechRecognition`
  mock + utterance / language / volume assertions.

For per-module reference also see [`CLAUDE.md`](CLAUDE.md), the
auto-generated [`docs/reference/command_reference.md`](docs/reference/command_reference.md),
and the Sphinx chapter under
`docs/source/Eng/doc/specialized_modules/`.

## Advanced WebDriverWrapper

The Selenium wrapper is now composed via mixins under
`je_web_runner/webdriver/_wrapper_mixins/` (lifecycle / element / wait stay in
`webdriver_wrapper.py`; cookies / actions / media / navigation / scripting are
the mixin themes). External imports — `webdriver_wrapper_instance`,
`WebDriverWrapper`, the `_options_dict` / `_webdriver_dict` /
`_webdriver_manager_dict` patch targets — are unchanged.

### Stealth / anti-bot launch

```python
from je_web_runner import webdriver_wrapper_instance

webdriver_wrapper_instance.set_driver(
    "chrome",
    options=[
        "--disable-blink-features=AutomationControlled",
        f"--user-data-dir={profile_dir}",
    ],
    experimental_options={
        "excludeSwitches": ["enable-automation"],
        "useAutomationExtension": False,
    },
    extension_paths=["/path/to/extension.crx"],  # optional
    enable_bidi=True,                            # for add_console_listener etc.
    install_driver=False,                        # skip webdriver-manager; Selenium Manager finds the driver
)
webdriver_wrapper_instance.add_script_to_evaluate_on_new_document(
    "Object.defineProperty(navigator, 'webdriver', {get: () => undefined});"
)
```

### Attach to a manually started Chrome (session reuse)

```python
# Step 1 — user starts Chrome themselves:
#   chrome.exe --remote-debugging-port=9222 --user-data-dir="C:/temp/profile"
webdriver_wrapper_instance.attach_to_existing_browser("127.0.0.1:9222")
```

### CDP shortcuts (emulation, network, downloads)

```python
w = webdriver_wrapper_instance
w.set_timezone("Asia/Tokyo")
w.set_locale("ja-JP")
w.set_device_metrics(390, 844, device_scale_factor=3, mobile=True)
w.set_user_agent("Mozilla/5.0 (custom)")
w.set_extra_http_headers({"X-Test-Run": "ci-123"})
w.set_geolocation(35.68, 139.69, accuracy=50)
w.set_network_conditions(offline=False, latency=200,
                          download_throughput=50_000, upload_throughput=10_000)
w.block_urls(["*.doubleclick.net/*", "*.googletagmanager.com/*"])
w.set_cache_disabled(True)
w.set_download_directory("./downloads")
w.clear_origin_storage("https://example.com")    # cookies + localStorage + IDB + cache
```

### Session persistence

```python
w.to_url("https://example.com/")
# … log in, etc. …
w.save_cookies("./cookies.json")

# Later (after browser restart):
w.to_url("https://example.com/")
added = w.load_cookies("./cookies.json")          # → number of cookies applied
```

### Fetch interception primitives

```python
w.enable_fetch_interception(patterns=["*/api/*"])
# In a Fetch.requestPaused event callback (subscribe via CDPEventListener):
w.fulfill_request(req_id, response_code=200,
                  body=b'{"ok": true}',
                  response_headers={"Content-Type": "application/json"})
# Or:  w.continue_request(req_id, url=rewritten_url)
# Or:  w.fail_request(req_id, error_reason="AccessDenied")
```

### Page metadata + multi-tab navigation

```python
w.get_current_url(); w.get_title(); w.get_page_source()
w.get_window_handles(); w.get_current_window_handle()
w.new_window("tab")
w.switch_to_window_by_url("checkout")    # restores original if no match
w.close_window()                         # vs quit() which terminates the driver
w.reload(ignore_cache=True)              # CDP Page.reload — Ctrl+Shift+R equivalent
w.bring_to_front()
w.save_full_page_screenshot("./shot.png")   # full page, beyond viewport
w.print_page("./page.pdf")
```

### W3C BiDi listeners (Selenium 4.16+)

```python
webdriver_wrapper_instance.set_driver("chrome", enable_bidi=True)
sub_id = webdriver_wrapper_instance.add_console_listener(
    lambda entry: print(entry.text)
)
err_id = webdriver_wrapper_instance.add_js_error_listener(
    lambda err: print("page exception:", err)
)
# …later…
webdriver_wrapper_instance.remove_console_listener(sub_id)
webdriver_wrapper_instance.remove_js_error_listener(err_id)
```

### Background CDP event loop (independent module)

`CDPEventListener` opens its own CDP WebSocket on a worker thread so commands
and events share the same target session — required because Selenium's
`execute_cdp_cmd` cannot subscribe to events.

```python
from je_web_runner import CDPEventListener

with CDPEventListener.from_driver(driver) as listener:
    listener.on("Fetch.requestPaused", handle_paused)
    listener.send("Fetch.enable", {"patterns": [{"urlPattern": "*"}]})
    # … drive the browser …
```

Requires `pip install websocket-client` (lazy-loaded; a clear `CDPEventLoopError`
is raised if missing).

### Performance tracing

```python
from je_web_runner import record_trace

record_trace(
    driver, "perf.json",
    categories=["devtools.timeline", "loading"],
    duration=10.0,
)
# Open perf.json in chrome://tracing or DevTools "Performance".
```

### Cross-browser BiDi network (Selenium 4.16+, Firefox-compatible)

```python
from je_web_runner import (
    bidi_add_request_handler,
    bidi_add_response_handler,
    bidi_clear_network_handlers,
)

sub = bidi_add_request_handler(driver, lambda req: print(req.url))
bidi_clear_network_handlers(driver)
```

Every method above is also reachable from action JSON via `WR_*` aliases
(`WR_set_timezone`, `WR_save_cookies`, `WR_enable_fetch_interception`, …) so
the same surface drives the MCP server too.

## Browser Internals

```python
from je_web_runner import (
    selenium_cdp,                 # raw CDP
    pw_emulate, pw_set_locale,    # mobile / locale
)
from je_web_runner.utils.storage.browser_storage import (
    selenium_local_storage_set,
    selenium_indexed_db_drop,
)
from je_web_runner.utils.observability.event_capture import (
    start_event_capture,
    assert_no_console_errors,
    assert_no_5xx,
)
from je_web_runner.utils.dom_traversal.shadow_iframe import (
    selenium_query_in_shadow,
    playwright_shadow_selector,
    selenium_switch_iframe_chain,
)
from je_web_runner.utils.file_transfer.file_helpers import (
    selenium_upload_file,
    wait_for_download,
)
from je_web_runner.utils.extensions.extension_loader import (
    selenium_chrome_options_with_extension,
    playwright_extension_launch_args,
)
```

Service worker / cache control, console + network event capture and assertions, file upload via element + download dir watcher, browser extension loaders for Chromium-family.

## Test Data

```python
from je_web_runner import (
    load_env, get_env, expand_in_action,                   # .env + ${ENV.X}
    load_dataset_csv, load_dataset_json, run_with_dataset, # data-driven + ${ROW.x}
)
from je_web_runner.utils.test_data.faker_integration import (
    fake_email, fake_name, fake_credit_card, fake_value,   # faker
)
from je_web_runner.utils.factories.factory import user_factory, order_factory
from je_web_runner.utils.testcontainers_integration.containers import (
    start_postgres,
    start_redis,
    cleanup_all,
)
```

Every helper is JSON-callable too (`WR_load_env`, `WR_load_dataset_csv`, `WR_run_with_dataset`, `WR_faker_email`, `WR_user_factory`, `WR_tc_postgres`, …).

## Auth & APIs

```python
from je_web_runner import (
    http_get, http_post, http_assert_status, http_assert_json_contains,
)
from je_web_runner.utils.auth.oauth import (
    client_credentials_token,
    bearer_header,
)
from je_web_runner.utils.database.db_validate import (
    db_query,
    db_assert_count,
    db_assert_value,
)

token = client_credentials_token(
    "https://idp.example/oauth2/token",
    "client-id", "client-secret",
    cache_key="default",
)
http_get("https://api.example/users/me", headers=bearer_header(token["access_token"]))
http_assert_status(200)
http_assert_json_contains("role", "admin")

db_assert_count(
    "postgresql+psycopg://user:pw@host/db",
    "SELECT 1 FROM orders WHERE user_id = :uid",
    expected=1,
    params={"uid": 42},
)
```

OAuth2 helpers cache tokens in-process and refresh 30 seconds before expiry.

## Recorder

```python
from je_web_runner import (
    recorder_start,
    recorder_stop,
    recorder_save_recording,
)

recorder_start(webdriver_wrapper_instance)
# … user clicks / inputs in the browser …
recorder_save_recording(
    webdriver_wrapper_instance,
    output_path="./recorded.json",
    raw_events_path="./raw.json",  # optional — debugging
)
```

The recorder injects a static JS listener (no CDP, no eval), so it works on Chrome / Firefox / Edge alike. **Sensitive fields are masked by default** — `type=password`, names matching `password / card_number / cvv / ssn / secret / token / api_key / otp / passcode`, and 13–19-digit numeric values are replaced with `***MASKED***`.

## CI / Integrations

```python
from je_web_runner.utils.notifier.webhook_notifier import notify_run_summary
from je_web_runner.utils.test_management.jira_client import jira_create_failure_issues
from je_web_runner.utils.test_management.testrail_client import (
    testrail_send_results,
    testrail_results_from_pairs,
)
from je_web_runner.utils.ci_annotations.github_annotations import (
    emit_failure_annotations,
    emit_from_junit_xml,
)
```

For GitHub Actions inline annotations, run `emit_from_junit_xml("run_junit.xml")` after `generate_junit_xml_report` — failed test cases surface as `::error file=…::` lines on the PR diff.

`docker/docker-compose.yml` ships a Selenium Grid 4 stack (hub + Chrome + Firefox nodes); `docker/.env.example` exposes the version pin and concurrency settings.

The IDE config examples under [`docs/ide/`](docs/ide/) wire VS Code and JetBrains to the action JSON schema produced by `WR_export_action_schema`.

## Desktop control (AutoControl)

`pip install je_web_runner[autocontrol]` adds [AutoControl](https://github.com/Integration-Automation/AutoControlGUI) (`je_auto_control`), which drives the real mouse, keyboard and screen. The `WR_ac_*` commands reach it from an action file, and WebRunner imports it only when one of them runs.

| Command | What it does |
| --- | --- |
| `WR_ac_available` | Whether `je_auto_control` is installed (looked up, not imported) |
| `WR_ac_list_commands` | The `AC_*` commands the bridge will run |
| `WR_ac_run` | Run one AutoControl action and return its value: `["WR_ac_run", [["AC_write", {"write_string": "hello"}]]]` |
| `WR_ac_run_actions` | Run a list of AutoControl actions in order and return their values; the first failure stops it |
| `WR_ac_fill_native_file_dialog` | Type a path (made absolute) into the operating system's open or save dialog, then press Enter (`submit=false` skips it); waits `wait_seconds` (1) for the dialog first |
| `WR_ac_assert_image_on_screen` | Fail unless a template image is on the screen; returns its centre `[x, y]` (`detect_threshold` is AutoControl's match threshold) |
| `WR_ac_click_element_native` | Click an element with the real mouse (a trusted click, not WebDriver's): the element `selector` / `by` finds, else the current element; `mouse_button`, and `scale` for the monitor's display scale; returns the clicked `[x, y]` |
| `WR_ac_basic_auth` | Answer the browser's HTTP basic-auth dialog with the credentials in two environment variables (`username_env`, `password_env`: names, never values); with `url` it first opens that page without waiting; then types username, Tab, password and Enter (`submit=false` skips it) after `wait_seconds` (1) |

A failed AutoControl action fails its `WR_ac_*` action with `AutoControlBridgeError`. The bridge refuses the AutoControl commands that reach beyond the desktop: shell commands and programs (`AC_shell_command`, `AC_execute_process`), package loading (`AC_add_package_*`), action lists and files (`AC_execute_action`, `AC_execute_files`), `AC_run_agent`, and calls back into WebRunner (`AC_web_*`). They are refused anywhere in the action, including loop bodies and bodies passed as JSON strings. The MCP server refuses every `WR_ac_*` command but `WR_ac_available` and `WR_ac_list_commands` unless `WEBRUNNER_MCP_ALLOW_UNSAFE_COMMANDS=1`.

`WR_ac_click_element_native` scrolls the element to the middle of the viewport and maps its centre from the page to the screen with the window's `screenX/Y`, its outer and inner size and `devicePixelRatio`. Without `scale` it takes `devicePixelRatio` as the display scale, which is right at 100 % page zoom; on a zoomed page pass the monitor's scale (`1.25` for 125 %). The window must not be covered.

`WR_ac_basic_auth` exists because a classic `get` of a page behind basic auth waits, while the browser's dialog is open, until it times out, and chromedriver does not answer the dialog through BiDi. So pass the page as `url`: the command checks that the page has the keyboard focus (`document.hasFocus()`; AutoControl types into whichever window is in front, so it refuses otherwise), starts the navigation without waiting, then types into the dialog. The credentials go through AutoControl's `AC_write_secret`, which types them exactly and keeps them out of its log, its record and its return value; the action file, WebRunner's log and both run records hold only the variable names. It needs a `je_auto_control` that has `AC_write_secret`. For Playwright, set `http_credentials` with `WR_pw_set_context_options` instead.

The native commands refuse a Selenium driver whose window is not on this machine's screen: a headless browser, or a remote one (a grid or a device cloud). A Playwright browser is not checked, because Playwright does not report whether it was launched headless: launch it with `headless=False`.

The other direction, AutoControl's `AC_web_*` commands running `WR_*` commands, is part of AutoControl.

## AI Assistance

```python
from je_web_runner.utils.ai_assist.llm_assist import (
    set_llm_callable,
    suggest_locator,
    generate_actions_from_prompt,
)

# Plug in any callable that returns a string:
def my_llm(prompt: str) -> str:
    # call OpenAI / Anthropic / local Ollama / mock
    ...

set_llm_callable(my_llm)

locator = suggest_locator(html_blob, description="primary submit button")
draft = generate_actions_from_prompt("log in as alice and place an order")
```

WebRunner intentionally ships **no built-in LLM client** — the boundary is a single `Callable[[str], str]` so swapping provider is one line.

## CLI Usage

```bash
# Original entry points (unchanged):
python -m je_web_runner -e actions.json
python -m je_web_runner -d ./actions/
python -m je_web_runner --execute_str '[["WR_quit_all"]]'

# Newer flags:
python -m je_web_runner -d ./actions --tag smoke --exclude-tag slow
python -m je_web_runner -d ./actions --parallel 4 --parallel-mode process
python -m je_web_runner -d ./actions --ledger ledger.json
python -m je_web_runner -d ./actions --rerun-failed ledger.json
python -m je_web_runner -d ./actions --shard 1/4
python -m je_web_runner -d ./actions --watch ./actions
python -m je_web_runner --report run                          # JSON + HTML + XML + JUnit
python -m je_web_runner --validate ./action_smoke.json
python -m je_web_runner --migrate ./actions --migrate-dry-run
python -m je_web_runner -d ./actions --affected-by locator:login_button --affected-by url:/checkout
python -m je_web_runner -d ./actions --changed-since origin/main --impact-cache .webrunner_impact_cache.json
```

Installing the package also installs two console scripts, `webrunner` and `web_runner`, with the same flags, output and exit codes as `python -m je_web_runner` (`webrunner -e actions.json`, `pipx run --spec je_web_runner webrunner -d ./actions`).

Compose any of the flags above; the dispatcher applies impact selection → tag filters → ledger / re-run-failed → sharding → dependency-aware ordering before handing files to the runner.

**Impact-based selection.** `--affected-by KIND:VALUE` (repeatable; `KIND` is `locator`, `url`, `template` or `command`; `@file` reads one per line) runs only the action files that use one of those names. `--changed-since REF` runs the action files `git diff REF...HEAD` changed, plus every file sharing a locator or template with them, so changing a setup file that saves `login_button` also runs the tests that use it (URLs and commands are not followed: almost every file shares them). Both can be combined; tags, `--rerun-failed` and `--shard` then apply to what they selected. `--impact-cache PATH` keeps the impact index between runs.

## Test Record

```python
from je_web_runner import test_record_instance

test_record_instance.set_record_enable(True)
# … perform automation …
records = test_record_instance.test_record_list
# Each record: {"function_name", "local_param", "time", "program_exception"}
test_record_instance.clean_record()
```

## Exception Handling

WebRunner provides a hierarchy of custom exceptions — every helper raises a domain-specific subclass of `WebRunnerException`:

| Exception                                  | Description                                      |
|--------------------------------------------|--------------------------------------------------|
| `WebRunnerException`                       | Base                                             |
| `WebRunnerWebDriverNotFoundException`      | WebDriver not found                              |
| `WebRunnerOptionsWrongTypeException`       | Invalid options type                             |
| `WebRunnerArgumentWrongTypeException`      | Invalid argument type                            |
| `WebRunnerWebDriverIsNoneException`        | WebDriver is None                                |
| `WebRunnerExecuteException`                | Action execution error                           |
| `WebRunnerJsonException`                   | JSON processing error                            |
| `WebRunnerGenerateJsonReportException`     | JSON / XML / JUnit / Allure report error         |
| `WebRunnerHTMLException`                   | HTML report error                                |
| `WebRunnerAddCommandException`             | Custom command registration error                |
| `WebRunnerAssertException`                 | Assertion failure                                |
| `XMLException` / `XMLTypeException`        | XML processing error                             |
| `CallbackExecutorException`                | Callback execution error                         |
| `PlaywrightBackendError`                   | Playwright backend / element failure             |
| `PlaywrightLocatorError`                   | TestObject → Playwright selector mapping         |
| `RecorderError` / `VisualRegressionError`  | Recorder / visual regression                     |
| `HealingError` / `EnvConfigError` / `DataDrivenError` | Self-healing / env / dataset            |
| `HttpAssertionError` / `HttpError`         | HTTP API assertions                              |
| `AccessibilityError` / `LighthouseError`   | a11y / Lighthouse                                |
| `NotifierError` / `JiraError` / `TestRailError` | Notifications / test management            |
| `CDPError` / `StorageError` / `ServiceWorkerError` | Browser internals                        |
| `OAuthError` / `DatabaseValidationError`   | Auth / DB                                        |
| `NetworkEmulationError` / `LoadTestError`  | Throttling / Locust                              |
| `ShardingError` / `MigrationError` / `ActionLinterError` | Orchestration / linting              |
| `LLMAssistError` / `OTelTracingError`      | AI / observability                               |

## Logging

- **Log file:** `$WEBRUNNER_LOG_PATH` (a full file path), else `$WEBRUNNER_LOG_DIR/WEBRunner.log`, else `~/.je_web_runner/logs/WEBRunner.log`. A relative path resolves against the cwd at import; `os.devnull` turns the file off.
- **When:** the file (and its directory) is created on the first record; importing `je_web_runner` writes nothing.
- **Mode:** append; each line carries the process id; a file over 10 MB is moved to `<name>.1` when a process opens it.
- **Level:** WARNING+
- **Format:** `%(asctime)s | %(process)d | %(name)s | %(levelname)s | %(message)s`

## Supported Browsers

| Browser           | Selenium key | Playwright   |
|-------------------|--------------|--------------|
| Google Chrome     | `chrome`     | `chromium`   |
| Chromium          | `chromium`   | `chromium`   |
| Mozilla Firefox   | `firefox`    | `firefox`    |
| Microsoft Edge    | `edge`       | `chromium`   |
| Internet Explorer | `ie`         | n/a          |
| Apple Safari      | `safari`     | `webkit`     |

## Supported Platforms

- Windows
- macOS
- Ubuntu / Linux
- Raspberry Pi

## License

This project is licensed under the [MIT License](LICENSE).
