Metadata-Version: 2.4
Name: gecko-web-runtime-client
Version: 1.2.0
Summary: Thin Python client for the Firefox-derived WPR Gecko Worker protocol
Author: WPR project
License: MPL-2.0-compatible client; see the runtime bundle notices
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Provides-Extra: diagnostics
Requires-Dist: loguru>=0.7.2; extra == "diagnostics"

# Python client

The client provides a standard-library core plus an optional Loguru-backed
diagnostics layer. From this directory:

```powershell
$env:PYTHONPATH = "$PWD"
python ..\examples\normal_navigation.py
```

The package resolves `runtime/gecko-web-worker.exe` relative to its own
bundle. Override paths with `WPR_RUNTIME_HOME`, `WPR_RUNTIME_DIR`,
`WPR_WORKER_EXE`, `WPR_PROFILE_ROOT`, or `WPR_LOG_ROOT` when embedding it in a
larger application.

## Firefox Necko network client

The version-neutral `wpr-http-v1` API is exposed by
`gecko_web_runtime.AsyncWprHttpClient` (also importable from
`gecko_web_runtime.network`):

```python
from gecko_web_runtime import AsyncWprHttpClient, Timeout

async with AsyncWprHttpClient(
    base_url="https://example.com",
    headers={"X-Client": "demo"},
    timeout=Timeout(total=20),
) as client:
    request = client.build_request(
        "GET", "/search", params={"q": "gecko"}
    )
    response = await client.send(request)
    print(response.request.method, response.status_code)
    response.raise_for_status()
    payload = await response.json()
```

`Request` is the prepared, inspectable input (`method`, `url`, case-insensitive
mutable `headers`, `content`, `timeout`); `HttpResponse` carries that request,
status, case-insensitive headers, redirects, timing, charset and a leased body.
`send()` sends an existing `Request`; `request()` and `get()` / `post()` / `put()` /
`patch()` / `delete()` / `head()` / `options()` are convenience methods. For
async body handling use `await response.read()`, `await response.text()`, or
`await response.json()`; `aread()` / `atext()` / `ajson()` remain aliases. Use
`async with response` or `await response.aclose()` when leaving a response
unread. `await client.aclose()` closes its outstanding response leases before
shutting down the Worker.

The HTTP API is split into `network/client.py` (client defaults and request
helpers), `network/transport.py` (the versioned Worker boundary),
`network/models.py` (timeouts, response and body lifetime), and
`network/exceptions.py` (error hierarchy and Gecko error translation).
`gecko_web_runtime.http` remains a compatibility import path.

`timeout` accepts seconds, `Timeout(total=seconds, pool=seconds)`, or `None`.
`total` covers queueing plus the active request; `pool` can bound waiting for
an available Worker slot. By default `max_workers=1`, so `asyncio.gather()` calls
on one client queue behind one Worker. Set `max_workers=N` to let the same
client instance dispatch up to N requests at once:

```python
import asyncio
from gecko_web_runtime import AsyncWprHttpClient

async def main():
    async with AsyncWprHttpClient(max_workers=4, timeout=30) as client:
        responses = await asyncio.gather(
            *(client.get(f"https://example.com/api/{i}") for i in range(8))
        )
        try:
            print([response.status_code for response in responses])
        finally:
            await asyncio.gather(*(response.aclose() for response in responses))

asyncio.run(main())
```

This is a bounded Gecko Worker pool, not an HTTP socket-connection limit: the
current named-pipe RPC accepts one in-flight request per Worker. Each extra
Worker is a separate Firefox process/profile and therefore has its own Gecko
CookieService state; multi-Worker cookie continuity is not shared or ordered.
Keep the default `max_workers=1` when requests depend on a single browser cookie
jar or when minimizing memory is more important than parallel throughput. The
pool grows lazily up to the configured cap and reuses Workers after requests.
For latency-sensitive bursts, call `await client.warmup(url)` immediately after
entering the client and before sending application requests. It starts the
remaining Workers and makes one full GET per Worker to the supplied URL; the
responses are read and released while the Worker connections stay open. This
is real traffic, so use a safe, idempotent endpoint. Cookies set by that
endpoint remain isolated in each Worker profile.
An active client-side timeout or cancellation stops that Worker because the
current protocol has no per-request cancellation; the next call replaces it.
A queued request timeout leaves active requests and Workers untouched.
Connect/read/write phase timeouts still need native Worker support.

Request bodies currently accept UTF-8 text, JSON, or form data. Arbitrary
binary uploads are rejected explicitly until the protocol has a binary upload
data plane. Responses are buffered by the Worker up to its configured size
limit before Python receives the shared-memory body; streaming download/upload
remain future protocol work.

Transport failures raise a readable `WprHttpError` subclass (`WprProxyError`,
`WprConnectError`, or `WprTimeoutError`) and emit a standard-library WARNING
log record. It includes the Gecko `rv` and hex form, request method, redacted
URL, proxy endpoint without credentials, and `status_code=None` because no HTTP
response arrived. HTTP 4xx/5xx responses are returned normally, like
`requests`/`httpx`; `response.raise_for_status()` raises `WprHTTPStatusError`
with the HTTP status and response attached.

The selected Firefox bundle launches `runtime/net-v155/gecko-net-worker.exe`
(the versioned adapter may be replaced by a later bundle). Its response body
uses shared memory and is released when `atext()`, `aread()`, or `aclose()`
completes. Set `WPR_RUNTIME_HOME`, `WPR_RUNTIME_DIR`, and
`WPR_NETWORK_WORKER_EXE` to select a different Firefox-version adapter.

Client 1.2.0 with runtime 155.0.5 supports `ipc_backend="iocp"` and
`max_channels_per_worker=10` with `max_workers=1`, keeping one shared
cookie/proxy session while issuing concurrent Necko requests. Defaults remain
the independent-Worker threaded path. Older runtimes lack these capabilities;
upgrade both the client and runtime before enabling multi-Channel requests.
See [selection, protocol, and reproducible build instructions](../source/gecko-net-worker/experiments/multiplex-network/README.md).

### Measured public-site and resource comparisons

The release client and matching Worker were tested against Baidu
with 100 sequential requests and 100 requests at concurrency ten per library.
Gecko used one network Worker with up to ten Channels, not ten Workers.
Each resource-test arm ran in a fresh Python process. Sampling included that
Python process and every Worker descendant, excluding the external sampler
and the independent local fixture server.

Public-site elapsed times can be close even when CPU and memory costs differ.
The server also changed response size under concurrency, so the resource
comparison includes equal-hash fixed-body local controls. CPU uses cumulative
user+system deltas; memory fields are observed simultaneous tree peaks with
the actual sampling intervals recorded. They are not parent-only figures,
exact instantaneous maxima, or a claim of equivalent physical RAM consumption.

See [results, limitations, and reproduction commands](tests/results/http-resource-comparison-20261011.md).
The test entry points are `tests/integration/baidu_live_ab.py` and
`tests/integration/baidu_resource_ab.py`. They remain development tests; the
release keeps opt-in concurrency and transport defaults unchanged.

## Network IPC diagnostics and pipe A/B

The client defaults to chunked JSON-line reads (`WPR_PIPE_READ_MODE=chunk`),
compatible with the existing one-RPC-per-connection Worker. Set `byte` only
to reproduce the old unbuffered `FileIO.readline()` behavior.

`WPR_PIPE_MODE=rpc` remains the default. The optional `persistent` mode is
supported by runtime 155.0.5 (patches remain documented in
`source/gecko-net-worker/experiments/persistent-pipe`). In this mode each Worker retains
one control-pipe connection, serializes RPCs through its session lock, and closes
the channel on shutdown or abort. It does not multiplex network requests or
silently retry a failed request, including POST.

Set `WPR_RPC_DIAGNOSTICS=1` before starting the patched Worker to add high-resolution
Worker dispatch, channel setup, header wait, body delivery, blob creation and
response-finalization timings. The Python side independently records pipe
connect/write/read/close, JSON encode/decode, executor scheduling, pool wait,
mapping and body-copy durations. After consuming the body, inspect:

```python
response = await client.get("http://127.0.0.1:8080/")
body = await response.aread()
print(response.diagnostics)
```

`fetch_rpc.pipe_read_ms` includes waiting for the network; it is not pure IPC
cost. Worker and Python durations use their respective monotonic clocks. Do not
subtract timestamps across processes. Missing Worker timing is an empty mapping,
not a claimed zero-duration measurement. Diagnostics are returned as data and
do not print automatically.

From this directory, with the experimental binary selected:

```powershell
$env:WPR_RUNTIME_HOME = 'G:\wpr-runtime-155-win64'
$env:WPR_NETWORK_WORKER_EXE = 'G:\firefox1\firefox\obj-wpr-context\dist\bin\gecko-web-worker.exe'
uv run --with httpx --with aiohttp python tests/integration/http_pipe_ab.py `
  --count 100 --rounds 3 --output tests/results/http-pipe-ab.json
```

This compares four Gecko combinations and aiohttp using exact fixed local
responses, warmed pools, rotating backend order, and separately recorded cold
pool startup. Add `--no-worker-diagnostics --small-only` to quantify performance
without diagnostic metadata expansion. Results include phase medians and aiohttp
connection-create/reuse trace counts. The server runs in the controller process,
so its CPU is included in `controller_and_local_server_cpu_seconds`.

For a focused 500-request comparison rather than the full matrix:

```powershell
uv run --with httpx --with aiohttp python tests/integration/http_pipe_ab.py `
  --count 500 --rounds 1 --parallel-only --small-only --delays 0 `
  --variants persistent/chunk aiohttp `
  --output tests/results/http-gecko-aiohttp-500.json
```

### Experimental piggyback response release

After building Worker patches 0018 and 0019, select the matching Release DLL
directory and opt into deferred network-body cleanup:

```powershell
$env:WPR_RUNTIME_HOME = 'G:\wpr-runtime-155-win64'
$env:WPR_RUNTIME_DIR = 'G:\firefox1\firefox\obj-wpr-release\dist\bin'
$env:WPR_NETWORK_WORKER_EXE = "$env:WPR_RUNTIME_DIR\gecko-web-worker.exe"
$env:WPR_PIPE_MODE = 'persistent'
```

```python
async with AsyncWprHttpClient(max_workers=10, release_mode="piggyback") as client:
    response = await client.get("http://127.0.0.1:8080/")
    body = await response.aread()
    await client.flush_releases()  # Optional explicit Worker-side confirmation.
    print(client.transport_stats)
```

`aread()`/`aclose()` immediately close the local named mapping. Worker-side lease
release is acknowledged on the next Fetch, a timer/threshold flush, an explicit
flush, or client shutdown. The timer attempts a flush after 50 ms, but serial
pipe occupancy can delay it. Thresholds of 32 entries or 8 MiB trigger drains;
they are not global caps on unread response memory. Only acknowledged entries
are removed from the pending queue. Abort stops the Worker and discards its queue.

The default remains `release_mode="immediate"`. Old Workers are rejected during
capability negotiation if piggyback mode is requested, before any HTTP request
is sent. `transport_stats` covers the currently resident pool, not retired Workers.

```powershell
uv run --with httpx --with aiohttp python tests/integration/http_pipe_ab.py `
  --count 500 --rounds 1 --parallel-only --small-only --delays 0 `
  --variants persistent/chunk persistent/chunk/piggyback aiohttp `
  --output tests/results/http-release-piggyback-500.json
```

The benchmark includes final release draining in total request-phase time.

To install the pinned runtime automatically from the GitHub Release:

```powershell
python -m gecko_web_runtime install-runtime --version 155
```

The command verifies the pinned SHA256 and stores the selected runtime under
`%LOCALAPPDATA%\WPR\runtime\155`.

For development, use `uv` from this directory so the Python environment and
lock file are reproducible:

```powershell
cd G:\wpr-runtime-155-win64\python
uv sync --extra diagnostics
uv run python -m unittest discover -s tests\unit -v
```

The Gecko binary is intentionally not a Python dependency.  Install or select
the matching runtime separately after the environment is ready:

```powershell
uv run python -m gecko_web_runtime install-runtime --version 155
```

Live network workflow tests are opt-in:

```powershell
$env:WPR_RUN_ONLINE_TESTS = "1"
uv run python -m unittest discover -s tests\integration -v
```

To measure cold/warm navigation and the current isolation boundary:

```powershell
uv run python tests\integration\context_reuse_benchmark.py `
  --url https://example.com/ `
  --out context-reuse.json
```

The benchmark records first/second navigation timing, frame remote types,
WindowGlobal replacement, Worker PIDs and whether an isolated context sees a
Cookie from the persistent context.

An optional per-Worker `fpfile` path can be passed to the session. The Worker
reads the file before creating the remote page; its Firefox profile remains a
separate Cookie/Storage directory:

```python
async with AsyncGeckoWorkerSession(
    fpfile=r"G:\browser-runtime\fpfile.txt"
) as session:
    await session.page_create()
```

For application code, prefer the high-level runtime facade and workflow
adapter.  Frame handles, pointer coordinates, WindowGlobal rebinding and
challenge waits remain internal:

```python
from gecko_web_runtime import AsyncGeckoRuntime

async with AsyncGeckoRuntime(fpfile=r"G:\browser-runtime\fpfile.txt") as runtime:
    page = await runtime.new_page()
    result = await page.navigate(
        "https://nopecha.com/demo/cloudflare",
        workflow="turnstile",
    )
    print(result.ok, result.url, len(result.html))
```

For WeChat public-account articles, use the site adapter instead of the
Cloudflare-specific Turnstile workflow:

```python
async with AsyncGeckoRuntime() as runtime:
    page = await runtime.new_page()
    result = await page.navigate(article_url, workflow="wechat_article")
    if result.ok:
        article_html = result.html
```

Use `WorkflowMetrics` when an application needs a stable result/metrics
boundary. The default output contains the product chain stages (`T0`...`T9`)
plus a compact report with success, final URL, HTML size,
total/navigation/final-ready time, and parent/whole-process-tree memory peaks.
Detailed attempts, phase timings, and per-stage resource snapshots are
emitted only with `debug=True` or `emit="debug"`:

```python
from gecko_web_runtime import WorkflowMetrics

metrics = WorkflowMetrics(name="wechat", emit="summary", print_summary=True)
summary = metrics.emit(result)
```

Library code can set `emit="none"` and consume the returned dictionary without
printing. `WPR_METRICS_EMIT`, `WPR_METRICS_LOG_LEVEL`, `WPR_METRICS_DEBUG`,
`WPR_METRICS_PRINT`, and `WPR_METRICS_MEMORY` provide process-level defaults.

The adapter recognizes the WeChat/Tencent gate, rebinds the current top
WindowGlobal, and dispatches a real pointer sequence to the visible
verification target.  It reports slider/image challenges as a bounded result
for a future dedicated solver rather than treating the challenge page as an
article.  It is one-shot by default and closes the Page after the workflow
boundary; pass `WechatArticleWorkflow(close_after=False)` when intentionally
reusing the Page.

For multiple logical sessions, keep the Worker runtime alive but make the
storage boundary explicit:

```python
async with AsyncGeckoRuntime(fpfile="fpfile-a.txt") as runtime:
    async with runtime.new_context(isolation="persistent") as context_a:
        page_a = await context_a.new_page()
        result_a = await page_a.navigate(url_a)

    async with runtime.new_context(isolation="isolated") as context_b:
        page_b = await context_b.new_page()
        result_b = await page_b.navigate(url_b)
```

`persistent` reuses the runtime's current Worker while each Context receives
its native Firefox OriginAttributes boundary (`user_context_id` and optional
`private_browsing_id`).  `isolated` remains the explicit separate Worker and
profile mode for callers that require process-level isolation.

Turnstile results expose phase timings in `result.timing_ms`, including
per-attempt iframe/action waits, POST, redirect, final Document readiness, and
HTML serialization.  Worker PID, Working Set, Private Bytes, and CPU samples
are available in `result.resource_usage`.  Sampling can be disabled for a
low-overhead benchmark with `TurnstileWorkflow(resource_sampling=False)` or
`WPR_RESOURCE_SAMPLING=0`.  The integration test writes the complete result
to the path in `WPR_RESULT_OUT` when set.

Challenge retries use the `adaptive` policy by default.  After a top-level
state transition it waits for the network settle window and stops immediately
on POST/redirect; it only dispatches another action when no submit arrives.
`retry_policy="legacy"` preserves the previous retry loop for A/B experiments,
while `retry_policy="generation"` is a strict diagnostic policy that requires a
new `BrowsingContext`/`WindowGlobal` before another action.  The third-party
client exposes the same switch as `--retry-policy`.  Debug results include
`state_change_to_submit_ms` when both observations were captured on the same
client monotonic clock.

The final boundary is configurable.  `final_wait_until="complete"` waits for
the full DOM/load boundary and is the compatibility default.  Set
`WPR_TURNSTILE_WAIT_UNTIL=document` (or pass `final_wait_until="document"`)
when the caller only needs the final Document HTML; the result then reports
`final_document`, `final_load`, and the selected `final_ready` boundary
separately.  Document mode does not wait for the page's scripts, stylesheets,
images, or analytics resources to finish.

Large results use a separate data plane.  `page.eval_shared_blob()` uses a
one-shot mapping, while `page.eval_shared_ring()`/`page.html_shared_ring()` use
a Worker-lifetime fixed-slot ring.  Both return a small descriptor over the
control pipe and a read-only Windows named mapping for the bytes.  Read it as
a `memoryview` or decode it with `read_text()`, then release the lease
explicitly:

```python
blob = await page.html_shared_ring()
try:
    html = blob.read_text()
finally:
    await blob.release()
```

This keeps multi-megabyte HTML out of JSON serialization and the named-pipe
response.  The ring mapping is created once per Worker and reuses slots using
generation-bearing leases; `data.release` returns a slot to the ring.  Set
`WPR_HTML_TRANSPORT=json`, `blob`, or `ring` to A/B the adapter boundary.  The
WeChat and Turnstile adapters default to `ring` and fall back to ordinary
`page.eval` when an older Worker has no data-plane method.

The long-lived comparison probe keeps one Worker per site/transport, uses a
fresh OriginAttributes context for every sample, and records success rate,
P95, CPU, Worker/process-tree memory, and Gecko reporter stages:

```powershell
uv run python tests\integration\data_plane_ab_benchmark.py `
  --runs 10 --modes json ring
```

The low-level `AsyncGeckoWorkerSession` API remains available for custom
workflows and backward compatibility.  Turnstile is implemented in
`gecko_web_runtime.workflows.turnstile`; future site adapters belong in the
same package and are not part of the Gecko core.

The client project keeps its tests alongside the package source:

```text
python/tests/unit/          # import/API contract tests
python/tests/integration/   # opt-in live Worker/workflow tests
```

The larger `G:\browser-runtime\tests\probes` tree is a separate black-box
runtime audit harness.  It exercises the complete bundled Gecko runtime and
stores diagnostic artifacts, so it is intentionally not included in the
Python wheel.

To enable structured application diagnostics:

```bash
pip install gecko-web-runtime-client[diagnostics]
```

```python
from gecko_web_runtime.diagnostics import configure_logging

logger = configure_logging(log_file="runtime.log")
logger.info("application.start")
```

For a real page WebRTC permission boundary, use the same session and answer
the native pending request.  The Worker forwards Firefox's
`PeerConnection:response:allow` or `PeerConnection:response:deny` topic; it
does not manufacture SDP, ICE candidates, or media devices:

```python
pending = await session.page_permission_requests()
for request in pending:
    await session.page_permission_respond(request["callID"], "allow")
```

Leaving the list unanswered preserves Firefox's normal prompt-pending
behavior.  A response is scoped to the current WindowGlobal, so a navigation
that replaces it requires polling the new page's request list.

For a private GitHub Release, set `GITHUB_TOKEN`, `GH_TOKEN`, or
`WPR_GITHUB_TOKEN`, or configure Git Credential Manager for the GitHub account.
# Firefox Necko network client

The version-neutral `wpr-http-v1` API is exposed by
`gecko_web_runtime.AsyncWprHttpClient`:

```python
from gecko_web_runtime import AsyncWprHttpClient

async with AsyncWprHttpClient() as client:
    response = await client.get("https://example.com/")
    html = await response.atext()
```

The selected Firefox bundle launches `gecko-net-worker.exe`. Its response body
uses shared memory and is released when `atext()`, `aread()`, or `aclose()`
completes. Set `WPR_RUNTIME_HOME`, `WPR_RUNTIME_DIR`, and
`WPR_NETWORK_WORKER_EXE` to select a different Firefox-version adapter.
