Metadata-Version: 2.4
Name: vfa-mcp
Version: 1.0.5
Summary: VFA — Visual & Functional Auditor: auditoria web autonoma (visual + funcional) expuesta como servidor MCP, con Playwright local y motor universal LiteLLM.
Author-email: AIDevTeam <red.line.v.o.s@outlook.com>
License-Expression: LicenseRef-Proprietary
Project-URL: Homepage, https://vaillan.github.io/vfa-docs
Project-URL: Documentation, https://vaillan.github.io/vfa-docs
Project-URL: Buy License, https://www.creem.io/payment/prod_6qWpfXtntbKRUvBifbObZj
Keywords: mcp,mcp-server,qa-automation,visual-regression,playwright,litellm,accessibility,llm-agents
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mcp<2.0.0,>=1.2.0
Requires-Dist: litellm==1.106.3
Requires-Dist: deepagents>=0.1.0
Requires-Dist: langchain-litellm>=0.1.0
Requires-Dist: playwright>=1.40.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: truststore>=0.9.0
Requires-Dist: mcp-types>=0.1.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
Dynamic: license-file

# VFA — Visual & Functional Web Auditor (MCP Server)

Autonomous black-box web QA agent exposed as an MCP server over `stdio`.
Pure hexagonal architecture (Ports & Adapters), one unified inference engine
powered by **LiteLLM**, and a local-first **Playwright** Chromium sandbox —
no Docker, no sidecars, no infrastructure to maintain.

## Why VFA

| Pillar                       | What it means                                                                                                                                                                                                                       |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Native Zero-Docker**       | Playwright downloads and manages Chromium on the host itself. A built-in ephemeral `BrowserContext` sandbox plus an Anti-SSRF firewall ship by default, and local development is first-class when `VFA_ALLOW_LOCAL_HOST=true` (`localhost`, `127.0.0.1`, `*.local`). |
| **Universal LiteLLM Engine** | A single adapter drives DeepSeek, OpenAI, Anthropic, Gemini, xAI/Grok, OpenRouter and Ollama. Switching providers is a two-variable change.                                                                                         |
| **Active Visual Grounding**  | Multimodal vision inspects real UIs, reads raster image CAPTCHAs and clicks by visual coordinates whenever the accessibility tree is not enough.                                                                                    |
| **1-Click Execution**        | Plug-and-play: `uvx vfa-mcp` starts the server with no permanent install, `pip install vfa-mcp` if you prefer an isolated virtualenv.                                                                                               |

```bash
uvx vfa-mcp
```

Goal-driven QA ships in the same engine: one sentence becomes a validated
`TestPlan` with deterministic oracles, the agent executes and recovers each
case, and the verdict is computed from evidence instead of from "the steps
ran". See [Goal mode lifecycle](#goal-mode-lifecycle).

## Quickstart

### ⚠️ License Key (Required)

VFA is a commercial tool. An active **VFA Pro License** is required to execute audits and user flows.
* **Get your license:** [Purchase on Creem](https://www.creem.io/payment/prod_6qWpfXtntbKRUvBifbObZj)
* **Configuration:** Set the `VFA_LICENSE_KEY` environment variable in your MCP client or `.env`.

### 1. Install

One command, permanent install in your own virtualenv:

```bash
pip install vfa-mcp && playwright install chromium
```

Zero-install alternative (recommended for MCP clients):

```bash
uvx vfa-mcp
playwright install chromium   # first run only
```

### 2. Pick a provider

VFA resolves the API key from `VFA_LLM_API_KEY` first, then from the standard
variable of the selected provider (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
`GEMINI_API_KEY`, `DEEPSEEK_API_KEY`, `XAI_API_KEY`, `OPENROUTER_API_KEY`).

| Provider | `VFA_LLM_PROVIDER` | Example `VFA_LLM_MODEL` | API key variable | Base URL / Endpoint variable |
| :--- | :--- | :--- | :--- | :--- |
| OpenAI | `openai` | `gpt-4o` | `OPENAI_API_KEY` | — *(Default)* |
| Anthropic | `anthropic` | `claude-3-5-sonnet-20241022` | `ANTHROPIC_API_KEY` | — *(Default)* |
| Google Gemini | `gemini` | `gemini/gemini-2.0-flash` | `GEMINI_API_KEY` | — *(Default)* |
| DeepSeek (Direct) | `deepseek` | `deepseek/deepseek-chat` | `DEEPSEEK_API_KEY` | — *(Default)* |
| **DeepSeek (OpenAI-compat)** | `openai` | `openai/deepseek-v4-pro` | `VFA_LLM_API_KEY` / `OPENAI_API_KEY` | `VFA_LLM_BASE_URL="https://api.deepseek.com"` |
| **DeepSeek Vision (OpenAI-compat)** | `openai` | `openai/deepseek-flash` | `VFA_VISION_API_KEY` / `VFA_LLM_API_KEY` | `VFA_VISION_BASE_URL="https://api.deepseek.com"` |
| **DeepSeek (Anthropic-compat)** | `anthropic` | `claude-3-5-sonnet` *(alias)* | `DEEPSEEK_API_KEY` / `ANTHROPIC_API_KEY` | `ANTHROPIC_BASE_URL="https://api.deepseek.com/anthropic"` |
| xAI / Grok | `xai` | `xai/grok-2-vision-1212` | `XAI_API_KEY` | — *(Default)* |
| OpenRouter | `openrouter` | `openrouter/anthropic/claude-3.5-sonnet` | `OPENROUTER_API_KEY` | — *(Default)* |
| Ollama (local) | `ollama` | `ollama/llama3.2-vision` | — *(Placeholder)* | `OLLAMA_API_BASE="http://localhost:11434"` |

Aliases accepted by the provider resolver: `google` (Gemini) and `grok` (xAI).
The vision role inherits the reasoning provider, model and endpoint when
`VFA_VISION_PROVIDER`, `VFA_VISION_MODEL` and `VFA_VISION_BASE_URL` are left
empty. When both roles use the `openai` protocol but different endpoints,
`VFA_VISION_BASE_URL` is mandatory.

### 3. Wire it into your client

#### Zoo Code / Roo Code (`mcpSettings.json`)

Click the **MCP Servers** plug icon ⚙️ → **Edit MCP Settings**, then add:

```json
{
  "mcpServers": {
    "vfa": {
      "command": "uvx",
      "args": ["vfa-mcp"],
      "env": {
        "VFA_LICENSE_KEY": "PRO-XXXX-XXXX",
        "OPENAI_API_KEY": "sk-...",
        "HEADLESS": "true",
        "VFA_ALLOW_LOCAL_HOST": "true",
        "LITELLM_LOG": "ERROR"
      }
    }
  }
}
```

---

#### Claude Desktop (`claude_desktop_config.json`)

* **macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
* **Windows:** `%APPDATA%\Claude\claude_desktop_config.json`

```json
{
  "mcpServers": {
    "vfa": {
      "command": "uvx",
      "args": ["vfa-mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "HEADLESS": "true",
        "VFA_ALLOW_LOCAL_HOST": "true",
        "LITELLM_LOG": "ERROR"
      }
    }
  }
}
```

---

#### Cursor (`.cursor/mcp.json` or `Settings → Features → MCP`)

```json
{
  "mcpServers": {
    "vfa": {
      "command": "uvx",
      "args": ["vfa-mcp"],
      "env": {
        "OPENAI_API_KEY": "sk-...",
        "HEADLESS": "true",
        "VFA_ALLOW_LOCAL_HOST": "true",
        "LITELLM_LOG": "ERROR"
      }
    }
  }
}
```

---

### Provider Drop-in Presets (Replace the `"env"` block)

Simply swap the `"env"` object in any configuration above with your preferred model:

#### ⚡ DeepSeek (OpenAI-compatible: reasoning + fast vision)
```json
"env": {
  "VFA_LLM_PROVIDER": "openai",
  "VFA_LLM_MODEL": "openai/deepseek-v4-pro",
  "VFA_VISION_PROVIDER": "openai",
  "VFA_VISION_MODEL": "openai/deepseek-flash",
  "VFA_LLM_BASE_URL": "https://api.deepseek.com",
  "VFA_VISION_BASE_URL": "https://api.deepseek.com",
  "VFA_LLM_API_KEY": "sk-...",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

> DeepSeek's native provider is also supported with
> `VFA_LLM_PROVIDER=deepseek` + `VFA_LLM_MODEL=deepseek/deepseek-chat`. Use the
> OpenAI-compatible form above when your model names are not present in
> LiteLLM's native model map.

#### 🧠 Anthropic Claude 3.5 Sonnet
```json
"env": {
  "VFA_LLM_PROVIDER": "anthropic",
  "VFA_LLM_MODEL": "claude-3-5-sonnet-20241022",
  "ANTHROPIC_API_KEY": "sk-ant-...",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

#### 🚀 Google Gemini 2.0 Flash (Multimodal)
```json
"env": {
  "VFA_LLM_PROVIDER": "gemini",
  "VFA_LLM_MODEL": "gemini/gemini-2.0-flash",
  "GEMINI_API_KEY": "AIzaSy...",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

#### 🌐 OpenRouter (Universal Gateway)
```json
"env": {
  "VFA_LLM_PROVIDER": "openrouter",
  "VFA_LLM_MODEL": "openrouter/anthropic/claude-3.5-sonnet",
  "VFA_VISION_MODEL": "openrouter/google/gemini-2.0-flash-001",
  "OPENROUTER_API_KEY": "sk-or-v1-...",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

#### 🏠 Ollama (100% Local / Offline)
```json
"env": {
  "VFA_LLM_PROVIDER": "ollama",
  "VFA_LLM_MODEL": "ollama/llama3.2-vision",
  "OLLAMA_API_BASE": "http://localhost:11434",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

#### ⚙️ OpenAI-compatible gateway:
```json
"env": {
  "VFA_LLM_PROVIDER": "openai",
  "VFA_LLM_MODEL": "openai/your-reasoning-model",
  "VFA_VISION_PROVIDER": "openai",
  "VFA_VISION_MODEL": "openai/your-vision-model",
  "VFA_LLM_BASE_URL": "https://your-gateway.example.com/v1",
  "VFA_VISION_BASE_URL": "https://your-gateway.example.com/v1",
  "VFA_LLM_API_KEY": "sk-...",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

#### ⚙️ Anthropic-compatible:
```json
"env": {
  "VFA_LLM_PROVIDER": "anthropic",
  "VFA_LLM_MODEL": "claude-3-5-sonnet",
  "ANTHROPIC_BASE_URL": "https://api.deepseek.com/anthropic",
  "ANTHROPIC_API_KEY": "sk-...",
  "HEADLESS": "true",
  "VFA_ALLOW_LOCAL_HOST": "true",
  "LITELLM_LOG": "ERROR"
}
```

> Agent-facing usage guide (when to invoke each tool, report template, security
> rules): [`SKILL.md`](SKILL.md). Marketplace manifest: [`skill.json`](skill.json).

## Tool reference

The v2 contract is **additive**: every parameter listed here is optional and all
previously published keys are still present.

### `qa_audit_url(url, session_id=None, screenshots_mode="auto")`

Vision analysis and runtime error collection in a single call.

| Parameter    | Type   | Required | Description                                                                                             |
| ------------ | ------ | -------- | ------------------------------------------------------------------------------------------------------- |
| `url`        | string | yes      | Reachable URL to audit. Only `http`/`https`; private or loopback targets are rejected unless `VFA_ALLOW_LOCAL_HOST` is enabled. |
| `session_id` | string | no       | Stable session key (`default` when omitted) so consecutive calls share browser, cookies and state.       |
| `screenshots_mode` | `auto`\|`inline`\|`paths`\|`none` | no | How the capture travels (see below); `auto` is the default. |

`auto` (default), `inline`, `paths` and `none` select how the capture travels:
`auto` returns evidence references only, `inline` pushes base64 (opt-in, one
capture, capped by `VFA_MAX_INLINE_BYTES`), `paths` writes under
`VFA_ARTIFACTS_DIR` and `none` captures nothing.

Returns `url`, `ok` (`true` only when the audit passed and no runtime event was
captured), `summary`, `issues[]` (visual, accessibility and layout findings),
`errors[]`, plus the additive keys `warnings[]`, `session_id`, `run_id`,
`verdict`, `observation` and `evidence`. The aliases `screenshot`,
`screenshot_base64` and `findings` are still present, with the capture keys
empty by default: pixels are pulled on demand with `qa_get_screenshot`.

The vision reply is parsed fail-closed: a truncated JSON object is salvaged
(`ok`, `summary`, `findings`, `truncated: true`) instead of losing the whole
analysis, and a reply without an explicit boolean `ok` is treated as a failure,
never as a pass. The audit prompt asks for at most five findings of 160
characters to keep the reply inside the token budget.

### `qa_execute_user_flow(url, steps=None, session_id=None, goal=None, observe=True, screenshots_mode="auto", case_id="", test_id="", max_minutes=None, max_iterations=None, max_modules=None, data=None)`

Replays a user journey from natural-language or structured steps, or plans and
executes it from a goal.

In goal mode the server observes the page before planning, feeds that reading to
the planner, executes the cases and, when a step fails, hands the case to the
agent loop so it can re-observe and recover. A recovered step is reported with
`recovered: true` and a `warning` finding instead of a critical one.

| Parameter          | Type              | Required | Description                                                                 |
| ------------------ | ----------------- | -------- | --------------------------------------------------------------------------- |
| `url`              | string            | yes      | Starting URL of the flow.                                                   |
| `steps`            | (string\|object)[]| no       | Ordered steps. A `dict` step is executed literally (no parser, no selector LLM). |
| `session_id`       | string            | no       | Stable session key for continuity across calls.                             |
| `goal`             | string            | no       | Objective in one sentence; when `steps` is empty the planner builds cases.  |
| `observe`          | boolean           | no       | Keep the observation usable (default `true`); `false` never exposes pixels.  |
| `screenshots_mode` | `auto`\|`inline`\|`paths`\|`none` | no | `auto` (default) returns evidence references; `inline` pushes base64; `paths` writes under `VFA_ARTIFACTS_DIR`; `none` captures nothing. |
| `case_id`          | string            | no       | Correlates the payload with the caller's test identifier.                   |
| `test_id`          | string            | no       | Stable runner identifier, e.g. `checkout.spec.ts:42`.                       |
| `max_minutes`      | number            | no       | Time budget for goal mode.                                                  |
| `max_iterations`   | integer           | no       | Per-call budget for the agent loop; defaults to `VFA_AGENT_MAX_ITERATIONS`. |
| `max_modules`      | integer           | no       | Cap for inventory and traversal; defaults to `VFA_MAX_MODULES` (6).         |
| `data`             | object            | no       | Fixed values for the planner (for example credentials); never echoed back.  |

Structured step example:

```json
{"action": "fill", "target": "Usuario", "value": "qa@test.dev"}
{"action": "click", "selector": "#login"}
{"action": "assert", "oracle": {"kind": "url_contains", "value": "/dashboard"}}
{"action": "assert", "oracle": {"kind": "text_present", "value": "Bienvenido"}}
{"action": "set_viewport", "width": 390, "height": 844}
{"action": "enter_frame", "selector": "#panel"}
{"action": "wait_for", "condition": {"kind": "element_visible", "target": "#panel", "timeout_ms": 3000}}
{"action": "assert", "oracle": {"kind": "field_value", "target": "#total", "value": "1234.5", "normalize": "number"}}
```

#### Navigation primitives (P0 and P1)

Every capability is a browser primitive, never a business task; the domain of
the site enters only through `url`, `steps`, `goal`, `data` and `oracles`.

| Action | Payload | Notes |
| --- | --- | --- |
| `enter_frame` / `leave_frame` | `selector` | Actions and oracles apply to the active frame. Navigating clears it. |
| `switch_popup` / `close_popup` | `index` (default last) | Moves focus to a popup tab and back; events follow the active tab. |
| `wait_for` | `condition: {kind, target, value, timeout_ms}` | `element_visible`, `text_present`, `network_idle`, `url_contains`. |
| `upload` | `selector`, `files: [...]` | Requires `VFA_UPLOAD_DIR`; paths outside it are rejected. |
| `drag` | `selector`, `to` | Native drag & drop between two elements. |

The import/export of an authenticated session is configuration-side: point
`VFA_SESSION_STORAGE_STATE` at a Playwright `storageState` to seed every new
context, and set `VFA_STORAGE_STATE_EXPORT=true` to receive the final state in
`storage_state` (a security warning is added to `warnings[]`; the server never
writes it to disk).

Returns `url`, `goal`, `success`, `ok`, `steps[{step, ok, action, selector, detail, duration_ms}]`,
`screenshots[]`, `errors[]` plus the additive keys `warnings[]`, `observation`,
`plan`, `cases[]`, `inventory[]`, `storage_state`, `verdict`, `findings[]`,
`counts`, `session_id`, `run_id`, `case_id`, `test_id` and `duration_ms`.

`plan.status` is `ready` or `blocked`; `plan.cases[]` declares the oracles,
`cases[]` carries the per-case verdict and `inventory[]` is what the agent
registered with `report_inventory` (`[{label, role, url}]`). Findings reported
by the agent carry `kind: "agent"`; `findings[]` also carries `kind` values
`step`, `oracle`, `recovery`, `budget` and `plan`. Secrets are redacted before
the payload leaves the server: a `fill` value is masked in `steps[]` even when
its key is not named like a credential.

`success` still means "the steps ran"; `verdict` is the QA judgement computed by
the deterministic oracle engine (`passed` | `failed` | `blocked`). `ok` is
fail-closed and never contradicts `verdict`.

#### Goal mode lifecycle

Goal mode is the server-side QA pipeline: the host sends one sentence and the
server plans, executes, recovers and judges. Planning is a single text
completion; execution is deterministic first and agentic only when needed.

```mermaid
sequenceDiagram
    autonumber
    participant C as MCP client
    participant F as qa_flow
    participant O as initial_context
    participant P as TestPlanner
    participant V as domain.test_plan
    participant X as execute_cases
    participant A as DeepAgent
    participant B as Browser

    C->>F: qa_execute_user_flow(url, goal, data, budgets)
    F->>O: observe before planning
    O->>B: goto + read_observation
    O-->>F: url, title, text, accessibility nodes
    F->>P: plan(goal, data, context)
    P->>V: parse_test_plan(reply)
    V-->>P: validated TestPlan
    P-->>F: ready or blocked plan
    alt plan is ready
        F->>F: use the plan as returned
    else blocked, empty or transport failure
        F->>F: fallback_plan adds a plan-fallback warning
    end
    F->>X: execute_cases(plan)
    loop each case
        X->>B: goto(url) and optional viewport
        alt prose steps
            X->>A: run_case(case)
        else structured steps
            X->>X: deterministic step engine
            X->>A: recovery only if a step failed
        end
        X->>B: observation and screenshot
        X->>X: evaluate oracles and aggregate the case verdict
    end
    X-->>C: plan, cases, inventory, findings, verdict, counts
```

**Planner contract**

- One LLM call per goal, JSON only, `status` is `ready` or `blocked`. An
  ambiguous objective returns `blocked` with `questions` instead of invented
  cases.
- Every case needs an `id`, at least one step and at least one oracle. Unknown
  oracle kinds are discarded and leave a `warning` finding.
- The planner never raises: any exception, empty reply or non-JSON text
  degrades to `blocked`. After a transport failure or an unusable reply a
  five-minute circuit breaker serves the fallback plan, and LiteLLM runs with
  `num_retries=0` so a timeout is not multiplied.
- A blocked plan does not abort the run: `fallback_plan()` builds one agentic
  case (`agent-goal`) with a `no_console_errors` oracle plus the
  `plan-fallback` warning, so the agent can still fulfil the objective.

**Execution contract**

- Structured steps run deterministically; a case whose prose the parser cannot
  execute is handed whole to the agent.
- If a deterministic step fails, the case is retried agentically. A recovered
  step is reported with `recovered: true` and a `recovery` warning; an
  unrecovered case produces a `critical` finding.
- Oracles are evaluated per case over the real observation. `visual` requires
  an explicit `true` from the vision model, otherwise it fails with `major`
  instead of being skipped silently.
- Exhausting `max_minutes`, `max_iterations` or `max_modules` fails honestly
  with a `budget` finding; it never becomes green.

**Agent tools**

| Tool | Purpose |
| --- | --- |
| `snapshot(limit)` | URL, title, visible text and accessibility nodes |
| `inspect()` | Current URL and visible text only |
| `look(question)` | Short answer about the current viewport screenshot |
| `click`, `fill`, `press`, `scroll`, `click_point`, `type_text` | Page actions |
| `report_finding(severity, message, evidence)` | Registers a defect (`kind: "agent"`) |
| `report_inventory(modules)` | Registers `[{label, role, url}]` for fan-out |
| `finish(summary, success)` | Closes the case with an explicit verdict |

Closing is nudged, not left to chance: the system prompt orders an immediate
`finish`, the recovery prompt forbids exploring past `max_modules`,
`report_inventory` replies with a reminder, and `snapshot` adds a closing hint
after eight observations. It is not mathematically guaranteed: an exhausted
budget fails the case instead of faking success.

### `qa_get_runtime_errors(peek=False)`

Drains the browser error buffer: `console.error`, uncaught exceptions
(`pageerror`), failed requests (`requestfailed`) and HTTP responses with status
`>= 400`. Console warnings are returned separately and never fail a run.

| Parameter | Type    | Required | Description                                     |
| --------- | ------- | -------- | ----------------------------------------------- |
| `peek`    | boolean | no       | Return the buffer without draining it (default `false`). |

Returns `count`, `errors[{kind, message, url, status}]`, `warnings[]`,
`error_count`, `errors_truncated` and `session_id`. `kind` is one of `console`,
`pageerror`, `http` or `requestfailed`. `VFA_MAX_ERRORS` (50) and
`VFA_MAX_ERROR_CHARS` (500) keep a noisy page from flooding the context: the
remaining events are counted, never serialized.

### `qa_get_screenshot(run_id, evidence_id="", index=0, quality=60, max_width=1280)`

Pulls a capture from a previous run as an MCP image (`ImageContent`). It is the
only path that moves pixels: the audit and flow payloads carry evidence
references, so the model asks for the frame only when the text is not enough.

| Parameter     | Type    | Required | Description                                                    |
| ------------- | ------- | -------- | -------------------------------------------------------------- |
| `run_id`      | string  | yes      | `run_id` returned by `qa_audit_url` or `qa_execute_user_flow`.  |
| `evidence_id` | string  | no       | `evidence.items[].id`; it takes precedence over `index`.        |
| `index`       | integer | no       | Position inside the run when `evidence_id` is empty.            |
| `quality`     | integer | no       | Requested JPEG quality for future captures (default 60).        |
| `max_width`   | integer | no       | Requested maximum width for future captures (default 1280).     |

A client without image support receives metadata instead: `path`, `sha256`,
`bytes`, `format` and `hint`. An expired or unknown id returns a readable error
(never an exception) that names the cause and the fix.

## Context budget and evidence

A response never carries a capture twice and, by default, carries none:

- `evidence.items[]` holds `{id, reason, format, bytes, sha256, width, height, path, retention, inline}`.
- `evidence.hint` names `qa_get_screenshot`; `evidence.doubt` marks a truncated vision reply.
- `evidence.retention` is `memory` (TTL `VFA_EVIDENCE_TTL`, default 600 s) or `disk` when `VFA_ARTIFACTS_DIR` is set.

| Ceiling | Default | Effect |
| ------- | ------- | ------ |
| `VFA_EVIDENCE_MODE` | `auto` | `off` disables every capture (personal data). |
| `VFA_INLINE_EVIDENCE` | `off` | The only switch that pushes pixels into a response. |
| `VFA_MAX_INLINE_BYTES` | `32768` | Inline base64 budget (~8-10k tokens). |
| `VFA_MAX_RESPONSE_BYTES` | `65536` | Hard ceiling of the whole response; degradation happens field by field, so the JSON stays valid. |
| `VFA_MAX_CAPTURES` | `8` | Captures per run. |
| `VFA_MAX_ERRORS` / `VFA_MAX_ERROR_CHARS` | `50` / `500` | Runtime events returned. |
| `VFA_MAX_STEPS` / `VFA_MAX_FINDINGS` | `200` / `50` | Serialized steps and findings. |

Conversation growth is the host's responsibility: the server caps each response,
not the session. In CI, set `VFA_ARTIFACTS_DIR` so `evidence[].path` survives the
process and `qa_get_screenshot` can read from disk.

### Oracle catalog (deterministic verdict)

| `kind`                  | Value | Fails with |
| ----------------------- | ----- | ---------- |
| `url_contains`          | yes   | critical   |
| `url_equals`            | yes   | critical   |
| `text_present`          | yes   | critical   |
| `text_absent`           | yes   | critical   |
| `element_present`       | yes   | major      |
| `no_console_errors`     | no    | critical   |
| `no_page_errors`        | no    | critical   |
| `no_http_server_errors` | no    | critical   |
| `no_request_failures`   | no    | major      |
| `visual`                | yes   | major (requires an explicit `true` from the vision model) |

Live oracles read the browser at evaluation time; they are reported as "not
evaluable" (never as passed) when no session is available.

| `kind` | Parameters | Fails with |
| --- | --- | --- |
| `element_visible` | `target`, `timeout_ms` | major (waits until it becomes visible) |
| `network_idle` | `timeout_ms` | minor |
| `field_value` | `target`, `value`, `normalize` (`text`\|`number`\|`date`\|`raw`), `tolerance` | critical |
| `collection_count` | `target`, `value` or `min`/`max` | major |
| `collection_order` | `target`, `value` (`asc`\|`desc`), `normalize` | major |
| `collection_aggregate` | `target`, `op` (`count`\|`sum`\|`min`\|`max`\|`avg`), `value`, `tolerance` | major |
| `pagination_state` | `value` (active page label, read from `aria-current`) | major |
| `response_status` | `url_contains`, `value` or `min`/`max` | critical |
| `response_body_contains` | `url_contains`, `value` | major (needs `VFA_RESPONSE_BODY_BYTES` > 0) |

Kinds outside this catalog are discarded by the plan validator and leave a
`warning` finding; they never downgrade a case to green.

## CLI: `vfa-qa` (pipeline gate)

The MCP surface is for conversational agents; `vfa-qa` is the surface for
pipelines. Both share the same engine, oracles and verdict.

```bash
vfa-qa audit --url https://app.dev --report out/report.json --junit out/junit.xml
vfa-qa flow  --url https://app.dev --step "escribe X en el campo Usuario" --step "haz clic en Entrar"
vfa-qa test  --url https://app.dev --objective "verifica el checkout con tarjeta valida" \
             --max-cases 3 --max-minutes 5 --fail-on major
vfa-qa report merge --inputs artifacts/*/report.json --out artifacts/report.json \
                    --junit artifacts/junit.xml
```

| Exit code | Meaning                                                              |
| --------- | -------------------------------------------------------------------- |
| `0`       | `verdict == passed`                                                  |
| `1`       | `verdict == failed` with a finding at or above `--fail-on`           |
| `2`       | `verdict == blocked` (ambiguous objective, environment, timeout)     |
| `3`       | Configuration error: invalid license, missing key, rejected URL      |

`--json` prints only the verdict JSON on stdout (captures are stripped);
`--allow-blocked` downgrades `2` to `0` for exploratory pipelines and never
degrades a real `1`. `--shard 2/4` splits cases deterministically by
`sha1(case_id) % total`; `VFA_RETRIES` retries only blocked runs.

## Configuration reference

VFA is configured **exclusively through environment variables** — there are no
config files, no database and no flags to pass. Variables are grouped in three
independent layers:

| Layer                    | Variables                                                                  | Consumed by                                    |
| ------------------------ | -------------------------------------------------------------------------- | ---------------------------------------------- |
| **Reasoning engine**     | `VFA_LLM_PROVIDER`, `VFA_LLM_MODEL`, `VFA_LLM_API_KEY`, `VFA_LLM_BASE_URL`, `VFA_LLM_MAX_TOKENS` | The **Deep Agent** (LiteLLM tool-calling loop) and the goal-mode planner |
| **Vision engine**        | `VFA_VISION_PROVIDER`, `VFA_VISION_MODEL`, `VFA_VISION_API_KEY`, `VFA_VISION_BASE_URL`, `VFA_VISION_MAX_TOKENS` | Visual grounding, `look`, CAPTCHA fallback and screenshot audit |
| **Browser & runtime**    | `HEADLESS`, `VFA_ALLOW_LOCAL_HOST`, `BROWSER_CDP_*`, `VFA_SESSION_*`, …    | Playwright adapter, session pool, Anti-SSRF    |

### Is anything mandatory?

Only **two things** are strictly mandatory:
1. `VFA_LICENSE_KEY`: Your commercial license key purchased from [Creem](https://www.creem.io/payment/prod_6qWpfXtntbKRUvBifbObZj).
2. An inference API key for your chosen provider (e.g. `OPENAI_API_KEY`, `DEEPSEEK_API_KEY`, etc.).

```bash
# Minimal production setup — works because the defaults are openai/gpt-4o
export OPENAI_API_KEY="sk-..."
```

### Does `VFA_LLM_MODEL` drive the Deep Agent?

**Yes.** `VFA_LLM_MODEL` is the model used by the Deep Agent — the component that
executes `qa_execute_user_flow` and performs the reasoning step of
`qa_audit_url`. Both tools go through LiteLLM with
`VFA_LLM_PROVIDER` + `VFA_LLM_MODEL`, so:

- **You may omit them** when you use OpenAI `gpt-4o` and export
  `OPENAI_API_KEY` (defaults: `openai` / `gpt-4o`).
- **You must set them** when you use any other provider, any other model, or an
  OpenAI-compatible gateway (DeepSeek, OpenRouter, Ollama, vLLM, Azure proxy…).
- The model **must support tool calling** (native function calling), because the
  ReAct loop sends a JSON action schema on every iteration.

The goal-mode planner uses the same `VFA_LLM_MODEL` for a single JSON planning
call. It is bounded by `VFA_LLM_TIMEOUT`, runs with `num_retries=0`, and after a
transport failure a five-minute circuit breaker serves the fallback plan instead
of paying the timeout again on every call.

For OpenAI-compatible endpoints, pair the model with a base URL:

```bash
export VFA_LLM_PROVIDER="openai"          # protocol, not the vendor
export VFA_LLM_MODEL="openai/deepseek-chat"
export OPENAI_API_KEY="sk-..."            # or VFA_LLM_API_KEY
export OPENAI_BASE_URL="https://api.deepseek.com"
```

### Credential and endpoint precedence

| What                 | Resolution order                                                                                                                          |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Reasoning API key    | `VFA_LLM_API_KEY` → provider standard variable (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `DEEPSEEK_API_KEY`, `XAI_API_KEY`, `OPENROUTER_API_KEY`, `GROQ_API_KEY`, `MISTRAL_API_KEY`, `AZURE_API_KEY`) → `OPENAI_API_KEY` |
| Reasoning base URL   | `VFA_LLM_BASE_URL` → provider standard base (`OPENAI_BASE_URL`/`OPENAI_API_BASE` for openai/azure, `ANTHROPIC_BASE_URL`, `OPENROUTER_API_BASE`, `XAI_API_BASE`, `OLLAMA_API_BASE`) |
| Reasoning max tokens | `VFA_LLM_MAX_TOKENS` → `1024`                                                                                                             |
| Vision provider      | `VFA_VISION_PROVIDER` → `VFA_LLM_PROVIDER` → `openai`                                                                                     |
| Vision model         | `VFA_VISION_MODEL` → `VFA_LLM_MODEL` → `gpt-4o`                                                                                           |
| Vision API key       | `VFA_VISION_API_KEY` → `VFA_LLM_API_KEY` when the vision provider is inherited or equals the reasoning provider → provider standard variable → `VFA_LLM_API_KEY` fallback → `OPENAI_API_KEY` |
| Vision base URL      | `VFA_VISION_BASE_URL` → `VFA_LLM_BASE_URL` when the vision provider is inherited or equals the reasoning provider → provider standard base → native endpoint |
| Vision max tokens    | `VFA_VISION_MAX_TOKENS` → `VFA_LLM_MAX_TOKENS` → `1024`                                                                                   |

`VFA_VISION_*` is therefore **optional by design**: if the reasoning model is
multimodal (`gpt-4o`, `claude-3-5-sonnet`, `gemini/gemini-2.0-flash`,
`xai/grok-2-vision-1212`), leave it empty and one model does everything. Set it
only when you want a **cheaper/faster model for screenshots**, when your
reasoning model is text-only, or when vision lives on a **different
provider/endpoint**.

> **Endpoint rule:** if both roles use the `openai` protocol but different
> endpoints, `VFA_VISION_BASE_URL` is mandatory. The vision role would otherwise
> inherit `VFA_LLM_BASE_URL`.

```bash
# Same OpenAI-compatible endpoint, two models
export VFA_LLM_PROVIDER=openai
export VFA_LLM_MODEL=openai/deepseek-v4-pro
export VFA_LLM_BASE_URL=https://api.deepseek.com
export VFA_LLM_API_KEY=sk-...
export VFA_VISION_PROVIDER=openai
export VFA_VISION_MODEL=openai/deepseek-flash
export VFA_VISION_BASE_URL=https://api.deepseek.com

# Vision on a different vendor: the endpoint is not inherited
export VFA_LLM_PROVIDER=deepseek
export VFA_LLM_MODEL=deepseek/deepseek-chat
export DEEPSEEK_API_KEY=sk-...
export VFA_VISION_PROVIDER=openai
export VFA_VISION_MODEL=gpt-4o-mini
export OPENAI_API_KEY=sk-...
```

### Full variable list

| Variable | Default | Required when | Purpose |
| :--- | :--- | :--- | :--- |
| `VFA_LICENSE_KEY` | _(empty)_ | Mandatory for all tool executions | Commercial Pro license key from [Creem](https://www.creem.io/payment/prod_6qWpfXtntbKRUvBifbObZj) |
| `VFA_LLM_PROVIDER` | `openai` | Using a non-OpenAI provider | SDK protocol/routing for the Deep Agent (`openai`, `anthropic`, `gemini`, `deepseek`, `xai`, `openrouter`, `ollama`; aliases `google`, `grok`) |
| `VFA_LLM_MODEL` | `gpt-4o` | Using a model other than `gpt-4o` | Reasoning model (must support tool calling) |
| `VFA_LLM_API_KEY` | _(empty)_ | Sharing one credential across providers | Generic key that overrides the provider standard variable |
| `VFA_LLM_BASE_URL` | _(empty)_ | OpenAI-compatible gateways, self-hosted or proxied endpoints | Base URL for reasoning; provider standard variables are also honoured |
| `VFA_LLM_MAX_TOKENS` | `1024` | Reasoning models or long JSON outputs | Output-token budget per reasoning completion (8192+ for reasoning models) |
| `VFA_VISION_PROVIDER` | inherits | Vision served by a different vendor | Routing for multimodal calls |
| `VFA_VISION_MODEL` | inherits | Reasoning model lacks image input, or cost split desired | Model used for screenshots and visual grounding |
| `VFA_VISION_API_KEY` | inherits | Vision served by a different vendor | Credential for multimodal calls |
| `VFA_VISION_BASE_URL` | inherits | Vision on a different endpoint | Base URL for multimodal calls; mandatory when both roles use `openai` with different endpoints |
| `VFA_VISION_MAX_TOKENS` | inherits | Vision needs a different budget | Output-token budget for screenshots; empty inherits `VFA_LLM_MAX_TOKENS` |
| `OLLAMA_API_BASE` | _(empty)_ | `VFA_LLM_PROVIDER=ollama` | Local endpoint, e.g. `http://localhost:11434` (no key needed, a placeholder is used) |
| `VFA_AGENT_MAX_ITERATIONS` | `12` | Long flows that exceed the ReAct budget | Iteration ceiling of the Deep Agent loop |
| `VFA_LLM_TEMPERATURE` | `0.0` | Never (deterministic QA by default) | Sampling temperature |
| `VFA_LLM_TIMEOUT` | `60` | Slow/self-hosted models | Per-call timeout in seconds |
| `VFA_LLM_REQUESTS_PER_SECOND` | `0` | Free tiers or strict quotas | Token-bucket rate limiter; `0` disables it |
| `VFA_LLM_CHECKS_PER_SECOND` | `10.0` | Never | Rate limiter polling frequency |
| `HEADLESS` | `true` | Local debugging of the flow step by step | Run Chromium without a window |
| `VFA_ALLOW_LOCAL_HOST` | `false` | Auditing `localhost`, RFC 1918 or `*.local` targets | Allow `localhost`, `127.0.0.1`, `0.0.0.0`, `*.local` and RFC 1918; keep it `false` on internet-facing hosts |
| `BROWSER_CDP_URL` | _(empty)_ | Attaching to an already running Chromium | Optional custom CDP endpoint; empty uses the managed local browser |
| `BROWSER_CDP_TIMEOUT_MS` | `2000` | Remote CDP endpoint slower than 2s | Timeout of the CDP connection attempt |
| `VFA_ACTION_TIMEOUT_MS` | `10000` | Heavy SPAs with slow first paint | Timeout of every individual Playwright action |
| `VFA_SCREENSHOT_FULL_PAGE` | `true` | Never (viewport-only reports needed) | Full-page screenshots instead of viewport only |
| `VFA_SESSION_POOL_ENABLED` | `true` | Stateless/multi-tenant deployments may disable it | Reuse browser sessions between tool calls |
| `VFA_SESSION_POOL_TTL` | `300` | Never | Idle seconds before a pooled session is closed |
| `VFA_SESSION_POOL_MAX_SIZE` | `5` | High-concurrency hosts | Maximum concurrent pooled sessions (LRU eviction) |
| `VFA_SESSION_ID` | _(empty)_ | Fixed key for CLI/CI runs | Default session key when the tool does not receive `session_id` |
| `VFA_OBSERVE_TEXT_CHARS` | `4000` | Very large pages | Visible-text cap inside `observation` |
| `VFA_OBSERVE_MAX_NODES` | `60` | Context-constrained hosts | Accessibility nodes included in `observation` |
| `VFA_MAX_INLINE_SCREENSHOTS` | `3` | Minimising host context | Inline captures per response; the rest degrade to paths |
| `VFA_TEST_MAX_CASES` | `5` | Cost control in goal mode | Cases kept from the planner |
| `VFA_TEST_STOP_ON_FAILURE` | `true` | Full plan coverage desired | Stop the plan on the first critical finding |
| `VFA_MAX_MODULES` | `6` | Deep inventories of large products | Cap for `report_inventory` and module traversal in goal mode |
| `VFA_ARTIFACTS_DIR` | _(empty)_ | CI evidence retention | Enable on-disk screenshots/report; empty keeps the sandbox in memory |
| `VFA_ARTIFACTS_MAX_FILES` | `200` | Disk-constrained runners | Artefact retention by age |
| `VFA_RETRIES` | `0` | Flaky environments | Retries **only** for blocked runs and timeouts |
| `VFA_LLM_SELECTOR_FALLBACK` | `true` | Fully reproducible CI runs | Allow the LLM to resolve selectors; `false` keeps runs deterministic |
| `VFA_BROWSER` | `chromium` | Cross-engine coverage | `chromium`, `firefox` or `webkit` (invalid values fall back to chromium) |
| `VFA_VIEWPORT` | `1280x720` | Mobile/webview coverage | Default viewport in `WxH` format |
| `VFA_MCP_TRANSPORT` | `stdio` | Remote dev containers | `stdio`, `http` or `sse` (http/sse require the security checklist and are experimental) |
| `VFA_MCP_TOKEN` | _(empty)_ | Non-loopback `http`/`sse` | Bearer token compared in constant time |
| `VFA_MCP_ALLOWED_ORIGINS` | _(empty)_ | Non-loopback `http`/`sse` | Explicit `Origin` allowlist |
| `VFA_SESSION_STORAGE_STATE` | _(empty)_ | Replaying an authenticated journey | Imported Playwright `storageState`; empty keeps cookies in memory only |
| `VFA_STORAGE_STATE_EXPORT` | `false` | The host needs the final session state | Returns `storage_state` in the flow payload with a warning; never written to disk |
| `VFA_WAIT_TIMEOUT_MS` | `5000` | Slow asynchronous pages | Default timeout of `wait_for` conditions and live visibility oracles |
| `VFA_UPLOAD_DIR` | _(empty)_ | Flows that upload files | Only directory the `upload` primitive may read; empty disables uploads |
| `VFA_UPLOAD_MAX_FILES` | `5` | Multi-file inputs | Files per upload step (hard cap 10) |
| `VFA_UPLOAD_MAX_BYTES` | `10485760` | Small quotas | Total size allowed per upload step |
| `VFA_STEP_LATENCY_MS` | `15000` | Latency-sensitive suites | Per-step budget; exceeding it adds a `minor` finding and `duration_ms` |
| `VFA_ORACLE_LATENCY_MS` | `10000` | Expensive live oracles | Per-oracle-batch budget |
| `VFA_RESPONSE_RECORD` | `true` | Privacy-restricted runs | Record response status/URL for the `response_*` oracles |
| `VFA_RESPONSE_BODY_BYTES` | `0` | `response_body_contains` needed | Bytes of XHR/fetch body kept in memory (never on disk) |
| `VFA_MAX_RESPONSES` | `50` | Very chatty pages | Responses kept per page (FIFO) |
| `SSL_VERIFY` | `true` | Self-signed certificates in local/staging | TLS verification switch; `false`, `0`, `no`, `off` or an empty value disables it for LiteLLM and Chromium |

A copy-ready template with the same defaults lives in [`.env.example`](.env.example).

### Production checklist

1. **One credential per deployment**, injected by your secret manager — never
   committed and never placed in the MCP client JSON for shared machines.
2. **Keep `VFA_ALLOW_LOCAL_HOST=false`** on any host that can reach an internal
   network; it is the only guard against SSRF from an LLM-driven navigation.
3. **Pin `VFA_LLM_MODEL`** explicitly instead of relying on `gpt-4o`; upstream
   providers rotate model names and a silent change alters QA verdicts.
4. **Set `VFA_LLM_REQUESTS_PER_SECOND`** when you run parallel audits against a
   rate-limited provider, and raise `VFA_LLM_TIMEOUT` for slow self-hosted models.
5. **Keep `HEADLESS=true`** on servers (no X server is available) and keep
   `SSL_VERIFY` enabled outside staging.
6. **Do not mount `VFA_SESSION_STORAGE_STATE`** unless the flow needs a reused
   login: it writes real session cookies to the host filesystem.

## Architecture

### Runtime view

```mermaid
flowchart TB
    subgraph Ports["app/ports - hexagonal contracts"]
        BrowserPort["BrowserSessionPort / BrowserGatewayPort"]
        LLMPort["LLMPort / UsageLimiterPort"]
    end
    subgraph Adapters["app/adapters - I/O only"]
        Playwright["PlaywrightAdapter<br/>Chromium sandbox"]
        Guard["network_guard<br/>Anti-SSRF firewall"]
        Pool["session_pool<br/>TTL and LRU"]
        Capture["capture + error_registry<br/>console, pageerror, HTTP 400+"]
        LiteLLM["LiteLLMAdapter<br/>reasoning and vision"]
        Grounding["vision_grounding<br/>visual fallback"]
        License["license_guard<br/>worker validation"]
    end

    Client["MCP client<br/>Zoo Code, Claude, Cursor, CI"] -->|"JSON-RPC over stdio"| Server["server_mcp.py<br/>FastMCP entrypoint"]
    Server --> Tools["app/tools/qa_tools.py<br/>the three published QA tools"]

    Tools --> Audit["Audit path<br/>screenshot, vision analysis, runtime errors"]
    Tools --> Goal["Goal mode<br/>qa_flow.run_goal_flow"]
    Tools --> Runtime["Runtime buffer<br/>read-destructive drain"]

    Goal --> Context["qa_goal.initial_context<br/>observe before planning"]
    Goal --> Planner["TestPlanner<br/>one LLM planning call"]
    Planner --> Plan["domain.test_plan<br/>fail-closed validation"]
    Plan -->|"not ready"| Fallback["qa_goal.fallback_plan<br/>one agentic case"]
    Plan -->|"ready"| Cases["qa_cases.execute_cases"]
    Fallback --> Cases

    Cases --> Deterministic["Deterministic steps<br/>domain.step_engine"]
    Cases --> Agent["DeepAgent.run_case<br/>snapshot, look, click, fill"]
    Deterministic -->|"step failed"| Agent
    Agent -->|"report_finding / report_inventory / finish"| Cases
    Cases --> Oracles["domain.oracles<br/>evaluate and aggregate"]

    Audit --> BrowserPort
    Audit --> LLMPort
    Context --> BrowserPort
    Deterministic --> BrowserPort
    Agent --> BrowserPort
    Agent --> LLMPort
    Runtime --> Capture
    Tools --> License

    BrowserPort --> Playwright
    BrowserPort --> Guard
    BrowserPort --> Pool
    BrowserPort --> Capture
    LLMPort --> LiteLLM
    LLMPort --> Grounding
    Pool --> Playwright
    Playwright --> Chromium["Chromium<br/>ephemeral context"]
    LiteLLM --> Provider["LLM provider<br/>OpenAI, DeepSeek, Anthropic"]
    Grounding --> Provider
    License --> Worker["License worker"]
```

### Layer responsibilities

| Layer | Rule |
| --- | --- |
| `app/domain` | Pure logic and contracts: no I/O, no frameworks, no browser or LLM imports |
| `app/ports` | Abstract ports consumed by the tools and implemented by adapters |
| `app/adapters` | All I/O: Playwright, LiteLLM, vision, license worker, filesystem artifacts |
| `app/tools` | Thin MCP surface plus goal orchestration and payload assembly |
| `server_mcp.py` | FastMCP entrypoint and transport only |

`tests/test_architecture.py` enforces the hexagonal boundaries and the
350-line-per-file budget.

### Repository layout

```
app/
  domain/                    # Pure domain: no I/O, no frameworks
    models.py                # AuditReport, StepResult, RuntimeErrorEvent
    parser.py                # Deterministic NLP parser (ES/EN)
    step_spec.py             # Structured step validation
    semantic.py              # regex -> a11y snapshot -> LLM fallback
    observation.py           # Observation and Finding contracts
    oracles.py               # Oracle catalog, evaluation, aggregate verdict
    test_plan.py             # Fail-closed TestPlan validation
    redaction.py             # Secret masking for payloads and steps
    vision.py                # Visual primitives and grounding contracts
  ports/                     # Abstract contracts
    browser.py               # BrowserSessionPort / BrowserGatewayPort
    llm.py                   # LLMPort / UsageLimiterPort
    license.py               # LicensePort
  adapters/
    browser/                 # Playwright sandbox, Anti-SSRF, error buffers, pool
    llm/
      litellm_adapter.py     # Multi-provider acompletion + base64 vision
      deep_agent.py          # ReAct loop over tool calling
      test_planner.py        # Goal -> TestPlan, fail-closed, circuit breaker
      agent_toolkit.py       # snapshot, look, report_finding, report_inventory
      agent_tools.py         # JSON schemas for the agent tools
      vision_reply.py        # Truncation-tolerant vision JSON parser
      rate_limiter.py        # Async token bucket
      visual_fallback.py     # Heuristic fallback chain
    license/                 # License guard and worker validation
  tools/
    qa_tools.py              # The three published QA tools
    qa_flow.py               # Goal orchestration and payload assembly
    qa_goal.py               # Initial context, inventory, fallback plan
    qa_cases.py              # Deterministic and agentic case execution
    qa_payloads.py           # Fail-closed payloads and redaction
    qa_session.py            # Session keys, run ids, observation
    qa_wiring.py             # Dependency injection
  config.py                  # Single source of truth for configuration
server_mcp.py                # FastMCP entrypoint (stdio/http/sse)
test_live.py                 # Live MCP-only end-to-end harness
```

## Security model

- Chromium starts with strict isolation flags (`--disable-background-networking`,
  `--disable-default-apps`, `--disable-sync`, `--no-first-run`, `--hide-scrollbars`).
- Every session uses an ephemeral in-memory `BrowserContext`, destroyed on close:
  no cache, cookies or storage are persisted to the host disk.
- A context-level Anti-SSRF firewall aborts (`accessdenied`) requests to loopback,
  private ranges (`10/8`, `172.16/12`, `192.168/16`) and cloud metadata
  (`169.254.169.254`). Local development targets are allowed only while
  `VFA_ALLOW_LOCAL_HOST` is `true`; set it to `false` for strict production mode.
- The LLM credential never reaches the audited page: only the server process
  talks to the inference provider.
- Payloads are redacted before they leave the server: `fill` values are masked
  in `steps[]`, keys matching `password`, `token`, `secret`, `cookie` or
  `api_key` are replaced, and free text patterns (`password=`,
  `Bearer <token>`, card numbers) are scrubbed.

## Development

```bash
pip install -e ".[dev]"
playwright install chromium
pytest
```

Confirm the tool surface before wiring a client: the server must expose exactly
`qa_audit_url`, `qa_execute_user_flow`, `qa_get_runtime_errors` and
`qa_get_screenshot`. `test_live.py`
asserts that surface as its first check, and any MCP client can list it with a
plain `tools/list` request. If your MCP client points at another checkout,
update its `command`, `args` and working directory to this workspace.

Tests run without a real browser or network: every adapter is replaced by an
in-memory double (`tests/conftest.py`, `tests/visual_fakes.py`). The suite
covers the goal pipeline (planner, fallback plan, case execution, inventory
merge), oracle evaluation including the fail-closed visual oracle, secret
redaction, the planner circuit breaker and the live harness itself against a
simulated MCP server.

### Live MCP-only test

`test_live.py` is a thin stdio client of this server: it spawns
`server_mcp.py`, calls only the three QA tools and asserts only on published
payload keys. It never synthesizes steps, selectors or labels, never calls a
vision model locally and never changes server behaviour through environment
variables (only `LITELLM_LOG=ERROR`, to keep stdout clean for JSON-RPC).

```bash
set VFA_LIVE_EMAIL=user@example.com
set VFA_LIVE_PASSWORD=secret
python test_live.py --url https://app.example.com
```

A single process runs the whole sequence: tool-surface check, fail-closed
contracts (`""` and `file://`), anonymous home audit, login by `goal` with
`data`, inventory by `goal`, fan-out with one `goal` per module returned in
`inventory[]`, authenticated audit and a read-destructive
`qa_get_runtime_errors` whose second call must be empty.

| Flag | Effect |
| --- | --- |
| `--url` / `VFA_LIVE_URL` | Target; required |
| `--timeout` | Timeout per MCP call, 420 s by default |
| `--max-minutes` | Time budget per flow |
| `--max-iterations` | Agent loop budget per flow |
| `--max-modules` | Inventory and fan-out cap, 6 by default |
| `--simple-only` | Contracts plus home audit only |
| `--skip-login`, `--skip-modules` | Narrow the scope without faking results |
| `--artifacts-dir` | Evidence directory, `live_artifacts` by default |

| Exit code | Meaning |
| --- | --- |
| `0` | Every check passed |
| `1` | At least one check failed |
| `2` | Preflight failed: missing URL, missing credentials or no `server_mcp.py` |
| `130` | Interrupted by the user |

Evidence lands in `live_artifacts/` (`live_report.json`, `live_report.md` and
`*.png`) and every row cites its payload path (`login.findings[0]`,
`traversal[1].cases[0].findings[0]`). There is no `SKIP`: missing credentials, a
blocked plan, an empty payload or a critical finding fails the run. Credentials
are required even with `--simple-only`, because preflight validates them before
reading the flag.

## License

Commercial, single-seat Pro license. Distributed on PyPI as `vfa-mcp`.  
Acquire your permanent license key at [Creem Checkout](https://www.creem.io/payment/prod_6qWpfXtntbKRUvBifbObZj).
