Metadata-Version: 2.4
Name: refuta
Version: 0.0.12
Summary: Refuta — Refutation-driven Agentic Code Security Platform for DGX Spark
Author: Refuta contributors
Author-email: "DAPADA Inc." <contact@dapada.co.kr>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/memtomem/refuta
Project-URL: Repository, https://github.com/memtomem/refuta
Project-URL: Issues, https://github.com/memtomem/refuta/issues
Keywords: security,code-audit,llm,agent,sast,refutation,sandbox,bubblewrap
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# Refuta — Refutation-driven, self-hosted agentic code security platform

**Refuta** (`refuta`) combines local LLM inference (developed on an NVIDIA DGX Spark with `nvidia/Qwen3.6-35B-A3B-NVFP4` served by vLLM on `:8000`) with deterministic code, scanners, and a `bwrap` isolation sandbox. It is an agentic security audit platform that **forms a hypothesis, tries to refute it, and reports only the vulnerabilities and patches that survive**.

The model's output is treated as a claim, not as evidence: every quote it cites is matched against the original file text, line numbers are computed by code, and a finding is only marked `refuted` when the model points to a guard that verifiably exists in the snapshot.

---

## 1. Requirements

- Linux, Python 3.11+. No runtime dependencies (standard library only).
- [bubblewrap](https://github.com/containers/bubblewrap) (`bwrap`) for isolated patch verification. Without it, tests fall back to a plain subprocess and the result records `sandbox_backend: subprocess`.
- For `--runner local`: an OpenAI-compatible endpoint with JSON-schema structured output (e.g. vLLM). Set `QWEN_BASE` (default `http://127.0.0.1:8000/v1`); the model is discovered from `/models`. The endpoint must be this host (`localhost`, `127.0.0.0/8` or `::1`, written literally) unless `--allow-external` is given, requests never go through an `http_proxy`/`https_proxy` from the environment, and an endpoint that does not answer or lists no model stops the audit before the snapshot (exit 1) instead of running every sample into a connection error under a guessed model name.

```bash
pip install refuta          # or, from a source checkout: uv sync --group dev
```

---

## 2. Package layout

```text
refuta/
├── pyproject.toml          # Package settings and the `refuta` CLI entry point
├── refuta/
│   ├── __init__.py
│   ├── schemas.py          # FindingStatus (9 states), RepoPolicy, ScannerCandidate, EvidenceQuote, CrossReviewRecord, ...
│   ├── repo_provider.py    # Immutable snapshots, .git credential stripping, untrusted-instruction-file quarantine, Loop 1 diff scope
│   ├── scanners.py         # Static/secret/authorization scanners, Semgrep/Gitleaks/OSV/Syft parsers, Loop 4 attack-surface analyzer
│   ├── sandbox.py          # bwrap (--unshare-all) isolated runner for pytest and scanners, SHA256 supply-chain integrity checks
│   ├── verifier.py         # Quote and line-number verifier, refutation adjudicator, cross-model review (option D), bwrap patch verifier
│   ├── external.py         # Semgrep and OSV-Scanner in Docker on a read-only tool view of the snapshot
│   ├── osv.py              # OSV results decided by code: manifest records, pins, trusted indexes
│   ├── gitleaks.py         # Gitleaks on the snapshot, value masking, containment checks at every output
│   ├── data/               # Vendored Gitleaks default config without repository-steerable allowlists
│   ├── baseline.py         # Finding fingerprints, per-run config, comparison with a baseline run
│   ├── origin.py           # AST proof that an outbound URL is built only from literals/environment reads (`not_applicable`)
│   ├── runners.py          # LocalRunner (:8000), ClaudeRunner, CodexRunner, FixtureRunner, RunnerRouter, memory guard
│   ├── store.py            # SQLite (WAL) store for evidence, state, cache, and accepted risk + code/deployment/SBOM graph
│   ├── loops.py            # Loop 3 (new-CVE impact), Loop 4 (attack-surface reduction), Loop 6 (post-deploy re-verification), quiet-window guard
│   ├── risk.py             # accepted-risk exceptions: keyed bindings, matching, `refuta risk` commands
│   ├── publisher.py        # GitHub webhook HMAC verification, outbound secret redaction, PR/SARIF publish payloads
│   └── cli.py              # `refuta scan | surface | audit | eval` CLI
├── sample/repo/            # Multi-file mini service for evaluation (includes a prompt-injection trap AGENTS.md)
├── patches/                # CASE-01-IDOR fix patch and verification manifest
├── fixtures/               # Canned responses for offline regression runs
├── expected.json           # Ground truth used only by the independent evaluator (`refuta eval`)
├── results/                # fixture / qwen / qwen-cached run outputs and metrics
└── tests/
    ├── test_refuta.py      # Unit, integration, and security-boundary regression tests
    ├── test_input_origin.py # Input-origin proofs and the shapes they must reject
    ├── test_external_scanners.py # Semgrep tool view, Docker command, result checks, pipeline identity and cache
    ├── test_osv.py         # OSV manifest records, requirements-set trust, lockfile sources, Docker command
    ├── test_gitleaks.py    # Gitleaks stop conditions, masking, canary containment, cache and fingerprint identity
    ├── test_local_diagnostics.py # Local-response failure and diagnostic privacy regressions
    ├── test_baseline.py    # Fingerprints, run config, baseline selection and matching, two-commit audit
    ├── test_containment.py # Sandbox host sockets and timeouts, snapshot cleanup, git environment, CLI refusals
    ├── test_evidence.py    # In-span confirmation, production guards, payload types, line numbering, patch reproduction
    └── test_runners.py     # No proxy, loopback endpoint, no guessed model, HTTP reasons, memory stop, manifest preflight
```

---

## 3. Benchmark and verification results

On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), the recorded run (`--samples 3`) found **4/4 real vulnerabilities (including 1 `fixed_verified` through bwrap-isolated patch verification)** and **cleared 3/3 safe cases with 0 false positives**: 2 `refuted` by verified upstream guards and 1 `not_applicable` (CASE-07, a module-level literal URL proven by the input-origin check without a model call). It had **0 quote or line-number evidence errors**, **0 raw secret leaks**, and the **untrusted `AGENTS.md` quarantined (`quarantine_verified: true`)**. At recording time, all **225 unit and regression tests passed** (`8.14s`).

| Mode | Vulnerabilities found | Safe cases cleared | False positives | Evidence errors | Secrets exposed | Model calls | Time |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline scanners only (reference) | 4/4 | 0/3 | **3** | — | 0 | 0 | 0.009 s |
| Canned responses (`fixture`) | 4/4 (`fixed_verified` 1) | 3/3 | **0** | 0 | 0 | 5 (stub) | 0.328 s |
| Real Qwen3.6 audit (`local`, `--samples 3`) | 4/4 (`fixed_verified` 1) | 3/3 | **0** | 0 | 0 | 15 | 105.341 s |
| Re-run from cached investigations (`qwen-cached`) | 4/4 (`fixed_verified` 1) | 3/3 | **0** | 0 | 0 | 0 (15 cache hits) | 0.279 s |

The `local` row is one recorded run. vLLM output is not bit-exact at temperature 0, so a single investigation can flip: on 2026-10-26, single-sample runs (`--samples 1`) matched the ground truth in 6 of 8 runs, and both failures left `CASE-02-GUARDED-ADMIN` `confirmed` instead of `refuted` (1 false positive). With `--samples 3` (strict-majority vote over independently adjudicated investigations), 10 of 11 runs matched; the other had no majority on CASE-02 and escalated it to `needs_review`. No run in either mode missed a vulnerability, produced an evidence error, or exposed a secret. CASE-07 is decided by the input-origin check before any model call, so it does not change these rates or the 15 model calls per `--samples 3` run.

- Real Qwen run: [report](https://github.com/memtomem/refuta/blob/main/results/qwen/report.md), [structured result](https://github.com/memtomem/refuta/blob/main/results/qwen/result.json), [SARIF](https://github.com/memtomem/refuta/blob/main/results/qwen/sarif.json), [metrics](https://github.com/memtomem/refuta/blob/main/results/qwen/metrics.json)
- Cached re-run: [report](https://github.com/memtomem/refuta/blob/main/results/qwen-cached/report.md) and [metrics](https://github.com/memtomem/refuta/blob/main/results/qwen-cached/metrics.json)
- Canned responses: [report](https://github.com/memtomem/refuta/blob/main/results/fixture/report.md) and [metrics](https://github.com/memtomem/refuta/blob/main/results/fixture/metrics.json)

---

## 4. CLI usage

Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip install refuta`, use `refuta` instead of `.venv/bin/refuta`.

```bash
# 1) All unit and integration tests
.venv/bin/pytest -q

# 2) Static, secret, and authorization scanners only
.venv/bin/refuta scan \
  --repo sample/repo \
  --out /tmp/refuta-scan

# 3) Loop 4 attack-surface analysis (unauthenticated endpoints, debug routes, privileged routes)
.venv/bin/refuta surface \
  --repo sample/repo \
  --out /tmp/refuta-surface

# 4) Full security audit on the local vLLM (Qwen3.6 :8000) + bwrap patch verification
.venv/bin/refuta audit \
  --repo sample/repo \
  --runner local \
  --samples 3 \
  --patches patches/manifest.json \
  --out results/qwen

# 5) Evaluate against the independent ground truth (expected.json)
.venv/bin/refuta eval \
  --result results/qwen/result.json \
  --expected expected.json \
  --repo sample/repo

# Offline, no model needed: --runner fixture --fixture fixtures/responses.json
```

`audit` defaults to 3 independent investigations per candidate for `--runner local`, with a strict-majority vote; other runners default to 1. Set `--samples N` to override either default (`N >= 1`). On an uncached audit, the local default uses about three times as many model calls as `--samples 1`. Existing sample-0 cache entries are reused; only missing samples require new calls.

Each investigation prompt holds the candidate file plus related files: guard-like paths (`security`, `auth`, `deps`, `middleware`, `main.py`) first, then files that import the module, then other files that mention it, then tests. At most `RELATED_FILES_MAX_COUNT` (12) files and `RELATED_FILES_MAX_BYTES` (160,000 bytes) are included, listed in path order; the number left out is recorded as `metadata.related_files_omitted` on the candidate.

Local investigations run sequentially, and remaining samples are still attempted after a failure. The HTTP timeout (`RepoPolicy.max_timeout_sec`, default 120 seconds) applies to each request rather than setting a deadline for the entire audit. A stalled endpoint can therefore cause three consecutive timeout waits per candidate with the local default. Use `--samples 1` to reduce the number of requests when troubleshooting the endpoint.

### External scanners (Semgrep)

`audit --semgrep-config PATH` (repeatable) also runs Semgrep on the snapshot with your local rule files or directories. Each result becomes a candidate under rule ID `semgrep:<check_id>` and is investigated like a built-in one. Semgrep runs only from the pinned image `semgrep/semgrep:1.180.0` (by digest), which Refuta never pulls:

```bash
docker pull semgrep/semgrep@sha256:529ee8a277ec8adc5b534d7c74eea0a47e9de21d62852b6ba7ac6ba9566845c3
.venv/bin/refuta audit --repo <repo> --runner local --semgrep-config ~/rules/python.yaml --out <out>
```

The container has no network, a read-only root filesystem, no capabilities, and runs as your user. It sees only read-only mounts of the rule paths and a hard-link view of the snapshot that leaves out `.semgrepignore`, `.gitignore`, `.semgrep.yml` and `.semgrep/`. The flags `--disable-nosem`, `--no-git-ignore` and `--x-ignore-semgrepignore-files` mean the audited repository cannot hide findings. Registry configs (`p/...`, `auto`, URLs), missing rule paths, a missing image or Docker, and running as root fail before the audit starts (exit 1). Results outside the snapshot or with line ranges past the end of a file, Semgrep's own errors, a timeout (`RepoPolicy.scanner_timeout_sec`, default 600 s), and output over 64 MB are recorded in `errors` and make the run `partial`. Rule paths are copied (regular files only) into a private directory at preflight; the config digest and the Docker mounts both use that copy, so editing a rule during the audit cannot run under an old digest. Semgrep severities map to Refuta's scale (`ERROR` → `high`, `WARNING` → `medium`, `INFO` → `low`), so `ERROR` findings are SARIF errors. `coverage.external_scanners` lists the image, rule paths, a config digest, the files left out of the tool view, and result counts. Design: `docs/design/external-scanners.md`.

### External scanners (OSV-Scanner)

`audit --osv --osv-db DIR` checks the snapshot's PyPI manifests (`*requirements*.txt`, `uv.lock`, `poetry.lock`) against an offline OSV database, using the pinned image `ghcr.io/google/osv-scanner:v2.7.0` (by digest) under the same container restrictions as Semgrep. `DIR` must hold `osv-scalibr/PyPI/all.zip` (the OSV data dump); online queries are not supported:

```bash
docker pull ghcr.io/google/osv-scanner@sha256:4f3b7e2c64b9d38a5febc5a7e4a3dfef1d1362f726dea7a2ef4762b46469fcd0
mkdir -p ~/.cache/refuta/osv-db/osv-scalibr/PyPI
curl -o ~/.cache/refuta/osv-db/osv-scalibr/PyPI/all.zip https://osv-vulnerabilities.storage.googleapis.com/PyPI/all.zip
.venv/bin/refuta audit --repo <repo> --runner local --osv --osv-db ~/.cache/refuta/osv-db --out <out>
```

OSV results never reach the model. Refuta passes each supported manifest explicitly (`-L`) with an empty or operator (`--osv-config`) config, so `.gitignore` and `osv-scanner.toml` in the audited repository cannot hide anything. Each advisory group (PYSEC and GHSA aliases of one issue) becomes one `osv:<id>` finding, decided by code against the manifest record that carries the package:

- `confirmed` only for an exact pin (`==` with a concrete version, or `===`) or a locked version from a trusted index: PyPI, or a mirror listed with `--osv-trusted-index URL`.
- `needs_review` for unpinned requirements (`>=`, `~=`, `==1.*`), editable, path, URL and git sources, other registries, and any requirements file when the requirements files and their includes use an untrusted index, an include outside the snapshot, or any option outside a short allowlist (so `--no-index`, `--find-links`, abbreviations and unknown options all count as untrusted).
- Never `invalid_evidence`: OSV ran on this snapshot, so when Refuta's reading of a manifest disagrees with OSV's (another pin, no record, a lockfile that does not parse), the advisory stays `needs_review` with the mismatch in its reason instead of being dismissed.

For a requirements file the finding's evidence is the record's line, re-read from the snapshot and masked. For `uv.lock` and `poetry.lock`, the status reason names the parsed entry (`uv.lock entry urllib3 1.26.4 from https://pypi.org/simple`); `tomllib` gives no line numbers, so these findings point at line 1 with no quote. Index URLs in reasons and coverage are shown without user info or query. Whether the vulnerable code is reachable is not assessed. Lockfiles of other ecosystems (`package-lock.json`, `go.mod`, ...) are listed in `coverage.external_scanners.osv.unsupported_manifests` and make the run `partial`.

### Gitleaks and secret masking

`audit --gitleaks` runs Gitleaks on the snapshot before anything else reads it, from the pinned image `zricethezav/gitleaks:v8.24.3` (by digest) under the same container restrictions as Semgrep. Every value Gitleaks or the built-in secret rules find is then replaced by `*` in the snapshot itself, at every occurrence and in every file (same length, newlines kept, so line numbers do not change). Prompts, Semgrep, OSV, quote verification and every output only ever see the masked text:

```bash
docker pull zricethezav/gitleaks@sha256:e1b35e12a8c6fa8901f060459cfb6b2fc4c484d3afbe3b029733a3bbfab07055
.venv/bin/refuta audit --repo <repo> --runner local --gitleaks --state <fresh-state.db> --out <out>
```

- **The repository cannot steer Gitleaks.** Refuta passes its vendored copy of the pinned version's default config (`refuta/data/`, sha256 pinned) with every allowlist that looks at paths or at the surrounding line removed, so code under `node_modules/`-style or `gitleaks.toml`-like paths, or on a line ending in `# --mount=type=secret,`, is still scanned. `.gitleaks.toml`, `.gitleaksignore` and `gitleaks:allow` comments in the repository have no effect. `--gitleaks-config FILE` uses your own config instead; its results are all `needs_review`.
- **If masking cannot be established, the audit stops** (exit 1) before any prompt, cache entry or finding is written. That covers a Gitleaks failure or timeout, an unparseable report, a result without a value or outside the snapshot, a value that occurs in a file or directory name (also of skipped, quarantined and unreadable paths), a rule ID, a status or another identifier. The message names the cause only.
- **Findings are decided by code**: `gitleaks:<rule>` with a keyed hash (`REDACTED_HMAC:<12 hex>`, key per state database, so short or guessable values cannot be reversed from reports). `confirmed` only for a reviewed list of fixed-format rules (private keys, GitHub, GitLab, Slack, AWS, GCP, Stripe, ... tokens) with the vendored config; `generic-api-key`, operator rules and results whose value is not in the reported lines are `needs_review`, which SARIF exports as `note` and `--fail-on-new` counts. A value the built-in rules already report stays one built-in finding.
- **Only `--runner fixture` and `--runner local`** are allowed: `claude` and `codex` read files with their own tools. Patch verification is skipped whenever anything was masked (`coverage.external_scanners.gitleaks.patch_verification`), so `eval` on the benchmark needs a run without `--gitleaks`.
- As a backstop, values in prose fields (titles, reasons, quotes, errors, cached payloads) are replaced, the rendered outputs and every string bound to SQLite are checked before they are written, and diagnostics mask escaped spellings too. Data written to an existing `--state` by earlier runs without `--gitleaks` is not rewritten; use a fresh `--state` when that matters.

Without `--gitleaks` nothing changes: cache keys, fingerprints and outputs are as before. Design: `docs/design/gitleaks-scanner.md`.

### Baseline comparison

Every audit fingerprints its findings and records its configuration, so a later audit of the same repository can compare against it. With `--baseline RUN_ID`, or `--baseline-commit SHA` (the newest `completed` run of the repository at that commit; 7 or more hex digits), the audit labels each reportable finding (the SARIF set: `confirmed`, `fixed_verified`, OSV `needs_review`) `new` or `existing`:

```bash
.venv/bin/refuta audit --repo <repo> --state ~/.cache/refuta/state.db --out <out-main>        # on main
.venv/bin/refuta audit --repo <repo> --state ~/.cache/refuta/state.db --out <out-pr> \
  --baseline-commit <main-sha> --sarif-new-only --fail-on-new                                # on the PR
```

The baseline comes only from the `--state` database (the default database is new for every `--out`), never from a report file. It must be of the same repository (`repo_id`) and written by 0.0.6 or later; anything else exits 1 before the snapshot. The full snapshot is always audited: a verdict in an unchanged file can depend on a changed one, and the investigation cache already makes unchanged candidates free.

A fingerprint is the rule, the file, and the candidate's lines read from the snapshot with only line endings normalized (secrets: the masked hash; OSV: the package name, so a version bump keeps it). Line numbers are not part of it, so moved code stays `existing`; any edit inside the flagged span, including re-indentation, makes it `new`. Matching errs towards `new`: a finding is `existing` only when its fingerprint occurs exactly once in each run and the baseline's match was reportable too (a `refuted` finding that is now `confirmed` is new). `result.json.baseline` lists the baseline's reportable findings that are gone, in two groups: `resolved` when the same fingerprint now has a settled verdict (`refuted`, `not_applicable`, `accepted_risk`), and `not_reported` for every other case, with no claim about why (fixed, edited, inconclusive now, or not scanned). `baseline.warnings` names each configuration field that differs (model, prompt version, scanner digests, ...); a difference never makes a new finding existing. SARIF results carry `partialFingerprints` (`refuta/v1`) and, with a baseline, `baselineState` (`new` or `unchanged`); `--sarif-new-only` exports only new ones. `report.md` adds a "Compared with baseline" section. Design: `docs/design/baseline-comparison.md`.

`repo_id` scopes baselines and risk exceptions in the state database. By default it is the directory name, or `<checkout>/<path>` for a subdirectory of a git checkout (so `services/api` and `tools/api` of one monorepo stay apart); `audit --repo-id ID` sets it explicitly (1-200 of `A-Za-z0-9._/:@-`), for example when CI checks the repository out under a varying directory name. `config.repo_id_source` records which (`directory`, `git-subdir`, `--repo-id`).

`audit` exits 0 when the run completes and 2 when any investigation sample fails (`partial`), even if the other samples reach a strict majority and decide the finding. With `--fail-on-new` (needs a baseline), it exits 3 when any finding is new, ahead of 2: `result.json.status` still says `partial`. Check `result.json.errors` for failed samples and each finding's `sample_statuses` for the vote. A refusal before or during the snapshot (a runner the policy blocks, a missing path or `--fixture`, a repository git cannot read faithfully, bad options) prints one `refuta audit: ...` line and exits 1. `scan` and `surface` exit 1 on the same snapshot refusals and 2 when paths could not be read (`unreadable_files`). `eval` exits 1 when the result does not match the ground truth.

### Accepted risk

`refuta risk` accepts one reportable finding (`confirmed`, or an OSV / Gitleaks `needs_review`) of a stored run, for at most 90 days. A finding with a failed investigation sample is refused: that sample is investigated again by the next audit, which would release the exception, so re-audit with the same `--state` first. It reads only the state database:

```bash
.venv/bin/refuta risk accept --state ~/.cache/refuta/state.db --run <run_id> --finding FIND-CAND-004 \
  --approver sec-team --reason "internal tool; host fixed by config" --expires 2027-01-15
.venv/bin/refuta risk list --state ~/.cache/refuta/state.db [--repo-id ID] [--all]
.venv/bin/refuta risk renew EXC-… --state ~/.cache/refuta/state.db --run <run where it applied> \
  --approver sec-team --reason "reviewed again" --expires 2027-03-01
.venv/bin/refuta risk revoke EXC-… --state ~/.cache/refuta/state.db
```

An exception is bound to `risk_binding`, a keyed digest (per state database) of every input the verdict consumed: the rule, the candidate span, the claim (Semgrep rule digest, Gitleaks and OSV configuration), each sample's prompt and response, the original and post-mask hashes of every file the decision read (related files, files any sample quoted, input-origin writers, the OSV manifests and their includes), or the whole snapshot for the `claude`/`codex` runners. It applies to a later finding only when that digest occurs once, the status equals the accepted one, and it has not expired; it is matched after patch verification, so a `fixed_verified` finding stays visible. A re-audit against the same `--state` with unchanged inputs reuses cached responses and keeps the binding; any change releases the exception, and the finding's `status_reason` says so. Accepted findings are `accepted_risk`: out of SARIF and `--fail-on-new`, and `resolved` against a baseline where they were reportable. Every string a `risk` command writes or prints is checked against the values masked audits of the repository found (`known_values` in the state database); a hit refuses the command and prints nothing that holds the value. Context Refuta never read (another module that decides exposure) is not bound: name such assumptions in `--reason`. Design: `docs/design/risk-exceptions.md`.

### Local-response diagnostics

A local generation can loop (typically on whitespace, which the JSON grammar allows between tokens) until `max_tokens`. When such a response does not parse, it is reported as stage `response_truncated` instead of `investigation_json`. The local runner sends the same request once more after a truncated response (`TRUNCATION_RETRIES`, 1) before recording the sample as failed; `result.json` `usage.truncation_retries` counts those extra requests, separately from `model_calls`. The schema deliberately sets no `maxLength`: with the local vLLM/xgrammar backend a length-bounded string cannot contain escape sequences, which cuts code quotes at their first `"` or newline.

When a local model response has invalid UTF-8, malformed JSON, an invalid response envelope, or investigation content that is not a JSON object, the audit automatically saves a diagnostic under `<out>/diagnostics/<run_id>/<candidate_id>-sample<N>.json`. The sample number is zero-based. Each entry in `result.json.errors` points to its diagnostic; the same error list is saved in the audit's SQLite record. Failed responses are not cached. Successful responses create no diagnostic files, and a diagnostic write failure is recorded while the audit continues as `partial`.

Diagnostics contain the model, failure stage, response text, completion reason and token usage when available, and candidate/rule/sample identity. JSON errors also include the exact parser input after redaction, with character position and one-based line/column coordinates. Secrets in that parser input use equal-length `*` masks so the coordinates remain accurate after code-fence or reasoning-block removal and fallback extraction. Other diagnostic text uses the existing hashed secret markers, including for tokens split by Unicode escapes inside nested JSON or following short escapes such as `\n`. Diagnostic masking conservatively recognizes token prefixes within surrounding text and extends masking beyond scanner length caps; it can redact more text than the source scanner. The serialized diagnostic is checked again before writing.

Diagnostic directories are restricted to the owner (`0700`), files use `0600`, and symlink directories are rejected. Diagnostics are ignored by Git. Redaction covers the existing GitHub-token, `sk-`, and AWS-access-key families; truncated tokens or tokens interrupted by invalid encoding may not match. A field with unresolved escape obfuscation is replaced entirely with a redaction marker; parser input instead uses stars while preserving its length and line breaks. Diagnostics can still contain source code or other sensitive values and should be handled as sensitively as the audited snapshot. These permissions apply to diagnostics, not to the audit's other output files. HTTP errors and timeouts retain their existing error handling and create no diagnostic; an HTTP error's message carries the server's reason (vLLM's `error.message`, masked, at most 300 characters), such as a context-length overflow. Responses over 8 MiB are rejected as stage `response_size`.

---

## 5. Security and verification boundaries covered by tests

1. **Untrusted instruction-file quarantine and credential stripping (`RepoProvider`)**: a git checkout (including a subdirectory of one) is snapshotted from the tree of its captured `HEAD` commit, read object by object through `git cat-file --batch`; every commit, tree and blob is re-hashed against its object ID, so the snapshot matches `commit_sha` exactly. Git runs with an empty protocol allowlist (`GIT_ALLOW_PROTOCOL=""`), `GIT_NO_LAZY_FETCH`, `--no-replace-objects` and `core.fsmonitor=false`, so repository-controlled config cannot fetch, run transport commands or substitute objects, and without any inherited `GIT_*` variable, so a `GIT_DIR` or `GIT_WORK_TREE` from a git hook or the shell cannot point it at another repository; a repository git refuses because another user owns it (`safe.directory`) aborts with `RepoIntegrityError` instead of being copied from its working tree, and malformed trees or commits with valid object IDs do too; missing objects are reported as unreadable, and integrity or protocol failures (or a 300 s deadline) abort the snapshot with `RepoIntegrityError`. A directory the commit does not track is copied from disk under a `tree-…` id instead of the repository's commit; only a missing `git` executable, a non-git directory or a repository without commits falls back to the filesystem copy, and any other git failure raises. uncommitted and untracked files are not audited, and symlinks and submodules in the tree are skipped. Other directories are copied from the filesystem. `coverage.snapshot_source` records which (`git-head` or `filesystem`). When a snapshot is created, tokens in `.git/config` are removed and agent instruction and tool-config files are quarantined automatically: `AGENTS.md`, `AGENTS.override.md`, `CLAUDE.md`, `CLAUDE.local.md`, `GEMINI.md`, `.mcp.json`, `.cursorrules`, `.windsurfrules`, `.clinerules`, `copilot-instructions.md`, `.github/instructions/**/*.instructions.md`, and everything under `.claude/`, `.codex/`, `.cursor/`, `.agents/`, `.gemini/`, `.windsurf/` and `.clinerules/` (names compared case-insensitively). A snapshot that fails for any reason is removed before the error propagates, and the audit selects its runner (and applies the runner policy) before creating one, so a refused run leaves no copy of the repository behind. Only regular files are copied: symlinks (to files or directories, inside or outside the repository, including dangling ones), FIFOs, other special files, and files with more than one hard link are skipped and listed in `skipped_non_regular_files` (in `result.json` `coverage` and in `scan` output), so a link cannot pull host files into the snapshot or the prompt. In the filesystem copy, entries are opened relative to their parent directory's descriptor with `O_NOFOLLOW` and re-checked with `fstat` (same inode as the `lstat`), so this holds even if a path, including the repository root or one of its ancestors, is swapped for a link while the snapshot is being copied, and a directory swapped for another directory is reported as unreadable. The filesystem copy is meant for directories without git history that the operator controls; for repositories you do not trust, audit a git checkout, whose snapshot never reads the working tree. Paths that exist but cannot be read (for example, permission denied) are listed in `coverage.unreadable_files` and recorded in `errors`, which makes the audit `partial` (exit 2).
2. **Loop 1 diff scope (`compute_diff_scope`)**: computes exactly which files were added, modified, or deleted between two snapshots.
3. **Immediate secret masking (`scanners.py`)**: `ghp_...`, `sk-...`, and `AKIA...` values are replaced with `REDACTED_SHA256:<12 hex>` as soon as they are detected, every distinct value on a line is its own candidate and is masked in every snippet, and longer values are masked before values that are their prefix; tests verify that no raw value reaches prompts, SQLite, reports, or SARIF.
4. **External scanner parsers (`parse_semgrep_json`, `parse_gitleaks_json`, `parse_osv_scanner_json`, `parse_syft_cyclonedx_json`)**: normalize Semgrep, Gitleaks, OSV-Scanner, and Syft CycloneDX SBOM output into the standard schema, masking secrets automatically.
4a. **Semgrep on the snapshot (`external.py`, `audit --semgrep-config`)**: the docker command has only read-only mounts, no network and a pinned digest; the tool view omits suppression files at any depth; registry configs, missing images and root are refused before the audit. Every result must name a snapshot file and a line range inside it, its snippet is re-read from the snapshot (not taken from the report), and its message is masked. Imported rule IDs are namespaced (`semgrep:`), so built-in behaviour keyed by a rule ID (secret adjudication, the input-origin proof, patch verification) never applies to them and a built-in risk exception never covers them. Their cache key adds a digest of the claim and the rule files, while built-in keys stay unchanged. An opt-in test runs the real image and checks that `nosemgrep`, `.semgrepignore` and `.gitignore` in the audited repository suppress nothing.
4b. **Gitleaks containment (`gitleaks.py`, `audit --gitleaks`)**: canary values in a flagged function, on a built-in token's line, repeated in an unreported file, under 8 bytes, in an unlocated result, in an operator rule description, in a Semgrep message, or echoed by the model reach no prompt, no output file and no SQLite row (WAL included). Each stop condition (tool failure, timeout, bad report, value in a path, rule ID or status) ends the audit before any prompt is built. A rotated secret keeps the cached investigation but is `new` against a baseline; runs without values keep today's keys and fingerprints. An opt-in test runs the real image against repository allowlists, `.gitleaksignore`, `gitleaks:allow` and allowlisted paths.
5. **Deterministic evidence verifier (`verifier.py`)**: checks that the `vulnerable_quotes` and `guard_quotes` the model cites exist verbatim in the original files, and computes line numbers itself. Missing files, path escapes (`../`), and altered code quotes are rejected immediately as `invalid_evidence`. A quote is only checked to exist, so `confirmed` also needs at least one verified vulnerable quote at the flagged lines of the candidate's file (others, such as an entrypoint, only support it); without one the finding is `needs_review`. When a quote occurs more than once, its lines come from the occurrence inside the flagged span, so the finding points at the flagged call rather than an earlier `def` or constant. Line numbers count newlines only, as Python, Semgrep and Gitleaks do: a form feed or U+2028 does not shift them. Before a payload is cached or adjudicated, its types are checked for every runner (an object, a boolean `is_refuted`, lists of quote objects with string fields); anything else is a failed sample, never cached, and an unusable cached row is investigated again.
6. **Strict refutation**: even if the model claims `is_refuted: true`, the refutation is rejected (`needs_review`) unless it presents guard-code quotes (`guard_quotes`) verified against the snapshot files, at least one of them outside test code (`tests/`, `test/`, `test_*`, `*_test.py`, `conftest.py`): a test asserting the right behaviour guards nothing.
6a. **Input-origin refutation (`origin.py`, `PY-SSRF-URLOPEN` only)**: before any model call, the verifier tries to prove from the candidate file's AST that every `urlopen` URL is built entirely from string literals and module-level constants. Which binding a name refers to comes from the standard library's `symtable`: parameters, local variables, closures, lambdas and comprehensions are not followed, and each module constant must be bound exactly once by a plain assignment (no `global`, import or star import). The whole URL must be trusted, not just its host, because `urlopen` follows redirects: an attacker-chosen path or query on a fixed host can still reach another host through an open redirect. Only the standard library's `urlopen` counts as the sink (a custom `client.urlopen`, one replaced from another module, or one a repository `urllib.py`/`urllib/__init__.py` could shadow goes to the model), and only a `Request(...)` written inline at the sink is unwrapped, with its body, headers and method trusted as well, since they can steer a redirect like the URL (header dicts only when written inline in the call; `**kwargs` rejects); a named one is mutable (`req.full_url = ...`) and is not followed. A proven candidate ends `not_applicable` with its sink and assignments as verified `input_origin` evidence, makes no model call, and is excluded from SARIF; anything else goes to the model as before, with the secret-masked rejection reason kept in `input_origin.verification_error`. A module-level name must also not be overridden by non-test code anywhere in the snapshot (`settings.BASE = url`, `setattr(settings, "BASE", url)`). `audit --trust-environment` (`RepoPolicy.trust_environment=True`) also accepts environment reads with a literal, non-CGI name; it is opt-in because Python code can rewrite its own environment in ways a static check cannot rule out, so the operator vouches for it. The proof assumes honest code that may contain mistakes, as the text-matching scanner does; deliberately disguised writes (`globals()[...]`, computed-name `setattr`, replacing `os.getenv`) are out of scope. Design: `docs/design/input-origin-refutation.md`.
7. **Option D cross-model review (`adjudicate_cross_review`)**: adjudicates agreement and disagreement between the primary analyst and a second refuting reviewer, switching to `refuted` only when the refuting side presents a valid guard-code quote.
8. **Sample voting (`adjudicate_samples`, `audit --samples N`)**: runs N independent investigations per candidate, adjudicates each one separately, and keeps a status only when a strict majority reaches it; anything else escalates to `needs_review`. A sample counts as `refuted` only if its guard quote verified, so a single-sample refutation without evidence cannot win the vote. Per-sample statuses are kept in `sample_statuses`, and each sample has its own cache entry. Cache entries are keyed by candidate as well as file, including the candidate's line span, so two matches of one rule in the same file never share an investigation (caches written by 0.0.3 and earlier are re-investigated once).
9. **`bwrap` (bubblewrap) isolated patch and scanner runner (`sandbox.py`)**:
   - Isolates test and scanner processes with `/usr/bin/bwrap` (`--unshare-all --die-with-parent --new-session --ro-bind / / --tmpfs /run --tmpfs $HOME --ro-bind <venv> <venv> --ro-bind <base interpreter under $HOME, e.g. uv Python> --tmpfs /tmp --ro-bind <work_repo> /tmp/work --clearenv`).
   - Demonstrated inside the sandbox: (1) `$HOME` is not accessible, (2) the source under `/tmp/work` cannot be written (`Read-only file system`), (3) neither the host's `127.0.0.1:8000` nor external networks are reachable (`Network is unreachable`), (4) `/run` is empty, so host service sockets such as Docker's (`/run/docker.sock`, root-equivalent for members of the `docker` group), D-Bus and `/run/user/<uid>` agents cannot be reached: a read-only bind of `/` would not stop `connect()` on them.
   - Whether bwrap can create a network namespace is probed once with a trivial command; the nested backend (`bwrap-nested`, no network namespace) is chosen only from that probe, never from text the audited tests print. Tests run with stdin from `/dev/null` in their own session, and a timeout kills the whole process group; a patch whose tests time out ends `needs_review` (`sandbox_backend: timeout`) instead of aborting the audit.
   - A finding is promoted to `fixed_verified` only when all four conditions hold: (1) test files and test infrastructure are unmodified (`tests/`, `test/`, `test_*`, `*_test.py`, `conftest.py`, `pytest.ini`, `pyproject.toml`, `setup.cfg`, `tox.ini`, `sitecustomize.py`, `usercustomize.py`, `*.pth`, a root `pytest.py`), (2) the security test fails before the patch (the vulnerability reproduces: pytest exits 1 and reports failed tests; a collection error, a usage error or nothing collected is not a reproduction), (3) the security test passes after the patch, (4) the functional regression tests pass after the patch.
   - Execution is blocked when an external binary or model weight does not match its pinned SHA256 digest (`verify_artifact_sha256`).
10. **Default deny for sending code off-host, and a memory guard (`RunnerRouter`)**: with `local_only=True` (the default), `ClaudeRunner` and external `CodexRunner` calls are blocked; local inference is refused with `InsufficientMemoryError` when `MemAvailable` is below the threshold (`min_mem_available_gb`, default 8) or cannot be read, and that stops the audit (exit 1) instead of failing every remaining sample. The Claude runner sends its prompt on stdin, since Linux caps one command-line argument at 128 KiB. A `--patches` manifest is read and checked (required keys, readable patch files, no `..` targets) before the snapshot, so a typo no longer surfaces after every model call.
11. **Accepted-risk binding, expiry and containment (`risk.py`, `tests/test_risk_exceptions.py`)**: an exception covers only the finding it was created for (not another finding of the same rule in the file, not a verdict that escalated, not one whose binding another finding shares), is released when it expires or any input of the verdict changes (the file, a related or quoted file, the response, the post-mask bytes), never hides a `fixed_verified` finding, and never writes or prints a value an earlier masked audit found. Library-created exceptions without a binding keep the older rule: `confirmed` findings of the same rule and file while the file's SHA256 is unchanged.
12. **Continuous security loops and publishing safeguards (`loops.py`, `publisher.py`)**:
    - **Loop 3 (new-CVE response)**: queries the deployment/SBOM graph to identify affected services, environments, and runtime images immediately, without any code change.
    - **Loop 4 (attack-surface reduction)**: automatically identifies unauthenticated public routes, debug endpoints, and privileged admin paths.
    - **Loop 6 (post-deploy re-verification)**: fails while a pre-patch commit is still running in production, and completes only once the patched commit is deployed and passes its health check.
    - **Quiet-window guard (`QuietWindow`, `is_in_quiet_window`)**: tells heavy scheduled audits when to yield the host, e.g. to another model server sharing the GPU. Windows are configured by the caller (time zone, start/end, weekdays); the test uses a DGX Spark schedule (Mon/Thu 03:50–09:00 KST, daily 06:50–07:30 KST) as its example.
    - **Publishing safeguards**: GitHub webhook HMAC-SHA256 signature verification, blocking publication on a commit SHA mismatch (`StaleCommitPublishError`), and automatic redaction of secret tokens in outbound text.
13. **Local-response diagnostics (`tests/test_local_diagnostics.py`)**: redacts supported tokens (including overlapping patterns and Unicode escapes), preserves parser coordinates, separates runs, restricts file permissions, rejects symlink directories, and keeps partial audits usable when diagnostic writes fail. Non-object local responses are rejected before caching, and malformed legacy cache entries are re-investigated.

---

## License

[Apache License 2.0](https://github.com/memtomem/refuta/blob/main/LICENSE). Copyright 2026 DAPADA Inc. and Refuta contributors.
