Metadata-Version: 2.4
Name: control-coverage
Version: 0.1.0
Summary: Control-first coverage and blind-spot analysis over an evidence corpus, with a Statement of Applicability.
Author: Audit Labs
License: GPL-3.0-or-later
Project-URL: Homepage, https://audit-labs.dev
Project-URL: Repository, https://github.com/audit-labs/control-coverage
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Dynamic: license-file

# control-coverage

[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)]()

Control-first coverage and blind-spot analysis over an evidence corpus.

The rest of the Audit Labs toolchain is **evidence-first**: [audit-tools](https://github.com/audit-labs/audit-tools)
collects raw signals, [audit-report](https://github.com/audit-labs/audit-report)
maps each finding onto the controls it touches, and [evidence-seal](https://github.com/audit-labs/evidence-seal)
proves the package is authentic. That answers *"what did I collect, and what does it
map to?"* — but it can never tell you what you are **not** looking at, because it has
no list of everything a framework requires.

`control-coverage` supplies that missing list — the **denominator**. It starts from
the *complete* catalog of a framework's controls and scores your evidence against it,
so it can report two numbers nothing else in the pipeline can:

- **Coverage %** — of everything the framework requires, how much the evidence corpus
  addresses at all.
- **Blind spots** — the in-scope controls that *no* finding touches. These are the
  gaps an auditor finds for you if you don't find them first.

It also produces a **Statement of Applicability** — the ISO 27001 artifact that lists
every Annex A control, whether it applies, and why — derived from your evidence
instead of hand-maintained.

> Like every Audit Labs tool, this produces *evidence*, not a verdict. An unaddressed
> control is a gap in *evidence*, which may reflect a real gap in *controls* or simply
> a signal not yet collected. The final judgment belongs to the organization and its
> auditor.

## Install

```bash
git clone https://github.com/audit-labs/control-coverage
cd control-coverage
python -m venv .venv && source .venv/bin/activate
pip install -e .
```

Pure standard library plus PyYAML — no other dependencies.

## Usage

The input is one or more JSON reports from `audit-report` (its `--format json`
output). A corpus is typically one report per platform and date — AWS, GitHub,
GitLab — which `control-coverage` folds into a single per-framework picture.

```bash
# Coverage across every framework the corpus cites, Markdown to stdout
control-coverage aws.json github.json

# Just the blind spots — the controls nothing evidences yet
control-coverage aws.json github.json --framework SOC2 --blind-spots

# A whole directory of reports, all formats into ./out/
control-coverage ./reports/ --format md,html,json,soa --out out/

# Gate CI: exit non-zero if any framework's coverage is under 60%
control-coverage ./reports/ --fail-under 60
```

### Trend — how coverage moved

Point `--baseline` at an earlier corpus (a file or a directory) to see what changed:
controls that improved, regressed, and — the two that move the coverage number —
were *gained* (a blind spot became addressed) or *lost* (an addressed control became
a blind spot).

```bash
control-coverage ./reports/2026-02/ --baseline ./reports/2026-01/ --framework SOC2

# Gate CI: fail the build if any control regressed or lost coverage
control-coverage ./reports/2026-02/ --baseline ./reports/2026-01/ --fail-on-regression
```

Trend mode outputs Markdown, HTML, or JSON (`--format md,html,json`).

### Crosswalk — evidence leverage and the minimal set

One check is rarely worth one control: enforced 2FA is evidence for SOC 2 CC6.1,
ISO A.5.17, and NIST IA-2 at once. `--crosswalk` shows that leverage per check and
computes the **minimal evidence set** — the fewest checks that still touch every
addressed control, which is what you want when scoping a walkthrough or a sample.

```bash
control-coverage ./reports/ --framework SOC2,ISO,NIST --crosswalk
```

Crosswalk mode outputs Markdown, HTML, or JSON (`--format md,html,json`).

### Scope and the Statement of Applicability

Not every control applies to every organization. A **scope file** records which
controls are excluded and — required, never optional — *why*:

```yaml
# soa.yaml
subject: Acme Production
frameworks: [SOC2, ISO]
exclusions:
  - control: ISO:A.7.1
    reason: "Fully cloud-hosted; no physical premises are in scope for the ISMS."
  - control: ISO:A.5.7
    reason: "No formal threat-intelligence program; risk accepted by the CISO for 2026."
owners:
  SOC2:CC6.1: platform-team
```

```bash
# Coverage over in-scope controls, plus a ready-to-file SoA
control-coverage ./reports/ --scope soa.yaml --format md,soa --out out/
```

Excluded controls are recorded with their justification rather than counted as gaps.
An exclusion with no reason is rejected — an unjustified exclusion is the single most
common SoA audit finding.

## Assurance states

Every in-scope control lands in exactly one state:

| State | Meaning |
| --- | --- |
| **supported** | At least one mapped finding passes, and none fail. |
| **failing** | At least one mapped finding fails. The worst observation wins. |
| **asserted** | Findings map here, but their data was absent — evidence attempted, not obtained. |
| **unaddressed** | No finding maps here at all. **The blind spot.** |
| **out of scope** | Excluded by the scope file, with a recorded justification. |

`coverage %` is the share of in-scope controls in any of the first three states;
`assured %` is the share that are `supported`.

## Bundled catalogs

| Framework | Code | Catalog |
| --- | --- | --- |
| SOC 2 (Trust Services Criteria) | `SOC2` | Complete — all five categories (Common Criteria + Availability, Confidentiality, Processing Integrity, Privacy), 61 controls |
| ISO/IEC 27001:2022 Annex A | `ISO` | Complete — all 93 controls |
| NIST SP 800-53 Rev. 5 | `NIST` | Moderate baseline — 177 base controls across 18 families |

Most SOC 2 reports scope only some categories (Security is near-universal; Privacy and
Processing Integrity often are not). Use `exclude_families` in the scope file to drop a
whole category — or an ISO theme, or a NIST family — from the denominator in one line:

```yaml
exclude_families:
  - {framework: SOC2, family: Privacy, reason: "Privacy category not in the SOC 2 audit scope."}
  - {framework: SOC2, family: Processing Integrity, reason: "PI category not in the SOC 2 audit scope."}
```

Control codes are written `FRAMEWORK:ID` (`SOC2:CC6.1`, `ISO:A.5.17`), matching the
codes `audit-report` rulesets already cite. A partial catalog is reported honestly as
coverage of the shipped subset, never as the whole standard.

If the corpus cites a code whose framework is loaded but the catalog does not define
it — a typo or a renamed control — it is surfaced as an **unmatched control code**
rather than silently ignored.

## How it fits the pipeline

```
audit-tools ──► CSV package ──► evidence-seal (seal + verify)
                     │
                     ▼
               audit-report ──► per-package report (--format json)
                                        │
                                        ▼  one or more reports = a corpus
                               control-coverage ──► coverage %, blind spots, SoA,
                                                    trend over time, evidence crosswalk
```

## Development

```bash
pip install -e ".[dev]"
pytest
ruff check .
```

## License

GPL-3.0-or-later. See [LICENSE](LICENSE).
