Metadata-Version: 2.5
Name: nyms
Version: 0.0.2
Summary: Reversible pseudonymisation of text: deterministic detection of personal and sensitive identifiers, keyed surrogate generation (readable pseudonyms included), and re-identification through a mapping store.
Project-URL: Homepage, https://github.com/thorwhalen/nyms
Project-URL: Repository, https://github.com/thorwhalen/nyms
Project-URL: Documentation, https://thorwhalen.github.io/nyms
Author: Thor Whalen
License-Expression: MIT
License-File: LICENSE
Keywords: anonymization,cpr,danish,de-identification,gdpr,pii,privacy,pseudonymisation,redaction
Requires-Python: >=3.10
Requires-Dist: cw>=0.1
Requires-Dist: dol>=0.3
Provides-Extra: crypto
Requires-Dist: cryptography>=42; extra == 'crypto'
Provides-Extra: dev
Requires-Dist: cryptography>=42; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.1.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: sphinx-rtd-theme>=1.0; extra == 'docs'
Requires-Dist: sphinx>=6.0; extra == 'docs'
Description-Content-Type: text/markdown

# nyms

Reversible pseudonymisation of text before it reaches an AI. A deterministic (non-AI) pass finds personal and sensitive identifiers in Danish and English text, replaces them with surrogates, remembers the mapping per project so the same person is `[PERSON_1]` in every document, and puts the originals back into whatever the AI answers.

```bash
pip install nyms            # add "nyms[crypto]" for an encrypted mapping store
```

```python
from nyms import Pseudonymiser

p = Pseudonymiser("case-2026", names=["Rasmus Østergaard Jensen"])
r = p.pseudonymise("Rasmus Jensen (CPR 010190-1234) skylder 12.500 kr. Rasmus Jensens advokat ringede.")
r.text
# '[PERSON_1] (CPR [CPR_1]) skylder [AMOUNT_1]. [PERSON_1]s advokat ringede.'
p.reidentify("Svar: [PERSON_1] skal betale [AMOUNT_1].").text
# 'Svar: Rasmus Jensen skal betale 12.500 kr.'
p.check("Rasmus Jensen bor her")      # what would leak if this were sent
# ['Rasmus Jensen']
```

The same four verbs from the shell:

```bash
nyms pseudonymise --path letter.md --project case-2026 --names "Rasmus Østergaard Jensen" --out letter.pseud.md
nyms reidentify --path answer.md --project case-2026 --out answer.clear.md
nyms check --path draft.md --project case-2026       # {"ok": false, "leaks": [...]} if an original is in it
nyms detect "Rasmus Jensen, tlf. 12 34 56 78"        # every span and the policy's decision; nothing stored
nyms readable 010190-1234 --project case-2026        # a keyed hash as two readable words
```

Content is positional text or `--path` (a file, or `-` for stdin), never guessed from one another.

```bash
```

`r.report` says what was replaced (by type and by detector), what was detected and kept (dates, postcodes and URLs, under the default policy) and which detectors ran, and ends with the caveat that this is pseudonymisation, not anonymisation.

## What it finds

Danish: CPR numbers (the date part must be valid; the pre-2007 modulus-11 check only raises confidence, because numbers issued since October 2007 need not pass it), CVR numbers (modulus-11 and context), DK VAT numbers, phone numbers, bank account numbers (registreringsnummer and kontonummer), IBAN (mod-97), street addresses with floor and postcode, postcode and town, amounts (`12.500 kr.`, `DKK 12.500`, `36.000,-`), dates, court and journal numbers. English: addresses, phone numbers, amounts (`£4,250.00`, `$5,000`), dates. Both: e-mail, URLs, IBAN, credentials (API keys, tokens, private keys), organisation names by legal form (`ApS`, `A/S`, `Ltd`, `GmbH`), names after a title (`Hr.`, `advokat`, `Mr`, `Dr`), names from a bundled Danish and English first-name list, names you pass in (`names=`), and a low-confidence heuristic for capitalised sequences.

Once a value has been replaced anywhere in a project, every later occurrence is replaced too, including a Danish genitive (`Rasmus Jensens` becomes `[PERSON_1]s`), ALL-CAPS headings, and a surname or first name alone when it identifies exactly one known person (`Jensen blev` becomes `[PERSON_1] blev`). An identifier inside something the policy keeps (a CPR number in a URL) is still replaced. Pseudonymising already pseudonymised text changes nothing.

It does not find: names it has never seen and the heuristic misses, roles and rare facts that identify without an identifier, inflected compounds, OCR-damaged identifiers. The report's `kept` list and the `check` command exist so a reader sees what stayed.

## Choosing what is sensitive

A `Policy` decides after detection, so the report can say what it saw and kept. `Policy.default()` replaces credentials, persons, CPR, e-mail, phone, addresses, IBAN, bank accounts, case numbers, CVR, organisations and amounts; `Policy.identifiers()` leaves amounts and organisations; `Policy.strict()` replaces everything detected, dates and postcodes included. `include`, `exclude`, per-type `min_score`, `allow` (never replace) and `deny` (always replace) lists refine any of them. The taxonomy behind the types, with tiers and the equivalent Presidio, Google DLP and HIPAA names, is `nyms.taxonomy.ENTITY_TYPES`.

## Choosing the surrogates

| Strategy | Example | When |
|---|---|---|
| `Counter()` (default) | `[PERSON_1]`, `[CPR_2]` | the form LLMs alter least in the one published benchmark; numbered per project |
| `KeyedWords(key)` | `Alder Brook`, `[CPR:mustard-suitcase]` | the same surrogate on any machine that has the key, no shared store needed; readable |
| `RandomWords()` | `Lunar Cobweb` | a classic tokenisation vault: random, readable, no key |
| `Aliases({...}, fallback=...)` | `the landlord` | you choose the pseudonym for some values yourself |

`KeyedWords` is an HMAC-SHA-256 under the project key rendered as words from the EFF short wordlist (1,296 words, 10.3 bits each). The renderer stands alone as `nyms.readable(value, key=...)` and `nyms readable`. There is no unkeyed option: a plain hash of a CPR number is undone by enumerating the ten billion candidates.

Re-identification is tolerant of what models do to bracketed tokens (case, spacing, a dropped bracket, a genitive after it); name-shaped surrogates and aliases are matched exactly, so ordinary prose is left alone. It never guesses: tokens it did not issue come back in `unmatched`.

## Where the mapping and the key live

The mapping store (surrogate, original, type) is the key to the text. By default it is a set of JSON files under `~/.local/share/nyms/<project>/mapping/`, and the project key is 32 random bytes in `~/.config/nyms/keys/<project>.key` (mode 0600), created on first use; `NYMS_KEY` in the environment overrides it, and `Pseudonymiser(..., key=...)` accepts any bytes. With `pip install "nyms[crypto]"` and `encrypt=True` the records are Fernet-encrypted under the key before they hit disk; without it they are plaintext in your user directory, which is the trade-off of needing no dependency. Lose the store and `[PERSON_1]` cannot be reversed; lose the key and an encrypted store cannot be read. Back both up, apart from each other.

Any `MutableMapping` works as the store (`store=`), so [dol](https://github.com/i2mint/dol) backends (SQLite, S3) drop in, and `store={}` keeps a run in memory.

## For AI agents

<!-- epythet:agentic-readme:start -->
`nyms` ships a skill for coding agents, in [Agent Skills](https://agentskills.io) format:

```bash
gh skill install thorwhalen/nyms nyms --agent claude-code     # or copilot, cursor, codex, gemini
```

| Skill | Use it to |
|---|---|
| `nyms` | pseudonymise documents before they reach a model, re-identify the answer, and check that nothing leaks |

The skill folder also ships inside the wheel: `python -c "import nyms, os; print(os.path.join(nyms.__path__[0], 'data', 'skills'))"`.
<!-- epythet:agentic-readme:end -->

## Design

The research behind the design (terminology and regulation, detection libraries and methods, surrogates and key management, pipeline architecture) is in [`docs/research/`](docs/research/), with a [synthesis](docs/research/deidentification-synthesis.md); the decisions are in [`docs/adr/`](docs/adr/). Six seams, each one keyword argument of `Pseudonymiser`: `detectors`, `post_detectors`, `policy`, `strategy`, `store`, `key`. The model-based second pass is the `post_detectors` seam: any callable `text -> spans`, run on the already-pseudonymised text with the surrogates protected; none ships in v1.

The EFF wordlist is CC BY 3.0 (see `nyms/data/wordlists/NOTICE.md`). Everything else is MIT.
