Metadata-Version: 2.4
Name: dewatermark
Version: 0.7.0
Summary: Verification-first AI text watermark remover for Unicode artifacts and detector-scoped statistical watermark mitigation.
Project-URL: Homepage, https://github.com/cyzanfar/text-watermark-remover
Project-URL: Repository, https://github.com/cyzanfar/text-watermark-remover
Project-URL: Issues, https://github.com/cyzanfar/text-watermark-remover/issues
Project-URL: Changelog, https://github.com/cyzanfar/text-watermark-remover/blob/main/CHANGELOG.md
Project-URL: Documentation, https://cyzanfar.github.io/text-watermark-remover/
Author: Cyrus Anfar
License-Expression: MIT
License-File: LICENSE
License-File: UNICODE_LICENSE.txt
Keywords: AI agents,AI watermark remover,LLM watermark,SynthID,statistical watermark,steganography,text watermark remover,unicode,watermark detector
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Security
Classifier: Topic :: Text Processing :: Filters
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: requests>=2.31
Requires-Dist: tomli>=2; python_version < '3.11'
Provides-Extra: agents
Requires-Dist: mcp<3,>=1.27; (python_version >= '3.10') and extra == 'agents'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: hypothesis>=6; extra == 'dev'
Requires-Dist: jsonschema>=4.23; extra == 'dev'
Requires-Dist: mypy<2,>=1.11; extra == 'dev'
Requires-Dist: pytest-cov>=5; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Requires-Dist: twine>=5; extra == 'dev'
Requires-Dist: types-requests>=2.31; extra == 'dev'
Provides-Extra: eval
Requires-Dist: bert-score>=0.3.13; extra == 'eval'
Requires-Dist: mauve-text>=0.4.0; extra == 'eval'
Requires-Dist: sentence-transformers>=3.0; extra == 'eval'
Requires-Dist: torch>=2.2; extra == 'eval'
Requires-Dist: transformers>=4.45; extra == 'eval'
Provides-Extra: local
Requires-Dist: torch>=2.2; extra == 'local'
Requires-Dist: transformers>=4.44; extra == 'local'
Description-Content-Type: text/markdown

# dewatermark — Verification-first text watermark remover

[![PyPI version](https://img.shields.io/pypi/v/dewatermark.svg)](https://pypi.org/project/dewatermark/)
[![Python versions](https://img.shields.io/pypi/pyversions/dewatermark.svg)](https://pypi.org/project/dewatermark/)
[![CI](https://github.com/cyzanfar/text-watermark-remover/actions/workflows/ci.yml/badge.svg)](https://github.com/cyzanfar/text-watermark-remover/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/cyzanfar/text-watermark-remover/blob/main/LICENSE)
[![GitHub stars](https://img.shields.io/github/stars/cyzanfar/text-watermark-remover?style=social)](https://github.com/cyzanfar/text-watermark-remover/stargazers)

`dewatermark` removes suspicious hidden Unicode and helps test known statistical
text watermarks. It can clean one string, scan a repository, locate a detector
signal, or search for the smallest rewrite that clears named detectors. Basic
Unicode cleanup runs locally, gives the same result every time, and needs no
model or network connection.

For statistical LLM watermarks, it can run experiments and check the result
with a detector built for that watermark. It never treats changed text as proof
that every watermark is gone, and it does not claim to remove a vendor
watermark when no compatible detector is available.

[Try the browser playground (text stays in your browser)](https://cyzanfar.github.io/text-watermark-remover/)
· [View on PyPI](https://pypi.org/project/dewatermark/)
· [Explore integrations](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/INTEGRATIONS.md)

## Install

Python 3.9 or newer is required. The core package includes Unicode cleanup,
analysis, repository scanning, and the CLI:

```bash
python -m pip install dewatermark
dewatermark --version
```

> **Current release:** This page documents `0.7.0`, including detector
> localization, detector-guided mitigation, and the KGW and Unigram reference
> packs.

Install optional features only when you need them:

```bash
python -m pip install "dewatermark[local]"   # local model-backed rewriting
python -m pip install "dewatermark[eval]"    # research and evaluation tools
python -m pip install "dewatermark[agents]"  # MCP server; Python 3.10+
```

Models are not downloaded automatically, even when an optional package is
installed.

## Clean hidden Unicode

```python
import dewatermark

text = "he\u200bllo"  # contains an invisible zero-width character
clean = dewatermark.sanitize(text)

print(repr(text))   # 'he\u200bllo'
print(repr(clean))  # 'hello'
```

`sanitize()` returns a string. Its default `safe` profile removes or normalizes
characters covered by the policy while preserving recognized emoji,
right-to-left, and writing-system contexts.

Use `analyze()` when you want to inspect the text without changing it:

```python
report = dewatermark.analyze(text)
print(report)
```

The `aggressive` profile also normalizes compatibility characters and look-alike
letters. It is intentionally lossy, so use it only when that tradeoff is
acceptable:

```python
clean = dewatermark.sanitize(text, profile="aggressive")
```

## Command line

```bash
python -c "print('he\u200bllo', end='')" | dewatermark sanitize
# hello

python -c "print('he\u200bllo', end='')" | dewatermark analyze
dewatermark check .
```

The first command writes cleaned text. `analyze` reports findings without
changing the input. `check` scans files and changes nothing unless you pass
`--fix`; it exits with status `1` when it finds actionable hidden Unicode.

## Choose the right tool

| Goal | Start here |
| --- | --- |
| Remove clearly suspicious Unicode | `sanitize()` or `dewatermark sanitize` |
| Inspect text without changing it | `analyze()` or `dewatermark analyze` |
| Scan a repository | `dewatermark check PATH` |
| Get a JSON-ready report of what changed | `remove(..., mode="sanitize").to_dict()` |
| Try model-backed rewriting | [Statistical LLM watermarks](#statistical-llm-watermarks-advanced) |
| Verify a statistical watermark | [Verify with a detector](#verify-statistical-watermarks-with-a-detector) |
| Find where a known detector sees a signal | [Locate and mitigate a known signal](#locate-and-mitigate-a-known-signal) |
| Search for a verified, minimal rewrite | [Locate and mitigate a known signal](#locate-and-mitigate-a-known-signal) |
| Review exact text and settings before applying | [Agents and automation](#agents-and-automation) |
| Use editors, CI, HTTP, MCP, or Docker | [Integrations](#integrations) |

## What the results mean

Unicode cleanup and statistical watermark testing are separate operations.

| Result | Meaning |
| --- | --- |
| `unicode_sanitized` | Policy-covered characters were removed or normalized |
| `mitigation_verified` | A named independent detector was positive before rewriting and clear afterward at its tested decision boundary, and every required quality check passed |
| `mitigation_unverified` | Text changed and passed quality checks, but compatible verification was unavailable |
| `unsupported_scheme` | The requested watermark cannot currently be tested |
| `rejected_quality` | Rewritten candidates failed quality checks, so the original text was kept |

These results do not identify who wrote the text and do not prove that it is
universally watermark-free. See the
[assurance model](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/ASSURANCE.md)
for the full status contract.

## Scan files and repositories

```bash
dewatermark check .
dewatermark check . --fix
dewatermark check . --format sarif --output dewatermark.sarif
```

The scanner reports the file, line, column, Unicode code point, and reason for
each finding. `--fix` modifies files in place using atomic replacement; edit
details are available in the JSON and Python reports. You can share one
`.dewatermark.toml` policy across local development, pre-commit, CI, and the
editor integrations.

See the
[integration guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/INTEGRATIONS.md)
for shared policies, ignore lists, checking only changed lines, pre-commit, and
GitHub code scanning.

## Privacy and safety

- `sanitize`, `analyze`, repository scanning, and listing installed features run
  locally without a learned model.
- Managed model downloads run only after `dewatermark download-model`,
  `allow_model_download=True`, or the matching environment setting.
- Managed remote backends send text only when
  `allow_remote_processing=True` is set separately.
- The default Unicode profile preserves contextual characters. The
  `aggressive` profile may change legitimate text.
- Model-generated rewrites are treated as candidates. Detector-guided search
  accepts one only after quality checks and held-out verification pass; every
  other outcome returns the exact source.
- Errors and result receipts omit source text and credentials. Configuration
  output also hides credentials. `analyze()` intentionally returns annotated
  input, so treat its output as sensitive.
- Third-party Python extensions are trusted code and keep the permissions of
  the current process; this package is not an operating-system sandbox.

See the
[configuration guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/CONFIGURATION.md)
and [quality-check guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/QUALITY_GATES.md)
for advanced settings.

## Statistical LLM watermarks (advanced)

Removing hidden Unicode is deterministic. Statistical watermark removal is
experimental: the result depends on how the watermark was created, which
detector checks it, and the text being tested.

`remove()` provides several research modes:

- `sanitize` performs Unicode cleanup only.
- `bias_inversion` and `sira` are experimental implementations inspired by the
  [BIRA](https://arxiv.org/abs/2509.23019) and
  [SIRA](https://arxiv.org/abs/2505.05190) papers.
- `paraphrase`, `full`, and `adversarial` provide rewrite baselines.
- `auto` chooses an available mode and falls back safely when a backend cannot
  run.

These modes are not proof against a vendor deployment. Use a compatible,
independent detector for any removal claim. Model downloads and remote text
processing remain disabled until enabled separately.

## Verify statistical watermarks with a detector

```bash
dewatermark detectors list
dewatermark detectors doctor
dewatermark detectors conformance
dewatermark detectors packs
```

The built-in KGW-, Unigram-, and tournament-style detectors are small test
cases for integration code, not production detectors. The packaged KGW and
Unigram profiles score one exact, closed-vocabulary reference configuration;
the KGW pack also keeps its older token example. The SynthID pack is only a
disabled template until its required configuration and independent tests are
supplied.

A passing conformance test means the integration passes its known test cases.
It does not prove that the tool removes a production watermark. See the
[detector guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/DETECTORS.md)
and [reference detector guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/REFERENCE_DETECTORS.md).

## Locate and mitigate a known signal

When you have a compatible detector, `localize` finds the character ranges that
contribute to its result. Native detector ranges are preferred. Otherwise the
tool scans bounded, overlapping windows and adjusts the confidence threshold so
that scanning more windows does not make a positive result easier to obtain.
Only a calibrated detector with compatible p-values and declared family-wise
error control can produce a confirmatory `localized` result. Other ranges are
labeled `localized_exploratory`: useful editing hints, not verification.

```bash
dewatermark localize --input input.txt --detector your-primary-detector
```

`mitigate` tries bounded candidate-generation strategies and ranks only
candidates that pass the central quality checks. It returns changed text only
when the primary detector clears and another calibrated, independent detector
that was not used to guide the search also clears. Every other outcome returns
the exact original text.

```bash
dewatermark mitigate \
  --input input.txt \
  --detector your-primary-detector \
  --verifier your-held-out-detector \
  --strategy your-rewrite-strategy \
  --consent
```

The detector and strategy names above are installed extensions, not bundled
production services. The included KGW and Unigram profiles are exact, offline
reference configurations for integration and conformance work. They are
deliberately uncalibrated and cannot produce a production removal claim. See
[detector-guided mitigation](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/DETECTOR_GUIDED_MITIGATION.md)
for the Python API, budgets, subprocess strategy protocol, and acceptance rules.

### Current Claude limitation

Anthropic has
[confirmed text marking](https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content)
for supported Claude models, but it has not published the detector and
verification procedure needed for an independent test. `dewatermark` therefore
returns `unsupported`; its capability metadata records
`status=unsupported_pending_spec`. It does not claim that Unicode cleanup or a
generic rewrite removes a Claude watermark.

## Agents and automation

Use the review-before-apply API when a person or agent wants to inspect the
exact text and settings before execution:

```python
from dewatermark import apply_plan, create_plan, inspect_text, verify_text

text = "he\u200bllo"
inspection = inspect_text(text, detector="unicode")
plan = create_plan(text, mode="sanitize", detector="unicode")
applied = apply_plan(
    text,
    plan["plan_digest"],
    mode="sanitize",
    detector="unicode",
    consent=True,
)
verification = verify_text(
    text,
    applied["result"]["cleaned_text"],
    detector="unicode",
)

print(inspection["detector_evidence"]["status"])  # detected
print(verification["verification_status"])         # verified_cleared
```

The plan digest changes when the input or approved settings change, so
`apply_plan` rejects stale or mismatched plans. It does not authenticate the
approver or turn third-party Python plugins into sandboxed code.

The same workflow is available through the CLI, HTTP/OpenAPI, and MCP. See the
[agent workflow guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/AGENT_WORKFLOWS.md)
or locate the bundled skill with:

```bash
dewatermark skill path
dewatermark skill install --output ./remove-text-watermarks
```

## Integrations

- **Browser and JavaScript source:** use the
  [browser playground](https://cyzanfar.github.io/text-watermark-remover/),
  where text stays in the browser, or package the browser module from source.
- **Editors:** local-only VS Code and JetBrains integrations are included.
- **Git hooks and CI:** use pre-commit, the composite GitHub Action, or SARIF
  output for GitHub code scanning.
- **Services and agents:** run the local HTTP/OpenAPI server, MCP stdio server,
  or generated API clients.
- **Containers:** build the non-root Docker image; it does not start a network
  server unless you configure one.

All setup instructions are in the
[integration guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/INTEGRATIONS.md).

## Test results and evaluation

The checked-in Unicode fixture report records 50 of 50 embedded examples
removed across five known hidden-character families using the intentionally
lossy `aggressive` profile. This only tests those examples; it says nothing
about statistical or undocumented vendor watermarks.

The evaluation tools keep setup data separate from final test data and count
errors as failures. No tracked statistical result currently satisfies the full
benchmark protocol.

See the
[Unicode fixture report](https://github.com/cyzanfar/text-watermark-remover/blob/main/benchmarks/unicode-v0.4.md),
[evaluation guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/eval/README.md),
and [benchmark protocol](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/BENCHMARK_PROTOCOL.md).

## Extending and contributing

Rewriting backends, detectors, quality checks, and ways to split long input can
be added without changing the package's core. Start with the
[extension guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/EXTENSIONS.md),
[architecture](https://github.com/cyzanfar/text-watermark-remover/blob/main/docs/ARCHITECTURE.md),
and [contributor guide](https://github.com/cyzanfar/text-watermark-remover/blob/main/CONTRIBUTING.md).

Good first contributions include detector adapters, editor integrations,
Unicode examples from real systems, and independent benchmark replications.
See the [roadmap](https://github.com/cyzanfar/text-watermark-remover/blob/main/ROADMAP.md)
or open a
[feature proposal](https://github.com/cyzanfar/text-watermark-remover/issues/new/choose).

If the project is useful to you, consider
[starring it on GitHub](https://github.com/cyzanfar/text-watermark-remover).

## Scope limits

- The deterministic sanitizer covers known Unicode artifacts, not every
  possible text watermark.
- Statistical results apply only to the named detector and configuration used
  for that run.
- Editing text cannot erase records kept by a model provider or matching
  service.
- This project handles text. It does not remove image watermarks, EXIF/XMP,
  C2PA, or document metadata.
- Claude remains unsupported until a compatible public detector and procedure
  are available.

## License

The package is MIT-licensed; see
[LICENSE](https://github.com/cyzanfar/text-watermark-remover/blob/main/LICENSE).
The generated confusables table includes Unicode data covered by the
[Unicode License v3](https://github.com/cyzanfar/text-watermark-remover/blob/main/UNICODE_LICENSE.txt).
