Metadata-Version: 2.4
Name: annotations4all
Version: 0.3.0
Summary: Schema-first Python library for LLM-assisted span annotation
Keywords: ner,llm,nlp,span-annotation,tagger
Author: Nicole Dresselhaus
Author-email: Nicole Dresselhaus <nicole.dresselhaus@hu-berlin.de>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing
Classifier: Topic :: Text Processing :: Linguistic
Requires-Dist: fuzzysearch>=0.7.3,<1.0.0
Requires-Dist: httpx>=0.28.1,<1.0.0
Requires-Dist: openai>=2.31.0
Requires-Dist: typing-extensions>=4.0.0,<5.0.0
Requires-Dist: ollama>=0.4.7,<1.0.0 ; extra == 'ollama'
Requires-Dist: textual>=8.2.8,<9 ; extra == 'view'
Requires-Dist: rich>=14.2,<16 ; extra == 'view'
Requires-Dist: pyyaml>=6.0.3,<7 ; extra == 'view'
Requires-Python: >=3.11, <4.0
Project-URL: Repository, https://scm.cms.hu-berlin.de/annotations4all/annotations4all
Project-URL: Issues, https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/issues
Provides-Extra: ollama
Provides-Extra: view
Description-Content-Type: text/markdown

# annotations4all

**Languages:** [Deutsch](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/README.de.md) ·
[English](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/README.en.md)

`annotations4all` is a schema-first Python library for LLM-assisted span annotation (commonly used for named-entity
recognition). It generates prompts from a user-defined tag schema, parses annotated LLM responses in
`<<TAG>>…</TAG>>` format, and returns matches as offset-based spans.

> **Version 0.3.0:** Schema-first span annotation with prompt taggers, parsing and merge, provider policies,
> capability reporting, packaged hints and an optional annotation viewer. Legacy native wrappers are deprecated for
> removal in 0.4.0; the new provider clients are not deprecated.

![annotations* cosmos — libraries, data flow and
evaluation](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/raw/main/docs/diagrams/annotations-kosmos.en.png)

Overview of the `annotations-*` library family — raw data and gold annotations,
annotating, running experiments, evaluating, reporting:

## Documentation

The [tutorial](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/docs/tutorial.md) documents
the version 0.3 API and examples against local OpenAI-compatible installations. [Capability
reporting](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/docs/provider-capabilities.md)
and [provider hints](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/docs/hints.md)
describe the additional interfaces.

## Browse saved annotations (optional)

The read-only terminal viewer displays saved Prodigy/annotations4all records and
annotated TXT answers, with file statistics, gold/prediction layers, colored
span stacks and label toggles. Install from the checkout and run:

```bash
python -m pip install -e '.[view]'
annotations-view ./data
```

Split and compact nested-diff views compare distinct stored layers. Label colors
are optimized against the actual opaque theme background. Split/Diff compare
two stored layers of the same document, not independent files. Without a path, a
directory picker opens; `..` navigates up. Deliberate theme choices in the command
palette are saved immediately; other settings require explicit Save. Settings use
`$XDG_CONFIG_HOME/annotations4all/view_tui.yaml`.
[Viewer guide (German)](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/docs/view-tui.md).

## Installation

Release notes with screenshots: [0.2 as the initial feature
baseline](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/release-notes/0.2.md)
and [all changes from 0.2 to
0.3](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/release-notes/0.3.md).

Install from PyPI:

```bash
python -m pip install 'annotations4all==0.3.0'
```

For development, from the repository:

```bash
python -m venv .venv
. .venv/bin/activate
python -m pip install -e .
python -m pytest
```

## Schema-first quick start

The following example shows the preferred entry point: define the tag schema first, then generate prompt messages
and map the model response back to spans.

```python
from annotations4all import ConfigurableTagger

text = "Max Mustermann lives in Berlin."
tagger = ConfigurableTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)

messages = tagger.get_prompt(text)
for message in messages:
    print(message["role"])
    print(message["content"])

# Response of an LLM, e.g. from an OpenAI-compatible endpoint:
response = "<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>."
spans = tagger.parse_response(response, text)
print(spans)
```

`tags` describes the tag schema of the concrete workflow. `context` holds material- or task-specific annotation
hints, not arbitrary runtime data. Common NER tags such as `PER`, `LOC`, and `ORG` are well suited for quick starts,
but the library does not enforce a fixed ontology.

## Minimal example without an LLM call

The following example shows the smallest stable core: an already annotated response is mapped back to character
positions in the original text.

```python
from annotations4all import parse_region_response, parse_region_response_detailed

text = "Max Mustermann lives in Berlin."
response = "<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>."

spans = parse_region_response(response, text)
detailed = parse_region_response_detailed(response, text)
print(spans)
print(detailed.warnings)
# [{'label': 'PER', 'meta': None, 'start': 0, 'end': 14}, {'label': 'LOC', 'meta': None, 'start': 24, 'end': 30}]
```

## Writing custom taggers

For advanced use cases, custom taggers can be implemented. The base class `ChatTagger` remains importable for this
purpose but is not part of the highlighted package-root API.

```python
from annotations4all.taggers.base import ChatTagger, Message
from annotations4all.utils.response_parser import parse_region_response


class MyTagger(ChatTagger):
    def name(self) -> str:
        return "my-tagger"

    def get_prompt(self, text: str) -> list[Message]:
        return [
            {
                "role": "system",
                "content": "Annotate persons as <<PER>>…</PER>> and places as <<LOC>>…</LOC>>.",
            },
            {"role": "user", "content": text},
        ]

    def parse_response(self, response: str, text: str, logfile=None):
        return parse_region_response(response, text, logfile=logfile)
```

The model response must reproduce the original text as exactly as possible and mark spans with opening and closing tags:

```text
<<PER>>Max Mustermann</PER>> lives in <<LOC>>Berlin</LOC>>.
```

## Tag schemas

The library does not enforce a fixed ontology. The tag schema belongs to the respective research or annotation
workflow. Common quick-start tags include:

- `PER`: persons
- `LOC`: places
- `ORG`: organizations
- project-specific tags such as `DOM`, `DAT`, `KG`, etc.

Tags can optionally carry metadata, e.g. `<<LOC:city>>Berlin</LOC>>`. The parser returns this metadata as `meta`
when present. The returned objects are spans with at least `label`, `start`, and `end`.

## Hierarchical tagging in one pass

Nested answers are parsed directly — the parser is a stack decoder, not a flat one:

```python
from annotations4all import ConfigurableTagger

text = "Ernst August von Testingen"
tagger = ConfigurableTagger(
    tags=[("PER", "person names (outer span)"), ("FN", "given name"), ("LN", "family name")],
    context="PER encloses the whole name mention; FN and LN are nested inside it.",
    language="en",
)
response = "<<PER>><<FN>>Ernst August</FN>> <<LN>>von Testingen</LN>></PER>>"
print(tagger.parse_response(response, text))
# [{'label': 'PER', 'meta': None, 'start': 0, 'end': 26},
#  {'label': 'FN', 'meta': None, 'start': 0, 'end': 12},
#  {'label': 'LN', 'meta': None, 'start': 13, 'end': 26}]
```

- The prompt decides whether the model nests: the packaged templates allow nesting explicitly and forbid crossing
  tags, and the intended structure belongs in `context` as well.
- Children are always searched **inside** their parent span, and repeated text is anchored via the tag position in
  the response (no silent first-match drift). A child that cannot be placed inside its parent is reported as
  `nested-out-of-parent`: `nested_policy="strict"` (default) drops it, `"repair"` keeps it at the anchor that was
  found.
- A closing tag written one `>` short (`</TAG>` instead of `</TAG>>`) is accepted and reported as
  `close-tag-single-bracket`; `tolerate_single_bracket_close=False` rejects it.
- Overlap is allowed as long as no span encloses the other — two flat spans such as `<<GIV>>…</GIV>>
  <<RCV>>…</RCV>>` are both parsed.
- **Crossing** tags (`<<GIV>>…<<RCV>>…</GIV>>…</RCV>>`) cannot be represented in the `<<TAG>>` format: the parser
  keeps the span that closes first, drops the other one and reports `unclosed-nested-tag` / `unmatched-close-tag`.
  Use the per-label mode for those (see below) — it keeps both spans and reports the `crossing` conflict.
- For long outer spans raise `max_match_length` (default 100); otherwise they are truncated (`span-too-long`).

## Nested fallback: per-label mode + merge

For nested or hierarchical schemas a single, nested model answer can be too unreliable (see the section above). The
fallback annotates **one label (or one label group) per model call** and merges the flat runs deterministically
afterwards:

```python
from annotations4all import OpenAICompatClient, PerLabelTagger

text = "Anna Müller lives in Berlin."
tagger = PerLabelTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)
client = OpenAICompatClient(api_url="https://example.invalid/v1", api_key="<token>")

result = tagger.annotate(text, client, model="my-model", stream=False)
print(result.flat_spans())
# [{'label': 'PER', 'meta': None, 'start': 0, 'end': 11}, {'label': 'LOC', 'meta': None, 'start': 21, 'end': 27}]
print(result.spans)  # nested: parent spans with `children`
print(result.conflicts)  # conflict list
```

The merge is also usable on its own — a pure function on flat spans (no client, no prompt):

```python
from annotations4all import merge_span_sets

result = merge_span_sets(
    {
        "PER": [{"label": "PER", "meta": None, "start": 0, "end": 11}],
        "LOC": [{"label": "LOC", "meta": None, "start": 21, "end": 27}],
    }
)
```

Merge rules:

- **Anchor rule:** the hierarchy is derived from containment only — a span's parent is the smallest span that
  strictly encloses it, never a similar span found elsewhere in the document. Bounds are half-open (`[start, end)`),
  so touching spans are adjacent, not overlapping.
- **Conflicts are reported, never silently repaired:** `duplicate` (same label, same bounds, twice),
  `intra-label-overlap` (same label, partial overlap) and `crossing` (different labels, partial overlap).
- **Order-independent:** permuting runs or spans yields identical output.

Cost: **one model call per run** — with one label per run that is ≈ the number of labels. For large hierarchical
schemas the mode is therefore only sensible for the outer level; recursive prompting per detected element is the
alternative. In addition, `parse_region_response` and `parse_region_response_detailed` accept `bounds=(start, end)`
to anchor candidates only inside an already known parent span (no global fuzzy search).

## Backends and clients

The legacy `utils.llamacpp_client.LlamaCppClient` and
`utils.ollama_client.OllamaClient` emit `DeprecationWarning` on construction and
will be removed in **0.4.0**. Use `annotations4all.OpenAICompatClient`, or the
new public `annotations4all.OllamaClient` for Ollama's OpenAI-compatible endpoint.
The latter is not the legacy native-SDK wrapper and is not deprecated. Python
normally hides `DeprecationWarning`; use `python -Wd` to display it.

The inference contract remains a narrow, explicit OpenAI-compatible chat-completions interface. The target server
must offer a compatible endpoint such as `/v1/chat/completions`. Version 0.3 also exposes `create_chat_client` and
provider-policy wrappers; these are not native SDKs or a promise that every provider endpoint supports every field.
Explicit `reasoning` and `reasoning_effort` options are validated separately from generic `extra_body` payloads.
Capability discovery is offline by default and requires an explicit opt-in for remote verification.

```python
from annotations4all import ConfigurableTagger, OpenAICompatClient

text = "Max Mustermann lives in Berlin."
tagger = ConfigurableTagger(
    tags=[("PER", "Person names"), ("LOC", "Places")],
    context="Short modern example sentence.",
    language="en",
)
client = OpenAICompatClient(
    api_url="https://example.invalid/v1",
    api_key="<token>",
)

chunks = client.chat.create(
    model="my-model",
    messages=tagger.get_prompt(text),
    stream=True,
    temperature=0,
)
response = "".join(chunk.answer for chunk in chunks)
spans = tagger.parse_response(response, text)
print(spans)
```

`OpenAICompatClient` reads API keys either from `api_key=` or from an environment variable. The default is
`OPENAI_API_KEY`; `api_key_env=` selects a different name. Local servers that do not require authentication
automatically receive a dummy key, because the underlying OpenAI SDK still expects a value.

Provider-specific additional fields can be supplied through `extra_body=`. Known reasoning representations are
structurally validated and checked for conflicts even when no explicit reasoning argument is set. Explicit
`reasoning`/`reasoning_effort` controls may translate or supplement the payload; other additional fields remain the
endpoint's responsibility:

```python
client.chat.create(
    model="my-model",
    messages=tagger.get_prompt(text),
    extra_body={"reasoning": {"enabled": False}},
)
```

The compatibility promise remains narrow: standard content responses and the tested streaming form are the target.
Existing schema/tagger/parser/merge entry points remain available; 0.3 adds endpoint-aware request validation and
policy. Arbitrary provider-specific fields are not a universal stability promise.

## Provider clients, capabilities and hints

Version 0.3 adds policy clients over documented OpenAI-compatible routes:
`OpenAIClient`, `OpenRouterClient`, `DeepSeekClient`, `AnthropicClient`,
`GeminiClient`, `MistralClient`, `GroqClient`, and `OllamaClient`. They inherit
the generic connection/request interface and add `discovery_url`/`trusted`
metadata settings. Default URLs and key-variable names are in the
[tutorial](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/docs/tutorial.md).

`create_chat_client(...)` selects policy by the exact configured endpoint;
unknown destinations retain `OpenAICompatClient`. Configuration and construction
perform no network requests or state-file writes. Keyless construction uses a
dummy SDK key where needed; that does not authenticate against a hosted provider.

`ProviderCapabilitiesClient.list_models()` and `.capabilities(model)` default to
offline, unverified documented evidence. Remote metadata requests need explicit
`verify=True` and appropriate destination/key trust. `None` means unknown, not
unsupported; `checked_fields` identifies only facts actually established by
discovery, not inference success or account entitlement.

`get_hints()` supplies packaged German/English advice about reasoning, costs,
verification and evaluation, filterable by provider, topic and effort. Static
advice is not capability detection, an allowlist or a benchmark guarantee.

Boolean `reasoning` and named `reasoning_effort` are independent controls.
`reasoning=False` plus an explicit effort is rejected, as are conflicting
representations in `extra_body`. Tokens are validated rather than silently
normalized. Translation depends on the selected endpoint; `True` may leave a
model's default unchanged rather than force thinking. Provider wrappers are not
native SDKs and do not establish per-model support for every option.

```python
from annotations4all import create_chat_client, get_hints

client = create_chat_client(
    api_url="https://openrouter.ai/api/v1",
    api_key="<token>",
    reasoning=True,
)
for hint in get_hints(provider="openrouter", topic="reasoning", effort="high"):
    print(hint.title)
```

## Tests

```bash
python -m pytest
```

The tests cover prompts, parser golden cases/fuzzing, merge/per-label workflows, request policies, capability
evidence, hints and legacy deprecations. The optional viewer adds rendering, mouse, comparison, subprocess and
settings regressions. Install the `view` extra for the complete viewer suite. The separately configured live
endpoint smoke test in `tests/integration/test_endpoint_smoke.py` is skipped without configuration.

## Citation

> Dresselhaus, Nicole. (2026). *annotations4all* (Version 0.3.0) [Software]. Humboldt-Universität zu Berlin.
> <https://doi.org/10.5281/zenodo.23471533>

Project-wide concept DOI: [10.5281/zenodo.22011370](https://doi.org/10.5281/zenodo.22011370).
Version-specific citation metadata is supplied in `CITATION.cff`. The
[0.2 release note](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/release-notes/0.2.md)
documents the preceding version.

Machine-readable metadata is available in
[`CITATION.cff`](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/CITATION.cff).

## License

This software is licensed under the MIT License. See
[`LICENSE`](https://scm.cms.hu-berlin.de/annotations4all/annotations4all/-/blob/main/LICENSE) for details.
