Metadata-Version: 2.5
Name: jev-lite
Version: 0.0.3
Summary: OpenAI-style sync and async Pydantic client for TypeSafe Jev (System One).
Project-URL: Documentation, https://github.com/allen2c/jev-lite#readme
Project-URL: Homepage, https://github.com/allen2c/jev-lite
Project-URL: Issues, https://github.com/allen2c/jev-lite/issues
Project-URL: Repository, https://github.com/allen2c/jev-lite
Author-email: Allen Chou <f1470891079@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: jev,openai,opentelemetry,pydantic,system-one,typesafe
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: httpx>=0.23
Requires-Dist: opentelemetry-api>=1.20
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.20
Requires-Dist: opentelemetry-sdk>=1.20
Requires-Dist: pydantic>=2
Provides-Extra: cache
Requires-Dist: diskcache>=5.6; extra == 'cache'
Provides-Extra: distill
Requires-Dist: datasets>=2; extra == 'distill'
Provides-Extra: ml
Requires-Dist: accelerate>=1.0; extra == 'ml'
Requires-Dist: pillow>=9; extra == 'ml'
Requires-Dist: torch>=2.1; extra == 'ml'
Requires-Dist: torchvision>=0.16; extra == 'ml'
Requires-Dist: transformers>=4.40; extra == 'ml'
Description-Content-Type: text/markdown

# jev-lite

OpenAI-style Python client for [TypeSafe Jev](https://typesafe.ai/) (System One).

Jev does not generate text. Send `state` plus typed questions; get structured
decisions with probabilities. `jev-lite` is the official TypeSafe HTTP client — sync
and async, same shape as the OpenAI Python SDK. `jev-lite@0.0.2` added Dialogflow-style
system entities; `jev-lite@0.0.3` adds a privacy filter and a reranker (all
three are previews, below).

```bash
pip install jev-lite
# or: uv add jev-lite
```

Python 3.11+. Set `TYPESAFE_API_KEY`. Docs: <https://docs.typesafe.ai/>

## Usage

```python
from jev_lite import Choice, Noul, Score, TypeSafe

client = TypeSafe()  # TYPESAFE_API_KEY

result = client.system_one.create(
    state="I was charged twice. Please fix this ASAP.",
    questions={
        "billing": Noul(instructions="Is this ticket about billing?"),
        "tone": Choice(
            instructions="What is the customer's tone?",
            criteria={"calm": None, "frustrated": None, "angry": None},
        ),
        "urgency": Score(
            instructions="How urgent is this ticket?",
            criteria=["can wait", "this week", "today"],
        ),
    },
)

print(result.nouls["billing"].noul)
print(result.choices["tone"].choice)
print(result.scores["urgency"].score)
print(result.elapsed_ms, result.request_id, result.usage)
print(client.models.list())  # GET /v1/models
```

Async is the same surface with `await`:

```python
from jev_lite import AsyncTypeSafe, Noul

async with AsyncTypeSafe() as client:
    result = await client.system_one.create(
        state="I was charged twice. Please fix this ASAP.",
        questions={"billing": Noul(instructions="Is this ticket about billing?")},
    )
    models = await client.models.list()
```

Pin the model with `TypeSafe(model="jev-1.13.0")` (or `TYPESAFE_DEFAULT_MODEL`);
`jev-latest` moves. Prefer one `system_one.create` with many questions over a
loop of calls.

Re-running unchanged questions can replay answers from disk
(`pip install "jev-lite[cache]"`):

```python
client = TypeSafe(cache="~/.cache/jev")  # or a configured diskcache.Cache
```

Only 200 responses are stored, keyed on the request body (which carries the
model id). Leave the cache off when you measure repeat or order stability.

## System entities (preview)

`jev_lite.system_entities` extracts the 59 Dialogflow ES system entities
(`@sys.date-time`, `@sys.unit-currency`, `@sys.geo-city`, …) and fills
typed slots, in ten languages (en and zh-Hant primary). Code proposes
candidates, Jev `jev-1.13.0` decides, code normalizes. **Preview:** the API
may change within `0.0.x`. Same install; no extra dependency.

```python
from datetime import datetime
from zoneinfo import ZoneInfo

from jev_lite import TypeSafe
from jev_lite.system_entities import extract

now = datetime(2026, 10, 1, 16, 0, tzinfo=ZoneInfo("Asia/Taipei"))
with TypeSafe() as client:
    result = extract(client, "Book a table for 4 next Friday at 7pm", ["date-time", "number"], now=now)
for match in result.matches:
    print(match.entity, match.span.text, match.value)  # number 4 …; date-time … 2026-10-09 19:00
```

Slot filling, async and the value models: [API](https://github.com/allen2c/jev-lite/blob/main/docs/system-entities/api.md).
Read the [known limitations](https://github.com/allen2c/jev-lite/blob/main/docs/system-entities/limitations.md) first:
extraction is not a security boundary. On the held-out set
`system-entities-cases@v0.0.1` (301 cases, ten languages) it is 99.3% exact
on accepted values ([release gates](https://github.com/allen2c/jev-lite/blob/main/docs/system-entities/probes/release-candidate-gates.md)).

## Privacy filter and reranker (preview)

`jev_lite.privacy_filter` and `jev_lite.reranker` ship in `jev-lite@0.0.3`
as previews: the APIs may change within `0.0.x`. Import them as modules
(`from jev_lite.privacy_filter import detect`, `from jev_lite.reranker import
rerank`). All numbers pin `jev-1.13.0`.

**Privacy filter**: private-span detection. Public multilingual gold
(ai4privacy OpenPII, Gretel finance, NVIDIA Nemotron-PII), 300 test texts,
exact-span precision / recall:

| Setup | P / R | Over-redaction | Speed |
| --- | --- | --- | --- |
| [openai/privacy-filter](https://huggingface.co/openai/privacy-filter), local GPU | 63.4 / 62.9 | 5.2% | ~20 ms per text |
| openai/privacy-filter proposes, Jev decides | 75.4 / 60.1 | 2.5% | p50 210 ms |
| Jev only, no GPU (broader proposals, per-label cuts) | 68.6 / 60.5 | 2.4% | p50 215 ms |
| Jev only, round-04 default | 80.4 / 57.6 | 1.5% | p50 210 ms |
| Jev only, before the person proposer | 78.3 / 63.6 | 1.6% | p50 209 ms |
| **Jev only, default `detect(client, text)`**, no GPU | **79.3 / 69.2** | **1.6%** | p50 ~0.7 s |

The default uses per-label proposers and cuts fitted on dev, and finds
people with `system_entities` person spans plus a person detect Noul (person
recall 43.9 → 71.9), with an order-stable name boundary. On overlap it
scores 88.9 / 77.6; exact address recall is 0 because the gold splits an
address into components while the filter masks the whole address. It fails
most written robustness bars, so it is a preview. Hosted Jev sees the raw
text before masking. Details:
[docs/privacy-filter/](docs/privacy-filter/README.md).

**Reranker**: batched evidence Noul over a first-stage pool (balanced
batches of at most 30, independent of the caller's order), nDCG@10:

| Bank | First stage | Reranked | Query p50 |
| --- | --- | --- | --- |
| SciFact (50 q) | .556 | .638 | 0.30 s |
| FiQA (50 q) | .219 | .346 | 0.26 s |
| zh, BM25 (50 q) | .526 | .587 | 0.30 s |
| zh, E5 (50 q) | .551 | .647 | 0.27 s |
| Mr. TyDi ko / ja / ar (30 q each) | .415 / .605 / .529 | .675 / .791 / .754 | 0.23–0.24 s |

A batch score is relative to the other passages in its request, not a
standalone probability. A passage that echoes the relevance criteria can be
lifted into the top 10; `rerank(..., check_echo=True)` (off by default)
re-checks the top-k and moves such passages down, at about +0.23 s. Details: [docs/reranker/](docs/reranker/README.md).

## Tracing

Off by default. Opt in and pick the destination:

```python
from jev_lite import instrument

instrument(endpoint="http://127.0.0.1:4318")  # local Grafana LGTM
```

Or set `OTEL_EXPORTER_OTLP_ENDPOINT` and call `instrument()`, or install your
own `TracerProvider` first (Logfire, etc.). Spans record prompt, result, and
latency.

## Develop

```bash
uv sync --dev
uv run ruff check . && uv run pytest
```

The supported public API is the TypeSafe client. Optional local engines live
under `jev_lite.engine` (`pip install 'jev-lite[ml]'`). See
[docs/](https://github.com/allen2c/jev-lite/tree/main/docs).

## Semantic decisions v0.0.2

Dataset `semantic-decisions@v0.0.2`, versioned separately from the package
([versions](docs/versions.md)).

[252 frozen zh-Hant rows](data/semantic-decisions/v0.0.2/natural-zh-Hant/README.md)
in 18 families (36 easy, 48 medium, 168 hard), gold derived from visible text.
This natural zh-Hant v1 qualified freeze **is** v0.0.2, and it is the only
semantic-decision eval in this repo. Every model answers the same 504 locked
requests: each row in canonical and left-rotated option order. Charts are
percent correct. The families were seen during development before the
freeze, so this is not unseen confirmation.

[![All 504](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22AutoTrust%20JEV-27B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22Jev%201.13.0%22%2C%22AutoJev%2027B%22%2C%22Rune%20v3%2026B-A4B%22%2C%22decider-4b%20v2%22%2C%22NeoHorse-Jev%204B%22%2C%22Tev1%204B%22%2C%22AutoTrust%20JEV-9B%22%2C%22Clef%2027B%22%2C%22Mica%20v0.1%204B%22%2C%22Kev%204B%22%2C%22Clef-Flash%209B%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22RSI-Jev%20v3.0%22%2C%22Strands%202B%22%2C%22Tev1%200.8B%22%2C%22semif%22%2C%22JevEmbed%204B%22%2C%22Lev%204B%22%2C%22decider%200.8B%22%2C%22JevK5%20v0.2%22%2C%22Plumb%204B%22%2C%22GLiNER%22%2C%22Laya%22%2C%22jeff%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B96.4%2C96.4%2C95.4%2C93.3%2C92.9%2C86.5%2C84.9%2C84.3%2C82.5%2C80.2%2C80.0%2C79.4%2C76.2%2C74.6%2C72.4%2C69.8%2C66.3%2C65.9%2C65.5%2C62.9%2C62.1%2C57.5%2C55.8%2C51.0%2C47.6%2C43.5%5D%2C%22backgroundColor%22%3A%5B%22%23c026d3%22%2C%22%239333ea%22%2C%22%230d9488%22%2C%22%23d97706%22%2C%22%23475569%22%2C%22%230f766e%22%2C%22%232563eb%22%2C%22%23ea580c%22%2C%22%23a21caf%22%2C%22%23059669%22%2C%22%23dc2626%22%2C%22%2392400e%22%2C%22%2310b981%22%2C%22%230ea5e9%22%2C%22%2338bdf8%22%2C%22%2384cc16%22%2C%22%230891b2%22%2C%22%234f46e5%22%2C%22%23a16207%22%2C%22%237c3aed%22%2C%22%2316a34a%22%2C%22%230369a1%22%2C%22%23b45309%22%2C%22%2365a30d%22%2C%22%23e11d48%22%2C%22%23db2777%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22All%20504%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22AutoTrust%20JEV-27B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22Jev%201.13.0%22%2C%22AutoJev%2027B%22%2C%22Rune%20v3%2026B-A4B%22%2C%22decider-4b%20v2%22%2C%22NeoHorse-Jev%204B%22%2C%22Tev1%204B%22%2C%22AutoTrust%20JEV-9B%22%2C%22Clef%2027B%22%2C%22Mica%20v0.1%204B%22%2C%22Kev%204B%22%2C%22Clef-Flash%209B%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22RSI-Jev%20v3.0%22%2C%22Strands%202B%22%2C%22Tev1%200.8B%22%2C%22semif%22%2C%22JevEmbed%204B%22%2C%22Lev%204B%22%2C%22decider%200.8B%22%2C%22JevK5%20v0.2%22%2C%22Plumb%204B%22%2C%22GLiNER%22%2C%22Laya%22%2C%22jeff%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B96.4%2C96.4%2C95.4%2C93.3%2C92.9%2C86.5%2C84.9%2C84.3%2C82.5%2C80.2%2C80.0%2C79.4%2C76.2%2C74.6%2C72.4%2C69.8%2C66.3%2C65.9%2C65.5%2C62.9%2C62.1%2C57.5%2C55.8%2C51.0%2C47.6%2C43.5%5D%2C%22backgroundColor%22%3A%5B%22%23c026d3%22%2C%22%239333ea%22%2C%22%230d9488%22%2C%22%23d97706%22%2C%22%23475569%22%2C%22%230f766e%22%2C%22%232563eb%22%2C%22%23ea580c%22%2C%22%23a21caf%22%2C%22%23059669%22%2C%22%23dc2626%22%2C%22%2392400e%22%2C%22%2310b981%22%2C%22%230ea5e9%22%2C%22%2338bdf8%22%2C%22%2384cc16%22%2C%22%230891b2%22%2C%22%234f46e5%22%2C%22%23a16207%22%2C%22%237c3aed%22%2C%22%2316a34a%22%2C%22%230369a1%22%2C%22%23b45309%22%2C%22%2365a30d%22%2C%22%23e11d48%22%2C%22%23db2777%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22All%20504%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)

[![Easy](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22Jev%201.13.0%22%2C%22Lev%204B%22%2C%22Kev%204B%22%2C%22Tev1%204B%22%2C%22Tev1%200.8B%22%2C%22semif%22%2C%22decider-4b%20v2%22%2C%22JevK5%20v0.2%22%2C%22Plumb%204B%22%2C%22RSI-Jev%20v3.0%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22Strands%202B%22%2C%22Clef-Flash%209B%22%2C%22AutoTrust%20JEV-9B%22%2C%22NeoHorse-Jev%204B%22%2C%22Mica%20v0.1%204B%22%2C%22Clef%2027B%22%2C%22AutoTrust%20JEV-27B%22%2C%22AutoJev%2027B%22%2C%22Rune%20v3%2026B-A4B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22decider%200.8B%22%2C%22JevEmbed%204B%22%2C%22jeff%22%2C%22Laya%22%2C%22GLiNER%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C93.1%2C83.3%2C62.5%2C52.8%2C33.3%5D%2C%22backgroundColor%22%3A%5B%22%230d9488%22%2C%22%237c3aed%22%2C%22%2392400e%22%2C%22%23ea580c%22%2C%22%230891b2%22%2C%22%234f46e5%22%2C%22%230f766e%22%2C%22%230369a1%22%2C%22%23b45309%22%2C%22%2338bdf8%22%2C%22%230ea5e9%22%2C%22%2384cc16%22%2C%22%2310b981%22%2C%22%23a21caf%22%2C%22%232563eb%22%2C%22%23dc2626%22%2C%22%23059669%22%2C%22%23c026d3%22%2C%22%23d97706%22%2C%22%23475569%22%2C%22%239333ea%22%2C%22%2316a34a%22%2C%22%23a16207%22%2C%22%23db2777%22%2C%22%23e11d48%22%2C%22%2365a30d%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Easy%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22Jev%201.13.0%22%2C%22Lev%204B%22%2C%22Kev%204B%22%2C%22Tev1%204B%22%2C%22Tev1%200.8B%22%2C%22semif%22%2C%22decider-4b%20v2%22%2C%22JevK5%20v0.2%22%2C%22Plumb%204B%22%2C%22RSI-Jev%20v3.0%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22Strands%202B%22%2C%22Clef-Flash%209B%22%2C%22AutoTrust%20JEV-9B%22%2C%22NeoHorse-Jev%204B%22%2C%22Mica%20v0.1%204B%22%2C%22Clef%2027B%22%2C%22AutoTrust%20JEV-27B%22%2C%22AutoJev%2027B%22%2C%22Rune%20v3%2026B-A4B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22decider%200.8B%22%2C%22JevEmbed%204B%22%2C%22jeff%22%2C%22Laya%22%2C%22GLiNER%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C93.1%2C83.3%2C62.5%2C52.8%2C33.3%5D%2C%22backgroundColor%22%3A%5B%22%230d9488%22%2C%22%237c3aed%22%2C%22%2392400e%22%2C%22%23ea580c%22%2C%22%230891b2%22%2C%22%234f46e5%22%2C%22%230f766e%22%2C%22%230369a1%22%2C%22%23b45309%22%2C%22%2338bdf8%22%2C%22%230ea5e9%22%2C%22%2384cc16%22%2C%22%2310b981%22%2C%22%23a21caf%22%2C%22%232563eb%22%2C%22%23dc2626%22%2C%22%23059669%22%2C%22%23c026d3%22%2C%22%23d97706%22%2C%22%23475569%22%2C%22%239333ea%22%2C%22%2316a34a%22%2C%22%23a16207%22%2C%22%23db2777%22%2C%22%23e11d48%22%2C%22%2365a30d%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Easy%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)

[![Medium](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22Jev%201.13.0%22%2C%22Tev1%204B%22%2C%22AutoTrust%20JEV-9B%22%2C%22Mica%20v0.1%204B%22%2C%22AutoTrust%20JEV-27B%22%2C%22AutoJev%2027B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22Kev%204B%22%2C%22decider-4b%20v2%22%2C%22Rune%20v3%2026B-A4B%22%2C%22Clef%2027B%22%2C%22NeoHorse-Jev%204B%22%2C%22Clef-Flash%209B%22%2C%22JevK5%20v0.2%22%2C%22Tev1%200.8B%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22Plumb%204B%22%2C%22semif%22%2C%22JevEmbed%204B%22%2C%22RSI-Jev%20v3.0%22%2C%22Lev%204B%22%2C%22Strands%202B%22%2C%22decider%200.8B%22%2C%22GLiNER%22%2C%22Laya%22%2C%22jeff%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C99.0%2C99.0%2C99.0%2C97.9%2C95.8%2C93.8%2C91.7%2C89.6%2C87.5%2C86.5%2C85.4%2C85.4%2C80.2%2C79.2%2C76.0%2C72.9%2C40.6%2C38.5%2C35.4%5D%2C%22backgroundColor%22%3A%5B%22%230d9488%22%2C%22%23ea580c%22%2C%22%23a21caf%22%2C%22%23dc2626%22%2C%22%23c026d3%22%2C%22%23d97706%22%2C%22%239333ea%22%2C%22%2392400e%22%2C%22%230f766e%22%2C%22%23475569%22%2C%22%23059669%22%2C%22%232563eb%22%2C%22%2310b981%22%2C%22%230369a1%22%2C%22%230891b2%22%2C%22%230ea5e9%22%2C%22%23b45309%22%2C%22%234f46e5%22%2C%22%23a16207%22%2C%22%2338bdf8%22%2C%22%237c3aed%22%2C%22%2384cc16%22%2C%22%2316a34a%22%2C%22%2365a30d%22%2C%22%23e11d48%22%2C%22%23db2777%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Medium%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22Jev%201.13.0%22%2C%22Tev1%204B%22%2C%22AutoTrust%20JEV-9B%22%2C%22Mica%20v0.1%204B%22%2C%22AutoTrust%20JEV-27B%22%2C%22AutoJev%2027B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22Kev%204B%22%2C%22decider-4b%20v2%22%2C%22Rune%20v3%2026B-A4B%22%2C%22Clef%2027B%22%2C%22NeoHorse-Jev%204B%22%2C%22Clef-Flash%209B%22%2C%22JevK5%20v0.2%22%2C%22Tev1%200.8B%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22Plumb%204B%22%2C%22semif%22%2C%22JevEmbed%204B%22%2C%22RSI-Jev%20v3.0%22%2C%22Lev%204B%22%2C%22Strands%202B%22%2C%22decider%200.8B%22%2C%22GLiNER%22%2C%22Laya%22%2C%22jeff%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C100.0%2C99.0%2C99.0%2C99.0%2C97.9%2C95.8%2C93.8%2C91.7%2C89.6%2C87.5%2C86.5%2C85.4%2C85.4%2C80.2%2C79.2%2C76.0%2C72.9%2C40.6%2C38.5%2C35.4%5D%2C%22backgroundColor%22%3A%5B%22%230d9488%22%2C%22%23ea580c%22%2C%22%23a21caf%22%2C%22%23dc2626%22%2C%22%23c026d3%22%2C%22%23d97706%22%2C%22%239333ea%22%2C%22%2392400e%22%2C%22%230f766e%22%2C%22%23475569%22%2C%22%23059669%22%2C%22%232563eb%22%2C%22%2310b981%22%2C%22%230369a1%22%2C%22%230891b2%22%2C%22%230ea5e9%22%2C%22%23b45309%22%2C%22%234f46e5%22%2C%22%23a16207%22%2C%22%2338bdf8%22%2C%22%237c3aed%22%2C%22%2384cc16%22%2C%22%2316a34a%22%2C%22%2365a30d%22%2C%22%23e11d48%22%2C%22%23db2777%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Medium%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)

[![Hard](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22AutoTrust%20JEV-27B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22Jev%201.13.0%22%2C%22AutoJev%2027B%22%2C%22Rune%20v3%2026B-A4B%22%2C%22decider-4b%20v2%22%2C%22NeoHorse-Jev%204B%22%2C%22Tev1%204B%22%2C%22AutoTrust%20JEV-9B%22%2C%22Clef%2027B%22%2C%22Mica%20v0.1%204B%22%2C%22Kev%204B%22%2C%22Clef-Flash%209B%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22RSI-Jev%20v3.0%22%2C%22Strands%202B%22%2C%22GLiNER%22%2C%22JevEmbed%204B%22%2C%22semif%22%2C%22decider%200.8B%22%2C%22Tev1%200.8B%22%2C%22Lev%204B%22%2C%22Laya%22%2C%22jeff%22%2C%22JevK5%20v0.2%22%2C%22Plumb%204B%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B94.6%2C94.6%2C93.2%2C89.9%2C89.6%2C80.1%2C78.6%2C76.5%2C73.8%2C70.8%2C69.9%2C69.3%2C66.1%2C65.5%2C64.3%2C61.6%2C57.7%2C56.0%2C53.0%2C52.4%2C52.4%2C50.3%2C49.1%2C41.7%2C38.7%2C37.5%5D%2C%22backgroundColor%22%3A%5B%22%23c026d3%22%2C%22%239333ea%22%2C%22%230d9488%22%2C%22%23d97706%22%2C%22%23475569%22%2C%22%230f766e%22%2C%22%232563eb%22%2C%22%23ea580c%22%2C%22%23a21caf%22%2C%22%23059669%22%2C%22%23dc2626%22%2C%22%2392400e%22%2C%22%2310b981%22%2C%22%230ea5e9%22%2C%22%2338bdf8%22%2C%22%2384cc16%22%2C%22%2365a30d%22%2C%22%23a16207%22%2C%22%234f46e5%22%2C%22%2316a34a%22%2C%22%230891b2%22%2C%22%237c3aed%22%2C%22%23e11d48%22%2C%22%23db2777%22%2C%22%230369a1%22%2C%22%23b45309%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Hard%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)](https://quickchart.io/chart?version=3&w=640&h=716&bkg=white&c=%7B%22type%22%3A%22bar%22%2C%22data%22%3A%7B%22labels%22%3A%5B%22AutoTrust%20JEV-27B%22%2C%22AutoTrust%20JEV-27B-VL%22%2C%22Jev%201.13.0%22%2C%22AutoJev%2027B%22%2C%22Rune%20v3%2026B-A4B%22%2C%22decider-4b%20v2%22%2C%22NeoHorse-Jev%204B%22%2C%22Tev1%204B%22%2C%22AutoTrust%20JEV-9B%22%2C%22Clef%2027B%22%2C%22Mica%20v0.1%204B%22%2C%22Kev%204B%22%2C%22Clef-Flash%209B%22%2C%22RSI-Jev%20v4.0-VL%22%2C%22RSI-Jev%20v3.0%22%2C%22Strands%202B%22%2C%22GLiNER%22%2C%22JevEmbed%204B%22%2C%22semif%22%2C%22decider%200.8B%22%2C%22Tev1%200.8B%22%2C%22Lev%204B%22%2C%22Laya%22%2C%22jeff%22%2C%22JevK5%20v0.2%22%2C%22Plumb%204B%22%5D%2C%22datasets%22%3A%5B%7B%22data%22%3A%5B94.6%2C94.6%2C93.2%2C89.9%2C89.6%2C80.1%2C78.6%2C76.5%2C73.8%2C70.8%2C69.9%2C69.3%2C66.1%2C65.5%2C64.3%2C61.6%2C57.7%2C56.0%2C53.0%2C52.4%2C52.4%2C50.3%2C49.1%2C41.7%2C38.7%2C37.5%5D%2C%22backgroundColor%22%3A%5B%22%23c026d3%22%2C%22%239333ea%22%2C%22%230d9488%22%2C%22%23d97706%22%2C%22%23475569%22%2C%22%230f766e%22%2C%22%232563eb%22%2C%22%23ea580c%22%2C%22%23a21caf%22%2C%22%23059669%22%2C%22%23dc2626%22%2C%22%2392400e%22%2C%22%2310b981%22%2C%22%230ea5e9%22%2C%22%2338bdf8%22%2C%22%2384cc16%22%2C%22%2365a30d%22%2C%22%23a16207%22%2C%22%234f46e5%22%2C%22%2316a34a%22%2C%22%230891b2%22%2C%22%237c3aed%22%2C%22%23e11d48%22%2C%22%23db2777%22%2C%22%230369a1%22%2C%22%23b45309%22%5D%2C%22borderWidth%22%3A0%2C%22barPercentage%22%3A0.82%7D%5D%7D%2C%22options%22%3A%7B%22indexAxis%22%3A%22y%22%2C%22layout%22%3A%7B%22padding%22%3A%7B%22right%22%3A40%7D%7D%2C%22plugins%22%3A%7B%22legend%22%3A%7B%22display%22%3Afalse%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22Hard%22%2C%22font%22%3A%7B%22size%22%3A16%7D%7D%2C%22datalabels%22%3A%7B%22display%22%3Atrue%2C%22anchor%22%3A%22end%22%2C%22align%22%3A%22end%22%2C%22clamp%22%3Atrue%2C%22color%22%3A%22%23111827%22%2C%22font%22%3A%7B%22size%22%3A11%7D%2C%22formatter%22%3Afunction%28v%29%7Breturn%20v.toFixed%281%29%7D%7D%7D%2C%22scales%22%3A%7B%22x%22%3A%7B%22min%22%3A0%2C%22max%22%3A100%2C%22ticks%22%3A%7B%22stepSize%22%3A20%7D%2C%22title%22%3A%7B%22display%22%3Atrue%2C%22text%22%3A%22%25%22%7D%7D%2C%22y%22%3A%7B%22grid%22%3A%7B%22display%22%3Afalse%7D%7D%7D%7D%7D)

| Model | Easy /72 | Medium /96 | Hard /336 | All /504 | Hard priority reversals /29 |
| --- | ---: | ---: | ---: | ---: | ---: |
| AutoTrust JEV-27B | 72 | 96 | 318 | 486 | 28 |
| AutoTrust JEV-27B-VL | 72 | 96 | 318 | 486 | 28 |
| Jev `jev-1.13.0` | 72 | 96 | 313 | 481 | 28 |
| AutoJev-27B | 72 | 96 | 302 | 470 | 29 |
| Rune v3 26B-A4B | 72 | 95 | 301 | 468 | 24 |
| decider-4b v2 | 72 | 95 | 269 | 436 | 14 |
| NeoHorse-Jev-4B | 72 | 92 | 264 | 428 | 5 |
| Tev1-4B | 72 | 96 | 257 | 425 | 15 |
| AutoTrust JEV-9B | 72 | 96 | 248 | 416 | 4 |
| Clef 27B | 72 | 94 | 238 | 404 | 4 |
| Mica v0.1 4B | 72 | 96 | 235 | 403 | 7 |
| Kev 4B | 72 | 95 | 233 | 400 | 9 |
| Clef-Flash 9B | 72 | 90 | 222 | 384 | 3 |
| RSI-Jev v4.0-VL 2B | 72 | 84 | 220 | 376 | 0 |
| RSI-Jev v3.0 2B | 72 | 77 | 216 | 365 | 2 |
| Strands Decider 2B | 72 | 73 | 207 | 352 | 1 |
| Tev1-0.8B | 72 | 86 | 176 | 334 | 0 |
| SemIf, Qwen3.5-4B | 72 | 82 | 178 | 332 | 2 |
| JevEmbed Qwen3-Embedding-4B | 60 | 82 | 188 | 330 | 1 |
| Lev 4B | 72 | 76 | 169 | 317 | 1 |
| decider-0.8b | 67 | 70 | 176 | 313 | 1 |
| JevK5 4B v0.2 | 72 | 88 | 130 | 290 | 2 |
| Plumb-4B | 72 | 83 | 126 | 281 | 1 |
| GLiNER2.5-Decide | 24 | 39 | 194 | 257 | 0 |
| Laya multilingual | 38 | 37 | 165 | 240 | 0 |
| jeff (GLiFormer large) | 45 | 34 | 140 | 219 | 0 |

Easy and medium are regression and "insufficient information" controls.
Hard carries the difference: a priority reversal counts only if both ends
of a same-case pair are right in both orders. Always answering B already
scores 194/336 on hard. All 6,552 calls are valid, with no errors. Adapters
differ, so these are task-level scores, not weight comparisons.
[Details and disclosures](data/semantic-decisions/v0.0.2/natural-zh-Hant/open-models.md).

Earlier banks (`semantic-decisions@v0.0.1`, the controlled slices, r2) are
retired. v0.0.1 survives only at package tag `v0.0.1`, and none of their
scores is pooled with v0.0.2. No model redistribution is authorized.

## License

Apache-2.0
