Metadata-Version: 2.4
Name: triepick
Version: 0.1.0
Summary: Fast trie-planned candidate scoring for user-managed vLLM instances.
Author: Son The Nguyen
License: MIT
Project-URL: Homepage, https://github.com/sonthenguyen/triepick
Project-URL: Repository, https://github.com/sonthenguyen/triepick
Project-URL: Issues, https://github.com/sonthenguyen/triepick/issues
Keywords: vllm,llm,inference,scoring,classification,decision-making
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Dynamic: license-file

# TriePick

### Turn your LLM into an efficient decision model.

Most agent decisions are small: choose a route, pick the next action, select a tool, classify a state.

You probably don't need a separate decision model architecture. TriePick takes the LLM you already run with vLLM, scores freeform candidate decisions directly, and returns the winner with candidate probabilities.

<p align="center">
  <img
    src="https://raw.githubusercontent.com/sonthenguyen/triepick/main/assets/triepick-demo.gif"
    alt="TriePick choosing among freeform candidates"
    width="900"
  >
</p>


## Quick start

```python
from vllm import LLM
from triepick import TriePick

llm = LLM(
    model="Qwen/Qwen3.5-4B",
)

tp = TriePick(
    llm=llm,
)

result = tp.choose(
    state=(
        "The system is routing requests to the wrong category. "
        "What is the most likely cause?"
    ),
    candidates=[
        "misconfiguration",
        "misclassification",
        "bad training data",
        "outdated routing rules",
    ],
)

print(result.candidate)
print(result.probabilities)
```

TriePick wraps the exact `llm` instance you give it; model loading and vLLM configuration stay yours.


## How it works

Most multiple-choice approaches turn answers into labels like `A`, `B`, `C`, `D`.

TriePick scores the **actual candidate strings**.

It builds a token trie and evaluates only the next-token branches needed to distinguish the candidates. Shared prefixes are reused, and scoring stops once a candidate is uniquely identified.

All scoring rows are flattened into **one batched `llm.generate()` call**.

<p align="center">
  <img
    src="https://raw.githubusercontent.com/sonthenguyen/triepick/main/assets/triepick-scoring.png"
    alt="How TriePick scores freeform candidates"
    width="900"
  >
</p>

For example, `misconfiguration` and `misclassification` share the token branch `mis`, so TriePick evaluates that shared branch once and only scores the next branch needed to distinguish them.


## Batch decisions

TriePick can score multiple independent decisions together.

```python
batch = tp.choose_batch([
    {
        "state": "The payment failed because the card expired.",
        "candidates": [
            "retry payment",
            "request new payment method",
            "cancel account",
        ],
    },
    {
        "state": "A support request contains a billing dispute.",
        "candidates": [
            "billing",
            "technical support",
            "sales",
        ],
    },
])
```

Decision points across the batch are flattened together before inference and scored in a single vLLM call.

```python
print("generate_calls:", batch.generate_calls)
print("scoring_rows:", batch.scoring_rows)
print("total_candidates:", batch.total_candidates)
print("inference_ms:", round(batch.inference_ms, 2))
```


## Jev / System One

**LLMs are great System One models.**

TriePick includes a Jev-compatible adapter for System One-style decision APIs.

```python
from triepick.adapters import JevAdapter

jev = JevAdapter(tp)
```

The adapter supports Jev-style `noul`, `choice`, and `score` operations while using the same TriePick scoring and batching path.


## Multimodal

If your vLLM model supports image inputs, TriePick can score decisions using images directly.

```python
result = tp.choose(
    state="Which category best describes this image?",
    candidates=[
        "document",
        "screenshot",
        "photograph",
    ],
    image=image,
)
```

Images stay in-process and are passed directly to vLLM.


## Results

`choose()` returns a `ChoiceResult` containing the selected candidate and candidate probabilities.

```python
result.candidate
result.probabilities
```

It also exposes scoring diagnostics:

```python
result.decision_nodes
result.scoring_rows
result.inference_ms
result.generate_calls
```

Tie information is available too:

```python
result.is_tie
result.tied_candidates
result.margin
```

On an exact tie, `is_tie` is `True` and `tied_candidates` contains the tied candidates.


## Benchmark

On all **231 redistributable JevBench public decisions** with `Qwen/Qwen3.5-4B`:

| Path | Accuracy | Median latency |
| --- | ---: | ---: |
| Native TriePick | **73.6%** | **16.5 ms** |
| JevAdapter | **73.6%** | **17.7 ms** |

Native TriePick and `JevAdapter` agreed on **99.6%** of predictions.

The benchmark used an NVIDIA RTX PRO 6000 Blackwell Server Edition with vLLM `0.30.1rc1.dev190+g4bb804cc5`, Model Runner V2, and Fastokens.

This is a reproducible public-set evaluation, not an official sealed Benchmark Heaven leaderboard submission.

[Full benchmark details](benchmarks/jevbench-public-qwen3.5-4b.md)


## Compatibility

Currently tested with:

| Model | vLLM | Runner |
| --- | --- | --- |
| `Qwen/Qwen3.5-4B` | `0.30.1rc1.dev190+g4bb804cc5` | Model Runner V2 |

TriePick is not tied to Qwen.

At the scoring layer, it needs a supplied vLLM instance with the capabilities required for targeted token scoring, especially `logprob_token_ids`.

Model Runner V2 is the currently tested path. TriePick does **not** select, configure, or force a particular model runner.


## Installation

From a local checkout:

```bash
pip install .
```

Or install without changing the surrounding vLLM environment:

```bash
pip install --no-deps .
```

Then pass your existing `vllm.LLM` instance to TriePick:

```python
from triepick import TriePick

tp = TriePick(
    llm=llm,
)
```


## Tutorials

- [**Quick Start**](https://github.com/sonthenguyen/triepick/blob/main/tutorials/01_quickstart.ipynb) — install TriePick, load Qwen3.5-4B, and make your first decision
- [**Batching and Multimodal**](https://github.com/sonthenguyen/triepick/blob/main/tutorials/02_batch_and_multimodal.ipynb) — batch decisions efficiently and score image-based inputs
- [**Jev / System One**](https://github.com/sonthenguyen/triepick/blob/main/tutorials/03_jev_system_one.ipynb) — use `noul`, `choice`, `score`, and mixed System One requests


## The core idea

A decision model usually does not need to generate every answer.

TriePick scores the token branches that distinguish the candidates and batches those evaluations into one model call.
