Metadata-Version: 2.4
Name: haspi
Version: 0.1.3
Summary: Inverse soft-Q learning (IQ-Learn) reward extraction for hate-speech analysis on the One Million Posts Corpus
Author-email: Tobias Kietreiber <tobias.kietreiber@ustp.at>
License-Expression: MIT
Project-URL: Homepage, https://github.com/fhstp/haspi
Project-URL: Documentation, https://haspi.readthedocs.io
Project-URL: Repository, https://github.com/fhstp/haspi
Project-URL: Issues, https://github.com/fhstp/haspi/issues
Keywords: hate-speech,content-moderation,nlp,german,imitation-learning,iq-learn,reward-learning,explainability
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Natural Language :: German
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jax<0.8,>=0.7
Requires-Dist: flax<0.13,>=0.12
Requires-Dist: optax<0.3,>=0.2
Requires-Dist: transformers<4.40,>=4.38
Requires-Dist: safetensors
Requires-Dist: numpy
Requires-Dist: scikit-learn
Requires-Dist: tqdm
Provides-Extra: cuda12
Requires-Dist: jax[cuda12]<0.8,>=0.7; extra == "cuda12"
Provides-Extra: gemma
Requires-Dist: torch; extra == "gemma"
Provides-Extra: leolm
Requires-Dist: torch; extra == "leolm"
Requires-Dist: accelerate; extra == "leolm"
Requires-Dist: sentencepiece; extra == "leolm"
Provides-Extra: demo
Requires-Dist: gradio<6,>=4; extra == "demo"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=7; extra == "docs"
Requires-Dist: myst-parser; extra == "docs"
Requires-Dist: furo; extra == "docs"
Dynamic: license-file

<div align="center">
<img src="https://raw.githubusercontent.com/fhstp/HaSPI/main/images/Logo_with_background.svg" alt="logo"></img>
</div>

# HaSPI

**haspi** scores German comments for hate speech and moderation removal, and tells you
**which words drove each verdict** — an explainable reward built on the
[One Million Posts Corpus](https://ofai.github.io/million-post-corpus/) (DerStandard,
OFAI). Rooted in inverse soft-Q learning (IQ-Learn); JAX / flax-nnx with an optional
PyTorch encoder.

```console
$ haspi-explain
> Du bist ein widerlicher Idiot und gehörst abgeschoben.
  → HATE   (score +1.06, threshold -0.02)
     toward HATE:     'bist'+0.39  'widerlicher'+0.22  'gehörst'+0.22  'Du'+0.17
     toward non-hate: 'abgeschoben.'-0.10
> Danke für den sehr informativen Artikel.
  → non-hate   (score -0.35, threshold -0.02)
     toward HATE:     'Danke'+0.15  'informativen'+0.11
     toward non-hate: 'den'-0.10  'für'-0.08  'sehr'-0.05
```

The current method — the **sequence-χ² reward** — runs a comment through a frozen German
language model (LeoLM-7b), mean-pools its hidden states, and applies a small linear χ²
reward head. Because the whole pipeline is linear, the score decomposes **exactly** into
per-word contributions, and scoring is one model pass plus a dot product.

| held-out AUROC | |
|---|---|
| hate-vs-neutral (10-fold CV) | **0.76** |
| moderation removal (RP-Mod, same splits as Assenmacher et al.) | **0.80** (their fine-tuned BERT: 0.79) |

**Full documentation & the method diagram:** https://haspi.readthedocs.io

## Install

```bash
pip install "haspi[leolm]"          # the current method: + torch, accelerate, sentencepiece
pip install "haspi[leolm,cuda12]"   # …with CUDA-12 JAX for GPU
```

Python ≥ 3.10. The reward head is CPU/JAX; running the language model (feature extraction
and the explainer) needs a GPU. Note the pinned `transformers>=4.38,<4.40` — Hugging Face
v5 dropped Flax support; the 4.39.x line is the last that ships it.

The fitted reward files are distributed via [GitHub Releases](https://github.com/fhstp/haspi/releases),
not committed to the repo. Fetch them into `models/` before running the explainers or the demo
(or [fit your own](#fit-your-own-reward)):

```bash
gh release download --repo fhstp/haspi --pattern '*.pkl' --dir models/
```

## Try the demo

`haspi-demo` launches a small web app — the project schematic made real. Type or **sample** a German
comment and see the verdict, a green→red risk meter, and the comment with each word coloured by its
exact contribution (risky words highlighted). It handles hate and moderation (with an optional
article/thread context), and a seeded "🎲 Aus Korpus" button pulls a reproducible real corpus comment.

```bash
pip install "haspi[demo,leolm]"
haspi-demo                          # → http://<host>:8000   (needs a fitted reward + GPU)
```

## Classify and explain

The main tools are the two explainers. They load a fitted reward
(`models/leolm_chi2_reward.pkl`) and the language model, then score whatever you give them.

```bash
haspi-explain                 # interactive: type a comment, get a verdict + word drivers
haspi-explain-corpus --n 8    # sample labelled corpus posts, show prediction vs. truth
```

Useful flags: `--topk N` (words shown per direction), `--level {word,token}` (word-level by
default; `token` shows the raw sub-word pieces), `--reward PATH` (a different reward file).

### Reading the output

- **`score` vs `threshold`** — the verdict. Score above the calibrated threshold → `HATE`,
  below → `non-hate`. The threshold was chosen once on the training set.
- **`toward HATE` / `toward non-hate`** — the words that pushed the score up or down, each
  with its exact contribution. These are **real, additive attributions**: they sum to the
  decision (relative to an average comment), not a heuristic highlight. A neutral comment's
  words roughly cancel; a slur or aggressive phrasing spikes.

> **Note.** The attribution is faithful to the decision, but the model reasons at the
> sentence level — trust the **overall verdict** and the **ranking of content words**, not
> any single function word (you'll sometimes see `und`, `.`, etc. carry a little weight).

### In Python

```python
from haspi.sequence import LeoLMScorer, format_attribution

scorer = LeoLMScorer("models/leolm_chi2_reward.pkl")
res = scorer.score("Du bist ein Idiot.")
# res = {"score": ..., "label": "HATE"/"non-hate", "threshold": ..., "tokens": [(word, contribution), ...], "n": ...}

toward_hate, toward_non_hate = format_attribution(res, topk=5)
```

## Fit your own reward

Two steps — extract frozen features once (GPU), then fit the linear head (CPU/JAX). Needs
`data/labels.npz` and `data/corpus.sqlite3` from `haspi-prepare-corpus`.

```bash
haspi-extract-features --model LeoLM/leo-hessianai-7b --out data/leolm7b_feats.npz
haspi-fit-reward       --features data/leolm7b_feats.npz --out models/leolm_chi2_reward.pkl
```

`haspi-fit-reward` standardises the features, reduces them with whitened PCA-512, fits the
χ² head, **folds** the standardise→PCA→head chain into a single vector (printing
`fold max|Δ| ≈ 1e-6` as a reproduction check), calibrates the decision threshold, and writes
a small reward file. A different encoder is fine — swap `--model` (any Llama/Mistral/Qwen2
architecture works with the pinned `transformers`).

To fit from cached features without any torch:

```python
import numpy as np
from haspi.data import hate_mask
from haspi.sequence import fit_reward

labels = dict(np.load("data/labels.npz"))
feats = np.load("data/leolm7b_feats.npz")["mean"]
payload, max_delta, auc = fit_reward(feats, hate_mask(labels))   # payload → pickle it
```

## Context-aware moderation

For the online/offline moderation task (a post kept vs. removed by moderators), a comment can be
scored with its **thread and article context** — the parent comment and the article's title and
topic path — prepended before encoding.

```bash
haspi-moderation-context --variants comment article topic
```

```console
variant       random split  article-disjoint
--------------------------------------------
comment              0.722             0.726
article              0.757             0.750
topic                0.697             0.653
```

Two things make this honest. First, the context conditions the encoder but the reward **pools and
attributes only the comment tokens** — the article/thread words shape the comment's representation
(causal attention) yet never enter the pooled feature, so the reward can't just memorise *which
article* a post is under. Second, evaluation is on an **article-disjoint** split (whole articles
assigned to train or test, never both). The proof that this matters is model-free: predicting a
post's label purely from the training-set removal rate of its **article ID** (no text, no model)
scores **0.70** AUROC on a random split and exactly **0.50** (chance) when articles are disjoint — the
leakage is real and there to be exploited. Because we pool comment tokens only, the reward barely
touches it: the article-context reward moves just 0.757 → 0.750 from random to disjoint (a whole-blob
pooling of context+comment drops twice as far), and the small article-context gain over comment-only
(0.726 → 0.750) **survives** the honest split. The rule still stands — **evaluate context-aware
moderation on article-/thread-disjoint splits** — because the inflation is large in
article-concentrated samples (the article-ID control reaches 0.98 on a random split there).

`haspi-moderation-context --save-reward models/moderation.pkl` writes a moderation reward that
`haspi-explain --reward models/moderation.pkl` scores and explains as `remove`/`keep`. (Our text-only
reward is comparable to the prior text-only baseline on this corpus, 0.728; the rigorous external
comparison is on the **RP-Mod** benchmark, where the frozen reward matches a fine-tuned German BERT —
see the docs.)

## How it works (in one paragraph)

A frozen German decoder LM encodes the comment; its final hidden states are mean-pooled to
one vector *φ*. A linear χ² reward head — the one-step reduction of two-distribution
IQ-Learn — scores it, trained so an *expert* class (non-hate) and an *anchor* class (hate)
are separated. Standardisation and PCA-whitening condition the head; the whole linear chain
then folds into a single weight vector *g* with `r = g·φ + const`. Since *φ* is a mean over
token states, each token's contribution is exactly `orient · (1/L) · g·(hₜ − μ)` and the
parts sum to the score — that's where the per-word explanations come from. The diagram and
the full derivation are in the [documentation](https://haspi.readthedocs.io).

---

## Superseded: two-distribution IQ-Learn

The original method — a token-level IQ-Learn reward trained end to end on frozen GPT-2 — is
kept for reproducibility. It plateaus at ~0.55 AUROC on this corpus (the reward tracks post
length more than hate semantics), which is what motivated the sequence-χ² reformulation. Its
CLIs print a superseded notice; `haspi.CURRENT_METHOD` / `haspi.METHODS` record the full
method-evolution story.

```bash
haspi-prepare-corpus --num-sequences 50000 --maxlength 128
haspi-train --task hate --avenue B --lm-mode frozen --qhead factored \
    --weight-decay 1.0 --epochs 25 --save-path models/hate_frozen.pkl
haspi-classify --model models/hate_frozen.pkl        # REPL with per-token IQ-Learn rewards
haspi-evaluate --model models/hate_frozen.pkl --sweep
```

`haspi-train --cv` runs the 10-fold cross-validation protocol. See the
[architecture guide](https://haspi.readthedocs.io) for the agent internals, tuning notes
(`gamma ≤ 0.95`, `--target-entropy` near the policy's natural entropy, the `factored`
Q-head), and the library API.

## Tests & docs

```bash
pytest                                  # fast unit suite, no model downloads
pytest -m slow                          # end-to-end on real models (needs the download / GPU)
pip install -e ".[docs]" && sphinx-build -W -b html docs docs/_build/html   # build the docs
```
