Metadata-Version: 2.4
Name: toc-cluster
Version: 0.3.0
Summary: Trust Orbit Computation: clustering with confidence
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.20
Requires-Dist: scikit-learn>=1.0
Requires-Dist: scipy>=1.7
Provides-Extra: gpu
Requires-Dist: torch>=2.0; extra == "gpu"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"

# toc-cluster

**Trust Orbit Computation** - a clustering algorithm that tells you not just which cluster each point belongs to, but *how confident* that assignment is.

## Install

```bash
pip install toc-cluster
```

## Quick Start

```python
from toc import TOC
import numpy as np

X = np.random.randn(1000, 50)   # your data, shape (n_samples, n_features)

model = TOC(n_clusters=5)
model.fit(X)

print(model.labels_)          # cluster label for every point, shape (1000,)
print(model.states_)          # 'MERGE', 'ORBIT', or 'ESCAPE' per point
print(model.orbit_percent_)   # ORBIT% per cluster — measures boundary ambiguity
model.summary()               # print full results table
```

## The Three States

| State | Meaning | What it tells you |
|-------|---------|-------------------|
| MERGE | Confidently inside a cluster | High-quality assignment |
| ORBIT | Genuinely between clusters | Boundary — assignment uncertain |
| ESCAPE | Outlier | Far from any cluster structure |

## Why Use TOC Instead of K-Means?

K-Means assigns every point with equal confidence. TOC tells you which assignments are confident (MERGE) and which are uncertain (ORBIT).

Numbers below are ARI, measured against this package's current (v0.3.0)
code. Two TOC numbers are reported for each dataset: **unsupervised**
(`model.fit(X)`, no labels — the mode shown in Quick Start above) and
**semi-supervised** (`model.fit(X, y=...)`, dominance computed from known
labels).

**Samusik CyTOF** (514,386 cells, 24 populations):
- K-Means: ARI 0.597
- FlowSOM (measured head-to-head, 3 seeds): ARI mean 0.763, best 0.890
- TOC, unsupervised: ARI 0.823
- TOC, semi-supervised: ARI 0.818

**Levine32 CyTOF** (104,184 cells, 14 populations):
- K-Means: ARI 0.649
- FlowSOM (measured head-to-head, 3 seeds): ARI mean 0.913, best 0.920
- TOC, unsupervised: ARI 0.907
- TOC, semi-supervised: ARI 0.849

**PBMC 10k scRNA-seq** (11,627 cells, 15 Leiden clusters as ground truth):
- K-Means: ARI 0.684
- TOC, unsupervised: ARI 0.782
- TOC, semi-supervised: ARI 0.897

**Human bone marrow** (263,159 cells, 55 cell states):
- K-Means: ARI 0.368
- TOC, unsupervised: ARI 0.397
- TOC, semi-supervised: ARI 0.522

Unsupervised TOC beats K-Means on all four datasets. The unsupervised
refinement loop (v0.3.0) uses a **label-consistent centroid update**: each
refinement iteration recomputes cluster centres as the mean of the converged
MERGE points per current cluster and reassigns points to the nearest centre,
instead of re-clustering the converged positions with a fresh K-Means. The
re-clustering variant tended to merge adjacent clusters that Phase 1
contracts into nearby clumps, which on PBMC 10k destroyed a good
preliminary partition outright. One caveat: on continuum-like data with
many overlapping states and a weak preliminary partition (the bone-marrow
dataset above), preserving cluster identity also preserves the partition's
initial mistakes — there the re-clustering variant scored higher
(ARI 0.471); it remains available via `_assign_strategy="seeded_kmeans"`.

**About the FlowSOM comparison.** FlowSOM numbers were measured with the
official Python port (flowsom 0.2.2) on the identical labeled-cell subsets,
with the true K given as the metacluster count, 3 seeds, default 10×10 grid.
FlowSOM was scored on its native input (arcsinh(x/5), unstandardised) — it
degrades badly on the StandardScaler-ed matrix TOC uses (e.g. Levine32 mean
ARI 0.62 there). On these terms TOC beats FlowSOM's per-seed mean on Samusik
(0.823 vs 0.763) but not FlowSOM's best seed (0.890), and the two are
statistically tied on Levine32 (0.907 vs mean 0.913). FlowSOM shows large
seed-to-seed variance on Samusik (0.697–0.890) and is substantially faster
(~35 s vs ~17 min on 514k cells). TOC's differentiator is the per-point
confidence output (MERGE/ORBIT/ESCAPE), not raw ARI dominance over FlowSOM.

## When to Use TOC

TOC works best when:
- IntrD (intrinsic dimensionality) > 10
- N/K (average points per cluster) > 150
- Data is preprocessed (normalized, PCA-reduced)

Not recommended for: raw count scRNA-seq, small tabular datasets, graph-structured data.

## GPU Support

TOC automatically uses GPU if CUDA is available. No code changes needed.

```python
# Same code works on CPU and GPU
model = TOC(n_clusters=5)
model.fit(X)   # uses GPU automatically if available
```

## What ORBIT Points Converge To

Each ORBIT point is pushed toward its trusted neighbours by a fixed-size step
per neighbour (independent of distance): `s_ij = pi * T_ij * alpha_fixed`.
Summed over all trusted neighbours, this update is exactly one step of
unit-step gradient descent on the convex objective

```
f_i(p) = sum_j  s_ij * ||p - x_j||
```

— the trust-weighted sum of distances from a candidate position `p` to point
`i`'s neighbours. Gradient descent on this objective converges to its
minimizer: the **trust-weighted geometric median (Fermat–Weber point)** of
the point's local neighbourhood. That's what an ORBIT point actually settles
at — a data-dependent location, not a fixed universal distance. We verified
this directly: solving for the weighted geometric median with an independent
optimizer landed on the exact same point (to solver precision) that the real
Phase 2 dynamics converges a real ORBIT point to, and with the many
neighbours a typical point has (`k_trust=25` by default), the point locks
onto that median with zero residual oscillation.

**The degenerate case.** With only a single dominant neighbour (or two
neighbours of equal weight on opposite sides), the weighted-median objective
has no smooth interior minimum — fixed-length-step descent can never land
exactly on it, so the point overshoots forever and locks into a stable
2-cycle instead of a single point. The half-width of that oscillation is

```
r* = (pi * alpha_fixed) / 2
```

which is `0.12` with the default `pi=0.6`, `alpha_fixed=0.4`. This is a real
and exact result, but it describes the single/two-neighbour limit
specifically — **it is not the general behaviour of ORBIT points**, which
typically have many neighbours and converge to their local weighted median
instead of oscillating. Phase 2's internal step cap defaults to exactly
`pi * alpha_fixed` (a no-op, since trust values are always in `[0, 1]`); pass
`_max_step_fixed` explicitly for a tighter cap, which pulls the degenerate-case
radius down to `_max_step_fixed / 2` instead of `r*`.

## ORBIT% — The Novel Output

ORBIT% per cluster measures how many of a cluster's members sit at the boundary with other clusters. High ORBIT% = ambiguous cluster. Low ORBIT% = well-defined cluster.

Example from an earlier run on human bone marrow (263,159 cells), semi-supervised:
- MPP-MyLy (multipotent progenitor): 89.4% ORBIT — between myeloid and lymphoid lineages
- Plasma Cell (terminally differentiated): 0.9% ORBIT — most distinct cell type

This qualitative pattern (progenitors high-ORBIT, terminally differentiated
cells low-ORBIT) is the kind of result TOC is meant to produce, but these
exact percentages predate the v0.2.0 fixes above and have not been
re-verified against the current package — only the aggregate ARI numbers in
the previous section have been. Treat the specific figures here as
illustrative, not confirmed, until re-run.

## Citation

[Paper link — coming soon]

## License

MIT
