Metadata-Version: 2.4
Name: GGDLPC
Version: 0.0.2
Summary: Lossless compression via generalized Gaussian modeling of linear predictive coding residuals
Author-email: Dan Jacobellis <danjacobellis@utexas.edu>
Project-URL: Homepage, https://danjacobellis.net
Project-URL: Repository, https://github.com/danjacobellis/GGDLPC
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: torch

# GGDLPC

**Generalized Gaussian Distribution Linear Predictive Coding** — a lossless compression method for scalar-quantized signals, autoencoder latents, and neural network parameter values.

## How it works

The core observation is that prediction residuals of many integer-valued signals — particularly quantized autoencoder latents — are well-approximated by a Generalized Gaussian Distribution (GGD). Since the KL divergence between the actual distribution and a fitted GGD is small (~0.75% above the conditional entropy bound), Huffman codes designed for the GGD are near-optimal for the actual data.

**Linear prediction** removes spatial/temporal correlation: each sample is predicted as a linear combination of causal neighbors, and the prediction residual is entropy coded.

**Per-channel parametric model**: each channel is described by a small number of scalars:
- A **scale relationship**: the GGD scale parameter varies as an affine function of a local activity statistic
- A **shape relationship**: the GGD shape parameter β varies as an affine function of log-scale
- Linear predictor weights and bias

From these scalars, all Huffman tables across ~41 log-spaced context bins are **generated analytically** at load time — no fitted tables are stored or transmitted.

**Embedded run mode**: for contexts where the conditional entropy falls below 1 bit (where symbol codes are inherently wasteful), a zero-run mode with elementary Golomb codes recovers the sub-bit rate, with the trigger and order derived from the same parametric model.

## Design principles (from LOCO-I/JPEG-LS)

1. **Structure**: prediction → context statistic → parametric conditional model
2. **Model cost**: few parameters per channel, with the model generating every context's distribution
3. **Sufficient statistics**: code selection from a decoder-computable causal statistic — no signaling
4. **Symbol codes only**: canonical Huffman + Golomb family — table lookups and shifts, no multiplications, suitable for FPGA and microcontroller targets

## Origin

GGDLPC was developed as the entropy coding stage for the [FRAPPE](https://ut-sysml.github.io/FRAPPE/) family of asymmetric neural codecs (v3+), replacing the off-the-shelf JPEG-LS codec used in earlier versions. It applies to any integer-valued signal with GGD-distributed prediction residuals.

## Package

`src/GGDLPC/` is the Python package (C coding engine JIT-built on first import; requires a C compiler). One call codes one integer tensor (values in [-N, N], N ≤ 65536 chosen at fit time; rank 1, 2, or 3) as one self-contained byte-padded bitstream, truncatable at any channel boundary. Macroregions, pre-quantization, multi-call file layouts, and metadata are caller compositions.

```python
import GGDLPC
channels, prov = GGDLPC.fit(loader_factory, N=31, rank=2)
codec = GGDLPC.Codec(GGDLPC.new_blob(31, 2, channels, prov))
data, ch_bits = codec.encode(z)      # (C, *S) integers -> bytes
z2 = codec.decode(data, z.shape)     # bit-exact
bits = GGDLPC.proxy_call_bits(z_noisy, codec.blob)  # differentiable rate
```

For a deployment that keeps the tables in flash, `GGDLPC.emit_c` emits them as C99 `static const` data from the same blob — the pooled table arrays (one entry per unique cell across all channels), per-channel cell arrays pointing into the pool, and a per-channel record so that boot-time setup is one `glpc_channel_config` call:

```python
src, info = GGDLPC.emit_c(codec.blob, enc=True, dec=False, prefix='spk')
open('spk_tables.c', 'w').write(src)   # info: pool entries and const bytes
```

```c
#include "spk_tables.c"                 /* static const spk_enc_pool[], spk_channels[] */
glpc_channel_t ch;                      /* the only per-channel RAM */
GLPC_CHANNEL_CONFIG_REC(&ch, &spk_channels[0]);
```

The codewords in the emitted tables are read back from the engine's own `glpc_tables_init`, so the C engine remains the single implementation of the canonical code; the self-test compiles the emitted source in every direction combination and checks it reproduces the package's streams bit-exactly.

Local development install: `./install_for_debugging.sh` (builds a wheel, installs into the `~/g` venv, runs `GGDLPC.selftest()`).

## Related

- [FRAPPEv5](https://github.com/danjacobellis/FRAPPEv5) — autoencoder training code that uses GGDLPC
- [compressors](https://github.com/danjacobellis/compressors) — codec library with FRAPPE v1–v3 inference implementations
