quantize — ONED signal op

Data kinds: signalsignal

Call: import fullseye as fs; fs.ledger.quantize(x, bits=8, mode='round', dither=None, seed=0) (to call the implementation directly, import dsp; dsp.quantize(x, bits=8, mode='round', dither=None, seed=0); from the registry, ops1d.get("quantize"))

Usage

Scalar quantiser with the error model stated, plus optional dither.

*bits* sets the number of levels `L = 2**bits` over the signal's own

min..max range. *mode* is `"round" (mid-tread, unbiased) or "truncate"`

(floor, the convention PIL's posterize and most fixed-point casts use).

The two differ by more than a rounding convention. With step

`Delta = range/(L-1) both have error variance Delta**2/12`, but

truncation also carries a mean of `-Delta/2`, so its mean square error is

truncate: Delta2/12 + (Delta/2)2 = Delta**2/3

round: Delta**2/12

— a factor of 4. Anything that measures a level (not just displays it)

must round.

*dither* adds noise before quantising so the error stops being a function

of the signal: `"tpdf"` (triangular, the audio standard — two uniform draws

summed, so the error's variance no longer depends on the sample value) or

`"rpdf"` (one uniform draw). Dither raises the total error power but removes

the correlation that makes quantisation audible as distortion rather than as

hiss. `seed` fixes the draw so the op stays deterministic.

Applicability. (1) The range is taken from *this* signal, so two signals

quantised separately do not share a scale. (2) `bits=1` with no dither is a

comparator, not a quantiser — the error model does not apply. (3) The error

model assumes the signal moves by more than a step between samples; on a flat

stretch the error is a constant offset, not noise.

References (sample data, literature)

• Sample-data catalog (download URLs / licences) — 2-D uses skimage.data (BSD/public domain) plus synthetic images; 3-D lists download URLs for real data sources (Stanford, PDS, …).

• Operator provenance and references — the sources of the research/methods this op family came from.

• The canonical algorithm (author, year) and its uses are named in the family usage guide above.

Runnable examples (verified samples that actually call this op)

poc_cold_chain_excursionpy -3.11 examples/poc_cold_chain_excursion.py

Ops the type connects to (they accept signal as input)

create_funct_1d_array · create_funct_1d_pairs · smooth_funct_1d_gauss · smooth_funct_1d_mean · derivate_funct_1d · integrate_funct_1d · zero_crossings_funct_1d · local_min_max_funct_1d

Same category (signal)

lowpass · highpass · bandpass · envelope · rms · local_std · companding_mu_law · resample


*Provenance: dsp.py — ONED operator registry. This per-op note is generated by tools/opdocs.py md (do not hand-edit).*

© 2026 Kazufumi Furuse — Fullseye operator documentation. Licensed under Apache-2.0.