Origin Story

Before there was a fit() method, there were three pages of pen-and-paper sketching. Nothing here was implemented as written — but almost everything zoneboost does today is a more rigorous version of a question asked on one of these pages.

The notes below are transcribed close to verbatim, in the order they were written, with a short note on how each idea eventually landed (or didn't) in the shipped model. See How It Works for what actually got built.

Page 1 — "Advanced Feature Engineering"

Handwritten notebook page titled Advanced Feature Engineering, sketching a 2D grid split into labeled regions (1a, 2, 3) along two variables x and y, with a note about standardizing data since some values may be large and others small. Below it, a table headed ORIGINAL lists an outcome column of raw numbers (76, 79, 80, 100, 30, 120, 200, 18, ..., 276, 120, 130) alongside ten variable columns x, y, z, a, b, c, d, e, f, h, with a red annotation distinguishing a valid outlier the business has confirmed from an invalid value that should be dropped. — The first sketch: splitting two variables into a labeled grid of regions, and a worked table of ten variables plus an outcome.

The page opens with "creating your own transformer to use in improving predictive power," then draws a grid over two variables (x and y) and labels the cells — a core region, an inner ring, an outer ring. A side note flags that variables will need to be standardized first, "since some might be huge values vs small values" — the raw problem that zone boundaries have to be scale-aware.

It generalizes almost immediately: "can be ideally 2 variables, x & y," but also "can continue filtering on the variable... i.e. (x&y) of a∈1 & b=1 & c=1 & d=1 etc." That's stacking conditions across more than two variables — the seed of what became pairwise (and eventually higher-order) interactions and adaptive interaction order rather than a single fixed 2D grid.

A starred note reframes the whole thing: "coming up with a new profile almost similar to decision trees" — for a simplified 10-variable example (x, y, z, a, b, c, d, e, f, h, outcome), with "outcome could be a number or classes." That regression-or-classification framing is exactly the ZoneBoostRegressor / ZoneBoostClassifier split, years before either existed. The red-ink aside on the outcome table — a business-confirmed outlier is a valid value, an invalid one should be dropped — is an early, informal version of the trust-the-data-boundary judgment that trim_fraction now makes automatic and consistent instead of a manual call per dataset.

Page 2 — Regions of Interest

Naming the pieces: weighting a variable by its relationship to the outcome, correcting for interaction effects, and re-checking importance as the model (and the data) evolves.

This page names the object directly — "regions of interest" — and starts asking the harder questions about them: "what is the weight of that [region/]variable with the outcome? But this could be affected by how it interacts with another variable." That's the moment a single-variable zone average stops being enough and an interaction term becomes necessary — you can't judge one variable's zone in isolation if its effect depends on another variable's zone too.

Two circled notes push further: variable importance needs to run before modeling, not just be read off the fitted model after the fact — and "the model will keep changing... using older data might affect its performance," so boxplots comparing outcome against each variable are proposed as a running sanity check. That's an early, manual stand-in for what empirical Bayes shrinkage now does automatically per zone, and what a time-based drift comparison does automatically across model versions.

The closing numbered list — "1. the ado-transformer, 2. correlation, 3. covariance, 4. original value," with min-max scaling and "direction included" — is the first draft of a feature-engineering pipeline: rescale, then compute per-region relationships, keeping the sign of the relationship rather than just its magnitude. "ado-transformer" never became a real class name, but the idea — a transformer built from region statistics rather than a fixed formula — is the through-line to every zone in zoneboost today.

Page 3 — Weighting Regions and Resampling

Handwritten notebook page describing an approach that works with samples of data, breaking it down into smaller samples and averaging the results. It proposes weighting sections of the data by density, e.g. giving a region between 0.8 and 1.0 on one axis and 90 to 100 on another a higher weight such as 50 percent. It defines four kinds of regions: inner core, outer core (split into important-to-one and important-to-neither variants), and outliers (split into important-to-both, outlier-to-both, and further outlier variants), with a note to try changing the importance weighting of each region as a scenario. A list of important steps to include follows: correlation (filtered per region), covariance, joint probability, conditional probability, interaction effects, point-biserial correlation, mean, median, mode, variance/standard deviation, and range. A final note says that if the data follows a date pattern it should be broken down sequentially, and if not, it should be sampled without replacement, since generating many such datasets and averaging their output imputes the final prediction. — The density-weighting and resampling idea: split the data into many small samples, weight each region by how much data actually falls in it, and average many such runs into one prediction.

The last page is the most complete draft of the mechanism. It opens: "works with samples of data, i.e. breaks down the data to smaller samples & then average it out" — followed by "the idea is to see what weight you give each section of the data," with a worked example (a region between 0.8–1.0 on one axis and 90–100 on another getting a 50% weight). That's density-aware shrinkage in everything but name: a zone with more supporting data should count for more than a sparse one.

It then defines four kinds of regions by name — inner core, outer core, outliers, with sub-cases for whether a point matters to one variable, both, or neither — which is a hand-drawn precursor to the model shrinking sparse or disagreeing zones toward a shared prior rather than trusting every region equally. The "important steps to include" list underneath — correlation, covariance, joint/conditional probability, interaction effects, point-biserial correlation, mean/median/mode, variance, range — is essentially a checklist of the descriptive statistics zoneboost's zones are built from, rather than anything resembling a decision-tree split criterion or a gradient-descent update.

The closing note is the earliest version of an evaluation discipline that stuck: "if data follows a date pattern, it's broken down to follow [a] sequential date pattern; if no date pattern, data is chosen without replacement" — and because that process "creates a lot of datasets," the output has to be imputed (averaged) back into one number. That distinction — sequential holdout for ordered data, resampling without replacement otherwise, many runs averaged into one estimate — is the direct ancestor of honest data splits, cross-fitted cell means, and bootstrap stability.

What Stuck, and What Didn't

Read together, three ideas from these pages survived essentially intact into the shipped model:

Regions, not splits or gradients. Every version of the sketch reaches for the same object — a labeled section of the data with its own average — rather than a tree's recursive split or a neural net's learned weight. That's still exactly what a zoneboost zone is.
Weight a region by how much data actually supports it. The page 3 density-weighting example is the informal version of empirical Bayes shrinkage.
A variable's effect on the outcome can depend on another variable. Flagged as a caveat on page 2, this became a first-class output — interactions are reported as their own term, never split back between the two variables that produced them.

A few things didn't survive, or survived only loosely:

The fixed 2D grid on page 1 gave way to adaptive, data-driven zone boundaries re-derived every boosting round, rather than one grid drawn once.
"The ado-transformer" as a named, standalone preprocessing step never shipped — its job got absorbed into the zone-fitting step itself rather than staying a separate transform.
The manual "re-check boxplots as the model changes" habit on page 2 is now closer to an explicit tool — time-based drift comparison — than something a person has to remember to do.

None of this was planned as a coherent design when it was written — it reads like someone working out, page by page, why a decision tree's opacity bothered them and what a transparent alternative would have to track instead. The rest of these docs describe what that alternative turned into.