Metadata-Version: 2.5
Name: coarsegate
Version: 0.2.0
Summary: Measure what a coarsening removes, before interpreting a spatial-resolution comparison
Author: Joshua Wang
License-Expression: MIT
License-File: LICENSE
Keywords: change of support,cross-validation,downscaling,ecology,spatial resolution,statistical power
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Provides-Extra: dev
Requires-Dist: pyarrow>=14; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Provides-Extra: sim
Requires-Dist: scikit-learn>=1.3; extra == 'sim'
Requires-Dist: scipy>=1.10; extra == 'sim'
Description-Content-Type: text/markdown

# coarsegate

Measure what a coarsening removes, before you interpret a spatial-resolution
comparison.

## The problem

A common design asks whether finer predictors beat coarser ones: take a
predictor field, block-average it onto a coarse grid, fit the same model to
both arms, compare. The design looks airtight, because resolution is the only
thing that varies.

It can still fail to test its own hypothesis. Over a small extent, block
averaging barely changes the predictors. The two arms are fed nearly the same
numbers, the comparison cannot come out either way, and it returns a null that
means nothing. Nothing in the output distinguishes that case from a real
negative result.

The condition is measurable from the predictors alone, with no model fitted, so
it is available while the design can still be changed.

## Install

```bash
pip install coarsegate
```

## Use

```python
from coarsegate import precondition_report

report = precondition_report(
    df, features=["tmax", "ppt", "cmd"],
    cell_size=4000,          # same units as x and y
    x="x_m", y="y_m",        # projected coordinates, not degrees
    time="year",             # optional; a panel needs it, one slice does not
)
print(report)
```

```
coarsening 153 sites to 25 cells of 4000 units
  spatial variance removed (median over 14 features): f = 0.123
  fine-coarse correlation (median):            r = 0.998
  verdict: UNINFORMATIVE

f = 0.123 -> detection rate 0.08 [0.04, 0.15] at k = 5
  f is below the 0.4 floor: a null here says nothing about the hypothesis.

The two predictor sets are close to the same numbers, so a comparison between them
cannot come out either way. Report this as unanswerable at this design, not as a null.
```

`report.passes` is the verdict as a boolean. `report.per_feature` and
`report.correlations` carry the detail behind the two summary numbers, in case
one predictor is behaving differently from the rest.

The pieces are usable on their own:

```python
from coarsegate import variance_removed, correlation_retained, power_at, upscale

variance_removed(df, features, cell_size=25_000, x="x_m", y="y_m", time="year")
power_at(0.484)                    # what that fraction buys in detection
upscale(df, features, cell_size=25_000, x="x_m", y="y_m", time="year")
```

## What the floor is, and what it is not

`f` is the fraction of between-site predictor variance that block averaging
destroys. It is the within-cell share of the change-of-support decomposition,
and geostatistics has computed that quantity from the variogram since long
before this package existed. That part is not new.

What is not available analytically is the probability that a comparison at a
given `f` returns a verdict at all. That has to be measured, and the shipped
calibration is the measurement: 800 simulated panels across eight correlation
lengths, plus a block-count sweep, all under the same spatially-blocked
cross-validation and paired bootstrap the method is meant to be used with.

```python
from coarsegate import calibration_table
calibration_table("f")        # the correlation-length sweep, k = 5
calibration_table("blocks")   # the block-count sweep, at the two highest f
```

Read the floor in one direction only. Below `f` of about 0.4, a null carries no
information, so the diagnostic disqualifies the comparison. Above it, detection
depends on the number of validation blocks as much as on `f`, so clearing the
floor licenses interpretation rather than guaranteeing power. Detection rates
below the floor drift back up, and those are not detections in any useful sense.
They are what a bootstrap does with two nearly identical predictor sets, which
is exactly why the floor is a floor and not a point on a curve.

## Caveats worth reading before you cite a number

- The calibration is measured at five validation blocks and 300 sites. A design
  far from that should re-run the simulation rather than read the table.
- `x`, `y` and `cell_size` must be projected and share units. Averaging degrees
  gives cells whose ground area varies with latitude, which puts a spurious
  north-south gradient into `f`.
- Pass `time` for a panel. Without it the coarse arm loses the interannual
  signal too, so `f` counts variance the coarsening was never meant to touch and
  comes out too **high**, which is the dangerous direction: it makes an
  unanswerable design look answerable. Repeated coordinates with no `time`
  raise a warning.
- `f` is summarised as the median over features, so one nearly flat predictor
  cannot drag the verdict.

## Citation

The method, the calibration and the two worked extents:

> Wang, J. Measure what the coarsening removes: a precondition for interpreting
> spatial-resolution comparisons in ecological models. EcoEvoRxiv.
> https://doi.org/10.32942/X2ZH5C

That is a preprint. It has not been peer reviewed.

## License

MIT.
