Metadata-Version: 2.4
Name: pygndc
Version: 1.0.14
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Dist: numpy>=1.21.0
Requires-Dist: zstandard>=0.21.0
Requires-Dist: rasterio>=1.3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: cryptography>=41.0.0
Requires-Dist: matplotlib>=3.5.0 ; extra == 'all'
Requires-Dist: pillow>=9.0.0 ; extra == 'all'
Requires-Dist: xarray>=2022.0 ; extra == 'all'
Requires-Dist: geopandas>=0.12.0 ; extra == 'all'
Requires-Dist: shapely>=2.0.0 ; extra == 'all'
Requires-Dist: scipy>=1.9.0 ; extra == 'all'
Requires-Dist: pytest>=7.0.0 ; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0 ; extra == 'dev'
Requires-Dist: black>=23.0.0 ; extra == 'dev'
Requires-Dist: ruff>=0.1.0 ; extra == 'dev'
Requires-Dist: mypy>=1.0.0 ; extra == 'dev'
Requires-Dist: twine>=4.0.0 ; extra == 'dev'
Requires-Dist: geopandas>=0.12.0 ; extra == 'geo'
Requires-Dist: shapely>=2.0.0 ; extra == 'geo'
Requires-Dist: matplotlib>=3.5.0 ; extra == 'viewer'
Requires-Dist: pillow>=9.0.0 ; extra == 'viewer'
Requires-Dist: xarray>=2022.0 ; extra == 'xarray'
Provides-Extra: all
Provides-Extra: dev
Provides-Extra: geo
Provides-Extra: viewer
Provides-Extra: xarray
License-File: LICENSE
Summary: Geographic Neural Data Cube - Read and analyze .gndc compressed geospatial time-series data
Keywords: geospatial,satellite,remote-sensing,neural-networks,compression
Author-email: Jianbo Qi <jianboqi@126.com>
Requires-Python: >=3.10, <3.14
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://geondc.org
Project-URL: Issues, https://github.com/jianboqi/pygndc/issues
Project-URL: Repository, https://github.com/jianboqi/pygndc

# pygndc

**Geographic Neural Data Cube** — a Python SDK for reading and analyzing `.gndc` compressed geospatial time-series data.

<img width="286" height="320" alt="GeoNDC" src="https://raw.githubusercontent.com/jianboqi/pygndc/main/assets/logo.png" />

## What is GeoNDC?

GeoNDC is a **continuous-time, AI-ready representation** of Earth observation archives. Unlike traditional Analysis-Ready Data (cloud-corrected raster files) or geospatial foundation model embeddings (abstract feature vectors), GeoNDC preserves the original physical observables — surface reflectance, vegetation indices, biophysical variables — while enabling millisecond-level random-access queries at any *(x, y, t)* coordinate.

Each archive (MODIS, Sentinel-2, Landsat, HiGLASS, …) is encoded into a single self-contained `.gndc` file (typically 0.5–2 GB) that runs on a laptop, a server, or directly in a browser via WebGPU. Data providers train the model once and publish the file; users download it and run inference locally — the compressed form *is* the analysis-ready form. No hosted runtime, no API quota, no vendor lock-in.

## Key Capabilities

- **Continuous-time reconstruction** — query data at any moment, not just original observation times
- **Millisecond random access** — point time series in ~7 ms, full-frame reconstruction in ~2 s on a consumer GPU
- **Analytic gradients** — compute spatial/temporal derivatives directly from the neural network
- **Compact storage** — typically ~100:1 versus Int16 raster baselines, up to ~400:1 versus raw float archives
- **Portable Native encoder and decoder** — the default CPU/WGPU path needs no PyTorch, Numba, CUDA toolkit, or runtime JIT; WGPU targets Vulkan, DX12, and Metal, with an optional PyTorch/tiny-cuda-nn integration
- **Implicit gap-filling** — cloud-occluded surfaces are reconstructed from the learned spatiotemporal field
- **Multi-sensor support** — Sentinel-2, Landsat, MODIS, HiGLASS, and more

## Online Viewer & Sample Data

- **Web Viewer**: Browse `.gndc` files directly in the browser via WebGPU at [geondc.org/viewer](https://www.geondc.org/viewer/) — no installation required, GPU-accelerated, runs entirely client-side.
- **Sample Data**: Download `.gndc` datasets from [Hugging Face](https://huggingface.co/datasets/geondc/geondc-data).

## Querying a Collection

Many independently encoded geographic products can be exposed as one logical cube
without repacking their neural payloads:

```python
from pygndc import GNDCCollection

collection = GNDCCollection(mode="native_wgpu")
collection.add(r"Y:\HiGLASS\*.gndc")
collection.save("higlass_china.gndc-collection.json")

collection = GNDCCollection.load("higlass_china.gndc-collection.json")
values = collection.query(
    points=[(116.4, 39.9), (118.2, 36.5)],
    time="2023-07-15",
    bands=[0, 1],
)
```

The collection index loads without opening its members. Queries are grouped by
member and decoded through the existing `GNDCDataset` runtime with a bounded LRU
cache. Spatiotemporal windows support in-memory results, `stream=True`, or direct
output to a NumPy `.npy` file.

## Documentation

- **[TUTORIAL.md](TUTORIAL.md)** — Installation, quick-start, CLI commands, end-to-end usage examples.
- **[API_Reference.md](API_Reference.md)** — Full Python API for `pygndc.open()`, `GNDCDataset`, `GNDCReader`, analysis functions.

## Installation Extras

Prebuilt packages support Python 3.10–3.13 on Windows x86-64 and Linux x86-64
(glibc 2.28 or newer). They include the encoder, CPU/GPU decoder, GeoTIFF I/O,
and Python API. GPU acceleration requires a compatible graphics driver;
CPU decoding is available without a GPU. The desktop viewer is optional:

```bash
pip install pygndc[viewer]
```

The desktop viewer caches up to nine chunk readers by default, enough for a 3×3
HiGLASS product, and keeps up to nine model slots resident on WGPU. Set
`PYGNDC_VIEWER_MAX_CACHED_CHUNKS` to tune the host-side LRU and
`PYGNDC_VIEWER_WGPU_SLOTS` to tune device-resident model slots, or change the
reader limit at runtime from **Tools > Chunk Cache**.

## Encoding large datasets

GPU training reuses data already loaded into device memory across training
rounds. Set `train.train_dtype: float16` to store training targets in half
precision, or `float32` to retain full-precision targets. Model calculations use
float32 in either case. The default training-data budget is 4096 MB; set
`PYGNDC_NATIVE_PERSISTENT_MAX_MB` to adjust it for your GPU.

For tiled inputs, the encoder can read the next tile while training the current
one. Set `tiled.prefetch_chunks: false` when available RAM is limited.

## Reading large archives

Masks and frame-indexed residual corrections are loaded for requested dates.
Repeated queries to the same date reuse recently loaded data. Existing `.gndc`
files can use these improvements without being re-encoded. Older residual
formats without a date index continue to use their existing decoding behavior.

GPU loading transfers supported quantized weights directly and restores them
on the device. This reduces work during model loading; it does not reduce the
memory required by the loaded float32 model. CPU decoding remains available
when GPU loading is unavailable.

For applications that process many chunks in sequence, `GNDCDataset` provides
optional prefetch budgets. See [the API reference](API_Reference.md#3-gndcdataset)
for the parameters and memory limits. Ordinary `read()` calls do not enable
background prefetch automatically.

## Changes in 1.0.14

Encoding and decoding APIs now include type declarations for editor completion,
parameter hints, and static type checking. Implementation source code is not
included in the prebuilt packages.

- Faster encoder startup by reusing hardware identification within a process.
- More efficient reuse of GPU training data across epochs.
- Lower memory use for queries with frame-indexed residual corrections.
- Faster loading of supported quantized models and repeated queries to a date.
- Clear errors when a file declares residual corrections that cannot be read.

Existing file formats remain supported. See [release notes](CHANGELOG.md).

## License

GeoNDC is proprietary binary software. Reading and decoding do not require an
encoder license. Creating `.gndc` files requires a valid encoder license. Contact
`jianboqi@126.com` for evaluation or commercial use. See [LICENSE](LICENSE) for
the complete terms.

The GeoNDC file-format specification is published separately to support
interoperability. Previously released copies remain governed by the license
that accompanied those copies.

## Citation

```bibtex
@misc{qi2026geondcqueryableneuraldata,
  title={GeoNDC: A Queryable Neural Data Cube for Planetary-Scale Earth Observation},
  author={Jianbo Qi and Mengyao Li and Baogui Jiang and Yidan Chen and Qiao Wang},
  year={2026},
  eprint={2603.25037},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2603.25037},
}
```

## Contact

- Author: Jianbo Qi
- Email: jianboqi@126.com
- Issues: [GitHub Issues](https://github.com/jianboqi/pygndc/issues)

