Metadata-Version: 2.4
Name: calicropyield
Version: 0.4.0
Summary: A multi-modal data downloader and processing library for California crop yield benchmarking
Author-email: Hamid Kamangir <hamid.kamangir@gmail.com>
License-Expression: BSD-3-Clause
Project-URL: Homepage, https://github.com/plant-ai-biophysics-lab/calicropyield-dev
Project-URL: Repository, https://github.com/plant-ai-biophysics-lab/calicropyield-dev
Project-URL: Bug Tracker, https://github.com/plant-ai-biophysics-lab/calicropyield-dev/issues
Keywords: california,crop yield,benchmark,remote sensing,agriculture
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.3
Requires-Dist: pyarrow>=14.0
Requires-Dist: xarray>=0.19
Requires-Dist: netCDF4>=1.6
Requires-Dist: rasterio>=1.2
Requires-Dist: geopandas>=0.10
Requires-Dist: shapely>=1.8
Requires-Dist: numpy>=1.21
Requires-Dist: huggingface_hub>=0.30
Requires-Dist: zarr<3.2,>=3.0.8
Provides-Extra: dev
Requires-Dist: setuptools-scm>=8.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest>=7.0; extra == "test"
Requires-Dist: pytest-cov>=4.0; extra == "test"
Dynamic: license-file

# calicropyield

Python client library for downloading and preparing county-level data used in the California Crop Yield Benchmark.

## What this package does

`calicropyield` provides a single client to retrieve benchmark-ready files from the public Hugging Face dataset [`hkaman/california-crop-yield-benchmark`](https://huggingface.co/datasets/hkaman/california-crop-yield-benchmark):

- ET (OpenET monthly GeoTIFF)
- Landsat monthly GeoTIFF
- CDL annual GeoTIFF
- Irrigation annual GeoTIFF (IrrMapper)
- DayMet climate (stored as Zarr, saved locally as NetCDF)
- Soil (stored as Zarr, saved locally as NetCDF)
- CDFA crop yield (statewide parquet, saved locally as one CSV per county)

The package supports optional geometry-based cropping and lightweight filtering for CDL/USDA outputs.

## Installation

### From source (this repository)

```bash
git clone https://github.com/plant-ai-biophysics-lab/calicropyield-dev.git
cd calicropyield-dev
pip install .
```

### Development install (recommended)

```bash
uv sync --extra dev --extra test
```

## Authentication setup

None required. The dataset is public on Hugging Face, so the client downloads anonymously.

If you are logged in to Hugging Face (`hf auth login`, or the `HF_TOKEN` environment variable), your token is picked up automatically, which gives you higher rate limits for large downloads.

## Quickstart

```python
from calicropyield import CalCropYieldClient

client = CalCropYieldClient(
    cache_dir="~/data/calicropyield",
    dataset_version="v2",   # optional; defaults to latest
    overwrite=False,
)

print(client.list_datasets())
print(client.list_years("landsat"))

# Download 2021 Landsat files for two counties
paths = client.download_landsat(["Yolo", "Fresno"], years=[2021])
print(f"Downloaded {len(paths)} files")
```

## Public API

Main exports:

- `CalCropYieldClient`
- `DatasetCatalog`
- `DataDownloader` (backward-compatible alias of `CalCropYieldClient`)

### Download methods

All download methods return `List[pathlib.Path]`:

- `download_et(county_names, years=None, geometry=None)`
- `download_landsat(county_names, years=None, geometry=None)`
- `download_climate(county_names, years=None, geometry=None, variables=None)`
- `download_soil(county_names, geometry=None, variables=None)`
- `download_cdl(county_names, years=None, geometry=None, crop_types=None)`
- `download_irrigation(county_names, years=None, geometry=None)`
- `download_yield(county_names, years=None, crop_types=None)`

### Introspection helpers

- `list_datasets()`
- `list_years(dataset="landsat")`
- `list_counties()`

## Dataset catalog and versioning

Dataset layout is defined in a versioned manifest file: `calicropyield/catalog.json`.

- Current latest dataset version: `v2`, served from the Hugging Face repo named by `hf_repo` (at `hf_revision`) in the catalog
- Years are declared per dataset (`list_years("<dataset>")`); 2012 is absent for every dataset
- Available dataset keys: `et`, `landsat`, `cdl`, `irrigation`, `climate`, `soil`, `yield`
- `v1` (the original Google Drive release) is still listed for reference but is no longer downloadable

You can inspect catalog metadata via:

```python
from calicropyield import DatasetCatalog

catalog = DatasetCatalog()  # latest
print(catalog.version)
print(catalog.available_datasets)
print(catalog.years_for("landsat"))
```

## Output structure

Downloaded files are stored under:

```text
<cache_dir>/counties/<County>/data/<dataset_type>/<year>/...
```

For non-yearly datasets (for example `soil`), year subfolders are omitted.

## Development workflow

This project uses:

- `uv` for environment and lockfile management
- `ruff` for linting
- `pytest` for tests
- `setuptools-scm` for dynamic package versioning from git tags

Common commands:

```bash
uv sync --extra dev --extra test
uv run ruff check calicropyield/
uv run pytest
uv lock
```

## Releases

Package version is derived from git tags (for example `v0.2.0`).

Typical release flow:

```bash
git tag vX.Y.Z
git push origin vX.Y.Z
```

On tag push, GitHub Actions builds wheel and sdist artifacts and validates them with `twine check`.

## Repository layout

- `calicropyield/`: installable Python package
- `dataset/`: internal dataset generation scripts/notebooks (these wrote the original GCS release, which has since been moved to Hugging Face)
- `.github/workflows/ci.yml`: lint/test/build CI

## Related project

End-user tutorial notebooks and benchmark usage examples live in the companion repository:

- https://github.com/plant-ai-biophysics-lab/california-crop-yield-benchmark

## License

BSD 3-Clause License. See `LICENSE`.
