Metadata-Version: 2.4
Name: rasteret
Version: 0.4.0
Summary: Index-first GeoTIFF access layer for ML and analysis, powered by queryable Parquet indexes.
Project-URL: Repository, https://github.com/terrafloww/rasteret
Project-URL: Documentation, https://terrafloww.github.io/rasteret
Project-URL: Issues, https://github.com/terrafloww/rasteret/issues
Project-URL: Changelog, https://terrafloww.github.io/rasteret/changelog/
Author-email: Sidharth Subramaniam <sid@terrafloww.com>
License: Apache-2.0
License-File: LICENSE
Keywords: cloud-optimized,cog,geospatial,geotiff,imagery,raster,satellite
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: GIS
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: <3.14,>=3.12
Requires-Dist: affine>=2.4.0
Requires-Dist: cachetools>=5.3.2
Requires-Dist: duckdb>=1.4.4
Requires-Dist: geoarrow-pandas>=0.1.0
Requires-Dist: geoarrow-pyarrow>=0.1.0
Requires-Dist: geopandas>=0.13
Requires-Dist: huggingface-hub>=0.23.0
Requires-Dist: imagecodecs>=2023.9.18
Requires-Dist: ipykernel>=6.31.0
Requires-Dist: nbconvert>=7.17.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: obstore>=0.8.0
Requires-Dist: pyarrow>=14.0.1
Requires-Dist: pyproj>=3.6.1
Requires-Dist: pystac-client>=0.7.5
Requires-Dist: rasterio<1.5.0,>=1.4.3
Requires-Dist: tqdm>=4.60
Requires-Dist: xarray>=2025.1.0
Requires-Dist: zstandard>=0.22.0
Provides-Extra: all
Requires-Dist: boto3>=1.34.0; extra == 'all'
Requires-Dist: datasets>=2.20.0; extra == 'all'
Requires-Dist: duckdb>=1.1.0; extra == 'all'
Requires-Dist: folium>=0.18.0; extra == 'all'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'all'
Requires-Dist: mkdocs-jupyter>=0.25; extra == 'all'
Requires-Dist: mkdocs-llmstxt>=0.2; extra == 'all'
Requires-Dist: mkdocs-material>=9.5; extra == 'all'
Requires-Dist: mkdocs-section-index>=0.3; extra == 'all'
Requires-Dist: mkdocs>=1.6; extra == 'all'
Requires-Dist: mkdocstrings[python]>=0.27; extra == 'all'
Requires-Dist: planetary-computer>=1.0.0; extra == 'all'
Requires-Dist: pre-commit>=3.7.0; extra == 'all'
Requires-Dist: pytest-asyncio>=0.23.2; extra == 'all'
Requires-Dist: pytest-cov>=7.0.0; extra == 'all'
Requires-Dist: pytest-timeout>=2.3.0; extra == 'all'
Requires-Dist: pytest>=8.4.2; extra == 'all'
Requires-Dist: requests>=2.31.0; extra == 'all'
Requires-Dist: ruff==0.8.6; extra == 'all'
Requires-Dist: scikit-learn>=1.5.0; extra == 'all'
Requires-Dist: sedonadb>=0.2.0; extra == 'all'
Requires-Dist: stac-geoparquet>=0.6.0; extra == 'all'
Requires-Dist: tifffile>=2023.9.18; extra == 'all'
Requires-Dist: xarray<2027,>=2024.1.0; extra == 'all'
Provides-Extra: aws
Requires-Dist: boto3>=1.34.0; extra == 'aws'
Provides-Extra: azure
Requires-Dist: planetary-computer>=1.0.0; extra == 'azure'
Requires-Dist: requests>=2.31.0; extra == 'azure'
Provides-Extra: dev
Requires-Dist: pre-commit>=3.7.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.2; extra == 'dev'
Requires-Dist: pytest-cov>=7.0.0; extra == 'dev'
Requires-Dist: pytest-timeout>=2.3.0; extra == 'dev'
Requires-Dist: pytest>=8.4.2; extra == 'dev'
Requires-Dist: ruff==0.8.6; extra == 'dev'
Requires-Dist: tifffile>=2023.9.18; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-jupyter>=0.25; extra == 'docs'
Requires-Dist: mkdocs-llmstxt>=0.2; extra == 'docs'
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocs-section-index>=0.3; extra == 'docs'
Requires-Dist: mkdocs>=1.6; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.27; extra == 'docs'
Provides-Extra: earthdata
Requires-Dist: requests>=2.31.0; extra == 'earthdata'
Provides-Extra: examples
Requires-Dist: datasets>=2.20.0; extra == 'examples'
Requires-Dist: duckdb>=1.1.0; extra == 'examples'
Requires-Dist: folium>=0.18.0; extra == 'examples'
Requires-Dist: huggingface-hub>=0.23.0; extra == 'examples'
Requires-Dist: scikit-learn>=1.5.0; extra == 'examples'
Requires-Dist: sedonadb>=0.2.0; extra == 'examples'
Requires-Dist: stac-geoparquet>=0.6.0; extra == 'examples'
Provides-Extra: xarray
Requires-Dist: xarray<2027,>=2024.1.0; extra == 'xarray'
Description-Content-Type: text/markdown

<h1 align="center">Rasteret</h1>

<p align="center">
  <strong>Build a collection once. Query it like a table. Read pixels 20x faster from cloud COGs.</strong>
</p>

<p align="center">
  <a href="https://terrafloww.github.io/rasteret"><img src="https://img.shields.io/badge/docs-terrafloww.github.io%2Frasteret-009DD1" alt="Documentation"></a>
  <a href="https://discord.gg/86NgTB3Xa"><img src="https://img.shields.io/badge/Discord-chat-5865F2?logo=discord&logoColor=white" alt="Discord"></a>
  <a href="https://pypi.org/project/rasteret/"><img src="https://img.shields.io/pypi/v/rasteret?color=blue" alt="PyPI"></a>
  <a href="https://pypi.org/project/rasteret/"><img src="https://img.shields.io/pypi/pyversions/rasteret" alt="Python"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache--2.0-blue" alt="License"></a>
</p>

Rasteret is an index-first reader for cloud-hosted tiled GeoTIFFs and COGs. It
builds a queryable Arrow/Parquet collection with scene metadata, asset URLs, CRS
sidecars, and parsed COG header metadata. Pixels stay in the original COGs.

After that, you can filter, join, and enrich the collection as a table, then
read only the pixels you need into NumPy, xarray, GeoPandas, TorchGeo, or Arrow
point-sample tables.

```text
STAC / Parquet / Arrow table -> Rasteret Collection -> NumPy / xarray / GeoPandas / TorchGeo
external labels / plots / points    filter/join/share          read pixels on demand
```

## Why Rasteret

Remote raster workflows often repeat the same setup work: STAC loops, COG header
parsing, tile byte-range planning, CRS transforms, retries, and output assembly.

Rasteret moves the expensive raster metadata discovery into a `Collection` build
step and reuses that metadata for later reads.

That helps when you:

- train or evaluate models over many remote COG scenes
- repeatedly sample the same imagery with different AOIs, points, labels, or splits
- avoid rediscovering raster header metadata in new notebooks, containers, or machines
- want one source collection to feed TorchGeo, xarray, NumPy, GeoPandas, and Arrow tools
- need DuckDB, Polars, PyArrow, or GeoPandas to work on metadata and external
  geometries before pixel reads

## Quick Example

```python
import rasteret

sentinel2_collection = rasteret.build(
    "earthsearch/sentinel-2-l2a",
    name="s2_bangalore",
    bbox=(77.5, 12.9, 77.7, 13.1),
    date_range=("2024-01-01", "2024-01-31"),
)

clear = sentinel2_collection.subset(cloud_cover_lt=50)

arr = clear.get_numpy(
    geometries=(77.55, 13.01, 77.58, 13.08),
    bands=["B04", "B08"],
)
```

The same collection can feed a downstream chip-based dataset:

```python
metadata = clear.to_table(
    columns=["id", "datetime", "geometry", "proj:epsg", "B04_metadata", "B03_metadata", "B02_metadata", "B08_metadata"],
)
```

## Bring Your Own Geometry And Metadata

Rasteret works well with the table tools you already use. External labels, farm
plots, asset locations, fire boundaries, or point samples can stay in
GeoPandas, DuckDB, Polars, or PyArrow until you need pixels.

```python
import duckdb
import geopandas as gpd
import rasteret
from shapely.geometry import box

plots = gpd.GeoDataFrame(
    {
        "plot_id": ["plot-a"],
        "crop": ["rice"],
    },
    geometry=[box(77.55, 13.01, 77.58, 13.08)],
    crs="OGC:CRS84",
)
plots_arrow = plots.to_arrow(geometry_encoding="WKB")

con = duckdb.connect()
con.sql("INSTALL spatial; LOAD spatial;")
con.register("sen2_rasteret", clear)
con.register("plots", plots_arrow)

# Bring your own geometries
plot_aois = con.sql("""
    SELECT
        plots.plot_id,
        plots.crop,
        plots.geometry AS plot_geometry
    FROM sen2_rasteret, plots
    WHERE sen2_rasteret."eo:cloud_cover" < 10
      AND ST_Intersects(
          ST_GeomFromWKB(sen2_rasteret.geometry),
          ST_GeomFromWKB(plots.geometry)
      )
""")

plot_patches = clear.get_gdf(
    geometries=plot_aois,
    geometry_column="plot_geometry",
    geometry_crs=4326,
    bands=["B04", "B08"],
)
```

The same pattern works with Polars or PyArrow for split/label columns, and with
`sample_points(...)` when your external data is point-based. `get_gdf(...)` and
`sample_points(...)` keep business columns such as `plot_id` in their outputs.

## What You Can Do

| Task | Rasteret surface |
| --- | --- |
| Build from a registered dataset | `rasteret.build("catalog/id", ...)` |
| Build from your own Parquet, GeoParquet, DuckDB, Polars, or Arrow record table | `rasteret.build_from_table(...)` |
| Reopen a saved or prebuilt Collection | `rasteret.load(path_or_dataset_id)` |
| Re-wrap a read-ready Arrow object | `rasteret.as_collection(...)` |
| Get numpy arrays | `Collection.get_numpy(...)` |
| Get xarray dataset | `Collection.get_xarray(...)` |
| Get GeoPandas rows with pixel arrays | `Collection.get_gdf(...)` |
| Sample pixels at points | `Collection.sample_points(...)` |
| Build a downstream TorchGeo dataset | `Collection.to_table(...)` + `Collection.read_window(...)` |

## Dataset Catalog

Rasteret ships with dataset IDs so you do not have to remember STAC endpoints,
band maps, license metadata, or cloud access settings. Most catalog entries are
recipes for `rasteret.build(...)`: Rasteret searches the source catalog, parses
the COG metadata once, and writes a reusable local Collection.

Only one built-in ID is already a read-ready Rasteret Collection:
`aef/v1-annual`. Use `rasteret.load("aef/v1-annual")` for AlphaEarth Foundation
Embeddings. The built-in alias loads Rasteret's maintained Source Cooperative
Collection. You do not need to call `build()` for this dataset.

| ID | Dataset | Coverage | Auth | Use |
| --- | --- | --- | --- | --- |
| `aef/v1-annual` | AlphaEarth Foundation Embeddings (Annual) | global | none | `rasteret.load(...)` |
| `earthsearch/sentinel-2-l2a` | Sentinel-2 Level-2A | global | none | `rasteret.build(...)` |
| `earthsearch/landsat-c2-l2` | Landsat Collection 2 Level-2 | global | required | `rasteret.build(...)` |
| `earthsearch/naip` | NAIP | north-america | required | `rasteret.build(...)` |
| `earthsearch/cop-dem-glo-30` | Copernicus DEM 30m | global | none | `rasteret.build(...)` |
| `earthsearch/cop-dem-glo-90` | Copernicus DEM 90m | global | none | `rasteret.build(...)` |
| `pc/sentinel-2-l2a` | Sentinel-2 Level-2A (Planetary Computer) | global | required | `rasteret.build(...)` |
| `pc/io-lulc-annual-v02` | ESRI 10m Land Use/Land Cover | global | required | `rasteret.build(...)` |
| `pc/alos-dem` | ALOS World 3D 30m DEM | global | required | `rasteret.build(...)` |
| `pc/nasadem` | NASADEM | global | required | `rasteret.build(...)` |
| `pc/esa-worldcover` | ESA WorldCover | global | required | `rasteret.build(...)` |
| `pc/usda-cdl` | USDA Cropland Data Layer | conus | required | `rasteret.build(...)` |

You can browse the same list from the CLI:

```bash
rasteret datasets list
rasteret datasets info aef/v1-annual
```

To make your own dataset ID for a reusable local collection or Parquet record
table, see [Register A Local Collection into Dataset Catalog](https://terrafloww.github.io/rasteret/how-to/dataset-catalog/#register-a-local-collection-into-dataset-catalog).

## Performance

Rasteret is 10x to 20x faster than rasterio/GDAL

| Scenario | TorchGeo/rasterio | Rasteret | Speedup |
| --- | ---: | ---: | ---: |
| Single AOI, 15 scenes | 9.08 s | 1.14 s | 8.0x |
| Multi-AOI, 30 scenes | 42.05 s | 2.25 s | 18.7x |
| Cross-CRS, 12 scenes | 12.47 s | 0.59 s | 21.3x |

![Processing time comparison](./assets/benchmark_results.png)

Rasteret also compares well against time-series workflows that use Google Earth
Engine or thread-pooled rasterio for the measured setup:

| Library | First run (cold) | Subsequent runs (hot) |
| --- | ---: | ---: |
| Rasterio + ThreadPool | 32 s | 24 s |
| Google Earth Engine | 10-30 s | 3-5 s |
| Rasteret | 3 s | 3 s |

![Single request performance](./assets/single_timeseries_request.png)

See the [Benchmarks guide](https://terrafloww.github.io/rasteret/explanation/benchmark/)
for methodology, environment details, and additional Hugging Face `datasets`
comparisons.

## Install

```bash
uv pip install rasteret
```

Optional integrations:

```bash
uv pip install "rasteret[aws]"
uv pip install "rasteret[azure]"
uv pip install "rasteret[all]"  # all optional integrations for exploration
```

Rasteret requires Python 3.12 or later.

## Learn More

- [Getting Started](https://terrafloww.github.io/rasteret/getting-started/)
- [Build from Parquet and Arrow Tables](https://terrafloww.github.io/rasteret/how-to/build-from-tables/)
- [Bring Your Own AOIs, Points, And Metadata](https://terrafloww.github.io/rasteret/how-to/enriched-collection-workflows/)
- [TorchGeo Integration](https://terrafloww.github.io/rasteret/how-to/torchgeo-integration/)
- [Benchmarks](https://terrafloww.github.io/rasteret/explanation/benchmark/)
- [API Reference](https://terrafloww.github.io/rasteret/reference/)

## License

Code: [Apache-2.0](LICENSE)
