Metadata-Version: 2.4
Name: scan-google-sheet
Version: 0.3.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Dist: polars>=1.0
License-File: LICENSE
Summary: Read public Google Sheets into Polars DataFrames and LazyFrames — no auth required, powered by Rust
Keywords: polars,google-sheets,csv,dataframe,rust
Author-email: Attica-oss <g.mounac@gmail.com>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Issues, https://github.com/Attica-oss/scan_google_sheet/issues
Project-URL: Repository, https://github.com/Attica-oss/scan_google_sheet

# Scan Google Sheet

Read public Google Sheets into Polars DataFrames and LazyFrames — no auth, no service accounts, no API keys.

[![PyPI](https://img.shields.io/pypi/v/scan-google-sheet)](https://pypi.org/project/scan-google-sheet/)
[![Python](https://img.shields.io/pypi/pyversions/scan-google-sheet)](https://pypi.org/project/scan-google-sheet/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![CI](https://github.com/Attica-oss/scan_google_sheet/actions/workflows/ci.yml/badge.svg)](https://github.com/Attica-oss/scan_google_sheet/actions/workflows/ci.yml)

---

## Quick start

```bash
pip install scan-google-sheet
# or
uv add scan-google-sheet
```

```python
import polars as pl
from scan_google_sheet import read_google_sheet, scan_google_sheet

# Eager — returns a DataFrame immediately
df = read_google_sheet("Sheet1", sheet_id="1BxiMVs0XRA5nFMdKvBdBZjgm...")

# Lazy — returns a LazyFrame that participates in Polars query optimisation
df = (
    scan_google_sheet("Sheet1", sheet_id="1BxiMVs0XRA5nFMdKvBdBZjgm...")
    .filter(pl.col("year") == 2025)
    .select("vessel", "amount")
    .collect()
)
```

A full Google Sheets URL works in place of a bare ID:

```python
df = read_google_sheet(
    "Sheet1",
    url="https://docs.google.com/spreadsheets/d/1BxiMVs0.../edit#gid=0",
)
```

**Requirements:** Python ≥ 3.9, and the spreadsheet must be shared as **Anyone with the link can view**.

Prebuilt wheels currently cover:

| Platform | Interpreters |
|---|---|
| Linux x86-64 (manylinux 2.17+) | CPython 3.9–3.15, including free-threaded 3.14t/3.15t, plus PyPy 3.11 |
| macOS (arm64 and x86-64) | CPython 3.13 |
| Windows x86-64 | CPython 3.13 |

Anything outside that grid — notably macOS or Windows on a Python other than 3.13, and Linux on ARM — falls back to building from source, which requires a Rust toolchain.

---

## Rust core

Version `0.2.0` rewrote the fetch-and-parse core in Rust from scratch, compiled to a native extension via [PyO3](https://pyo3.rs) and [maturin](https://www.maturin.rs/). The public API is unchanged.

| | 0.1.x | 0.2.0+ |
|---|---|---|
| HTTP client | `httpx` (Python) | `reqwest` (Rust, blocking + rustls) |
| CSV parsing | `polars` via its Python API | `polars` called directly as a Rust crate |
| GIL | Released by `httpx`/`polars` around their own I/O and parse work | Released across the whole fetch + parse (`Python::detach`) |
| Distribution | Pure-Python wheel | Compiled extension module (`_core`) per platform |
| Public API | `read_google_sheet`, `scan_google_sheet` | Unchanged |
| Errors | Python exception hierarchy | Same hierarchy, raised from Rust |

### Upgrading from 0.1.x

Drop-in: same function signatures, same return types, same exception classes and attributes. Existing calling code needs no changes. Two optional keyword arguments are new — [`query`](#server-side-queries) and [`warn_if_filtered`](#detecting-an-active-filter) — and both default to off.

### Performance

Set expectations correctly: for a single call, don't expect a dramatic speedup. Fetching a public sheet is dominated by network round-trips and Google's server-side export of the sheet to CSV, neither of which any client-side language touches. Parsing was already Rust in `0.1.x` — `polars`' Python bindings call into the same crate this rewrite calls directly — so there was never a slow "Python parser" to replace.

What actually changes:

- **Server-side queries** (`query=`) cut the dominant cost. Google filters and projects before exporting, so less data is generated, transferred, and parsed. On a wide or long sheet this is the single biggest win available here.
- **Pooled connections.** A process-wide `reqwest::Client` (`src/core/fetch.rs`) keeps connections alive, where `0.1.x` re-negotiated TLS on every call. This is the lever that matters for an app fetching repeatedly.
- **One fewer conversion per call.** Bytes go straight from the HTTP response into the parser instead of round-tripping through a Python string. Negligible on small sheets, worth a few milliseconds on multi-megabyte ones.
- **GIL released across the whole call** (`Python::detach`). This is parity with `0.1.x` rather than a gain — CPython already releases the GIL around blocking socket I/O and polars releases it while parsing — but it means the native extension doesn't regress threaded callers.

If you benchmark, measure repeated calls in one process rather than a single cold fetch, since connection reuse is where the difference shows. Note that Google also caches exports server-side, so re-fetching the same sheet is faster for any client; vary the sheet to isolate the library.

---

## Server-side queries

Pass a [Google Visualization API Query Language](https://developers.google.com/chart/interactive/docs/querylanguage) string to have Google filter, project, and aggregate before the data is downloaded:

```python
# Columns are referenced by spreadsheet letter (A, B, C...), not by header name
df = read_google_sheet(
    "RawData",
    sheet_id="1BxiMVs0XRA5nFMdKvBdBZjgm...",
    query="select A, C, G, H where YEAR(K) = 2026",
)
```

Smaller transfer, less to parse. See [`docs/query-language.md`](docs/query-language.md) for a summary of the language, or Google's reference for the full spec.

---

## Detecting an active filter

Google's plain export endpoint (`/export?format=csv`) returns every row, but the `/gviz/tq` endpoint this library uses — needed for `sheet_name` and `query` support — silently **omits rows hidden by an active filter** on that tab. If a collaborator left a filter on, your results can be quietly incomplete.

`warn_if_filtered=True` checks for this and emits a `UserWarning`, at the cost of two extra HTTP requests:

```python
import warnings

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    df = read_google_sheet(
        "RawData", sheet_id="1BxiMVs0...", warn_if_filtered=True
    )
    if caught:
        print(caught[0].message)
        # sheet 'RawData' appears to have an active filter hiding 12 row(s)
        # (88 visible vs 100 in the full export) — query results may be incomplete
```

This is a best-effort heuristic, not a guarantee. It is most reliable when you pass `url` including `#gid=...`; with a bare `sheet_id` there is no gid to compare against, so the check falls back to the spreadsheet's first tab.

---

## API

### `read_google_sheet`

```python
def read_google_sheet(
    sheet_name: str,
    sheet_id: str | None = None,
    url: str | None = None,
    *,
    timeout: int = 10,
    parse_dates: bool = True,
    query: str | None = None,
    warn_if_filtered: bool = False,
) -> pl.DataFrame
```

Fetches the sheet and returns a collected `DataFrame`. Fetch and parse both run in Rust with the GIL released.

### `scan_google_sheet`

```python
def scan_google_sheet(
    sheet_name: str,
    sheet_id: str | None = None,
    url: str | None = None,
    *,
    timeout: int = 10,
    parse_dates: bool = True,
    batch_size: int = 1_000,
    query: str | None = None,
    warn_if_filtered: bool = False,
) -> pl.LazyFrame
```

Returns a `LazyFrame` registered through the Polars IO plugin API. Projection pushdown, predicate pushdown, `head()`, and streaming all work.

> **Note:** Google Sheets does not support partial HTTP reads, so the full sheet is always downloaded in one request. Pushdowns reduce processing cost, not network cost — use `query` to reduce network cost. This is also the one part of the API that stays in Python: `register_io_source` is a Polars Python-only hook with no Rust-side equivalent, so `scan_google_sheet` is a thin wrapper around the Rust-backed `read_google_sheet`.

### Shared parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `sheet_name` | `str` | — | Tab name as shown in Google Sheets |
| `sheet_id` | `str \| None` | `None` | Spreadsheet ID from the URL |
| `url` | `str \| None` | `None` | Full Google Sheets URL (ID extracted automatically) |
| `timeout` | `int` | `10` | HTTP timeout in seconds |
| `parse_dates` | `bool` | `True` | Attempt automatic date/datetime parsing |
| `query` | `str \| None` | `None` | Query Language string, applied server-side |
| `warn_if_filtered` | `bool` | `False` | Warn if the tab appears to have rows hidden by a filter (2 extra requests) |

Provide either `sheet_id` or `url`, not both.

### URL utilities

```python
from scan_google_sheet import extract_sheet_id, build_gviz_url, from_url

extract_sheet_id("https://docs.google.com/spreadsheets/d/ABC123/edit")
# "ABC123"

build_gviz_url("ABC123", "Sheet1")
# "https://docs.google.com/spreadsheets/d/ABC123/gviz/tq?tqx=out:csv&sheet=Sheet1"

build_gviz_url("ABC123", "Sheet1", "select A, C where B = 1")
# ".../gviz/tq?tqx=out:csv&sheet=Sheet1&tq=select%20A%2C%20C%20where%20B%20%3D%201"

from_url("https://docs.google.com/spreadsheets/d/ABC123/edit", "Sheet1")
```

---

## Error handling

Every exception inherits from `ReadSheetError`, so you can catch everything with one handler or branch on specific types:

```python
from scan_google_sheet import (
    read_google_sheet,
    ReadSheetError,
    SheetFetchError,
    SheetURLError,
    SheetParseError,
    NetworkError,
    ConfigurationError,
)

try:
    df = read_google_sheet("Sheet1", sheet_id="...")
except ReadSheetError as e:
    match e:  # structural pattern matching requires Python ≥ 3.10
        case SheetFetchError() if e.is_auth_error:
            print("Make the sheet public (Share → Anyone with the link)")
        case SheetFetchError() if e.is_not_found:
            print(f"Sheet not found — check the ID: {e.url}")
        case NetworkError():
            print(f"No connection: {e.cause}")
        case SheetURLError(raw=r):
            print(f"Could not parse URL: {r!r}")
        case SheetParseError():
            print(f"CSV parse failed: {e.cause}")
        case ConfigurationError():
            print(str(e))
```

```
ReadSheetError
├── SheetURLError       malformed URL or unextractable sheet ID  (.raw)
├── SheetFetchError     non-200 HTTP response                    (.url, .status_code)
│                                                                (.is_auth_error, .is_not_found)
├── SheetParseError     CSV or Polars parsing failure            (.column)
├── NetworkError        transport failure, no response received  (.url)
└── ConfigurationError  invalid argument combination
```

These are raised from Rust via PyO3, but the classes, hierarchy, messages, and attributes are the ones `0.1.x` raised.

---

## Making your sheet public

In Google Sheets: **Share → Change to Anyone with the link → Viewer → Done.**

The `gviz/tq?tqx=out:csv` endpoint requires the sheet to be publicly readable. This library only ever reads; it never writes.

---

## How it works

```
Google Sheets URL / ID
        │
        ▼
  build_gviz_url()      constructs the CSV export URL (+ tq query)   [Rust]
        │
        ▼
    fetch_raw()         reqwest GET, pooled client, GIL released     [Rust]
        │
        ▼
    parse_csv()         polars CSV → DataFrame                       [Rust]
        │
        ▼
  read_google_sheet()   PyO3 boundary → pl.DataFrame          [Rust → Python]
        │
        ▼
  scan_google_sheet()   register_io_source() lazy wrapper          [Python]
        │
        ▼
  LazyFrame / DataFrame
```

The crate also builds as a standalone Rust library with `cargo build` and no Python involved — see `read_public_sheet` in `src/lib.rs`, which the `python` feature's PyO3 bindings wrap.

---

## Development

```bash
git clone https://github.com/Attica-oss/scan_google_sheet
cd scan_google_sheet

# Rust build and tests (HTTP mocked via mockito, no network)
cargo build
cargo test

# Python extension, editable install into a venv
uv venv
source .venv/bin/activate
uv sync --group dev
maturin develop --release --features python

# Python test suite
uv run pytest
```

> `maturin develop` and `maturin build` without `--release` produce unoptimized debug builds — no inlining, no bounds-check elision. It won't show on small sheets, but always benchmark and publish `--release` builds.

---

## Changelog

### 0.3.0

- New `query` parameter: a Google Visualization API Query Language string applied server-side, so filtering, projection, and aggregation happen before download
- New `warn_if_filtered` parameter: warns when the target tab appears to have rows hidden by an active filter
- Prebuilt wheels published for the platforms listed under [Quick start](#quick-start); `0.2.x` shipped as a source distribution only

### 0.2.1

- Packaging fix

### 0.2.0

- Fetch/parse core rewritten in Rust (PyO3 + maturin); GIL released across network and parse work via `Python::detach`
- Process-wide pooled `reqwest::Client` (`src/core/fetch.rs`), reusing keep-alive connections instead of a fresh TLS handshake per call
- Public API and exception hierarchy unchanged from `0.1.x`, including structured attributes (`raw`, `url`, `status_code`, `is_auth_error`, `is_not_found`, `column`, `cause`)
- `scan_google_sheet`'s lazy IO-plugin wrapper stays in Python (`register_io_source` has no Rust-side equivalent)
- Rust unit and integration tests (`cargo test`, mocked HTTP via `mockito`) alongside the Python suite

### 0.1.x

- Pure-Python implementation (`httpx` + `polars`)
- `read_google_sheet` and `scan_google_sheet`
- Polars IO plugin for lazy evaluation
- Structured exception hierarchy
- Test suite using `pytest-httpx`

---

## License

[MIT](LICENSE) © 2026 Garry (Attica-oss)

