Metadata-Version: 2.5
Name: bigquery-data-science
Version: 0.10.0
Summary: BigQuery ML, BigQuery AI, and 4-Lens Agent Skills: 104 live-verified SQL/Python workflows, zero-dependency CLI catalog & skill installer, and stdio MCP server.
Project-URL: Homepage, https://github.com/statmike/bigquery-data-science
Project-URL: Repository, https://github.com/statmike/bigquery-data-science
Project-URL: Documentation, https://github.com/statmike/bigquery-data-science#readme
Project-URL: Changelog, https://github.com/statmike/bigquery-data-science/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/statmike/bigquery-data-science/issues
Author: Mike Henderson
License: Apache-2.0
License-File: LICENSE
Keywords: agent-skills,bigframes,bigquery,bigquery-ai,bigquery-ml,gemini-enterprise,generative-ai,google-cloud,machine-learning,mcp,sql,vertex-ai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.11
Provides-Extra: all
Requires-Dist: bigframes>=1.25; extra == 'all'
Requires-Dist: bigquery-magics>=0.4; extra == 'all'
Requires-Dist: db-dtypes>=1.2; extra == 'all'
Requires-Dist: differences>=0.2; extra == 'all'
Requires-Dist: google-cloud-aiplatform>=1.60; extra == 'all'
Requires-Dist: google-cloud-bigquery-datatransfer>=3.15; extra == 'all'
Requires-Dist: google-cloud-bigquery-reservation>=1.16; extra == 'all'
Requires-Dist: google-cloud-bigquery>=3.25; extra == 'all'
Requires-Dist: google-cloud-dataform>=0.5; extra == 'all'
Requires-Dist: google-cloud-documentai>=2.29; extra == 'all'
Requires-Dist: google-cloud-orchestration-airflow>=1.13; extra == 'all'
Requires-Dist: google-cloud-pipeline-components>=2.14; extra == 'all'
Requires-Dist: google-cloud-storage>=2.17; extra == 'all'
Requires-Dist: google-cloud-workflows>=1.14; extra == 'all'
Requires-Dist: graphviz>=0.20; extra == 'all'
Requires-Dist: ipykernel>=6.29; extra == 'all'
Requires-Dist: jinja2>=3.1.6; extra == 'all'
Requires-Dist: kfp>=2.7; extra == 'all'
Requires-Dist: lifelines>=0.29; extra == 'all'
Requires-Dist: matplotlib>=3.8; extra == 'all'
Requires-Dist: numpy>=1.26; extra == 'all'
Requires-Dist: onnx>=1.16; extra == 'all'
Requires-Dist: onnxruntime>=1.18; extra == 'all'
Requires-Dist: pillow>=10.0; extra == 'all'
Requires-Dist: pydantic>=2.7; extra == 'all'
Requires-Dist: scikit-learn>=1.5; extra == 'all'
Requires-Dist: scipy>=1.11; extra == 'all'
Requires-Dist: skl2onnx>=1.17; extra == 'all'
Requires-Dist: tqdm>=4.66; extra == 'all'
Requires-Dist: weasyprint>=68.1; extra == 'all'
Requires-Dist: xgboost>=2.0; extra == 'all'
Provides-Extra: notebooks
Requires-Dist: bigframes>=1.25; extra == 'notebooks'
Requires-Dist: bigquery-magics>=0.4; extra == 'notebooks'
Requires-Dist: db-dtypes>=1.2; extra == 'notebooks'
Requires-Dist: google-cloud-bigquery>=3.25; extra == 'notebooks'
Requires-Dist: google-cloud-documentai>=2.29; extra == 'notebooks'
Requires-Dist: google-cloud-storage>=2.17; extra == 'notebooks'
Requires-Dist: ipykernel>=6.29; extra == 'notebooks'
Requires-Dist: jinja2>=3.1.6; extra == 'notebooks'
Requires-Dist: matplotlib>=3.8; extra == 'notebooks'
Requires-Dist: pillow>=10.0; extra == 'notebooks'
Requires-Dist: pydantic>=2.7; extra == 'notebooks'
Requires-Dist: tqdm>=4.66; extra == 'notebooks'
Requires-Dist: weasyprint>=68.1; extra == 'notebooks'
Provides-Extra: pipelines
Requires-Dist: bigframes>=1.25; extra == 'pipelines'
Requires-Dist: bigquery-magics>=0.4; extra == 'pipelines'
Requires-Dist: db-dtypes>=1.2; extra == 'pipelines'
Requires-Dist: google-cloud-aiplatform>=1.60; extra == 'pipelines'
Requires-Dist: google-cloud-bigquery-datatransfer>=3.15; extra == 'pipelines'
Requires-Dist: google-cloud-bigquery-reservation>=1.16; extra == 'pipelines'
Requires-Dist: google-cloud-bigquery>=3.25; extra == 'pipelines'
Requires-Dist: google-cloud-dataform>=0.5; extra == 'pipelines'
Requires-Dist: google-cloud-documentai>=2.29; extra == 'pipelines'
Requires-Dist: google-cloud-orchestration-airflow>=1.13; extra == 'pipelines'
Requires-Dist: google-cloud-pipeline-components>=2.14; extra == 'pipelines'
Requires-Dist: google-cloud-storage>=2.17; extra == 'pipelines'
Requires-Dist: google-cloud-workflows>=1.14; extra == 'pipelines'
Requires-Dist: ipykernel>=6.29; extra == 'pipelines'
Requires-Dist: jinja2>=3.1.6; extra == 'pipelines'
Requires-Dist: kfp>=2.7; extra == 'pipelines'
Requires-Dist: matplotlib>=3.8; extra == 'pipelines'
Requires-Dist: pillow>=10.0; extra == 'pipelines'
Requires-Dist: pydantic>=2.7; extra == 'pipelines'
Requires-Dist: tqdm>=4.66; extra == 'pipelines'
Requires-Dist: weasyprint>=68.1; extra == 'pipelines'
Provides-Extra: workflows
Requires-Dist: bigframes>=1.25; extra == 'workflows'
Requires-Dist: bigquery-magics>=0.4; extra == 'workflows'
Requires-Dist: db-dtypes>=1.2; extra == 'workflows'
Requires-Dist: differences>=0.2; extra == 'workflows'
Requires-Dist: google-cloud-bigquery>=3.25; extra == 'workflows'
Requires-Dist: google-cloud-documentai>=2.29; extra == 'workflows'
Requires-Dist: google-cloud-storage>=2.17; extra == 'workflows'
Requires-Dist: graphviz>=0.20; extra == 'workflows'
Requires-Dist: ipykernel>=6.29; extra == 'workflows'
Requires-Dist: jinja2>=3.1.6; extra == 'workflows'
Requires-Dist: lifelines>=0.29; extra == 'workflows'
Requires-Dist: matplotlib>=3.8; extra == 'workflows'
Requires-Dist: numpy>=1.26; extra == 'workflows'
Requires-Dist: onnx>=1.16; extra == 'workflows'
Requires-Dist: onnxruntime>=1.18; extra == 'workflows'
Requires-Dist: pillow>=10.0; extra == 'workflows'
Requires-Dist: pydantic>=2.7; extra == 'workflows'
Requires-Dist: scikit-learn>=1.5; extra == 'workflows'
Requires-Dist: scipy>=1.11; extra == 'workflows'
Requires-Dist: skl2onnx>=1.17; extra == 'workflows'
Requires-Dist: tqdm>=4.66; extra == 'workflows'
Requires-Dist: weasyprint>=68.1; extra == 'workflows'
Requires-Dist: xgboost>=2.0; extra == 'workflows'
Description-Content-Type: text/markdown

# bigquery-data-science

**Production Agent Skills, zero-dependency CLI, stdio MCP server & Google ADK tools for BigQuery ML, BigQuery AI, and BigQuery MLOps — distilled directly from 104 live-executed notebooks and 87 SQL scripts in Google Cloud.**

[![PyPI](https://img.shields.io/pypi/v/bigquery-data-science?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/bigquery-data-science/)
[![BigQuery ML Hub](https://img.shields.io/badge/BigQuery%20ML-61%20Notebooks%20%26%2058%20SQL-4285F4?style=for-the-badge&logo=google-cloud&logoColor=white)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md)
[![BigQuery AI Hub](https://img.shields.io/badge/BigQuery%20AI-43%20Notebooks%20%26%2029%20SQL-0F9D58?style=for-the-badge&logo=google-cloud&logoColor=white)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md)
[![BigQuery MLOps Skill](https://img.shields.io/badge/BigQuery%20MLOps-8%20Pipelines%20%C2%B7%20Registry%20%C2%B7%20Drift-F9AB00?style=for-the-badge&logo=google-cloud&logoColor=white)](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md)
[![Multi-Skill Playbooks](https://img.shields.io/badge/Multi--Skill-6%20Playbooks%20%C2%B7%207--Stage%20Map-8E24AA?style=for-the-badge)](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/cross-skill-playbooks.md)
[![Namespace Matrix](https://img.shields.io/badge/ML.*%20vs%20AI.*-Namespace%20Matrix-EA4335?style=for-the-badge)](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/namespace-matrix.md)
[![License](https://img.shields.io/badge/License-Apache%202.0-4E5D6C?style=for-the-badge)](https://github.com/statmike/bigquery-data-science/blob/main/LICENSE)

`104 live-verified notebooks` · `87 SQL scripts` · `4 Composable Agent Skills` · `20 Reference & Playbook Guides` · `137 <25 KB syntax cards` · `zero-dependency CLI, MCP & ADK`

![bigquery-data-science architecture](https://raw.githubusercontent.com/statmike/bigquery-data-science/main/docs/assets/readme_architecture.png)

---

## What you just found

This repository pairs **three tightly connected layers** that stay synchronized through automated tooling (`bigquery-data-science sweep`):

1. **Ground-Truth Notebooks, SQL & Reference Hubs ([`bigquery-ml/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) & [`bigquery-ai/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md)):** Every model type (`23`), `ML.*` function (`12` families), `AI.*` / Cloud AI function (`29`), composed workflow (`32`), and MLOps orchestration pipeline (`8`) is validated end-to-end in a live-executed Jupyter notebook (`.ipynb` with committed outputs and one-click Colab / Vertex AI Workbench / BigQuery Studio headers) paired with a pure SQL companion script (`87` `.sql` files) and `13` deep domain reference guides.
2. **4 Composable Agent Skills ([`skills/`](https://github.com/statmike/bigquery-data-science/blob/main/skills/PLANS.md)):** An umbrella router and multi-skill composer ([`bigquery-data-science`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/SKILL.md)) routes across three specialist skills — trained models & causal inference ([`bigquery-ml`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ml/SKILL.md)), zero-training foundation models & vector/hybrid search ([`bigquery-ai`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ai/SKILL.md)), and production orchestration, model registry, hybrid serving, drift monitoring & point-in-time feature stores ([`bigquery-mlops`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md)) — and explicitly recommends multi-skill combinations via [`cross-skill-playbooks.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/cross-skill-playbooks.md).
3. **4-Lens Progressive Disclosure, CLI, Stdio MCP & Google ADK ([`src/bigquery_data_science/`](https://github.com/statmike/bigquery-data-science/blob/main/src/bigquery_data_science/__init__.py)):** Instead of dumping 24 MB of raw notebooks or 200+ KB monolithic reference files into an AI coding agent's context window, `bigquery-data-science` compiles the repository into **4 progressive lenses** (`SKILL.md` $\to$ `20` reference guides $\to$ `137` `<25 KB` syntax cards $\to$ `104` clean narratives) and exposes them through a zero-dependency CLI (`uvx bigquery-data-science`), stdio MCP server (`bigquery-data-science mcp`), and Google ADK toolset (`get_bigquery_tools()`).

---

## Three ways to start

### Journey 1 — Equip your AI coding agent (`uvx`, `uv`, `pip`, `pipx`, `npx skills`, or MCP)

The core `bigquery-data-science` package has **zero third-party runtime dependencies** (`< 1.5 MB` wheel) and installs the 4 skills (`bigquery-data-science`, `bigquery-ml`, `bigquery-ai`, `bigquery-mlops`) into `.agents/skills/` and `.claude/skills/`:

```bash
# Option A — Zero-install ephemeral CLI via uvx (recommended for instant setup):
uvx bigquery-data-science install                     # installs into ./.agents/skills & ./.claude/skills
uvx bigquery-data-science install --mcp               # also registers stdio MCP server in ./.mcp.json
uvx bigquery-data-science install --global            # installs into ~/.agents/skills & ~/.claude/skills
uvx bigquery-data-science install --target agents     # only .agents/skills (Antigravity, Cursor, Codex, Windsurf)
uvx bigquery-data-science install --target claude     # only .claude/skills (Claude Code)

# Option B — Persistent CLI install via uv tool, pipx, or pip:
uv tool install bigquery-data-science && bigquery-data-science install
pipx install bigquery-data-science && bigquery-data-science install
pip install bigquery-data-science && bigquery-data-science install

# Option C — Add to an existing Python project (uv or pip):
uv add bigquery-data-science && uv run bigquery-data-science install
pip install bigquery-data-science && python3 -m bigquery_data_science.cli install

# Option D — skills.sh CLI (installs all 4 skills or an individual skill from GitHub):
npx skills add statmike/bigquery-data-science
npx skills add statmike/bigquery-data-science --skill bigquery-ml
npx skills add statmike/bigquery-data-science --skill bigquery-mlops

# Option E — Claude Code plugin marketplace & stdio MCP server:
# In Claude Code: /plugin marketplace add statmike/bigquery-data-science
#                 /plugin install bigquery-data-science
claude mcp add bigquery-data-science -- uvx bigquery-data-science mcp
# Or with a local pip/uv install:
claude mcp add bigquery-data-science -- python3 -m bigquery_data_science.cli mcp
```

### Journey 2 — Triage tasks & look up exact BigQuery syntax from the terminal

Run any catalog command directly with `uvx bigquery-data-science ...` (zero install) or `bigquery-data-science ...` (after `uv tool install` or `pip install`):

```bash
# 1. List bundled skills and lens counts:
uvx bigquery-data-science list

# 2. Triage any BigQuery ML, BigQuery AI, or BigQuery MLOps task (10 comparisons + 6 multi-skill playbooks):
uvx bigquery-data-science triage "forecast daily retail sales with holiday promotions"
uvx bigquery-data-science triage "schedule retraining pipeline with drift gate and Vertex AI Model Registry"

# 3. Fetch the surgical syntax card (< 25 KB) for any model or function:
uvx bigquery-data-science lookup ARIMA_PLUS
uvx bigquery-data-science lookup AI.FORECAST
uvx bigquery-data-science lookup ML.ENTITY_FEATURES_AT_TIME

# 4. Search across all 137 syntax cards, 20 reference guides, and 104 narratives:
uvx bigquery-data-science search "point-in-time feature leakage"

# 5. Statically audit a BigQuery SQL query or .sql file for prerequisites & gotchas:
uvx bigquery-data-science review-sql "SELECT AI.GENERATE('Summarize', endpoint => 'gemini-3.5-flash')"

# 6. Add --json to any catalog command (list, lookup, search, triage, review-sql) for automation:
uvx bigquery-data-science lookup AI.AGG --json
```

### Journey 3 — Run the 104 ground-truth notebooks & SQL scripts (`uv` or `pip`)

Every notebook in [`bigquery-ml/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) and [`bigquery-ai/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md) includes committed outputs for immediate reading on GitHub and one-click launch links for **Colab**, **Colab Enterprise**, **Vertex AI Workbench**, and **BigQuery Studio**.

To run notebooks locally, choose the dependency extra that matches the folders you want to execute:

| Extra | Scope | With `uv` (from a repository clone) | With `uv pip` or standard `pip` |
| :--- | :--- | :--- | :--- |
| **`(none)`** | Skills, CLI (`install`, `triage`, `lookup`, `search`), and stdio MCP server (`0` third-party dependencies) | `uv sync --no-dev` | `pip install bigquery-data-science` |
| **`[notebooks]`** | Core runtime for all 104 notebooks (`google-cloud-bigquery`, `bigframes`, `bigquery-magics`, `matplotlib`, `ipykernel`, `weasyprint`) | `uv sync --extra notebooks` | `pip install "bigquery-data-science[notebooks]"` |
| **`[workflows]`** | Adds causal inference, survival analysis, and ONNX/XGBoost import/export (`scikit-learn`, `scipy`, `xgboost`, `lifelines`, `differences`, `onnxruntime`) | `uv sync --extra workflows` | `pip install "bigquery-data-science[workflows]"` |
| **`[pipelines]`** | Adds GCP orchestration SDKs for `bigquery-ml/pipelines/*` (`kfp`, `google-cloud-aiplatform`, `dataform`, `workflows`, `orchestration-airflow`) | `uv sync --extra pipelines` | `pip install "bigquery-data-science[pipelines]"` |
| **`[all]`** | Complete notebook, workflow, and pipeline suite | `uv sync --extra all` | `pip install "bigquery-data-science[all]"` |

```bash
git clone https://github.com/statmike/bigquery-data-science.git
cd bigquery-data-science

# With uv (recommended — uses the locked uv.lock environment):
uv sync --extra all
uv run bigquery-data-science kernel --user

# Or with standard Python venv + pip:
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[all]"
bigquery-data-science kernel --user
```

---

## Front Door 1 — Agent Skills & 4-Lens Progressive Disclosure ([`skills/`](https://github.com/statmike/bigquery-data-science/blob/main/skills/PLANS.md))

### The 4 Bundled Skills

| Skill | Canonical Role | What It Covers | Entry Point |
| :--- | :--- | :--- | :--- |
| **[`bigquery-data-science`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/SKILL.md)** | **Umbrella Router & Multi-Skill Composer** (`Workflow` + `Reviewer`) | Routes across `bigquery-ml`, `bigquery-ai`, and `bigquery-mlops` (and recommends multi-skill combinations when a task spans modeling + GenAI + MLOps); 10 empirical head-to-head task comparisons; complete `ML.*` vs `AI.*` prerequisite matrix ([`namespace-matrix.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/namespace-matrix.md)); and 6 multi-skill composition playbooks ([`cross-skill-playbooks.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/cross-skill-playbooks.md)). | [`skills/bigquery-data-science/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/SKILL.md) |
| **[`bigquery-ml`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ml/SKILL.md)** | **Trained Models, Preprocessing & Pipelines** (`Workflow` + `Generator` + `Tool Wrapper` + `Reviewer`) | 23 `CREATE MODEL` types (`LINEAR_REG`, `BOOSTED_TREE_*`, `ARIMA_PLUS`, `ARIMA_PLUS_XREG`, `KMEANS`, `PCA`, `MATRIX_FACTORIZATION`, `DNN_*`, `AUTOML_*`, `CONTRIBUTION_ANALYSIS`, imported `ONNX`/`TENSORFLOW`/`XGBOOST`), 20+ model-free `ML.*` preprocessing/drift/feature-time functions, 18 workflows, and 8 production orchestration patterns. | [`skills/bigquery-ml/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ml/SKILL.md) |
| **[`bigquery-ai`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ai/SKILL.md)** | **Foundation Models, Embeddings, Search & Document AI** (`Workflow` + `Generator` + `Tool Wrapper` + `Reviewer`) | 29 SQL functions across `AI.GENERATE*`, `AI.IF`/`SCORE`/`CLASSIFY`/`AGG`, `AI.EMBED` + `VECTOR_SEARCH` + `AI.SEARCH`, `AI.FORECAST` + `AI.DETECT_ANOMALIES` (TimesFM), `AI.PREDICT` + `AI.EVALUATE` (TabFM), `AI.KEY_DRIVERS`, `AI.PARSE_DOCUMENT` + `ML.PROCESS_DOCUMENT`, and Cloud AI `ML.TRANSLATE`/`UNDERSTAND_TEXT`/`ANNOTATE_IMAGE`/`TRANSCRIBE`, plus 14 multi-function workflows. | [`skills/bigquery-ai/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ai/SKILL.md) |
| **[`bigquery-mlops`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md)** | **Production MLOps, Registry, Serving, Monitoring & Feature Store** (`Workflow` + `Generator` + `Tool Wrapper` + `Reviewer`) | Operationalizes both `ML.*` (`bigquery-ml`), `AI.*` (`bigquery-ai`), and external models logging to BigQuery across 8 orchestration engines, Vertex AI Model Registry & online endpoints (`model_registry='VERTEX_AI'`), `EXPORT MODEL` (`xgboost_version='2.1'`) / `IMPORT MODEL` (`ONNX`/`XGBOOST`/`TENSORFLOW`), drift & skew monitoring (`ML.VALIDATE_DATA_DRIFT`, `ML.VALIDATE_DATA_SKEW`, `ML.METRICS`), and point-in-time Feature Store (`ML.ENTITY_FEATURES_AT_TIME`). | [`skills/bigquery-mlops/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md) |

### How the 4 Progressive Lenses Prevent Context Bloat

| Lens | Skill Path | Typical Size | When the Agent Loads It |
| :--- | :--- | :--- | :--- |
| **Lens 1 — Router & Guardrails** | `skills/*/SKILL.md` | `~6–9 KB` | **Always first.** Task routing tree, critical SQL generation rules, cross-cutting empirical gotchas, and pre-delivery verification checklist. |
| **Lens 2 — Decision & Gotcha Guide** | `skills/*/reference/*.md` | `~15–45 KB` | **When comparing options within a family** (e.g., choosing among forecasting models, orchestration engines, or `AI.IF` vs `AI.CLASSIFY` vs `AI.GENERATE_TABLE`). |
| **Lens 3 — Surgical Syntax Card** | `skills/*/syntax/*.md` ([`ML Index`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ml/syntax/INDEX.md) · [`AI Index`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ai/syntax/INDEX.md)) | `4–15 KB` (`< 25 KB` enforced) | **When writing SQL for a specific model or function.** Sliced automatically by H2 (`## `) from `reference/*.md` so agents never load a 218 KB monolith. |
| **Lens 4 — Worked Textbook Narrative** | `skills/*/narrative/*.md` (`104` files) | `~15–30 KB` | **Only when building a full end-to-end pipeline** (including GCP IAM, Cloud Resource Connections, reservations, and multi-step SQL / BigFrames choreography). Never loads raw `.ipynb` JSON. |

---

## Front Door 2 — Ground-Truth Notebooks, SQL & Reference Hubs ([`bigquery-ml/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) & [`bigquery-ai/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md))

### Decoding BigQuery's `ML.*` vs `AI.*` SQL Namespaces

BigQuery exposes machine learning and generative AI across two SQL prefixes (`ML.*` and `AI.*`) that reflect **when** a function was introduced and **whether** it requires a `CREATE MODEL` DDL object. Full matrix: **[`skills/bigquery-data-science/reference/namespace-matrix.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/namespace-matrix.md)**.

| Pattern | SQL Prefix | Requires `CREATE MODEL`? | Requires Cloud Resource Connection? | Representative Examples | Primary Hub |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **1. Customer-Trained Models & Lifecycle** | `CREATE MODEL` + `ML.*` | **Yes** (trained on your table) | Only for `MATRIX_FACTORIZATION` / `AUTOML_*` / `REMOTE` | `LINEAR_REG`, `BOOSTED_TREE_*`, `ARIMA_PLUS`, `KMEANS`, `PCA`, `ML.PREDICT`, `ML.EVALUATE`, `ML.EXPLAIN_PREDICT`, `ML.FORECAST`, `ML.GET_INSIGHTS` | **[`bigquery-ml/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md)** |
| **2. Model-Free Feature Engineering, Drift & Point-in-Time Joins** | `ML.*` | **No** (operates directly on tables/queries) | No | `ML.STANDARD_SCALER`, `ML.QUANTILE_BUCKETIZE`, `ML.ONE_HOT_ENCODER`, `ML.VALIDATE_DATA_DRIFT`, `ML.TF_IDF`, `ML.NGRAMS`, `ML.ENTITY_FEATURES_AT_TIME` | **[`bigquery-ml/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md)** |
| **3. Zero-Training Foundation Model Functions (Gemini, TimesFM, TabFM)** | `AI.*` (plus `VECTOR_SEARCH`) | **No** (`endpoint => '...'` inline or managed) | For multimodal `ObjectRef` or custom connections | `AI.GENERATE`, `AI.GENERATE_TABLE`, `AI.IF`, `AI.SCORE`, `AI.CLASSIFY`, `AI.AGG`, `AI.EMBED`, `AI.SEARCH`, `AI.FORECAST`, `AI.DETECT_ANOMALIES`, `AI.PREDICT`, `AI.EVALUATE`, `AI.KEY_DRIVERS`, `AI.PARSE_DOCUMENT` | **[`bigquery-ai/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md)** |
| **4. Remote Cloud AI APIs & First-Gen Vertex Wrappers** | `CREATE MODEL ... REMOTE` + `ML.*` | **Yes** (thin metadata pointer to Vertex / Cloud AI API) | **Yes** (`CLOUD_RESOURCE` connection + IAM) | `ML.PROCESS_DOCUMENT`, `ML.TRANSLATE`, `ML.UNDERSTAND_TEXT`, `ML.ANNOTATE_IMAGE`, `ML.TRANSCRIBE`, `ML.GENERATE_TEXT`, `ML.GENERATE_EMBEDDING` | **[`bigquery-ai/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md)** |
| **5. Production MLOps, Model Registry, Export/Import & Universal Monitoring** | `CREATE MODEL ... OPTIONS(model_registry='VERTEX_AI')`, `EXPORT MODEL`, `ML.*` | Varies (`CREATE MODEL` for Registry/Import; **No** for `ML.VALIDATE_DATA_DRIFT` / `ML.METRICS`) | Only for `REMOTE` / Vertex endpoints | `OPTIONS(model_registry='VERTEX_AI', vertex_ai_model_id='...')`, `EXPORT MODEL` (`xgboost_version='2.1'`), `MODEL_TYPE='ONNX'\|'XGBOOST'\|'TENSORFLOW'`, `ML.VALIDATE_DATA_DRIFT`, `ML.VALIDATE_DATA_SKEW`, `ML.METRICS`, `ML.ENTITY_FEATURES_AT_TIME`, 8 orchestrators | **[`bigquery-ml/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md)** · **[`skills/bigquery-mlops/`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md)** |

### 7-Stage Problem-First Data Science Navigator (All 32 Workflows & 8 Pipelines)

Whether you are reading the repository as a human practitioner or routing an AI coding agent, every workflow and pipeline maps to one of **7 problem-first stages** (full playbook & SQL choreography: **[`skills/bigquery-data-science/reference/cross-skill-playbooks.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/cross-skill-playbooks.md)**):

| Lifecycle Stage | Skills Composed | Ground-Truth Workflows & Pipelines Across Both Hubs |
| :--- | :--- | :--- |
| **1. Data Preparation, Enrichment & Point-in-Time Feature Engineering** | `bigquery-ml` + `bigquery-ai` + `bigquery-mlops` | [`data_enrichment`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/data_enrichment/) · [`feature_store` (workflow)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/feature_store/) · [`feature_store` (function)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/feature_store/) · [`exploration`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/exploration/) · [`scalers`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/scalers/) · [`feature_engineering`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/feature_engineering/) · [`encoding`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/encoding/) · [`bucketizing`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/bucketizing/) |
| **2. Unstructured Media, Document AI & Multimodal Extraction** | `bigquery-ai` + `bigquery-ml` | [`document_intelligence`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/document_intelligence/) · [`multimodal_analysis`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/multimodal_analysis/) · [`image_deduplication`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/image_deduplication/) · [`content_analysis`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/content_analysis/) · [`content_moderation`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/content_moderation/) · [`image`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/image/) |
| **3. Embeddings, Semantic & Hybrid Search + RAG** | `bigquery-ai` + `bigquery-ml` | [`semantic_search`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/semantic_search/) · [`catalog_search`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/catalog_search/) · [`rag_pipeline`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/rag_pipeline/) · [`document_rag`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/document_rag/) · [`embeddings_classification`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/embeddings_classification/) |
| **4. Predictive Modeling (Classification, Regression, Survival & Recommendations)** | `bigquery-ml` + `bigquery-ai` | [`tabular_prediction`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/tabular_prediction/) · [`churn_retention`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/churn_retention/) · [`ga4_churn_prediction`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/ga4_churn_prediction/) · [`survival_analysis`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/survival_analysis/) · [`anomaly_fraud_detection`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/anomaly_fraud_detection/) · [`customer_segmentation`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/customer_segmentation/) · [`recommendation`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/recommendation/) · [`cross_validation`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/cross_validation/) · [`ensembling`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/ensembling/) |
| **5. Time-Series Intelligence & Forecasting** | `bigquery-ml` + `bigquery-ai` | [`time_series_intelligence`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/time_series_intelligence/) · [`regression_based_forecasting`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/regression_based_forecasting/) · [`hierarchical_forecasting`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/hierarchical_forecasting/) · [`time_series`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/time_series/) · [`log_analysis`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/log_analysis/) |
| **6. Causal Inference, Uplift & Key-Driver Attribution** | `bigquery-ml` + `bigquery-ai` | [`metric_diagnostics`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/metric_diagnostics/) · [`propensity_score_matching`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/propensity_score_matching/) · [`price_elasticity_dml`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/price_elasticity_dml/) · [`uplift_cate`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/uplift_cate/) · [`difference_in_differences`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/difference_in_differences/) · [`synthetic_control`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/synthetic_control/) · [`causal_effect`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/causal_effect/) |
| **7. Production MLOps, Registry, Serving & Drift Monitoring** | `bigquery-mlops` + `bigquery-ml` + `bigquery-ai` | [`model_monitoring`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/model_monitoring/) · [`evaluation`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/evaluation/) · [`export`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/models/export/) · [`imported`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/models/imported/) · [`remote`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/models/remote/) · [`transform_only`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/models/transform_only/) · [`sql_scripting`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/sql_scripting/) · [`scheduled_queries`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/scheduled_queries/) · [`dataform`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/dataform/) · [`cloud_workflows`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/cloud_workflows/) · [`dbt`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/dbt/) · [`vertex_kfp`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/vertex_kfp/) · [`composer_airflow`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/composer_airflow/) · [`airflow_with_kfp`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/airflow_with_kfp/) |

### Navigate the Notebook & Reference Hubs

Instead of scrolling through 104 individual rows here, jump directly into the domain hubs and category indexes below:

| Domain Hub | Sub-Folder / Resource | Count | What You Will Find Inside | Direct Entry Points |
| :--- | :--- | :--- | :--- | :--- |
| **[BigQuery ML (`bigquery-ml/`)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md)**<br/>*Trained `CREATE MODEL` + model-free `ML.*` + MLOps* | **Start Here & Reference Index** | `1` overview + `5` guides | Interactive tour of all BigQuery ML capabilities, environment setup, and the complete [`RESOURCES.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/RESOURCES.md) reference map. | [`bigquery-ml/README.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) · [`overview.ipynb`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/overview.ipynb) · [`RESOURCES.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/RESOURCES.md) · [`setup/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/setup/README.md) |
| | **[`bigquery-ml/models/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/models/)** | `23` folders (`23` `.ipynb`, `23` `.sql`) | Every `CREATE MODEL` type: `linear_reg`, `logistic_reg`, `boosted_tree_*`, `random_forest_*`, `dnn_*`, `wide_and_deep_*`, `automl_*`, `arima_plus`, `arima_plus_xreg`, `kmeans`, `pca`, `autoencoder`, `matrix_factorization`, `contribution_analysis`, `cox_proportional_hazards`, and imported `onnx`/`tensorflow`/`xgboost`. | [`bigquery-ml/README.md` (Model Catalog)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) · [`reference/create-model-model-types.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/create-model-model-types.md) · [`reference/comparison-tables.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/comparison-tables.md) |
| | **[`bigquery-ml/functions/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/functions/)** | `12` folders (`12` `.ipynb`, `12` `.sql`) | `ML.EVALUATE`, `ML.PREDICT`, `ML.EXPLAIN_PREDICT`, `ML.GLOBAL_EXPLAIN`, `ML.FEATURE_INFO`, `ML.TRANSFORM`, numerical/categorical/text preprocessing, `ML.VALIDATE_DATA_DRIFT`, `ML.DESCRIBE_DATA`, `ML.ENTITY_FEATURES_AT_TIME`, `ML.FEATURES_AT_TIME`, and hyperparameter tuning. | [`bigquery-ml/README.md` (Function Catalog)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) · [`reference/model-lifecycle-functions.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-lifecycle-functions.md) · [`reference/model-free-functions.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-free-functions.md) · [`reference/model-management-monitoring.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/reference/model-management-monitoring.md) |
| | **[`bigquery-ml/workflows/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/workflows/)** | `18` folders (`18` `.ipynb`, `15` `.sql`) | End-to-end applied ML workflows: customer LTV, churn & survival analysis, demand forecasting at scale, fraud detection, propensity scoring, price elasticity, A/B uplift & causal inference, recommendation systems, anomaly detection, and model monitoring. | [`bigquery-ml/README.md` (Workflows)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) · [`skills/bigquery-ml/reference/workflows-and-pipelines.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-ml/reference/workflows-and-pipelines.md) |
| | **[`bigquery-ml/pipelines/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/pipelines/)** | `8` folders (`8` `.ipynb`, `8` `.sql`) | Production MLOps orchestration across 8 GCP & SQL engines: `sql_scripting`, `scheduled_queries`, `dataform`, `cloud_workflows`, `dbt`, `vertex_kfp`, `composer_airflow`, and `airflow_with_kfp`. | [`bigquery-ml/README.md` (Pipelines)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ml/README.md) · [`skills/bigquery-mlops/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md) · [`skills/bigquery-mlops/reference/orchestration-and-pipelines.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/reference/orchestration-and-pipelines.md) |
| **[BigQuery AI (`bigquery-ai/`)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md)**<br/>*Zero-training `AI.*` foundation models + Cloud AI `ML.*`* | **Start Here & Reference Index** | `2` setup/overview + `8` guides | Interactive tour of BigQuery AI functions, synthetic multimodal/PDF dataset generator ([`data/documents/generate.ipynb`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/data/documents/generate.ipynb)), and [`RESOURCES.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/RESOURCES.md). | [`bigquery-ai/README.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md) · [`overview.ipynb`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/overview.ipynb) · [`RESOURCES.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/RESOURCES.md) · [`setup/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/setup/README.md) |
| | **[`bigquery-ai/functions/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/functions/)** | `29` folders (`29` `.ipynb`, `29` `.sql`) | General-purpose generation (`AI.GENERATE*`, `AI.COUNT_TOKENS`), managed task functions (`AI.IF`, `AI.SCORE`, `AI.CLASSIFY`, `AI.AGG`), embeddings & vector search (`AI.EMBED`, `AI.SIMILARITY`, `VECTOR_SEARCH`, `AI.SEARCH`), Predictive AI (`AI.FORECAST`, `AI.DETECT_ANOMALIES`, `AI.PREDICT`, `AI.EVALUATE`), `AI.KEY_DRIVERS`, Document AI (`AI.PARSE_DOCUMENT`, `ML.PROCESS_DOCUMENT`), and Cloud AI (`ML.TRANSLATE`, `ML.UNDERSTAND_TEXT`, `ML.ANNOTATE_IMAGE`, `ML.TRANSCRIBE`). | [`bigquery-ai/README.md` (Function Catalog)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md) · [`reference/general-purpose-functions.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/general-purpose-functions.md) · [`reference/managed-functions.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/managed-functions.md) · [`reference/embedding-generation-and-semantic-search.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/embedding-generation-and-semantic-search.md) · [`reference/predictive-ai.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/predictive-ai.md) · [`reference/document-processing.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/document-processing.md) |
| | **[`bigquery-ai/workflows/`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/workflows/)** | `14` folders (`14` `.ipynb`) | Composed multi-function pipelines: semantic search, RAG, hybrid catalog search (BM25 + vector + RRF + `AI.SCORE`), content analysis & moderation, data enrichment, zero-shot vs trained tabular prediction, time-series intelligence, metric diagnostics, log analysis, document intelligence & PDF RAG, multimodal analysis, and image deduplication. | [`bigquery-ai/README.md` (Workflows)](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/README.md) · [`reference/unstructured-data-infrastructure.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/unstructured-data-infrastructure.md) · [`reference/cloud-ai-service-models.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/cloud-ai-service-models.md) · [`reference/augmented-analytics.md`](https://github.com/statmike/bigquery-data-science/blob/main/bigquery-ai/reference/augmented-analytics.md) |
| **[Agent Skills (`skills/`)](https://github.com/statmike/bigquery-data-science/blob/main/skills/PLANS.md)**<br/>*4 Composable Skills · 4 Progressive Lenses* | **Umbrella Router + 3 Specialist Skills** | `4` `SKILL.md` + `20` guides + `137` syntax cards + `104` narratives | Multi-skill task routing (`bigquery-data-science`), trained models & preprocessing (`bigquery-ml`), zero-training foundation models & search (`bigquery-ai`), and production orchestration, registry, export/import, drift & feature store (`bigquery-mlops`). | [`skills/bigquery-data-science/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/SKILL.md) · [`cross-skill-playbooks.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/cross-skill-playbooks.md) · [`namespace-matrix.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-data-science/reference/namespace-matrix.md) · [`skills/bigquery-mlops/SKILL.md`](https://github.com/statmike/bigquery-data-science/blob/main/skills/bigquery-mlops/SKILL.md) |

---

## Contributing, Pre-Commit & CI/CD

Contributions are welcome — see [`CONTRIBUTING.md`](https://github.com/statmike/bigquery-data-science/blob/main/CONTRIBUTING.md), [`CODE_OF_CONDUCT.md`](https://github.com/statmike/bigquery-data-science/blob/main/CODE_OF_CONDUCT.md), and [`SECURITY.md`](https://github.com/statmike/bigquery-data-science/blob/main/SECURITY.md).

- **GitHub Actions CI & Release Workflows:** [`.github/workflows/ci.yml`](https://github.com/statmike/bigquery-data-science/blob/main/github/workflows/ci.yml) runs `ruff`, `mypy`, `pytest`, `bigquery-data-science sweep`, `sync_headers.py --check`, and `./scripts/check_install.sh` across Python 3.11, 3.12, and 3.13 on every push and pull request. [`.github/workflows/release.yml`](https://github.com/statmike/bigquery-data-science/blob/main/github/workflows/release.yml) publishes release candidates (`v*rc*`) to TestPyPI and production tags (`v*`) to PyPI via OIDC Trusted Publishing.
- **Local Pre-Commit Hooks ([`.pre-commit-config.yaml`](https://github.com/statmike/bigquery-data-science/blob/main/pre-commit-config.yaml)):** Install with `uv run pre-commit install` to run `ruff`, `mypy`, `bigquery-data-science sweep`, and `sync_headers.py --check` automatically before each commit.

Run the complete local verification and cold-install gate before submitting a pull request:

```bash
uv run ruff check src tests && \
  uv run mypy src && \
  uv run pytest -q && \
  uv run bigquery-data-science sweep && \
  python3 architectures/headers/sync_headers.py --check && \
  ./scripts/check_install.sh
```

## Versioning & Release Cadence

`bigquery-data-science` adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) (`Major.Minor.Patch`):

- **Major (`X.0.0`) — Skills architecture, names & structural contract shifts:** Changes in skills hierarchy/naming (`skills/`), packaging overhauls, or breaking changes to CLI subcommands, MCP tool schemas, or library exports.
- **Minor (`X.Y.0`) — Multi-area monthly sweeps & capability rollouts:** Scheduled monthly sweeps across BigQuery release notes — introducing new foundation models, options, notebooks, SQL workflows, and synchronized 4-lens skill updates.
- **Patch (`X.Y.Z`) — Targeted single-area updates & fixes:** Isolated bug fixes, typo corrections, single-notebook adjustments, or targeted hotfixes.

See [`CONTRIBUTING.md`](https://github.com/statmike/bigquery-data-science/blob/main/CONTRIBUTING.md#releasing) for the full release process and TestPyPI/PyPI tag flow.

## License

Apache-2.0 — see [`LICENSE`](https://github.com/statmike/bigquery-data-science/blob/main/LICENSE).

This is not an officially supported Google product. It is built on Google products and is neither
endorsed nor supported by Google.
