Metadata-Version: 2.4
Name: mostlyright-data
Version: 0.14.0
Summary: Mostly Right CLI for reproducible, reviewed Parquet datasets
Project-URL: Homepage, https://mostlyright.md/
Project-URL: Documentation, https://mostlyright.md/docs/guides/cli/
Requires-Python: <3.14,>=3.13
Provides-Extra: build
Requires-Dist: hatchling==1.28.0; extra == 'build'
Requires-Dist: uv-build==0.11.3; extra == 'build'
Provides-Extra: dev
Requires-Dist: antlr4-python3-runtime==4.9.3; extra == 'dev'
Requires-Dist: astropy==8.0.1; extra == 'dev'
Requires-Dist: cffi==2.1.0; extra == 'dev'
Requires-Dist: cmarkgfm>=2025.10.22; extra == 'dev'
Requires-Dist: colorlog==6.12.0; extra == 'dev'
Requires-Dist: croniter==6.2.4; extra == 'dev'
Requires-Dist: cryptography==50.0.0; extra == 'dev'
Requires-Dist: duckdb==1.5.5; extra == 'dev'
Requires-Dist: eccodes==2.48.0; extra == 'dev'
Requires-Dist: eccodeslib==2.48.0.26; (sys_platform != 'win32') and extra == 'dev'
Requires-Dist: et-xmlfile==2.0.0; extra == 'dev'
Requires-Dist: flatbuffers==25.12.19; extra == 'dev'
Requires-Dist: h5py==3.16.0; extra == 'dev'
Requires-Dist: idna==3.18; extra == 'dev'
Requires-Dist: jsonschema[format]<5,>=4.23; extra == 'dev'
Requires-Dist: lxml==6.1.2; extra == 'dev'
Requires-Dist: narwhals==2.25.0; extra == 'dev'
Requires-Dist: netcdf4==1.7.4; extra == 'dev'
Requires-Dist: numpy==2.5.1; extra == 'dev'
Requires-Dist: omegaconf==2.3.1; extra == 'dev'
Requires-Dist: onnxruntime==1.28.0; extra == 'dev'
Requires-Dist: opencv-python-headless==5.0.0.93; extra == 'dev'
Requires-Dist: openpyxl==3.1.5; extra == 'dev'
Requires-Dist: packaging==26.3; extra == 'dev'
Requires-Dist: pandas==3.0.3; extra == 'dev'
Requires-Dist: pikepdf==10.10.0; extra == 'dev'
Requires-Dist: pillow==12.3.0; extra == 'dev'
Requires-Dist: polars-runtime-32==1.42.1; extra == 'dev'
Requires-Dist: polars==1.42.1; extra == 'dev'
Requires-Dist: protobuf==7.35.1; extra == 'dev'
Requires-Dist: pyarrow==23.0.1; extra == 'dev'
Requires-Dist: pyclipper==1.4.0; extra == 'dev'
Requires-Dist: pycparser==3.0; extra == 'dev'
Requires-Dist: pyhdf==0.11.7; ((sys_platform == 'linux' and platform_machine == 'x86_64') or (sys_platform == 'darwin' and platform_machine == 'arm64')) and extra == 'dev'
Requires-Dist: pypdfium2==5.13.0; extra == 'dev'
Requires-Dist: pypublicdecompwt==2.8.1.6; extra == 'dev'
Requires-Dist: pyreadstat==1.3.6; extra == 'dev'
Requires-Dist: pytest-xdist>=3.6; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: python-dateutil==2.9.0.post0; extra == 'dev'
Requires-Dist: pyyaml==6.0.3; extra == 'dev'
Requires-Dist: rasterio==1.5.1; extra == 'dev'
Requires-Dist: referencing<1,>=0.35; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Requires-Dist: shapely==2.1.2; extra == 'dev'
Requires-Dist: six==1.17.0; extra == 'dev'
Requires-Dist: zarr==3.3.0; extra == 'dev'
Provides-Extra: hosted
Requires-Dist: mostlyright-studio-client-v3==3.0.0; extra == 'hosted'
Provides-Extra: local
Requires-Dist: antlr4-python3-runtime==4.9.3; extra == 'local'
Requires-Dist: cffi==2.1.0; extra == 'local'
Requires-Dist: colorlog==6.12.0; extra == 'local'
Requires-Dist: croniter==6.2.4; extra == 'local'
Requires-Dist: cryptography==50.0.0; extra == 'local'
Requires-Dist: duckdb==1.5.5; extra == 'local'
Requires-Dist: et-xmlfile==2.0.0; extra == 'local'
Requires-Dist: flatbuffers==25.12.19; extra == 'local'
Requires-Dist: idna==3.18; extra == 'local'
Requires-Dist: lxml==6.1.2; extra == 'local'
Requires-Dist: narwhals==2.25.0; extra == 'local'
Requires-Dist: numpy==2.5.1; extra == 'local'
Requires-Dist: omegaconf==2.3.1; extra == 'local'
Requires-Dist: onnxruntime==1.28.0; extra == 'local'
Requires-Dist: opencv-python-headless==5.0.0.93; extra == 'local'
Requires-Dist: openpyxl==3.1.5; extra == 'local'
Requires-Dist: packaging==26.3; extra == 'local'
Requires-Dist: pandas==3.0.3; extra == 'local'
Requires-Dist: pikepdf==10.10.0; extra == 'local'
Requires-Dist: pillow==12.3.0; extra == 'local'
Requires-Dist: polars-runtime-32==1.42.1; extra == 'local'
Requires-Dist: polars==1.42.1; extra == 'local'
Requires-Dist: protobuf==7.35.1; extra == 'local'
Requires-Dist: pyarrow==23.0.1; extra == 'local'
Requires-Dist: pyclipper==1.4.0; extra == 'local'
Requires-Dist: pycparser==3.0; extra == 'local'
Requires-Dist: pypdfium2==5.13.0; extra == 'local'
Requires-Dist: pyreadstat==1.3.6; extra == 'local'
Requires-Dist: python-dateutil==2.9.0.post0; extra == 'local'
Requires-Dist: pyyaml==6.0.3; extra == 'local'
Requires-Dist: shapely==2.1.2; extra == 'local'
Requires-Dist: six==1.17.0; extra == 'local'
Provides-Extra: local-search-neural
Requires-Dist: anyio==4.14.2; extra == 'local-search-neural'
Requires-Dist: certifi==2026.7.22; extra == 'local-search-neural'
Requires-Dist: click==8.4.2; extra == 'local-search-neural'
Requires-Dist: filelock==3.32.2; extra == 'local-search-neural'
Requires-Dist: flatbuffers==25.12.19; extra == 'local-search-neural'
Requires-Dist: fsspec==2026.7.0; extra == 'local-search-neural'
Requires-Dist: h11==0.16.0; extra == 'local-search-neural'
Requires-Dist: hf-xet==1.6.0; extra == 'local-search-neural'
Requires-Dist: httpcore==1.0.9; extra == 'local-search-neural'
Requires-Dist: httpx==0.28.1; extra == 'local-search-neural'
Requires-Dist: huggingface-hub==1.27.0; extra == 'local-search-neural'
Requires-Dist: idna==3.18; extra == 'local-search-neural'
Requires-Dist: numpy==2.5.1; extra == 'local-search-neural'
Requires-Dist: onnxruntime==1.28.0; extra == 'local-search-neural'
Requires-Dist: packaging==26.3; extra == 'local-search-neural'
Requires-Dist: protobuf==7.35.1; extra == 'local-search-neural'
Requires-Dist: pyyaml==6.0.3; extra == 'local-search-neural'
Requires-Dist: tokenizers==0.23.1; extra == 'local-search-neural'
Requires-Dist: tqdm==4.70.0; extra == 'local-search-neural'
Requires-Dist: typing-extensions==4.16.0; extra == 'local-search-neural'
Provides-Extra: recorder
Requires-Dist: cryptography==50.0.0; extra == 'recorder'
Provides-Extra: release
Requires-Dist: antlr4-python3-runtime==4.9.3; extra == 'release'
Requires-Dist: cffi==2.1.0; extra == 'release'
Requires-Dist: colorlog==6.12.0; extra == 'release'
Requires-Dist: croniter==6.2.4; extra == 'release'
Requires-Dist: cryptography==50.0.0; extra == 'release'
Requires-Dist: duckdb==1.5.5; extra == 'release'
Requires-Dist: et-xmlfile==2.0.0; extra == 'release'
Requires-Dist: flatbuffers==25.12.19; extra == 'release'
Requires-Dist: idna==3.18; extra == 'release'
Requires-Dist: jsonschema[format]<5,>=4.23; extra == 'release'
Requires-Dist: lxml==6.1.2; extra == 'release'
Requires-Dist: narwhals==2.25.0; extra == 'release'
Requires-Dist: numpy==2.5.1; extra == 'release'
Requires-Dist: omegaconf==2.3.1; extra == 'release'
Requires-Dist: onnxruntime==1.28.0; extra == 'release'
Requires-Dist: opencv-python-headless==5.0.0.93; extra == 'release'
Requires-Dist: openpyxl==3.1.5; extra == 'release'
Requires-Dist: packaging==26.3; extra == 'release'
Requires-Dist: pandas==3.0.3; extra == 'release'
Requires-Dist: pikepdf==10.10.0; extra == 'release'
Requires-Dist: pillow==12.3.0; extra == 'release'
Requires-Dist: polars-runtime-32==1.42.1; extra == 'release'
Requires-Dist: polars==1.42.1; extra == 'release'
Requires-Dist: protobuf==7.35.1; extra == 'release'
Requires-Dist: pyarrow==23.0.1; extra == 'release'
Requires-Dist: pyclipper==1.4.0; extra == 'release'
Requires-Dist: pycparser==3.0; extra == 'release'
Requires-Dist: pypdfium2==5.13.0; extra == 'release'
Requires-Dist: pyreadstat==1.3.6; extra == 'release'
Requires-Dist: python-dateutil==2.9.0.post0; extra == 'release'
Requires-Dist: pyyaml==6.0.3; extra == 'release'
Requires-Dist: shapely==2.1.2; extra == 'release'
Requires-Dist: six==1.17.0; extra == 'release'
Provides-Extra: scientific
Requires-Dist: astropy==8.0.1; extra == 'scientific'
Requires-Dist: eccodes==2.48.0; extra == 'scientific'
Requires-Dist: eccodeslib==2.48.0.26; (sys_platform != 'win32') and extra == 'scientific'
Requires-Dist: h5py==3.16.0; extra == 'scientific'
Requires-Dist: netcdf4==1.7.4; extra == 'scientific'
Requires-Dist: pillow==12.3.0; extra == 'scientific'
Requires-Dist: pyhdf==0.11.7; ((sys_platform == 'linux' and platform_machine == 'x86_64') or (sys_platform == 'darwin' and platform_machine == 'arm64')) and extra == 'scientific'
Requires-Dist: pypublicdecompwt==2.8.1.6; extra == 'scientific'
Requires-Dist: rasterio==1.5.1; extra == 'scientific'
Requires-Dist: zarr==3.3.0; extra == 'scientific'
Description-Content-Type: text/markdown

# mostlyright-data

Browse datasets built by our community or create your own with agents that find the sources and
configure the pipeline. Mostly Right treats a successful run as one readable dataset version;
publishing that version does not by itself prove that catch-up or scheduled refresh is running.

`mostlyright-data` is the command-line client. It probes sources, registers one recipe document,
starts and follows hosted dataset runs, reads what they delivered, and downloads the datasets after
they finish.

## Requirements

Use CPython 3.13.

## Install

```bash
python -m pip install mostlyright-data
```

That is the whole install. There is no second profile and no lane to choose: every `mr-data`
command sends its work to the backend, and it works on Linux, macOS, and Windows.

It also carries the `mr-data-build` agent skill, which Claude Code and Codex read to author a
recipe without rediscovering the contract a run at a time. The first `mr-data` command you run
places it in your agent's skill directory; there is nothing to install by hand. A skill you have
edited yourself is never replaced, and `MOSTLYRIGHT_SKILL_AUTOINSTALL=0` switches the placement
off.

## Sign in

```bash
mr-data login
mr-data whoami
```

`login` stores a device key on the current machine. `whoami` checks that key.

## Check the installation

```bash
mr-data --version
mr-data --help
```

## Build a dataset

```bash
mr-data probe SOURCE_ID "does this feed carry city grain before 2021"
mr-data recipe RECIPE.json
mr-data run RECIPE_ID --sample --max-rows 50000
mr-data checks RUN
mr-data download RUN --output ./out
```

`probe` asks one registered source one question; the session behind it is opened, reused and closed
for you. `recipe` registers one document and prints the identifiers the server derived from it.
`run` starts a run and `mr-data status RUN`, `mr-data runs`, and `mr-data watch RUN` report on it.
The current Harness worker executes unwindowed `sample` and `full` runs. It refuses a stated run
window, and it does not execute `refresh`, `backfill`, promotion catch-up, or scheduled warehouse
continuation. `mr-data promote TABLE_ID` is still the human-gated publication request, but claim a
table is live and current only when the deployed Studio and warehouse services return that evidence.

## There is no local execution

There used to be. A second product lane built datasets on your own machine, and `mr-data` carried
about fifty commands for making, running, serving, indexing, checking and signing a build here.
That lane is gone: the engine it ran now ships only inside the backend's worker images.

`mr-data --help` lists every command this product has. A name that is not on it is not a command
and says so. The distribution still publishes a `[local]` extra — that is the engine's dependency
closure, which the backend's worker image installs — and installing it makes no command run that
does not run without it.

## Documentation

- [CLI guide](https://mostlyright.md/docs/guides/cli/)
- [Hosted research sessions](https://mostlyright.md/docs/guides/hosted-research/)
- [Credentials](https://mostlyright.md/docs/guides/credentials/)
- [Public datasets](https://mostlyright.md/docs/guides/use-public-datasets/)
- [Certified document extraction](docs/DOCUMENT-EXTRACTION.md)

Use `mr-data --help` for the full command list and options. Commands that support
`--json` write one JSON object.

`--output` may use a relative path.

## Behavior and limits

- Weather Reader versions 1 and 2 accept one extracted GRIB2 record and refuse a multi-record GFS
  or HRRR file. Version 3 reads a bounded collection and selects exactly one sealed record.
- Hosted credential-free acquisition supports one single-address public HTTPS object. Bounded collections have no hosted route, and the route they used to have was the deleted local one.
- A keyed stream venue is named by a credential reference the backend holds, never by a value. No command flag takes a secret as an argument. Venues reached over an actual WebSocket are covered; a venue whose credential comes from an interactive human login, or that is not WebSocket at all, is not.

## Verification

```bash
make test-fast
```

`make test-fast` is non-authoritative and is useful while editing. `make verify` is required
before review or merge.

## Development

```bash
uv sync --locked --extra dev
```

The test suite drives the execution engine in-process, and requires Linux for the worker-boundary
tests. The client itself is tested on Linux, macOS, and Windows.
