Metadata-Version: 2.5
Name: researchos-ci
Version: 0.4.0
Summary: Research CI verifier with evidence-bound quant and ML domain checks
Project-URL: Repository, https://github.com/demirelo/researchos
Project-URL: Issues, https://github.com/demirelo/researchos/issues
Author: ResearchOS
License: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Requires-Python: >=3.11
Requires-Dist: pyyaml>=6.0
Description-Content-Type: text/markdown

# ResearchOS / Research CI

**Verify claims and evidence in AI-assisted computational research.**

Research CI is the main product. It checks declared claims, linked evidence,
domain rules and optional reproduction before a research change passes CI.
The optional Conductor adds approved Builder steps and independent review around
that verification gate.

A passing configuration check is separate from a reproduced, supported claim.
The verdict `proved` means a declared computational predicate passed the configured
checks. It does not establish general scientific truth.

## Implementation status

Version **0.4.0** extends the implementation of all 12 improvement ideas from
[the proposal](https://researchos-10x-ideas-jai3.pages.grok.bot/).
[PR #16](https://github.com/demirelo/researchos/pull/16) merged the initial implementation.
This update completes the remaining code features and adds publishing workflows.
Registry publication and public report deployment remain separate service actions.

| Idea | Current implementation | Operating limit |
|---|---|---|
| GitHub Action | Sticky PR comment, Check Run, line annotations, job summary and report artifacts | Code scanning must be enabled before GitHub can accept SARIF uploads |
| Finding guidance | Shared reasons, required actions, config examples, `explain` and print-only `--fix` | Suggested fixes require review; they do not create evidence |
| Packaged domain packs | Quant and ML resources, verified wheel/source builds and GitHub/PyPI release workflow | PyPI requires the owning account to configure its trusted publisher |
| Claim extraction | Draft claims from Markdown, notebook Markdown and PR text; optional model review | Claims remain proposed until a person reviews them and links evidence |
| Reproduction | Explicit Docker execution, fresh outputs, timeouts and authenticated cache for pinned image digests | Experiments must supply their dependencies and reproduction scripts |
| Evidence binding | Scalar and interval predicates, percentile/basic bootstrap, tolerances, artifact hashes and fresh-result comparison | Sampling assumptions and interval coverage require review |
| Verdict semantics | Supported reproduced keystones can be terminal; valid contradictory results can be falsified | Skipped reproduction and legacy self-attestations remain non-terminal |
| Builder loop | Approved worktree edits, Venice and Claude Code adapters, independent review, new verification and iteration checkpoints | Custom adapters must enforce their provider budget |
| Quant checks | Typed assumptions, bounded def/use dataflow, computed pairing permutation controls and advisory deflated Sharpe | Dynamic Python flows and experiment refits require explicit project protocols |
| Reports and badge | Static reports, HTTP report/badge endpoint, checksummed sites and Pages deployment workflow | [Synthetic demo is live](https://demirelo.github.io/researchos/); real reports require an approved destination |
| ML and pack SDK | Split hashes/membership, seed results, baseline parity and 12 declarative SDK operators for config or reproduced metrics | Custom fixture execution requires an explicit Docker or trusted-host flag |
| Event and CLI consistency | Incremental reads, persisted settings, redirects, request recovery and durable cost records | Uncertain paid attempts require reconciliation before retry |

See the [implementation and validation record](docs/IMPLEMENTATION_10X.md) for
check details and the [workflow guide](docs/USAGE_10X.md) for configuration.

## Requirements and installation

Use Python **3.11+**. The examples below use [`uv`](https://docs.astral.sh/uv/).
Docker must be running when you request container reproduction. Prepare an image
with the experiment dependencies; reproduction does not install them.

```bash
git clone https://github.com/demirelo/researchos.git
cd researchos
uv sync
uv run researchos --version
```

Build and install a local wheel:

```bash
uv build --clear
python3.11 -m venv /tmp/researchos-venv
/tmp/researchos-venv/bin/python -m pip install dist/researchos_ci-0.4.0-py3-none-any.whl
/tmp/researchos-venv/bin/researchos --version
```

The distribution name is `researchos-ci`; the command and Python import remain
`researchos`. The wheel includes both domain packs. Version 0.4.0 was built and tested locally;
this change does not publish it to PyPI or create a downloadable release.

## Verify a project

```bash
uv run researchos verify --ci examples/quant-failing-pr
```

The bundled quant fixture is deliberately incomplete. Expect a non-zero exit
and blocking findings. Verification does **not execute project code by default**.

Request restricted Docker reproduction explicitly:

```bash
uv run researchos verify --ci --reproduce --timeout 600 \
  --image python:3.12-slim /path/to/project
```

Docker reproduction disables network access, removes inherited environment
variables, uses a read-only root filesystem and enforces resource limits.
Pin the image digest to enable authenticated reproduction caching.
`--unsafe-local` runs trusted project code on the host with no filesystem or
network isolation.

Declare claims and evidence in `researchos.yaml`. Supply a reproduction script
and fresh output files when you need a reproduced verdict. See
[structured claims and reproduction](docs/USAGE_10X.md) and
[statistical evidence and controls](docs/STATISTICAL_CHECKS.md).

## Extract claims, inspect findings and generate reports

```bash
uv run researchos init /path/to/project --domain ml
uv run researchos explain quant.transaction_costs_present
uv run researchos verify --ci --json-output verdict.json /path/to/project
uv run researchos report verdict.json --output report/index.html \
  --badge-output report/claims.svg
uv run researchos pack test quant-finance
uv run researchos pack test ml
```

`init` writes `researchos.draft.yaml` with proposed claims and candidate result
files. Review the draft and link measured evidence before accepting it.
`--fix` prints configuration examples for review; it does not edit the project
or invent measurements. Reports and badges remain local until you publish them.
Serve a live report and badge with `researchos report verdict.json --serve`, or
create a deployment directory with `--site-dir public-report`. See
[report hosting and releases](docs/PUBLISHING.md) and the
[pack rule reference](docs/PACK_RULES.md).

## GitHub integration

The [composite Action](docs/GITHUB_ACTION.md) verifies a project and can publish
one sticky bot comment, a Check Run and line annotations. It also writes JSON,
SARIF, Markdown, HTML and SVG files for artifact upload. Fork PRs receive no
privileged publication or project-code execution.

The repository workflows check tests, secrets, repository hygiene and the
expected-failure fixture. A separate maintainer smoke workflow tests a synthetic
project. Its comment and Check Run use a separate identity so that fixture
success cannot replace a project verdict.

Code scanning is disabled for this repository as of 6 November 2026. SARIF remains
available in report artifacts; the smoke workflow skips code-scanning upload
when the service is unavailable.

## Optional Conductor and model providers

The Conductor records verification results, model proposals, director decisions,
Builder diffs, review results and costs in an append-only event log.
Venice is the primary documented model path. Set `VENICE_API_KEY` through your
environment, an ignored `.env`/`.env.local`, or the supported macOS Keychain entry.
Do not put credentials in prompts, commits or logs.

Start a model-assisted planning run:

```bash
uv run researchos conductor start --project /path/to/project \
  --goal 'Resolve the reproduction blocker' \
  --mode venice --model qwen3-235b-a22b-instruct-2507 --budget-usd 2
uv run researchos conductor progress
uv run researchos conductor approve next
```

Planning alone does not configure a coding agent. For approved file edits and
independent review, configure both native adapters:

```bash
uv run researchos conductor start --project /path/to/project \
  --goal 'Resolve the reproduction blocker' \
  --mode venice --model qwen3-235b-a22b-instruct-2507 \
  --builder-command '["/ABS/VENV/bin/python","-m","researchos.conductor.venice_adapter"]' \
  --adversary-command '["/ABS/VENV/bin/python","-m","researchos.conductor.venice_adapter"]' \
  --allow-project-code-execution --budget-usd 2 --iteration-limit 2 \
  --execution-timeout 600
uv run researchos conductor approve next
```

Use an absolute path to the Python environment that contains ResearchOS.
The native Venice adapter loads in isolated mode, receives only its scoped
provider credential, and applies structured file edits. It does not execute
model-proposed commands. Docker reproduction runs separately without that key.
Each further iteration requires approval. No automatic commit or merge occurs.

Venice costs derived from the live catalog are labelled estimates, not invoices.
Missing usage or uncertain completions stop further work for reconciliation.
OpenRouter PI calls and a Claude Code Builder/reviewer adapter are also supported.
See the [Builder and provider guide](docs/USAGE_10X.md).

`--mode fake` provides deterministic planning guidance with no model calls. Its
STORM-style events are checklists, not independent model research. Live independent
review requires a configured review adapter.

## Agent guidance

```bash
uv run researchos skill install claude
uv run researchos skill install codex --dest .
```

These installers provide guidance for running ResearchOS before an agent reports
research progress. They do not make unsupported claims pass verification.

## Validation

Version 0.4.0 passed **156 local tests**, including clean-wheel installation,
resampling, interval verdicts, SDK rules, dataflow, HTTP and release regressions.
Astra reviewed the code at medium effort through the local CLI. Six findings
were corrected; the closure review found no blocking findings. See the
[completion record](docs/COMPLETION_PLAN.md).

Live checks passed Docker isolation and cache reuse, Venice claim-draft review,
Venice Builder/reviewer execution, conservative Claude review rejection, and
GitHub comment/check updates, warning annotations and downloaded report artifacts.
These checks used synthetic projects. They verify the software workflow, not
real financial or scientific performance.

## Project documents

- [Workflow and configuration guide](docs/USAGE_10X.md)
- [GitHub Action setup](docs/GITHUB_ACTION.md)
- [Implementation and validation record](docs/IMPLEMENTATION_10X.md)
- [Statistical evidence and controls](docs/STATISTICAL_CHECKS.md)
- [Pack rule reference](docs/PACK_RULES.md)
- [Report hosting and releases](docs/PUBLISHING.md)
- [Positioning](docs/POSITIONING.md)
- [Domain pack specification](docs/DOMAIN_PACK_SPEC.md)
- [Open-core boundary](docs/OPEN_CORE_BOUNDARY.md)
- [Research Conductor design](docs/RESEARCH_CONDUCTOR.md)
- [STORM integration design](docs/STORM_INTEGRATION.md)
