Metadata-Version: 2.4
Name: bolero-denmark
Version: 0.2.5
Summary: Danish literature and register-schema resources for Bolero, unpacked by `bolero-harness denmark`
Author: Northwestern-CSSI
License-Expression: MIT AND LicenseRef-Corpus-Rights
Project-URL: Homepage, https://github.com/Northwestern-CSSI/Bolero-denmark
Keywords: bolero,denmark,statistics-denmark,register-data,corpus
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE.md
Dynamic: license-file

# bolero-denmark

The canonical Danish literature and schema resource root for Bolero. The GitHub
repository preserves curated agent context plus legally redistributable source
snapshots; the data-only PyPI distribution carries the curated text, catalogs,
and templates while `.gitattributes` excludes bulk `originals/` from the wheel.

## Use

```bash
pip install bolero-ai bolero-denmark
bolero-harness denmark            # unpack this repository's runtime resources
bolero-harness denmark DIR --force
```

`bolero-harness denmark` (bolero-ai >= 0.1.8.post3) uses the same hardened tar
extraction as the harness source tree. Without bolero-ai:

```python
python3 -c "import bolero_denmark, tarfile; tarfile.open(bolero_denmark.archive_path()).extractall('.', filter='data')"
```

The package archive is produced with `git archive HEAD`. Paths marked
`export-ignore` are therefore available in GitHub clones but not the PyPI
payload. The `DST_agentic_ext/` submodule is also not embedded. To populate it
in a checkout with upstream access:

```bash
git clone --recurse-submodules https://github.com/Northwestern-CSSI/Bolero-denmark.git
# or: git submodule update --init
```

## What is inside

```text
resources/
  literature/             economics and medical disciplinary norms, catalogs,
                          reusable templates, target-paper source material,
                          9 paper Markdown conversions, and the systematic
                          Danish-register economics extraction corpus
  schema/                 agent-facing FSV/TIMES and lean HKV documentation,
                          normalized pilot inventory, approved-upload-list evidence,
                          official DST/SDS documents, SKS/LPR material, and
                          pinned parsed Danish/English/mapping source archives
replication/              navigation to the three current native replica Worlds
preregistration/bundles/  single current collection: seven Denmark Worlds and
                          the TechBridge benchmark; matching ZIPs and both PDFs
preregistration/          current index and separately labeled historical plans
DST_agentic_ext/          pinned private collaboration submodule; pointer only
tools/prereg-lane/        two debates + Transcriber, native prereg World output
deploy/gefion/            Qwen single/dual-srun and other model deployment recipes
```

Start with [`resources/README.md`](resources/README.md). The Chinese
classification and synthesis is
[`resources/RESOURCE_CLASSIFICATION.zh.md`](resources/RESOURCE_CLASSIFICATION.zh.md).

For two Qwen replicas on the **same eight-H100 compute node**, use the
[dual-srun deployment instructions](deploy/gefion/README.md). They reuse the
existing Qwen serving recipe and launch two disjoint four-GPU steps plus a
CPU-only SGLang Router inside one existing allocation. Bolero uses the single
router endpoint at port 30000. Preview is the default; `--execute` opts into launch.

The current **0.2.4** handoff is indexed in
[`preregistration/bundles/README.md`](preregistration/bundles/README.md), with
per-package source versions, current-standard audits and honest support gaps.
It contains seven Denmark studies (including DS1.1) and the TechBridge benchmark.
Every World includes a current paper template PDF and preregistration PDF.
The GH1 addition still awaits selection between two different source studies.
Superseded release trees and old study decks are no longer shipped as alternative
inputs; [Git history](https://github.com/Northwestern-CSSI/Bolero-denmark/tree/e76714d/preregistration)
and existing release tags retain them without altering their original bytes.

With Bolero 0.2.3 or newer, extract a finished bundle directly into a new
session so that it contains `world_model/{atoms,molecules,docs,README.md}`.
From the initialized project, launch with:

```bash
bash run-research.sh campaign.mode=prereg hydra.run.dir="$PWD/outputs/my-study"
```

Here the bundle is already at `outputs/my-study/world_model/` and
`campaign.prereg_world=null` (the default). The Harness validates and adopts
the World in place without rewriting or copying it. Use `campaign.mode=replica`
for a replication package. An external source may still be copied through
`campaign.prereg_world=<external/world_model>` into an absent target. Do not
nest two world_model/ directories, combine two sources, or overwrite an
initialized session; use `bolero-harness research -r <session>` to resume.
Configure governed resources, model and runtime as usual. The existing released
bundle ZIPs needed no retranscription for in-place adoption; newer handoff
revisions record navigation/provenance changes separately from science.
Only Atom/Molecule format and their graph gate Harness startup. Docs, README,
PDFs, LaTeX, templates, manifests, checksums and review status remain useful
producer deliverables, not required launch files. The same applies to dispatch,
replay and completion reporting; order comes from Molecule membership.
Preserve useful existing supporting materials generously under classified
docs/ directories. The documented supporting filenames are recommendations;
unavailable files may be omitted or empty without preventing execution.

The standalone preregistration lane lives in
[`tools/prereg-lane/`](tools/prereg-lane/). It gives every role a small,
stage-specific evidence bundle and exposes this repository's literature,
concept graph, schema, economics, medical, and package conventions through
narrow skills. It runs locally with a shared workspace and unrestricted live
search; it does not depend on the Bolero harness or confidential data runtime.
Its accepted artifact is a closed ex-ante epistemic graph: possible experiment
results update claims and conditionally activate later experiments or terminal
interpretation branches before outcome data are seen.

The schema scale currently represented by the curated layer is 531 FSV
registers (29,234 variables), 121 TIMES top-level registers (9,843 documented
variable occurrences), and a lean HKV index covering 71 datasets and 881
variables. The literature layer includes 828 systematic economics
corpus rows, 229 confirmed papers, structured extraction/audit records, eight
target/pilot papers in nine Markdown versions, and economics and medical
standards with source-level rights/provenance decisions.

## Source preservation versus runtime payload

- Exact public or licensed originals are retained under `resources/**/originals/`
  and bound by SHA-256. The Mikkelsen target originals live in
  `resources/literature/medical/target/`. These files remain in GitHub but are
  excluded from `git archive` so the runtime wheel stays focused and does not
  duplicate large provenance archives.
- Copyrighted sources without confirmed redistribution rights are represented
  by canonical citation, URL, rights status, and an original project summary;
  their PDFs are not copied.
- The expanded raw registry metadata layer (~876 MB) is preserved through the
  archived parsed Danish, unofficial-English, FSV, and mapping releases. The
  included generators deterministically rebuild the curated FSV, TIMES, and
  lean HKV views from a documented assembled source layout; the upstream crawl
  itself is not duplicated here.
- No row-level registry microdata is present. Public names, register codes,
  variable descriptions, and documentation URLs are schema, not observations.
- Private Slack/email transcripts and operational credentials are not packaged.
  Scientific conclusions derived from project coordination are rewritten as
  public, auditable summaries.

## Resource semantics

`literature/` answers “what makes this analysis scientifically defensible?”
`schema/` answers “what does this data source mean, and what evidence supports
its availability?” A pilot-sheet listing, public codebook, grant approval,
runtime mount, and validated table are distinct evidence states. See
[`resources/schema/README.md`](resources/schema/README.md).

Replication decks cite resources relative to the mounted `resources/` root:
for example `literature/papers/md/...` and
`schema/data-registry-bolero/fsv/...`. Their registered scientific protocol is
fixed; only equivalent data operationalization may adapt silently.

## Licensing and provenance

The project license expression is `MIT AND LicenseRef-Corpus-Rights`. MIT covers
the packaging and project-authored material. `NOTICE.md` identifies the rights
of every corpus class. Each discipline catalog records source URLs, licenses,
and whether an exact file was archived or kept link-only. Checksums are in:

- `resources/literature/CHECKSUMS.sha256`
- `resources/schema/provenance/CHECKSUMS.sha256`

Official Danish source content is authoritative relative to the English
translation. Parsed snapshots can be stale or erroneous; English schema text
is explicitly an unofficial machine translation for retrieval.

## Release

`.github/workflows/release.yml` builds from a tag and publishes three matching
artifacts: a GitHub Release, the data-only Python package on PyPI, and a
data-only image at `docker.io/erzhuoshao/bolero-denmark`. The Docker image is
built `FROM scratch`: it contains no OS, Python, command, or third-party
dependency. Its release tree is rooted at `/opt/bolero-denmark/`.

The image is intended as a transport or multi-stage-build source, not as a
runnable container. For example:

```dockerfile
FROM docker.io/erzhuoshao/bolero-denmark:0.1.0.post8 AS bolero_denmark
FROM scratch
COPY --from=bolero_denmark /opt/bolero-denmark/resources/ /resources/
```

Docker Hub publishing needs one repository Actions secret,
`DOCKERHUB_TOKEN`, containing a write-scoped Docker Hub personal access token.
The Docker Hub username is fixed in the workflow. Each release publishes the
`vVERSION`, `VERSION`, and `latest` image tags. The Git tag must equal `v` plus
the version in `pyproject.toml`.

```bash
# bump version in pyproject.toml, commit, then:
git tag v0.1.0.post8 && git push origin main v0.1.0.post8
```

`packaging/build-release.sh` performs the same local build. No release is made
merely by updating `main`.
