Metadata-Version: 2.4
Name: modchef
Version: 1.0.0
Summary: Cook EasyBuild environments from software ingredients.
Author-email: Nicolas Rapin <kuikuisven@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/nicorap/modchef
Project-URL: Repository, https://github.com/nicorap/modchef
Project-URL: Issues, https://github.com/nicorap/modchef/issues
Project-URL: Documentation, https://github.com/nicorap/modchef/blob/main/docs/user-guide.md
Keywords: easybuild,lmod,hpc,modules,rdf
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: System :: Installation/Setup
Classifier: Topic :: System :: Systems Administration
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: rdflib>=6.0
Requires-Dist: PyYAML>=5.4
Provides-Extra: index
Requires-Dist: easybuild-framework; extra == "index"
Requires-Dist: easybuild-easyblocks; extra == "index"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# modchef

Cook EasyBuild environments from software ingredients.

modchef indexes a cluster's EasyBuild modules into an RDF graph and turns a list
of requested tools and Python/R packages into a minimal, ready-to-run set of
`module load` commands — picking versions that share a compatible toolchain. It
distinguishes modules that are **installed** (loadable now) from those only
**available** in the official EasyBuild collection, and when something isn't
installed it tells you the exact easyconfig to request. When requested tools
can't share a toolchain, it can suggest the install that would unify them.

See **[docs/user-guide.md](docs/user-guide.md)** for the end-user guide.

## Usage

    modchef cook --tools samtools bcftools multiqc --python pandas numpy
    modchef cook recipe.yaml --output load_modules.sh
    modchef search bwa
    modchef explain SAMtools/1.22-GCC-14.3.0
    modchef menu

`cook` emits a `module purge` followed by the `module load` lines for the
fewest compatible toolchain clusters. It also surfaces, as comments and on
stderr: `REQUEST INSTALL` (available in EasyBuild, not installed),
`TO UNIFY` (install X so split tools share one toolchain), and
`NOT IN EASYBUILD` (no easyconfig anywhere). `search` tags each hit
`[installed]` or `[available — not installed]`.

## Daily index (cron, runs as the EasyBuild admin user)

`modchef-index` needs the EasyBuild framework importable, so the cron loads the
EasyBuild module first (the runtime `modchef` module deliberately does *not*
depend on EasyBuild, so `cook`/`search` users stay lean):

    module load EasyBuild/5.2.0
    modchef-index --installed-root /opt/easybuild/software \
                  --robot-repo     /opt/easybuild/easyconfigs \
                  --official-repo  /opt/easybuild/easyconfigs \
                  --output /opt/easybuild/modchef/modchef.ttl

`--installed-root` is the EasyBuild software install tree: modchef indexes the
easyconfig EasyBuild stamped for each *actually installed* module
(`<name>/<version>/easybuild/*.eb`), so the catalog matches what `module avail`
shows, including every bundle's extensions. `--robot-repo` is an easyconfigs
collection used only to resolve installed dependencies. `--official-repo` is the
official EasyBuild collection, indexed as "available, not installed" with the
same full facts (deps + packages): `cook` builds from installed modules first
and, when a tool or package is only available, tells you the easyconfig to ask
support to install. Any easyconfig that fails to parse is reported on stderr
rather than silently dropped. Full-parsing the official collection makes the
daily index noticeably heavier.

The runtime CLI reads the graph from `$MODCHEF_TTL` (set by the module file).

## Deploying the module (EasyBuild, on the HPC as the EasyBuild admin user)

Two easyconfigs ship in `easybuild/`, differing only in toolchain and dependency
versions:

| Easyconfig | Toolchain | Python |
|---|---|---|
| `modchef-1.0.0-GCCcore-12.3.0.eb` | GCCcore/12.3.0 | 3.11.3 |
| `modchef-1.0.0-GCCcore-15.2.0.eb` | GCCcore/15.2.0 | 3.14.2 |

Pick the one whose Python matches the rest of your interactive tooling —
loading modchef beside a module built on a different Python puts two Pythons in
one environment, and which one wins depends on load order.

Both need two site-specific edits before installing:

- **`modextravars = {'MODCHEF_TTL': ...}`** — where the `modchef-index` cron
  writes the daily graph. Users must be able to read it, your EasyBuild admin
  to write it.
- **checksum** — the tarball is built from a checkout, so its bytes differ per
  build and no published value can be correct for you. Generate yours with
  `eb <easyconfig> --inject-checksums`. Once modchef is on PyPI you can instead
  set `source_urls = [PYPI_SOURCE]` and pin the released hash.

The installed module is named after the toolchain, e.g.
`modchef/1.0.0-GCCcore-15.2.0`.

1. Build the source tarball from a checkout and copy it plus your chosen
   easyconfig to the HPC (`<eb>` below is the toolchain suffix you picked):

        python -m build                   # writes dist/modchef-1.0.0.tar.gz
        scp dist/modchef-1.0.0.tar.gz              <admin>@<hpc>:/opt/easybuild/ebfiles_repo/
        scp easybuild/modchef-1.0.0-<eb>.eb        <admin>@<hpc>:/opt/easybuild/ebfiles_repo/

2. EasyBuild caches sources by filename, so overwrite any stale copy in its sourcepath
   (`eb --show-config | grep sourcepath`) before reinstalling:

        cp /opt/easybuild/ebfiles_repo/modchef-1.0.0.tar.gz <sourcepath>/m/modchef/

3. Reinstall over the existing module and smoke-test:

        eb /opt/easybuild/ebfiles_repo/modchef-1.0.0-<eb>.eb --rebuild
        module load modchef/1.0.0-<eb>
        modchef --help
        modchef-index --help | grep official-repo

The version stays `1.0.0` (rebuild in place), so use `--rebuild`. An earlier
SYSTEM-toolchain `modchef/1.0.0`, if any, is a separate module name — remove it
to avoid confusion.
