Metadata-Version: 2.4
Name: methodology-framework
Version: 0.4.1
Summary: Portable process tooling for agent-driven delivery — scripts, playbooks, templates, and specs.
Author: whiteout59
License-Expression: Apache-2.0
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests<3,>=2.31
Requires-Dist: python-frontmatter<2,>=1.1
Requires-Dist: pyyaml<7,>=6.0
Requires-Dist: colorama<1,>=0.4
Provides-Extra: dev
Requires-Dist: pytest<9,>=8.0; extra == "dev"
Requires-Dist: requests-mock<2,>=1.12; extra == "dev"
Requires-Dist: ruff==0.15.14; extra == "dev"
Requires-Dist: mypy<2,>=1.10; extra == "dev"
Requires-Dist: types-requests<3,>=2.31; extra == "dev"
Requires-Dist: types-PyYAML<7,>=6.0; extra == "dev"
Requires-Dist: types-colorama<1,>=0.4; extra == "dev"
Requires-Dist: vcrpy<9,>=7.0; extra == "dev"
Dynamic: license-file

# methodology-framework

Portable process tooling for agent-driven delivery -- scripts, playbooks, templates, and specs.

## Installation

```bash
pip install -e .
```

## Package contents

- `methodology_framework.sync_stories_to_jira` -- one-way repo-to-Jira story sync
- `methodology_framework.build_playbook` -- build a concrete playbook from a parameterized body + bindings
- `methodology_framework.register_playbook_with_devin` -- register a rendered playbook with Devin's API
- `methodology_framework/playbooks/` -- parameterized playbook bodies (package data)
- `methodology_framework/templates/` -- story and process templates (package data)
- `methodology_framework/specs/` -- format specs (package data)
- `methodology_framework.bootstrap_jira` -- verify a Jira project matches the canonical shape (read-only spec-compliance checker)
- `methodology_framework/jira_shapes/` -- canonical Jira shape definitions (YAML, package data)

## Jira Bootstrap (Verify Mode)

The CLI checks whether a Jira project matches the methodology-framework's
canonical shape and reports gaps. It does **not** provision — operators apply
gaps via the Jira UI. This read-only design lets the tool run in CI as a gate.

> **v0.2.0 breaking change:** `--apply` and `--dry-run` were removed.
> The CLI is now a spec-compliance verifier. See `operator-verify-runbook.md`
> for the operator workflow.

### Prerequisites

- Python 3.12+
- `pip install methodology-framework` (or `pip install -e .` from source)
- For `--verify` mode: `JIRA_ADMIN_TOKEN` environment variable set to a Jira
  API token (admin scope is **not** required; a regular user token with read
  access is sufficient), and `JIRA_USER_EMAIL` set to the email of the
  Atlassian account that owns the token

### Usage

```bash
# Verify — query Jira and report spec-compliance gaps (read-only)
export JIRA_ADMIN_TOKEN="<your-token>"
export JIRA_USER_EMAIL="<your-email>"
python -m methodology_framework bootstrap-jira \
  --project-key=MYPROJ \
  --jira-host=myorg.atlassian.net \
  --verify

# JSON output (stable schema — safe for CI parsing)
python -m methodology_framework bootstrap-jira \
  --project-key=MYPROJ \
  --jira-host=myorg.atlassian.net \
  --verify --output=json

# Markdown output
python -m methodology_framework bootstrap-jira \
  --project-key=MYPROJ \
  --jira-host=myorg.atlassian.net \
  --verify --output=markdown

# Export — emit an importable config bundle (YAML); no API calls
python -m methodology_framework bootstrap-jira \
  --project-key=MYPROJ \
  --jira-host=myorg.atlassian.net \
  --export /tmp/jira-bundle.yaml
```

### Exit codes

| Code | Meaning |
|------|---------|
| `0` | All spec items verified OK |
| `1` | One or more items missing (gaps found) |
| `2` | One or more items unverifiable (API limitation; operator must inspect manually) |
| `3` | Error (connectivity, auth, or runtime failure) |

### Flags

| Flag | Required | Description |
|------|----------|-------------|
| `--project-key` | Yes | Jira project key (e.g. `SCRUM`). Substituted for `{{PROJECT_KEY}}` in shape defs. |
| `--jira-host` | Yes | Jira Cloud host (e.g. `myorg.atlassian.net`). No `https://` prefix. |
| `--verify` | No | Query Jira and report spec-compliance gaps (read-only). |
| `--export <path>` | No | Emit importable config bundle at `<path>`. No API calls. |
| `--output` | No | Output format for `--verify`: `text` (default), `json`, `markdown`. |

### Environment variables

| Variable | When needed | Description |
|----------|-------------|-------------|
| `JIRA_ADMIN_TOKEN` | `--verify` mode | Jira API token. Admin scope is **not** required; a regular user token with read access is sufficient. **Never** accepted as a CLI flag. |
| `JIRA_USER_EMAIL` | `--verify` mode | Email of the Atlassian account that owns the API token. Used with the token for HTTP Basic auth. **Never** accepted as a CLI flag. |

### JSON output schema (v0.2.0)

```json
{
  "spec_version": "0.2.0",
  "project_key": "METH",
  "verified_at": "2025-05-31T00:00:00Z",
  "items": [
    {"resource_type": "status", "name": "To Do", "result": "ok", "details": null},
    {"resource_type": "custom_field", "name": "Story File", "result": "missing", "details": "Operator must create via Settings → Custom fields → Create"}
  ],
  "summary": {"total": 14, "ok": 12, "missing": 1, "unverifiable": 1},
  "exit_code": 1
}
```

Any breaking change to this schema requires a MAJOR version bump.

### Shape definitions

The three canonical shape files shipped as package data:

- `jira_shapes/workflow.yaml` — workflow statuses (To Do, Ready for AI agent,
  In Progress, In Review, Waiting, Blocked, Done, Won't do) and transitions
  with actor permissions
- `jira_shapes/custom_fields.yaml` — Story File (URL), Requirement IDs
  (labels), Agent Estimate (number)
- `jira_shapes/automation_rules.yaml` — PR-title-key auto-transition,
  dependency resolution, BLOCKED protection

For full methodology context see the methodology requirements doc § 4.13.3
("Jira Bootstrap CLI").

## Adopter Integration

Adopting projects consume the methodology framework's CI pipelines via
**reusable GitHub Actions workflows**. Instead of copying workflow YAML
into each repo, adopters write a thin caller that references the
framework's workflows by SemVer tag.

> **Version pinning requirement:** adopters MUST pin to a specific tag
> (`@v0.X.Y`), never `@main`. The framework version must be explicit
> in the caller so updates are deliberate, not silent. This matches the
> version-pinning model per methodology requirements § 4.13.

### Required inputs

| Input | Required | Default | Description |
|-------|----------|---------|-------------|
| `project-key` | **Yes** | — | Jira project key (e.g. `METH`, `SCRUM`). No default — sync errors if absent. |
| `jira-base-url` | **Yes** | — | Jira Cloud base URL (e.g. `https://icpipeline.atlassian.net`). |
| `jira-user-email` | **Yes** | — | Jira service account email for HTTP Basic auth. |
| `stories-dir` | No | `docs/stories` | Relative path to the stories directory. |
| `phase-regex` | No | `^phase\d+$` | Regex for phase directory matching. Empty string (`""`) disables phase labeling entirely. |
| `repo-url` | No | *(auto)* | Override repo URL. Normally derived from `GITHUB_SERVER_URL` + `GITHUB_REPOSITORY` (GHA-native). |

### Per-tenant discovery

Sync discovers all tenant-specific Jira IDs at runtime via the REST API,
using canonical **names** as lookup keys. The names come from the framework's
shape definitions (`jira_shapes/*.yaml`) — the same spec the verifier
(`bootstrap-jira --verify`) enforces.

**Custom fields** (discovered via `GET /rest/api/3/field`):

| Name | Purpose |
|------|---------|
| `Requirement IDs` | Multi-value field for REQ-id traceability |
| `Story File` | URL to the canonical story `.md` file in the repo |
| `Agent Estimate` | Numeric estimate in agent-hours |

**Statuses** (discovered via `GET /rest/api/3/project/{key}/statuses`):

| Name | Category |
|------|----------|
| `To Do` | TODO |
| `Ready for AI agent` | TODO |
| `In Progress` | IN_PROGRESS |
| `In Review` | IN_PROGRESS |
| `Waiting` | IN_PROGRESS |
| `Blocked` | IN_PROGRESS |
| `Done` | DONE |
| `Won't do` | DONE |

**Transitions**: Sourced from the workflow spec (`jira_shapes/workflow.yaml`).

**Link types** (discovered via `GET /rest/api/3/issueLinkType`):
`Blocks` (inward: "is blocked by").

**Issue types** (discovered via `GET /rest/api/3/issuetype`):
`Story`, `Sub-task`.

If any required name is missing from the target project, sync fails fast:
```
ERROR: Required custom field 'Story File' not found in Jira project METH;
run 'methodology-framework bootstrap-jira --verify --project-key=METH' to identify the gap.
```

### Story sync workflow

Syncs story `.md` files from the adopter's repo to Jira. Create
`.github/workflows/sync-stories.yml` in the adopter repo:

```yaml
name: Sync stories to Jira

on:
  push:
    branches: [main]
    paths:
      - "docs/stories/**/*.md"
  workflow_dispatch:
    inputs:
      mode:
        description: "Sync mode"
        required: true
        default: "since-ref"
        type: choice
        options:
          - since-ref
          - all

jobs:
  sync:
    permissions:
      contents: write
      pull-requests: read
    uses: whiteout59/methodology-framework/.github/workflows/sync.yml@v0.3.0
    with:
      project_key: MYPROJ
      mode: ${{ github.event.inputs.mode || 'since-ref' }}
      jira_base_url: "https://myorg.atlassian.net"
      jira_user_email: "devin-sync@myorg.atlassian.net"
      # stories_dir: "docs/stories"    # default
      # phase_regex: "^phase\\d+$"     # default; set "" to disable
    secrets:
      JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}
```

The `mode` input defaults to `since-ref` for routine push-triggered
runs (preserves the 250-story scale guard). Pass `mode: all` explicitly
for nightly or `workflow_dispatch` full-corpus syncs.

> **Secrets forwarding:** the caller's `secrets:` block must explicitly
> map each secret the reusable workflow declares. Secrets are NOT
> inherited by default in reusable workflows. Using `secrets: inherit`
> works but is brittle — prefer explicit mapping.

#### Required caller permissions

The reusable sync workflow declares `permissions: contents: write`
internally (for `jira_key` + `agent_acus` writeback commits). GitHub
Actions requires the **caller** to grant at least the same permissions
at the job level — otherwise the workflow fails at the `uses:`
resolution step with:

```
The workflow is requesting 'contents: write', but is only allowed 'contents: read'.
```

Add a job-level `permissions:` block to your caller:

```yaml
jobs:
  sync:
    permissions:
      contents: write
      pull-requests: read
    uses: whiteout59/methodology-framework/.github/workflows/sync.yml@v0.2.0
    # ... with: / secrets: as above
```

> **Job-scoped, not workflow-scoped.** Keep `permissions:` on the job,
> not the top-level `on:` — if future jobs are added to the same file,
> they should not inherit write access they don't need.

### Playbook build + register workflow

Builds a concrete playbook from a parameterized body + bindings file,
then registers it with Devin's API. Create
`.github/workflows/build-and-register.yml` in the adopter repo:

```yaml
name: Build and register playbook

on:
  push:
    branches: [main]
    paths:
      - "methodology/playbooks/*.body.md"
      - "docs/jira-pickup-config.md"
  workflow_dispatch:

jobs:
  build-and-register:
    uses: whiteout59/methodology-framework/.github/workflows/build-and-register.yml@v0.1.0
    with:
      playbook_body_path: "methodology/playbooks/scrum-router.body.md"
      bindings_path: "docs/jira-pickup-config.md"
    secrets:
      DEVIN_API_TOKEN: ${{ secrets.DEVIN_API_TOKEN }}
```

> **Nesting limit:** GitHub allows reusable workflow nesting up to
> 4 levels deep. If your repo wraps these workflows in another
> caller layer, verify the total nesting depth stays within limits.

## How this repo evolves

The methodology-framework repo itself uses the same sync workflow it
ships to adopters. Framework-side evolution stories (new specs, CLI
features, workflow improvements) are drafted as `.md` files under
`docs/stories/` and synced to the **METH** Jira project via
[`.github/workflows/sync-stories.yml`](.github/workflows/sync-stories.yml).
Adopter-side stories (product-specific work) file in each adopter's own
Jira project (e.g. SCRUM for centralized-pipeline-ui).

## PyPI Publishing

The package is published to PyPI automatically via OIDC Trusted Publishers
when a SemVer tag is pushed:

```bash
git tag v0.1.0
git push origin v0.1.0
```

The `pypi-release.yml` workflow builds and publishes using
`pypa/gh-action-pypi-publish` in OIDC mode -- no `PYPI_API_TOKEN` secret
is needed. The job uses the `pypi` GitHub environment for protection rules.

**Operator setup (one-time):** bind the PyPI project `methodology-framework`
to the GitHub repo `whiteout59/methodology-framework`, workflow
`pypi-release.yml`, environment `pypi` at
[PyPI Trusted Publishers](https://pypi.org/manage/account/publishing/).

## Release lifecycle

Version bumps in `pyproject.toml` drive the entire release cycle
automatically. The operator's only decision surface is PR review.

### Primary flow (chained via `workflow_call`)

1. A PR bumps the `version` field in `pyproject.toml` (e.g. `"0.2.0"` → `"0.2.1"`).
2. Reviewer approves and merges the PR to `main`.
3. `auto-tag.yml` fires on the `pyproject.toml` change, extracts the version,
   validates it as stable SemVer (`^[0-9]+\.[0-9]+\.[0-9]+$`), and creates an
   annotated tag `v0.2.1`.
4. `auto-tag.yml`'s `publish` job invokes `pypi-release.yml` via `workflow_call`,
   passing the version as an input.
5. `pypi-release.yml` checks out the tag, builds the package, and publishes to
   PyPI via OIDC Trusted Publishers.

**Operator surface = PR review only.** No manual `git tag` step, no manual
publish trigger. The `workflow_call` chain bypasses GitHub's recursion guard
that prevents `GITHUB_TOKEN`-authored tag pushes from triggering downstream
workflows via `on: push: tags:`.

### Fallback (operator-pushed tag)

If `auto-tag.yml` doesn't fire (workflow disabled, branch protection
blocking the tag push, GitHub Actions outage, etc.), or the operator needs
to re-publish a yanked version, the manual fallback still works:

```bash
git tag -a vX.Y.Z -m "Release vX.Y.Z"
git push origin vX.Y.Z
```

An operator-pushed tag directly triggers `pypi-release.yml` via the
preserved `on: push: tags: ['v[0-9]+.[0-9]+.[0-9]+']` trigger. This
path is independent of `auto-tag.yml` and requires no `workflow_call`.

### Pre-release versions

The `auto-tag.yml` workflow only tags **stable** SemVer versions. Versions
containing `-rc`, `-beta`, `-alpha`, `.dev`, or `.pre` suffixes (or any
string not matching `^[0-9]+\.[0-9]+\.[0-9]+$`) are logged and skipped.
Pre-release publishing is out of scope; if needed, a separate workflow
would handle pre-release tag patterns.

## Cost tracking via `agent_acus`

The methodology's post-execution notes include an `agent_acus` field that
records per-story compute cost. Since agents cannot self-report ACU
consumption mid-session, a post-merge populator workflow backfills the
value from the Devin API after the session completes.

**Adopter wiring (3 steps):**

1. Copy the template caller into your repo:
   ```bash
   cp "$(python -c "import methodology_framework; import pathlib; \
     print(pathlib.Path(methodology_framework.__file__).parent / \
     'templates/github_workflows/populate-story-acus-caller.yml')")" \
     .github/workflows/populate-story-acus.yml
   ```
2. Set the `DEVIN_API_TOKEN` repo secret (Settings > Secrets and
   variables > Actions > New repository secret).
3. Pin the framework version in the caller's `uses:` line.

The caller fires on merged PRs that touch story files, invokes the
reusable `populate-story-acus.yml` workflow, and opens a follow-on PR
with the populated ACU values.

## Contributing

Framework evolution stories — new specs, CLI features, workflow improvements —
are filed as Markdown story files under [`docs/stories/`](docs/stories/).
See the [directory README](docs/stories/README.md) for the filing convention
(slug-named directories, frontmatter shape, `depends_on` hygiene). Stories
sync to the **METH** Jira project via the same sync workflow the framework
ships to adopters.

For the canonical story template and format spec, see
[`whiteout59/centralized-pipeline-ui/methodology/templates/story.md`](https://github.com/whiteout59/centralized-pipeline-ui/blob/main/methodology/templates/story.md)
and
[`whiteout59/centralized-pipeline-ui/methodology/devin-story-format.md`](https://github.com/whiteout59/centralized-pipeline-ui/blob/main/methodology/devin-story-format.md).

## Development

```bash
pip install -e ".[dev]"
pytest tests/ -v
ruff check src/methodology_framework
ruff format --check src/methodology_framework
mypy --strict src/methodology_framework
```

### Shape-def expansion (v0.1.2+)

`workflow.yaml` now includes a `statusCategory` field on each status
(`TODO`, `IN_PROGRESS`, or `DONE`). The `--apply` handler validates
this field at load time and exits with a clear error if any status is
missing it.

`custom_fields.yaml` uses canonical shorthand types (`text`, `string`,
`multi-value`, `numeric`) that are mapped to fully-qualified Jira
identifiers at apply time. Fields may also specify a fully-qualified
type directly (any value containing `:` is passed through unchanged).

### Recording new VCR cassettes

Integration tests under `tests/integration/` replay pre-recorded VCR
cassettes in CI (no network access required). To record fresh cassettes
against a real Jira instance:

```bash
export JIRA_USER_EMAIL="<your-email>"
export JIRA_ADMIN_TOKEN="<your-api-token>"
PYTEST_VCR_MODE=record pytest tests/integration/ -v
```

Cassettes are stored at `tests/fixtures/vcr/*.yaml`. The VCR config in
`tests/integration/conftest.py` automatically strips `Authorization`
headers from recorded interactions. **Never commit a cassette with a
real token in any header.**

After recording, verify no credentials leaked:

```bash
grep -i 'authorization' tests/fixtures/vcr/*.yaml
# Expected: no output
```

## License

Apache-2.0
