Metadata-Version: 2.1
Name: wexample-wex-addon-master
Version: 11.2.0
Summary: Extends wex to fan out commands across all managed apps and manage DNS zones against Cloudflare, Route53, OVH, and Gandi
Author-Email: weeger <contact@wexample.com>
License: MIT
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Requires-Dist: wexample-wex-addon-ai>=13.0.0
Requires-Dist: wexample-wex-addon-app>=30.0.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Provides-Extra: dns
Requires-Dist: octodns>=1.0.0; extra == "dns"
Requires-Dist: octodns-ovh>=0.0.4; extra == "dns"
Requires-Dist: octodns-cloudflare>=1.1.0; extra == "dns"
Requires-Dist: pyyaml>=6.0; extra == "dns"
Description-Content-Type: text/markdown

# wex_addon_master

Version: 11.2.0

`wex-addon-master` is a wex addon for teams that coordinate multiple wex-managed apps from a single control repository. It introduces a `master` workdir type that reads a `project.yml` listing apps by path or glob, then exposes commands to fan any wex command across all of them — sequentially or in parallel, scoped to a named stack if needed — and to manage DNS zones as local YAML files kept in sync with Cloudflare, Route53, OVH, or Gandi via octodns.

## Table of Contents

- [Installation](#installation)
- [Quickstart](#quickstart)
- [Fan out a command](#fan-out-a-command)
- [Local variable overrides](#local-variable-overrides)
- [Tests](#tests)
- [Architecture](#architecture)
- [Integration in the Suite](#integration-in-the-suite)
- [Dependencies](#dependencies)
- [Versioning & Compatibility Policy](#versioning--compatibility-policy)
- [License](#license)
- [About us](#about-us)
- [Known Limitations & Roadmap](#known-limitations--roadmap)
- [Status & Compatibility](#status--compatibility)
- [Useful Links](#useful-links)
- [Migration Notes](#migration-notes)

## Installation

```bash
pip install wexample-wex-addon-master
```

Requires Python >=3.10.

## Quickstart

Install the package:

```bash
pip install wexample-wex-addon-master
```

A master project is a directory that wex treats as a control tower for a set of apps. It needs two files.

**`project.yml`** — declare which apps the master manages:

```yaml
project:
  name: my-master

apps:
  - ../app-one
  - ../apps/*
```

Each entry is a path or glob relative to the master directory. A path is included only when it contains `.wex/config.yml`.

**`.wex/python/app_manager/app_workdir.py`** — tell wex to use the `MasterWorkdir` type (defined in src/wexample_wex_addon_master/workdir/master_workdir.py):

```python
from wexample_wex_addon_master.workdir.master_workdir import MasterWorkdir

class AppWorkdir(MasterWorkdir):
    pass
```

With those two files in place, run from the master directory:

```bash
wex master::info/show
```

wex prints a table of every resolved app — name, version, wex version, and git status.

## Fan out a command

`master::apps/run` calls any wex command on every app in sequence:

```bash
wex master::apps/run -c app::info/show
```

Pass `-a` / `--async_mode` to run all apps in parallel (output is grouped at the end). Pass `-s <stack>` to restrict execution to the apps belonging to a named stack.

## Local variable overrides

Create `master.local.yml` next to `project.yml` to define per-machine values. References in `project.yml` are expanded at load time:

```yaml
# master.local.yml
LOCAL: /home/user/projects
```

```yaml
# project.yml
apps:
  - ${LOCAL}/app-one
  - ${LOCAL}/apps/*
```

`master.local.yml` is never committed; add it to `.gitignore`.

## Tests

This project uses `pytest` for testing and `pytest-cov` for code coverage analysis.

### Installation

First, install the required testing dependencies:
```bash
.venv/bin/python -m pip install pytest pytest-cov
```

### Basic Usage

Run all tests with coverage:
```bash
.venv/bin/python -m pytest --cov --cov-report=html
```

### Common Commands
```bash
# Run tests with coverage for a specific module
.venv/bin/python -m pytest --cov=your_module

# Show which lines are not covered
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing

# Generate an HTML coverage report
.venv/bin/python -m pytest --cov=your_module --cov-report=html

# Combine terminal and HTML reports
.venv/bin/python -m pytest --cov=your_module --cov-report=term-missing --cov-report=html

# Run specific test file with coverage
.venv/bin/python -m pytest tests/test_file.py --cov=your_module --cov-report=term-missing
```

### Viewing HTML Reports

After generating an HTML report, open `htmlcov/index.html` in your browser to view detailed line-by-line coverage information.

### Coverage Threshold

To enforce a minimum coverage percentage:
```bash
.venv/bin/python -m pytest --cov=your_module --cov-fail-under=80
```

This will cause the test suite to fail if coverage drops below 80%.

## Architecture

The addon has four moving parts: an addon manager that plugs into the wex kernel, a middleware that resolves the workdir before every command, the workdir itself which is the object commands operate on, and a set of pure helper modules the workdir delegates to.

### Entry point

src/wexample_wex_addon_master/master_addon_manager.py (`MasterAddonManager`) extends `AppAddonManager`. Its two overrides wire the addon into the kernel:

- `get_middlewares_classes()` appends `MasterMiddleware` to the parent list, so every master command passes through it.
- `get_workdir_types()` registers the string `"master"` → `MasterWorkdir`, making `master` a legal value for the `type:` key in an app's `.wex/config.yml`.
- `get_package_module()` returns the `wexample_wex_addon_master` package object, which lets the kernel auto-discover all commands under `src/wexample_wex_addon_master/commands/`.

### Middleware

src/wexample_wex_addon_master/middleware/master_middleware.py (`MasterMiddleware`) runs before the body of every command that carries `@middleware(middleware=MasterMiddleware)`. It overrides `_create_app_workdir()`:

```python
app_workdir = super()._create_app_workdir(request=request, app_path=app_path)
if not isinstance(app_workdir, MasterWorkdir):
    raise RuntimeError(...)
return app_workdir
```

The parent call (from `AppMiddleware`) reads the current directory's `.wex/config.yml`, looks up the `type:` field in `get_workdir_types()`, and constructs the matching workdir. The `isinstance` check is the enforcer: if the project's `.wex/python/app_manager/app_workdir.py` does not subclass `MasterWorkdir`, the middleware stops the call before the command body runs and prints the fix.

### Workdir

src/wexample_wex_addon_master/workdir/master_workdir.py (`MasterWorkdir`) extends both `ManagedWorkdir` and `WithAiWorkdirMixin`. It is the object commands receive as `app_workdir`. All business logic is delegated to helper modules — the workdir provides the glue.

**App resolution** — `get_apps_paths()` and `get_apps_tree()` delegate to src/wexample_wex_addon_master/helper/apps.py. `get_all_apps_paths()` flattens `get_apps_tree()` into a dedup'd list:

```python
return list(dict.fromkeys(p for _, p in self.get_apps_tree()))
```

`get_code_paths()` overrides the parent by returning `get_all_apps_paths()`, so AI and code-scanning features follow the same app set.

**Fan-out** — `apps_execute_manager()` iterates resolved paths and calls `ManagedWorkdir.manager_run_from_path()` on each. Sequential by default; with `async_mode=True` it uses `parallel_map` from `wexample_helpers`. Captured outputs are printed grouped after all apps finish. `fail_fast=True` re-raises the first exception; `fail_fast=False` collects failures and reports them at the end.

**Manager bootstrap** — `apps_ensure_manager_bin()` creates a temporary `ManagedWorkdir` for each resolved app (with `configure=False` to avoid full initialisation) and calls `ensure_manager_bin()` on it.

**DNS** — `get_dns_providers()`, `get_dns_zones()`, `get_dns_zones_dir()`, and `scan_app_domains()` read `project.yml` + `master.local.yml` and delegate into the DNS helper module.

**Stacks** — `get_stacks()`, `get_stack()`, `get_stack_apps()`, `get_stack_groups()`, and `get_stack_missing()` delegate to src/wexample_wex_addon_master/helper/stacks.py.

### Configuration files

src/wexample_wex_addon_master/file/project_yaml_file.py (`ProjectYamlFile`) wraps `project.yml` — the committed file that declares apps, stacks, DNS zones, and provider types.

src/wexample_wex_addon_master/file/master_local_yaml_file.py (`MasterLocalYamlFile`) wraps `master.local.yml` — the per-machine file that holds values that vary by developer or server (local paths, credentials). It is never committed.

`load_project_data()` in src/wexample_wex_addon_master/helper/apps.py reads both files, then calls `dict_interpolate(data, env_vars)` to expand `${VAR}` placeholders from `master.local.yml` into `project.yml`. All callers receive the fully resolved dict and never need to handle the two-file split themselves.

`load_local_vars()` first expands `master.local.yml` against itself, so a variable like `LOCAL: /home/user` can appear inside the same file as `MASTER_APP_PATHS: ${LOCAL}/apps/*` and be resolved in one pass.

### App resolution

`get_apps_paths()` reads `project.yml`'s `apps:` list. Each entry is a path or glob string relative to the master directory. A match is kept only when it contains `.wex/config.yml`. Entries may also be dicts with a `path:` key; both forms produce the same result.

`get_apps_tree()` adds recursive suite expansion: when an app's `.wex/config.yml` declares `package_suite.location`, those glob patterns are resolved relative to that app's path and their matches become child entries. The returned `list[tuple[int, Path]]` (depth, path) powers the indented tree in `master::info/show`. Cycles are guarded with a `visited` set.

`read_app_summary()` reads an app's `.wex/config.yml` and returns a dict with `name`, `version`, `wex_version`, and `tags` — the four fields used in every table command.

### Stacks

src/wexample_wex_addon_master/helper/stacks.py resolves a stack declared under `stacks:` in `project.yml` into concrete app paths through three mechanisms:

- **`apps:`** — explicit path or glob entries resolved the same way as top-level apps.
- **`groups:`** — a named dict of subsets; each group can have its own `apps:` and `include_tags:`.
- **`include_tags:`** — selects apps from the full resolved list whose `.wex/config.yml` carries any of the listed tag strings (read via `read_app_summary()`).

`get_stack_groups()` returns `list[tuple[str, list[Path]]]` — top-level apps appear under the group name `"_"`. `get_stack_missing()` returns raw entries declared in the stack that did not resolve to any known app path; tag-based includes are not reported as missing because they are matchers, not references.

### DNS subsystem

The DNS subsystem lives entirely under `src/wexample_wex_addon_master/helper/dns/`. Its public surface is re-exported by src/wexample_wex_addon_master/helper/dns/__init__.py.

**Loading** — src/wexample_wex_addon_master/helper/dns/loader.py builds the runtime state from the merged config. `load_providers()` reads provider declarations from `project.yml` (`type:`) and credentials from `master.local.yml` (`dns.providers.<name>`), calls `build_provider()` for each, and raises `ValueError` for any provider that lacks credentials. `load_zones()` reads `dns.zones` from `project.yml`, resolves each `provider:` reference to a `DnsProvider` instance, and returns `dict[str, ZoneBinding]`.

**Provider abstraction** — src/wexample_wex_addon_master/helper/dns/providers/base.py defines `DnsProvider`, an abstract class with one required method:

```python
def to_octodns_config(self) -> dict[str, Any]: ...
```

src/wexample_wex_addon_master/helper/dns/providers/__init__.py registers four concrete implementations under string keys: `cloudflare`, `route53`, `ovh`, and `gandi`. `build_provider(type_name, credentials)` selects the class from `PROVIDERS` and calls `cls(**credentials)`. Adding a new provider means implementing `DnsProvider` and adding one entry to `PROVIDERS`.

**Octodns config** — src/wexample_wex_addon_master/helper/dns/octodns_config.py builds the YAML dict that `octodns-sync` expects via `--config-file`. A pull config sets the remote provider as `sources` and `octodns.provider.yaml.YamlProvider` (pointing at the local `dns/zones/` directory) as `targets`. A push config inverts source and target.

**Runner** — src/wexample_wex_addon_master/helper/dns/runner.py writes the config dict to a temp file and spawns octodns as a subprocess:

```python
subprocess.run([sys.executable, "-m", "octodns.cmds.sync", *args, "--config-file", config_path], check=True)
```

Using `python -m octodns.cmds.sync` rather than the `octodns-sync` console script means the call works regardless of whether the venv's `bin/` is on `PATH`. The temp file is removed in a `finally` block whether or not octodns succeeds.

**Service** — src/wexample_wex_addon_master/helper/dns/service.py exposes two high-level functions: `pull_zone()` (provider → local YAML, passes `--force` when `force=True` to bypass octodns' 30 % change threshold) and `push_zone()` (local YAML → provider, dry-run by default, live when `apply=True`).

**Zone YAML** — src/wexample_wex_addon_master/file/dns_zone_yaml_file.py (`DnsZoneYamlFile`) wraps `dns/zones/<zone>.yaml` files in the octodns format. src/wexample_wex_addon_master/helper/dns/zone_yaml.py provides `load_zone()`, `save_zone()`, `add_record()`, and `remove_record()` to read and mutate these files without calling octodns. `add_record()` refuses to overwrite an existing record of the same type — callers must call `remove_record()` first.

**App domain scanning** — src/wexample_wex_addon_master/helper/dns/app_domains.py walks every managed app's `.wex/config.yml` and per-env configs under `.wex/env/*/config.yml`, collecting `domain:` / `domains:` declarations as `AppDomain(app_path, domain, env)`. Three audit functions compare the scanned domains against the local zone YAMLs:

- `find_missing_records()` — domains declared by apps that have no entry in the zone YAML.
- `find_zone_records_without_app()` — FQDNs present in the zone YAML but not referenced by any app (NS/SOA and apex records are excluded as zone plumbing).
- `find_app_domains_without_zone()` — app domains that match none of the managed zones.

### Snapshot and Git helpers

src/wexample_wex_addon_master/helper/snapshot.py defines the `AppSnapshot` dataclass (name, version, wex\_version, git\_status, path, error, tags) and two collectors:

- `collect_local_snapshot()` calls `read_app_summary()` and `get_git_status()` — both safe to call in parallel via `parallel_map`.
- `collect_remote_snapshot()` opens one SSH connection, runs a compound command that cats `.wex/config.yml` then runs `git status` separated by a sentinel string (`===WEX_SNAPSHOT_SEP===`), and parses both halves from the single response. SSH multiplexing (`ControlMaster=auto`, `ControlPath=/tmp/wex-ssh-%C`, `ControlPersist=30s`) reuses a single TCP connection across parallel calls to the same host, keeping the total number of concurrent SSH sessions below `sshd`'s `MaxStartups`. `sudo -n` is prepended to each remote command to handle production directories owned by a different system user.

src/wexample_wex_addon_master/helper/git.py provides `get_git_status()` (runs `git status --porcelain=v1 --branch` locally) and `parse_git_status_output()`, which compresses the output into one of: `clean`, `↑N`, `*`, `↑N *`, `no upstream`, or `—`.

### Commands

Every command carries `@middleware(middleware=MasterMiddleware)`. The middleware runs first and delivers a `MasterWorkdir` as the `app_workdir` parameter before the command body executes.

**`apps` group**

- src/wexample_wex_addon_master/commands/apps/run.py (`master::apps/run`) — fans any wex command across all resolved apps. Options: `--command` (required), `--arguments`, `--continue_on_error`, `--async_mode`, `--stack`.
- src/wexample_wex_addon_master/commands/apps/setup.py (`master::apps/setup`) — ensures every app has a functional `.wex/bin/app-manager`, replacing broken symlinks or stale content.

**`info` group**

- src/wexample_wex_addon_master/commands/info/show.py (`master::info/show`) — prints the master's metadata then a table of all apps with local version, wex version, and git status. With `--remotes <env>` it also collects remote snapshots via SSH (parallel, capped at 4 workers).

**`stack` group**

- src/wexample_wex_addon_master/commands/stack/list.py (`master::stack/list`) — table of all stacks with app count and missing-reference count.
- src/wexample_wex_addon_master/commands/stack/show.py (`master::stack/show`) — detail view of one stack: properties frame, apps table grouped by group, and unresolved entries. At medium verbosity it adds per-app wex version and git status (fetched in parallel via `parallel_map`).

**`dns` group**

- src/wexample_wex_addon_master/commands/dns/pull.py — downloads records from the provider into `dns/zones/<zone>.yaml`.
- src/wexample_wex_addon_master/commands/dns/push.py — uploads local zone YAMLs to the provider (dry-run unless `--apply`).
- src/wexample_wex_addon_master/commands/dns/diff.py — runs `push_zone()` with `apply=False` to show what octodns would change without applying anything.
- src/wexample_wex_addon_master/commands/dns/check.py — DNS audit across all apps and zones: missing records, orphan records, and app domains outside managed zones. Accepts `--env` to restrict to one env scope.
- src/wexample_wex_addon_master/commands/dns/add.py — adds one record to a local zone YAML (does not push to provider).
- src/wexample_wex_addon_master/commands/dns/remove.py — removes a record (or all records on a subdomain) from a local zone YAML.
- src/wexample_wex_addon_master/commands/dns/zones.py — table of declared zones with provider name, type, local record count, and how many managed apps use each zone.
- src/wexample_wex_addon_master/commands/dns/domains.py — table of all domains found across managed apps, with the env and matched zone.
- src/wexample_wex_addon_master/commands/dns/sync_app_domain.py — reads an app's `remotes[0].host` and the `domain:` from its env config, matches the domain to a zone, then calls `add_record()` on the local zone YAML. The record is not pushed; the command prints the next steps.

### Call path example

`wex master::apps/run -c app::info/show` walks through these layers:

1. The wex kernel loads `MasterAddonManager`, which registers `MasterMiddleware` and the `master` workdir type.
2. `MasterMiddleware._create_app_workdir()` reads `.wex/config.yml` in the current directory, constructs a `MasterWorkdir`, and asserts `isinstance(app_workdir, MasterWorkdir)`.
3. `master__apps__run()` in src/wexample_wex_addon_master/commands/apps/run.py receives the verified `app_workdir` and calls `app_workdir.apps_execute_manager(command="app::info/show")`.
4. `apps_execute_manager()` calls `get_all_apps_paths()` → `get_apps_tree()` → `get_apps_paths()` → `load_project_data()`, which reads and interpolates `project.yml` against `master.local.yml`, expands globs, and returns a dedup'd list of app paths.
5. For each path, `ManagedWorkdir.manager_run_from_path(cmd=["app::info/show"], path=app_path)` spawns that app's own wex manager binary as a subprocess.

## Integration in the Suite

This package is part of the Wexample Suite — a collection of high-quality, modular tools designed to work seamlessly together across multiple languages and environments.

### Related Packages

The suite includes packages for configuration management, file handling, prompts, and more. Each package can be used independently or as part of the integrated suite.

Visit the [Wexample Suite documentation](https://docs.wexample.com) for the complete package ecosystem.

## Dependencies

- wexample-wex-addon-ai: >=13.0.0
- wexample-wex-addon-app: >=30.0.0

## Versioning & Compatibility Policy

Wexample packages follow **Semantic Versioning** (SemVer):

- **MAJOR**: Breaking changes
- **MINOR**: New features, backward compatible
- **PATCH**: Bug fixes, backward compatible

We maintain backward compatibility within major versions and provide clear migration guides for breaking changes.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

Free to use in both personal and commercial projects.

## About us

[Wexample](https://wexample.com) stands as a cornerstone of the digital ecosystem — a collective of seasoned engineers, researchers, and creators driven by a relentless pursuit of technological excellence. More than a media platform, it has grown into a vibrant community where innovation meets craftsmanship, and where every line of code reflects a commitment to clarity, durability, and shared intelligence.

This packages suite embodies this spirit. Trusted by professionals and enthusiasts alike, it delivers a consistent, high-quality foundation for modern development — open, elegant, and battle-tested. Its reputation is built on years of collaboration, refinement, and rigorous attention to detail, making it a natural choice for those who demand both robustness and beauty in their tools.

Wexample cultivates a culture of mastery. Each package, each contribution carries the mark of a community that values precision, ethics, and innovation — a community proud to shape the future of digital craftsmanship.

## Known Limitations & Roadmap

Current limitations and planned features are tracked in the GitHub issues.

See the [project roadmap](https://github.com/wexample/python-wex_addon_master/issues) for upcoming features and improvements.

## Status & Compatibility

**Maturity**: Production-ready

**Python Support**: >=3.10

**OS Support**: Linux, macOS, Windows

**Status**: Actively maintained

## Useful Links

- **Homepage**: https://github.com/wexample/python-wex-addon-master
- **Documentation**: [docs.wexample.com](https://docs.wexample.com)
- **Issue Tracker**: https://github.com/wexample/python-wex-addon-master/issues
- **Discussions**: https://github.com/wexample/python-wex-addon-master/discussions
- **PyPI**: [pypi.org/project/wexample-wex-addon-master](https://pypi.org/project/wexample-wex-addon-master/)

## Migration Notes

When upgrading between major versions, refer to the migration guides in the documentation.

Breaking changes are clearly documented with upgrade paths and examples.
