Metadata-Version: 2.4
Name: content-plugin-finder
Version: 0.1.0
Summary: Find Ansible modules, filters, and lookups used in scenario/target directories
Author: Ansible Community
License-Expression: MIT
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: LICENSE.ansible-content-capture
License-File: NOTICE
Requires-Dist: PyYAML
Requires-Dist: ruamel.yaml
Requires-Dist: jsonpickle
Requires-Dist: joblib
Requires-Dist: requests
Requires-Dist: filelock
Requires-Dist: rapidfuzz
Requires-Dist: tabulate
Requires-Dist: gitdb
Requires-Dist: smmap
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.9.0; extra == "dev"
Dynamic: license-file

# content-plugin-finder

> **Unsupported prototype.** This repository is an early experiment under
> active development. APIs, CLI flags, and behavior may change without notice.
> There is no support commitment, stability guarantee, or production readiness
> claim. Use at your own risk.

Find Ansible **modules**, **filters**, and **lookups** used in directories such as Molecule scenarios or ansible-test integration targets, and map git changes to molecule scenarios / ansible-test integration targets that should run.

Uses a pluggable **crawler subsystem**. v1 ships three crawlers; more plugin kinds can be added later without changing the orchestrator.

License: **MIT** (see [LICENSE](LICENSE)).

This project vendors [ansible-content-capture](https://github.com/ansible/ansible-content-capture) (Apache-2.0) under `src/ansible_content_capture/`. See [LICENSE.ansible-content-capture](LICENSE.ansible-content-capture) and [NOTICE](NOTICE).

Vendored note: `loader.get_scanner_version()` was patched to use `importlib.metadata` instead of removed `pkg_resources` (setuptools ≥83).

![Impact view: changed file → plugin FQCNs → molecule scenarios and integration targets](docs/images/visualizer-impact.png)

## Install

```bash
pip install -e ".[dev]"
```

## Visualizer

A zero-build dual-mode graph UI lives in [`viz/`](viz/) (d3 v7 from CDN).

### 1. Export JSON

From a collection checkout:

```bash
# Plugin dependency DAG (full graph, or focus one FQCN)
content-plugin-finder --collection-graph . --format json > graph.json
content-plugin-finder --collection-graph . --plugin ansible.platform.application --format json > plugin.json

# Impact flow (needs a git base, or pipe paths)
content-plugin-finder --impact . --base origin/main --format json > impact.json
echo 'plugins/action/base_action.py' \
  | content-plugin-finder --impact . --from-stdin --format json > impact.json
```

### 2. Open the UI

```bash
# optional: from the repo root
xdg-open viz/index.html   # or open viz/index.html in your browser
```

1. Open [`viz/index.html`](viz/index.html) (double-click or `file://` is fine).
2. Click **Choose File** / **Load JSON** and select `graph.json` or `impact.json`.
3. Mode is auto-detected from the JSON shape; you can override with the **Mode** control.
4. **Plugin dependencies**: pick a **Focus plugin** FQCN (recommended). Click a node for path + owning plugins.
5. **Impact**: explore changed files → plugins → molecule / integration roots. Counts appear in the toolbar.

| Mode | JSON source | View |
|------|-------------|------|
| Plugin dependencies | `--collection-graph --format json` (full or `--plugin FQCN`) | Focus one FQCN (recommended) or capped “All”; nodes are Python files |
| Impact | `--impact --format json` | Changed files → plugin FQCNs → molecule / integration roots |

No npm build. The file picker works with `file://` (browsers block `fetch` of local paths).

## Usage

```bash
content-plugin-finder path/to/scenario [path/to/target ...]
content-plugin-finder --format json --by-directory path/to/scenario
content-plugin-finder --types module,filter path/to/scenario
content-plugin-finder --list-crawlers

# Discover Molecule scenarios + integration targets under a collection/repo
content-plugin-finder --parent ../ansible.platform --depth 4 --list-roots
content-plugin-finder --parent ../ansible.platform --depth 4
```

`--parent` walks for:

- directories containing `molecule.yml`
- `tests/integration/targets/<name>/`

`--depth` is the max relative depth under `--parent` (default `4`).
For a collection root, depth `3` reaches `extensions/molecule/<scenario>`;
depth `4` also reaches `tests/integration/targets/<target>`.

## Crawler subsystem

| Crawler | Kind | Source |
|---------|------|--------|
| `module` | module | ansible-content-capture task/module trees |
| `filter` | filter | Jinja pipes in YAML scalars |
| `lookup` | lookup | ACC lookup/query tasks + Jinja `lookup()` / `query()` |

Modules and action plugins are reported together as modules (not distinguishable from content alone).

## Collection import graph

Build a Python import tree and plugin **name map** for a collection:

```bash
content-plugin-finder --collection-graph ../ansible.platform
content-plugin-finder --collection-graph ../ansible.platform --plugin application
content-plugin-finder --collection-graph ../ansible.platform --plugin ansible.platform.application --format json
```

Output is **per plugin** (FQCN): entry files (module + action when both exist) plus the
**transitive** in-collection dependency closure (`depends_on_files`). Direct
file→import edges remain available in the JSON under `plugins` / `imports`.

Plugin identity is always the **FQCN** `{namespace}.{name}.{plugin}` from `galaxy.yml`
plus the short plugin name (e.g. `ansible.platform.application`).
`--plugin application` is accepted as a short alias and resolved to that FQCN.

| Kind | Short name source (then prefixed with collection FQCN) |
|------|--------------------------------------------------------|
| module | `DOCUMENTATION` `module:` / file stem |
| action | `ActionModule.MODULE_NAME`, else matching `plugins/modules/<stem>.py`, else stem |
| filter | keys from `FilterModule.filters()` (one file may expose many filters) |
| lookup | `DOCUMENTATION` `name:` / file stem |

## Impact (git changes → tests to run)

```bash
# Local / CI with explicit base (PR target SHA or branch)
content-plugin-finder --impact . --parent . --base origin/main
content-plugin-finder --impact . --base "$PR_BASE_SHA" --head "$PR_HEAD_SHA"

# Pipe changed paths (CI escape hatch)
git diff --name-only "$BASE" "$HEAD" | content-plugin-finder --impact . --from-stdin

# Emit only names for a matrix
content-plugin-finder --impact . --base origin/main --emit molecule --names-only
content-plugin-finder --impact . --base origin/main --emit integration --names-only
content-plugin-finder --impact . --base origin/main --format json
```

Text output (default):

```text
molecule	extensions/molecule/application_mock
integration	tests/integration/targets/applications_test
```

In CI, pass the PR **base SHA** (or fetch the target branch). Bare `git diff` without a base is not reliable on shallow/detached checkouts. Three-dot `base...head` is the default; use `--two-dot` for `base head`.

```text
git changed files
        │
        ├─ under scenario/target     → that root
        │     (path layout works even if discovery depth missed the root)
        ├─ shared molecule file      → all discovered molecule scenarios
        │     (e.g. extensions/molecule/requirements.yml)
        └─ collection .py            → file_to_plugins → FQCNs
                │
                └─ content index → molecule scenarios / integration targets
```

Direct path matches cover scenario and integration-target edits themselves, not only
plugin→content reverse mapping. Shared files under a `molecule/` directory (outside
any scenario) select every discovered molecule scenario. Deleted paths are included
in the git diff (`ACMRD`).

### Running Molecule or ansible-test from the list

Emit leaf names, then drive the tools your collection already uses:

```bash
content-plugin-finder --impact . --base origin/main \
  --emit molecule --names-only > molecule.txt
content-plugin-finder --impact . --base origin/main \
  --emit integration --names-only > integration.txt
```

**Molecule** (scenario name = directory under `extensions/molecule/` or `molecule/`):

Serial (one scenario at a time):

```bash
while read -r scenario; do
  [ -n "$scenario" ] || continue
  molecule test -s "$scenario"
done < molecule.txt
```

With Molecule **workers** (experimental concurrent scenarios; multiple `-s` + `--workers`):

```bash
args=()
while read -r scenario; do
  [ -n "$scenario" ] || continue
  args+=(-s "$scenario")
done < molecule.txt

((${#args[@]})) || exit 0
molecule test "${args[@]}" --workers cpus-1
# optional: --continue-on-error
```

`--workers` accepts an integer, `cpus`, or `cpus-1`. Prefer this over a serial loop when the collection supports shared-state / ansible-native multi-scenario runs. See Molecule’s docs for `--workers` / `--shared-state` behavior (default scenario create/destroy stays on the main process).

**ansible-test integration** (target name = directory under `tests/integration/targets/`):

```bash
mapfile -t targets < integration.txt
if ((${#targets[@]})); then
  ansible-test integration --docker default "${targets[@]}"
fi
```

Adjust Molecule/ansible-test flags to match your collection CI. An empty file means nothing to run for that kind.

Minimal PR CI sketch (fetch full history or the base SHA first):

```yaml
- uses: actions/checkout@v4
  with:
    fetch-depth: 0

- name: Select tests
  run: |
    content-plugin-finder --impact . \
      --base "${{ github.event.pull_request.base.sha }}" \
      --head "${{ github.sha }}" \
      --emit molecule --names-only | tee molecule.txt
    content-plugin-finder --impact . \
      --base "${{ github.event.pull_request.base.sha }}" \
      --head "${{ github.sha }}" \
      --emit integration --names-only | tee integration.txt

- name: Molecule
  run: |
    args=()
    while read -r s; do
      [ -n "$s" ] || continue
      args+=(-s "$s")
    done < molecule.txt
    ((${#args[@]})) || exit 0
    molecule test "${args[@]}" --workers cpus-1

- name: ansible-test
  run: |
    mapfile -t t < integration.txt
    ((${#t[@]})) || exit 0
    ansible-test integration --docker default "${t[@]}"
```

## Library

```python
from pathlib import Path
from content_plugin_finder.crawl.orchestrator import Orchestrator
from content_plugin_finder.collection import build_collection_graph
from content_plugin_finder.impact import compute_impact

report = Orchestrator().scan([Path("extensions/molecule/default")])
print(report.to_dict())

graph = build_collection_graph(Path("../ansible.platform"))
print(graph.resolve("application").name)  # ansible.platform.application

impact = compute_impact(
    collection_root=Path("../ansible.platform"),
    changed_files=["plugins/action/base_action.py"],
)
print(impact.molecule_scenarios)
print(impact.integration_targets)
```
