Metadata-Version: 2.4
Name: repokg
Version: 0.4.4
Summary: Generate an AI-ready knowledge graph (KNOWLEDGE_GRAPH.md + kg.json) of any codebase: modules, dependency edges, branches, PRs, ops surface.
Author: Nihar Shah
License: MIT
Project-URL: Homepage, https://github.com/NehharShah/repokg
Keywords: knowledge-graph,codebase,ai,agents,documentation,repo-map
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Documentation
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# repokg

[![CI](https://github.com/NehharShah/repokg/actions/workflows/ci.yml/badge.svg)](https://github.com/NehharShah/repokg/actions/workflows/ci.yml)
[![PyPI version](https://img.shields.io/pypi/v/repokg.svg)](https://pypi.org/project/repokg/)
[![Python versions](https://img.shields.io/pypi/pyversions/repokg.svg)](https://pypi.org/project/repokg/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Generate an **AI-ready knowledge graph** of any codebase — so an AI agent (or a new
developer) can read one file and start building immediately.

`repokg` extracts everything that can be known *deterministically* about a repo —
module inventory, internal import graph, every branch classified against every PR
(merged / squash-merged / abandoned / stale), contributor stats, CI/Docker/Helm/Make
surface — and renders it as:

- **`KNOWLEDGE_GRAPH.md`** — a single human/AI-readable document with a mermaid architecture
  graph, module tables, branch & PR catalog, timeline, and ops inventory.
- **`.repokg/kg.json`** — the same graph, machine-readable.

The semantic layer (module purposes, data-flow narratives, project eras, gotchas)
can't be produced by static analysis without guessing — so repokg is
**agent-first**: it emits `.repokg/prompts/enrich.md`, a rigorous prompt any AI coding
agent (Claude Code, Cursor, Copilot Workspace…) executes to verify-and-fill the
narrative sections, writing `.repokg/narratives.json`. Re-render and the knowledge graph is
complete. No API keys, no LLM dependency in the tool itself.

## Install

```sh
pipx install repokg        # or: pip install repokg
# from source:
pipx install git+https://github.com/NehharShah/repokg
```

Requirements: Python ≥ 3.9, `git`. Optional: [`gh`](https://cli.github.com) (logged
in) for the PR/branch cross-reference — without it the knowledge graph still builds, minus PR data.

## Usage

```sh
cd your-repo
repokg                      # = generate: scan + prompts + render
```

Output:

```
.repokg/kg.json               # machine-readable knowledge graph
.repokg/prompts/enrich.md        # hand this to your AI agent
KNOWLEDGE_GRAPH.md                        # the knowledge graph document
```

Then, in your AI agent of choice:

> Follow the instructions in .repokg/prompts/enrich.md

The agent explores the code, writes `.repokg/narratives.json`, and runs
`repokg render` — KNOWLEDGE_GRAPH.md now carries verified purposes, data flows,
timeline eras, and gotchas alongside the deterministic structure.

### Commands

| Command | Effect |
|---|---|
| `repokg scan [path]` | Extract structure → `.repokg/kg.json` |
| `repokg prompts [path]` | Write the enrichment prompt |
| `repokg render [path]` | `kg.json` (+ `narratives.json`) → `KNOWLEDGE_GRAPH.md` |
| `repokg generate [path]` | All three (default) |
| `repokg inject [path]` | Wire the knowledge graph into `CLAUDE.md` / `AGENTS.md` / Cursor rules (`--diff` for dry run) |
| `repokg audit [path]` | Show every *inferred* conclusion with confidence + evidence (`--json` for machines) |
| `repokg clean [path]` | Remove everything repokg authored — never touches your content (`--diff` for dry run) |
| `repokg check [path]` | Exit 1 if the knowledge graph is stale vs `HEAD` (CI-friendly) |
| `repokg diff [path]` | Report what changed between two graphs; exit 1 if the shape changed |

Flags: `--out DIR` (default `<repo>/.repokg`), `--md FILE` (default `<repo>/KNOWLEDGE_GRAPH.md`),
`--exclude PATTERN` (repeatable), `--no-github`, `--no-cache`, `--pr-limit N`, `--diff`, `--json`,
`--from KG.JSON`, `--to KG.JSON`, `--format text|json|md`, `--no-renames`.

### Structural diff

`check` answers "is the graph stale?". `diff` answers what actually changed:

```
$ repokg diff .
comparing two graphs of 46e6f96f1a2b (the same commit)

modules  +1 ~1
  + billing                    Python, 4 files, 212 lines
  ~ api                        loc 980 -> 1044

edges  +1
  + api -> billing (Python)    3 imports

languages  ~1
  ~ Python                     files 120 -> 124, loc 18400 -> 18612

shape changed: modules, edges (exit 1)
```

That new edge is the point — "this adds a dependency from `api/` to `billing/`,
intentional?" is the review comment a raw file diff will not give you.

With no arguments it compares the graph the last scan left in `.repokg/kg.json`
against a fresh scan. **The stored graph is never written over** — it is the
baseline, so saving over it would destroy the answer to the next question.
(`.repokg/cache.json` is still updated, since it records what each file
contained rather than what the graph concluded, and is what keeps the scan
feeding the diff fast. `repokg clean` removes it either way.) Point `--from` and
`--to` at saved graphs to compare two of them without scanning at all.

`--format md` emits a report ready to paste into a PR comment, and `--format
json` prints the whole delta keyed by record, uncapped, with scan progress moved
to stderr so it stays pipeable.

**Exit codes** follow `diff(1)` and `git diff --exit-code`: **0** unchanged,
**1** the shape changed, **2** error. Shape means the *membership* of modules,
edges, languages and the ops surface — a module or dependency appearing or
disappearing, or a module switching primary language. LOC drift, import counts,
branch tips and PR states are all reported but deliberately do not move the exit
code, because they change on essentially every commit and a gate that fired
every time would be switched off within a week.

Not everything that differs is reported, either. A branch's tip, date and commit
subject move whenever anyone pushes, so comparing them would bury the
transitions that matter — a branch going stale, or merging — under noise from
unrelated work. A section recorded by only one of the two graphs is skipped and
noted rather than reported as wholly added, since a version gap is not an
architectural change.

#### Moved modules

A module is identified by its path, so moving one reads as a removal plus an
addition. Pairing those back up is the diff's only heuristic, and it says so:

```
modules  R1
  R lib -> shared    high, 1 of 1 imports across unmoved modules preserved
```

The evidence is the **import neighbourhood**, not size similarity — a moved
module keeps its dependencies, whereas two modules having a similar line count
is a coincidence waiting to happen. Confidence is `high` when every import to
and from the unmoved parts of the graph survived, `medium` when most did or the
directory name is unchanged, and `low` when the name is the only thing matching.
An ambiguous pairing — a module split in two, or two candidates for one move —
is not reported at all, and a language change is never a rename.

The pair stays in `added` and `removed` as well, so a consumer that distrusts
the pairing can ignore it and see exactly what it would have seen otherwise.
`--no-renames` turns it off. A rename still exits 1: no dependency changed, but
every path a doc, an agent or a CODEOWNERS entry referenced did.

### Incremental scans

`scan` caches what it extracted from each file in `.repokg/cache.json` and replays
it for files that have not changed, so a re-scan only parses what moved:

```
$ repokg scan .
cache: cold (no cache yet) — parsed 431 files
$ repokg scan .
cache: replayed 431 of 431 files, parsed 0
$ vim src/app.py && repokg scan .
cache: replayed 430 of 431 files, parsed 1
```

A file is replayed only when `git` has not flagged it — `git diff` against the
commit the cache was written at, plus `git status` for anything dirty, staged, or
untracked — **and** its size and mtime still match what was recorded. The second
check is what covers the files git cannot speak for: anything in `.gitignore`
that repokg still walks, submodule contents, or a directory that is not a git
checkout at all.

The cache is an optimization and nothing else. Output is identical either way
(a test enforces byte-for-byte equality), a missing or unreadable or unusable
cache degrades to a full scan and says so on stdout, and `--no-cache` forces one.
`repokg clean` removes it with everything else.

One caveat, shared with every build tool that trusts `stat`: a file rewritten
with its size and modification time preserved is invisible to both checks. Use
`--no-cache` if you have reason to think that happened.

Synthetic mixed Python/TypeScript/Java monorepos, one laptop, best of three —
indicative only, not a benchmark suite (see #17):

| files | LOC | edges | cold | warm | speedup | `cache.json` |
|---|---|---|---|---|---|---|
| 300 | 30k | 21 | 144 ms | 121 ms | 1.2× | 0.1 MB |
| 1,500 | 228k | 111 | 336 ms | 129 ms | 2.6× | 0.3 MB |
| 6,000 | 1.2M | 450 | 1,316 ms | 176 ms | 7.5× | 1.0 MB |

The ratio grows with the repo because the saving is essentially the whole parse
cost, while the warm floor does not shrink. That floor is worth knowing before
optimising further: at 6,000 files it is roughly 100 ms of git metadata
(branches, contributors, merge classification — none of which touches the
cache) and 30 ms of walking, stat'ing and writing the output. Parsing, the part
the cache removes, was around 1,150 ms of the cold scan.

### Excluding paths

Common noise (`node_modules`, `.git`, build output, …) is skipped automatically.
For repo-specific noise — fixtures, snapshots, vendored trees, generated docs —
add globs on the command line or in a committed `.repokgignore` at the repo root:

```sh
repokg scan --exclude '*fixtures' --exclude 'docs/gen'
```

```
# .repokgignore — one glob per line, same semantics as --exclude
*fixtures
*.snap
packages/*/gen
```

Patterns are matched (`fnmatch`) against repo-relative paths; matching
directories are pruned wholesale and matching files dropped, so modules, import
edges, and ops all inherit the exclusion. `*` crosses `/`: `*fixtures` matches
at any depth, `fixtures` only at the root. CLI patterns and `.repokgignore` are
unioned. Exclusions are never silent — `scan` prints what it dropped, `kg.json`
records the patterns and counts, and `repokg audit` carries an uncertainty note.

## Honesty layer

Most of the graph is measured fact. The parts that are *heuristics* are labeled
as findings with confidence and evidence, surfaced by `repokg audit`:

```
[git]
  trunk = master          high    detected via origin/HEAD symref
  integration = staging   medium  matched a well-known integration branch name
[modules]
  4 flagged generated     low     path-name heuristic; verify before excluding
```

`repokg diff` carries the same discipline. Its one heuristic — pairing a removed
module with an added one to call it a move — reports the confidence it matched
at and the evidence for it inline, refuses ambiguous pairings outright, and can
be turned off with `--no-renames`.

Agent-written `narratives.json` is schema-validated before rendering — malformed
enrichment fails loudly with errors precise enough for the agent to self-correct.
(Findings/confidence design inspired by [RepoCanon](https://github.com/NehharShah/repocanon).)

## Agent integration

`repokg inject` adds a **managed block** (delimited by
`<!-- repokg:begin/end -->`, idempotent, never touches your hand-written
content) pointing agents at KNOWLEDGE_GRAPH.md:

- **`CLAUDE.md`** (Claude Code) — updated if present
- **`AGENTS.md`** (the cross-tool agent standard) — updated if present, created if
  no agent file exists at all
- **`.github/copilot-instructions.md`** (Copilot) — updated if present
- **`.cursor/rules/repokg.mdc`** (Cursor, with `alwaysApply: true`) — created
  if `.cursor/rules/` exists; falls back to legacy `.cursorrules`

Keep it fresh in CI:

```yaml
- run: pipx run repokg check . || echo "::warning::KNOWLEDGE_GRAPH.md is stale"
```

Or surface the architectural change a PR makes, which is what the three exit
codes are for — a mistyped path must not read as a new dependency:

```yaml
- run: |
    rc=0; pipx run repokg diff . --format md > diff.md || rc=$?
    if [ "$rc" -gt 1 ]; then exit "$rc"; fi   # 2 = the diff failed, not the graph
    if [ "$rc" -eq 1 ]; then cat diff.md >> "$GITHUB_STEP_SUMMARY"; fi
```

KNOWLEDGE_GRAPH.md itself also lists any agent-context files it found, so an agent landing
on the knowledge graph discovers your rules — and vice versa.

## What gets extracted (all verified, never guessed)

| Area | How |
|---|---|
| Branch classification | `git for-each-ref` + `--merged` ancestry vs the integration branch (auto-detects `staging`/`develop`), cross-referenced with every PR's head ref via `gh` — distinguishes true merges from squash-merges from abandoned work |
| PR catalog | `gh pr list --state all` — open / merged / closed-unmerged, full appendix table |
| Module inventory | Filesystem walk with LOC per directory, language detection, generated-code flagging |
| Import graph | Go: `import` blocks resolved against `go.mod` module paths · Python: stdlib `ast` incl. relative imports · JS/TS: `import`/`require` resolution — relative paths, tsconfig/jsconfig `paths` + `baseUrl` aliases (nearest config wins), npm/yarn/pnpm workspace package names · Rust: `use` declarations resolved against Cargo crate names (cross-crate) and `src/` module trees (intra-crate) · Java/Kotlin: imports resolved by longest prefix against `package` declarations. Directory→directory edges with counts |
| Ops surface | CI workflow names, Dockerfiles, compose files, Helm charts, Makefile targets, config/docs/test/migration dirs |
| Timeline | Merged PRs grouped by month with conventional-commit scope frequencies (replaced by agent-written eras after enrichment) |

## Why agent-first instead of calling an LLM API?

Because the enrichment quality depends on *reading the code*, and your coding agent
already has the repo open, tools to search it, and your permission model. A prompt it
can execute beats a second LLM integration with its own keys, costs, and context limits.
The contract between tool and agent is one JSON file (`narratives.json`) with a fixed
schema — everything else stays deterministic and reproducible.

## Known limitations

- **JS/TS**: relative imports, tsconfig/jsconfig `paths`/`baseUrl` aliases and
  workspace package names are resolved; `extends` chains are not followed (a
  leaf config without its own aliases is skipped rather than shadowing the
  root's), and package `exports` maps are not modeled — subpath imports fall
  back to the package dir. Alias imports whose targets ground nowhere are
  counted in an uncertainty note.
- **Fork PRs**: a fork PR whose head branch name matches a local branch will be
  linked to it (GitHub's API reports bare head refs).
- **Python**: packages are discovered at the repo root and under `src/`;
  deeper monorepo layouts (`packages/*/src/…`) get file-level edges only.
- **Rust**: `use` declarations only — macro-generated imports, re-export
  chains, and `[dependencies] path = …` (non-workspace) crates are not
  resolved; `crate::` paths ground only in module dirs/files that exist.
- **Java/Kotlin**: explicit imports only — same-package references (no import
  needed) and fully-qualified inline names produce no edges; when a package is
  declared only in test roots, edges resolve there.
- Branch `ahead` counts use one batched git call on git ≥ 2.41, with a
  per-branch fallback on older git.
- **Incremental scans** cache per-file facts, not repo metadata: `go.mod`,
  `Cargo.toml`, `tsconfig.json`, `package.json` and `pnpm-workspace.yaml` are
  re-read every scan (one per package, against one read per source file).

## Roadmap

- [x] Rust import graph
- [x] Java / Kotlin import graphs
- [x] `--exclude` glob patterns + `.repokgignore`
- [x] Incremental scan cache for large monorepos
- [x] `repokg diff` — structural diff between two scans
- [ ] `llms.txt` emission alongside KNOWLEDGE_GRAPH.md
- [x] tsconfig `paths` alias + workspace package resolution
- [ ] PyPI release + prebuilt GitHub Action

## Development

```sh
pip install -e .
python -m unittest discover -s tests -v
```

No runtime dependencies — stdlib only.

## Contributing

All work goes through issue → branch (`issue-<N>/<desc>`) → PR → review → squash-merge to `main`.
See [CONTRIBUTING.md](CONTRIBUTING.md) for the workflow and the ground rules
(zero deps, findings for heuristics, `clean` reversibility).

## License

MIT
