Wikitext (MediaWiki) reader and writer for myform, implementing proposal P8 from the universal-graph spec.

Spec of record: docs/spec/universal-graph-and-sites.md §P8 in the myform repo. Parent epic: MEMY-1099 (the universal graph, wave 1+). This card covers deferred work past wave 1. Decision of record: D7 — import wikiparse as the reader engine, replacing the mwparserfromhell three-layer reader plan.

Context

MediaWiki has no formal grammar; its specification is the behavior of the PHP parser. The original P8 plan proposed mwparserfromhell as a low-level parser with a hand-built three-layer reader on top (template/link/style surface, line-oriented block pass for list containment, dedicated table sub-parser). D7 replaces that plan with a two-hop adapter using wikiparse (~/my/apps/wikiparse), a from-scratch wikitext translator whose project records report a full English Wikipedia dump run of 8,531,070 pages.

wikiparse.parse_article(text, title=...) -> Article translates wikitext to Markdown + YAML frontmatter. The adapter feeds that Markdown through myform's existing MyST reader to produce a Doc.

Dependency gate discovered 2026-08-08: no public PyPI package exists; the v0.7.0 tag pipeline failed, and current main is 166 commits ahead while still reporting 0.7.0. A wheel built from the old tag emits duplicate YAML frontmatter with current my-basis==1.0.0; current main contains the fix. Prototype against an exact fixed source commit if needed, but the released extra must depend on the prepared 0.8.0 artifact from APPS-113.

What D7 buys and costs

Buys: eliminates the three-layer reader, the table sub-parser, and the list-containment assembly — the hardest engineering in P8. wikiparse already wrote and tested all of it. The adapter is estimated at ~150-250 lines.

Costs (documented in §P8, not hidden):

1. No MappingGrade granularity. Every construct arrives as already-translated Markdown. The Doc tree is identical to reading any Markdown document. SyntaxRaw + MappingGrade does not apply. Acceptable at Tier 3.
2. Template preservation is upstream and bounded. wikiparse extracts {{cite ...}} / <ref> as standalone citation records in YAML frontmatter; non-citation templates are converted to text or dropped. The reader cannot recover what the dependency no longer carries. On write, current myform has ENCLAVE classification but no sidecar carrier surface, so unsupported typed nodes use honest CARRY / APPROX fates until MEMY-1139 lands.
3. Heavy optional-extra dependency chain.wikiparse pulls in pandas, pyarrow, pypandoc (needs system pandoc binary), jinja2, logfire, zstandard, more-itertools, packaging, python-dateutil, tabulate, tqdm. All behind myform[wikitext] with function-local imports; import myform budget unaffected.
4. GPL-2.0-only license. wikiparse is GPL-2.0-only; myform is MPL-2.0. Keeping it as an optional, function-local dependency preserves the import/runtime boundary, but downstream redistribution still needs to honor the dependency's license.
5. Bootstrap attestation.wikiparse.__init__ installs a custom MetaPathFinder that SHA-256-attests source files at import time. Unusual for a library dependency; flagged for awareness.
6. pandoc system binary.wikiparse needs pandoc on PATH for HTML table/list conversion. External requirement beyond Python packages.

Definition of done

Phase 1: the writer (D6 — easy win first)

1. * → wikitext writer. A WikitextWriter registered through Writer.register, lazy-loaded via _DIRECT_WRITERS. Emits headings (=, ==), lists (*, #, ;/:), bold/italic (''', ''), links ([[Article]], [url label]), tables ({| ... |}), and code (<code>, <pre>). Its capability table uses only truthful NATIVE / APPROX / CARRY / DROP fates; no built-in writer may claim ENCLAVE until MEMY-1139 delivers a carrier.
2. Matrix sweep picks up wikitext as a write target with no edit to test_matrix_sweep.py.
3. Tier 3 gate: must not crash; must report honestly; coverage measured, not gated.

Phase 2: the reader (D7 — two-hop adapter)

4. myform[wikitext] extra in pyproject.toml declaring wikiparse>=0.8.0, after APPS-113 produces the first public artifact with the duplicate-frontmatter fix.
5. WikitextReader adapter (~150-250 lines): takes a per-call title, calls wikiparse.parse_article(text, title=...), feeds the complete Article.serialize() output through the existing MyST reader, sets Doc.source = 'wikitext', and preserves Article.title / Article.name under Doc.env['wikiparse']. Registered through Reader.register; the CLI derives a default title from the input filename stem.
6. Matrix sweep picks up wikitext as a read source with no edit to test_matrix_sweep.py.
7. Tier 3 gate: must not crash on real Wikipedia articles; must report honestly; coverage measured, not gated.
8. Round-trip property:read(write(doc)) produces a Doc that is structurally similar (not byte-identical — wikitext round-trip through Markdown is lossy by construction).

Both phases

9. Gates green:uv run --no-sync python -m pytest, uv run ruff check, uv run ruff format --check, uv run pyrefly check.
10. Spec stays true. Where building disproves the spec, the spec is corrected and the correction is recorded.
11. Adversarial review of the adapter surface before the card is called done.

Recovered contract decisions

• Frontmatter mapping. Feed the complete serialization into MystReader. Its FrontMatter node already retains all current keys; splitting/remapping is unnecessary and would be lossy. Only article identity absent from serialization is added under Doc.env['wikiparse'].
• Pandoc fail-closed behavior. Native {| ... |} tables do not need Pandoc, but raw HTML <ul>, <ol>, and <table> do. Upstream currently catches a missing-Pandoc error and silently deletes the construct, so the adapter must preflight pandoc and refuse ingestion rather than describe that as a fallback.
• Bootstrap attestation compatibility. A fresh subprocess proved that wikiparse's attested finder, a pre-existing custom finder, myform lazy loading, and mdformat plugin discovery coexist. Real-dependency tests still isolate MY_DATA, bytecode caches, working directory, and process-global setup.
• Async-loop behavior. A cold synchronous parse_article() call inside a running event loop raises until await wikiparse.Interface.setup() has run. The adapter must test and document deterministic behavior for that context; it may not deadlock or leak a coroutine.
• Optional native reader. If MappingGrade granularity is wanted later, the path is to build a Doc-producing reader using wikiparse's internal parser components (Table, Tag, Template, WikiBuffer, regex.py). Those are unversioned implementation detail; this would require coordination with the wikiparse project. Deferred.

Sizing

Compare: writers/rst.py is 682 lines, writers/typst.py is 942, readers/rst.py is 792. The reader is dramatically smaller than the original three-layer plan (~700-950 lines) because wikiparse already did the parsing.

Sequencing

1. Writer first (D6). No dependency on wikiparse; pure myform code. Can land independently.
2. Reader second (D7). Depends on wikiparse being importable; needs the myform[wikitext] extra and the pandoc system binary.
3. Wave-1 dependency. P7 lands first because it overlaps registries, lowerings, and the matrix. The operator approved executing the remaining backlog without waiting for unrelated wave-2 sequencing.