Metadata-Version: 2.5
Name: laygen
Version: 0.2.0
Summary: Layout-preserving text extraction and reconstruction for office documents
Project-URL: Homepage, https://ondevice.neoali.com/laygen
Author-email: "NEOALI CO., LTD." <neoali.official@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: docx,hwpx,layout,localization,ooxml,pptx,translation,xlsx
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Localization
Classifier: Topic :: Text Processing :: Markup
Requires-Python: >=3.10
Requires-Dist: chardet>=5.0
Description-Content-Type: text/markdown

# laygen

Extract translatable text from office documents and rebuild them with the layout intact.

`laygen` does not translate anything. It splits a document into its text segments, hands
them over as JSON, and puts translated text back where it came from. Whatever does the
translating — an agent, a script, a person — sits in between. That keeps the tool free of
model dependencies, API keys and network access.

## Formats

| Extension | Segment |
|---|---|
| `.docx` | paragraph (body, headers, footers, notes, charts, SmartArt) |
| `.pptx` | paragraph (slides, speaker notes, charts, SmartArt) |
| `.xlsx` | shared string (deduplicated), inline strings, chart labels |
| `.hwpx` | paragraph |
| `.csv` | non-empty cell |
| `.txt` | non-empty line |

## Install

```bash
uv tool install laygen
```

The only dependency is `chardet`, for detecting text encodings. Everything else is
standard library.

## Use

```bash
laygen extract report.docx -w work/     # 1240 segments -> 39 batches
laygen status -w work/                  # which batch is next
# translate work/batches/batch_NNN.jsonl -> batch_NNN.done.jsonl
laygen check -w work/ --batch 0 -t work/batches/batch_000.done.jsonl
laygen merge -w work/ work/batches/*.done.jsonl -o translated.jsonl
laygen reconstruct -w work/ -t translated.jsonl -o report.ko.docx
```

Extraction always splits the segments into batches, capped by both segment count
(`--batch-size`, default 32) and character budget (`--batch-chars`, default 4000).
Segment count alone is a poor proxy for how much text a batch holds — a hundred table
cells and a hundred paragraphs differ by two orders of magnitude.

Finished batches stay on disk, so an interrupted run resumes rather than restarting;
`laygen status` reports where. Any id left out keeps its source text, so a partial
translation degrades into a partly-translated document rather than a broken one.

`laygen missing -w work/ -t translated.jsonl -o retry.jsonl` writes out whatever is
still outstanding, ready to translate and merge in.

Re-splitting with `laygen batches` refuses to run if it would orphan translations that
are already done; pass `--force` to override.

### Translation file formats

**JSONL (preferred)** — one record per line:

```jsonl
{"id": "cid0", "text": "번역된 텍스트"}
```

A line that fails to parse costs that one segment, which falls back to its source text.
This matters because translated text often contains straight double quotes, and one
unescaped `"` would otherwise invalidate the whole file.

**JSON object** — `{"cid0": "…", "cid1": "…"}`. Fine for short documents, but a single
syntax error loses every translation in the file.

Either way, generate the file with a JSON serialiser rather than typing it by hand.

## The workspace

```
work/
├── original.docx    copy of the source
├── plan.json        format, checksum and the map back to the original elements
├── segments.json    {"cid0": "…"}  — the full text, source of truth
└── batches/         batch_000.jsonl …  — what a translator actually works through
```

Self-contained: move the directory to another machine and reconstruction still works.

## Notes and limits

- **Segment ids are positional.** They are assigned in document order at extraction time.
  Reconstruction re-reads `original.*` from the workspace, so never edit the source
  document between the two steps — `plan.json` records a SHA-256 to help you notice.
- **Formatting that varies inside a paragraph is lost.** A translated paragraph goes into
  its first run and the remaining runs are emptied. Translation reorders words, so there
  is no honest way to keep a bold span attached to the word it started on. Paragraph and
  character styling itself is preserved.
- **XLSX is edited through its shared string table**, not rebuilt by a spreadsheet
  library, so charts, pivot tables, macros and formulas survive untouched. Numbers, dates
  and formulas are skipped by construction — they are not stored as shared strings.
  Because Excel deduplicates that table, a label repeated across fifty cells is one
  segment, translated once and consistently.
- **PPTX skips slide layouts and masters**, whose text is template boilerplate.
- **HWPX drops inline markup inside a rewritten paragraph** (highlight marks, line
  breaks). The paragraph's own text and styling survive.
- **No PDF.** It needs layout detection and OCR, which would pull in a GPU-scale
  dependency stack. Out of scope for this version.

## Development

```bash
uv sync
uv run pytest
```
