Metadata-Version: 2.4
Name: figcrop
Version: 0.2.3
Summary: High-quality figure extraction from dense research-paper PDFs
Author: TadaLab Keio
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/tadalab-keio/figcrop
Project-URL: Repository, https://github.com/tadalab-keio/figcrop
Project-URL: Issues, https://github.com/tadalab-keio/figcrop/issues
Keywords: pdf,figures,research-papers,layout-analysis,openvino
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Text Processing :: General
Requires-Python: <3.14,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: mineru[core]<4,>=2
Requires-Dist: openvino>=2025
Requires-Dist: pymupdf>=1.27.2
Requires-Dist: numpy
Requires-Dist: pillow
Requires-Dist: shapely>=2
Requires-Dist: fastapi
Requires-Dist: uvicorn
Requires-Dist: mcp>=1.0
Dynamic: license-file

# figcrop

Extract figures from research-paper PDFs by real figure number.

figcrop uses MinerU's PP-DocLayoutV2 layout model for visual-region detection,
then adds its own PDF-text and geometry logic to group panels into whole figures,
assign PDF source objects to figures/panels, and exclude page furniture. It is designed for dense semiconductor
papers where simple heuristic tools often miss figures or split panels badly.

## Features

- Extract whole `Fig.N` outputs, including multi-panel figures, by the real figure
  number found in the PDF text layer.
- Keep figure-internal labels, process-flow text, axes, table borders, and panels,
  while excluding the caption line.
- Output modes:
  - `figure` default: whole figure body, no caption.
  - `panel`: split figures into `(a)` / `(b)` subpanels when panel labels are
    available.
  - `caption`: whole figure body plus the matched caption text.
- Run as a persistent local HTTP server so the layout model is loaded once.
- Use OpenVINO by default for fast local inference; torch backends remain available.
- Copy the PDF's final rendered pixels through the union of precise per-object
  ownership bboxes, preserving paint order, clipping, transparency, and overlays.
- Handles slide/poster-style cells in some PDFs by clipping giant page-level
  fallback detections to the caption's local cell.

## Requirements

- Python 3.10-3.13. Windows is the main tested environment.
- `uv` is recommended; the setup scripts install into the project-local `.venv`.
- First run downloads model weights from Hugging Face.
- OpenVINO GPU is the recommended default. NPU is not currently useful for this
  RT-DETR layout model.

## Setup

Windows:

```powershell
powershell -ExecutionPolicy Bypass -File setup.ps1 -Device xpu
```

Linux/macOS:

```bash
bash setup.sh
```

`requirements.txt` is a manifest for the base packages. Device-specific torch
packages are handled by the setup scripts.

## Install

Install the released CLI from PyPI:

```powershell
pipx install figcrop
```

This exposes:

```powershell
figcrop help
figcrop extract paper.pdf out --mode caption
figcrop serve
figcrop montage out_figure out_panel review.jpg
figcrop review paper.pdf review_out auto
figcrop-mcp
```

`figcrop` is the main human/agent CLI. `figcrop-mcp` is only needed when an MCP
client should call figcrop as a registered tool instead of running shell
commands.

To install directly from the current GitHub source instead:

```powershell
pipx install git+https://github.com/tadalab-keio/figcrop.git
```

## Usage

`<py>` means `.venv\Scripts\python.exe` on Windows or `.venv/bin/python` on Unix.

For AI agents, the shortest path is:

```powershell
<py> figcrop.py extract paper.pdf out auto --mode figure
```

Then inspect the JPEGs and `out/figures.json`. Do not treat a successful command
as a successful crop until representative images have been viewed.

Start the local server:

```powershell
<py> figcrop.py serve auto
```

Then request crops:

```bash
curl -s -X POST http://127.0.0.1:8077/extract \
  -H "Content-Type: application/json" \
  -d '{"pdf":"paper.pdf","out_dir":"out","figs":[1,2],"mode":"caption"}'
```

Useful request fields:

- `figs`: real figure numbers to extract, for example `[1,2]`.
- `top`: fallback positional extraction, for example `2` for the first two visual
  regions per page.
- `mode`: `"figure"` default, `"panel"` for `(a)` / `(b)` subpanels, or
  `"caption"` for whole figure plus matched caption text.
- `panels`: legacy alias for `mode: "panel"`.
- `caption_mode`: legacy alias; `"include"` means `mode: "caption"`.

One-shot CLI:

```powershell
figcrop extract paper.pdf out auto
figcrop extract paper.pdf out auto --figs 1,2
figcrop extract paper.pdf out auto --top 3
figcrop extract paper.pdf out_panels auto --mode panel
figcrop extract paper.pdf out_with_captions auto --mode caption
figcrop help
```

Output files are JPEG crops plus a `figures.json` manifest in `out_dir`.
JPEG is the default output codec (quality 95, 4:4:4 chroma) because it keeps
600 dpi review sets and Markdown-embedded figures compact while encoding much
faster than lossless PNG on typical papers. The PDF itself is not inherently a
JPEG: figcrop rasterizes the selected PDF paint commands and then encodes the
result as JPEG.

Production extraction uses MuPDF CPU selective replay under the single renderer
name `mupdf`. The former renderer aliases `cpu` and `auto` are no longer
accepted. Skia/OpenGL rendering is disabled by default because scene
construction, readback, and output encoding did not provide a reliable
end-to-end advantage across PDFs.

The previous direct Skia renderer remains in the GitHub source tree under
`figcrop_gpu.py`, `native/`, and its focused tests as a development asset. It is
not included in the installed wheel and is not a supported extraction backend.
Developers investigating a source checkout can install its dependencies and run
the tests directly:

```powershell
pip install skia-python glfw PyOpenGL
python setup_native.py build_ext --inplace
python -m pytest tests/test_gpu.py
$env:FIGCROP_DEV_RENDERERS = "1"
figcrop extract paper.pdf out auto --renderer skia
```

Without `FIGCROP_DEV_RENDERERS=1`, passing `skia`, `gpu`, `opengl`, or
`skia-cpu` fails with an explicit development-only error instead of running or
silently falling back. This does not affect the OpenVINO GPU/XPU device used by
the layout detector; only production PDF replay is fixed to MuPDF CPU.

Create a borderless review montage from extracted outputs:

```powershell
figcrop extract paper.pdf out_figure auto --mode figure
figcrop extract paper.pdf out_panel auto --mode panel
figcrop montage out_figure out_panel review.jpg
```

`montage` reads only the JPEGs and `figures.json`; it does not need the original
PDF. By default it keeps the extracted images at native resolution, adds a
readable Fig/panel label above each tile, and does not draw tile borders. When
panel groups exist, each row is ordered as caption/full first, then each group
followed by its leaf panels.

JPEG decode and independent montage saves use up to eight workers by default,
bounded by the available logical CPUs and actual job count. Override this for a
particular machine with `FIGCROP_MONTAGE_WORKERS`; more workers can be slower on
memory-bandwidth-bound native-resolution montages:

```powershell
$env:FIGCROP_MONTAGE_WORKERS = "4"
figcrop review paper.pdf review_out auto
```

Full-resolution review images can expand to several GiB in memory even when the
saved JPEG is small. Row images are therefore encoded through a bounded pipeline
and released after saving instead of retaining every row until the end.

For visual QA, the preferred one-command path is:

```powershell
figcrop review paper.pdf review_out auto
```

This creates `review_out/caption`, `review_out/figure`, `review_out/panel`,
`review_out/review_montage.jpg`, and per-figure rows in
`review_out/fig_montages/` using one loaded model. The montage layout is:
caption crop, full figure crop, then panel groups and leaf panels in visual
hierarchy order.

## AI Connectors

- OpenAPI/REST: start `figcrop.py serve auto` and use `/openapi.json`.
- MCP: install `requirements-mcp.txt` and run `figcrop_mcp.py`.

See `CONNECTORS.md` for Claude/Codex/MCP examples and connector safety notes.

## How It Works

1. Render each PDF page at 150 dpi for layout detection.
2. Run PP-DocLayoutV2 through OpenVINO or torch.
3. Keep visual regions labeled `image`, `chart`, or `table`.
4. Read `Fig.N` / `Table N` captions directly from the PDF text layer.
5. Assign each region to the nearest same-column caption below it.
6. Run the detector locally on each figure and pair panel candidates with
   `(a)` / `(b)` label anchors.
7. Split PDF text commands into spatially meaningful word atoms, reconstruct
   glyph/vector/image paint geometry, and follow exact non-white paint contact
   beyond the detector bbox when a figure object crosses it.
8. Assign every source object to exactly one figure/group/panel, caption,
   page-furniture, external, or unresolved owner.
9. Replay only the requested ownership subtree's PDF paint callbacks in their
   original order while preserving transforms, clips, groups, masks, images,
   overlays, and per-glyph text selection; then crop the resulting 600 dpi
   canvas to the owned objects' outer bounds.

The numbering and whole-figure grouping are local geometry logic, not MinerU's
full reading-order pipeline.

## Performance

For repeated extraction, use the server. In local tests, server mode avoids
reloading the model for each request and is much faster than one-shot CLI runs,
especially on one-page PDFs.

For occasional use, the one-shot CLI is still practical: most ordinary papers
finish in a few seconds on a local OpenVINO GPU setup, so running the server is a
convenience rather than a hard requirement.

## Limitations

- Very dense pages can still confuse region-to-caption assignment. Use `top=` as
  a fallback.
- `mode=panel` only emits figures that have a clean `(a)`, `(b)`, ... sequence.
  Figures without panel labels are intentionally skipped.
- Some PDFs contain slide/poster grids or decorative page furniture that can look
  like a large table. figcrop has a local-cell fallback for common cases, but this
  class of PDF may still need review.
- Outputs are intended for local research workflow use. Always inspect crops when
  building datasets or publications.

## License

Apache-2.0. See `LICENSE`.

Built on MinerU and OpenVINO. See `NOTICE` for attribution and dependency notes.
