Metadata-Version: 2.4
Name: backstamp
Version: 0.1.1
Summary: Record how a figure was made inside the figure file itself: commit, calling function, parameters.
Author: Joschua Geuter
License-Expression: BSD-3-Clause
Project-URL: Repository, https://github.com/Joschua21/backstamp
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Framework :: Matplotlib
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Visualization
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: matplotlib
Requires-Dist: pypdf>=3.15
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Dynamic: license-file

# backstamp

A saved figure says nothing about the code that made it. Weeks later, finding the commit,
the function, the parameters and the data behind a plot means searching old scripts, or
making it again.

backstamp writes that record into the figure file when it is saved, and reads it back:
from the figure, or from the slide decks and LaTeX papers it ends up in. Before you
publish, it writes a copy without the record.

## Install

```
pip install backstamp
```

Python 3.9 or later. Installs matplotlib and pypdf, and the `backstamp` command.

## Save

```python
import backstamp

def plot_accuracy(subjids, dates, window=50, norm=1.0):
    fig, ax = plt.subplots()
    ...
    backstamp.save(fig, "accuracy", fig_dir="figures/", inputs={"subject": subjids, "date": dates})
```

This writes `figures/accuracy.pdf`. Call `save` inside the plotting function: the record
describes the function that calls it.

`save` takes a figure, an axes (what pandas' `.plot` returns) or a seaborn grid (what
`relplot` and `catplot` return).

### In your own save code

Already saving your figures? `backstamp.metadata()` returns the record as a `metadata=`
dict. Any of these, inside the plotting function:

```python
def plot_accuracy(subjids, dates, window=50):
    ...
    meta = backstamp.metadata(inputs={"subject": subjids, "date": dates})
    fig.savefig("figures/accuracy.pdf", metadata=meta)        # matplotlib
    plt.savefig("figures/accuracy.pdf", metadata=meta)        # pyplot
    ax.figure.savefig("figures/accuracy.pdf", metadata=meta)  # pandas: .plot returns ax
    g.savefig("figures/accuracy.pdf", metadata=meta)          # seaborn: relplot, catplot
    with PdfPages("figures/all.pdf", metadata=meta) as pdf:   # many pages, one record
        ...
```

- Call it inside the plotting function, as with `save`: the record describes the function
  that calls `metadata()`. Its first argument heads the `Title`; it defaults to the
  function's name.
- PDF only: a PNG, SVG or JPEG save raises an error when given it.

A save helper of your own that calls `metadata()` is marked with `@backstamp.helper`. The
record then names the function that called the helper, with that function's parameters:

```python
@backstamp.helper
def save_fig(fig, name, inputs=None):
    meta = backstamp.metadata(name, inputs=inputs)
    fig.savefig(f"figures/{name}.pdf", metadata=meta)
    fig.savefig(f"figures/{name}.png")

def plot_accuracy(subjids, dates, window=50):
    ...
    save_fig(fig, "accuracy", inputs={"subject": subjids, "date": dates})
```

- The helper can sit anywhere, in the same file as the plotting functions too.
- Mark each function between the plotting function and backstamp: a helper that calls
  another helper of yours marks both.

## What the record holds

- the commit of the repository the calling code lives in, marked `-dirty` when it has
  uncommitted changes or untracked files, with a hash of the diff to tracked files; the
  branch, and the `origin` URL without credentials
- the installed version of the calling package, the Python version and the backstamp
  version
- the function that called `save` or `metadata`, directly or through a marked helper: its
  module, file, line, and the functions enclosing it
- every parameter of that function, in full. Data objects are stored as their type and
  shape (`<DataFrame 50x2>`, `<ndarray (3, 4) float64>`), not their values.
- the inputs, as given (below)
- when the figure was made

The record identifies the data when the plotting function takes IDs (subjects, dates,
file names) and loads the data itself. A function handed a DataFrame records only its
shape.

Parameters are recorded with the values they have when `save` or `metadata` runs. A
parameter reassigned inside the function is recorded with its new value.

The record is JSON in the PDF's `Subject` field, with a one-line summary in `Title`, e.g.
`accuracy sub-40,45 20251124-20260203`.

## Inputs

`inputs=` records what data went in. Keys are free-form, and values are stored as given:

```python
inputs={"subject": [40, "mouseA"], "date": ["20261002", "03oct2026"]}
inputs={"date": ("20251124", "20251130")}           # a tuple of two is a range
inputs={"date": "20251124-20251130"}                # so is text
inputs={"subject": {40: ("20251124", "20251130"),   # each subject to its dates
                    41: ("20260101", "20260115")}}
inputs={"subject": {40: {"session": [1, 2]}}}       # or to further keys
inputs=["subjids", "dates"]                         # copy these parameters from the call
```

- A tuple of two values is stored as the range `"start/end"`. Use a list for two separate
  values.
- JSON stores mapping keys as text, so `40` reads back as `"40"`.
- A parameter name the function does not have gives a warning.
- `subject`, `date` and `session` are summarised in the `Title` as `sub-40,45`, the date
  span, and `ses-1,2`. Any other key shows as `key=value`.

## Paths and private data

Every parameter ends up in the PDF. Two settings control what that exposes:

| `paths=` | `/Users/me/data/sub-040/trials.parquet` is stored as |
| --- | --- |
| `"name"` (default) | `trials.parquet` |
| `"full"` | the full path |
| `"hide"` | `<path>` |

- A path is a `Path` object, or text starting `/`, `~/`, `C:\` or `\\`. Relative text such
  as `"figures/"` is kept as it is.
- `redact=["token"]` stores the named parameters, and inputs of that name, as
  `<redacted>`.
- `backstamp.configure(paths=..., redact=...)` sets both for the rest of the process.
  `paths=` on a call to `save` or `metadata` overrides it; `redact=` on a call adds to it,
  so a configured name is always redacted.

Other text, such as names in an ordinary string parameter, is stored as it is.

## Record size

A record is at most 30,000 characters. A larger one still saves, with a warning naming the
figure and the size. backstamp drops the data summaries first, then cuts parameters,
largest first, then inputs, and lists what it cut. The code fields are never cut. Reading
the figure back names the cut values too.

## File names

The file is `{name}.pdf`. `filename=` takes a string, or a function of the name and the
inputs:

```python
backstamp.save(fig, "accuracy", fig_dir="figures/", inputs={"subject": [40, 45]},
               filename=lambda name, inputs: f"{name}_sub-{'-'.join(map(str, inputs['subject']))}.pdf")
```

## Read

```
$ backstamp show figures/accuracy.pdf
figures/accuracy.pdf
  title     accuracy sub-40,45 20251124-20260203
  inputs    subject = [40, 45]
            date    = ["20251124", "20260203"]
  made by   lab.plots:plot_accuracy  (plots.py:212)
  chain     plot_accuracy <- main
  params    subjids = [40, 45]
            dates   = ["20251124", "20260203"]
            trials  = "<DataFrame 50x2>"
            window  = 50
            norm    = 1.0
  commit    a1b2c3d-dirty  diff 3f2a9c1e0b7d  on main  https://github.com/me/lab.git
  created   2026-10-05T14:02:11+01:00
  versions  lab 0.3.0, python 3.12.13, backstamp 0.1.0
```

- With no path, `backstamp show` waits for one: drop a figure into the terminal window, or
  paste its path, and press Enter. Several files dropped at once are read together. An
  empty line or Ctrl-D quits.
- Flags typed before or after a dropped figure apply to that line: `figure.pdf --reproduce`,
  `--slide 2 deck.key`. Flags given to `backstamp show` itself apply to every figure
  dropped.
- It also takes folders, searched with their subfolders, and globs.
- `--json` prints every field. `--reproduce` prints the commands that rebuild the figure.
- `python -m backstamp` runs the same command.

### Figures in decks and papers

A figure dragged into a Keynote or PowerPoint deck keeps its record inside the deck, and
one included in a LaTeX document keeps it inside the PDF that pdfLaTeX compiles.
`backstamp show` reads every figure in the deck or PDF, by slide or page:

```
$ backstamp show talk.key --slide 3
talk.key  slide 3  accuracy-9092.pdf
  title     accuracy sub-40,45 20251124-20260203
  inputs    subject = [40, 45]
            date    = ["20251124", "20260203"]
  made by   lab.plots:plot_accuracy  (plots.py:212)
  ...
```

- `--slide N` and `--page N` keep the figures on that slide or page. Both can be repeated.
- After the slide or page comes the name the figure was stored or included as. Keynote
  adds a number to the file name; PowerPoint stores each figure as `imageN.emf`.
- A figure on several slides is listed on each.
- Decks: `.key`, as a file or a package folder, and `.pptx`.
- If the slide order of a deck cannot be read, its figures are listed without slide
  numbers, and `show` says so.

### Find figures

`backstamp ls` lists the figures in a folder and its subfolders, one row each:

```
$ backstamp ls figures/
name            function       commit         inputs                       created
accuracy        plot_accuracy  a1b2c3d-dirty  sub-40,45 20251124-20260203  2026-10-05
sub-40/latency  plot_latency   a1b2c3d        sub-40 20251124-20251130     2026-10-06

$ backstamp ls figures/ --dirty --subject 40
name      function       commit         inputs                       created
accuracy  plot_accuracy  a1b2c3d-dirty  sub-40,45 20251124-20260203  2026-10-05

$ backstamp ls talks/ --function plot_accuracy
name                                   function       commit         inputs                       created
lab-meeting.key slide 3 accuracy-9092  plot_accuracy  a1b2c3d-dirty  sub-40,45 20251124-20260203  2026-10-05
```

A figure inside a deck or a pdfLaTeX PDF gets a row of its own, named by the file, the
slide or page, and the name the figure was stored as. The filters apply to it.

| Filter | Keeps figures |
| --- | --- |
| `--subject ID`, `--date DATE`, `--session ID` | with that value in the input of that name |
| `--input KEY=VALUE` | with `VALUE` in the input `KEY` |
| `--function NAME` | made by `NAME`, or inside a call to `NAME` |
| `--commit SHA` | made at that commit; a prefix is enough |
| `--dirty` | made with uncommitted changes |
| `--unstamped` | without a record |

- A figure must pass every filter given. Each can be repeated.
- Values match as text, inside mappings too: under `subject`, a mapping's keys are subjects
  and its values their dates.
- Whole numbers match by value, so `--subject 40` finds `"040"`.
- A date, `YYYYMMDD` or ISO, finds a stored range that holds it, and can be a range itself:
  `--date 20251101-20251130`. A range in any other format matches its ends exactly.

### Rebuild a figure

```
$ backstamp show --reproduce figures/accuracy.pdf
# figures/accuracy.pdf
# Not an exact rebuild:
# - The code had uncommitted changes (diff 3f2a9c1e0b7d): the checkout is not the code that ran.
# - The record holds a placeholder, not the value, for trials (<DataFrame 50x2>): pass the real values.

git clone https://github.com/me/lab.git
cd lab
git checkout a1b2c3d

# Then, in Python, with this checkout installed (e.g. pip install -e .):
from lab.plots import plot_accuracy

plot_accuracy(
    subjids=[40, 45],
    dates=['20251124', '20260203'],
    trials=...,  # <DataFrame 50x2>
    window=50,
    norm=1.0,
)
```

The commands check out the recorded commit and repeat the call with every parameter. Above
them is what keeps them from rebuilding the figure exactly: uncommitted changes, a
parameter stored as a summary or placeholder, a value cut to fit, a script run directly
rather than imported, or a repository or commit the record does not name.

### In Python

```python
rec = backstamp.read("figures/accuracy.pdf")
rec.status              # "ok"
rec.commit, rec.dirty   # "a1b2c3d", True
rec.call["function"]    # "plot_accuracy"; also module, file, line, chain and params
rec.inputs              # {"subject": [40, 45], "date": ["20251124", "20260203"]}
print(rec)              # the block backstamp show prints
print(rec.reproduce())
rec.to_dict()           # every field, as JSON-safe data

for rec in backstamp.read_all("talks/lab-meeting.key"):
    rec.where           # {"slide": 3, "file": "accuracy-9092.pdf"}
```

`read` reads a file as one figure. `read_all` returns a list with one `Record` per figure:
one for a figure's own PDF, and one for each figure on each slide or page of a deck or a
pdfLaTeX PDF. `where` names the `slide` or `page`, and the `file` the figure was stored or
included as; it is `None` for a figure's own PDF.

Neither raises. `rec.status` says what was found:

| `status` | The file is |
| --- | --- |
| `"ok"` | a figure with a record |
| `"unstamped"` | a PDF without a record, such as one another program saved, or a pdfLaTeX PDF read with `read` |
| `"unreadable"` | not a PDF, or a deck read with `read` |
| `"missing"` | not there |

- `commit` is the hash alone. `dirty` says the code had uncommitted changes, and
  `diff_sha` identifies the changes to tracked files. Untracked files alone leave it
  `None`.
- The other fields are `branch`, `repo`, `created`, `title`, and `versions` (`package`,
  `python`, `backstamp`). A field the record does not hold is `None`.
- `cut` names the values cut to fit the size limit, such as `params.trials`.
- `extra` holds any field passed in `save(provenance=...)` without an attribute of its own.
- In Jupyter, a record displays as a table.

## What the record survives

The record is part of the PDF file, so it goes wherever the file goes. It is lost when a
program draws the figure into a new PDF.

| Route | Record |
| --- | --- |
| Copy or move the file: Finder, `cp`, `mv` | kept |
| Upload to Dropbox, OneDrive or Google Drive, download again | kept |
| Commit to git, clone | kept |
| Send through Slack, or as an email attachment | kept |
| Open in Preview; annotate and save | kept |
| Open in Acrobat, save | kept |
| Open in Illustrator, save as PDF | kept; the `Title` becomes the file name |
| Open in Inkscape, save as PDF | lost |
| Drag into a Keynote or PowerPoint deck, save | kept inside the deck |
| Copy and paste from one deck into another | kept inside the new deck |
| Copy from PowerPoint, paste into Preview (New from Clipboard) | kept |
| Copy from Keynote, paste into Preview | lost |
| Export a deck as PDF | lost |
| Print to PDF | lost |
| Include in a LaTeX document, compile with pdfLaTeX (e.g. on Overleaf) | kept inside the compiled PDF |

- `backstamp show` reads the records inside decks and pdfLaTeX PDFs: see
  [Figures in decks and papers](#figures-in-decks-and-papers).
- Dropbox and OneDrive download several files at once as a zip. Unzip it to read them.
- Acrobat's Document Properties shows only the first 2,000 characters of a long record.
  The file holds all of it.
- Tested on macOS with Keynote 15, PowerPoint for Mac 16, Acrobat, Illustrator 30.8,
  Inkscape 1.4.3 and pdfTeX 1.40.29, using records of up to 30,000 characters and
  non-ASCII text. Untested: PowerPoint for Windows, LuaLaTeX and XeLaTeX.

## Strip the record before publishing

The record holds your repository, parameters and inputs. That helps you and the people
you work with, but a figure sent to a journal, or a paper posted as a preprint, need not
carry it. `backstamp strip` writes a copy without it:

```
$ backstamp strip figures/accuracy.pdf
figures/accuracy.pdf -> figures/accuracy_stripped.pdf  removed: Subject, Title, Creator
```

- The copy sits next to the file, with `_stripped` added to its name. A copy already there
  is replaced; the original is never changed.
- It removes the record, with backstamp's `Title` and `Creator` beside it, wherever it is:
  in the PDF's own fields; in the XMP metadata, where Acrobat and Illustrator copy it; and
  inside each figure of a paper compiled with pdfLaTeX. A `Title` or `Creator` another
  program wrote stays, as does the rest of the PDF: a paper's own title, its pages, links
  and outline.
- It then checks the copy. If any record text is left anywhere in the file, compressed
  parts included, it writes no copy and says where the text is.
- A PDF without a record gets no copy, and `strip` says so.
- It takes folders and globs, as `show` does. With no path, it waits for PDFs dropped
  into the terminal.

Decks are not stripped. `strip` names the figures in a deck that hold a record:

```
$ backstamp strip talk.key
backstamp: talk.key: a deck; backstamp strips PDFs, not decks. These figures in it hold a record:
  accuracy-9092.pdf  slides 3, 4  accuracy sub-40,45 20251124-20260203
Strip each figure before adding it to the deck, or share the deck exported as PDF, which holds no records.
```

In Python, `backstamp.strip("figures/accuracy.pdf")` returns the copy's path, and `out=`
names it. When it writes no copy, it raises a `ValueError` with the same message.

## Wrapping `save`

The record names the nearest calling function outside backstamp. A function that wraps
`save` is marked with `@backstamp.helper`, as above, so the record names the plotting
function instead of the wrapper:

```python
@backstamp.helper
def save_figure(fig, name):
    return backstamp.save(fig, name, fig_dir="figures/")
```

`skip_modules=(__name__,)` on the call to `save` or `metadata` skips every function in the
wrapper's module instead, for a module that holds only wrappers.

## Licence

BSD-3-Clause
