Metadata-Version: 2.4
Name: pygpack
Version: 1.0.0
Summary: Build, run, and inspect .pyg game packages (PYG1 binary container format).
License: MIT
Keywords: pygame,game-packaging,pyg,asset-bundling
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Games/Entertainment
Classifier: Topic :: Software Development :: Build Tools
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"

# PYG Format Specification

This document specifies the **PYG1** binary container format, the format
produced by `pyg build` and read directly (not via extraction) by `pyg run`,
`pyg list`, `pyg info`, `pyg validate`, and `pyg extract`. It also documents
the legacy ZIP-based `.pyg` format that PYG1 replaces, and the compatibility
contract between them.

Reference implementation: `pyg/format/` (PYG1) and `pyg/legacy/` (legacy ZIP).

---

## 1. Overview

A `.pyg` file is a single-file game package: game code, assets, and a JSON
manifest bundled together for distribution. Two things can produce a `.pyg`
file today:

| Format | Magic / signature | Status | Reader |
|---|---|---|---|
| **PYG1** | `50 59 47 31` (`"PYG1"`) at byte 0 | Current, default | `pyg.format.reader.Pyg1Package` |
| Legacy ZIP | Standard ZIP local-file-header signature (`PK\x03\x04`) | Read-only, deprecated | `pyg.legacy.zip_reader.LegacyZipPackage` |

`pyg.loader.open_package()` detects which one it's looking at and returns
the matching reader, both of which expose the same interface — see
[§8 Compatibility](#8-compatibility-and-format-detection).

PYG1 is a **game distribution container**, not a general-purpose filesystem.
It deliberately does not include (and is not intended to grow): encryption,
DRM, online activation, networking, or executable machine-code packaging.

---

## 2. Binary layout

A PYG1 file is four regions, laid out in this order:

```
Offset 0
┌────────────────────────────────────────────┐
│ PYG1 HEADER            (128 bytes, fixed)   │
├────────────────────────────────────────────┤
│ METADATA BLOCK          (meta.json bytes)   │  <- metadata_offset
├────────────────────────────────────────────┤
│ FILE TABLE              (N variable entries)│  <- file_table_offset
├────────────────────────────────────────────┤
│ FILE DATA               (raw/compressed     │  <- data_offset
│                           file bytes)        │
│   main.py                                   │
│   __init__.py                               │
│   icon.png                                  │
│   assets/player.png                         │
│   assets/background.png                     │
└────────────────────────────────────────────┘
```

Every region's location and size is given explicitly in the header as an
absolute byte offset + size pair, so a reader validates all four regions'
bounds before trusting any of them (see [§7 Validation](#7-validation-rules)).
The builder always lays regions out contiguously in this order, but nothing
in the reader *requires* that — a future writer could reorder them and old
readers would still work, as long as the header's offsets are correct.

All multi-byte integers are **little-endian**.

---

## 3. Header (128 bytes, fixed size)

| Offset | Size | Field | Type | Description |
|-------:|-----:|---|---|---|
| `0x00` | 4  | `magic` | bytes | Always `50 59 47 31` (`"PYG1"`) |
| `0x04` | 2  | `version` | u16 | Format version. `1` for PYG1. |
| `0x06` | 2  | `header_size` | u16 | Size of this header in bytes (`128` in v1). Lets a future reader know how much header to skip even if it doesn't understand every field. |
| `0x08` | 4  | `flags` | u32 | Bitflags — see below |
| `0x0C` | 8  | `metadata_offset` | u64 | Absolute byte offset of the metadata block |
| `0x14` | 8  | `metadata_size` | u64 | Size in bytes of the metadata block, as stored (see `METADATA_COMPRESSED` flag) |
| `0x1C` | 8  | `file_table_offset` | u64 | Absolute byte offset of the file table |
| `0x24` | 8  | `file_table_size` | u64 | Total size in bytes of the file table region |
| `0x2C` | 4  | `file_count` | u32 | Number of entries in the file table |
| `0x30` | 8  | `data_offset` | u64 | Absolute byte offset where file data begins |
| `0x38` | 8  | `data_size` | u64 | Total size in bytes of the data region |
| `0x40` | 4  | `header_crc32` | u32 | CRC32 of bytes `[0x00, 0x40)` — everything above, computed over the *unpacked* field values, not including this field or the reserved tail |
| `0x44` | 60 | `reserved` | bytes | Zero-filled. Reserved for PYG2+. |

Total: **128 bytes** (`0x00`–`0x80`).

### `flags` bit definitions (v1)

| Bit | Name | Meaning |
|---|---|---|
| 0 | `METADATA_COMPRESSED` | Metadata block is zlib-compressed |
| 1–31 | — | Reserved, must be `0` in v1 |

### Why a header CRC?

`header_crc32` lets a reader detect a corrupted or hand-edited header
*before* trusting any offset in it — this is checked first, ahead of
per-region bounds checking, so a single flipped byte anywhere in the header
is caught deterministically rather than potentially producing a
seemingly-valid-but-wrong read.

### Why `header_size` as its own field?

A PYG2 header can be longer than 128 bytes (using some of today's reserved
space, or growing past it) without breaking a PYG1-only reader's ability to
locate the metadata block, file table, and data — those are all absolute
offsets from byte 0, not "immediately after the header."

---

## 4. Metadata block

The metadata block holds the game manifest (conventionally authored as
`meta.json` in the source project) as UTF-8 JSON bytes, located via
`metadata_offset`/`metadata_size`. If `METADATA_COMPRESSED` is set, the
stored bytes are zlib-compressed and must be inflated before parsing.

### Manifest schema

```json
{
    "format": "pyg",
    "format_version": 1,
    "name": "My Game",
    "id": "com.example.mygame",
    "version": "1.0.0",
    "author": "Joven",
    "entry": "main.py",
    "icon": "icon.png"
}
```

| Key | Required | Notes |
|---|---|---|
| `name` | **yes** | Non-empty string |
| `id` | **yes** | Non-empty string (reverse-DNS style by convention) |
| `entry` | **yes** | Non-empty string; a path that must exist in the file table after build |
| `format` | no | Defaults to `"pyg"` if omitted |
| `format_version` | no | Defaults to `1` if omitted |
| `version`, `author`, `icon`, ... | no | Free-form; unknown keys are preserved, not rejected |

The reader can fetch and parse metadata with a single seek + read — no file
table parsing or extraction required, which is what makes `pyg info` fast
even on a large package.

`meta.json` itself is **not** duplicated as a regular file table entry; it
exists only as the metadata block.

---

## 5. File table

Located at `file_table_offset`, spanning exactly `file_table_size` bytes,
containing exactly `file_count` entries back-to-back. Each entry is
**variable-length** (because paths vary):

| Field | Type | Description |
|---|---|---|
| `entry_size` | u32 | Total byte length of this entry, including this field |
| `path_length` | u16 | Length in bytes of the path string that follows |
| `path` | bytes | UTF-8, POSIX-style (`/`), relative, no leading `/`, no `..` components |
| `data_offset` | u64 | Absolute file offset where this entry's (possibly compressed) data begins |
| `compressed_size` | u64 | Byte length of the stored data, as written (post-compression if compressed) |
| `original_size` | u64 | Byte length after decompression |
| `compression_method` | u8 | See [§6](#6-compression) |
| `checksum_method` | u8 | `0 = none`, `1 = CRC32` |
| `checksum` | u32 | CRC32 of the **original (decompressed)** data, or `0` if `checksum_method == 0` |
| `entry_flags` | u16 | Reserved, `0` in v1 |
| `reserved` | 8 bytes | Zero-filled, reserved for future per-entry metadata |

Fixed portion (everything except `path`) is **46 bytes**; total entry size
is `46 + path_length`.

`entry_size` being self-describing means a reader can walk the table by
repeatedly jumping `entry_size` bytes forward, without needing to fully
understand every field in an entry written by a newer format version —
the same forward-compatibility idea as `header_size`.

Loading the whole table is one seek + read (it's typically tiny — just
entries, not data); after that, locating any individual file is a
dictionary lookup, and reading it is one more seek + read into the data
region. No full-package scan or extraction is needed to read one file.

---

## 6. Compression

Compression is applied **per file**, not to the package as a whole — this
is what makes random access to a single file possible without touching
any other file's data.

### Method enum

| Value | Name | Status |
|---|---|---|
| `0` | `STORED` | Implemented — raw bytes, no compression |
| `1` | `ZLIB` | Implemented — zlib/deflate |
| `2` | `ZSTD` | Reserved for a future version |
| `3` | `LZMA` | Reserved for a future version |

### Builder policy

For each file, the builder tries zlib and keeps the compressed result only
if it's actually smaller than the original — otherwise it falls back to
`STORED`. Empty files are always `STORED` (zlib has non-zero overhead even
for zero input bytes). This means, for example, a short `main.py` might
stay `STORED` while a large, repetitive dialogue `.txt` file compresses to
a fraction of its size — both are legal, and the file table simply records
whichever method was actually used for that file.

---

## 7. Checksums

Every file entry (as written by the current builder) carries a **CRC32**
of its original, decompressed content, computed with `checksum_method = 1`.
On read, after decompression, the reader recomputes CRC32 over the
decompressed bytes and compares it to the stored value — a mismatch raises
`ChecksumMismatchError` rather than silently returning corrupted data.
Checksum verification can be explicitly skipped per-read
(`pkg.read(path, verify_checksum=False)`) for tooling that wants raw access
regardless.

There is no package-level checksum in PYG1 — `header_crc32` covers header
corruption, and per-file CRC32 covers data corruption; a whole-package hash
was deliberately left out to keep the format simple, per the "don't
overcomplicate PYG1" design goal.

---

## 8. Compatibility and format detection

### Format detection order

```
if file starts with PYG1 magic bytes:
    use PYG1 reader (pyg.format.reader.Pyg1Package)
elif file is a valid ZIP archive:
    use legacy reader (pyg.legacy.zip_reader.LegacyZipPackage)
else:
    reject as an invalid .pyg package
```

Implemented in `pyg.loader.open_package()`, used by every CLI command
except `build` (which always produces PYG1) — so a game distributed as an
old ZIP-based `.pyg` continues to run, list, validate, and extract exactly
as it did before, with no user-visible difference.

### Shared reader interface

Both `Pyg1Package` and `LegacyZipPackage` expose:

```
read_metadata() -> dict
list_files() -> list[str]
get_entry(path) -> object with at least .original_size
read(path, *, verify_checksum=True) -> bytes
extract_all(dest_dir) -> Path
close()
```

...plus context-manager support (`with open_package(path) as pkg:`). Code
written against this interface doesn't need to know or branch on which
concrete format it received.

### What's different for legacy packages

- No binary header/file-table to validate — `pyg validate` instead checks
  ZIP integrity (`zipfile.testzip()`) and manifest validity.
- `get_entry()` returns compression info as a string (`"STORED"` /
  `"DEFLATED"`) rather than the PYG1 integer enum, since ZIP's own
  compression model doesn't map onto PYG1's `compression_method` values.
- Checksum verification is handled internally by Python's `zipfile` module
  during `read()` (it raises on CRC mismatch itself); there's no separate
  verification step to opt in or out of.

### What's the same

Everything a caller actually needs day to day: metadata, file listing,
individual file reads, and safe extraction all work identically regardless
of format.

---

## 9. Validation rules

Run by `pyg validate` (`pyg.validator.validate_pyg()`, dispatching to
`validate_pyg1()` or `validate_legacy_zip()` by detected format).

### PYG1

- File is at least 128 bytes (header fits)
- `magic == b"PYG1"`
- `version` is supported by this build
- `header_size >= 128`
- `header_crc32` matches a recomputed CRC32 of the header
- All four regions' `offset + size <= file_size` (no out-of-bounds reads;
  checked with arbitrary-precision integers, so no overflow risk)
- No two regions overlap
- Metadata block parses as valid JSON and contains `name`, `id`, `entry`
- File table has exactly `file_count` entries, and parsing it consumes
  exactly `file_table_size` bytes
- Every path is safe: no `..`, no leading `/` or `\`, no drive letters,
  no empty path segments
- No two file entries share the same path
- Every entry's data span falls inside the declared data region
- No two entries' data spans overlap each other
- Every file decompresses successfully and its checksum (when present)
  matches its content

### Legacy ZIP

- File is a well-formed ZIP archive (`zipfile.testzip()` finds no
  corrupt members)
- `meta.json` is present, parses as JSON, and contains `name`, `id`, `entry`

---

## 10. Security model

`.pyg` files — of either format — are treated as **untrusted input**:

- Path safety is enforced on every entry, both when writing (defense
  against a buggy builder) and when reading (defense against a hostile or
  corrupted package): absolute paths (`/etc/passwd`, `C:\Windows\...`),
  parent-directory traversal (`../../file`), and empty/`.`/`..` path
  segments are all rejected before any file is opened for writing.
- `extract_all()` additionally resolves each target path and confirms it
  stays inside the destination directory before writing, as a second,
  independent check beyond path-string validation.
- All header and file-table offsets/sizes are bounds-checked against the
  actual file size before any seek/read happens, and sanity-bounded
  (`MAX_REASONABLE_SIZE`, `MAX_REASONABLE_FILE_COUNT` in
  `pyg/format/constants.py`) to reject obviously-corrupt or maliciously
  crafted header values before they're used for anything.
- Nothing in the header or file table is trusted for control flow beyond
  locating and bounds-checking bytes to read.

Deliberately **out of scope**: encryption, DRM, code signing, or any
attempt to make package *contents* tamper-*proof* rather than
tamper-*evident*. CRC32 (both levels) detects accidental corruption; it is
not a security mechanism against a deliberate attacker who can rewrite the
whole file, including its checksums.

---

## 11. Building and running packages

### Build

```bash
pyg build MyGame/                  # writes MyGame.pyg next to the project dir
pyg build MyGame/ -o dist/Game.pyg # explicit output path
pyg build MyGame/ --format pyg1    # explicit format (currently the only option)
pyg build MyGame/ --compile        # ship .pyc bytecode instead of .py source
```

`MyGame/` must contain `meta.json` with at least `name`, `id`, and `entry`.
Every other file under the directory is packaged; `meta.json` itself becomes
the metadata block rather than a regular file entry. See [§12](#12-source-protection--what-pyg1-does-and-does-not-do)
before relying on `--compile` for anything beyond casual source hiding.

### Inspect

```bash
pyg info MyGame.pyg       # print the manifest as JSON
pyg list MyGame.pyg       # print every file path (and size) in the package
pyg validate MyGame.pyg   # run every check in §9, print a pass/fail report
```

All three work directly against the binary layout — no extraction step —
and all three auto-detect PYG1 vs. legacy ZIP.

### Extract

```bash
pyg extract MyGame.pyg                  # extracts to ./MyGame/
pyg extract MyGame.pyg -o some/dir/     # extracts to a specific directory
```

### Run

```bash
pyg run MyGame.pyg
pyg run MyGame.pyg -- --level 3         # extra args are passed to the entry point
pyg run MyGame.pyg --keep-temp          # don't delete the runtime dir afterward (debugging)
```

`pyg run` (`pyg.runtime.run_pyg1`) does the following, matching the flow
described in the original design:

1. Open the package (format auto-detected)
2. Validate the manifest has an `entry`
3. Create a fresh temporary runtime directory
4. `extract_all()` the package into it
5. Confirm the entry point file actually exists post-extraction
6. Launch it with the current Python interpreter, with `cwd` set to the
   runtime directory (so relative asset paths like `assets/player.png`
   resolve exactly as they would unpackaged)
7. Wait for it to exit, propagating its exit code
8. Delete the runtime directory (in a `finally`, so this happens even if
   the game crashes or the process is interrupted) — unless `--keep-temp`
   was passed

---

## 12. Source protection — what PYG1 does and does not do

**PYG1 provides integrity, not confidentiality.** `pyg extract` (and the
same code path that powers `pyg run`) will always recover the exact bytes
that were packaged, by design — that's what makes the format actually
runnable. Renaming a `.pyg` file to `.zip` no longer works (unlike the old
format — see [§8](#8-compatibility-and-format-detection)), which stops
*casual* browsing with a generic archive tool, but it does not stop
`pyg extract` itself, since that's the legitimate "run this game" path and
can't be removed without breaking `pyg run`.

`pyg build --compile` ships compiled `.pyc` bytecode instead of `.py`
source: every `.py` file is compiled with the standard library's
`py_compile` and only the bytecode is packaged (the source text is not
included in the output file at all); `meta.json`'s `entry` field is
rewritten automatically (`main.py` → `main.pyc`) if it pointed at a
compiled file. This raises the bar from "open in a text editor" to "run a
decompiler" — meaningfully higher friction, genuinely useful against
casual viewing — but it is explicitly **not encryption** and does not stop
a determined person: bytecode decompilers (e.g. `decompyle3`,
`uncompyle6`) recover source close to the original from `.pyc` files.

Real encryption was deliberately excluded from PYG1's goals (see
[§1](#1-overview)) and isn't a natural fit for a Python-executed format
regardless: whatever key or scheme decrypts a file at runtime has to be
reachable by the interpreter that's about to execute it, so it slows down
extraction rather than preventing it. For a distributable *Python* game,
there is no tier that stops a sufficiently determined person — the
interpreter that runs the game always has to see real, executable source
or bytecode. `--compile` is the practical ceiling for this format, not a
missing feature waiting to be escalated.

---

## 13. Extending the format (PYG2+)

Designed extension points, so a future version doesn't require rewriting
readers from scratch:

- **Header**: `header_size` lets old readers skip past new trailing fields
  they don't understand, and 60 bytes of `reserved` space exist for new
  fixed-position fields before the header needs to grow past 128 bytes.
- **File table**: `entry_size` lets a reader skip to the next entry even if
  a newer format added fields to the end of the entry it doesn't recognize.
- **Compression**: `compression_method` is an open enum; `ZSTD` (`2`) and
  `LZMA` (`3`) are reserved values, not yet implemented — adding one is a
  change to `pyg/format/compression.py` plus the writer's method-selection
  logic, not a layout or reader change.
- **Metadata**: the `METADATA_COMPRESSED` flag is already defined even
  though the current builder doesn't set it, for exactly this kind of
  forward wiring.

None of this is implemented for a hypothetical PYG2 today — this section
just documents *why* the current layout looks the way it does.
