Metadata-Version: 2.4
Name: vcti-shader-compiler
Version: 2.0.1
Summary: Offline Slang to GLSL ES 3.00 shader compiler: validated, linkable artifacts with reflection-derived uniform layouts
Author: Visual Collaboration Technologies Inc.
License-Expression: LicenseRef-Proprietary
Project-URL: Repository, https://github.com/vcollab/vcti-python-shader-compiler
Project-URL: Changelog, https://github.com/vcollab/vcti-python-shader-compiler/blob/main/CHANGELOG.md
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: <3.14,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26
Requires-Dist: moderngl>=5.12
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff; extra == "lint"
Provides-Extra: typecheck
Requires-Dist: mypy; extra == "typecheck"
Dynamic: license-file

# vcti-shader-compiler

Offline Slang → GLSL ES 3.00 shader compiler.

## Overview

`vcti-shader-compiler` lets you write a shader **once**, in Slang, and get
validated shader text for the graphics API that has to run it. It drives a
pinned external toolchain (`slangc` → SPIR-V → SPIRV-Cross → glslang) at build
time, derives each shader's uniform layout from the Slang compiler's own
reflection data, repairs the defects that cross-compilation introduces, and can
execute a compiled shader headlessly so you can test its output in Python.
Nothing it produces needs a shader compiler at runtime.

**GLSL ES 3.00 is the only target implemented.** Slang itself can emit WGSL,
and adding it is mostly a matter of *removing* steps rather than adding them —
[docs/wgsl.md](docs/wgsl.md) records what was tried, what it would change, and
the one question worth settling first.

## Why author shaders this way

Shader source is awkward to work with in three ways, whatever you happen to be
rendering, and they show up even if you only ever target one graphics API:

- **There is no way to share code.** GLSL has no modules and no `import` — a
  shader is one flat translation unit. Anything two shaders both need gets
  copied, or assembled at runtime by string concatenation and `#define`
  switches. That is how most large shader codebases end up built, and it is why
  they are hard to change safely.
- **Nothing checks it until a GPU does.** A typo, a varying that does not match
  between stages, a uniform spelled two ways — none of it surfaces until a
  driver compiles the shader, and then the symptom is a blank frame or a subtly
  wrong image rather than an error naming the line.
- **It is hard to test.** Confirming that a shader computes the value you meant
  usually means rendering something and looking at it. There is no natural way
  to assert on the numbers.

A fourth reason appears later in a project's life: the API you compile for is
not necessarily the one you will always target. A renderer written against
WebGL2 wants GLSL ES 3.00; moving to WebGPU means WGSL; adding a native or a
server-side path means something else again. That is a rewrite of the shader
library unless the source was written independently of the target.

This package's answer to all four:

- **[Slang](https://shader-slang.org/) as the source language.** An
  open-source, Khronos-hosted shading language with HLSL-like syntax. It has
  real modules and `import`, so shader code composes like ordinary code; and it
  compiles to SPIR-V, the portable shader IR (Intermediate Representation),
  from which the same source can be emitted as GLSL, WGSL, MSL, or HLSL. Write
  the logic once, independently of the target.
- **Compilation offline, on a build machine.** The shader is generated and
  type-checked before anything ships, so a mistake fails your build rather than
  your users' frame.
- **Execution from Python.** `render_readback` runs the compiled shader headlessly
  and hands back what the GPU computed as a NumPy array, so shader math can be
  asserted on in an ordinary test.

The package compiles shaders and nothing else — it has no opinion about what a
shader computes, no catalogue of shader kinds, and no idea where your Slang
modules live. So none of this is specific to a subject area: it applies to any
shader you would rather write once, check before shipping, and test like normal
code. VCollab uses it, for example, to build the shaders behind its CAE
viewers, but the package knows nothing about that.

## How it works

Two passes over the same source, both driven by `slangc`:

```
              ┌─ slangc ─→ SPIR-V ─→ spirv-cross ─→ GLSL ES 3.00 ─→ glslang ✓
your.slang ───┤
              └─ slangc ─→ reflection JSON ─→ uniform layouts + attributes
```

SPIR-V sits in the middle because it is the interchange format both halves of
the toolchain speak: Slang emits it, SPIRV-Cross consumes it. `glslang` then
parses and type-checks the emitted GLSL, so invalid output never reaches
whatever consumes it.

Cross-compiled output is corrected rather than trusted, because each of these
defects is invisible until far downstream — or, in one case, stops the build
with a diagnostic about an extension nobody asked for:

- **Inter-stage varyings are renamed.** GLSL ES 3.00 links varyings *by name*
  and forbids `layout(location)` on them, but separately compiled stages get
  unrelated generated names — so the pair silently fails to link in the
  consumer. Both sides are forced to a shared `v{location}`.
- **`half` becomes `mediump`.** Slang's `half` cross-compiles to fp16 types
  behind two desktop vendor extensions, which no WebGL2 driver accepts and the
  validator rejects — so `half` would otherwise fail the build outright. Those
  types are rewritten to explicitly `mediump` fp32, the ES spelling of the same
  intent, lowering exactly the varyings and locals the source asked to lower.
  `half` in a *uniform block* is refused instead, because Slang packs it as two
  bytes and widening it would move every later member off its reflected offset.
- **Every stage declares its float precision, `highp` by default.** SPIRV-Cross
  emits `precision mediump float;` for a fragment stage — as little as 10 bits
  of mantissa, fine for colours but lossy for measured data or large
  coordinates — and emits nothing at all for a vertex stage, where ES 3.00's own
  default is `highp`. Left alone, one source computes at two precisions, and the
  symptom is quietly wrong pixels rather than an error. Pass `precision=` to
  choose something else; see [docs/precision.md](docs/precision.md) for what
  lowering it reaches.

Uniform offsets come from slangc's reflection rather than from hand-computed
std140 rules, so packing stays correct when a member is added.

## Composing shaders from modules

Because sources are Slang modules, a pipeline is assembled by importing rather
than by pasting text together:

```slang
import lighting;    // a shared lighting model
import colormap;    // mapping a value to a colour
```

You tell the compiler where those modules live by passing `include_dirs`, which
become slangc `-I` search paths. That is what lets one shader pull in modules
that live anywhere — a shared directory in your repository, or Slang files
shipped inside a separately installed package — while the compiler itself
discovers nothing and knows none of them by name.

## Validating and testing shaders

Two levels, both without a browser or a GPU farm:

- **Statically**, `glslang` type-checks the emitted GLSL as part of compiling,
  so a malformed shader fails the build.
- **Executably**, `render_readback` runs the compiled shader in a headless OpenGL
  context via [moderngl](https://moderngl.readthedocs.io/), feeding it an array
  of input values and handing back what the GPU computed. Because the result is
  a NumPy array, an ordinary `pytest` can diff real GPU output against a NumPy
  reference implementation — so a shader library can prove its math means what
  its authors think it means, on every commit.

## What you get back

Results come back in memory — nothing this package writes to disk is meant to
outlive the call, and it never chooses where anything goes.

| Call | Returns |
|---|---|
| `compile_stages` | `{stage: glsl}` — GLSL ES 3.00 text, precision declared, varyings renamed so the stages link |
| `reflect_uniforms` | `{name: UniformLayout}` — std140 offset, size, stride, and encoding per uniform |
| `reflect_attributes` | `{name: AttributeLayout}` — location, element type, and whether it binds as an integer attribute |
| `pack_ubo` | `bytes` — one std140 uniform block, ready to upload |
| `render_readback` | One `(N, out_channels)` array per uniform set, in the requested texel format |

You supply the source, an entry-point map, the import search paths, a work
directory, and a resolved `Toolchain`. Anything written along the way is an
intermediate in that directory and can be deleted the moment the call returns —
[docs/design.md](docs/design.md) explains why the package deliberately owns no
format of its own.

## Installation

```bash
pip install vcti-shader-compiler
```

Requires Python 3.12 or 3.13. The Python package is pure orchestration — the
actual compilers are external binaries you provision yourself, see below.

## Prerequisites: the shader toolchain

Three external executables must be on your machine before anything compiles:
`slangc`, `spirv-cross`, and `glslang`. They are not pip-installable — you
download or build them once and point three environment variables at them:

```bash
export SLANG_DIR=...        # extracted slang release
export SPIRV_CROSS_DIR=...  # SPIRV-Cross source tree you built
export GLSLANG_DIR=...      # glslang source tree you built
```

Then check that all three resolve:

```bash
python -c "from vcti.shader.compiler import discover_toolchain; print(discover_toolchain())"
```

**[docs/toolchain.md](docs/toolchain.md) has the full procedure** — where to
download each one, the `cmake` invocations for the two that need building, the
layouts each variable expects, and known-good versions.


## Quick Start

```python
import tempfile
from pathlib import Path
from vcti.shader.compiler import find_toolchain, compile_stages

toolchain = find_toolchain()  # or discover_toolchain() to raise if missing
with tempfile.TemporaryDirectory() as tmp:
    work = Path(tmp)
    (work / "demo.slang").write_text(slang_source)
    stages = compile_stages(
        toolchain,
        work / "demo.slang",
        {"vertex": "vertexMain", "fragment": "fragmentMain"},
        work,
        include_dirs=[...],  # Slang dirs the source imports from
    )
# stages == {"vertex": "<glsl es>", "fragment": "<glsl es>"}
```

## Dependencies

`numpy` and `moderngl` — the latter drives the headless GL context that
`render_readback` tests shaders in. Compiling and testing are one package, not two:
generating a shader you cannot execute is only half the job.

`moderngl`'s `glcontext` layer publishes no cp314 wheel, which is why
`requires-python` is capped below 3.14.

The three toolchain executables are external and pinned — they are provisioned
as described under [Prerequisites](#prerequisites-the-shader-toolchain), never
as pip dependencies.

## Documentation

| If you want to… | Read |
|---|---|
| Install the three external compilers | [docs/toolchain.md](docs/toolchain.md) |
| Lower precision to `mediump`, and what `half` does | [docs/precision.md](docs/precision.md) |
| Add a WGSL target (not implemented — the plan and what was tried) | [docs/wgsl.md](docs/wgsl.md) |
| Understand the architecture and design decisions | [docs/design.md](docs/design.md) |
| Navigate and modify the source | [docs/source-guide.md](docs/source-guide.md) |
