Metadata-Version: 2.4
Name: laya-gateway
Version: 0.1.0
Summary: Route every prompt to the right LLM and block unsafe prompts first, using Laya as a fast decision engine.
Author: amanjoshi2002
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/amanjoshi2002/laya-gateway
Project-URL: Issues, https://github.com/amanjoshi2002/laya-gateway/issues
Keywords: llm,model-routing,guardrails,ai-safety,laya,llm-gateway
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: laya>=0.3.20
Requires-Dist: pydantic>=2.0
Provides-Extra: demo
Requires-Dist: gradio>=4.0; extra == "demo"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

# 🛡️ Laya Gateway

[![tests](https://github.com/amanjoshi2002/laya-gateway/actions/workflows/tests.yml/badge.svg)](https://github.com/amanjoshi2002/laya-gateway/actions/workflows/tests.yml)
[![PyPI](https://img.shields.io/pypi/v/laya-gateway)](https://pypi.org/project/laya-gateway/)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
![Python](https://img.shields.io/badge/python-3.10%2B-blue)

**A lightweight LLM gateway: decide *whether* a prompt is safe and *which* model should answer it, before you pay for a big model call.**

Laya Gateway uses [Laya](https://huggingface.co/convaiinnovations/laya), a small and fast
non-autoregressive decision model, as the "front desk" of your AI app:

```
          ┌──────────────┐   allow   ┌──────────────┐        ┌──────────────┐
 prompt ─▶│  Guardrail   │──────────▶│    Router    │───────▶│  code-A      │
          │ safe/unsafe? │           │ best model?  │        │  thinking-B  │
          └──────┬───────┘           └──────────────┘        │  creative-C  │
                 │ block / review                            │  fast-D      │
                 ▼                                           └──────────────┘
           never reaches an LLM
```

- **Model routing:** describe your models in plain English and get the best match for each prompt.
- **Guardrails:** safe/unsafe classification with `allow` / `review` / `block` thresholds.
- **Fast and cheap:** one forward pass on a small model, which runs on CPU.
- **Typed results:** everything comes back as a Pydantic model with a confidence score.

![Laya Gateway demo](docs/screenshots/01-reasoning-routed.png)

## Installation

```bash
pip install laya-gateway
```

Or from source:

```bash
git clone https://github.com/amanjoshi2002/laya-gateway.git
cd laya-gateway
pip install -e .
```

On first use, the Laya model weights are downloaded from Hugging Face.

## Quick start

### Route a prompt to the right model

```python
from laya_gateway import ModelRouter

router = ModelRouter(
    models={
        "code-A": "Specialized for programming, debugging and APIs.",
        "thinking-B": "Specialized for math, logic and multi-step reasoning.",
        "creative-C": "Specialized for stories, poetry and creative writing.",
        "fast-D": "Specialized for simple factual questions and summaries.",
    }
)

result = router.select("Build a REST API using Python.")

print(result.model)       # code-A
print(result.confidence)  # e.g. 0.94
```

The keys are your own names. Map them to real providers (for example
`"code-A"` to a coding model and `"fast-D"` to a cheap model) in your app.

### Guard a prompt

```python
from laya_gateway import Guardrail, LayaEngine

guard = Guardrail(
    engine=LayaEngine(purpose="Safety guardrail for a customer-support bot"),
    review_threshold=0.60,
    block_threshold=0.80,
)

result = guard.check("Give me malware that steals credentials.")

print(result.action)      # GuardrailAction.BLOCK
print(result.blocked)     # True
print(result.reason)      # Unsafe content detected with 99.81% confidence.
```

| Laya says | Confidence | Action |
|---|---|---|
| `safe` | any | `allow` |
| not `safe` | ≥ `block_threshold` | `block` |
| not `safe` | ≥ `review_threshold` | `review` |
| not `safe` | below both | `allow` |

The guardrail fails closed: any category other than `"safe"` goes through the thresholds.
You can plug in your own backend by implementing the `DecisionEngine` protocol
(`evaluate(prompt) -> GuardrailResult`).

## Demo website

```bash
git clone https://github.com/amanjoshi2002/laya-gateway.git && cd laya-gateway
pip install -e ".[demo]"
python demo/app.py
```

Open http://127.0.0.1:7860, type a prompt, and watch the guardrail verdict and the
routed model appear along with their latency.

**Host it for free on Hugging Face Spaces:** create a new *Gradio* Space, upload
`demo/app.py` as `app.py` and `demo/requirements.txt` as `requirements.txt`.

More scripts are in [`examples/`](examples/):

```bash
python examples/route_queries.py
python examples/check_prompts.py
```

## Running tests

```bash
pip install -e ".[dev]"
pytest                      # offline unit tests (no model download)
pytest -m integration -s    # real Laya model
```

## Limitations

This is an early (0.1) release. Be honest with yourself about what a small
classifier can do:

- **Guardrail accuracy is limited.** On the prompts in
  `tests/test_laya_guardrail.py` plus some everyday prompts, the underlying model
  separates safe from unsafe prompts about 80–85% of the time. It catches clearly
  malicious requests reliably, but it can also flag harmless creative or general
  prompts. Treat it as a first-pass filter, not your only safety layer.
- **Routing is heuristic.** In a 16-prompt sanity check it picked the intended
  specialist 12 times. Clear, distinct model descriptions help a lot.
- Both components load their own copy of the model, so expect roughly 2× the memory.

Contributions that improve these numbers are very welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).

## Project structure

```
src/laya_gateway/
├── selector.py            # ModelRouter, LayaSelector
└── guardrails/
    ├── guard.py           # Guardrail (thresholds → allow/review/block)
    ├── laya.py            # LayaEngine (Laya-backed safety classifier)
    └── schemas.py         # GuardrailAction, GuardrailResult
demo/app.py                # Gradio demo website
examples/                  # runnable scripts
tests/                     # unit + integration tests
```

## Credits

- **[Laya](https://huggingface.co/convaiinnovations/laya)** by
  **[Convai Innovations](https://huggingface.co/convaiinnovations)** does all the
  decision-making in this project (Apache-2.0). Laya Gateway is an independent
  project and is not affiliated with or endorsed by Convai Innovations.
- [Pydantic](https://github.com/pydantic/pydantic) for typed results.
- [Gradio](https://github.com/gradio-app/gradio) for the demo UI.

See [NOTICE](NOTICE) for full attribution.

## License

[Apache License 2.0](LICENSE) © 2026 amanjoshi2002
