Metadata-Version: 2.4
Name: llserve
Version: 0.2.0
Summary: CLI wrapper for repeatable llama-server profiles
License-Expression: MIT
Project-URL: Homepage, https://github.com/frank101010/llserve
Project-URL: Repository, https://github.com/frank101010/llserve
Project-URL: Issues, https://github.com/frank101010/llserve/issues
Keywords: llama.cpp,llama-server,cli
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Environment :: Console
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp>=3.10
Requires-Dist: rich>=13.0
Dynamic: license-file

# llserve

`llserve` is a small CLI wrapper around `llama-server` from the [`llama.cpp`](https://github.com/ggml-org/llama.cpp) project for repeatable launch profiles.

Repository: <https://github.com/frank101010/llserve>

Instead of remembering a long `llama-server` command line, you store the options once in a named profile and then run:

```bash
llserve run my-profile
```

## What It Does

- stores `llama-server` setups as named JSONC profile files
- generates profile templates from `llama-server --help`
- resolves profiles from local repo scope or user scope
- launches `llama-server` from a profile
- prints the exact command it would run
- checks profiles against the currently installed `llama-server`
- refreshes profile templates when `llama-server` changes
- opens profiles in `$EDITOR`

## Status

Current version: `0.2.0`

Implemented commands:

- `llserve run <profile> [-- <llama-server args...>]`
- `llserve run [--trace] <profile> [-- <llama-server args...>]`
- `llserve print-command <profile> [-- <llama-server args...>]`
- `llserve profile list [--long]`
- `llserve profile create [--local|--user] [--force] <profile> -- [llama-server args...]`
- `llserve profile check [--local|--user] <profile>`
- `llserve profile update [--local|--user] <profile>`
- `llserve profile edit [--local|--user] <profile>`

## Requirements

- Python `>=3.11`
- `llama-server` from [`llama.cpp`](https://github.com/ggml-org/llama.cpp), installed and available on `PATH`

By default `llserve` uses `llama-server` from `PATH`, but you can override that per command with:

```bash
llserve --llama-server /path/to/llama-server ...
```

## Installation

Install for regular CLI use:

```bash
uv tool install llserve
```

Verify the install:

```bash
llserve --help
```

Upgrade:

```bash
uv tool upgrade llserve
```

Uninstall:

```bash
uv tool uninstall llserve
```

`llserve` does not bundle `llama-server`. Install `llama-server` from [`llama.cpp`](https://github.com/ggml-org/llama.cpp) separately and make sure it is available on `PATH`.

For local development:

```bash
uv sync --group dev
```

Run without installing globally:

```bash
uv run llserve --help
```

## Quick Start

Create a profile from a working `llama-server` command line:

```bash
llserve profile create chat -- \
  -hf repo/model:Q4_K_M \
  -c 65536 \
  -ngl 99 \
  -b 2048 \
  -ub 512
```

Check it:

```bash
llserve profile check chat
```

Edit it:

```bash
llserve profile edit chat
```

Run it:

```bash
llserve run chat
```

Run it with HTTP tracing enabled:

```bash
llserve run --trace chat
```

See the exact underlying command:

```bash
llserve print-command chat
```

## Profiles

Each profile is stored as one `.jsonc` file.

- file name without `.jsonc` is the profile name
- profile keys are the long `llama-server` option names without leading dashes
- active values are uncommented JSON values
- inactive template entries remain commented out
- comments are generated from `llama-server --help`

Example:

```jsonc
{
  // -hf,   -hfr, --hf-repo <user>/<model>[:quant]
  // Hugging Face model repository; quant is optional, case-insensitive,
  "hf-repo": "repo/model:Q4_K_M",

  // -c,    --ctx-size N
  // size of the prompt context
  "ctx-size": "65536",

  // --port PORT
  // "port": null,
}
```

## Profile Locations

`llserve` supports two scopes.

User scope:

```text
~/.config/llserve/profiles/
```

Local scope:

```text
.llserve/profiles/
```

Local lookup walks upward from the current directory until the git repo root.

Rules:

- if exactly one matching profile exists across visible scopes, it is used
- if both local and user profiles match, lookup fails and you must disambiguate
- `--local` searches only local scope
- `--user` searches only user scope
- `profile create` defaults to user scope
- `profile create --local` requires being inside a git repository

List visible profiles:

```bash
llserve profile list
llserve profile list --long
```

## Command Behavior

### `run`

- builds `llama-server` argv from the profile, then appends CLI passthrough args
- duplicate options between profile and CLI are allowed
- final argument order is preserved
- `llama-server` decides last-one-wins behavior
- Ctrl+C exits cleanly without a Python traceback
- `--trace` runs `llama-server` behind a local logging proxy and writes traced HTTP traffic into `./llserve-trace/<session>/`
- trace mode rewrites the child server to listen on `configured-port + 10000`
- trace mode fails with an error if that target port would exceed `65535`

### `print-command`

- prints the exact shell-ready command line that `run` would execute

### `profile create`

- creates a new generated profile template
- accepts the same `llama-server` options after `--`
- short flags like `-hf` and `-c` are accepted
- stores canonical long-form keys in the profile file
- rejects duplicate seed options during creation
- refuses to overwrite unless `--force` is passed

### `profile check`

- validates the profile against the currently installed `llama-server`
- prints `ok` on success
- reports all detectable compatibility problems on failure

### `profile update`

- regenerates the template/comments from current `llama-server --help`
- preserves active supported values
- preserves obsolete active values as clearly commented-out entries
- removes obsolete inactive template entries

### `profile edit`

- opens the resolved profile in `$EDITOR`
- runs `profile check` after the editor exits successfully

## Logging And Diagnostics

Verbose mode:

```bash
llserve -v ...
```

Verbose mode currently logs profile resolution paths.

Errors use structured logging in this format:

```text
ERROR(path:line:column): message
ERROR: message
```

Path formatting:

- local profiles use relative paths
- user profiles use `~/...`

## Caveats

- comments in generated profile files are tool-owned and may be replaced by `profile update`
- `profile create` validates what it can parse from `llama-server --help`, but some prose-only constraints are still enforced only by `llama-server`
- create-time duplicate options are rejected even though runtime duplicate options are allowed

## Development

Run tests:

```bash
uv run --group dev pytest
```

Run lint:

```bash
uv run ruff check .
uv run ruff format --check .
```

## License

MIT
