Metadata-Version: 2.5
Name: taskmaster-python
Version: 0.1.0
Summary: Client library that reports queue metrics to the Taskmaster autoscaling service.
Author-email: "Zagaran, Inc." <theo@zagaran.com>
License-Expression: MIT
License-File: LICENSE
Keywords: autoscaling,celery,metrics,queue,redis
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Celery
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: System :: Monitoring
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: requests>=2.25
Provides-Extra: celery
Requires-Dist: celery[redis]>=5.0; extra == 'celery'
Description-Content-Type: text/markdown

# taskmaster-python

Client library for Taskmaster, an autoscaling service. It runs a background
thread inside your app that periodically collects queue metrics and reports them
to the Taskmaster API.

Currently supported backends:

- **Celery with a Redis broker**, reporting per queue:
  - `queue_depth`: number of jobs waiting.
  - `queue_latency`: seconds the oldest waiting job has been on the queue (0 when empty).

## Installation

```bash
pip install "taskmaster-python[celery]"
```

## Configuration

| Environment variable         | Required | Description                                          |
| ---------------------------- | -------- | ---------------------------------------------------- |
| `TASKMASTER_URL`             | yes      | Base URL of the Taskmaster API.                      |
| `TASKMASTER_AUTH_TOKEN`      | yes      | Sent as `Authorization: Api-Key <token>`.            |
| `TASKMASTER_REPORT_INTERVAL` | no       | Seconds between reports. Defaults to `10`, max `30`. |

If `TASKMASTER_URL` is unset, reporting is disabled and nothing runs. That makes
local development need no setup.

## Usage (Celery)

Call `taskmaster_celery` once, after your Celery app is configured:

```python
from celery import Celery
from taskmaster_python.celery import taskmaster_celery

app = Celery("proj")
app.config_from_object("proj.celeryconfig")

taskmaster_celery(app)
```

This starts a daemon thread in the current process. Every interval it measures
the depth and latency of each queue and POSTs a report to
`TASKMASTER_URL/v1/report_metric`. Errors are logged to the `taskmaster_python`
logger and the thread keeps running.

It also connects a `before_task_publish` signal handler. The handler stamps each
task with its publish time in a `taskmaster_published_at` message header. Queue
latency is measured from that stamp, so **call `taskmaster_celery` in every
process that enqueues tasks** (e.g. your web server), with `TASKMASTER_URL` set.
If the oldest job in any of a queue's priority levels has no stamp, that queue's
latency is left out of the report. Its depth is still reported. The stamp uses
the publishing host's clock, so keep hosts NTP-synced.

Queues are discovered from `task_queues`, `task_default_queue`, and dict-style
`task_routes`. If you route tasks with a custom router function, list the queues
explicitly:

```python
taskmaster_celery(app, queues=["default", "emails", "reports"])
```

Multiple apps can be registered. They share one thread and one report per interval.

## Report format

```json
POST /v1/report_metric
Authorization: Api-Key <TASKMASTER_AUTH_TOKEN>

{
  "metrics": [
    {"queue": "celery", "type": "queue_depth", "value": 12},
    {"queue": "celery", "type": "queue_latency", "value": 4.82},
    {"queue": "emails", "type": "queue_depth", "value": 0},
    {"queue": "emails", "type": "queue_latency", "value": 0.0}
  ]
}
```

## Adding a backend

1. Subclass `taskmaster_python.Collector` and implement `collect()` to return a
   list of `Metric` objects.
2. Register it with `taskmaster_python.reporter.add_collector(...)` and call
   `reporter.ensure_running()`, typically from a setup function in a new
   `taskmaster_python.<backend>` subpackage.
3. Add the backend's dependencies as an optional extra in `pyproject.toml`.

New metric kinds subclass `taskmaster_python.Metric` and set a `type` marker.

## License

MIT