Metadata-Version: 2.2
Name: agently-devtools
Version: 0.1.8
Summary: Observation and developer tooling companion package for Agently
Author-email: Agently Team <developer@agently.tech>
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: agently<4.2.0,>=4.1.0.2
Requires-Dist: fastapi<1.0,>=0.104
Requires-Dist: httpx<0.29.0,>=0.28.1
Requires-Dist: uvicorn<1.0,>=0.30
Provides-Extra: dev
Requires-Dist: anyio<5,>=4; extra == "dev"
Requires-Dist: build<2.0,>=1.2; extra == "dev"
Requires-Dist: pytest<9.0.0,>=8.4.0; extra == "dev"
Requires-Dist: pytest-asyncio<2.0.0,>=1.0.0; extra == "dev"

# agently-devtools

`agently-devtools` is the local observation, evaluation, and playground companion package for `agently`.

## Install

```bash
pip install -U agently agently-devtools
```

Compatible with `agently >=4.1.0.2,<4.2.0`.

Compatibility resolution uses the current Agently offline manifest first, then the Agently release registry, then a bounded legacy fallback for `4.1.0.2` through `4.1.1`.

Python `>=3.10`

## Start

```bash
agently-devtools start
```

Default local address:

- Console: `http://127.0.0.1:15596/`
- Ingest: `http://127.0.0.1:15596/observation/ingest`

Explicit listener options:

```bash
agently-devtools start --host 0.0.0.0 --port 15596 --path /observation/ingest --no-open
```

## Observation Bridge

```python
from agently import Agently
from agently_devtools import ObservationBridge

bridge = ObservationBridge(
    Agently,
    app_id="your_app_id",
    group_id="your_group_id",
)
bridge.watch(Agently)
```

Explicit endpoint styles are also supported:

```python
ObservationBridge(Agently, host="127.0.0.1", port=15596, path="/observation/ingest")
ObservationBridge(Agently, endpoint="https://devtools.example.com/observation/ingest")
```

Use selective watch when you do not want to upload every observation event:

```python
bridge = ObservationBridge(
    Agently,
    app_id="your_app_id",
)
bridge.watch(agent, flow, lookup_reference)
```

The older `bridge = ObservationBridge(...); bridge.register(Agently)` style remains compatible and emits a deprecation warning. Prefer binding the Agently object or event center in the constructor, then selecting global or scoped observation with `bridge.watch(...)`.

The bridge uses a background queue and coalesces high-frequency streaming events before upload, so passive observation does not block normal request output. For short-lived scripts, call `await bridge.flush()` before exit when you need all buffered events delivered.

TriggerFlow executions automatically publish their flow definition, so the execution graph can show the full static structure, including branches that have not been hit yet.

## Scenario Evaluations

```python
from agently_devtools import EvaluationBridge, EvaluationCase, EvaluationRunner

bridge = EvaluationBridge(
    base_url="http://127.0.0.1:15596",
    app_id="your_app_id",
    group_id="your_eval_group",
)
runner = EvaluationRunner(bridge=bridge)

binding = bridge.bind_agent_factory(
    lambda: build_agent(),
    suite_id="support-suite",
    target_name="support-agent",
)

report = runner.run(
    binding,
    cases=[
        EvaluationCase(case_id="refund", input="Need a refund for a duplicate payment."),
        EvaluationCase(case_id="shipping", input="Where is my shipment now?"),
    ],
    rules=[
        lambda record: bool(record.output),
        lambda record: record.error is None,
    ],
    rounds=2,
)
```

Use the binding helper that matches your target:

- `bind_agent(...)` / `bind_agent_factory(...)`
- `bind_request(...)` / `bind_request_factory(...)`
- `bind_triggerflow(...)` / `bind_triggerflow_factory(...)`

`EvaluationBridge` already records rounds into DevTools and keeps observed run linkage automatically. You do not need an extra `ObservationBridge` just for Scenario Evaluations.
