Metadata-Version: 2.5
Name: getvda-evaluator
Version: 0.1.0
Summary: Governance evaluation for AI agents — gate every tool call against signed governance. LangChain DeepAgents middleware included.
Project-URL: Homepage, https://getvda.ai
Project-URL: Documentation, https://getvda.ai/blocks/acp.html
Project-URL: Source, https://github.com/getvda-ai/acp-agent/tree/main/packages/evaluator-py
Project-URL: Changelog, https://github.com/getvda-ai/acp-agent/releases
Author-email: Verified Digital Agents <hello@getvda.ai>
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: agent,ai-governance,audit,compliance,deepagents,eu-ai-act,guardrails,langchain,middleware
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Legal Industry
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Typing :: Typed
Requires-Python: >=3.11
Provides-Extra: deepagents
Requires-Dist: deepagents>=0.2; extra == 'deepagents'
Requires-Dist: langchain>=1.0; extra == 'deepagents'
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# getvda-evaluator

Governance evaluation for AI agents, in Python — **an entry point into the existing
[getvda.ai](https://getvda.ai) framework, not a parallel one.**

It calls the same services an `@getvda/evaluator-sdk` user calls: ACP for the signed
bundle, Witness for the seal, HITL for escalation. An agent governed through this path
is governed by identical machinery. Nothing here re-implements policy.

## LangChain DeepAgents middleware

```python
from deepagents import create_deep_agent
from getvda_evaluator.middleware import build

agent = create_deep_agent(
    tools=[...],
    middleware=[build(
        environment="ref-env",
        bundle_source=fetch_signed_bundle,     # ACP
        capability_map={"send_email": "notify_guest"},
        sealer=seal_to_witness,                # Witness
    )],
)
```

It binds `wrap_tool_call`, not `wrap_model_call`. Governance gates **actions**: a model
deciding to do something is not yet doing it, and an evidence trail of intentions is
not an evidence trail of conduct.

## Two behaviours worth knowing

**Fail-static.** If governance cannot be fetched or verified, the middleware **denies**.
An agent that keeps acting when its rules are unreachable is ungoverned while appearing
governed, which is worse than being stopped.

**An unmapped tool is denied, not exempt.** Tools with no `capability_map` entry use
their own name, fail to match a capability, and are refused. A tool does not escape
governance because nobody wrote a mapping line. Use `ungoverned_tools` to exempt
deliberately, in writing.

## Divergence is the risk this package carries

There are now two implementations of the same decision. Both are tested against
`packages/evaluator-conformance/cases.json` — one contract, one set of verdicts,
checked in CI on every change.

**Change the contract before changing either implementation, never after.** If they
disagree you have two answers to "was this action allowed", and an evidence product
with two answers has none.

That gate earned its place on its first run: this port had invented a SKILL.md format
(markdown headings) that the real parser rejects. The suite caught it before it could
grant permissions nobody had written down.

## Upstream drift

The middleware binds to LangChain's `wrap_tool_call` / `awrap_tool_call`, which live in
`libs/langchain_v1` — a v1 rewrite still in flight. `tests/test_api_drift.py` pins that
surface and a weekly CI job runs it, because a middleware that stops intercepting still
looks installed.

The drift test skips when LangChain is absent: the evaluation core is deliberately
testable without an agent framework, since it is the part that must not drift.

```sh
pytest packages/evaluator-py                       # core + conformance
pip install -e '.[deepagents]' && pytest           # + upstream API check
```
