Metadata-Version: 2.3
Name: agentdojo
Version: 0.1.0
Summary: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents
Author: Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner
Author-email: Edoardo Debenedetti <edoardo.debenedetti@inf.ethz.ch>
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: anthropic>=0.28.0
Requires-Dist: click>=8.1.7
Requires-Dist: cohere>=5.3.4
Requires-Dist: deepdiff>=7.0.1
Requires-Dist: docstring-parser>=0.15
Requires-Dist: jsonref>=1.1.0
Requires-Dist: langchain>=0.1.17
Requires-Dist: openai>=1.0.0
Requires-Dist: openapi-pydantic>=0.4.0
Requires-Dist: pydantic[email]>=2.7.1
Requires-Dist: pyyaml>=6.0.1
Requires-Dist: rich>=13.7.1
Requires-Dist: tenacity>=8.2.3
Requires-Dist: typing-extensions>=4.11.0
Requires-Dist: vertexai>=1.49.0
Provides-Extra: docs
Requires-Dist: black>=24.4.2; extra == 'docs'
Requires-Dist: mkdocs-material>=9.5.26; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25.1; extra == 'docs'
Provides-Extra: transformers
Requires-Dist: torch>=2.2.1; extra == 'transformers'
Requires-Dist: transformers>=4.41.2; extra == 'transformers'
Description-Content-Type: text/markdown

# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents

## Quickstart

```bash
pip install agentdojo
```

If you want to use the prompt injection detector, you need to install the `transformers` extra:

```bash
pip install "agentdojo[transformers]"
```

## Running the benchmark

The benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.

For example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:

```bash
python -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \
    -ut user_task_1 --model gpt-4o-2024-05-13 \
    --defense tool_filter --attacker tool_knowledge
```

To run the above, but on all suites and tasks, run the following:

```bash
python -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \
    --defense tool_filter --attacker tool_knowledge
```

## Documentation of the Dojo

Take a look at our [documentation](https://agentdojo.spylab.ai/).

## Development set-up

Take a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.
