Metadata-Version: 2.4
Name: thrdi
Version: 0.2.2
Summary: Trace any agentic CLI to a unified local store, queryable from the CLI.
Author: Duncan McKinnon
License: MIT
Project-URL: Homepage, https://github.com/duncankmckinnon/thirdeye
Project-URL: Issues, https://github.com/duncankmckinnon/thirdeye/issues
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: POSIX
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Topic :: Software Development
Classifier: Topic :: Utilities
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: msgpack>=1.0
Requires-Dist: zstandard>=0.22
Requires-Dist: pyaml>=23.0
Requires-Dist: tomli>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.5; extra == "dev"
Requires-Dist: pre-commit>=3.7; extra == "dev"
Requires-Dist: ruff>=0.7; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Provides-Extra: ui
Requires-Dist: starlette>=0.36; extra == "ui"
Requires-Dist: uvicorn>=0.27; extra == "ui"
Requires-Dist: jinja2>=3.1; extra == "ui"
Requires-Dist: python-multipart>=0.0.20; extra == "ui"
Provides-Extra: logfire
Requires-Dist: logfire[datasets]>=4.25; extra == "logfire"
Requires-Dist: genai-prices>=0.1.4; extra == "logfire"
Dynamic: license-file

<p align="center">
  <img src="docs/img/logo_transparent.png" alt="thirdeye" width="160" />
</p>

[![PyPI](https://img.shields.io/pypi/v/thrdi.svg)](https://pypi.org/project/thrdi/)
[![Homebrew](https://img.shields.io/badge/homebrew-duncankmckinnon%2Ftap-orange)](https://github.com/duncankmckinnon/homebrew-tap)
[![CI](https://github.com/duncankmckinnon/thirdeye/actions/workflows/test.yml/badge.svg)](https://github.com/duncankmckinnon/thirdeye/actions/workflows/test.yml)
[![codecov](https://codecov.io/gh/duncankmckinnon/thirdeye/branch/main/graph/badge.svg)](https://codecov.io/gh/duncankmckinnon/thirdeye)
[![Python](https://img.shields.io/pypi/pyversions/thrdi.svg)](https://pypi.org/project/thrdi/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

Trace every agent session on your machine — Claude Code, Codex, Cursor, GitHub Copilot CLI — into one history you and your agents can manage, search, and evaluate.

## Install

> **Windows support is experimental.** The test suite runs on Windows in CI, and
> Claude Code tracing is the verified integration there. The Codex CLI, Cursor,
> and Copilot CLI installers are implemented but have not been live-certified
> against those tools on Windows. Please report Windows problems at the
> [issue tracker](https://github.com/duncankmckinnon/thirdeye/issues). See
> [docs/windows.md](docs/windows.md) for the deliberate platform differences.

The easiest installation approach is to use [homebrew](https://brew.sh/). This will package all thirdeye extras as a globally available CLI tool. Homebrew is
macOS / Linux only — on Windows, install with `pipx` or `uv` (below).

```bash
brew install duncankmckinnon/tap/thirdeye    # macOS / Linux
```

Alternatively, you can get the same result with [pipx](https://pipx.pypa.io/stable/)

```bash
pipx install 'thrdi[ui,logfire]' # global install
```

For local installation, you can also use `uv` or `pip` with the optional ui and logfire extras that add the ability to navigate thirdeye as a webpage and export traces via OpenTelemetry to Pydantic Logfire.

```bash
uv tool install thrdi # or: uv tool install 'thrdi[ui,logfire]'
```

```bash
pip install thrdi # or: pip install 'thrdi[ui,logfire]'
```

## Interactive setup

Configure tracing, bundled agent skills, and optional Pydantic Logfire export in
one guided flow:

```bash
thirdeye setup
```

The wizard prompts for each supported agent, installs skills in the matching
agent directories, and can sign in to Logfire in the browser to mint a write
token if you choose to enable remote export. The commands below remain available
when you want to configure each piece separately.

## Install agent skills locally

Install thirdeye's bundled skills for agents working in the current project:

```bash
thirdeye skills list                                 # show bundled skill names
thirdeye skills add                                  # all skills → .agents/skills/
thirdeye skills add --claude --codex                 # both agent-specific folders
thirdeye skills add -p path/to/folder                # custom parent folder
thirdeye skills add --only thirdeye-review           # just one
thirdeye skills add --force                          # replace existing entries
```

On macOS and Linux, skills install as symlinks, so upgrading thirdeye
(`brew upgrade thirdeye` or `pipx upgrade thrdi`) automatically refreshes them.
On Windows without Developer Mode, skills are copied instead; after upgrading,
rerun `thirdeye skills add --force` to refresh them. See
[Windows support](docs/windows.md#2-copied-skills) for details.

## Enable tracing

```bash
thirdeye add --claude        # also: --cursor, --codex, --copilot
```

To detach: `thirdeye remove --claude` (also `--cursor`, `--codex`, `--copilot`).

## Copilot CLI capture

GitHub Copilot CLI capture keeps a durable, local V1 archive and derives V2
views from that archive. V1 source records are immutable: V2 never rewrites
them, and it can rebuild its derived turns, usage accounting, and export queue
after the original Copilot files have gone away.

```bash
thirdeye add --copilot
thirdeye copilot status --source-home "$COPILOT_HOME"
thirdeye copilot sync --source-home "$COPILOT_HOME"
thirdeye copilot reconcile
thirdeye copilot reconcile --session-id <stored-session-id> --rebuild
thirdeye copilot watch --source-home "$COPILOT_HOME" --interval 1
thirdeye remove --copilot
```

`--source-home` is optional. Resolution is `--source-home`, then `COPILOT_HOME`,
then `~/.copilot`. Capture reads only that home's `session-state/**/events.jsonl`,
`workspace.yaml`, and `session-store.db` (`sessions`, `turns`,
`assistant_usage_events`). It does not read credentials, token-bearing config,
or other Copilot files.

`thirdeye add --copilot` writes user-level hooks at
`$COPILOT_HOME/hooks/thirdeye.json` (default `~/.copilot/hooks/thirdeye.json`).
The 1.0.83 live probe used repository hooks; user-level installation is not a
claim that every Copilot runtime invokes them. `watch` is an explicit foreground
poller and is not started by add or setup. It can import transcripts and SQLite
rows even when hooks are absent. `sync` and `status` print recoverable source
diagnostics (locations and reasons, not prompt bodies).

Normal `sync` and `reconcile` refresh local V2 projections only; they do not
initialize export eligibility. To mark already-completed retained history as
export-eligible, opt in explicitly:

```bash
thirdeye copilot sync --export --source-home "$COPILOT_HOME"
thirdeye copilot reconcile --export
```

Watch and hook follow-up activate live-style export without that history
opt-in. Their first activation records a durable eligibility boundary even if
remote export is not presently configured: interactions already complete stay
local-only, while interactions still open at activation may be queued later if
they complete and remote export is configured. `--export` instead activates
with retained completed history included, or later removes those identities
from an existing boundary. Jobs are dispatched only when remote export is
configured. `--rebuild` resets only reproducible derived state; it preserves
the raw archive and the separate delivery ledger. See
[Copilot CLI capture and reconciliation](docs/copilot-capture.md) for the
archive schema, attribution rules, correction behavior, and delivery limits.

Passing unit tests is not live certification of Copilot CLI, native VS Code, or
Copilot cloud integrations.

## Read your history

```bash
thirdeye list                          # every session, every platform
thirdeye events <id>                   # one session, terse
thirdeye tail <id> -n 5                # last few events
thirdeye event <id> <seq>              # one event, fully expanded
thirdeye search "migration"            # substring across all sessions
thirdeye stats                         # totals
```

## Tag and filter

```bash
thirdeye tag <id> <seq> --add bug,review     # tag an event
thirdeye tag <id> --list                     # list tagged events in a session
thirdeye tag <id> <seq> --remove bug         # untag
thirdeye tags                                # global tag inventory
thirdeye search "migration" --tag review --platform claude --since 2026-05-01
```

Add `--json` for parseable JSONL, `--tree` for human-readable, `--platform` / `--cwd` / `--tag` / `--since` / `--until` to filter. Session IDs accept any unique prefix. Run `thirdeye --help` for the full reference.

## Per-turn usage

thirdeye captures model name and token counts per turn into an append-only
sidecar (`usage.jsonl`) and a global SQLite index (`usage.db`). Capture starts
automatically on the next agent run after `thirdeye add`.

```bash
thirdeye usage                          # global rollup, sessions by token spend
thirdeye usage <id>                     # per-turn detail for one session
thirdeye usage --top 5 --since 2026-05-01
thirdeye usage <id> --json              # parseable JSONL rows
thirdeye usage reindex                  # rebuild SQLite from sidecars
thirdeye usage errors                   # tail the capture audit log
```

Filters: `--platform` / `--harness`, `--model SUBSTR`, `--since` / `--until`,
`--top N`, `--sort total|input|output|ts`.

## Export to Pydantic Logfire

Mirror every captured session into [Logfire](https://pydantic.dev/logfire) live, as traces — no separate export step. Once enabled, each thirdeye session becomes one Logfire trace: tool calls appear as spans with real durations (paired from `PreToolUse`/`PostToolUse`, or Codex's `call_id`), everything else (messages, notifications, compaction, ...) as timeline markers, all searchable by `gen_ai.conversation.id`.

On Claude Code, each individual model call within a turn gets its own `chat <model>` span. Codex reconstructs calls from its rollout JSONL. Cursor reconstructs each generation from its IDE/CLI hooks, pairing shell and MCP callbacks and recording file and generic tool events. All three use OpenTelemetry GenAI semantic conventions for agent invocation, chat, tool execution, messages, models, and token/cache usage; no OpenInference span taxonomy is used.

Within a trace, subagents dispatched in local Cursor IDE and CLI sessions are exported beneath their dispatching `Task` span — including background, parallel, and nested children — and each child's tool calls are attributed to it by the child hook generation derived from its Task call identity, never by timing, nearest-turn heuristics, or tool-completion order.

```bash
thirdeye logfire enable                                           # reuse a saved token, or sign in to mint one
thirdeye logfire enable --auth                                    # force a fresh Logfire login, then mint
thirdeye logfire status
thirdeye logfire disable                                          # keeps the saved key
```

Or from `thirdeye ui`, under **settings**: paste the gateway key and hit Enable — persisted the same way, in `~/.thirdeye/config.yaml`.

Export is dispatched from the same Claude Code, Codex, and Cursor hooks that already capture events, but the actual Logfire call (including a flush, a real network round trip) runs in a detached background process — the hook itself never waits on the network, so enabling this adds no network latency to your tool calls.

From the sessions page, you can also send the currently filtered sessions to
Logfire as a named managed dataset. Configure a separate project API key with
`project:write_datasets` scope under **Settings → Pydantic Logfire**, apply the
session filters you want, enter a dataset name, and choose **Send to Logfire**.
Each session becomes one case containing its metadata and ordered event stream.
The managed-datasets feature must be enabled for the Logfire project.
Choose **one case per turn** to export each captured user-to-assistant turn as
its own case. A turn-content query searches every turn in every session selected
by the broader filters; comma-separated terms are ANDed within the same turn.
An optional exact selector in the form `<session-id>:<platform-turn-id>` remains
available for direct lookup.

### Exporting environment context

Set `THIRDEYE_CAPTURE_ENV=WB_*` and individual variables such as `WB_PLAN`,
`WB_AGENT`, `WB_TASK`, `WB_STEP`, and `WB_WAVE` on the agent process. Thirdeye
captures matching variables into each export job and attaches them to every
exported span for Claude Code, Codex, and Cursor. For example:

```sh
THIRDEYE_CAPTURE_ENV='WB_*' WB_PLAN=my-plan WB_TASK=task-1 claude
```

The patterns can also be persisted so they do not depend on the launching
shell exporting `THIRDEYE_CAPTURE_ENV` (an agent dispatched from a context
that never sources your shell rc would otherwise capture nothing):

```sh
thirdeye capture-env set 'WB_*'    # writes capture_env to ~/.thirdeye/config.yaml
thirdeye capture-env show          # what is active, and where it came from
```

`THIRDEYE_CAPTURE_ENV` still overrides the persisted value when set, so a
one-off run can change it.

Each captured name becomes a dotted span-attribute key — lowercased, with
`_` mapped to `.` — so `WB_PLAN` is `wb.plan` and `BUILD_LABEL` is
`build.label`. No pattern is privileged; the transform is uniform. Values
remain strings with their original case and contents;
nonempty values are also exported directly as Logfire tags on the top-level
session span only, with duplicates removed. Child spans retain the named
attributes without repeating these session tags. For example,
`WB_PLAN=my-plan` adds the `my-plan` tag alongside `wb.plan="my-plan"`.
Logfire tags preserve the original case and contents. Local session tags
retain their existing key prefixes, sanitization, and 64-character limit.
Existing span attributes take precedence on a name collision.

Patterns match variable names, not fields inside a composite value. This path
requires no `OTEL_RESOURCE_ATTRIBUTES`, which was observed to be absent from
Claude Code hook environments. Workbench must supply the individual `WB_*`
variables in each child's environment; changing thirdeye alone cannot extract
values the launcher does not supply. Export jobs snapshot context before
worker dispatch, keeping concurrent agents' values separate. Only opt in
variables you intend to export to Logfire.

## Browse in a browser

For a richer experience than the CLI, install the UI extra and launch:

```bash
thirdeye ui      # or: thirdeye serve
```

The local browser UI covers:

- **Sessions list** with platform / cwd / status / date filters and a
  tag multi-select drawn from every tag in your history, defaulting to
  the last 7 days, newest first.
- **Ask panel** — type "find sessions about the workbench plan" or
  "long-lasting claude runs this week" and a CLI agent of your choice
  (claude / codex) auto-fills the filter form. Review the
  populated fields and hit Search / Filter to run.
- **Saved filter views** — name a filter combination and pin it to the
  sidebar; restored across browser sessions via local storage.
- **Session view** — collapsible event tree color-coded by event type,
  inline tag editing, live-tail via Server-Sent Events for open sessions.
- **Evals** — author and edit YAML rubrics; dispatch a run on one
  session or a batch on a selection. Two complementary tables: per-
  definition cross-cut (`/evals/defs/<name>/results`) for comparing a
  rubric across sessions, and a per-(session, definition) panel showing
  the directive text plus every run on that session with parsed verdict
  and score columns.
- **Usage charts** — daily tokens-over-time and sessions-per-day with a
  platform filter and totals cards.

The server binds loopback only.

See [docs/ui.md](docs/ui.md) for full reference.

## Evaluations

Grade a recorded session by dispatching one of your installed CLI agents
(claude / codex) as an LLM-as-judge. Eval definitions are named
rubrics — directive text shipped with sensible defaults and editable per-user.

```bash
thirdeye eval def list                                          # available rubrics
thirdeye eval def show default                                  # see the directive
thirdeye eval def create my-rubric --directive "<text>"         # custom rubric

thirdeye eval run <id> --agent claude                           # foreground
thirdeye eval run <id> --agent codex --using token-efficiency --background

thirdeye eval show <id>                                         # latest result
thirdeye eval list --since 2026-05-01 --verdict warn            # history
thirdeye eval status                                            # background jobs
```

Per-turn findings are stored with the event `seq` they anchor to, and
`thirdeye events <id>` annotates the timeline inline by default (suppress with
`--no-findings`, filter with `--eval NAME`). The eval invocation itself is a
thirdeye-traced session, so every grading run has its own audit trail.

Dispatched agents run in read-only mode (Claude `--allowedTools` allowlist,
Codex `--sandbox read-only`). No new Python
deps — thirdeye shells out to the agent binaries you already have installed.

## Agent

Dispatch an AI agent (Claude Code or Codex) against your thirdeye
history directly from the CLI. The agent is pre-loaded with its analysis and
evaluation skills and runs in read-only mode by default.

```bash
thirdeye agent "review my sessions from the last week"
thirdeye agent "find sessions where token usage spiked" --stream
thirdeye agent "fix inefficient tool use in session abc123" --fix
thirdeye agent "summarize eval findings" --agent codex
```

Flags:

| Flag | Description |
|------|-------------|
| `--stream` | Print tool calls and results in real time as the agent explores |
| `--fix` | Unlock full tool access so the agent can edit files (default: read-only) |
| `--agent NAME` | Agent to dispatch: `claude` (default) or `codex` |
| `--skill PATH` | Inject an additional skill from a local file (repeatable) |
| `--skills` | List the built-in skills and exit |
| `--cwd PATH` | Working directory context injected into the prompt |

New sessions opened by the agent are automatically tagged `thirdeye-agent`
so you can filter them with `thirdeye list --tag thirdeye-agent`.

### Skills used by `thirdeye agent`

Four bundled skills are injected into every `thirdeye agent` run by default:

- **`use-thirdeye`** — basic CLI fluency: enable tracing, search sessions,
  debug tool calls, analyze token usage.
- **`thirdeye-evals`** — eval workflow: create rubrics, dispatch
  evaluators, view per-turn findings.
- **`thirdeye-review`** — audit other agents' traces to find
  inefficiencies and propose invocation, skill, or convention changes
  (cohort scoping, tool-mix patterns, token spikes, recommendation
  templates).
- **`thirdeye-filter`** — directive used by the browser UI's Ask panel
  to translate natural-language queries into filter JSON. Installed
  alongside the others; not invoked directly by agents.

Pass `--skill path/to/skill.md` to inject additional skills from local files
alongside the defaults. Run `thirdeye agent --skills` to see the default list.

## License

MIT.
