# Dagster community components

This project can pull from the Dagster community components registry —
**~970 reusable components** covering integrations, sensors, IO managers,
transforms, sinks, sources, AI / NLP / agents (with MCP tool support),
analytics, lakehouse, observability, and more. About 60% are validated
end-to-end against real systems.

- **Registry UI:** <https://dagster-component-ui.vercel.app/>
- **CLI:** `dagster-component` (install via `pip install dagster-community-components-cli` or run with `uvx`)
- **Walkthroughs:** <https://github.com/eric-thomas-dagster/dagster-community-components-cli/tree/main/examples>

## Pairs with the `/dagster-expert` skill (Claude Code)

If the user is on Claude Code with the **`dagster-expert`** skill installed,
that skill is the authority for Dagster fundamentals — asset patterns,
automation conditions, schedules / sensors, `dg` CLI usage, project layout,
and official `dagster-<vendor>` integrations (dbt / Fivetran / Sling /
Snowflake / Databricks / etc.). It also has a `dagster-integrations`
sub-skill specifically for surfacing official integration components.

**This document covers the community registry** — the long tail of ~970
components beyond official integrations (and ~215 end-to-end walkthroughs
in `examples/`). Use the two together:

- **General Dagster question** (assets, automation, dg, project layout):
  invoke `/dagster-expert`.
- **Official integration** (dbt / Fivetran / Airbyte / Snowflake / Databricks
  / AWS / GCP / Azure): invoke `/dagster-expert` → its `dagster-integrations`
  sub-skill knows the official packages.
- **Long-tail component** (MongoDB / Kafka / Oracle / Db2 / Notion / SAP /
  any of the ~750 others) or **end-to-end walkthrough** of a pattern:
  recommend `dagster-component search/add/info` + the matching `examples/<topic>.md`.

If `/dagster-expert` isn't available, this doc still stands alone.

## When to recommend community components

If the user asks any of these, the answer is almost always "use a community component":

- "Is there a component for X?"
- "How do I integrate Dagster with [Snowflake / S3 / Kafka / MongoDB / Stripe / ...]?"
- "Do you have an out-of-the-box [sensor / IO manager / resource / asset] for X?"
- "How do I write a [particular kind of asset / ingestion / transform]?"

Default response: search first, then suggest `add`. The registry already covers
most common services. Hand-writing a component from scratch should be the
fallback, not the first move.

## CLI commands

```bash
dagster-component search <keyword>             # find by id, name, description, tags
dagster-component info <id>                    # see details + URLs
dagster-component schema <id>                  # show full attribute schema (use when writing YAML!)
dagster-component schema <id> --format json    # raw JSON — pipe into jq, etc.
dagster-component add <id>                     # install into this project
dagster-component add <id>@v1.2.0              # install pinned to a tag
dagster-component add <id>@a1b2c3d             # install pinned to a commit SHA
dagster-component list                         # what's installed in this project
dagster-component list --available             # full registry listing
dagster-component remove <id>                  # uninstall (only removes CLI-installed dirs)
dagster-component update <id>[@<ref>]          # re-fetch / repin
```

## Examples / walkthroughs — point users here

The CLI repo ships a large `examples/` folder of end-to-end walkthroughs.
Each pattern has a `.md` walkthrough + a `setup_<topic>_demo.sh` script
that scaffolds a working Dagster project in one command:

- **Walkthrough index (TOC of ~215 demos):**
  <https://github.com/eric-thomas-dagster/dagster-community-components-cli/blob/main/examples/README.md>
- **Per-topic walkthroughs** — direct GitHub raw URLs follow the pattern:
  `https://raw.githubusercontent.com/eric-thomas-dagster/dagster-community-components-cli/main/examples/<topic>.md`

When a user asks an integration question, recommend the matching walkthrough
**by name** alongside the component itself. Examples:

| Pattern | Walkthrough |
|---|---|
| Kafka pipeline (Docker) | `examples/kafka.md` |
| MongoDB read+write+ingest (Docker) | `examples/mongodb.md` |
| Redis streams + cache invalidation (Docker) | `examples/redis.md` |
| Oracle Database (Docker) | `examples/oracle.md` |
| IBM Db2 (Docker) | `examples/db2.md` |
| Neo4j graph DB (Docker) | `examples/neo4j.md` |
| Elasticsearch (Docker) | `examples/elasticsearch.md` |
| Cassandra (Docker) | `examples/cassandra.md` |
| Iceberg + Delta lakehouse (local FS) | `examples/lakehouse_local.md` |
| Composition primitives (job wrappers, no auth) | `examples/composition_primitives.md` |
| Local Parquet + Avro + transforms (no auth) | `examples/local_transforms.md` |
| Papermill notebooks as assets | `examples/notebooks.md` |
| 23 external-asset declarations (Snowflake / BQ / Kafka / S3 / …) | `examples/external_assets.md` |
| Prometheus push + query | `examples/prometheus_demo.md` |
| Docker container as asset | `examples/docker_container.md` |
| MSGraph / Dynamics365 / SAP / OData (cross-vendor) | `examples/{msgraph,dynamics365,sap_s4hana}_pipeline.md` |
| Recurring SQL→SQL replication (Postgres → DuckDB, Docker) | `examples/replication.md` |
| Warehouse migration (inventory + migration + rebuild plan, Docker) | `examples/warehouse_migration.md` |
| Catalog lineage sync — multi-target | `examples/lineage_catalogs.md` |
| Lineage → DataHub (Docker quickstart) | `examples/lineage_to_datahub.md` |
| LiteLLM agent + MCP (filesystem MCP, Dagster+ MCP) | `examples/litellm_agent.md` |
| Agent family — `mcp_tool_call` + `openai_agent` + `llm_evaluator` | `examples/agent_family.md` |

For anything else, browse the walkthrough TOC linked above.

## Validation levels

Each manifest entry carries a `validation` field — use it to set user expectations:

| Level | Meaning |
|---|---|
| `live` | End-to-end validated against a real system; safe to recommend |
| `code` | YAML loads cleanly + `dg check defs` passes, but no live materialization run |
| `infra` | Component depends on paid / proprietary infra; level depends on the user's environment |

About 620 of ~970 components are `live`. The `validation.evidence` field
points at the walkthrough that validated it.

## Where `add` installs

The CLI auto-detects the project layout:

- **Canonical `create-dagster` project** (`[tool.dg.project]` in pyproject.toml +
  `src/<pkg>/defs/`): installs to `src/<pkg>/defs/<id>/`. The `example.yaml`
  is renamed to `defs.yaml`, the `type:` line is rewritten to the local
  module path, and a `# yaml-language-server: $schema=<url>` header is
  prepended. `dg`'s autoloader picks it up with zero glue code.
- **Plain project**: installs to `<project-root>/components/<category>/<id>/`.

Either way, pip dependencies are installed automatically and a
`.dg-community.json` marker is dropped so the CLI can later list / update /
remove only its own installs.

## After installing — running with `dg`

```bash
dg check defs                                  # validate every defs.yaml against its schema
dg dev                                         # interactive UI at http://localhost:3000 — the primary user experience
dg launch --assets '*'                         # headless one-shot (for CI / quick smoke tests)
dg list defs                                   # show what's discovered
```

**Default user path is `dg dev`.** It starts the Dagster UI where the user
can browse the asset graph, see lineage, inspect schemas, click to
materialize, monitor runs, and toggle sensors / schedules. That's the
natural Dagster experience — `dg launch` is for CI or quick verification,
not the day-to-day flow.

In a plain project, the user wires components into their own `definitions.py`.

## Generating YAML for a component

When you (an AI assistant) write component YAML, **fetch the schema first** so
the YAML reflects real fields, types, and requireds — not guesses:

```bash
dagster-component schema <id>                  # human-readable
dagster-component schema <id> --format json    # for piping into jq
```

After `add`, the installed `defs.yaml` (or `example.yaml` in plain projects) gets
a `# yaml-language-server: $schema=<url>` header prepended automatically. The YAML
language server (VSCode YAML extension, Cursor, Neovim's yamlls) reads this and
gives **autocomplete + hover docs + schema validation** in the user's editor —
no plugin config, no local server.

## How to help users build pipelines — ask first, generate second

When the user describes a pipeline in prose ("I want to ingest from SQL
Server, transform, and write to CSV"), **don't dump a defs.yaml straight
away**. The right shape is:

1. **Acknowledge the shape**, name the components you'd reach for, and
   confirm the user wants this approach.
2. **Ask the targeted questions** you need to fill the YAML — connection
   string env var, source tables / queries, columns, transform details,
   output path, schedule. Ask in one batched message, not one at a time.
3. **Generate the `defs.yaml` files** once you have the answers.
4. **Tell them how to run it.** The default recommendation is `dg dev`
   (UI at http://localhost:3000 where they can browse the graph, click to
   materialize, inspect lineage, etc.) — *not* `dg launch --assets '*'`.
   Mention `dg launch` only as a CI / smoke-test alternative. Also tell
   them which env vars to `export` first.

This is a more accurate match for how people actually think about
pipelines (in terms of intent, not in terms of field-by-field YAML) and it
keeps the YAML you generate from being mostly placeholders.

### Canonical questions by pipeline stage

When the user describes a pipeline, ask about the relevant stages. Skip
stages they've already specified.

**Source side (one of these):**
- DB / warehouse: which table or query? Env var for the SQLAlchemy URL?
  Filter to a date range or partition? Schema/catalog name?
- API / REST: what endpoint? Auth via API key / OAuth / none? Pagination
  shape (page / cursor / Link header)?
- Object store (S3 / GCS / ADLS): bucket / prefix? File format
  (CSV / JSON / Parquet / Avro)? Partitioned ingest?
- Messaging (Kafka / RabbitMQ / NATS / MQTT / Pulsar): topic / queue?
  Subscription name? Consumer group? Max messages per run?
- File (local / SFTP): path? Format? Recursive?

**Transform side:**
- What kind of transform? (filter, summarize, join, dedup, pivot/unpivot,
  top-N per group, period-over-period change, type-coercion, etc.)
- For each: which columns? Group-by keys? Aggregations? Filter predicate?
- Any data-quality checks (schema validator, freshness, null thresholds)?

**Sink side:**
- File: path, format (csv / json / parquet / avro / excel), partitioned?
- Table: which DB? Connection env var? Table name? `if_exists` =
  replace / append / upsert?
- Object store: bucket / prefix / partition layout?

**Orchestration:**
- One-shot or scheduled? Cron expression?
- Partitioned (daily / hourly / static / dynamic)?
- Retry policy on failure?

### Example dialog

> **User:** "I want to ingest from SQL Server, transform it some way, and
> store it as a CSV."

> **Good response:**
>
> "Got it — here's the component shape I'd reach for:
>
> 1. `mssql_resource` — shared SQL Server connection
> 2. `dataframe_to_table` upstream OR `sql_transform` (with
>    `return_dataframe: true`) — read from SQL Server
> 3. One of `filter` / `summarize` / `top_n_per_group` / `pct_change` /
>    `dataframe_join` for the transform
> 4. `dataframe_to_csv` — write to disk
>
> A few questions before I generate the YAML:
>
> 1. **Connection:** what env var holds the SQL Server connection
>    string? (e.g. `MSSQL_URL`). Or do you want to use a
>    `mssql_resource` with explicit host/port/user/database fields?
> 2. **Source:** which table(s) or query? Filter to a date range?
> 3. **Transform:** what kind? Filter rows? Group + aggregate?
>    Top N per group? Join with another asset?
> 4. **Output:** what filename / path for the CSV? Should it be
>    partitioned (one file per day, etc.)?
> 5. **Schedule:** one-shot, or do you want a cron schedule?"

That's it — once the user answers, generate the 3-5 `defs.yaml` files,
list the env vars to export, and tell them to run `dg dev` (the UI is
the natural Dagster experience). Mention `dg launch --assets '*'` only
as a headless / CI alternative.

### When to stop asking and just generate

The user has given you enough when:
- All source-side connection details are concrete (URL/path/topic + auth)
- The transform is named or "no transform" was specified
- The output destination is concrete (filename or table or bucket)

If you have those, generate the YAML. Don't ask about every optional
field (`group_name`, `retry_policy`, partition shape) unless the user
brings them up — pick reasonable defaults and call them out in a one-line
comment so the user can override.

## Common gotchas to avoid

1. **YAML 1.1 `on:` is a boolean.** If a component has an `on:` field, quote it:
   `"on": true` (not `on: true`) — otherwise YAML parses the key as `True`.
2. **Demos should be 100% components.** Avoid custom Python files in `defs/`.
   If a transform / generator / glue is needed, the right move is to use (or
   build) a component, not to drop a `.py` file into the project.
3. **`upstream_asset_key` vs `deps:`** — these are different:
   - `upstream_asset_key: foo` → the asset reads data from `foo` (the
     upstream DataFrame is passed in)
   - `deps: [foo]` → ordering-only lineage; nothing is loaded at runtime
4. **No future annotations.** Don't use `from __future__ import annotations`
   in Dagster code — annotations are read at runtime and the future import
   turns them into strings, breaking context-type validation.
5. **Sinks return `Output(value=None)`.** Components like `dataframe_to_csv`,
   `mongodb_writer`, `dataframe_to_avro` are sinks — they write to their own
   destination and return `None`. When combined with a project-level IO
   manager, the IO manager should treat `obj is None` as a no-op.
6. **Multi-step launches need persistent storage.** Dagster's default
   in-memory IO manager doesn't survive between subprocesses with the
   multiprocess executor. For chains of DataFrame assets, install
   `local_parquet_io_manager` (or a cloud equivalent) as the project's
   `io_manager`.

## Reading the registry without the CLI

Static GitHub raw content — no auth, no server. If the CLI isn't installed:

- **Full manifest:** <https://raw.githubusercontent.com/eric-thomas-dagster/dagster-component-templates/main/manifest.json>
- **Per-component files:** swap `component.py` in the manifest entry's
  `component_url` for `schema.json` / `README.md` / `example.yaml` /
  `requirements.txt`.
- **Walkthroughs:** raw-content URLs at
  `https://raw.githubusercontent.com/eric-thomas-dagster/dagster-community-components-cli/main/examples/<topic>.md`

## Version pinning (`id@ref`)

Components evolve. For production, prefer pinning:

| Spec | Resolves to |
|---|---|
| `postgres_resource` | latest (HEAD of main) |
| `postgres_resource@v1.2.0` | tag `v1.2.0` |
| `postgres_resource@a1b2c3d` | commit `a1b2c3d` |

The `.dg-community.json` marker records which ref was installed so future
tooling can detect drift between pinned and latest.

## Component categories

`resource`, `io_manager`, `sensor`, `observation`, `external`, `integration`,
`check`, `transformation`, `ingestion`, `ai`, `analytics`, `infrastructure`,
`source`, `sink`, `jobs`, `data-warehouse`, `dbt`.

Filter with `--category`: `dagster-component search "" --category io_manager`.

## Quick task → component cheatsheet

| Task | Likely component(s) |
|---|---|
| Connect to PostgreSQL / MySQL / MSSQL / Oracle / Db2 | `postgres_resource` / `mysql_resource` / `mssql_resource` / `oracle_resource` / `db2_resource` |
| Land DataFrames as parquet on S3 / GCS / ADLS | `s3_parquet_io_manager` / `gcs_parquet_io_manager` / `azure_blob_parquet_io_manager` |
| Watch S3 / GCS / ADLS for new objects | `s3_monitor` / `gcs_monitor` / `adls_monitor` (dynamic-partition mode) |
| Read REST API → DataFrame | `rest_api_fetcher` |
| OData reads (SAP / MS Graph / Dynamics) | `odata_ingestion` |
| Kafka / NATS / RabbitMQ / MQTT / Pulsar | `<broker>_to_database_asset` + `<broker>_monitor` + `<broker>_observation_sensor` |
| MongoDB / Cassandra / Neo4j / Elasticsearch | `<db>_resource` + `<db>_reader` + `<db>_writer` |
| Iceberg / Delta read+write | `iceberg_ingestion` + `dataframe_to_iceberg_table` (or delta_*) |
| Sync external table into the catalog (declare-only) | `external_<vendor>_table` (Snowflake / BigQuery / Iceberg / Delta / Kafka / S3 / GCS / Kinesis / Pub/Sub / SharePoint / …) |
| Pandas profile / pct change / top-N per group | `dataframe_describe` / `pct_change` / `top_n_per_group` |
| Filter / summarize / pivot / unpivot / join / dedup | `filter` / `summarize` / `pivot` / `unpivot` / `dataframe_join` / `unique_dedup` |
| Templated SQL CTAS or inline read | `sql_transform` (Jinja2, auto-injects partition_key + run_id) |
| Materialize a Jupyter notebook as an asset | `jupyter_notebook_asset` (papermill) |
| Run a container as an asset | `docker_container_asset` |
| Push metrics to Prometheus / query Prometheus | `dataframe_to_prometheus` / `dataframe_from_prometheus` |
| Push Dagster run lifecycle to StatsD / DogStatsD / Datadog Agent | `dagster_runs_to_statsd_sensor` |
| Push Dagster run lifecycle as OTel **metrics** (Datadog OTel ingest, Grafana OTel, Honeycomb metrics) | `dagster_runs_to_otlp_metrics_sensor` |
| Push Dagster run lifecycle as OTel **logs** (Splunk OTC, Honeycomb logs, Loki via OTC) | `dagster_runs_to_otlp_sensor` |
| Synthetic data for demos (orders / events / customers / etc.) | `synthetic_data_generator` (many `schema_type` values) |
| **Run an LLM agent with MCP tool support (single-shot loop)** | `litellm_agent` (multi-vendor) / `openai_agent` / `anthropic_agent` / `gemini_agent` / `snowflake_cortex_agent` |
| **Deterministic single MCP tool call (no LLM)** | `mcp_tool_call` — scheduled query to any MCP server with `{partition_key}` templating |
| **Score an LLM/agent output (LLM-as-judge)** | `llm_evaluator` — answer_relevance / groundedness / harmfulness / helpfulness / coherence |
| **NL → SQL → result via Databricks Genie** | `databricks_genie_query` (single-question or per-row) |
| **Push Dagster asset lineage to a data catalog** | `lineage_graph_extractor` + `lineage_to_datahub` / `lineage_to_openmetadata` / `lineage_to_purview` / `lineage_to_alation` / `lineage_to_collibra` / `lineage_to_file` / `lineage_to_webhook` |
| **One-time warehouse migration** (Oracle / Db2 / MSSQL / Postgres → Snowflake / BigQuery / DuckDB) | `database_schema_inventory` + `database_migration_assessment` + `database_tables_migration` + `database_constraints_migration` + `database_replication` + `database_views_migration` |
| **Recurring DB → warehouse replication** | `database_replication` (Sling-backed) |

When in doubt: `dagster-component search <keyword>` — almost always a hit.

---

## Recent additions worth knowing (2026-06)

The agent stack and observability sensors are recent — AI assistants
following older guidance may have missed them:

- **MCP agent family** (`litellm_agent` + native `openai_agent` /
  `anthropic_agent` / `gemini_agent` / `snowflake_cortex_agent`).
  Single-shot LLM agent with Model Context Protocol tool support.
  Supports stdio / streamable-HTTP / SSE transports; HTTP carries
  literal + env-backed headers so customers can wire the Dagster+ MCP
  server at `mcp.agent.dagster.cloud/mcp/` (34 tools) without leaking
  bearer tokens to YAML. Each agent supports the full Dagster pattern
  (partitions with `{partition_key}` templated into prompts, kinds,
  FreshnessPolicy via `cron`/`time_window` factories, RetryPolicy).
- **`mcp_tool_call`** — deterministic single-shot tool call against any
  MCP server. No LLM in the loop. Same partition/templating pattern.
- **`llm_evaluator`** — LLM-as-judge scoring for agent outputs. Curated
  prompts per metric, OpenAI/Claude/Gemini judges via LiteLLM, drop-in
  downstream of any `*_agent` (matches the agent output dict shape).
- **Run-event sensors**: `dagster_runs_to_statsd_sensor` (DogStatsD UDP),
  `dagster_runs_to_otlp_metrics_sensor` (distinct from the existing
  OTel-logs sensor). Both no-deps / no-auth where possible.
- **Lineage family**: `lineage_graph_extractor` → `lineage_to_<catalog>`
  fan-out across DataHub / OpenMetadata / Purview / Alation / Collibra /
  file / webhook with per-sink change-detection skip via payload hashing.
- **Warehouse migration**: 6-component playbook that handles the bulk
  of an Oracle / Db2 / MSSQL → Snowflake / BigQuery migration with
  dry-run assessment + per-object status DataFrame.

See `examples/agent_family.md`, `examples/litellm_agent.md`,
`examples/lineage_to_datahub.md`, `examples/lineage_catalogs.md`, and
`examples/warehouse_migration.md` for end-to-end walkthroughs of each.

## Recent additions worth knowing (2026-08)

Big shipment on the agentic-orchestration + HITL side. If the user asks
about any of these, use the specific op / component; don't fall back
to generic patterns.

### AgenticPipelineComponent — now 9 ops (previously 5–6)

One YAML with `source: + steps: + outputs:`. Every step becomes a
first-class Dagster asset with cost/latency/tokens in metadata. All 9 ops:

| Op | Purpose | Calls | Best for |
|---|---|---|---|
| `llm_call` | one LLM call over source | 1 | simple analysis / summarization |
| `route` | router LLM picks a specialist; specialist answers | 2 | soft branch by content ("does this read as a question?") |
| **`conditional_route`** | deterministic branching (`regex` / `contains` / `equals` / `jsonpath`) picks specialist; NO router LLM | 1 | hard branch by label / regex / structured field — cheap, testable, reviewable |
| `debate` | N proposers write in parallel; arbitrator picks winner | N+1 | multi-perspective decisions |
| `critique_loop` | drafter + critic × N iterations. Optional **`until_score_gte: N`** — critic scores each draft `SCORE: X/100`; stops early when X ≥ N (saves ~half the calls when the first draft is already good) | 2N+1 (or fewer) | iterative refinement with cost control |
| `synthesize` | join N upstream steps via typed named `inputs:` (or positional `sources:`) | 1 | multi-input merge with per-port templating |
| `mcp_call` | deterministic MCP tool call (no LLM). Transports: `stdio` / `http` / `sse` / **`fastmcp`** (v2 client, auto-transport + bearer/OAuth) | 0 LLM, 1 MCP | fetch grounding data (GitHub, filesystem, remote MCP server) |
| `tool_use_loop` | LLM has MCP tools; iteratively picks tools until `finalize`. Bounded by `max_iterations` | 3–30 (LLM + tool) | open-ended agent tool-use, LangGraph-shape |
| `handoff` | invoke user-provided callable (LangGraph / AutoGen / CrewAI / DSPy). Framework's per-node lineage lives in asset metadata; adjacent Dagster steps stay first-class | 1 wrapped call | bring existing framework code as ONE pipeline step |

**Every step's output flows in a standard `{text, cost_usd, latency_ms,
tokens_total, model_fingerprint, materialized_at, op, ...}` dict shape.**
Downstream steps consume via `source: <step_id>` OR typed `inputs: {port:
{from: <step_id>} | {literal: <value>}}`. Every text-emitting op
supports `{text}` + `{port_name}` + `{partition_key}` + `{partition.<name>}`
substitution in prompts / tool_args / sink paths.

### AgenticPipelineComponent — new opt-in flags

- **`per_step_ops: true`** — emits as `@dg.graph_multi_asset` with ONE OP
  PER STEP instead of a single `@multi_asset`. Runs page shows N step_keys
  (`{prefix}_ingest`, `{prefix}_{step_id}` per step, `{prefix}_extract`)
  instead of one composite op. Native per-op retry via Dagster's
  re-execution machinery — restart from any failed step. State dict
  serializes through the IO manager between ops. Default stays False for
  zero-overhead single-op behavior.
- **`partition_key_parser: "{owner}/{repo}#{issue_number:int}"`** —
  parses composite partition keys into named fields available as
  `{partition.<name>}` in tool_args + prompts + source.text. Optional
  `:int` / `:float` / `:bool` type suffixes preserve types when the
  substitution target is a pure single-placeholder value (needed for
  GitHub MCP `get_issue` which rejects string `issue_number`).

### PartitionedAssetLauncherJobComponent — config-driven entry

Takes run config (owner/repo/issue_number, tenant_id, etc.), formats a
partition key from `partition_key_template`, registers on the target's
DynamicPartitionsDefinition, materializes the target assets with that
key. Same entry point for humans (form-fill in the UI) + external
systems (POST run_config via GraphQL). Pairs with any dynamic-partitioned
multi_asset — typically an AgenticPipelineComponent with a
`partition_key_parser` set.

### Local-AI A/B stack (3 components — should-we-go-local as a pipeline)

- **`InferenceProviderABTestComponent`** — same prompt through N LiteLLM
  providers side-by-side; each response an asset with cost / latency /
  tokens. Failure isolation (one provider erroring doesn't tank the A/B).
- **`ProviderABEvaluatorComponent`** — LLM-as-judge scores N candidates
  against a rubric in ONE pass (avoids judge-drift across separate
  calls). Optional `min_winner_score` asset check turns the pair into
  a branch-deploy merge gate: swap the provider in a PR, quality holds
  → merge; drops → merge blocked automatically.
- **`InferenceCostReportComponent`** — aggregates per-provider cost +
  latency + quality into one report asset (baseline deltas, projected
  daily savings, winner-by-cost / winner-by-quality / winner-by-value
  composite). Materialize daily → time-series in Insights.

### SlackApprovalGateComponent — Slack quorum HITL

Reactions-polling (not interactive buttons) so it works in Dagster+
Serverless without a public webhook. Multi-approver quorum + allowlist
+ timeout policies (`escalate` / `reject` / `approve`) + escalation
pings. Writes the same JSON token file `HumanApprovalGateComponent`
reads — everything downstream unchanged. Sanitizes composite partition
keys (`/` → `_`) so per-partition token filenames land as single
segments.

### Key walkthroughs for these

- `examples/local_ai_ab.md` — the A/B stack in one project
- `examples/slack_approval_gate.md` — Slack quorum HITL
- `examples/maintainer_investigation_room.md` — MIR-v1 (5-op mcp_call + fan-out + file-drop HITL)
- `examples/maintainer_investigation_room_v3.md` — MIR-v3 (every 2026-08
  primitive composed into one pipeline: per_step_ops + tool_use_loop +
  handoff + debate + critique_loop + Slack quorum HITL)

### Rules of thumb when composing agentic pipelines

- **`route` vs `conditional_route`.** If the branch signal is deterministic
  (a label field, a regex on a subject line, a JSON path into a payload),
  reach for `conditional_route` — it's a code path, cheap ($0), and
  reviewable in a diff. If the signal is soft ("does this issue read as
  a question or a bug report?"), keep `route` (LLM picks). Rule: if you
  can write the picker as a unit test, use `conditional_route`.
- **`critique_loop` with `until_score_gte:`.** Set an iteration cap AND
  a quality floor — critic scores each draft `SCORE: N/100` and the loop
  stops the moment quality clears the bar (skipping the revise step
  since the current draft is already good enough). Cuts cost roughly in
  half on easy cases without capping quality on hard ones.
- **Vague prompt → `tool_use_loop`. Known shape → typed steps.**
  If the user's ask is exploratory ("grab an issue, figure out what I
  need, give me a report"), the whole thing is ONE `tool_use_loop` op:
  give it the source, list the MCP tools, set `max_iterations`, let the
  LLM decide what to fetch each turn. If the user already knows the
  shape ("intake → 4 specialists → synthesize → debate → report"), use
  the prescriptive multi-step DAG — every step a first-class asset.
  Hybrid works too: deterministic intake (`mcp_call`) → exploratory
  middle (`tool_use_loop`) → deterministic report (`critique_loop`).
- **Fan-out with typed inputs:** N specialists all read
  `inputs: {issue_facts: {from: intake}}`. Cleaner than passing the
  whole state dict.
- **Multi-input synthesize:** typed `inputs:` map port names to prior
  step ids or literals. Each port becomes a `{port_name}` placeholder
  in `prompt_template` + `system_prompt`. Prefer typed inputs over
  positional `sources:` for multi-input joins.
- **`per_step_ops: true` when the run view matters:** for demos + when
  customers want dbt-style retry, flip it on. Default off keeps the
  in-memory state performance for high-throughput pipelines.
- **`tool_use_loop` for exploratory reasoning:** when the LLM needs
  to make N calls that depend on prior tool results (search → read →
  search → …). One asset materializes with the full trace in metadata.
- **`handoff` for framework composition:** if the customer already has
  LangGraph / AutoGen / CrewAI code, wrap it in a callable + use
  `handoff` — don't force a rewrite to the Dagster ops. Adjacent
  Dagster steps stay first-class.
- **Composite partition keys + parser:** for per-entity pipelines
  (per-issue triage, per-tenant analysis), key on
  `{owner}/{repo}#{issue_number:int}` (or similar) and parse back with
  `partition_key_parser` so each field is available as
  `{partition.<name>}` in tool_args / prompts.
