Metadata-Version: 2.4
Name: llm-microsoft-foundry
Version: 0.3.0
Summary: LLM plugin for Microsoft Foundry models, authenticated via Azure CLI
License: Apache-2.0
Project-URL: Homepage, https://github.com/stefanstranger/llm-microsoft-foundry
Project-URL: Issues, https://github.com/stefanstranger/llm-microsoft-foundry/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: llm>=0.32
Requires-Dist: httpx>=0.27
Requires-Dist: azure-identity>=1.16
Requires-Dist: pydantic>=2
Requires-Dist: pyyaml>=6
Dynamic: license-file

---
title: LLM Microsoft Foundry
description: Use Microsoft Foundry chat and decision models with the LLM command-line tool
---

LLM plugin for [Microsoft Foundry](https://learn.microsoft.com/azure/ai-foundry/) models.

Authenticates transparently using `az login` (no API key needed) and lets you
use chat and decision models deployed in a Foundry resource directly from the
[llm](https://llm.datasette.io/) CLI.

## Install

```powershell
llm install llm-microsoft-foundry
# or, from a local clone:
llm install -e .
```

Dependencies (`azure-identity`, `httpx`) are installed automatically.

## Authenticate

```powershell
az login
az account set --subscription "<subscription-id-or-name>"
```

The plugin uses `DefaultAzureCredential`, so any credential that works with the
Azure SDK (az cli cache, managed identity, env vars) works here too.

## Configure a model

```powershell
llm foundry add-model \
  --model-id foundry-gpt-5.4 \
  --api-base https://<your-resource>.services.ai.azure.com \
  --deployment gpt-5.4 \
  --api-type chat \
  --tools \
  --alias gpt54
```

| Option | Description |
|--------|-------------|
| `--model-id` | Unique local identifier (used with `llm -m`) |
| `--api-base` | Foundry resource root URL (the **resource root**, not a project URL) |
| `--deployment` | Exact deployment name in Foundry (defaults to `--model-id`) |
| `--api-type` | Inference API exposed by the deployment: `chat` or `decision` |
| `--tools` / `--no-tools` | Advertise tool calling for a chat deployment whose model supports it |
| `--vision` / `--no-vision` | Enable image attachment support |
| `--alias` | Short alias(es), repeatable |

The API type defaults to `chat`, so configurations created by earlier plugin
versions continue to work. Decision models do not support the `--tools` or
`--vision` options.

Settings are stored in `<llm user dir>/foundry_models.yaml`, in the same folder
as LLM's own configuration files
(`C:\Users\<you>\AppData\Roaming\io.datasette.llm\` on Windows). You can edit it
directly:

```yaml
models:
- model_id: foundry-gpt-5.4
  api_base: https://my-resource.services.ai.azure.com
  deployment: gpt-5.4
  api_type: chat
  supports_tools: true
  supports_vision: false

- model_id: foundry-deepseek-v3.2
  api_base: https://my-resource.services.ai.azure.com
  deployment: DeepSeek-V3.2
api_type: chat
supports_tools: false
supports_vision: false
aliases:
  - deepseek
```

> **Migrating from an older version?** If you have a `foundry_config.json`, it is migrated to `foundry_models.yaml` automatically on the next `llm` invocation.


## Use

```powershell
# List configured models
llm foundry list-models

# Prompt a model
llm -m foundry-gpt-5.4 "What is the capital of France?"

# Or using an alias
llm -m gpt54 "Explain quantum entanglement in one sentence"

# Set as default model
llm models default foundry-gpt-5.4

# Check which Azure identity is active
llm foundry whoami
```

### Options

```powershell
llm -m foundry-gpt-5.4 -o temperature 0.2 -o max_tokens 512 "Write a haiku"
```

| Option | Default | Description |
|--------|---------|-------------|
| `temperature` | model default | Sampling temperature (0–2) |
| `max_tokens` | model default | Max output tokens |
| `top_p` | model default | Nucleus sampling probability |

### Tools

The plugin supports [LLM tool calling](https://llm.datasette.io/en/stable/tools.html)
through the Foundry chat completions API. Two separate settings are involved:

* `--tools` on `llm foundry add-model` (or `supports_tools: true` in the YAML
  file) advertises that the deployment can accept tool definitions. Enable it
  only for chat deployments whose model supports function calling. It is off by
  default.
* `--tool` (or `-T`) on an `llm` prompt selects which local tools LLM offers to
  the model for that invocation. `--functions` works the same way for ad-hoc
  Python functions.

Pass an explicit `-m` so the request goes to the intended Foundry model rather
than your default model:

```powershell
llm -m foundry-gpt-5.4 "What is the current UTC time?" --tool llm_time --td
```

`--td` (`--tools-debug`) prints each tool call and its result. Add
`--ta` (`--tools-approve`) to confirm every call before it runs.

When the model requests a tool, the plugin hands the call back to LLM. LLM runs
the tool on your machine, applies any approval prompts, turns tool exceptions
into error results, and sends the results to Foundry in a follow-up request
until the model returns a final answer. Tools never execute inside Foundry, and
the plugin does not run them itself. Conversations continued with `llm -c`
include earlier tool calls and results.

> [!NOTE]
> Offering tools does not force the model to use them. The model decides
> whether a tool is relevant, so it may answer directly in plain text. Tool
> selection quality depends on the deployed model.

If Foundry returns a tool call whose arguments are not a valid JSON object, the
command stops with an error that shows the tool name, call ID, and raw
arguments instead of running the tool with guessed input.

## Use Microsoft-Decision-1

[Microsoft-Decision-1](https://learn.microsoft.com/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision)
returns bounded, numerical decisions instead of generated prose. It supports
`noul` questions for probabilities, `choice` questions for classification, and
`score` questions for ordered scales.

Deploy the model in Foundry, then configure the deployment:

```powershell
llm foundry add-model `
  --model-id foundry-decision-1 `
  --api-base https://<your-resource>.services.ai.azure.com `
  --deployment <your-decision-deployment> `
  --api-type decision `
  --alias decision
```

The `--deployment` value must be the deployment name from Foundry, not the
underlying `Microsoft-Decision-1` model name.

### Pass state text and questions

Provide the state as the prompt and the questions as a JSON model option:

```powershell
$questions = @'
{
  "refund": {
    "type": "noul",
    "instructions": "Does the customer request a refund?"
  }
}
'@

llm -m decision `
  -o questions $questions `
  "I was charged twice. Please refund the duplicate charge."
```

### Pass a complete decision request

Pass a complete request through standard input when the state is structured or
the request contains several questions:

```powershell
@'
{
  "state": {
    "message": "Checkout is down again. This is my third request.",
    "customer_contact_count": 3
  },
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this request?",
      "criteria": {
        "billing": "Charges, invoices, and refunds",
        "engineering": "Bugs, errors, and outages",
        "support": "How-to and account questions"
      }
    },
    "severity": {
      "type": "score",
      "instructions": "How severe is this issue?",
      "criteria": ["Cosmetic", "Minor", "Major", "Critical"]
    },
    "repeat_contact": {
      "type": "noul",
      "instructions": "Has the customer contacted support before?"
    }
  }
}
'@ | llm -m decision
```

The plugin adds the configured deployment name to the request and emits the
complete Decision API response as JSON. Token counts from the response are
available through LLM's usage metadata.

> [!NOTE]
> Decision models do not support streaming, conversations, system prompts,
> attachments, tools, or schemas.

## Remove a model

```powershell
llm foundry remove-model foundry-gpt-5.4
```

## Required Azure role

The identity must have the **Cognitive Services User** role on the Foundry
resource. Owner and Contributor roles alone do not grant inference access.

## How it works

```
llm CLI
  └─ llm-microsoft-foundry plugin
       ├─ DefaultAzureCredential (azure-identity)
       │    └─ uses the Azure CLI credential cache
       ├─ Chat
       │    └─ POST <api_base>/openai/v1/chat/completions
       │         (tool calls are returned to LLM and run locally)
       └─ Decision
            └─ POST <api_base>/providers/microsoft/v1/systemone
```

Chat and decision requests use the Entra token scope required by their
respective Foundry APIs. No proxy, extra service, or master key is involved.

## Development

```powershell
git clone https://github.com/stefanstranger/llm-microsoft-foundry
cd llm-microsoft-foundry
llm install -e .
uv sync
uv run pytest
```
