Metadata-Version: 2.4
Name: ds-workspace-mcp
Version: 1.2.0
Summary: A Model Context Protocol server for safely inspecting and profiling local analytical datasets.
License-Expression: MIT
License-File: LICENSE
Keywords: mcp,model-context-protocol,data-science,analytics,python
Author: Diogo Ribeiro
Author-email: hansolo.dj@gmail.com
Requires-Python: >=3.11,<3.13
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Provides-Extra: opentelemetry
Requires-Dist: duckdb (>=1.5.5,<2.0.0)
Requires-Dist: mcp[cli] (>=1.28.1,<1.29.0)
Requires-Dist: openpyxl (>=3.1.0,<4.0.0)
Requires-Dist: opentelemetry-api (>=1.36.0,<2.0.0) ; extra == "opentelemetry"
Requires-Dist: opentelemetry-sdk (>=1.36.0,<2.0.0) ; extra == "opentelemetry"
Requires-Dist: pandas (>=2.2.0,<3.0.0)
Requires-Dist: pydantic (>=2.7.0,<3.0.0)
Requires-Dist: pydantic-settings (>=2.14.1,<3.0.0)
Requires-Dist: scikit-learn (>=1.7.0,<2.0.0)
Requires-Dist: sqlglot (>=30.15.0,<31.0.0)
Project-URL: Changelog, https://github.com/DiogoRibeiro7/ds-workspace-mcp/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/DiogoRibeiro7/ds-workspace-mcp#readme
Project-URL: Homepage, https://github.com/DiogoRibeiro7/ds-workspace-mcp
Project-URL: Issue Tracker, https://github.com/DiogoRibeiro7/ds-workspace-mcp/issues
Project-URL: Repository, https://github.com/DiogoRibeiro7/ds-workspace-mcp
Project-URL: Security Policy, https://github.com/DiogoRibeiro7/ds-workspace-mcp/security/policy
Description-Content-Type: text/markdown

# Data Science Workspace MCP Server

![CI](https://github.com/DiogoRibeiro7/ds-workspace-mcp/actions/workflows/ci.yml/badge.svg)
![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12-blue)
![License](https://img.shields.io/badge/license-MIT-green)
![Release](https://img.shields.io/github/v/release/DiogoRibeiro7/ds-workspace-mcp?include_prereleases)

A local-first **Model Context Protocol (MCP)** server for data science workspaces.

`ds-workspace-mcp` gives MCP-compatible assistants structured access to analytical data
without handing them arbitrary filesystem or database control. It exposes a guarded local
workspace where an assistant can discover datasets, inspect schemas, profile columns, run
bounded read-only queries, evaluate baseline modeling readiness, and create reusable
markdown modeling reports.

The project is built as a practical reference implementation for AI-assisted data tooling:
typed Python, bounded outputs, path traversal protection, read-only SQL guardrails,
release metadata, Docker support, and CI-backed documentation.

## At a glance

| Area | What this server provides |
| --- | --- |
| Dataset access | Safe reads for CSV, Parquet, JSON, Excel, SQLite, and DuckDB-backed CSV queries |
| Exploration | Preview, profiling, missingness summaries, categorical summaries, correlations, and issue detection |
| Modeling readiness | Target suggestions, feature review, leakage checks, time-series validation, forecast baselines, and dummy baselines |
| Report workflow | Generate, save, search, preview, compare, copy, rename, delete, and extract markdown report sections |
| MCP integration | Resources, tools, prompts, stdio transport, Streamable HTTP transport, and runnable client examples |
| Safety controls | Configured data/report roots, path traversal rejection, size limits, row limits, query length limits, and SQL timeouts |

## Current scope

Today the repository covers five practical workflows:

- Safe local dataset access for CSV, Parquet, JSON, Excel, and SQLite sources.
- Structured aggregations for common EDA without free-form SQL.
- Lightweight analytical diagnostics, including leakage review, correlations, and time-series checks.
- Baseline modeling planning and markdown report generation.
- Saved report lifecycle management, including cataloging, search, section extraction, section export, and section-aware diffs.

---

## Why this project exists

Many AI assistants can reason well, but useful analytical work needs controlled access to
real data and real tools.

This MCP server exposes a local analytical workspace without giving the model arbitrary file-system access. The server only reads supported dataset files from a configured data directory, and saved modeling reports live under a separate configured reports directory.

Good use cases:

- connecting an MCP client to a local data science workspace;
- inspecting unfamiliar datasets before deeper analysis;
- creating first-pass EDA, readiness, and baseline modeling reports;
- preserving reusable markdown report artifacts from assistant-led analysis;
- demonstrating AI tooling, data engineering, and security-minded local service design.

## Quick start

Install dependencies:

```bash
poetry install
```

Generate a sample dataset:

```bash
poetry run generate-sample-healthcare-data
```

Start the MCP server over Streamable HTTP:

```bash
MCP_TRANSPORT=streamable-http poetry run ds-workspace-mcp serve
```

Open the MCP Inspector and connect to:

```text
http://localhost:8000/mcp
```

Useful first tool calls:

- `summarize_dataset` for a compact dataset overview.
- `profile_csv` for detailed column summaries, distributions, and bounded quality diagnostics.
- `aggregate_dataset` for allowlisted grouped metrics.
- `assess_modeling_readiness` for target, feature, leakage, and next-step guidance.
- `save_modeling_report` for a reusable markdown report artifact.

---

## Project structure

```text
ds-workspace-mcp/
├── data/
├── docs/
├── examples/
├── notebooks/
├── src/
│   └── ds_workspace_mcp/
│       ├── cli.py
│       ├── config.py
│       ├── core.py
│       ├── datasets/
│       ├── diagnostics.py
│       ├── experiment_plan.py
│       ├── feature_selection.py
│       ├── modeling_readiness.py
│       ├── modeling_report.py
│       ├── overview.py
│       ├── profiling.py
│       ├── report_export.py
│       ├── reports/
│       ├── server.py
│       ├── timeseries.py
│       ├── ml/
│       ├── sql/
│       └── synthetic/
├── tests/
├── .github/
├── docker-compose.yml
├── Dockerfile
├── README.md
└── pyproject.toml
```

---

## Requirements

- Python 3.11+
- Poetry
- `openpyxl` is installed as a runtime dependency for `.xlsx` ingestion.
- Node.js, only if you want to use the MCP Inspector through `npx`

---

## Quality Gate

Run the same checks expected before pull requests:

```bash
poetry run ruff check .
poetry run ruff format --check src tests examples
poetry run mypy
poetry run pytest -m "not integration"
```

On systems with `make`, the same gate is available as:

```bash
make quality
```

---

## Installation

```bash
poetry install
```

## Configuration

The server reads configuration from environment variables or a local `.env` file.

- `MCP_DATA_ROOT`: directory that contains readable datasets. Defaults to `data`.
- `MCP_REPORTS_ROOT`: directory that contains saved modeling reports. Defaults to `reports`.
- `MCP_TRANSPORT`: `stdio` or `streamable-http`. Defaults to `streamable-http`.
- `MCP_HOST`: bind host for HTTP mode. Defaults to `127.0.0.1`.
- `MCP_PORT`: bind port for HTTP mode. Defaults to `8000`.
- `MCP_MAX_PREVIEW_ROWS`: maximum allowed value for `preview_csv`. Defaults to `50`.
- `MCP_MAX_SQL_ROWS`: maximum rows returned by SQL queries. Defaults to `1000`.
- `MCP_MAX_SQL_QUERY_LENGTH`: maximum allowed SQL text length for query tools. Defaults to `20000`.
- `MCP_SQL_TIMEOUT_MS`: maximum SQL execution time before interruption. Defaults to `5000`.
- `MCP_MAX_CATEGORICAL_VALUES`: maximum top-value examples returned per categorical column in `profile_csv`. Defaults to `5`.
- `MCP_MAX_DATASET_BYTES`: maximum readable dataset size in bytes. Defaults to `25000000`.
- `MCP_PROFILE_CACHE_ENABLED`: enable or disable in-memory profile caching. Defaults to `true`.
- `MCP_PROFILE_CACHE_MAX_ENTRIES`: maximum cached profile entries. Defaults to `128`.
- `MCP_QUERY_CACHE_ENABLED`: cache successful DuckDB query results in memory. Defaults to `true`.
- `MCP_QUERY_CACHE_MAX_ENTRIES`: maximum cached DuckDB query results. Defaults to `32`.
- `MCP_QUERY_EXPORT_ENABLED`: permit bounded query results to be exported as new CSV files. Defaults to `false`.
- `MCP_MAX_EXPORT_ROWS`: maximum rows written by a single CSV export. Defaults to `1000`.
- `MCP_MAX_EXPORT_COLUMNS`: maximum columns written by a single CSV export. Defaults to `100`.
- `MCP_MAX_EXPORT_BYTES`: maximum UTF-8 CSV export size. Defaults to `5000000`; `MCP_MAX_DATASET_BYTES` is also enforced.
- `MCP_LOG_LEVEL`: server log level. Defaults to `INFO`.
- `MCP_API_KEY`: optional shared bearer token for Streamable HTTP mode. Disabled by default.
- `MCP_TRACING_ENABLED`: enable optional OpenTelemetry spans. Defaults to `false`.
- `MCP_TRACING_SERVICE_NAME`: service name reported in traces. Defaults to `ds-workspace-mcp`.
- `MCP_TRACING_CONSOLE_EXPORTER`: print spans to stdout through the OpenTelemetry console exporter. Defaults to `false`.

Example:

```bash
MCP_DATA_ROOT=./data
MCP_REPORTS_ROOT=./reports
MCP_TRANSPORT=streamable-http
MCP_HOST=127.0.0.1
MCP_PORT=8000
MCP_MAX_PREVIEW_ROWS=50
MCP_MAX_SQL_ROWS=1000
MCP_MAX_SQL_QUERY_LENGTH=20000
MCP_SQL_TIMEOUT_MS=5000
MCP_MAX_CATEGORICAL_VALUES=5
MCP_MAX_DATASET_BYTES=25000000
MCP_PROFILE_CACHE_ENABLED=true
MCP_PROFILE_CACHE_MAX_ENTRIES=128
MCP_QUERY_CACHE_ENABLED=true
MCP_QUERY_CACHE_MAX_ENTRIES=32
MCP_QUERY_EXPORT_ENABLED=false
MCP_MAX_EXPORT_ROWS=1000
MCP_MAX_EXPORT_COLUMNS=100
MCP_MAX_EXPORT_BYTES=5000000
MCP_LOG_LEVEL=INFO
MCP_API_KEY=
MCP_TRACING_ENABLED=false
MCP_TRACING_SERVICE_NAME=ds-workspace-mcp
MCP_TRACING_CONSOLE_EXPORTER=false
```

## Logging

The server emits structured logs with timestamp, level, module, and message.

- Tool and resource invocations are logged with dataset names and safe counts only.
- Validation failures are logged clearly.
- Full dataset contents and preview row payloads are intentionally not logged.

Set the log level with `MCP_LOG_LEVEL`, for example:

```bash
MCP_LOG_LEVEL=DEBUG
```

## Tracing

OpenTelemetry tracing is optional production polish.

Install the optional dependencies with:

```bash
poetry install --extras opentelemetry
```

To enable local tracing with console output:

```bash
MCP_TRACING_ENABLED=true
MCP_TRACING_CONSOLE_EXPORTER=true
poetry run ds-workspace-mcp serve
```

Current spans cover dataset resolution, CSV reads, profiling, SQL query execution, and MCP tool boundaries.

If tracing is enabled without the optional dependencies installed, the server continues to run and logs a warning instead of failing.

---

## Troubleshooting

- `Dataset not found: ...`: the file name must exist inside `MCP_DATA_ROOT`, and the server only sees files under that directory.
- `Only CSV files are supported.`, `Only SQLite files are supported.`, or `Unsupported dataset format: ...`: the requested file extension is not allowed for that tool.
- `Access outside the configured data directory is not allowed.`: the request attempted path traversal such as `../...`.
- `Could not profile dataset: ...`: the file could be opened but could not be profiled safely; validate the CSV structure and encoding.
- `Could not read dataset: ...`: the CSV could not be decoded or parsed safely.
- `Dataset exceeds the maximum allowed size ...`: the file is larger than the configured dataset-size guardrail.
- `Destructive or schema-changing SQL is not allowed.`: the SQL tool only accepts bounded read-only `SELECT` or `WITH` queries.
- `SQL query exceeded the timeout ...`: the query ran longer than the configured SQL timeout.

These errors intentionally avoid exposing absolute local paths in responses.

---

## Run with Streamable HTTP

```bash
MCP_TRANSPORT=streamable-http poetry run ds-workspace-mcp
```

By default, the server reads datasets from `./data`.

You can override that directory:

```bash
MCP_DATA_ROOT=/path/to/csv/files MCP_TRANSPORT=streamable-http poetry run ds-workspace-mcp
```

Then connect with the MCP Inspector:

```bash
npx -y @modelcontextprotocol/inspector
```

Use this URL:

```text
http://localhost:8000/mcp
```

---

## Run with stdio

```bash
MCP_TRANSPORT=stdio poetry run ds-workspace-mcp
```

This is useful when connecting the server directly to an MCP client that launches local servers.

## Synthetic Dataset

The repository also includes a reproducible healthcare operations dataset generator.

Generate a fresh sample into `data/`:

```bash
poetry run generate-sample-healthcare-data
```

Custom example:

```bash
poetry run generate-sample-healthcare-data --output data/custom_clinic_usage.csv --days 180 --clinics 6 --seed 7
```

Generated columns:

- `clinic_id`
- `date`
- `appointments_scheduled`
- `appointments_completed`
- `cancellations`
- `no_shows`
- `marketing_campaign`
- `local_holiday`
- `staff_available`
- `average_wait_time`
- `patient_satisfaction_score`

The generator includes weekday effects, seasonal demand, campaign lift, holiday effects, random noise, and light missingness in a couple of operational fields.

## CLI

The main console command now exposes a small local workflow CLI.

Serve the MCP server:

```bash
poetry run ds-workspace-mcp serve
```

`poetry run ds-workspace-mcp` also defaults to `serve`.

List available datasets:

```bash
poetry run ds-workspace-mcp list-datasets
```

Profile one dataset as JSON:

```bash
poetry run ds-workspace-mcp profile-dataset sample_clinic_usage.csv
```

Build a first-pass modeling plan as JSON:

```bash
poetry run ds-workspace-mcp plan-modeling sample_clinic_usage.csv --target-column appointments_completed
```

Build a markdown modeling report:

```bash
poetry run ds-workspace-mcp report-modeling sample_clinic_usage.csv --target-column appointments_completed
```

Save a markdown modeling report into `reports/`:

```bash
poetry run ds-workspace-mcp save-modeling-report sample_clinic_usage.csv --target-column appointments_completed --output-name clinic-usage-report.md
```

Saved reports, copied reports, renamed reports, and saved report sections fail when the
target file already exists unless `--overwrite` is passed or the MCP tool argument
`overwrite` is set to `true`.

List saved modeling reports:

```bash
poetry run ds-workspace-mcp list-modeling-reports
```

Search saved modeling reports:

```bash
poetry run ds-workspace-mcp search-modeling-reports clinic
```

Search saved modeling report content:

```bash
poetry run ds-workspace-mcp search-modeling-report-content elevated
```

Search the latest saved modeling report content:

```bash
poetry run ds-workspace-mcp search-latest-modeling-report-content elevated
```

Search saved modeling report sections:

```bash
poetry run ds-workspace-mcp search-modeling-report-sections risk
```

Search section headings in the latest saved modeling report:

```bash
poetry run ds-workspace-mcp search-latest-modeling-report-sections risk
```

Summarize recurring modeling report sections:

```bash
poetry run ds-workspace-mcp summarize-modeling-report-sections
```

List the most recent modeling reports:

```bash
poetry run ds-workspace-mcp list-recent-modeling-reports --limit 3
```

Summarize saved modeling reports:

```bash
poetry run ds-workspace-mcp summarize-modeling-reports --limit 3
```

Read one saved modeling report:

```bash
poetry run ds-workspace-mcp read-modeling-report clinic-usage-report.md
```

List section headings from one saved modeling report:

```bash
poetry run ds-workspace-mcp list-modeling-report-sections clinic-usage-report.md
```

List section headings from the latest saved modeling report:

```bash
poetry run ds-workspace-mcp list-latest-modeling-report-sections
```

Read one section from a saved modeling report:

```bash
poetry run ds-workspace-mcp read-modeling-report-section clinic-usage-report.md "Summary"
```

Read one section from the latest saved modeling report:

```bash
poetry run ds-workspace-mcp read-latest-modeling-report-section "Summary"
```

Save one section from a saved modeling report:

```bash
poetry run ds-workspace-mcp save-modeling-report-section clinic-usage-report.md "Risks"
```

Save one section from the latest saved modeling report:

```bash
poetry run ds-workspace-mcp save-latest-modeling-report-section "Risks"
```

Compare one section across two saved modeling reports:

```bash
poetry run ds-workspace-mcp compare-modeling-report-sections clinic-usage-report.md clinic-usage-v2.md "Risks"
```

Compare one section across the two most recent saved modeling reports:

```bash
poetry run ds-workspace-mcp compare-latest-modeling-report-sections "Risks"
```

Read the latest saved modeling report:

```bash
poetry run ds-workspace-mcp read-latest-modeling-report
```

Delete one saved modeling report:

```bash
poetry run ds-workspace-mcp delete-modeling-report clinic-usage-report.md
```

Rename one saved modeling report:

```bash
poetry run ds-workspace-mcp rename-modeling-report clinic-usage-report.md clinic-usage-v2.md
```

Rename the latest saved modeling report:

```bash
poetry run ds-workspace-mcp rename-latest-modeling-report clinic-usage-latest.md
```

Copy one saved modeling report:

```bash
poetry run ds-workspace-mcp copy-modeling-report clinic-usage-report.md clinic-usage-snapshot.md
```

Copy the latest saved modeling report:

```bash
poetry run ds-workspace-mcp copy-latest-modeling-report latest-clinic-snapshot.md
```

Inspect one saved modeling report:

```bash
poetry run ds-workspace-mcp inspect-modeling-report clinic-usage-report.md
```

Inspect the latest saved modeling report:

```bash
poetry run ds-workspace-mcp inspect-latest-modeling-report
```

Preview one saved modeling report:

```bash
poetry run ds-workspace-mcp preview-modeling-report clinic-usage-report.md
```

Compare two saved modeling reports:

```bash
poetry run ds-workspace-mcp compare-modeling-reports clinic-usage-report.md clinic-usage-v2.md
```

Compare the two most recent modeling reports:

```bash
poetry run ds-workspace-mcp compare-latest-modeling-reports
```

Preview the latest saved modeling report:

```bash
poetry run ds-workspace-mcp preview-latest-modeling-report
```

Generate a synthetic dataset through the main CLI:

```bash
poetry run ds-workspace-mcp generate-sample-healthcare-data --days 120 --clinics 5
```

## Examples

The repository includes runnable MCP client examples in [`examples/`](examples/).

Use stdio against a locally launched process:

```bash
poetry run python examples/stdio_client.py
```

Expected output starts with:

```text
Resources:
{
  "resources": [
```

Use Streamable HTTP after starting the server:

```bash
poetry run ds-workspace-mcp serve
poetry run python examples/http_client.py
```

Expected output starts with:

```text
Tools:
{
  "tools": [
```

The examples show how to:

- initialize an MCP client session;
- list resources or tools;
- call `preview_csv`, `profile_csv`, `detect_csv_issues`, and `summarize_correlations`.
- exit with a non-zero status and a concise stderr message if the connection fails.

---

## Available MCP capabilities

### Resource

```text
datasets://catalog
```

Returns the list of CSV files available in the configured data directory.

```text
databases://sqlite
```

Returns the list of SQLite database files available in the configured data directory.

```text
reports://modeling
```

Returns the list of saved modeling report file names from the local `reports/` directory.

```text
reports://modeling/latest
```

Returns the most recently modified saved modeling report as markdown.

```text
reports://modeling/{output_name}
```

Returns one saved modeling report as markdown for the requested report file name.

```text
reports://modeling/latest/sections
```

Returns section headings from the most recently modified saved modeling report.

```text
reports://modeling/{output_name}/sections
```

Returns section headings from one saved modeling report.

```text
reports://modeling/latest/sections/{section_heading}
```

Returns one section from the most recently modified saved modeling report as markdown.

```text
reports://modeling/{output_name}/sections/{section_heading}
```

Returns one section from a saved modeling report as markdown.

### Tools

#### `summarize_dataset`

Return a compact first-pass overview of a dataset, including shape, missingness highlights, identifier-like columns, strongest correlations, and recommended next tools.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv"
}
```

#### `aggregate_dataset`

Run a bounded allowlisted aggregation over any supported tabular dataset without accepting
free-form executable expressions.

Arguments:

```json
{
  "request": {
    "file_name": "sample_clinic_usage.csv",
    "group_by": ["clinic_id"],
    "metrics": [
      {
        "operation": "mean",
        "column": "appointments_completed",
        "output_name": "avg_completed"
      },
      {
        "operation": "count",
        "output_name": "row_count"
      }
    ],
    "filters": [
      {
        "column": "appointments_scheduled",
        "operation": "gt",
        "value": 0
      }
    ],
    "order_by": [
      {
        "column": "avg_completed",
        "direction": "desc"
      }
    ],
    "limit": 10
  }
}
```

#### `compare_datasets`

Compare two supported datasets for structural changes and lightweight drift diagnostics.
Schema differences and statistical drift are reported separately, and the response states
whether the comparison used full data or a bounded head sample.

Arguments:

```json
{
  "left_file_name": "sample_clinic_usage.csv",
  "right_file_name": "sample_clinic_usage_v2.csv"
}
```

The result includes added/removed columns, dtype changes, null-rate changes, row-count
changes, bounded categorical cardinality changes, numeric effect-size diagnostics,
categorical distribution shifts, timestamp range shifts, and the sampling policy used.

#### `suggest_target_columns`

Suggest plausible target columns for modeling, with task-type hints and reasoning.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv"
}
```

#### `suggest_feature_columns`

Suggest which columns to include, review, or exclude for a supervised modeling target.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed"
}
```

#### `assess_modeling_readiness`

Return a compact modeling-readiness summary that combines target suggestion, feature selection, leakage review, and next-step guidance.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed"
}
```

#### `build_experiment_plan`

Build a concrete first-pass modeling plan with starter models, validation guidance, risks,
metrics, next steps, and a reproducible evaluation manifest.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed"
}
```

#### `build_modeling_report`

Build a compact markdown report artifact for review or handoff. The structured result includes
an `evaluation_manifest`, and the markdown includes a compact manifest section with provenance
details such as dataset fingerprint, executable validation strategy, selected feature counts,
runtime versions, and generation time.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed"
}
```

#### `save_modeling_report`

Build and save a markdown report artifact inside the local `reports/` directory.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed",
  "output_name": "clinic-usage-report.md"
}
```

#### `list_modeling_reports`

Return markdown modeling reports saved inside the local `reports/` directory.

#### `search_modeling_reports`

Return saved markdown modeling reports whose file names match a case-insensitive substring.

Arguments:

```json
{
  "query": "clinic"
}
```

#### `search_modeling_report_content`

Return saved markdown modeling reports whose content matches a case-insensitive text query.

Arguments:

```json
{
  "query": "elevated"
}
```

#### `search_latest_modeling_report_content`

Return content matches from the most recently modified saved modeling report.

Arguments:

```json
{
  "query": "elevated"
}
```

#### `search_modeling_report_sections`

Return saved modeling report sections whose headings match a case-insensitive text query.

Arguments:

```json
{
  "query": "risk"
}
```

#### `search_latest_modeling_report_sections`

Return section-heading matches from the most recently modified saved modeling report.

Arguments:

```json
{
  "query": "risk"
}
```

#### `summarize_modeling_report_sections`

Return a compact summary of recurring section headings across saved modeling reports.

#### `list_recent_modeling_reports`

Return the most recently modified saved markdown modeling reports.

Arguments:

```json
{
  "limit": 5
}
```

#### `summarize_modeling_report_catalog`

Return a compact summary of saved modeling reports, including total count, total size, and recent entries.

Arguments:

```json
{
  "limit": 5
}
```

#### `read_modeling_report`

Read one saved markdown modeling report from the local `reports/` directory.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md"
}
```

#### `list_modeling_report_sections`

List markdown section headings discovered inside one saved modeling report.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md"
}
```

#### `list_latest_modeling_report_sections`

List markdown section headings discovered inside the most recently modified saved modeling report.

#### `read_modeling_report_section`

Read one markdown section from a saved modeling report.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "section_heading": "Summary"
}
```

#### `read_latest_modeling_report_section`

Read one markdown section from the most recently modified saved modeling report.

Arguments:

```json
{
  "section_heading": "Summary"
}
```

#### `save_modeling_report_section`

Save one markdown section from a saved modeling report as a new markdown artifact.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "section_heading": "Risks",
  "new_output_name": "clinic-risks.md"
}
```

#### `save_latest_modeling_report_section`

Save one markdown section from the most recently modified saved modeling report as a new markdown artifact.

Arguments:

```json
{
  "section_heading": "Risks",
  "new_output_name": "latest-risks.md"
}
```

#### `compare_modeling_report_sections`

Return a bounded diff summary between matching sections in two saved modeling reports.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "other_output_name": "clinic-usage-v2.md",
  "section_heading": "Risks"
}
```

#### `compare_latest_modeling_report_sections`

Return a bounded diff summary for one section across the two most recently modified saved modeling reports.

Arguments:

```json
{
  "section_heading": "Risks"
}
```

#### `read_latest_modeling_report`

Read the most recently modified saved markdown modeling report from the local `reports/` directory.

#### `delete_modeling_report`

Delete one saved markdown modeling report from the local `reports/` directory.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md"
}
```

#### `rename_modeling_report`

Rename one saved markdown modeling report inside the local `reports/` directory.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "new_output_name": "clinic-usage-v2.md"
}
```

#### `rename_latest_modeling_report`

Rename the most recently modified saved modeling report inside the local `reports/` directory.

Arguments:

```json
{
  "new_output_name": "clinic-usage-latest.md"
}
```

#### `copy_modeling_report`

Copy one saved markdown modeling report into a new artifact inside the local `reports/` directory.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "new_output_name": "clinic-usage-snapshot.md"
}
```

#### `copy_latest_modeling_report`

Copy the most recently modified saved modeling report into a new artifact inside the local `reports/` directory.

Arguments:

```json
{
  "new_output_name": "latest-clinic-snapshot.md"
}
```

#### `inspect_modeling_report`

Return metadata for one saved markdown modeling report.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md"
}
```

#### `inspect_latest_modeling_report`

Return metadata for the most recently modified saved modeling report.

#### `preview_modeling_report`

Return a bounded preview of one saved markdown modeling report.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md"
}
```

#### `compare_modeling_reports`

Return a bounded unified diff summary between two saved markdown modeling reports.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "other_output_name": "clinic-usage-v2.md"
}
```

#### `compare_latest_modeling_reports`

Return a bounded unified diff summary between the two most recently modified saved modeling reports.

#### `preview_latest_modeling_report`

Return a bounded preview of the most recently modified modeling report from the local `reports/` directory.

#### `preview_csv`

Preview the first rows of a CSV file.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "rows": 5
}
```

#### `profile_csv`

Return row count, column count, dtypes, missing values, missing percentages, bounded column summaries for numeric, categorical, boolean, and datetime fields, and advisory data-quality diagnostics.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv"
}
```

Example response shape:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "row_count": 120,
  "column_count": 6,
  "missing_values": {"wait_time": 4},
  "numeric_columns": [
    {
      "column": "appointments_completed",
      "count": 120,
      "mean": 83.2,
      "std": 9.4,
      "min": 61.0,
      "q25": 77.0,
      "median": 84.0,
      "q75": 90.0,
      "max": 103.0,
      "iqr": 13.0,
      "robust_spread": 9.64,
      "histogram": [{"lower_bound": 61.0, "upper_bound": 70.0, "count": 4}],
      "iqr_outlier_count": 1,
      "z_score_outlier_count": 0,
      "skewness": 0.12,
      "quality_signals": []
    }
  ],
  "categorical_columns": [
    {
      "column": "clinic_id",
      "count": 120,
      "unique_count": 4,
      "top_value": "north",
      "top_value_frequency": 32,
      "top_values": [{"value": "north", "count": 32}],
      "rare_category_count": 0,
      "rare_category_mass": 0.0,
      "entropy": 1.97,
      "normalized_entropy": 0.98,
      "quality_signals": []
    }
  ],
  "boolean_columns": [
    {
      "column": "local_holiday",
      "true_count": 8,
      "false_count": 112,
      "missing_count": 0
    }
  ],
  "datetime_columns": [
    {
      "column": "date",
      "count": 120,
      "min": "2024-01-01T00:00:00",
      "max": "2024-04-29T00:00:00"
    }
  ],
  "profiling_limits": {
    "max_categorical_values": 5,
    "max_histogram_bins": 10,
    "max_candidate_key_columns": 12,
    "max_candidate_key_combinations": 25
  },
  "data_quality": {
    "duplicate_row_count": 0,
    "duplicate_row_percentage": 0.0,
    "candidate_keys": [],
    "empty_columns": [],
    "one_value_columns": [],
    "probable_free_text_columns": [],
    "possible_identifier_columns": []
  }
}
```

Histogram bins, top values, and candidate key searches are capped. Quality signals are advisory and include a reason plus severity and confidence metadata.

Profiling limits:

- Categorical examples are intentionally bounded by `MCP_MAX_CATEGORICAL_VALUES`.
- The profiler returns summaries rather than raw wide-table payloads.
- Datetime detection is conservative to avoid misclassifying free-text columns.
- Profile results are cached in memory by file path, size, modified time, and profiling options.
- The profile cache is thread-safe and reports path-free hit, miss, eviction, and entry metrics.

#### `detect_csv_issues`

Detect simple data-quality issues, such as high missingness and likely identifier columns.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv"
}
```

#### `query_csv_with_duckdb`

Run a safe read-only DuckDB query against the CSV loaded as a temporary table named `dataset`.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "sql": "SELECT clinic_id, AVG(appointments_completed) AS avg_completed FROM dataset GROUP BY clinic_id ORDER BY avg_completed DESC",
  "limit": 10
}
```

#### `export_query_result_to_csv`

Optionally export a safe `SELECT` or `WITH` query against a CSV, Parquet, JSON or
Excel dataset into a new UTF-8 CSV inside `MCP_DATA_ROOT`. This tool is disabled
unless the operator explicitly sets `MCP_QUERY_EXPORT_ENABLED=true`.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "sql": "SELECT clinic_id, SUM(appointments_completed) AS completed FROM dataset GROUP BY clinic_id",
  "output_name": "completed_by_clinic.csv",
  "limit": 100
}
```

The response reports the output file name, row and column counts, bytes written,
and applied row limit. The export is created atomically and never overwrites an
existing file. Only a single `.csv` file name is accepted; directories, selectors,
and traversal are rejected. Export size is bounded by `MCP_MAX_EXPORT_BYTES` and
`MCP_MAX_DATASET_BYTES`, with separate row and column caps. CSV contents are
preserved exactly as data, including strings beginning with spreadsheet formula
characters; review untrusted data before opening exports in spreadsheet software.

#### `list_sqlite_databases_tool`

Return the SQLite database files available in the configured data directory.

#### `list_sqlite_tables`

List user tables from a SQLite database.

Arguments:

```json
{
  "file_name": "clinic_metrics.sqlite"
}
```

#### `describe_sqlite_table`

Describe the columns for a SQLite table.

Arguments:

```json
{
  "file_name": "clinic_metrics.sqlite",
  "table_name": "visits"
}
```

#### `query_sqlite`

Run a safe read-only SQLite query against a database inside the configured data directory.

Arguments:

```json
{
  "file_name": "clinic_metrics.sqlite",
  "sql": "SELECT clinic, SUM(appointments) AS total_appointments FROM visits GROUP BY clinic ORDER BY total_appointments DESC",
  "limit": 10
}
```

Example response shape:

```json
{
  "file_name": "clinic_metrics.sqlite",
  "columns": ["clinic", "total_appointments"],
  "rows": [
    {"clinic": "north", "total_appointments": 22}
  ],
  "row_count": 1,
  "limit_applied": 10
}
```

#### `summarize_correlations`

Rank the top absolute correlations among numeric columns.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "method": "pearson"
}
```

#### `detect_possible_target_leakage`

Return heuristic warnings about columns that may leak the target.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed"
}
```

#### `validate_time_series_dataset_tool`

Validate whether a dataset looks ready for forecasting or time-series modeling.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "time_column": "date",
  "target_column": "appointments_completed",
  "group_column": "clinic_id"
}
```

#### `evaluate_forecast_baselines`

Evaluate transparent forecasting baselines with chronological rolling-origin backtesting.
Regular frequencies are required; irregular series are rejected instead of being scored.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "time_column": "date",
  "target_column": "appointments_completed",
  "group_column": "clinic_id",
  "forecast_horizon": 1,
  "test_size": 0.2,
  "seasonal_period": 7
}
```

The result includes last-value naive and drift baselines, plus seasonal naive when the
frequency and history support a seasonal period. Metrics include MAE, RMSE, MASE when
the training denominator is valid, and sMAPE with both-zero terms contributing `0`.
The structured result also includes an evaluation manifest with dataset fingerprint,
rolling-origin validation metadata, metric definitions, and runtime bounds.

#### `evaluate_baseline_model`

Evaluate a dummy baseline model for regression, binary classification, or multiclass classification.
The result includes an evaluation manifest with the dataset fingerprint, selected features,
split metadata, baseline definition, metric definitions, package version, Python version,
and relevant configuration bounds.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "target_column": "appointments_completed",
  "task_type": "regression",
  "test_size": 0.2,
  "random_state": 42,
  "validation_strategy": "chronological",
  "time_column": "date"
}
```

### Prompt

#### `dataset_analysis_prompt`

Creates a reusable analysis prompt for a dataset.

Arguments:

```json
{
  "file_name": "sample_clinic_usage.csv",
  "objective": "forecast clinic usage and staffing demand"
}
```

#### `modeling_report_review_prompt`

Creates a reusable review prompt for a saved modeling report.

Arguments:

```json
{
  "output_name": "clinic-usage-report.md",
  "focus": "decide whether the report is ready for stakeholder review"
}
```

Use `"latest"` for `output_name` to target the most recently modified saved modeling report.

---

## Test

```bash
poetry run pytest
```

Run static checks:

```bash
poetry run ruff check .
poetry run mypy
```

## Development Workflow

Local development uses the same quality gate as CI:

```bash
poetry run ruff check .
poetry run mypy
poetry run pytest
```

GitHub Actions runs those checks on pushes to `main` and `develop`, and on pull requests, across Python 3.11 and 3.12.

## Documentation Rule

README maintenance is part of the definition of done for this repository.

- Any user-facing capability change should update `README.md` in the same change.
- CLI, MCP tool, workflow, and setup documentation should stay aligned with the implemented repo state.
- We use the same rule as a default for other repositories unless there is a stronger repo-specific convention.

## Security

For local-only work, the default configuration keeps HTTP auth disabled.

If you set `MCP_API_KEY`, Streamable HTTP mode requires:

```text
Authorization: Bearer <your-key>
```

Limitations of this approach:

- it is a simple shared secret, not a full user or OAuth system;
- it does not apply to `stdio` transport;
- it should be paired with HTTPS and normal secret management if exposed beyond local development.

## Packaging and Release

Build local release artifacts with:

```bash
poetry build
```

This produces both a wheel and a source distribution in `dist/`.

GitHub Actions also includes a release workflow in `.github/workflows/release.yml`:

- `workflow_dispatch` builds artifacts for a manual release dry run.
- pushing a tag such as `v1.0.0` builds release artifacts from the tagged repository state.

This repository treats Git tags and GitHub releases as the primary release mechanism. Package indexes are optional and are not part of the core product workflow.

## Notebooks

The repository includes two walkthrough notebooks in `notebooks/`:

- `01_mcp_dataset_inspection.ipynb`: dataset discovery, preview, profiling, issue detection, and correlation inspection on the bundled sample CSV.
- `02_forecasting_readiness_workflow.ipynb`: synthetic healthcare dataset generation, time-series validation, leakage review, and baseline regression evaluation.

To run them locally, start Jupyter from the project environment:

```bash
poetry run jupyter notebook
```

If Jupyter is not installed in your Poetry environment yet, install it in that environment first with your preferred tool.

## Project Docs

- `CHANGELOG.md`: user-facing change history
- `CITATION.cff`: citation metadata for GitHub and Zenodo
- `CONTRIBUTING.md`: local development and PR expectations
- `SECURITY.md`: security reporting and hardening expectations
- `docs/API_CONTRACT.md`: versioned public MCP, CLI, and configuration contract
- `docs/SECURITY_MODEL.md`: trust boundaries, controls, and known security gaps
- `docs/DEPLOYMENT.md`: supported local, HTTP, Docker, and Compose deployment paths
- `docs/ARCHITECTURE.md`: component responsibilities and request flow
- `docs/RELEASE_CHECKLIST.md`: release prep checklist

---

## Docker

Build:

```bash
docker build -t ds-workspace-mcp .
```

Run:

```bash
docker run --rm -p 8000:8000 ds-workspace-mcp
```

Mount your own datasets:

```bash
docker run --rm \
  -p 8000:8000 \
  -v "$(pwd)/data:/app/data" \
  -v "$(pwd)/reports:/app/reports" \
  ds-workspace-mcp
```

Run with Docker Compose:

```bash
docker compose up --build
```

The Compose setup reads defaults from `.env.example`, overrides from `.env` when present, binds the server to `0.0.0.0`, publishes `MCP_PORT`, mounts local `data/` at `/app/data`, and mounts local `reports/` at `/app/reports` so saved reports survive container recreation.

To inspect the running HTTP server with MCP Inspector:

```bash
npx -y @modelcontextprotocol/inspector
```

To stop the stack:

```bash
docker compose down
```

---

## Safety design

The server does not read arbitrary files.

All dataset paths are resolved inside `MCP_DATA_ROOT`, which defaults to `./data`. Path traversal attempts such as `../secret.csv` are rejected.

Supported tabular files are CSV, Parquet, JSON, and Excel. SQLite database files are
supported through the dedicated read-only SQLite tools.

DuckDB query safety notes:

- CSV files are resolved through the existing safe data-root checks before DuckDB sees any data.
- Queries must be a single `SELECT` or `WITH` statement against the temporary `dataset` table.
- DuckDB external access and extension autoload/autoinstall are disabled before user SQL executes.
- Parsed SQL relation references must resolve only to the registered in-memory `dataset` relation.
- Destructive or schema-changing SQL is rejected.
- External file-reading functions such as `read_csv(...)` are rejected.
- A final row limit is always applied, with `MCP_MAX_SQL_ROWS` acting as the upper bound.
- SQL text is bounded by `MCP_MAX_SQL_QUERY_LENGTH`.
- Interruption is requested when execution or result materialization exceeds `MCP_SQL_TIMEOUT_MS`.
- Successful query results are cached in memory by normalized SQL, resolved dataset
  identity, file size, file modification time, and row limit. Disable caching with
  `MCP_QUERY_CACHE_ENABLED=false` if queries rely on volatile functions or always
  need immediate re-execution.
- CSV reads require UTF-8 (optional UTF-8 BOM). Empty or malformed CSV and mixed
  encodings fail with explicit, path-free errors; header-only CSV remains valid.
- Query result export is the only optional dataset-writing tool. It is off by
  default, writes only new CSV files under `MCP_DATA_ROOT`, and enforces limits
  before publishing a complete file.

SQLite safety notes:

- Only `.sqlite`, `.sqlite3`, and `.db` files inside `MCP_DATA_ROOT` are allowed.
- SQLite connections are opened in read-only mode.
- Schema-changing and destructive SQL is rejected.
- Query execution is limited to a single `SELECT` or `WITH` statement.
- A final row limit is always applied, with `MCP_MAX_SQL_ROWS` acting as the upper bound.
- SQL text is bounded by `MCP_MAX_SQL_QUERY_LENGTH`.
- Long-running queries and result fetching are interrupted via SQLite progress handlers when they exceed `MCP_SQL_TIMEOUT_MS`.

Diagnostics notes:

- Correlation summaries are limited to numeric columns and return only the top ranked pairs.
- Leakage warnings include severity and confidence, and remain review signals rather than proof.
- Exact target duplicates and likely identifiers are strong exclusion evidence for baseline feature selection.
- Name overlap and strong numeric correlation are review evidence by themselves, not automatic exclusion proof.

Time-series validation notes:

- Timestamp parsing is conservative and fails if the declared time column cannot be parsed at all.
- The validator checks sorting, duplicate timestamps, inferred frequency, missing intervals, grouped gaps, and missing target values.
- Frequency inference reports a structured kind: `regular`, `approximately_regular`, `irregular`, `insufficient_data`, or grouped `heterogeneous`.
- Missing intervals are counted only after a regular or sufficiently supported approximate fixed interval is established.
- Grouped series infer frequency per group and avoid reporting one global frequency when groups disagree.
- History-length warnings are heuristic and intended for baseline forecasting readiness, not strict modeling requirements.
- Forecast baselines use chronological rolling-origin evaluation and require regular inferred frequencies.
- Seasonal naive runs only when an explicit or inferred seasonal period is valid for the horizon and training history.
- Forecast baseline metrics include MAE, RMSE, MASE when the denominator is valid, and sMAPE with both-zero terms contributing `0`.

Baseline model notes:

- Only scikit-learn dummy baselines are used.
- `regression`, `binary_classification`, and `multiclass_classification` are supported.
- Validation can be `random`, `stratified`, `chronological`, or `grouped`; classification defaults to stratified validation when feasible.
- Classification baselines report full, train, and test class counts and fail clearly when a split cannot represent every class.
- Classification metrics include accuracy, balanced accuracy, macro F1, and weighted F1.
- Chronological validation requires a parseable time column, never shuffles, and reports train/test time boundaries.
- Grouped validation requires a group column and reports train/test group counts plus group-overlap status.
- These metrics are reference baselines for comparison, not final model quality targets.

---

## Suggested Portfolio Positioning

> An MCP server that gives AI assistants safe, structured access to local analytical datasets, SQL sources, and reusable modeling report workflows.

This project demonstrates AI engineering, data tooling, typed Python service design, MCP integration, analytical workflow ergonomics, and security-minded local tool design.

