Metadata-Version: 2.5
Name: tcia_utils
Version: 4.0.0a17
Summary: A small Python client for TCIA research discovery and evidence.
Project-URL: Homepage, https://github.com/kirbyju/tcia_utils
Project-URL: Bug Tracker, https://github.com/kirbyju/tcia_utils/issues
Author-email: Justin Kirby <justin.kirby@nih.gov>
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Requires-Dist: httpx<1,>=0.27
Provides-Extra: downloads
Requires-Dist: tcia-data-retriever==0.1.0rc3; extra == 'downloads'
Provides-Extra: examples
Requires-Dist: ipykernel<8,>=7; extra == 'examples'
Requires-Dist: nbclient<1,>=0.11; extra == 'examples'
Requires-Dist: nbformat<6,>=5; extra == 'examples'
Requires-Dist: nibabel<6,>=5; extra == 'examples'
Requires-Dist: pandas<4,>=2; extra == 'examples'
Requires-Dist: tcia-data-retriever==0.1.0rc3; extra == 'examples'
Provides-Extra: tables
Requires-Dist: pandas>=2; extra == 'tables'
Provides-Extra: test
Requires-Dist: build>=1; extra == 'test'
Requires-Dist: pytest>=8; extra == 'test'
Description-Content-Type: text/markdown

# TCIA Utils 4.0

TCIA Utils is a Python client for discovering TCIA datasets, participants, clinical evidence, available data, citations, and verified analytical publications. It uses the shared TCIA V2 REST service. A researcher can start with a question about a dataset without choosing the underlying catalog or storage system.

Version **4.0.0a17** is an opt-in preview for testing the redesigned interface.
Ordinary pip installations continue selecting stable 3.x. Install this version
explicitly in a fresh Python 3.11+ environment:

```sh
python -m pip install "tcia_utils[tables,downloads]==4.0.0a17"
```

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/kirbyju/TCIA_Notebooks/blob/main/tcia_utils_v4/TCIA_Dataset_and_Participant_Queries.ipynb)

The [guided workshop](https://github.com/kirbyju/TCIA_Notebooks/blob/main/tcia_utils_v4/TCIA_Dataset_and_Participant_Queries.ipynb) pins its own tested preview version and queries real public metadata. Public samples use `max_items=1`; controlled transfers remain opt-in. Report feedback through [GitHub issues](https://github.com/kirbyju/tcia_utils/issues), including the failing operation, package/Python versions, and error message. Do not include credentials. Existing 3.x scripts should stay in a separate environment; see [migration guidance](MIGRATION.md).

The client requires Python 3.11 or newer and `httpx`. Notebook dependencies are in [the tutorial requirements](https://github.com/kirbyju/TCIA_Notebooks/blob/main/tcia_utils_v4/requirements.txt). The built wheel is the package used for notebook verification; an editable checkout is not required.

Provide the URL of a TCIA V2 REST host that has a ready installed bundle. The client does not choose a host or make a request during import or construction.

```python
from tcia_utils import TCIAClient

with TCIAClient(base_url="https://tcia.duckdns.org") as tcia:
    service = tcia.status()
    page = tcia.search_datasets(query="lung", limit=10)
    for dataset in page.items:
        print(dataset["dataset_type"], dataset["short_title"], dataset.get("doi"))
    print(page.completeness.state, service.provenance.release_fingerprint)
```

Searches return one `Page` at a time. Call `page.next_page()` or iterate `page.iter_pages()` when more results are needed. Detail calls return a `Result` containing the service's named fields and notes. Both expose observed release provenance and completeness. `partial` means more cursor pages are available; `unknown` means the route has no exhaustive cursor or contains a bounded sublist. An empty successful page is a real zero-match result. Failures raise typed `TCIAError` subclasses: for example, `ServiceUnavailableError` for a missing capability or HTTP 503, `HTTPResponseError` for HTTP 500, and `TransportError` for a connection failure.

For pandas-free applications, install `tcia_utils==4.0.0a17` without extras.
Every public result provides `.to_dict()` and `.to_json()`; these retain records,
provenance and completeness instead of requiring a DataFrame. For example,
`page.to_json(indent=2)` serializes the current search page, while
`details.to_dict()["data"]` accesses a detail result's native dictionaries and lists.
Exports never fetch extra pages. See [the JSON guide](docs/v4/user-guide.md#work-with-dictionaries-and-json-without-pandas)
for saving results and collecting pages explicitly.

Collection search now returns `data_categories` directly from the shared service, without per-dataset enrichment calls. `plan_downloads()` consumes the shared REST/MCP download plan, including WordPress-derived software requirements and one software-help catalog. Runtime prerequisite checks remain in the Python client.

To plan and download a dataset from Python:

```python
with TCIAClient(base_url="https://tcia.duckdns.org", pin_snapshot=True) as tcia:
    plan = tcia.plan_downloads("CMB-GEC")
    print(plan.to_dataframe()[["download_id", "download_category", "access_level"]])
    readiness = tcia.download_dataset("CMB-GEC", "downloads", dry_run=True)
    print(readiness.to_dataframe())  # Metadata and local prerequisite checks only.
    # Download public groups when ready (can be large):
    # After choosing a supported imaging download_id from plan:
    # report = tcia.download_dataset("CMB-GEC", "downloads",
    #     download_ids=[chosen_download_id], max_items=1, accept_data_policy=True)
```

Optional transfers delegate to `tcia-data-retriever==0.1.0rc3`, including its pip-bundled headless executable. `tcia_utils[downloads]` installs it. The shared TCIA service resolves basic public DICOM and non-DICOM participant selections; it reuses IDC metadata for DICOM. The Python client does not install `idc-index`. Advanced IDC queries and viewers remain available through IDC's own tools. The SDK prepares exact, bounded selections and reports; TDR owns downloads, retries, verification, authorization and output layout. No separate Go/desktop installation is needed. Aspera groups remain external-only. See the [download guide](docs/v4/user-guide.md#download-from-python) for policy acceptance, credential handling and current interface limits.

Use [the task-oriented guide](docs/v4/user-guide.md) for methods, evidence scope, selection exports, and tool handoffs. The [migration guide](MIGRATION.md) explains the 3.x API change; [troubleshooting](docs/v4/troubleshooting.md) covers setup and response errors. The canonical [guided workshop](https://github.com/kirbyju/TCIA_Notebooks/blob/main/tcia_utils_v4/TCIA_Dataset_and_Participant_Queries.ipynb) teaches dataset discovery/download planning, participant cohorts, brain MR segmentations, and controlled CT/SEG access in one conversational lesson. It uses real public metadata by default; public image cells run bounded samples; controlled transfers remain opt-in. It shows download categories and uses package helpers for page collection, clinical enrichment, manifest saving, and direct Python downloads. Original slide packages need configured transfer software; indexed NIfTI paths still require their published package rather than an invented per-file URL. The participant example forms a final cohort by combining imaging modality,
diagnosis and primary disease site; the initial modality candidates are labeled
separately from the clinically filtered results.

This alpha is for community testing. The [service compatibility review](docs/v4/design/service-compatibility.md) and [notebook validation](docs/v4/design/notebook-validation.md) preserve dated verification and earlier failure history. The four original hosted-route failures were repaired and subsequently passed live checks. API availability, indexed coverage, and the active metadata snapshot can change; retain response provenance with research exports. Clinical summaries are derived metadata, and some routes are bounded rather than exhaustive.

Visible TCIA Collection and Analysis Result records establish dataset publication. DataCite metadata and publication metrics enrich those records but do not establish publication by themselves. Public metadata about controlled data does not grant payload access. For advanced public DICOM metadata queries and viewers, use IDC's own tools. Selection export prepares route-specific manifests without transferring payloads. Explicit `download_dataset()` and `download_files()` calls perform transfers; public notebook samples run with max_items=1, while controlled examples require the reader's approved key and explicit selection.

TCIA data use remains subject to the [TCIA Data Usage Policy](https://www.cancerimagingarchive.net/data-usage-policies-and-restrictions/).

## Manifest paths and URLs

```python
with TCIAClient(base_url="https://tcia.duckdns.org") as tcia:
    report = tcia.download_manifest("cohort.tcia", "downloads", dry_run=True)
    # When ready, omit dry_run to transfer the entire supported selection.
    # report = tcia.download_manifest("https://example.org/cohort.csv", "downloads", accept_data_policy=True)
```

Supported inputs are `.tcia` and CSV/TSV with exactly one route column:
`SeriesInstanceUID`, `imageUrl`, or `drs_uri`. Keys are required for controlled DRS
transfers. Public UID manifests are checked against the maintained imaging index
before any payload starts; missing series are reported rather than silently
skipped. The client handles parsing and transfer adapters internally. The REST
service does not receive local manifests or credentials.

`plan_downloads()` is an optional metadata-only review of download groups, access
terms, software requirements and Python route support. `download_dataset()` does
this planning internally; you do not need to call both. `download_manifest()` can
use an existing manifest directly. The shared service owns the published-requirement
interpretation; inspecting a plan does not check personal access approval.
See [the download guide](docs/v4/user-guide.md#download-from-python).


To test a transfer or keep a demo small, pass `max_items=1` to
`download_manifest()`, `download_dataset()`, or `download_files()`. The report
marks a sample and counts what was selected and omitted. The limit counts whole
DICOM series/files, not bytes; original packages without file-level selection
cannot be sampled. See the sample-download section of the v4 user guide.

The destination is passed directly to TDR, which arranges files using available
metadata. Public DICOM has collection, participant, study and series folders;
file manifests can have fewer levels. The SDK adds no dataset or manifest-hash
directory layers. Normalized transfer lists are kept in
`destination/manifests/`. See the download guide for resuming older output trees.

Download preparation is visible immediately. Actual transfers require explicit
`accept_data_policy=True`. To expand a sample, keep the same destination and omit
`max_items`; TDR automatically checks and reuses existing files. To retry an
interrupted download, rerun the same call with the same destination. This does
not promise partial-file byte-range resume. TDR RC3 counts accepted existing files
as `Skipped`; `Synced` identifies
public DICOM sync jobs, which can still fetch missing files. See the guide for details.
TCIA Utils requests plain progress lines, which TDR streams into Python/notebook output. Controlled keys are
invocation-only and are not copied into the GUI credential cache.
Read the download guide before using credentials. No dependency is installed at runtime.

For selected participants whose files require Aspera, `export_aspera_manifest()`
writes the exact `packageUrl,packagePath` CSV used by the TCIA Participant Explorer.
It queries metadata only, preserves dataset-scoped identity and current package
links, and supports `max_items` for a small file-list sample. The separate
[Aspera notebook](https://github.com/kirbyju/TCIA_Notebooks/blob/main/TCIA_Aspera_CLI_Downloads.ipynb)
installs and invokes Aspera; the SDK does not.
