Metadata-Version: 2.5
Name: pidatametrics
Version: 0.7.1
Summary: Pi Datametrics API client and reporting tools with CSV, Excel, Google Sheets and BigQuery exports.
Requires-Python: >=3.9
Requires-Dist: google-auth
Requires-Dist: google-cloud-bigquery
Requires-Dist: gspread
Requires-Dist: openpyxl
Requires-Dist: pandas
Requires-Dist: python-dateutil
Requires-Dist: requests
Description-Content-Type: text/markdown

# pidatametrics

Python client for the Pi Datametrics API, with report exports to CSV, Excel,
Google Sheets and BigQuery. Implements every non-deprecated endpoint in the
**13 January 2026 API specification**, including sections 2–5 and 7–12.
Section 6 (Position Reports) is deliberately excluded.

IDs, names, query text, and export destinations in these examples are illustrative
placeholders. Replace them with your own configuration. The Colab script ships
with no account or source IDs selected and prompts for them at runtime. It also
accepts `PI_ACCOUNT_ID` from an environment variable. Credentials come from
secrets/environment variables or hidden prompts, never from embedded values.

**New in 0.7.1:** [one Colab script for all endpoints](examples/colab_all_endpoints.py),
numbered menus, multiple sources, latest-period defaults, and custom dates/filters.
See [Colab and multiple sources](#colab-and-multiple-sources) below.

## Install

Install the updated code from this directory:

```bash
python -m pip install -e .
```

For the published PyPI version:

```python
# Google Colab cell
!pip install -q pidatametrics
```

The PyPI command installs the published release; changes in this repository will
only be available through PyPI after a new release is published. To use this
checkout in Colab, upload/build its wheel and install that wheel instead.

## Connect

```python
import os
from pidatametrics import PiDataMetrics, PiReportManager

CLIENT_ID = os.environ["PI_CLIENT_ID"]
CLIENT_SECRET = os.environ["PI_CLIENT_SECRET"]
ACCOUNT_ID = 100

pi = PiReportManager(CLIENT_ID, CLIENT_SECRET, ACCOUNT_ID)
# PiDataMetrics has the same endpoint methods without the report workflows.
```

In Colab, credentials can instead come from secrets:

```python
from google.colab import userdata
CLIENT_ID = userdata.get("PI_CLIENT_ID")
CLIENT_SECRET = userdata.get("PI_CLIENT_SECRET")
pi = PiReportManager(CLIENT_ID, CLIENT_SECRET, ACCOUNT_ID)
```

Google authentication is only needed for Google Sheets/BigQuery exports:

```python
from google.colab import auth
auth.authenticate_user()
```

Every endpoint method below is available on both `PiDataMetrics` and
`PiReportManager`. Methods return the API's `data` unchanged: a list, dictionary,
or `None`, depending on the endpoint. Raw SERP responses can also be a top-level
list. They do not export automatically or flatten nested data.

Use IDs returned by discovery methods for your own account. The example IDs and
periods below are illustrative; choose periods in the workspace's tracking history.

```python
WORKSPACE_ID = 1001
ENGINE_ID = 1
GROUP_ID = 1028
SITE_ID = 10
INDEX_ID = 3
RAPID_WORKSPACE_ID = 1304
CAMPAIGN_ID = 1200
MI_ID = 100
VERSION = 1
PERIOD = "2025-01-01"
RAPID_PERIOD = "2025-01-01T04/PT2H"
```

## 2. Accounts and user — all six endpoints

```python
user = pi.get_user()                                     # 2.1 /api/me
accounts = pi.get_accounts()                             # 2.2 /api/accounts
account = pi.get_account()                               # 2.3 /api/accounts/{account}
quota = pi.get_account_quota()                           # 2.4 .../quota
sites = pi.get_account_sites()                           # 2.5 .../sites
structure = pi.get_account_structure()                   # 2.6 .../account-structure
filtered_structure = pi.get_account_structure(workspace_ids=[WORKSPACE_ID])
```

The account is selected by `ACCOUNT_ID` when constructing the client. If you
need to discover accessible accounts first, omit it:

```python
with PiDataMetrics(CLIENT_ID, CLIENT_SECRET) as discovery:
    print(discovery.get_accounts())
# Then construct a client with the chosen account ID.
```

## 3. Global search engines — both endpoints

```python
engines = pi.get_search_engines()                         # 3.1 /api/search-engines
uk_english_engines = pi.get_search_engines(market="gb", language="en")
engine = pi.get_search_engine(ENGINE_ID)                  # 3.2 /api/search-engines/{engine}
```

`market` is an ISO 3166-1 alpha-2 country code, and `language` is an ISO 639-1
code. The specification's URI uses `language`, although its parameter table
calls it `language-code`; this client follows the URI.

## 4. Standard workspace setup — all eight endpoints

These routes are under `/api/accounts/{account}/workspaces`.

```python
workspaces = pi.get_workspaces()                         # 4.1
workspace = pi.get_workspace(WORKSPACE_ID)                # 4.2 /{workspace}
history = pi.get_workspace_tracking_history(WORKSPACE_ID) # 4.3 /{workspace}/tracking-history
sites = pi.get_workspace_sites(WORKSPACE_ID)              # 4.4 /{workspace}/sites
groups = pi.get_stgs(WORKSPACE_ID)                        # 4.5 /{workspace}/search-term-groups
groups = pi.get_search_term_groups(WORKSPACE_ID)          # descriptive alias of get_stgs
all_terms = pi.get_workspace_search_terms(WORKSPACE_ID)   # 4.6 /{workspace}/search-terms
group_terms = pi.get_search_terms(WORKSPACE_ID, GROUP_ID)  # 4.7 /{workspace}/search-term-groups/{group}/search-terms
site = pi.get_site(WORKSPACE_ID, SITE_ID)                  # 4.8 /{workspace}/sites/{site}
```

## 5. Visibility indexes — all four operations

Routes are under `/api/accounts/{account}/workspaces/{workspace}/visibility-indexes`.
Group scores use the same `site-scores` route as site scores, adding `site-id`.

```python
indexes = pi.get_visibility_indexes(WORKSPACE_ID)         # 5.1
index = pi.get_visibility_index(WORKSPACE_ID, INDEX_ID)    # 5.2 /{index}
site_scores = pi.get_visibility_index_site_scores(        # 5.3 /{index}/site-scores
    WORKSPACE_ID, INDEX_ID, period=PERIOD,
)
group_scores = pi.get_visibility_index_search_term_group_scores(  # 5.4
    WORKSPACE_ID, INDEX_ID, SITE_ID, period=PERIOD,
)
```

## 7. Standard workspace search data — all three endpoints

Routes are under `/api/accounts/{account}/workspaces/{workspace}/search-data`.
Match `period` to the tracking frequency:

| Frequency | Format | Example |
| --- | --- | --- |
| Daily | `YYYY-MM-DD` | `2025-01-01` |
| Weekly | `YYYY-Www` | `2025-W01` |
| Monthly | `YYYY-MM` | `2025-01` |

### 7.1 Returning site URLs

```python
urls = pi.get_returning_site_urls(
    WORKSPACE_ID, ENGINE_ID, SITE_ID, PERIOD,
    stg_ids=[GROUP_ID],       # optional: search-term-group-id[]
    ranking_mode="classic", # optional: classic or hypergraph (API default)
)
```

### 7.2 Bulk search results

All documented filters are supported:

```python
results = pi.get_bulk_search_results(
    WORKSPACE_ID, ENGINE_ID, PERIOD,
    stg_ids=[GROUP_ID],                    # search-term-group-id[]
    number_of_results=25,                 # number-of-results; API default 100
    ranking_mode="hypergraph",           # ranking-mode; or classic
    site_id=SITE_ID,                      # site-id
    top_url_only=True,                    # top-url-only; requires site_id
    ranked_features_only=False,           # ranked-features-only
    serp_features=["classicLink", "popularProducts"],  # serp-feature[]
)
# All filters are optional; omit site_id/top_url_only to include other sites.
results = pi.get_bulk_search_results(WORKSPACE_ID, ENGINE_ID, PERIOD)
```

Existing scripts can continue using the original method and API-style keys:

```python
results = pi.get_bulk_serp_data(
    WORKSPACE_ID, ENGINE_ID, PERIOD,
    **{"number-of-results": 25, "serp-feature[]": ["popularProducts"]},
)
```

`get_bulk_serp_data` retains the package's original `ranking-mode=absolute`
default for compatibility. **The supplied specification only documents
`classic` and `hypergraph`.** Use `get_bulk_search_results` for the documented
default, or pass `ranking_mode="hypergraph"` to the original method.

### 7.3 Raw bulk search results

```python
raw_results = pi.get_bulk_search_results_raw(
    WORKSPACE_ID, ENGINE_ID, PERIOD,
    stg_ids=[GROUP_ID],
    serp_features=["popularProducts", "images"],
)
```

Raw data retains the nested sections and items without ranking or flattening.
The standard raw endpoint's example includes `number-of-results`, but its
parameter table does not; the named method exposes the documented table's
filters. For that example-only parameter, use the escape hatch below.

Updated feature examples supplied for this package (camelCase):

```python
FEATURES = [
    "aiOverview", "textAds", "answerCard", "appPacks", "carousel", "citations",
    "classicLink", "classicLinkCarousel", "classicLinkSitelinks", "datasets",
    "sportsResults", "filteredResults", "finance", "findResultsOn", "flightPack",
    "hotelPack", "images", "jobs", "knowledgePanel", "maps", "peopleAlsoAsk",
    "peopleAlsoBuyFrom", "peopleAlsoSearch", "products", "productListedAds",
    "sitesCarousel", "siteLinks", "sourcePanel", "topPosts", "twitterCards", "videos",
]
```

`from pidatametrics import SERP_FEATURES` provides the corresponding display-name
to camelCase mapping. `aiOverview` and `textAds` use the explicitly supplied
updated spellings; other updated labels are converted to camelCase from the
supplied names. These updated values have not been verified against a live API.
The older PDF used some different identifiers, including `apps`, `eventResults`,
`knowledgeGraph`, `adwords`, and `popularProducts`. Filters are not restricted to
this list: pass the identifier accepted by your API/data source, including old
or custom values. No automatic substitution changes your filters.

## 8. Search volume — one endpoint

`/api/accounts/{account}/workspaces/{workspace}/volume-data/bulk-search-volume`

```python
# Most recent 36 available months, according to the specification:
volume = pi.get_bulk_volume(WORKSPACE_ID)

volume = pi.get_bulk_volume(
    WORKSPACE_ID,
    start_date="2025-01",  # maps to start-period; monthly format
    end_date="2025-03",    # maps to end-period; provide both dates or neither
    stg_ids=[GROUP_ID],
    search_terms=["example query", "work suits"],
    currency="GBP",        # ISO 4217; defaults to workspace currency
)
```

Returns original fields including monthly volume, CPC, competition, currency,
root term (when present), and status. Unavailable/invalid terms may omit volume.

## 9. Rapid workspace setup — all five endpoints

Routes are under `/api/accounts/{account}/rapid-workspaces`.

```python
rapid_workspaces = pi.get_rapid_workspaces()                             # 9.1
rapid_workspace = pi.get_rapid_workspace(RAPID_WORKSPACE_ID)              # 9.2 /{workspace}
rapid_sites = pi.get_rapid_workspace_sites(RAPID_WORKSPACE_ID)             # 9.3 /{workspace}/sites
campaigns = pi.get_rapid_workspace_campaigns(RAPID_WORKSPACE_ID)           # 9.4 /{workspace}/campaigns
campaign = pi.get_rapid_workspace_campaign(RAPID_WORKSPACE_ID, CAMPAIGN_ID) # 9.5 /{workspace}/campaigns/{campaign}
```

## 10. Rapid workspace search data — all three endpoints

Routes are under
`/api/accounts/{account}/rapid-workspaces/{workspace}/campaigns/{campaign}/search-data`.
Periods use `YYYY-MM-DDTHH/PT2H`, for example `2025-01-01T04/PT2H`.

```python
# 10.1 /returning-site-urls
rapid_urls = pi.get_rapid_returning_site_urls(
    RAPID_WORKSPACE_ID, CAMPAIGN_ID, ENGINE_ID, SITE_ID, RAPID_PERIOD,
    ranking_mode="hypergraph",  # optional; or classic
)

# 10.2 /bulk-search-results
rapid_results = pi.get_rapid_bulk_search_results(
    RAPID_WORKSPACE_ID, CAMPAIGN_ID, ENGINE_ID, RAPID_PERIOD,
    number_of_results=25,       # optional; API default 100 URLs
    ranking_mode="hypergraph", # optional; or classic
)

# 10.3 /bulk-search-results-raw
rapid_raw = pi.get_rapid_bulk_search_results_raw(
    RAPID_WORKSPACE_ID, CAMPAIGN_ID, ENGINE_ID, RAPID_PERIOD,
    number_of_results=5,        # optional; limits blocks per search result
)
```

## 11. Market intelligence — all six endpoints

Routes are under `/api/accounts/{account}/market-intelligence`.

```python
reports = pi.get_market_intelligence_list()                              # 11.1
report = pi.get_market_intelligence(MI_ID)                                # 11.2 /{mi}
versions = pi.get_market_intelligence_versions(MI_ID)                      # 11.3 /{mi}/versions
version = pi.get_market_intelligence_version(MI_ID, VERSION)               # 11.4 /{mi}/versions/{version}
components = pi.get_market_intelligence_components(MI_ID)                  # 11.5 /{mi}/components
older_components = pi.get_market_intelligence_components(MI_ID, version=VERSION)
component = pi.get_market_intelligence_component(MI_ID, "sov-leaders")     # 11.6 /{mi}/components/{key}
older_component = pi.get_market_intelligence_component(
    MI_ID, "sov-leaders", version=VERSION,
)
```

Omit `version` for the most recent version. Discover available component keys
with `get_market_intelligence_components`; each report may include a different
set. The single component method supports all eight examples in the document:

```python
COMPONENT_KEYS = [
    "sov-leaders",
    "sov-timeline",
    "market-rank",
    "estimated-revenue-performance",
    "estimated-revenue-opportunity",
    "historic-revenue-potential",
    "estimated-organic-traffic",
    "search-volume-timeline",
]
available = pi.get_market_intelligence_components(MI_ID)
component_data = {
    key: pi.get_market_intelligence_component(MI_ID, key)
    for key in COMPONENT_KEYS if key in available
}
```

Component data stays nested to preserve categories, subcategories, sites,
timelines, totals, currencies, and date ranges. Each component request costs
200 API units according to the supplied specification.

## 12. LLM mentions — one endpoint

This uses the global route `/api/data/llm/mentions`, passing account and
workspace IDs as query parameters.

```python
mentions = pi.get_llm_mentions(
    WORKSPACE_ID, ENGINE_ID,
    start_period="2025-01-01",
    end_period="2025-01-07",
    stg_ids=[GROUP_ID],  # optional
)
# Weekly LLM tracking:
weekly_mentions = pi.get_llm_mentions(
    WORKSPACE_ID, ENGINE_ID, "2025-W36", "2025-W38",
)
```

Returns query/response text and nested mentions with entity, citation,
source site/domain/URL, sentiment, and sentiment category. The client uses
`search-engine-id`, correcting the typo in the specification's example URL.

## Existing report workflows and exports

Your four existing `PiReportManager` workflows remain available with their
existing arguments. Google Sheets and BigQuery require Google credentials and
permissions; CSV and Excel do not. A report fetches multiple endpoints and can
therefore consume more quota than a single endpoint call.

```python
SERP_SOURCES = [
    ["UK", 1001, "Example workspace", 1, "Example engine A"],
    ["UK", 1001, "Example workspace", 2, "Example engine B"],
    ["UK", 1001, "Example workspace", 3, "Example engine C"],
]
BQ_CONFIG = {"project": "project-id", "dataset": "example_dataset", "table": "example_report"}

# 1. Search volume: comma-separated IDs, or a workspace-name substring.
pi.run_volume_report(workspace_ids="1001", output_mode="csv", filename="volume")
# Alternative: workspace_name="Example workspace" selects matching tracked workspaces.

# 2. Yesterday's SERP data.
pi.run_serp_report(data_sources=SERP_SOURCES, output_mode="excel", filename="serp")

# 3. Historical SERP data, stepping backwards from the start date.
pi.run_historical_serp_report(
    data_sources=SERP_SOURCES, duration=7, frequency="daily",
    start_date="2025-01-07", features=["popularProducts"], num_results=25,
    output_mode="csv", filename="historical",
)

# 4. Flattened LLM mentions.
pi.run_llm_report(
    data_sources=SERP_SOURCES, start_period="2025-01-01", end_period="2025-01-07",
    output_mode="gsheet", spreadsheet_name="Pi LLM Mentions", filename=None,
)

# BigQuery appends rows to the configured table; repeat runs can duplicate data.
pi.run_llm_report(
    data_sources=SERP_SOURCES, start_period="2025-01-01", end_period="2025-01-07",
    output_mode="bigquery",
    bq_config={**BQ_CONFIG, "table": BQ_CONFIG["table"] + "_llm"}, filename=None,
)
```

`output_mode` accepts `csv`, `excel`, `gsheet`, or `bigquery`.
`filename` is now honored as the output file base name or Google Sheets tab
name; pass `None` for a generated name. CSV/Excel extensions are added as needed.
The existing Sheets exporter clears an existing tab with the same name before
writing. Use a new name to retain an earlier export.

The four report helpers preserve their existing flattened schemas and return
`None`. Historical reports accept `daily`, `weekly`, or `monthly`; supply the
starting date as `YYYY-MM-DD`. Weekly requests now use the ISO week containing
that date (`YYYY-Www`), and monthly requests use `YYYY-MM`, matching the API.
New endpoint data is available directly through the methods above. To
save a complete nested response without losing fields:

```python
import json
from pathlib import Path

data = pi.get_market_intelligence_component(MI_ID, "sov-leaders")
Path("sov-leaders.json").write_text(
    json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8",
)
```

For a flat list of dictionaries, use the existing exporters directly:

```python
from pidatametrics import PiExporter

rows = pi.get_visibility_index_site_scores(WORKSPACE_ID, INDEX_ID, PERIOD)
PiExporter.to_csv(rows, "visibility_scores")
PiExporter.to_excel(rows, "visibility_scores")
# PiExporter.to_google_sheet(rows, "Pi Scores", tab_name="Scores")
# PiExporter.to_bigquery(rows, "project-id", "dataset", "table")
```

## Authentication lifecycle and authorization-code applications

Construction posts a client-credentials grant to `/api/auth`. Access tokens
last one hour according to the specification. Token renewal is explicit:
construct a new client for client-credentials access when its token expires.

The optional authorization-code flow is also supported. Register the
application in Pi first, then direct the user to this URL:

```python
import secrets
state = secrets.token_urlsafe(32)  # Store in the user's server-side session.
url = PiDataMetrics.authorization_url(CLIENT_ID, "https://example.com/callback", state)
print(url)
```

After the callback, verify its `state` equals your stored value, then exchange
the returned code:

```python
tokens = PiDataMetrics.request_token(
    CLIENT_ID, CLIENT_SECRET, grant_type="authorization_code",
    code="CODE_FROM_CALLBACK", redirect_uri="https://example.com/callback",
)
pi = PiDataMetrics(account_id=ACCOUNT_ID, access_token=tokens["access_token"])

# Later, renew using the stored refresh token:
refreshed = PiDataMetrics.request_token(
    CLIENT_ID, CLIENT_SECRET, grant_type="refresh_token",
    refresh_token=tokens["refresh_token"],
)
pi.close()
pi = PiDataMetrics(account_id=ACCOUNT_ID, access_token=refreshed["access_token"])
# Store a replacement refresh_token too, if the server returns one.
```

For a full client-credentials token response, including expiry fields returned
by Pi, call `PiDataMetrics.request_token(CLIENT_ID, CLIENT_SECRET)`.

## Errors, quota, timeouts, and custom parameters

```python
import requests
from pidatametrics import PiAPIError

try:
    data = pi.get_workspaces()
    print(pi.last_meta)      # Includes status/quota when supplied by the API.
    # pi.last_response contains the complete last successful JSON response.
except PiAPIError as exc:
    print(exc.meta)          # API errors reported within JSON metadata.
except requests.exceptions.HTTPError as exc:
    print(exc.response.status_code, exc.response.text)
except requests.exceptions.RequestException as exc:
    print("Request failed:", exc)
```

Requests use a 60-second timeout by default. GET requests retry server-side
5xx errors up to three times with exponential backoff; authentication POSTs
are not retried automatically. HTTP failures raise exceptions instead of
terminating the Python process. Existing report/export helpers may catch
errors and print them; use endpoint methods when your code needs to handle
exceptions directly.

```python
with PiDataMetrics(CLIENT_ID, CLIENT_SECRET, ACCOUNT_ID,
                   timeout=120, max_retries=5) as client:
    print(client.get_account_quota())
```

`max_retries` accepts 0–9 (at most ten attempts). Array parameters are encoded
as repeated bracketed keys, and booleans as `true`/`false`. Optional `None`
values are omitted. Quota is charged by the server; inspect `get_account_quota`
and `last_meta` before making large numbers of requests.

The existing account-relative escape hatch remains available:

```python
data = pi.fetch_endpoint(
    f"workspaces/{WORKSPACE_ID}/search-data/bulk-search-results-raw",
    params={"search-engine-id": ENGINE_ID, "period": PERIOD,
            "number-of-results": 5, "serp-feature[]": ["images"]},
)
pi.close()  # Or use a with block.
```

## Development checks

```bash
python -m pip install -e . pytest build
python -m pytest
python -m build
```

Tests mock HTTP responses: they check every endpoint's route, query encoding,
response shapes, OAuth forms, errors, and compatibility with existing reports
without consuming API quota. Live requests require your own Pi credentials.

## Colab and multiple sources

### One script, numbered choices, latest by default

Use [examples/colab_all_endpoints.py](examples/colab_all_endpoints.py).
It contains configuration, installation, credential prompts, all 39 endpoint
choices, the four original report workflows, and export handling in one file.

1. Download `examples/colab_all_endpoints.py` and the
   `dist/pidatametrics-0.7.1-py3-none-any.whl` built from this repository.
2. Open a fresh Google Colab session. Paste the entire Python script into one
   cell. Edit `ACCOUNT_ID`, `SOURCES`, and optionally `RAPID_SOURCES`,
   `MI_SOURCES`, and `BQ_CONFIG` near the top.
3. Run the cell. If needed, the script asks you to upload the wheel. Version
   0.7.1 is a local build; plain `pip install pidatametrics` will not install
   these changes until this version has been published to PyPI.
4. Add Colab secrets `PI_CLIENT_ID` and `PI_CLIENT_SECRET`, or enter them at the
   hidden prompts. Google authentication is requested only for Google exports.
5. Pick an operation number. Press Enter to use all configured sources and
   latest defaults. Enter source numbers such as `1,3` to select a subset.
6. Choose `csv`, `excel`, `gsheet`, or `bigquery`. Direct endpoints also offer
   `json` and `preview`. The menu returns after each operation; enter `0` to exit.

Alternatively, upload the Python script into Colab's Files panel and run this
single cell (the script will prompt for the wheel if it is missing):

```python
%run /content/colab_all_endpoints.py
```

The script's `LAST_RESULTS` variable holds the last endpoint batch, including
original nested data, source labels, request parameters, quota metadata, and
per-request errors. JSON exports retain that entire structure. Tabular exports
use one row per top-level result item, with `data_json`, `sources_json`,
`params_json`, and `meta_json` columns, so unrelated nested lists are never
multiplied or discarded. Empty responses and failures are also represented.
Options 1–4 retain the existing report-specific flattened column layouts.

For every endpoint export except preview, a full `.json` backup is also saved
in the Colab working directory. CSV/Excel/JSON files are downloaded automatically.
Large response cells that exceed Excel/Sheets limits are rejected with a message
pointing to the JSON backup. BigQuery uses a consistent schema and appends rows;
use a separate table from your older flattened reports. Google Sheets clears an
existing tab with the selected name. Choose a new file/tab name to keep old results.

### What “latest” means

These defaults apply to `run_batch` and the Colab menu. Low-level endpoint
methods documented above still accept explicit periods, preserving their API.

| Data | Default behavior | Historical/custom override |
| --- | --- | --- |
| Standard SERP and visibility scores | Latest end period in workspace tracking history; an open-ended range uses the previous completed day/week/month in UTC | `period="2025-01-01"`, `"2025-W01"`, or `"2025-01"` |
| LLM mentions | Latest tracked daily/weekly period for both start and end | `start_period` and `end_period` |
| Rapid search data | Most recently completed two-hour UTC window | Explicit `period="2025-01-01T04/PT2H"` |
| Search volume | API default: most recent 36 available months | Both `start_date="2025-01"` and `end_date="2025-03"` |
| MI components | API's most recent version | `version=1` |
| MI version details | Highest available version ID | `version=1` |
| Account/setup/list endpoints | Current setup returned by the API | Supported ID/filter parameters |

Tracking history describes workspace tracking, not engine-specific processing
completion. Rapid workspaces have no documented tracking-history endpoint, so
the default rapid window is calculated, not confirmed available. A paused
campaign or processing delay can return no results; supply an older period in
that case. The runner does not silently keep querying older periods. If a
standard workspace has no usable history, supply a period explicitly.
Automatic history/version discovery adds API requests and can consume quota.
History lookups are reused for the same workspace within a batch.

Colab option 2 defaults to latest, and accepts an older period at its prompt.
The direct report helper supports the same choice explicitly:

```python
pi.run_serp_report(SERP_SOURCES, period="latest", output_mode="csv", filename="latest_serp")
pi.run_serp_report(SERP_SOURCES, period="2025-W01", output_mode="csv", filename="old_serp")
pi.run_llm_report(SERP_SOURCES, output_mode="csv", filename="latest_llm")
```

For compatibility, omitting `period` on the direct `run_serp_report` helper
continues to request yesterday. The Colab script always supplies `latest`
unless you enter a custom value. Historical report option 3 asks for a start
date and how many periods to step backwards.

### Reuse the original five-column sources

Yes: this structure works with `run_batch` for endpoints whose required IDs it
contains. Methods needing additional IDs require dictionary sources or shared
parameters. Account/global list operations run once, not once per SERP engine.

```python
SERP_SOURCES = [
    ["UK", 1001, "Example workspace", 1, "Example engine A"],
    ["UK", 1001, "Example workspace", 2, "Example engine B"],
    ["UK", 1001, "Example workspace", 3, "Example engine C"],
]

# Latest data for three engines: three data requests, one tracking-history lookup.
results = pi.run_batch("get_bulk_search_results", SERP_SOURCES)

# One workspace repeated across engines: volume is fetched only once.
volume = pi.run_batch("get_bulk_volume", SERP_SOURCES)

# Three engine requests with an explicit old date and custom filters.
older = pi.run_batch(
    "get_bulk_search_results", SERP_SOURCES,
    period="2025-01-01", number_of_results=25,
    serp_features=["aiOverview", "textAds"],
)
```

### Recommended: dictionary sources for all endpoint types

Use one dictionary per intended combination of IDs. Add another row for
another workspace, site, group, engine, campaign, index, or MI report.
**Scalar IDs are not expanded into every possible combination.** This keeps
site/campaign/group IDs associated with the correct workspace.

```python
SOURCES = [
    {
        "market_label": "UK",       # metadata label, not the search-engine market filter
        "workspace_id": 1001,
        "workspace_name": "Example workspace",
        "search_engine_id": 1,
        "search_engine_name": "Example engine A",
        # Fill in real IDs from your account when needed:
        # "site_id": 10,
        # "index_id": 3,
        # "stg_id": 1028,            # scalar: get_search_terms endpoint
        # "stg_ids": [1028, 1029],   # array filter: SERP, volume, LLM
    },
    {
        "market_label": "UK", "workspace_id": 1001,
        "workspace_name": "Example workspace",
        "search_engine_id": 2, "search_engine_name": "Example engine B",
    },
    {
        "market_label": "UK", "workspace_id": 1001,
        "workspace_name": "Example workspace",
        "search_engine_id": 3, "search_engine_name": "Example engine C",
    },
]

latest = pi.run_batch("get_bulk_search_results", SOURCES)
mentions = pi.run_batch("get_llm_mentions", SOURCES)

# Multiple workspaces; use your own workspace IDs.
workspaces = pi.run_batch("get_workspace", [
    {"workspace_id": 1001}, {"workspace_id": 1002},
])

# Site IDs remain paired with their workspace IDs.
sites = pi.run_batch("get_site", [
    {"workspace_id": 1001, "site_id": 10},
    {"workspace_id": 1001, "site_id": 11},
])

# Each source can override a shared period, engine, or filter.
mixed_dates = pi.run_batch("get_bulk_search_results", [
    {"workspace_id": 1001, "search_engine_id": 1, "period": "2025-01-01"},
    {"workspace_id": 1001, "search_engine_id": 2, "period": "2025-01-07"},
], number_of_results=25, serp_features=["aiOverview", "textAds"])

# Rapid data: latest calculated two-hour window unless period is supplied.
rapid = pi.run_batch("get_rapid_bulk_search_results", [
    {"workspace_id": 1304, "campaign_id": 1200, "search_engine_id": 58},
    {"workspace_id": 1304, "campaign_id": 1201, "search_engine_id": 58},
])

# Latest MI component versions by default; version can differ per source.
components = pi.run_batch("get_market_intelligence_component", [
    {"market_intelligence_id": 100, "component_key": "sov-leaders"},
    {"market_intelligence_id": 101, "component_key": "sov-timeline", "version": 1},
])
```

Only source fields used by the selected method are sent to the API. Other
fields remain as context in `sources`. Named source values override shared
keyword parameters. The five-column legacy format is converted to those named
fields automatically; its `UK` label becomes `market_label`.

Each batch uses the client account (`ACCOUNT_ID`). To query another account,
create another client; conflicting `account_id` values in sources are rejected.
This avoids silently sending a workspace ID under the wrong account.

An array-valued API filter remains an array in a single request:

```python
filtered = pi.run_batch("get_bulk_search_results", SOURCES,
                        stg_ids=[1028, 1029], serp_features=["aiOverview", "textAds"])
structure = pi.run_batch("get_account_structure", workspace_ids=[1001, 1002])
```

Duplicate requests are removed within each batch by default, and their source
labels are grouped together. Use `deduplicate=False` to intentionally repeat
them. `sources=None` makes one request with shared keyword parameters;
`sources=[]` makes zero requests. Missing required IDs are reported before any
data retrieval. Network/API errors stop the batch by default. To keep the
successful results and capture failed requests separately:

```python
results = pi.run_batch("get_bulk_search_results", SOURCES, continue_on_error=True)
for result in results:
    if result["error"]:
        print("Failed:", result["params"], result["error"])
    else:
        print("Succeeded:", result["params"], result["meta"])
```

### Custom parameters in the Colab menu

The script displays the selected method's parameter names. Press Enter at
the shared-parameters prompt for defaults, or paste a JSON object. Examples:

```json
{"period": "2025-01-01", "number_of_results": 25, "serp_features": ["aiOverview", "textAds"]}
```

```json
{"start_period": "2025-01-01", "end_period": "2025-01-07", "stg_ids": [1028, 1029]}
```

```json
{"start_date": "2025-01", "end_date": "2025-03", "currency": "GBP", "search_terms": ["example query"]}
```

Use only parameters displayed for the chosen endpoint. These are Python-style
names (`serp_features`), not HTTP spellings (`serp-feature[]`). Dates and filters
can also be saved directly in individual source dictionaries. To reset a saved
date to automatic selection, remove it or set it to `"latest"`.

### Full Colab menu

| Choice | Operation |
| --- | --- |
| 0 | Exit |
| 1 | Search volume report |
| 2 | Latest SERP report (custom period optional) |
| 3 | Historical SERP report |
| 4 | LLM mentions report (latest by default) |
| 5 | Current user |
| 6 | Accessible accounts |
| 7 | Account details |
| 8 | Account quota |
| 9 | Account sites |
| 10 | Account structure |
| 11 | Search engines |
| 12 | Search engine details |
| 13 | Workspaces |
| 14 | Workspace details |
| 15 | Workspace tracking history |
| 16 | Workspace sites |
| 17 | Search term groups |
| 18 | All workspace search terms |
| 19 | Search terms within groups |
| 20 | Site details |
| 21 | Visibility indexes |
| 22 | Visibility index details |
| 23 | Visibility site scores |
| 24 | Visibility group scores |
| 25 | Returning site URLs |
| 26 | Bulk search results |
| 27 | Raw bulk search results |
| 28 | Search volume (raw) |
| 29 | Rapid workspaces |
| 30 | Rapid workspace details |
| 31 | Rapid workspace sites |
| 32 | Rapid campaigns |
| 33 | Rapid campaign details |
| 34 | Rapid returning site URLs |
| 35 | Rapid bulk search results |
| 36 | Rapid raw search results |
| 37 | Market intelligence reports |
| 38 | Market intelligence details |
| 39 | Market intelligence versions |
| 40 | Market intelligence version details |
| 41 | Market intelligence components |
| 42 | Market intelligence component data |
| 43 | LLM mentions (raw) |

Deprecated Position Reports are excluded from both the package catalogue and
this menu.
