Metadata-Version: 2.5
Name: scrapyio-sdk
Version: 0.1.3
Summary: Official Python SDK for the Scrapy.io Platform API and publisher execution APIs
Project-URL: Homepage, https://scrapy.io
Project-URL: Documentation, https://docs.scrapy.io
Project-URL: Repository, https://github.com/scrapyio/public-dd
Author: Scrapy.io
License-Expression: MIT
Keywords: scrapy.io,scrapyio,sdk,web-scraping
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Provides-Extra: dev
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Description-Content-Type: text/markdown

# scrapyio-sdk

Official Python SDK for [Scrapy.io](https://scrapy.io).

**Server-side only.** Keep API keys off client devices.

## Install

```bash
pip install scrapyio-sdk
```

## Authentication

```python
import os
from scrapyio_sdk import ScrapyIO

client = ScrapyIO(api_key=os.environ["x-api-key"])
```

Always sends `x-api-key: <api_key>`.

## First request

```python
tools = client.tools.list(q="instagram", limit=20)
# {"items", "total", "offset", "limit"}

tool = client.tools.get("datadoping", "instagram-profile-scraper")
# or:
same = client.tool("datadoping/instagram-profile-scraper").get()
```

## Run a scraper

### Sync (`/v1/api`)

```python
result = client.tool("datadoping/instagram-profile-scraper").call({
    "username": "nasa",
})
```

### Async (`/v1/scraper`) + poll

```python
run = client.tool("datadoping/instagram-profile-scraper").start({
    "usernames": ["nasa"],
})

finished = client.run(run["id"]).wait(
    poll_interval_ms=3000,
    timeout_ms=15 * 60_000,
)

page = client.run(run["id"]).list_items(offset=0, limit=100)
```

## Schedules

```python
schedule = client.schedules.create(
    {
        "publisher": "datadoping",
        "slug": "instagram-profile-scraper",
        "runName": "nightly-ig",
        "timezone": "UTC",
        "frequency": "one-time",
        "date": "2026-09-20",
        "time": "23:50",
        "inputs": ["nasa"],
    },
    idempotency_key="nightly-ig-v1",
)

client.schedules.update(schedule["id"], {"isActive": False})
client.schedules.delete(schedule["id"])
```

## Errors

```python
from scrapyio_sdk import ScrapyIO, ScrapyIOError

try:
    client.tools.get("nope", "missing")
except ScrapyIOError as err:
    # err.type, err.status, err.message, err.doc_url
    ...
```

## Mental model

```text
client
├── tools.list / tools.get / tools.get_readme / tools.list_reviews
├── account.get / account.usage
├── tool("publisher/slug").get / .call / .start
├── runs.list / runs.get / runs.abort / runs.resume
├── run(id).get / .wait / .list_items / .abort / .resume
└── schedules.list / get / create / update / delete
```

## Development

```bash
cd sdk/python
python -m pip install -e ".[dev]"
pytest
```

## Notes

- GET requests may retry on transient 5xx/429; execution POSTs are not auto-retried.
- `list_items()` is JSON-only in v0.1.
- Companion JS package: [`@scrapyio/sdk`](https://www.npmjs.com/package/@scrapyio/sdk).
