Metadata-Version: 2.5
Name: aice-crawler
Version: 0.1.2
Summary: Python SDK for the AICE Crawler Service
Author-email: Adanian Labs <engineering@adanianlabs.io>
License: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: aice-core<0.4,>=0.3.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8.3; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Description-Content-Type: text/markdown

# aice-crawler

Python SDK for the [Crawler Service](../../../aice_packages/crawler_service/) —
multi-tenant web crawler with a page budget and an approve-then-publish flow
into the Knowledge Service.

```python
from aice_crawler import CrawlerClient

client = CrawlerClient(base_url="https://crawler.example.com", api_key="sk-…")

job = client.jobs.create(
    start_urls=["https://docs.example.com"],
    page_budget=500,
)

client.jobs.start(job["id"])

for page in client.jobs.list_pages(job["id"]):
    ...

client.jobs.approve(job["id"])
client.jobs.publish(job["id"])
```

Endpoints covered (see `crawler_service/app/api/routes.py`):

* `POST /jobs`, `GET /jobs`, `GET /jobs/{id}`
* `POST /jobs/{id}/start`, `GET /jobs/{id}/pages`, `GET /pages/{id}`
* `POST /jobs/{id}/approve`, `POST /jobs/{id}/publish`
* `PUT  /jobs/{id}/schedule`, `GET /jobs/{id}/runs`, `POST /jobs/{id}/reclean`
* `GET  /policy`, `PUT /policy`
