Metadata-Version: 2.4
Name: scrapenest
Version: 0.4.0
Summary: Official Python client for the ScrapeNest API — web scraping, search, AI answers, screenshots, PDFs, Google Maps reviews, and transcription.
Project-URL: Homepage, https://scrapenest.dev
Project-URL: Documentation, https://scrapenest.dev/docs
Author-email: ScrapeNest <support@scrapenest.dev>
License: MIT
License-File: LICENSE
Keywords: api,crawler,scrapenest,scraping,screenshot,search,serp,web-scraping
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: httpx>=0.27.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Description-Content-Type: text/markdown

# scrapenest

Official Python client for the [ScrapeNest](https://scrapenest.dev) API — web scraping, search, AI answers, screenshots, PDFs, Google Maps reviews, gas prices, and audio transcription.

- Docs: https://scrapenest.dev/docs
- Get an API key: https://scrapenest.dev

## Install

```bash
pip install scrapenest
```

Requires Python 3.9+.

## Quick start

```python
from scrapenest import Client

client = Client("sn_your_key_here")

result = client.search("python web scraping", num_results=5)
for r in result.results:
    print(r.title, r.url)
```

Every method returns a typed dataclass. Pass `include_usage=True` (where supported) to get a `usage` block with the credits charged.

## Search

```python
# SERP results. depth = "fast" | "balanced" | "thorough"
hits = client.search("openai news", depth="balanced", num_results=10)

# Search + fetch & clean the top pages
deep = client.search_deep("what is rust lang", fetch_top=3)
for page in deep.pages:
    print(page.url, len(page.text or ""))

# Cited answer synthesized from live sources
answer = client.search_answer("who wrote the odyssey", max_sources=3)
print(answer.answer)
for c in answer.citations:
    print(c.index, c.url)
```

## Scrape a page

```python
page = client.scrape_url(
    "https://example.com",
    render="auto",          # auto | direct | browser | stealth
    extract=True,
    return_markdown=True,
)
print(page.title)
print(page.markdown[:500])

# AI extraction
data = client.scrape_url("https://example.com/product", ai_query="Return the product name and price")
print(data.ai_extract)
```

## Screenshots & PDF

```python
shot = client.screenshot("https://example.com", full_page=True, format="png")
with open("page.png", "wb") as f:
    import base64
    f.write(base64.b64decode(shot.image_base64))

doc = client.pdf("https://example.com", format="A4", landscape=False)
with open("page.pdf", "wb") as f:
    import base64
    f.write(base64.b64decode(doc.pdf_base64))
```

## Google Maps reviews

Pass a place URL, a `0x…:0x…` FID, or a `ChIJ…` place id. Use `place_url` to bypass FID-built URLs for tricky listings.

```python
res = client.maps_reviews("https://www.google.com/maps/place/...", max_reviews=50, sort="newest")
print(res.name, res.rating, res.review_count)
for rev in res.reviews:
    print(rev.author_name, rev.rating, rev.text)

# Large pulls run as async jobs
run = client.maps_reviews_async("ChIJN1t_tDeuEmsRUsoyG83frY4", max_reviews=2000)
final = client.wait_for_run(run.run_id)
print(f"{len(final.result['reviews'])} reviews fetched")
# Need to stop a long job early? client.cancel_run(run.run_id)
```

## Gas prices

```python
gas = client.gas_prices("Los Angeles, CA", grade="regular", limit=10)
for s in gas.stations:
    print(s.brand, s.address, s.price)
if gas.region_stats:
    print(gas.region_stats.region, gas.region_stats.average_price)
```

## Translate & transcribe

```python
tr = client.translate("Hello, how are you?", "es")
print(tr.translated)

audio = client.audio_transcript("https://www.youtube.com/watch?v=...")
print(audio.text)
```

## Async client

```python
import asyncio
from scrapenest import AsyncClient

async def main():
    async with AsyncClient("sn_your_key_here") as client:
        result = await client.search("python web scraping")
        print(result.results[0].title)

asyncio.run(main())
```

`AsyncClient` exposes the same methods as `Client`.

## Error handling

```python
from scrapenest import RateLimitError, AuthenticationError, PaymentRequiredError

try:
    result = client.search("test")
except RateLimitError as e:
    print(f"Rate limited. Retry after {e.retry_after}s")
except PaymentRequiredError:
    print("Out of credits")
except AuthenticationError:
    print("Invalid API key")
```

## License

MIT
