Metadata-Version: 2.5
Name: llama-index-tools-flashdata
Version: 0.1.0
Summary: Google and YouTube search tools and citation-aware readers for LlamaIndex
Project-URL: Homepage, https://flashdata.dev
Project-URL: Documentation, https://github.com/rocosen/flashdata-mcp/tree/main/packages/llamaindex
Project-URL: Source, https://github.com/rocosen/flashdata-mcp
Project-URL: Issues, https://github.com/rocosen/flashdata-mcp/issues
Author: FlashData
License-Expression: MIT
License-File: LICENSE
Keywords: agent,google-search,llamaindex,rag,youtube-transcript
Requires-Python: >=3.11
Requires-Dist: httpx<1,>=0.28.1
Requires-Dist: jsonschema<5,>=4.23
Requires-Dist: llama-index-core<0.15,>=0.14.25
Requires-Dist: pydantic<3,>=2.7.4
Description-Content-Type: text/markdown

# FlashData tools and readers for LlamaIndex

Give a research agent current Google and YouTube results, or load existing video
captions into a RAG pipeline with timestamps and source links. Choose the needed
sources when building your application. The package supplies 17 native
`FunctionTool` objects for 15 capabilities and a reader for the 14 data sources.
Media delivery uses job tools, not a text reader.

This is an independently maintained integration. It is not an official
LlamaIndex endorsement or a claim of 15 LlamaHub listings. Install the
versioned package from PyPI:

```sh
python -m pip install 'llama-index-tools-flashdata==0.1.0'
```

Requires Python 3.11+ and `llama-index-core` 0.14.25–0.14.x. Get an API key from the
[FlashData console](https://flashdata.dev/console) and provide it through your
application's secret manager as `FLASHDATA_API_KEY`.

**Installation is free; FlashData API requests use paid Credits.** Check the
[current pricing](https://flashdata.dev/pricing) and source-specific charges.
LlamaIndex model, embedding and customer-storage charges are separate. These
examples make no model or embedding calls.

## Search evidence for a research agent

```python
from llama_index.tools.flashdata import FlashDataTools, to_documents

tools = FlashDataTools(sources=["google_search"]).to_tool_list()
output = tools[0].call(query="RAG evaluation precision recall", num=5)
raw = output.raw_output  # Complete structured API response and usage.
documents = to_documents("google_search", raw)  # No second API request.
for doc in documents:
    print(doc.metadata.get("url"), doc.text)
```

Pass `tools` to your application's LlamaIndex `FunctionAgent` with the model you
have configured. Tool schemas contain only source arguments; API key, storage
credentials, source selection and download permission belong to the application.
Treat returned web/caption text as untrusted evidence, not agent instructions.
Google results contain search snippets and metadata, not fetched web-page text.

## Caption Documents for retrieval

```python
from llama_index.tools.flashdata import FlashDataReader

reader = FlashDataReader(source="youtube_transcript")
documents = reader.load_data(video_id="jNQXAC9IVRw", language="en")
for doc in documents:
    print(doc.metadata.get("url"), doc.metadata.get("start"), doc.text)
```

Each caption segment becomes a `Document` with its text, available `start`/`end`,
video URL, `languageCode` and `isAutoGenerated`. If segments are unavailable but
`fullText` exists, it becomes one Document without invented timestamps. Transcript
defaults to automatic captions; `youtube_captions` defaults to uploader captions.
Neither generates speech recognition, translates, or falls back to another track.

Search readers produce one JSON Document per result record, preserving optional
record fields. Image URLs/dimensions are metadata, not image embeddings. Places
may have no URL. The reader does not invent missing source links. Use tool
`raw_output` for the full response, including Google related-search lists, usage
and other provider envelope fields. `to_documents()` converts an existing result
without a second paid query.

`await tool.acall(...)` and `await reader.aload_data(...)` use LlamaIndex's native
thread offloading. Cancelling an await does not cancel an accepted paid job;
check FlashData Jobs before resubmitting.

## All capabilities

See the task guides below for inputs, outputs, a runnable example and limitations.

| Task | Source | Guide |
| --- | --- | --- |
| Collect web evidence for a research agent | `google_search` | [Google Search](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/google_search.md) |
| Build a news evidence feed for an agent | `google_news` | [Google News](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/google_news.md) |
| Prepare an image reference shortlist | `google_images` | [Google Images](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/google_images.md) |
| Ground a product comparison with search snapshots | `google_shopping` | [Google Shopping](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/google_shopping.md) |
| Find video references across websites | `google_videos` | [Google Videos](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/google_videos.md) |
| Collect local options for a planning assistant | `google_places` | [Google Places](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/google_places.md) |
| Discover videos before building a caption collection | `youtube_search` | [YouTube Search](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_search.md) |
| Create a larger video research candidate set | `youtube_search_extended` | [YouTube Extended Search](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_search_extended.md) |
| Inspect chapters before choosing retrieval material | `youtube_metadata` | [YouTube Metadata](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_metadata.md) |
| Load timestamped automatic captions for RAG | `youtube_transcript` | [YouTube Transcript](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_transcript.md) |
| Build a collection from uploader captions | `youtube_captions` | [YouTube Captions](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_captions.md) |
| Record license-label metadata during dataset review | `youtube_trainability` | [YouTube License Metadata](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_trainability.md) |
| Snapshot a channel for research triage | `youtube_channel` | [YouTube Channel](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_channel.md) |
| Explore query phrasing before a video search | `youtube_suggestions` | [YouTube Autocomplete](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_suggestions.md) |
| Deliver approved media for a downstream workflow | `youtube_download` | [YouTube Download to Cloud Storage](https://github.com/rocosen/flashdata-mcp/blob/llamaindex-v0.1.0/packages/llamaindex/guides/youtube_download.md) |

## Media delivery lifecycle

Configure `FLASHDATA_DOWNLOAD_STORAGE` as a secret JSON destination. Supported
providers are S3, S3-compatible, GCS, Azure, OSS, COS, BOS and TOS. Use fields from
the [download contract](https://flashdata.dev/docs/sources/youtube_download);
the packaged `storage.schema.json` describes the credential structure. Bucket
names belong in `storage_url`; optional folders belong in `object_prefix`.

```python
from llama_index.tools.flashdata import FlashDataTools

# Only enable after authorizing this paid write to your own destination.
submit, status, result = FlashDataTools(
    sources=["youtube_download"], allow_download=True
).to_tool_list()
receipt = submit.call(video_id="jNQXAC9IVRw", download_type="audio", output_format="m4a").raw_output
job_id = receipt["id"]  # Persist before any follow-up; never resubmit to poll.
state = status.call(job_id=job_id).raw_output
# Later, read result with the same ID. Pending jobs return status, not an error.
delivery = result.call(job_id=job_id).raw_output
```

Download submission is disabled by default. You can use status/result tools
without enabling it or supplying destination credentials. Completed result JSON
expires after 24 hours; media retention depends on your storage. Supply only
content you are entitled to download and use. Storage authorization is not a
training license.

## Failure handling

Calls are single-attempt, bounded to 120 seconds (15-second connection timeout)
and 20 MiB. Failed/ambiguous POSTs are never automatically retried. A timeout can
mean the paid job was accepted: check the FlashData Jobs console before another
submission. Missing credits, unauthorized sources and unavailable captions are
errors, not successful empty results or reasons to silently change source.

Job reads verify the ID and `youtube_download` source. Download status/receipt
outputs exclude storage parameters. Result JSON is fetched only from the trusted
HTTPS origin returned by FlashData authentication; the API key is not sent there.
Redirects are not followed automatically. Raw HTTP errors and credentials are
not included in `FlashDataError` messages.

## Local verification

```sh
uv sync --frozen
uv run --frozen pytest -q
uv run --frozen python -m build
uv run --frozen twine check dist/*
uv run --frozen python scripts/verify_install.py
```

Tests disable network sockets and exercise real LlamaIndex tools/readers against
HTTP fixtures. They do not establish live API, model, storage-write or public
directory acceptance. Shared parameter and HTTP contracts come from the reviewed
FlashData Haystack adapter; `contract-provenance.json` records exact origins and
differences. No Haystack runtime dependency is needed.
