Metadata-Version: 2.4
Name: concinno-skills-vector
Version: 0.1.0
Summary: Remote vector database agent skills (Pinecone + Chroma + Weaviate) for Concinno — thin adapters for agent upsert/query. Concinno core's ZIQRetrieval + FTRL remains the full retrieval stack; this is the outbound-to-vendor slot.
Project-URL: Homepage, https://github.com/aiking931931/concinno
Project-URL: Issues, https://github.com/aiking931931/concinno/issues
Project-URL: Changelog, https://github.com/aiking931931/concinno/blob/main/projects/concinno-skills-vector/CHANGELOG.md
Author-email: "AI King (Chen-Xuan Wang)" <me@ai-king.dev>
License-Expression: Apache-2.0
Keywords: agent,chromadb,concinno,embedding,pinecone,retrieval,skills,vector,weaviate
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: chromadb>=0.5
Requires-Dist: concinno>=2.15.1
Requires-Dist: pinecone>=5.0
Requires-Dist: weaviate-client>=4.9
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-cov>=5; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.7; extra == 'dev'
Description-Content-Type: text/markdown

# concinno-skills-vector

Remote vector database skills for [Concinno](https://pypi.org/project/concinno/).
Pinecone + Chroma + Weaviate via their official Python SDKs — thin
`call(**kwargs)` adapters for agents that need to upsert/query a
vendor-hosted vector store.

## Status

MVP (0.1.0) — six tools covering the canonical "talk to a vector DB"
agent need. Concinno core already ships `ZIQRetrieval` — a full BM25 +
dense + FTRL online-learning retrieval engine for local knowledge; this
sub-package is the orthogonal "read/write external vendor index" slot.

| Tool | Library | License | Purpose |
|---|---|---|---|
| `PineconeUpsert` | `pinecone` (v5+) | Apache-2.0 | Upsert vectors into a Pinecone index |
| `PineconeQuery` | `pinecone` (v5+) | Apache-2.0 | Query a Pinecone index by vector |
| `ChromaAdd` | `chromadb` (v0.5+) | Apache-2.0 | Add vectors to a Chroma collection (local or remote) |
| `ChromaQuery` | `chromadb` (v0.5+) | Apache-2.0 | Query a Chroma collection by vector |
| `WeaviateAdd` | `weaviate-client` (v4.9+) | BSD-3 | Insert vectors into a Weaviate v4 collection |
| `WeaviateQuery` | `weaviate-client` (v4.9+) | BSD-3 | Query a Weaviate v4 collection by vector |

## Install

```bash
pip install concinno-skills-vector
```

All three vendor SDKs come in as hard dependencies. Consumers who only
need one vendor path can install with `--no-deps` and pick the SDK
they want.

## Scope vs Concinno core

Concinno core's `concinno.rag.ZIQRetrieval` is a **full retrieval
stack** — BM25 + dense embedding + FTRL online-learning routing
between the two arms, with persistence, reranking, and
per-namespace SPS (Structural Prior SOTA) tuning. It targets the
"I want an intelligent retriever" slot.

This sub-package is an **outbound adapter** — the "my agent already
uses Pinecone / Chroma / Weaviate as its vector store, let the agent
upsert / query it directly" slot. No embedding model, no scoring
opinion, no router.

The two can coexist in the same `ToolRegistry`; they do not overlap.

### Why this sub-package vs LangChain retrievers

LangChain retrievers are readers only (no online learning,
per-query routing is pluggable but not built-in adaptive). Concinno's
own `ZIQRetrieval` in core has the FTRL online-learning router;
this sub-package just provides clean agent-callable access to the
three most common vendor stores without reinventing the router.

## Safety

Every tool routes through a shared `_safety` module before touching
the vendor:

1. **`top_k` cap** — `query` defaults to 10 and hard-caps at 1000.
   Larger pagination must be done caller-side.
2. **Batch cap** — `upsert` / `add` / `insert` caps at 10_000 rows
   per call. Split larger jobs into multiple calls.
3. **Vector dimension consistency** — the first row of a batch
   defines the dimension; all subsequent rows must match. Mixed-dim
   batches corrupt vendor indexes with no rollback.
4. **Filter type** — filters must be JSON-serialisable `dict` (or
   `None`). String / script / callable filter bodies are rejected
   at the tool layer. Weaviate's typed `Filter` DSL is not
   translatable from an untyped dict; `WeaviateQuery` rejects
   non-empty filters with a clear message pointing the caller at
   the Weaviate SDK directly.

## Credentials

No credentials are stored. Each `call` takes the vendor-specific
credential kwargs (`api_key`, `weaviate_url` + `weaviate_api_key`,
`chroma_host` + `chroma_port` or `chroma_persist_dir`). Callers who
want indirection resolve upstream:

```python
import os
from concinno_skills_vector import PineconeQuery

PineconeQuery().call(
    action="query",
    api_key=os.environ["PINECONE_API_KEY"],
    index="my-index",
    vector=[0.1] * 1536,
    top_k=10,
)
```

Or via Concinno's `CredentialStore`:

```python
from concinno.core.credentials import CredentialStore
cs = CredentialStore()
key = cs.resolve({"$ref": "env:PINECONE_API_KEY"})
```

## Usage via Concinno `ToolRegistry`

When the consumer sets `CONCINNO_LOAD_PLUGINS=1`, the default
registry auto-mounts all six tools:

```python
import os
os.environ["CONCINNO_LOAD_PLUGINS"] = "1"

from concinno.tools.registry import get_default_registry
reg = get_default_registry()
expected = {
    "PineconeUpsert", "PineconeQuery",
    "ChromaAdd", "ChromaQuery",
    "WeaviateAdd", "WeaviateQuery",
}
assert expected.issubset(set(reg.list_deferred()))
```

## Direct Python usage

```python
from concinno_skills_vector import (
    PineconeUpsert, PineconeQuery,
    ChromaAdd, ChromaQuery,
    WeaviateAdd, WeaviateQuery,
)

# ── Pinecone ──────────────────────────────────────────────────────
PineconeUpsert().call(
    action="upsert",
    api_key="pk-...",
    index="products",
    vectors=[[0.1, 0.2, ...], [0.3, 0.4, ...]],
    ids=["sku-1", "sku-2"],
    metadata=[{"category": "book"}, {"category": "book"}],
    namespace="prod",
)
# → {"ok": True, "upserted": 2}

PineconeQuery().call(
    action="query",
    api_key="pk-...",
    index="products",
    vector=[0.1, 0.2, ...],
    top_k=5,
    filter={"category": {"$eq": "book"}},
)
# → {"matches": [{"id": "sku-1", "score": 0.95, "metadata": {...}}, ...]}

# ── Chroma ────────────────────────────────────────────────────────
ChromaAdd().call(
    action="add",
    collection="docs",
    vectors=[[...]],
    ids=["doc-1"],
    metadata=[{"source": "manual"}],
    chroma_persist_dir="~/.myapp/chroma",
)
# → {"ok": True, "added": 1}

ChromaQuery().call(
    action="query",
    collection="docs",
    vector=[...],
    top_k=3,
    chroma_host="localhost",
    chroma_port=8000,
)
# → {"matches": [{"id": "doc-1", "score": 0.12, "metadata": {...}}]}

# ── Weaviate ──────────────────────────────────────────────────────
WeaviateAdd().call(
    action="add",
    collection="Article",
    vectors=[[...], [...]],
    ids=["550e8400-e29b-41d4-a716-446655440000", "..."],
    metadata=[{"title": "..."}, {"title": "..."}],
    weaviate_url="https://my-cluster.weaviate.network",
    weaviate_api_key="wk-...",
)
# → {"ok": True, "added": 2, "failed": 0}

WeaviateQuery().call(
    action="query",
    collection="Article",
    vector=[...],
    top_k=10,
    weaviate_url="https://my-cluster.weaviate.network",
    weaviate_api_key="wk-...",
)
# → {"matches": [{"id": "550e8400-...", "score": 0.08, "metadata": {...}}]}
```

All tools return either `{"ok": True, ...}` / `{"matches": [...]}` on
success or `{"error": "..."}` on any validation or vendor-SDK failure
— same shape as other Concinno built-in tools.

## What this package is NOT

- **Not a retrieval engine.** Use `concinno.rag.ZIQRetrieval` for
  BM25 + dense + FTRL routing.
- **Not a migration tool.** Index creation / schema changes go
  through each vendor's admin API, not these Tool `call` methods.
- **Not an embedding service.** Callers supply pre-computed
  vectors. Embedding models live in their own tools (e.g. Concinno
  core's `ZIQRetrieval` owns `sentence-transformers`).
- **Not a secrets store.** Credentials are per-call.
- **Not a full Weaviate `Filter` DSL translator.** Agents needing
  server-side typed filters should call the Weaviate SDK directly.

## License

Apache-2.0. See `LICENSE` in the Concinno monorepo.
