Metadata-Version: 2.4
Name: weaviate-orm
Version: 0.9.0
Summary: A lightweight ORM-style layer for Weaviate
Author-email: Tobias Heimig-Elschner <Tobias.Heimig-Elschner@bbr.bund.de>
License-Expression: GPL-3.0-or-later
Project-URL: Homepage, https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm
Project-URL: Changelog, https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/CHANGELOG.md
Project-URL: Documentation, https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/tree/main/docs/user-docs
Project-URL: Issues, https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/issues
Project-URL: Citation, https://doi.org/10.58007/x1wa-rt92
Keywords: weaviate,orm,vector-database,vector-search,embeddings,semantic-search
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: weaviate-client<5,>=4.16.0
Requires-Dist: python-dateutil>=2.8
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: python-dateutil; extra == "test"
Provides-Extra: dev
Requires-Dist: weaviate-orm[test]; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs; extra == "docs"
Requires-Dist: mkdocs-material; extra == "docs"
Requires-Dist: mkdocstrings[python]; extra == "docs"
Dynamic: license-file

# Weaviate ORM
### A Pythonic ORM-style layer for Weaviate

`weaviate-orm` provides a structured, object-oriented interface to the Weaviate vector database. Inspired by traditional ORM patterns, it allows you to define collections as Python classes, automatically generate Weaviate schemas, and perform CRUD and similarity-based queries without leaving the object-oriented paradigm.

> **Status: pre-1.0.** Versioning is still settling (see [`CHANGELOG.md`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/CHANGELOG.md)) and the public API may change between releases. See [`docs/dev-docs/ROADMAP.md`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/dev-docs/ROADMAP.md) for what's in progress.

## Requirements

- Python 3.11+
- weaviate-client >= 4.16.0 (CI and this doc are verified against the latest release, 4.23.1)
- Weaviate server >= 1.27.0 (CI and this doc are verified against the latest stable release, 1.39.6)
- For examples using Ollama: a reachable Ollama instance with the "snowflake-arctic-embed2" embedding model available

### Server Requirements

**Weaviate 1.27.0 or higher** is required. The `weaviate-client >= 4.16.0` enforces this constraint. During engine initialization, the ORM performs a runtime check:

```python
engine = WeaviateEngine()
engine.create_all_schemas()  # Raises RuntimeError if server version < 1.27.0
```

Why 1.27.0?
- Unified vector configuration API (`Configure.Vectors.*` instead of deprecated `vectorizer_config`)
- Robust named vector support
- Improved schema handling and stability

If your server doesn't meet this requirement, the engine will raise a clear error:

```
RuntimeError: Weaviate server version 1.25.4 is not supported. 
Please use Weaviate >= 1.27.0 to match the client requirements.
```

## Features

- **Declarative Schema Definition**  
  Define collections using Python classes and descriptors for properties and cross-references.

- **Dynamic Schema Generation via Metaclass**  
  A metaclass extracts property and reference definitions to auto-generate Weaviate-compatible schema structures.

- **Polymorphic References** ✨  
  A reference field can accept objects from several different collections that share a common `PolymorphicParent` — Weaviate has no native concept of this, so the parent collection stores a small discriminator entry per child and the ORM resolves it transparently. See [`examples/03_polymorphic_parent.py`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/examples/03_polymorphic_parent.py) and [`docs/dev-docs/POLYMORPHIC_PARENT_ARCHITECTURE.md`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/dev-docs/POLYMORPHIC_PARENT_ARCHITECTURE.md).

- **Object-Oriented CRUD Operations**  
  Create, retrieve, update, and delete data using instance methods instead of hand-written GraphQL/REST calls.

- **Recursive Save & Update**  
  Handles deeply nested references and automatically saves or updates related objects when desired.

- **Flexible Engine Binding**  
  The engine takes any connection method from the `weaviate` library, allowing support for local, remote, or custom configurations.

- **UUID Management**  
  Built-in support for both `uuid4` and `uuid5` strategies — including UUID validation and immutability enforcement.

- **Optional Auto-loading of References**  
  Cross-references can be automatically resolved into full model instances when accessed.

- **A Full Query Surface**  
  Similarity search (`near_vector`/`near_text`), filter-only fetch (`fetch_objects`), keyword search (`bm25`), hybrid search (`hybrid`), and `count()` - all on your model classes, with `filters`, pagination, and a scoped `get(uuid, filters=...)`. See [Querying](#querying) below.

- **Lifecycle Callbacks**  
  Register `pre_save`/`post_save`/`pre_update`/`post_update`/`pre_delete`/`post_delete` hooks per model. See [Lifecycle Callbacks](#lifecycle-callbacks) below.

- **Strict Typing and Validation**  
  Type enforcement and optional validation logic on properties and references using Python descriptors. The package itself ships [PEP 561](https://peps.python.org/pep-0561/) type hints (`py.typed`) - `mypy` runs clean and blocking in CI.

- **Unit + Integration Tested**  
  200+ unit tests run offline; the integration suite exercises real CRUD, references, and polymorphism against a Weaviate started with `docker-compose.test.yml`. See [Testing](#testing) below.

- **Batch Operations** ✨  
  `bulk_save()`/`bulk_update()`/`bulk_delete()` do many objects' worth of work in one batched Weaviate call each instead of one round-trip per object, with best-effort error reporting, recursive creation of not-yet-saved references, and `bulk_save_many()` for saving across several collections in one call. See [Batch Operations](#batch-operations) below.

- **Nested (Object) Properties** ✨  
  A `Property` with `weaviate_type=DataType.OBJECT`/`OBJECT_ARRAY` describes its own fields via `nested_properties`, with the same casting/validation/required-field rules as top-level properties, nestable to arbitrary depth. See [Nested Properties](#nested-properties) below.

---

## Design Patterns

- **Descriptor Pattern**  
  Custom `Property` and `Reference` descriptors manage validation, casting, and schema representation of scalar and relational fields.

- **Metaclass-Based Schema Introspection**  
  A metaclass (`Weaviate_Meta`) dynamically collects all descriptors from a class and builds the full Weaviate schema. It also generates a dynamic `__init__` constructor based on the field signatures.

- **Engine Abstraction Layer**  
  The `WeaviateEngine` encapsulates connection logic and schema management. It supports any connection method compatible with the `weaviate` Python client (e.g., local, remote, or cloud).

- **Instance-Oriented CRUD**  
  CRUD operations (including recursive reference handling and UUID safety checks) are exposed as instance methods on models that inherit from `BaseModel`.

- **Reference Resolution Strategy**  
  `Reference` descriptors can be configured to auto-load full objects (not just UUIDs), and support single or list cardinalities. `way_type=ReferenceType.TWOWAY` is reserved for a future bidirectional-sync feature and currently raises `NotImplementedError` (`ONEWAY` is the default) - see the note in [`weaviate_reference.py`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/src/weaviate_orm/weaviate_reference.py).

---

## Installation

You can install `weaviate_orm` either from PyPI (once published) or directly from the source repository.

### 📦 From PyPI (recommended)
```bash
pip install weaviate-orm
```

### 🛠 From Git (development and testing)
```bash
git clone https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm.git
cd weaviate_orm
pip install -e .
```
Note: Make sure your Python environment includes a compatible Weaviate client (>= 4.16.0):

```bash
pip install "weaviate-client>=4.16.0"
```

## Quick Start

The snippet below needs only a plain Weaviate instance (`docker compose -f docker-compose.test.yml up -d`) - no vectorizer required. It's a trimmed-down, hand-maintained copy of the real, tested file [`examples/01_quickstart.py`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/examples/01_quickstart.py) (which has two extra `Paper` fields and runs end-to-end) - if the two drift apart, trust the example file.

```python
import datetime
from uuid import UUID

from weaviate.classes.config import DataType
from weaviate_orm import BaseModel, Property, Reference
from weaviate_orm.weaviate_reference import ReferenceType

class Author(BaseModel):
    first_name = Property(cast_type=str, weaviate_type=DataType.TEXT, description="First name", required=True)
    last_name = Property(cast_type=str, weaviate_type=DataType.TEXT, description="Last name", required=True)

    _namespace = UUID("eb8bc242-5f59-4a47-8230-0cea6fcc1028")

    def _get_uuid_name_string(self) -> str:
        return f"{self.first_name} {self.last_name}"

class Paper(BaseModel):
    title = Property(cast_type=str, weaviate_type=DataType.TEXT, description="Title", required=True)
    doi = Property(cast_type=str, weaviate_type=DataType.TEXT, description="DOI", required=True)
    author = Reference(target_collection_name="Author", reference_type=ReferenceType.SINGLE, auto_loading=True)

    _namespace = UUID("eb8bc242-5f59-4a47-8230-0cea6fcc1028")

    def _get_uuid_name_string(self) -> str:
        return self.doi
```

Connect an engine, register the models, and create the schema:

```python
import os
from weaviate import connect_to_local
from weaviate_orm.weaviate_engine import WeaviateEngine

engine = WeaviateEngine(
    connect_to_local,
    host=os.getenv("WEAVIATE_HOST", "localhost"),
    port=int(os.getenv("WEAVIATE_PORT", 8080)),
    grpc_port=int(os.getenv("WEAVIATE_GRPC_PORT", 50051)),
)
engine.register_all_models(Author, Paper)
engine.create_all_schemas()
```

`create_all_schemas()` only creates collections that don't exist yet. To add a
new property/reference a model has gained to a collection that's already
there, pass `update_existing=True` - it's additive only (nothing existing is
changed or removed); see [Engine & Connections](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/user-docs/concepts/engine.md#registering-models-and-creating-schemas)
for what that does and doesn't cover.

Create, read, update, delete:

```python
author = Author(first_name="Ada", last_name="Lovelace")
paper = Paper(title="Notes on the Analytical Engine", doi="10.1234/example-quickstart", author=author)
paper.save(include_references=True, recursive=True)  # recursive=True also saves `author`

loaded = Paper.get(paper.get_uuid(), include_references=True)
print(loaded.title, "by", loaded.author.first_name, loaded.author.last_name)

loaded.title = "Notes on the Analytical Engine (revised)"
loaded.update()

loaded.delete(force=True)
```

More runnable examples live under [`examples/`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/tree/main/examples/): references (single/list, auto-loading), polymorphic references, named vectors, self-provided vectors, lifecycle callbacks, and batch operations. Every one of them is exercised by the integration test suite.

## Batch Operations

`bulk_save()`/`bulk_update()`/`bulk_delete()` do many objects' worth of work
in as few Weaviate round-trips as possible (`insert_many`/`reference_add_many`/
`delete_many`), instead of one call per object - useful for loading, updating,
or removing hundreds or thousands of objects at once. All three are
best-effort: instead of raising on the first bad item, they return a
`BulkSaveResult`/`BulkDeleteResult` so one invalid or already-broken instance
doesn't block the rest of the batch. See
[`examples/07_batch_operations.py`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/examples/07_batch_operations.py).

```python
articles = [Article(title=f"Article {i}", author=ada) for i in range(1000)]
# author doesn't have to be saved first - recursive=True batch-creates it too.
result = Article.bulk_save(articles, include_references=True, recursive=True)

print(f"{len(result.succeeded)} created, {len(result.failed)} failed")
if result.has_errors:
    print(result.failed, result.reference_errors)

# Update: every instance's *current* properties fully overwrite the stored
# object - not a diff-based patch like update(). Every instance must exist.
for a in articles:
    a.title = a.title.upper()
Article.bulk_update(articles)

# Delete: skips (and reports) anything still referenced elsewhere, unless force=True.
Article.bulk_delete(articles)
```

`bulk_save(instances, include_references=True, recursive=True)` also creates a
not-yet-saved referenced object (grouped by its own collection, so still a
batch insert, not one `save()` per object) and supports `PolymorphicParent`
reference fields. To save instances of *several* collections in one call, use
`weaviate_orm.bulk_save_many(instances)` instead of calling `bulk_save()` once
per collection yourself. See the
[Batch Operations guide](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/user-docs/guides/batch-operations.md)
for the full picture, including what's still deliberately out of scope and the
duplicate-UUID (upsert) semantics both `bulk_save()`/`bulk_update()` share.

## Nested Properties

A `Property` can describe a nested JSON object (Weaviate's `OBJECT`/`OBJECT_ARRAY`
data types) via `nested_properties` - one `Property` (or a list of them) for the
object's own fields, with the same casting/validation/`required` rules as any
top-level property, nestable to arbitrary depth.

```python
class Company(BaseModel):
    name = Property(cast_type=str, description="Name", weaviate_type=DataType.TEXT)
    address = Property(
        weaviate_type=DataType.OBJECT,
        cast_type=dict,
        description="Address",
        required=False,
        nested_properties=[
            Property(weaviate_type=DataType.TEXT, cast_type=str, description="City", name="city"),
            Property(weaviate_type=DataType.TEXT, cast_type=str, description="ZIP", name="zip"),
        ],
    )

company = Company(name="Acme", address={"city": "Berlin", "zip": "10115"})
company.save()
```

A nested `Property` needs an explicit `name=...` - unlike a top-level property,
it's never assigned as a class attribute, so it never gets a name via
`__set_name__`. An unknown key in the dict, or a missing `required` nested
field, raises `ValueError` the same way a top-level property does.

## Querying

Every model gets five query classmethods, all returning hydrated model
instances (not raw Weaviate SDK objects) and sharing the same
`include_references`/`include_vector`/`top_n`/`offset`/`filters` vocabulary:
`near_vector()`/`near_text()` (similarity search), `fetch_objects()`
(filter-only - "objects matching a condition", no vector/keyword query),
`bm25()` (pure keyword, no vectorizer needed) and `hybrid()` (keyword+vector,
blended by `alpha`). Plus `count(filters=...)`, returning an `int`.

```python
from weaviate.classes.query import Filter

results = Article.near_text(
    "notes on computing",
    top_n=5,                    # default 5
    include_references=True,    # resolve cross-references on each hit
    include_vector=False,       # include the object's own vector(s) in the result
)

# Named-vector collections: pick which vector space to search
results = Article.near_vector(query_vector, target_vector="title_vector", include_references=False, include_vector=False)

# Filter-only fetch, keyword search, and count - no vectorizer required for any of these
results = Article.fetch_objects(filters=Filter.by_property("tenant_id").equal("acme"), top_n=50)
results = Article.bm25("notes on computing", query_properties=["title"])
total = Article.count(Filter.by_property("tenant_id").equal("acme"))
```

All five accept an optional `filters` argument - a `weaviate-client` filter
expression, passed straight through to scope the query (e.g. multi-tenant
isolation, or restricting to a precomputed set of UUIDs):

```python
from weaviate.classes.query import Filter

results = Article.near_text(
    "notes on computing",
    filters=Filter.by_property("tenant_id").equal("acme"),
)
```

`get(uuid, filters=...)` also accepts one, to scope a UUID lookup the same
way - a UUID that exists but fails `filters` returns `None`, indistinguishable
from a UUID that doesn't exist at all. `filters` isn't supported on
`bulk_save()` or `instance_exists()`. See
[Querying](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/user-docs/guides/querying.md)
for the full guide (pagination, hybrid search, the escape hatches) and
[`NEAR_QUERY_FILTERS_DESIGN.md`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/dev-docs/NEAR_QUERY_FILTERS_DESIGN.md)
for the design record.

For anything the typed wrappers don't cover, `raw_near_vector()`,
`raw_near_text()`, `raw_fetch_objects()`, `raw_hybrid()`, and `raw_bm25()`
pass `*args`/`**kwargs` straight through to the underlying `weaviate-client`
collection query and return the raw SDK result - no reference resolution or
casting to model instances.

## Lifecycle Callbacks

Register hooks that fire around `save()`, `update()`, and `delete()` -
useful for logging, cache invalidation, or side effects that should run
whenever an instance is persisted, regardless of which code path triggered it.
Allowed events: `pre_save`, `post_save`, `pre_update`, `post_update`,
`pre_delete`, `post_delete`.

```python
@Note.register_callback("post_save")
def log_post_save(instance: Note, **_ctx) -> None:
    print(f"saved {instance.get_uuid()}: {instance.text!r}")
```

Callbacks are registered per class, apply across the full MRO (a callback
registered on a base class also fires for subclasses), and receive `instance=`
plus event-specific context (e.g. `post_update` also gets `changes=`) - a
callback that declares `**kwargs` gets everything, one that declares named
parameters only gets the ones it asks for. See
[`examples/06_callbacks.py`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/examples/06_callbacks.py) for a full pre/post
save/update/delete walkthrough.

## Schema Configuration Access

The ORM uses protected class-level attributes for schema configuration to prevent accidental overrides. All schema configs are prefixed with an underscore and accessed via read-only class properties.

### Configuring Collections

Define schema configurations using the underscored attributes:

```python
from weaviate.classes.config import Configure

class Article(BaseModel):
    # Class-level schema configuration (protected with underscore)
    _vector_config = Configure.Vectors.text2vec_ollama(
        api_endpoint="http://llm:11434",
        model="snowflake-arctic-embed2",
        vectorize_collection_name=False
    )
    
    _description = "A collection of articles with vector embeddings"
    
    _inverted_index_config = None  # Optional index tuning
    _generative_config = None      # Optional generative configuration
```

### Accessing Configurations

You can read schema configurations at both class and instance levels:

```python
# Class-level access (read-only)
print(Article.vector_config)  # Returns the configured vector
print(Article.description)     # Returns "A collection of articles..."

# Instance-level access
article = Article(...)
print(article.vector_config)   # Proxies to class-level config
```

### Deprecation Path

Older code using non-underscored names will still work but emit a deprecation warning:

```python
class LegacyModel(BaseModel):
    vector_config = Configure.Vectors.text2vec_ollama(...)  # DeprecationWarning
    description = "Legacy"  # DeprecationWarning
```

Migrate to the underscored versions to avoid future incompatibility:

```python
class ModernModel(BaseModel):
    _vector_config = Configure.Vectors.text2vec_ollama(...)
    _description = "Modern"
```

## Project Structure
```
weaviate_orm/
│
├── __init__.py              # Public interface and versioning
├── weaviate_base.py         # BaseModel with full CRUD logic and query support
├── weaviate_engine.py       # Manages Weaviate client and schema creation
├── weaviate_meta.py         # Metaclass for schema extraction and dynamic __init__
├── weaviate_property.py     # Descriptor for scalar fields
├── weaviate_reference.py    # Descriptor for references (one-way; single or list)
├── weaviate_decorators.py   # Client injection and async-to-sync conversion
└──  weaviate_utility.py      # Helpers for validation and reference comparison
```

## License
This project is licensed under the GNU General Public License v3.0 (GPLv3).

You are free to use, modify, and distribute this software under the terms of the GPLv3 license. Any derivative work must also be distributed under the same license.

For full details, see the [LICENSE](https://choosealicense.com/licenses/gpl-3.0/).

## Contributing & Credits
This project is created and maintained by the BBSR - IDA (Tobias Heimig-Elschner).
Feel free to open issues or submit pull requests if you encounter bugs, have ideas, or want to improve the package. See [`CONTRIBUTING.md`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/CONTRIBUTING.md) for setup, testing, and merge-request expectations.

### 📚 Citation
The project is published on Zenodo. The citation below is for the archived `0.1.0` release;
check the [DOI page](https://doi.org/10.58007/x1wa-rt92) for a version-specific citation matching
the release you're using:

Heimig, T. (2025). Weaviate ORM - (0.1.0). Bundesinstitut für Bau-, Stadt- und Raumforschung (BBSR). https://doi.org/10.58007/x1wa-rt92

## Roadmap & Open Development Topics

See [`docs/dev-docs/ROADMAP.md`](https://gitlab.opencode.de/bbsr_ida_public/weaviate_orm/-/blob/main/docs/dev-docs/ROADMAP.md) for the full refactoring
and hardening plan (test environment, documentation, code cleanup). Feature-level
topics:

- **Batch Operations (v2)**<br>`bulk_save()` (v1, batch create) shipped - see [Batch Operations](#batch-operations) above. Still open: `bulk_update`/`bulk_delete`, batching across multiple collections in one call, and recursive batch-creation of not-yet-saved referenced objects.

- **Generalized Query Interface**<br>`fetch_objects()`/`count()`/`hybrid()`/`bm25()`, `filters`+pagination on all query methods, and a scoped `get(uuid, filters=...)` have shipped as plain classmethods - see [Querying](#querying) above. Still open: unifying near-vector/near-text/filter/hybrid into one fluent, chained interface (a bigger API-design question, deliberately kept separate - see `docs/dev-docs/ROADMAP.md`).

---

## Testing

```bash
pip install -e ".[dev]"
```

### Unit tests

No external services needed:

```bash
make test-unit          # or: pytest -m "not integration"
```

### Integration tests

Most integration tests only need a plain Weaviate - no vectorizer, no Ollama:

```bash
make test-int           # starts docker-compose.test.yml, waits, then runs pytest -m integration
make down               # tear it down when you're done
```

A handful of tests need a real embedding model and are marked `@pytest.mark.vectorizer`
(e.g. named vectors, `near_text`); they're skipped automatically when no Ollama is
reachable. To run them too, point `LLM_HOST`/`LLM_PORT` at an Ollama instance serving
`snowflake-arctic-embed2` (the full `docker-compose.yml` dev stack provides one) and run
`pytest -m integration` directly.

To connect against your own Weaviate instead of the bundled compose file, set:

```bash
export WEAVIATE_HOST=localhost
export WEAVIATE_PORT=8080
export WEAVIATE_GRPC_PORT=50051
```

Run `make help` for the full list of targets (lint, fmt, build, ...).

### Migration note

The Weaviate Python client has deprecated `vectorizer_config` in favor of `vector_config`. This project now uses `vector_config` everywhere, including named vectors (e.g., `Configure.Vectors.text2vec_ollama(...)`). Ensure your server and client meet the versions above to avoid startup or schema warnings.
