Metadata-Version: 2.4
Name: dq-kernel
Version: 0.2.0
Summary: A flexible data quality framework with engine extensibility using Python reflection
License: Apache-2.0
License-File: LICENSE
Keywords: data-quality,spark,deequ,great-expectations,data-validation
Author: Data Quality Framework Contributors
Requires-Python: >=3.12,<3.14
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Quality Assurance
Provides-Extra: aws
Provides-Extra: cron
Provides-Extra: databricks
Provides-Extra: deequ
Provides-Extra: delta
Provides-Extra: greatexpectations
Provides-Extra: spark
Requires-Dist: boto3 (>=1.26.0,<2.0.0) ; extra == "aws"
Requires-Dist: cron-schedule-triggers (>=0.0.11,<0.0.12) ; extra == "cron" or extra == "databricks"
Requires-Dist: databricks-sdk (>=0.123,<0.155) ; extra == "databricks"
Requires-Dist: delta-spark (>=3.3,<3.4) ; extra == "delta"
Requires-Dist: great_expectations (>=1.24,<2) ; extra == "greatexpectations"
Requires-Dist: pydeequ (==1.7.0) ; extra == "deequ"
Requires-Dist: pyhocon (>=0.3.59,<0.4.0)
Requires-Dist: pyspark (==3.5.9) ; extra == "spark" or extra == "delta"
Requires-Dist: pywin32-ctypes (>=0.2.0,<0.3.0) ; sys_platform == "win32"
Requires-Dist: pyyaml (>=6.0,<7.0)
Requires-Dist: requests (>=2.33.0,<3.0.0)
Project-URL: Documentation, https://github.com/anvai-labs/dq-kernel#documentation
Project-URL: Homepage, https://github.com/anvai-labs/dq-kernel
Project-URL: Repository, https://github.com/anvai-labs/dq-kernel
Description-Content-Type: text/markdown

# dq-kernel

Certifiable data-quality evidence engine for lakehouse pipelines. Semantically
versioned rules, exact-arithmetic evaluation, and machine-checkable evidence —
engine-agnostic by design.

## Install

```bash
pip install dq-kernel[spark]
```

Requires Python 3.12/3.13, Java 17, and a Spark 3.5.9 runtime.

## Quick start

```python
from dq.dq_framework import DQFramework

results = DQFramework(spark, config, default_dataframe=df).run()
```

Each result carries `check`, `success`, and bounded `details`. Evidence is
bound to dataset and rule-set digests as `dq-report/v1`.

## Three semantic versions

| Version | Semantics | Example |
|---|---|---|
| counts/v1 | Completeness (present/rows) | id is 80% complete |
| groups/v1 | Grouped distinct-count bounds | Each region has ≥ 2 distinct ids |
| ranges/v1 | Value-range bounds | All values ≥ 0 and ≤ 100 |

Plans are single-semantics: compose them with `RulesetEnvelope` for mixed
rulesets and execute the whole ruleset with one call —
`dq.spark_adapter.execute_envelope(envelope, datasets)` returns the
aggregate verdict.

## Documentation

- [Quickstart](docs/quickstart.adoc)
- [Error catalog](docs/error-catalog.adoc)
- [ADRs](docs/decisions/adr/)
- [SPECs](docs/specs/)
- [Modernization plan](docs/modernization-plan.adoc)

## License

Apache-2.0

