Metadata-Version: 2.4
Name: validater-data
Version: 0.1.0
Summary: A YAML-driven data validation library with HTML reports.
Author: Validater contributors
License-Expression: MIT
Keywords: data-validation,data-quality,yaml,polars
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: jinja2>=3.1
Requires-Dist: polars>=0.20
Requires-Dist: pydantic>=2
Requires-Dist: pyyaml>=6
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: twine>=5; extra == "dev"
Provides-Extra: mssql
Requires-Dist: pyodbc>=5; extra == "mssql"
Requires-Dist: sqlalchemy>=2; extra == "mssql"
Dynamic: license-file

# validater

`validater` is a YAML-driven data validation tool for checking datasets and producing an HTML report.

For a full step-by-step guide, see [docs.md](docs.md).

## Supported Sources

`validater` currently supports:

- CSV files.
- Microsoft SQL Server.

Planned source support includes more relational databases, Excel workbooks, and Parquet files.

## Install

```bash
pip install validater-data
```

For local development:

```bash
pip install -e ".[dev]"
```

To validate data from Microsoft SQL Server:

```bash
pip install "validater-data[mssql]"
```

## Create `rules.yaml`

```yaml
name: Validate Workouts
description: Checks required fields before ingestion

source:
  type: csv
  path: ./data/workouts.csv

checks:
  - column: title
    type: not_null
    severity: error

  - column: reps
    type: not_null
    severity: warn
```

For Microsoft SQL Server, use either `table` or `query`. Prefer `connection_string_env` so credentials are not committed to source control.

```yaml
name: Validate Users

source:
  type: mssql
  connection_string_env: VALIDATER_MSSQL_URL
  schema: dbo
  table: users

checks:
  - column: user_id
    type: primary_key

  - column: email
    type: regex
    pattern: '^[\w\.-]+@[\w\.-]+\.\w+$'
```

Use `filters` when only part of a table should be validated. Multiple filters are combined with `AND`.

```yaml
name: Validate Active Customers

source:
  type: mssql
  connection_string_env: VALIDATER_MSSQL_URL
  schema: dbo
  table: customers
  filters:
    - column: ACTIVE_FLAG
      operator: "="
      value: "Y"

    - column: COUNTRY
      operator: "in"
      value: ["US", "CA"]

    - column: DELETED_AT
      operator: "is_null"

checks:
  - column: customer_id
    type: primary_key

  - column: email
    type: regex
    pattern: '^[\w\.-]+@[\w\.-]+\.\w+$'
```

Supported filter operators are `=`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `not_in`, `is_null`, and `is_not_null`.

```yaml
name: Validate Recent Orders

source:
  type: mssql
  connection_string_env: VALIDATER_MSSQL_URL
  query: |
    SELECT order_id, customer_id, order_total, discount_total
    FROM dbo.orders
    WHERE created_at >= DATEADD(day, -1, SYSUTCDATETIME())

checks:
  - column: order_id
    type: primary_key

  - column: order_total
    type: column_dependency
    operator: ">="
    right_column: discount_total
```

`filters` are available for table-based MS SQL sources. For advanced filtering, joins, grouped `AND`/`OR` logic, or warehouse-specific SQL, use `query`.

## Supported Checks

`validater` currently supports these checks:

- `not_null`: column values must not be null.
- `regex`: non-null column values must match a regex pattern.
- `unique`: a column, or set of columns, must be unique.
- `unique_combination`: alias for multi-column uniqueness.
- `primary_key`: a single column must be non-null and unique.
- `composite_primary_key`: multiple columns must be non-null and unique together.
- `column_dependency`: compare two columns row by row, such as `max_value > min_value`.

Example:

```yaml
checks:
  - column: email
    type: regex
    pattern: '^[\w\.-]+@[\w\.-]+\.\w+$'

  - column: user_id
    type: primary_key

  - columns: [user_id, created_at]
    type: composite_primary_key

  - columns: [user_id, event_name]
    type: unique_combination

  - column: max_value
    type: column_dependency
    operator: ">"
    right_column: min_value
```

## Run

```bash
validater /path/to/rules.yaml
```

`validater` exits with a non-zero status when an `error` severity check fails, so it can be used in CI pipelines. A report is written to `reports/`.

The misspelled command `validaer` is also installed as a compatibility alias.
