Metadata-Version: 2.4
Name: dpm-xl
Version: 1.4.0
Summary: Parse and evaluate DPM-XL validation rules of the EBA Data Point Model (DPM 2.0) in Python
Keywords: Data Point Model,DPM,DPM 2.0,DPM-XL,EBA,XBRL,xBRL-CSV,XBRL-XML,COREP,FINREP,regulatory reporting,supervisory reporting,EBA taxonomy,filing rules,validation rules,report package,dpm-toolkit
Author: Luuk Hopman
Author-email: Luuk Hopman <github@luukhopman.nl>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Financial and Insurance Industry
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Office/Business :: Financial
Classifier: Topic :: Office/Business :: Financial :: Accounting
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Dist: polars>=2.0,<3 ; extra == 'columnar'
Requires-Dist: dpm-xbrl>=1.1,<2 ; extra == 'xbrl'
Requires-Python: >=3.11
Project-URL: Homepage, https://luukhopman.github.io/dpm-xl/
Project-URL: Documentation, https://luukhopman.github.io/dpm-xl/
Project-URL: Source, https://github.com/luukhopman/dpm-xl
Project-URL: Issues, https://github.com/luukhopman/dpm-xl/issues
Project-URL: Changelog, https://luukhopman.github.io/dpm-xl/project/changelog/
Provides-Extra: columnar
Provides-Extra: xbrl
Description-Content-Type: text/markdown

<p align="center">
  <a href="https://luukhopman.github.io/dpm-xl/"><img src="https://luukhopman.github.io/dpm-xl/assets/logo.svg" alt="dpm-xl logo" width="240"></a>
</p>

<p align="center"><em>Parse and evaluate EBA DPM-XL validation rules in Python.</em></p>

dpm-xl is a Python parser and evaluator for DPM-XL, the language in which
the European Banking Authority (EBA) writes the validation rules of its Data
Point Model (DPM 2.0). Give it a rule and your report data, such as a COREP
or FINREP report in xBRL-CSV or XBRL-XML; it tells you which records fail
and explains each failure in plain numbers. It is for teams that check
supervisory reports before they are filed, build validation into a
reporting platform, or write and analyse DPM-XL rules. It works with two
sibling packages: [dpm-xbrl](https://pypi.org/project/dpm-xbrl/) reads and
writes XBRL reports, and [dpm-synth](https://pypi.org/project/dpm-synth/)
generates test reports.

dpm-xl is an independent project, not affiliated with or endorsed by the
European Banking Authority.

## Installation

```bash
pip install dpm-xl               # Python 3.11+, no dependencies
pip install "dpm-xl[columnar]"   # adds the polars evaluator
pip install "dpm-xl[xbrl]"       # adds dpm-xbrl, to check XBRL reports
```

## Quickstart

```python
from dpmxl import MemoryProvider, Number, TableLayout, evaluate_rule, explain

layout = TableLayout("C_01.00", rows=("0010", "0020", "0030"), columns=("0010", "0020"))
values = {  # (table, row, column, sheet, open keys) -> value with XBRL decimals
    ("C_01.00", "0010", "0010", None, ()): Number.of(100, 0),
    ("C_01.00", "0020", "0010", None, ()): Number.of(60, 0),
    ("C_01.00", "0030", "0010", None, ()): Number.of(40, 0),
    ("C_01.00", "0010", "0020", None, ()): Number.of(1204, 0),
    ("C_01.00", "0020", "0020", None, ()): Number.of(1180, 0),
    ("C_01.00", "0030", "0020", None, ()): Number.of(31, 0),
}
provider = MemoryProvider([layout], values)

expression = (
    "with {tC_01.00, c*, default: 0, interval: true}: {r0010} = {r0020} + {r0030}"
)
result = evaluate_rule(expression, provider, precondition="{v_C_01.00}")

print(result.status, result.counts)
for outcome in result.failures:
    print(outcome.key, explain(result.evaluation, expression, outcome.key).summary)
```

```text
failed {'passed': 1, 'failed': 1, 'null': 0}
{'c': '0020'} 1,204 should equal 1,211 (1,180 + 31): difference 7, at least 5.5 given the reported precision
```

On the command line, against a JSON data file or an XBRL report:

```bash
dpm-xl check rules.json                       # report every problem in every expression
dpm-xl eval rules.json data.json              # evaluate and explain every rule
dpm-xl check-report module.json report.zip    # filing rules and validation rules (xbrl extra)
```

From an EBA DPM release to explained failures in four commands:

```bash
dpm-xl list-modules dpm-4.4.sqlite --release 4.3   # the releases and modules
dpm-xl export-module dpm-4.4.sqlite COREP_OF --release 4.3 -o corep_of.json
dpm-synth generate corep_of.json -o report.zip     # or use your own report
dpm-xl check-report corep_of.json report.zip       # filing rules and validation rules
```

`dpm-4.4.sqlite` is the EBA's DPM 2.0 database as SQLite (the ZIP the EBA
publishes works too, with mdbtools) and `corep_of.json` a module file,
which dpm-xl and dpm-synth both read; see
[Start from an EBA DPM release](https://luukhopman.github.io/dpm-xl/guides/from-a-dpm-release/).

## Features

- Parses every construct of the DPM-XL grammar and all 14,174 distinct EBA
  validation rule and precondition expressions of DPM releases 3.4–4.4
  except the two with typos in the EBA source, about 0.1 ms each
  (measured with 1.3.0); a
  [test](https://github.com/luukhopman/dpm-xl/blob/main/tests/test_corpus.py)
  parses every one.
- Evaluates them with EBA's interval arithmetic, null logic and key joins, and
  points each failure at the cells involved.
- Finds problems before evaluation, for editors and CI: every syntax error,
  unknown functions and tables, coordinates a table lacks, keys and operand
  types, each with its position and, where one fits, a suggested fix.
  `non-portable` warnings mark syntax other DPM-XL engines may reject;
  `dpm-xl check --strict` fails on them.
- Exports a module of an EBA DPM 2.0 database as a
  [module file](https://luukhopman.github.io/dpm-xl/reference/module-file/)
  with `dpm-xl export-module` (`list-modules` lists them), read by dpm-xl
  and dpm-synth.
- Rules carry a `severity` and `active`; `--fail-on` sets which failures
  fail a run, and results come as text, JSON or JUnit XML.
- Has no dependencies. The optional `columnar` extra adds a polars evaluator
  for reports with millions of open-table rows.
- The evaluator reads no files and knows no DPM release: data comes through
  a three-method provider, so it runs against a report upload, a database or
  test fixtures. With the `xbrl` extra it evaluates a module file's rules on
  xBRL-CSV and XBRL-XML reports read by
  [dpm-xbrl](https://pypi.org/project/dpm-xbrl/), with earlier reports for
  `time_shift`.

## Toolkit

**Part of the DPM Toolkit**, three Python packages that work together:

| Package | What it does |
| :-- | :-- |
| <a href="https://luukhopman.github.io/dpm-xl/"><img src="https://luukhopman.github.io/dpm-xl/assets/logo.svg" alt="dpm-xl" height="32"></a> | Parse and evaluate DPM-XL validation rules **(this package)** |
| <a href="https://luukhopman.github.io/dpm-xbrl/"><img src="https://luukhopman.github.io/dpm-xbrl/assets/logo.svg" alt="dpm-xbrl" height="32"></a> | Read, write and check xBRL-CSV and XBRL-XML reports |
| <a href="https://luukhopman.github.io/dpm-synth/"><img src="https://luukhopman.github.io/dpm-synth/assets/logo.svg" alt="dpm-synth" height="32"></a> | Generate test reports that satisfy DPM-XL rules |

The [toolkit tour](https://luukhopman.github.io/dpm-xl/guides/toolkit-tour/)
takes one report through all three: generated with dpm-synth, read with
dpm-xbrl, then evaluated and explained with dpm-xl.

## Documentation

**Documentation: <https://luukhopman.github.io/dpm-xl/>**

- [Getting started](https://luukhopman.github.io/dpm-xl/getting-started/):
  a first rule, end to end.
- [How-to guides](https://luukhopman.github.io/dpm-xl/guides/): start from
  an EBA DPM release, connect your data, evaluate rule files, explain failures in a UI, evaluate large
  reports, run rules in threads, check rules in CI, build editor support, and
  [read an XBRL report and evaluate it](https://luukhopman.github.io/dpm-xl/guides/end-to-end/)
  with dpm-xbrl.
- [Concepts](https://luukhopman.github.io/dpm-xl/concepts/): records and
  keys, null logic, interval arithmetic and the rest of the semantics.
- [Reference](https://luukhopman.github.io/dpm-xl/reference/): the language,
  the CLI, the data formats and the Python API.
- [Using dpm-xl with AI agents](https://luukhopman.github.io/dpm-xl/agents/),
  with [`llms.txt`](https://luukhopman.github.io/dpm-xl/llms.txt) and
  [`llms-full.txt`](https://luukhopman.github.io/dpm-xl/llms-full.txt).
- [Changelog](https://github.com/luukhopman/dpm-xl/blob/main/CHANGELOG.md),
  [stability policy](https://luukhopman.github.io/dpm-xl/project/stability/),
  [benchmarks](https://luukhopman.github.io/dpm-xl/project/benchmarks/) and
  [security policy](https://luukhopman.github.io/dpm-xl/project/security/).

## Development

```text
uv sync
uv run pytest
uv run ruff check
uv run ruff format --check
uv run ty check
uv run python scripts/build_docs.py
```

See [contributing](https://luukhopman.github.io/dpm-xl/project/contributing/).

## Licence

MIT. See [LICENSE](https://github.com/luukhopman/dpm-xl/blob/main/LICENSE).
