Metadata-Version: 2.4
Name: dwgen
Version: 2.1.4
Summary: dwgen - Data Warehouse Generator
Author-email: Christoph Metz <info@dwgen.de>
License: dwgen Free Use License v1.0
        
        Copyright (c) 2026 Christoph Metz 
        
        1. Grant of License
        
        Subject to the terms and conditions of this License, permission is hereby granted, free of charge, to any person obtaining a copy of this Software and associated documentation files (the "Software") to use the Software for private, educational, internal business, and commercial purposes.
        
        2. Restrictions
        
        Except as expressly permitted by this License, you may not:
        
            a). modify, adapt, translate, reverse engineer, decompile, disassemble, or create derivative works based upon the Software;
        
            b) redistribute, publish, upload, transmit, sublicense, rent, lease, lend, or otherwise make the Software available to any third party;
        
            c) sell, resell, bundle, or commercially distribute the Software, whether as a standalone product or as part of another product, without prior written permission from the copyright holder;
        
            d) remove, alter, or obscure any copyright, trademark, or proprietary notices contained in the Software.
        
        3. Ownership
        
        The Software is licensed, not sold. The copyright holder retains all right, title, and interest in and to the Software, including all intellectual property rights.
        
        4. Permitted Copies
        
        You may make copies of the Software solely for backup and archival purposes, provided that all copyright and license notices are retained.
        
        5. Termination
        
        This License automatically terminates if you fail to comply with any of its terms. Upon termination, all rights granted under this License cease immediately, and you must stop using and destroy all copies of the Software in your possession or control.
        
        6. Disclaimer of Warranty
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, AND NON-INFRINGEMENT.
        
        7. Limitation of Liability
        
        IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
        
        
        --------------------------------
          Used third party modules
        --------------------------------
        
        - jinja
         
           Copyright 2007 Pallets
        
           BSD-3-Clause License - https://jinja.palletsprojects.com/en/3.1.x/license/
        
        - pyyaml
        
          Copyright (c) 2017-2021 Ingy döt Net
        
          Copyright (c) 2006-2016 Kirill Simonov   
            
          MIT License - https://github.com/yaml/pyyaml/blob/main/LICENSE
        
        - pydantic
        
          MIT License - https://github.com/pydantic/pydantic/blob/main/LICENSE
        
        - rich
          
          Copyright (c) 2020 Will McGugan
        
          MIT License - https://github.com/Textualize/rich/blob/master/LICENSE
        
Project-URL: Homepage, https://dwgen.de
Keywords: database,dwh,generator,data vault
Classifier: Programming Language :: Python :: 3
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Topic :: Software Development :: Code Generators
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml<7,>=6.0.0
Requires-Dist: jinja2<4,>=3.1.4
Requires-Dist: rich<15,>=14.3.3
Requires-Dist: sqlglot<27,>=26.0.1
Requires-Dist: pydantic<3,>=2.10.0
Requires-Dist: packaging>=24.0
Dynamic: license-file

# DWH Generator (dwgen)

**Metadata-driven code generation for Data Warehouse projects.**

`dwgen` is a command-line tool for generating Data Warehouse artifacts from YAML metadata and Jinja templates.
It processes metadata definitions written in YAML and applies project-specific Jinja templates to generate any type of text-based output, such as SQL scripts, SQLProj files, XML configurations, Markdown documents, CSV files, Python code, or HTML.

Define your data warehouse entities once, validate them, model their dependencies, and generate the required SQL, configuration, documentation, or integration code from reusable templates.

> **Define your metadata once. Generate everything from it.**

## Why dwgen?

Data Warehouse projects often require the same information in multiple places: database schemas, ETL pipelines, deployment projects, documentation, configuration files, and more.

Maintaining these artifacts manually can lead to duplication, inconsistencies, and a significant amount of repetitive work.

`dwgen` takes a metadata-driven approach:

```text
             YAML Metadata
                   │
                   ▼
          ┌─────────────────┐
          │      dwgen      │
          │                 │
          │  • Validate     │
          │  • Resolve      │
          │    dependencies │
          │  • Apply Jinja  │
          │    templates    │
          └────────┬────────┘
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
      SQL       Config     Docs
```

The generated files are ordinary text files. Once they have been generated, they do not require `dwgen` at runtime.

> `dwgen` turns a validated metadata model and its dependency graph into any text-based artifact through customizable Jinja templates.

## Key Features

* **Tool-Agnostic**

  No tool or process in your architecture depends on *dwgen*. Once *dwgen* has generated the required code, it can be removed from the project, and the generated code continues to run independently.

  *dwgen* is neither an engine nor a runtime — it is a code generator.


* **Template-Based**

  *dwgen* harnesses the power of customizable templates to generate outputs tailored to your specific Data Warehouse requirements.

  Whether you are defining database schemas, designing ETL processes, or documenting your data architecture, *dwgen* provides the flexibility to generate the artifacts you need.

  *dwgen* uses the Jinja template engine. Combined with a comprehensive API for all entities, this makes it easy to customize existing templates or create your own.


* **Entity Lineage & Dependency Graph**

  Entities can reference and depend on other entities.

  `dwgen` builds a dependency graph (DAG) from these relationships and exposes the graph through the Template API.

  This allows templates to take dependencies and entity lineage into account when generating output.


* **Rich Validation**

  *dwgen* validates the YAML schema for each entity type and ensures that properties and dependencies are defined correctly.


* **Variety of Outputs**

  Through customizable templates, *dwgen* can generate virtually anything that can be expressed as text output, including SQL scripts, SQLProj files, XML configurations, Markdown documents, CSV files, Python code, and HTML.

  The output is determined by your templates, not by `dwgen`.

  This makes it possible to use the same metadata for completely different targets.

  For example:


   * **Database Deployment**

      * SQL Server
      * MySQL
      * PostgreSQL
      * Snowflake
      * Microsoft Fabric / Delta

   * **Data Integration**

      * SSIS
      * Pentaho
      * Dagster
      * Airflow
      * Notebooks
      * Stored Procedures
      * Spark-Jobs

   * **Documentation**

      * Markdown
      * HTML
      * Data dictionaries
      * Entity documentation
      * End-to-End lineage documentation

   * **Configuration**

      * Microsoft Fabric Lakehouse
      * Microsoft Fabric Warehouse
      * SSDT / SQLProj
      * Custom configuration formats
      * Datawarehouse Domain Mappings
      * Integration-Job-Dependencies
   
   * **Reporting Artifacts**
      
      * MicroStrategy Schema-Objects (creation scripts)
      * Metabase schema (metadata-sql)
      * Power BI Semantic Models 



* **File-Based Configuration**

  By defining each entity in its own `.yaml` file, it is easy to track changes and manage individual definitions. Each entity definition is designed to require only a minimal number of declarations.

## Install

```sh
pip install dwgen
```

## Usage

### Generate

Run `dwgen` from your project directory:

```sh
dwgen
```

The project configuration is defined in `dwgen.yaml`:

`dwgen` reads the project configuration and generates the configured artifacts.

The project configuration file `dwgen.yaml` must be located in the directory from which you call `dwgen` (the current working directory, or `cwd`).

### Synchronize

Use `dwsync` to synchronize generated files from the build directory to their configured targets:

```sh
dwsync
```

## Project Structure

A typical project might look like this:

```text
my-dwh-project/
│
├── dwgen.yaml
│
├── metamodel/
│      ├── entities/
│      │   ├── customer.yaml
│      │   ├── product.yaml
│      │   └── sales.yaml
│      │
│      └── templates/
│          ├── sql/
│          ├── dataintegration/
│          ├── documentation/
│          └── configuration/
│
└── build/
    ├── sql/
    ├── dataintegration/
    ├── documentation/
    └── configuration/
```

The exact structure is configurable and depends on your project setup.

## Reproducible Generation

Because the generated artifacts are derived from version-controlled metadata and templates, the generation process can be reproduced consistently across environments.

This makes `dwgen` suitable for automated build and deployment workflows as well as local development.

## Use Cases

`dwgen` is particularly useful when the same Data Warehouse metadata needs to be represented in multiple artifacts or technologies.

Typical use cases include:

* Generating database DDL from a logical model
* Generating deployment projects
* Generating ETL or orchestration definitions
* Generating documentation from the data model
* Generating data dictionaries
* Generating configuration files

## Design Principles

`dwgen` follows a few simple principles:

* **Metadata first**
 
  Define the model and its relationships independently of the generated target.

* **Templates define the output**

  `dwgen` does not prescribe how your artifacts should look. Templates control the generated output.

* **No runtime dependency**

  Generated artifacts do not require `dwgen` to execute.

* **Reproducible**

  The same metadata, templates, and configuration produce the same result.

* **Extensible**

  New output formats and project-specific conventions can be implemented through templates.


In essence, **dwgen** provides a powerful, metadata-driven approach to automating Data Warehouse development. By combining YAML-based definitions with flexible Jinja templates, it enables the repeatable, consistent, and maintainable generation of text-based DWH artifacts.


## License

Excerpt (See [LICENSE](LICENSE) for details):

```text
1. Grant of License

Subject to the terms and conditions of this License, permission is hereby granted, free of charge, to any person obtaining a copy of this Software and associated documentation files (the "Software") to use the Software for private, educational, internal business, and commercial purposes.

2. Restrictions

Except as expressly permitted by this License, you may not:

    a). modify, adapt, translate, reverse engineer, decompile, disassemble, or create derivative works based upon the Software;

    b) redistribute, publish, upload, transmit, sublicense, rent, lease, lend, or otherwise make the Software available to any third party;

    c) sell, resell, bundle, or commercially distribute the Software, whether as a standalone product or as part of another product, without prior written permission from the copyright holder;

    d) remove, alter, or obscure any copyright, trademark, or proprietary notices contained in the Software.

...

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, AND NON-INFRINGEMENT.

... 

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


```


