Metadata-Version: 2.4
Name: datahub-ai-classifier
Version: 0.1.0
Summary: Customer-taxonomy LLM classification and tagging for DataHub
License-Expression: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: acryl-datahub[datahub-rest]>=1.7.0
Requires-Dist: openai>=1.40.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# DataHub Classifier

Classify DataHub fields against a customer-provided tag list using an
OpenAI-compatible LLM, then apply those tags.

Install:

```bash
pip install datahub-ai-classifier
```

Example recipe: `datahub_classifier.example.yml`.

Both plugins must be in the same recipe, with the same `taxonomy` map.

```yaml
source:
  type: postgres
  config:
    classification:
      enabled: true
      sample_size: 10
      max_workers: 1
      classifiers:
        - type: datahub_classifier.classifier:AIPIIClassifier
          config:
            llm_model: gpt-4.1-mini
            llm_base_url: https://api.openai.com/v1
            llm_api_key: "<your-api-key>"
            llm_timeout_seconds: 60
            llm_max_retries: 0
            confidence_threshold: 0.6
            max_samples: 8
            taxonomy: &taxonomy
              C1: >
                Restricted identifiers. Use for national IDs and tax numbers
                such as SSN values like 123-45-6789 stored in columns named
                ssn.
              C2: >
                Confidential personal and payment data. Use for a person's
                full_name, email, phone, home_address, and date_of_birth, and
                for order_total money amounts.
              C3: >
                Internal operational data. Use for customer_id, order_id,
                loyalty_tier, currency, order status, shipped_at, and
                created_at timestamps.

transformers:
  - type: datahub-classifier
    config:
      taxonomy: *taxonomy
```

Missing tags are created. Existing tags are not overwritten.
Classifier alone leaves glossary terms and does not create tags.

## Development

```bash
pip install -e '.[dev]'
pytest
```
