Metadata-Version: 2.4
Name: synth-data-tn
Version: 0.1.5
Summary: Synthetic Tunisian data generator
License: MIT
License-File: LICENSE
Author: Mahdi Hammi
Author-email: your@email.com
Requires-Python: >=3.11,<3.14
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: click (>=8.1.0)
Requires-Dist: faker (>=26.0.0)
Requires-Dist: hijridate (>=2.6.0,<3.0.0)
Requires-Dist: pandas (>=3.0.2,<4.0.0)
Requires-Dist: pyarrow (>=16.0.0)
Project-URL: Homepage, https://github.com/mahdihammi/synth_data_tn
Project-URL: Repository, https://github.com/mahdihammi/synth_data_tn
Description-Content-Type: text/markdown

# synth-data-tn

Synthetic Tunisian data generator — Faker-based, locale-aware, **seasonally context-aware**.
Generates realistic *ecommerce*, *banking* and *telecom* datasets as CSV or Parquet, in French,
with Tunisian geography, banks, operators, payment methods — and spending behaviour that
follows the Tunisian calendar (Ramadan, Eids, summer holidays, sales periods…).

---

## Features

- **3 correlated domains** — every dataset is join-ready (`*_id` foreign keys link the tables).
  | Domain | Output files | Row counts (given `n`) |
  |---|---|---|
  | ecommerce | `ecommerce_customers`, `ecommerce_products`, `ecommerce_orders` | `n` customers, 15 products, `n*3` orders |
  | banking | `banking_accounts`, `banking_transactions` | `n` accounts, `n*10` transactions |
  | telecom | `telecom_subscribers`, `telecom_usage` | `n` subscribers, `n*6` usage records |

- **Seasonal context engine** — each generated row's date drives its behaviour:
  transaction volume, amounts, late-night activity (22:00–03:00) and spend
  distribution across categories. Islamic calendar events are resolved **dynamically**
  with [`hijridate`](https://pypi.org/project/hijridate/) — no hardcoded years, valid ~1924→2077 CE.

- **Tunisian provider** — governorates & cities (consistent pairs), Ibn Khaldoon-class banks,
  `TN59` IBANs, `+216` phone numbers, Ooredoo/Orange/Tunisie Telecom plans,
  local payment methods (Flouci, D17, paiment à la livraison…), francophone name pools.

- **Two output formats** — CSV (`utf-8-sig`, Excel-friendly) or Parquet (pyarrow).

---

## Installation

Requires Python **3.11–3.13**.

```bash
# With Poetry (recommended)
poetry install --sync --no-interaction

# Or with pip
pip install pandas "faker>=26" "pyarrow>=16" "click>=8.1" "hijridate>=2.6"
```

---

## Usage — CLI

All commands below assume `poetry run` prefix (drop it if installed globally).

### 1. Quick start — all domains

```bash
poetry run synth-data-tn generate
```

Generates every domain with defaults: `n=100`, context `auto`, CSV, seed 42, output `output/`.

### 2. Choose the context

```bash
poetry run synth-data-tn generate --context ramadan
```

Available contexts (`--context`):

| Context | Meaning | Volume | Amounts | Late-night |
|---|---|---|---|---|
| `auto` *(default)* | Each row's date picks its own season automatically | ⟵(per row) | per row | per row |
| `normal` | No seasonal effect | ×1.0 | ×1.0 | 10 % |
| `ramadan` | Ramadan | ×0.7 | ×1.2 | 55 % |
| `eid_al_fitr` | Eid al-Fitr | ×2.2 | ×1.8 | 20 % |
| `eid_al_adha` | Eid al-Adha | ×1.8 | ×2.0 | 15 % |
| `summer` | Summer holidays (June & August) | ×1.3 | ×1.1 | 25 % |
| `back_to_school` | September | ×1.4 | ×1.3 | 8 % |
| `soldes` | Sales season (January & July) | ×3.0 | ×0.7 | 15 % |
| `end_of_month` | Any day ≥ 25 | ×1.6 | ×1.2 | 12 % |

> In `auto` mode, only the *weights of each row* change with the season; the total row count
> stays `n` (times the domain multiplier). In a *fixed* context, `transaction_volume`
> additionally scales the row counts (e.g. `--context soldes` generates ×3 rows).
>
> ⚠️ **Fixed contexts are uplift simulations, not calendar slices.** Dates keep spreading
> over the normal ~2-year lookback — only the seasonal *weights* (amounts, categories,
> late-night hours) are forced to the chosen context on every row. So with
> `--context eid_al_fitr` you may see `ordered_at = 2026-10-03` labelled `eid_al_fitr`:
> that row answers "what would this day's spending look like if it were Eid al-Fitr?".
> Only `auto` mode matches rows to the season their date actually falls in.

### 3. Common recipes

```bash
# 100 retail banking customers, transactions only, CSV
poetry run synth-data-tn generate --n 100 --domain banking

# Repeatable domains
poetry run synth-data-tn generate --n 50 --domain banking --domain ecommerce

# Analytics-ready Parquet, custom seed and folder
poetry run synth-data-tn generate --n 200 --format parquet --seed 99 --output ./data

# Simulate Eid al-Fitr shopping peak
poetry run synth-data-tn generate --n 100 --domain ecommerce --context eid_al_fitr
```

### 4. Full options

```text
Usage: synth-data-tn generate [OPTIONS]

Options:
  -n, --n INTEGER                 Number of base records (accounts/customers/subscribers).
                                  Default: 100
  -d, --domain [banking|ecommerce|telecom]  Repeatable. Default: all.
  -c, --context [...]            Seasonal context. Default: auto. (see table above)
  --format [csv|parquet]         Where the files land. Default: csv
  --seed INTEGER                 Seed. Same seed ⇒ same Faker data. Default: 42
  --output DIRECTORY             Output dir. Default: output/
```

### 5. Collect your data

Files are written to `output/` (or `--output`) as
`<domain>_<dataset>.{csv|parquet}`, e.g. `banking_transactions.csv`.

---

## Usage — as a library

```python
from pathlib import Path
from synth_data_tn.config import GeneratorConfig
from synth_data_tn.context import SeasonalContext
from synth_data_tn.pipeline import run_pipeline, available_domains

config = GeneratorConfig(
    seed=42,
    output_dir=Path("output"),
    output_format="csv",               # or "parquet"
    context="auto",                    # "ramadan", "soldes", … or SeasonalContext enum
)

written = run_pipeline(config, n=100, domains=["banking"])   # omit domains → all
for path in written:
    print(path)
```

---

## How seasonal weight rows work

Each row generator picks a timestamp (`seasonal_datetime`) and gets the
`WeightProfile` matching its date in `auto` mode (or the configured profile in fixed mode):

- `transaction_volume` scales the number of rows (fixed contexts only),
- `avg_amount` scales monetary values,
- `late_night_probability` shifts the time-of-day toward 22:00–03:00 (Ramadan nights),
- `category_weights` re-weights spend categories (e.g. Ramadan → `Alimentation` ×2.5).

Every `<transactions|orders|usage>` file carries a **`context` column** holding the
season actually applied to that row (e.g. `ramadan`, `soldes`) — in `auto` mode it reveals
the date-inferred season, in fixed mode it repeats the configured context — **without**
implying the date falls inside the real calendar window of that season (see the
uplift-simulation note above).

Transactions/orders also link a per-row spatial consistency: `city` / `governorate`
always come from the same governorate (a matched Tunisian pair, like `La Marsa`, `Tunis`).

---

## Reproducibility

Seeding covers all Faker-driven values (`--seed`), including order statuses and IBANs.
Exception: `uuid4()`-derived primary keys (`*_id`) are intentionally unseeded — the *shape*
and *content* are reproducible; row *IDs and ordering* may vary from run to run.

---

## Development

```bash
poetry install

# Lint + format check (CI gate)
poetry run ruff check src/
poetry run ruff format --check src/

# Tests
poetry run pytest

# Single test
poetry run pytest tests/test_context.py::test_from_date -q
```

Project structure (src layout):

```
src/synth_data_tn/
  cli.py            click entry point (`synth-data-tn generate`)
  config.py         GeneratorConfig (seed, format, locale, context)
  context.py        SeasonalContext + WeightProfile — single source of seasonal truth
  pipeline.py       domain registry + orchestration
  core/             BaseGenerator (seasonal helpers) & exporter
  domains/          banking / ecommerce / telecom generators
  providers/        TunisianProvider (Faker provider: names, IBANs, banks, plans…)
```

---

## License

MIT — see [LICENSE](LICENSE).

