Metadata-Version: 2.4
Name: wranglefy
Version: 0.1.0
Summary: Automated data import and wrangling from many formats (CSV, JSON, Excel, Parquet, and more).
Author-email: Devisri Bandaru <bandarudevisri.ds@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/devisri/wranglefy
Project-URL: Repository, https://github.com/devisri/wranglefy
Project-URL: Issues, https://github.com/devisri/wranglefy/issues
Keywords: data,etl,pandas,data-wrangling,data-cleaning,csv,json,excel,parquet,import
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5
Requires-Dist: numpy>=1.21
Provides-Extra: excel
Requires-Dist: openpyxl>=3.0; extra == "excel"
Requires-Dist: xlrd>=2.0; extra == "excel"
Requires-Dist: odfpy>=1.4; extra == "excel"
Provides-Extra: parquet
Requires-Dist: pyarrow>=10.0; extra == "parquet"
Provides-Extra: html
Requires-Dist: lxml>=4.6; extra == "html"
Requires-Dist: beautifulsoup4>=4.9; extra == "html"
Requires-Dist: html5lib>=1.1; extra == "html"
Provides-Extra: xml
Requires-Dist: lxml>=4.6; extra == "xml"
Provides-Extra: spss
Requires-Dist: pyreadstat>=1.1; extra == "spss"
Provides-Extra: all
Requires-Dist: openpyxl>=3.0; extra == "all"
Requires-Dist: xlrd>=2.0; extra == "all"
Requires-Dist: odfpy>=1.4; extra == "all"
Requires-Dist: pyarrow>=10.0; extra == "all"
Requires-Dist: lxml>=4.6; extra == "all"
Requires-Dist: beautifulsoup4>=4.9; extra == "all"
Requires-Dist: html5lib>=1.1; extra == "all"
Requires-Dist: pyreadstat>=1.1; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: openpyxl>=3.0; extra == "dev"
Requires-Dist: pyarrow>=10.0; extra == "dev"
Requires-Dist: lxml>=4.6; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: ruff>=0.1; extra == "dev"
Dynamic: license-file

# wranglefy

**Automated data import and wrangling from many formats.**

`wranglefy` is a small, pandas-powered library that takes the friction out of the first mile of data work: getting messy files of *any* common format into a clean `DataFrame`. Point it at a file and it figures out the format; hand it a frame and it tidies column names, normalises missing values, infers types, de-duplicates rows, and more — in one call or as a readable, chainable pipeline.

```python
import wranglefy as of

df = of.read("sales.xlsx")            # format auto-detected from extension/content
clean = of.auto_wrangle(df)           # snake_case cols, typed, de-duped, tidy NaNs

# …or do both at once:
clean = of.load("sales.csv.gz", wrangle=True)
```

## Why

Real datasets arrive as CSVs with `"N/A"` sprinkled through them, Excel exports with `"First Name"` columns, gzipped JSON lines, the occasional Parquet file, and worse. `wranglefy` collapses the repetitive boilerplate of detecting formats and cleaning frames into a couple of well-tested calls, while staying out of your way when you need full control (every reader passes `**kwargs` straight through to pandas).

## Installation

```bash
pip install wranglefy
```

The core install only depends on pandas/numpy and reads CSV, TSV, JSON, JSON
Lines, and Pickle. Heavier formats live behind optional extras:

```bash
pip install "wranglefy[excel]"      # .xlsx / .xls / .ods
pip install "wranglefy[parquet]"    # Parquet / Feather / ORC (pyarrow)
pip install "wranglefy[html]"       # HTML tables
pip install "wranglefy[xml]"        # XML
pip install "wranglefy[spss]"       # SPSS .sav
pip install "wranglefy[all]"        # everything
```

## Supported formats

| Format       | Extensions                              | Extra      |
|--------------|-----------------------------------------|------------|
| CSV          | `.csv` (+ `.gz`, `.zip`, `.bz2`, `.xz`) | core       |
| TSV          | `.tsv`, `.tab`                          | core       |
| JSON         | `.json`                                 | core       |
| JSON Lines   | `.jsonl`, `.ndjson`                     | core       |
| Pickle       | `.pkl`, `.pickle`                       | core       |
| Excel        | `.xlsx`, `.xls`, `.xlsm`, `.ods`        | `excel`    |
| Parquet      | `.parquet`, `.pq`                       | `parquet`  |
| Feather      | `.feather`                              | `parquet`  |
| ORC          | `.orc`                                  | `parquet`  |
| HTML tables  | `.html`, `.htm`                         | `html`     |
| XML          | `.xml`                                  | `xml`      |
| Stata        | `.dta`                                  | core       |
| SAS          | `.sas7bdat`, `.xpt`                     | core       |
| SPSS         | `.sav`, `.zsav`                         | `spss`     |

## Reading data

```python
import wranglefy as of

# Single file — format detected from extension, then content if needed.
df = of.read("data.parquet")

# Force a format (e.g. an extensionless file or a buffer).
df = of.read(file_obj, format="csv")

# Read & concatenate many files; tag each row with its source.
df = of.read_many("exports/2024-*.csv", add_source_column=True)

# Inspect what wranglefy can do.
of.supported_formats()
of.detect_format("archive/data.json.gz")   # -> "json"
```

## Wrangling data

### One-shot

```python
clean = of.auto_wrangle(df)
```

The default pipeline: clean column names → strip strings → standardise missing values → drop empty rows/columns → infer types → de-duplicate. Every step is a keyword you can switch off, and `fill_missing` / `drop_constant` can be switched
on:

```python
clean = of.auto_wrangle(
    df,
    infer_types=True,
    dedupe=False,
    drop_constant=True,
    fill_missing="median",   # or True for the "auto" strategy
    optimize_memory=True,    # downcast floats/ints & categorise strings
    redact_pii=True,         # auto-hash emails, ssns, etc.
    clip_outliers=True,      # clip numeric outliers using IQR
)
```

### Chainable pipeline

For full control, use the `Wrangler` directly. It copies the input (never
mutates it) and records each operation in `.log`:

```python
from wranglefy import Wrangler

wr = (
    Wrangler(df)
    .clean_columns()                      # "First Name" -> "first_name"
    .strip_strings()
    .standardize_missing(["TBD", "???"])  # extend the default sentinel set
    .drop_empty(axis="both")
    .infer_types(threshold=0.95, category=True)
    .fill_missing("auto")
    .clip_outliers(method="iqr")          # cap statistical anomalies
    .redact_pii(method="mask")            # *** out sensitive columns
    .optimize_memory()                    # shrink memory footprint safely
    .dedupe(subset=["id"])
)

clean = wr.to_frame()
print(wr.log)   # ['clean_columns', 'strip_strings', ...]

# Drop into raw pandas whenever you need to:
wr.apply(lambda d: d[d["score"] > 0])
```

### Operations at a glance

| Method                  | What it does                                                        |
|-------------------------|---------------------------------------------------------------------|
| `clean_columns()`       | Normalise headers to unique `snake_case` identifiers                |
| `strip_strings()`       | Trim whitespace from every text cell                                |
| `standardize_missing()` | Turn `""`, `"NA"`, `"null"`, `"-"`, … into real `NaN`               |
| `drop_empty()`          | Remove all-NaN rows and/or columns                                  |
| `drop_constant()`       | Remove columns with a single unique value                           |
| `infer_types()`         | Coerce text to bool / numeric / datetime / category (threshold-based) |
| `dedupe()`              | Drop duplicate rows                                                 |
| `fill_missing()`        | Impute via `auto`/`mean`/`median`/`mode`/`zero`/`ffill`/`bfill`/`constant`/`drop` |
| `optimize_memory()`     | Downcast numeric types (e.g. `float64` -> `float32`) and categorise strings |
| `redact_pii()`          | Automatically identify and hash/mask sensitive PII columns            |
| `clip_outliers()`       | Cap numeric outliers using statistical bounds (IQR or Z-Score)        |
| `apply(func)`           | Escape hatch for any `df -> df` transform                           |

## Development

```bash
pip install -e ".[dev]"
pytest
```

## License

MIT © Devisri Bandaru
