Metadata-Version: 2.4
Name: transferred
Version: 0.1.4
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries
Requires-Dist: pyarrow>=24 ; extra == 'arrow'
Requires-Dist: transferred[arrow] ; extra == 'iterable'
Provides-Extra: arrow
Provides-Extra: iterable
Summary: The most convenient data transfer tool.
Keywords: data-transfer,etl,arrow,parquet,postgres
Author: Vania Livadnyi
License-Expression: MIT
Requires-Python: >=3.14
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Repository, https://github.com/skatromb/transferred

# transferred

[![Check](https://github.com/skatromb/transferred/actions/workflows/check.yml/badge.svg)](https://github.com/skatromb/transferred/actions/workflows/check.yml)
[![Coverage](https://img.shields.io/codecov/c/github/skatromb/transferred.svg)](https://codecov.io/gh/skatromb/transferred)
[![Python](https://img.shields.io/pypi/pyversions/transferred.svg)](https://pypi.org/project/transferred/)
[![PyPI](https://img.shields.io/pypi/v/transferred.svg)](https://pypi.org/project/transferred/)
[![wemake-python-styleguide](https://img.shields.io/badge/style-wemake-000000.svg)](https://github.com/wemake-services/wemake-python-styleguide)

<img src="https://raw.githubusercontent.com/skatromb/transferred/main/logo.png" alt="transferred" width="240">

The most convenient batch data transfer tool. Inspired by [dlt](https://dlthub.com).

`transferred` moves structured data between files, databases, python iterables.
Blazing fast, no transformations supported — hand them over to your Data Warehouse.

## Install

```bash
pip install transferred
```

Requires Python 3.14.

## Usage

```python
from pathlib import Path

from transferred import FilesDestination, Parquet, PostgresSource, Transfer

source = PostgresSource(
    "postgres://user:pass@localhost:5432/db",
    table="public.orders",
)
destination = FilesDestination(
    Path("orders/"),
    format=Parquet(compression="zstd"),
)

report = Transfer(source, destination).run()

print(report)
# RunReport:
#   rows: 10,000,000
#   written: 243.24 MiB
#   duration: 13s 700ms
#   written objects:
#     orders/part-00001.parquet
```

More in [examples/](https://github.com/skatromb/transferred/tree/main/examples).

10M rows of 22 diverse columns including `jsonb` and PostGIS geometry — peaking at 414 MiB RAM, interpreter included.

Look at [docs/DLT_COMPARISON.md](https://github.com/skatromb/transferred/blob/main/docs/DLT_COMPARISON.md) for more performance insights.


## Supported

| Type | Source | Destination |
| ------ | ------ | ----------- |
| Parquet files | `FilesSource` | `FilesDestination` |
| Database table | `PostgresSource` | `PostgresDestination` |
| DataFrames | polars or pandas `DataFrame`, a duckdb result, a `pa.Table`, pyarrow's `RecordBatch` or `RecordBatchReader` | — |
| Python iterables | `dict` / `@dataclass` / `pydantic.BaseModel` (with `pip install transferred[iterable]`) | — |

BigQuery, S3/GCS and incremental loads land later. See [PLAN.md](https://github.com/skatromb/transferred/blob/main/PLAN.md).

## Promises

- Best practice by default, override if needed
- No boring sizing/config
- No Out-of-memory
- Blazing fast

## License

MIT. See [LICENSE](https://github.com/skatromb/transferred/blob/main/LICENSE).

