Metadata-Version: 2.1
Name: deltacat
Version: 2.0.0b2
Summary: A portable, scalable, fast, and Pythonic Data Lakehouse for AI.
Home-page: https://github.com/ray-project/deltacat
Author: Ray Team
License: UNKNOWN
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.9
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aws-embedded-metrics==3.2.0
Requires-Dist: boto3~=1.34
Requires-Dist: getdaft==0.3.6
Requires-Dist: intervaltree==3.1.0
Requires-Dist: numpy==1.21.5
Requires-Dist: pandas==1.3.5
Requires-Dist: pyarrow==17.0.0
Requires-Dist: pydantic!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<3
Requires-Dist: pymemcache==4.0.0
Requires-Dist: ray>=2.20.0
Requires-Dist: s3fs==2024.5.0
Requires-Dist: tenacity==8.2.3
Requires-Dist: typing-extensions==4.6.1
Requires-Dist: redis==4.6.0
Requires-Dist: schedule==1.2.0
Provides-Extra: iceberg
Requires-Dist: pyiceberg[glue]>=0.9.0; extra == "iceberg"
Requires-Dist: pyiceberg[hive]>=0.9.0; extra == "iceberg"
Requires-Dist: pyiceberg[sql-sqlite]>=0.9.0; extra == "iceberg"

<p align="center">
  <img src="media/deltacat-logo-alpha-750.png" alt="DeltaCAT Logo" style="width:55%; height:auto; text-align: center;">
</p>

DeltaCAT is a portable Pythonic Data Lakehouse powered by [Ray](https://github.com/ray-project/ray). It lets you define and manage
fast, scalable, ACID-compliant multimodal data lakes, and has been used to [successfully manage exabyte-scale enterprise
data lakes](https://aws.amazon.com/blogs/opensource/amazons-exabyte-scale-migration-from-apache-spark-to-ray-on-amazon-ec2/).

It uses the Ray distributed compute framework together with [Apache Arrow](https://github.com/apache/arrow) and
[Daft](https://github.com/Eventual-Inc/Daft) to efficiently scale common table management tasks, like petabyte-scale
merge-on-read and copy-on-write operations.

DeltaCAT provides four high-level components:
1. **Catalog**: High-level APIs to create, discover, organize, share, and manage datasets.
2. **Compute**: Distributed data management procedures to read, write, and optimize datasets.
3. **Storage**: In-memory and on-disk multimodal dataset formats.
4. **Sync**: Synchronize DeltaCAT datasets to data warehouses and other table formats.


## Getting Started

DeltaCAT is rapidly evolving. Usage instructions will be posted here soon!

For now, feel free to peruse some of our examples:
* https://github.com/ray-project/deltacat/tree/2.0/deltacat/examples/rivulet
* https://github.com/ray-project/deltacat/tree/2.0/deltacat/examples/iceberg


