Metadata-Version: 2.1
Name: tf-datachain
Version: 0.1.0
Summary: A local dataset loader based on tf.data input pipeline
Author: Yiming Liu
Author-email: YimingDesigner@gmail.com
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: matplotlib
Requires-Dist: pandas
Requires-Dist: tensorflow
Requires-Dist: keras-cv
Requires-Dist: keras-core
Requires-Dist: opencv-python

# tf-datachain

`tf-datachain` is a local dataset loader based on `tf.data` input pipeline. It handles the job of reading and encoding data direct in your disk and simplify the processing by providing several predefined methods.

## Object Detection

```python
from tf_datachain import ObjectDetection as od
```

Before using `ObjectDetection` functions, you have to define some basic information, like folder path and class name list.

```python
od.imageFolder = "data/images"

# hard-code class names
od.classNames = ["class1", "class2", "class3"]
# or read them from csv file
import pandas as pd
od.classNames = pd.read_csv("class.csv", header=None).iloc[:,0].values.tolist()
```

Then, a ready-to-use `tf.data` input pipeline can be built within 3 steps:

- Preparation: prepare the list to process without reading content.
- Data Loading: load data from prepared list via `tf.data`.
- Augmentation: shuffle, batch, and resize.

The best practices to load dataset with different format are shown below.

### Pascal VOC XML Format

```python
from tf_datachain.utils import split

BATCH_SIZE = 4
# read .xml file within data/annotaions folder
# then split them with the ratio of 6:2:2
trainDataset, validationDataset, testDataset = split(od.prepareAnnotation("data/annotations", ".xml"), 6, 2, 2)

trainDataset = tf.data.Dataset.from_tensor_slices(trainDataset)
trainDataset = trainDataset.map(lambda data: od.loadData(data, "Pascal VOC XML", "xyxy"), num_parallel_calls=tf.data.AUTOTUNE)
# shuffle, ragged batch, and jittered resize
trainDataset = od.datasetProcessing(trainDataset, BATCH_SIZE, "Jittered Resize", (960, 960), "xyxy")
trainDataset = trainDataset.prefetch(tf.data.AUTOTUNE)
```

### Visualize Dataset

```python
# visualize single data
for data in dataset.take(1):
  visualizeData(data, "xyxy")

# visualize dataset shown in 2x2 grid
visualizeDataset(dataset, "xyxy", rows=2, cols=2)
```

