Metadata-Version: 2.5
Name: slicksmith-ttom
Version: 0.2.0
Summary: Get, arrange and score the Ttom Sentinel-1 oil spill dataset. Model-agnostic.
Author-email: Halyjo <harald.l.joakimsen@uit.no>
License: MIT License
        
        Copyright (c) 2025 Harald Lykke Joakimsen
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: py7zr>=0.22.0
Requires-Dist: rasterio>=1.4.3
Requires-Dist: scipy
Description-Content-Type: text/markdown

# slicksmith-ttom

Get, arrange and score the **Ttom** Sentinel-1 oil spill dataset. Small, and it
works with any model.

It does four things:

1. **Downloads** the dataset from Zenodo.
2. **Arranges** it in `train`, `val` and `test` folders.
3. **Cleans the labels** (closes gaps, drops tiny blobs), and keeps the original ones.
4. **Scores a model on whole scenes**, in a way that does not reward empty tiles.

There is no training code here and no model. You bring the model.

## The data

Trujillo-Acatitla, Tuxpan-Vargas, Ovando-Vázquez and Monterrubio-Martínez (2024),
*Sentinel-1 SAR oil spill image dataset for train, validate, and test deep
learning models*, licence CC-BY 4.0. Cite the authors' three Zenodo records:
[Part I](https://doi.org/10.5281/zenodo.8346860),
[Part II](https://doi.org/10.5281/zenodo.8253899),
[Part III](https://doi.org/10.5281/zenodo.13761290).

3,020 scenes of 2048 x 2048 pixels, 10 m per pixel, two bands (VV and VH) in dB.
Each scene has a binary oil label.

| category | what it is | train | val | test |
|---|---|---|---|---|
| `oil` | scenes with oil | 960 | 240 | 150 |
| `lookalikes` | dark features that are not oil (label is empty) | 548 | 137 | 150 |
| `non_oil` | open water (label is empty) | 548 | 137 | 150 |

## Get it

You need [`uv`](https://docs.astral.sh/uv/). The download is about 120 GB and
unpacking needs about the same again, so `--move` in step 2 is worth using.

```bash
git clone <this repo> && cd slicksmith-ttom && uv sync

uv run slicksmith-ttom download --raw-dir /data/ttom_raw
uv run slicksmith-ttom arrange  --raw-dir /data/ttom_raw --root /data/Ttom --move
uv run slicksmith-ttom labels   --root /data/Ttom
uv run slicksmith-ttom check    --root /data/Ttom
```

Every step is safe to run twice. You end up with:

```
/data/Ttom/{split}/{category}/
    images/*.tif
    labels/original/*.tif                 as published
    labels/closed_10_rm_small_10/*.tif    cleaned
```

### How train and val are made

Zenodo ships a test set and one train-and-validation set. Here, for each category,
the scene names are sorted, the **first 80 % are `train`, the rest `val`**. No random
numbers, so everyone gets the same folders. It is easy to read and easy to repeat.

It says nothing about *where* the scenes were taken. Neighbouring names may show the
same slick, and nobody has checked whether they do. Treat `val` as "scenes the
model did not train on", not as "another place" or "another day".

### The cleaned labels

Some labels were made by an algorithm, so their edges are ragged and there are small
stray blobs. Counting blobs as slicks then counts every stray blob. `labels` **closes**
gaps with a disk of radius 10 pixels and **removes blobs under 10 pixels** (about
1,000 m²). Those two numbers were chosen by looking at many training scenes and
comparing the number of slicks a person sees with the number the computation gives.
They matched well. That was a look, not a search: no other setting was scored. If you
use `closed_10_rm_small_10`, say how it was made. Other settings:
`slicksmith-ttom labels --radius 5 --min-size 50`.

## Use it

```python
from slicksmith_ttom import TtomDataset

train = TtomDataset(
    "/data/Ttom", "train", resize=None
)  # 16 full-resolution tiles per scene
sample = train[0]  # {"image": (2, 512, 512) dB, "mask": (512, 512), "category", "path"}
```

It is plain numpy, so a PyTorch `DataLoader` takes it as it is. Values are raw dB
(about -36 to +13). Normalise them yourself. Bands come out **VV first, then VH**
(the files store VH first); pass `bands=("VH", "VV")` to change it. A model fed the
wrong order does not crash, it just does badly, so check this first.

## Score a model

Give it a function from tiles to probabilities:

```python
from slicksmith_ttom import evaluate, detection_by_size


def predict(tiles):  # (B, 2, 512, 512) float32, dB, VV then VH
    x = my_normalise(tiles)
    return my_model(x)  # (B, 512, 512) probabilities, same tile every time


result = evaluate(
    predict,
    "/data/Ttom",
    split="test",
    thresholds=[0.1, 0.5],
    out_dir="results/my_model",
)
print(result.summary)  # one row per category and threshold
print(detection_by_size(result.slicks))  # share of slicks found, by size in m²
```

Or from the command line: `slicksmith-ttom eval --root ... --predict my_module:predict`.

**Why whole scenes.** Averaging a score over tiles is wrong for sparse targets. An
empty tile with one false pixel scores dice 0, just like a tile that missed a big
slick, and most tiles are empty. So the model runs on overlapping tiles, the results
are merged into one map the size of the scene, and *that* is scored. Counts are
added up over the whole split.

**What is counted.** Pixels (true and false positives, false negatives). Slicks,
where a slick is a blob of connected pixels: a true slick is *found* if any predicted
blob touches it, and a predicted blob touching no true slick is a *false alarm*.
Blobs under `min_area_m2` (default 1,000 m²) are ignored on both sides. Every slick
and false alarm is also listed with its size, so you can ask how detection depends on
size.

`lookalikes` and `non_oil` have empty labels, so the number to read there is
`false_alarms_per_scene`.

## Things to know

- `resize=512` shrinks the label with nearest-neighbour, which can delete the
  smallest slicks. Use tiles (`resize=None`) and `evaluate` for anything about
  small slicks.
- Cleaning shaves a slick that touches the scene edge by up to the closing radius.
  This is how the published cleaned labels were made and it is kept.
- Scene names repeat across categories (`00000` is in all three). The scene is
  the name *and* the category.

## Development

```bash
uv run pytest
uv run ruff check . && uv run ruff format .
```

Each module ends in a short `__main__` block that shows how it is used.
