Metadata-Version: 2.4
Name: trakcorelib
Version: 0.2.0
Summary: Tracking utility library
Author-email: Antoine Cribellier <antoine.cribellier@wur.nl>, Cees Voesenek <cees.voesenek@vortech.nl>
License-Expression: LGPL-3.0-or-later
License-File: LICENCE.md
Requires-Python: >=3.9
Requires-Dist: numpy>=1.21.6
Requires-Dist: opencv-python>=4.5.5.64
Requires-Dist: pillow>=11.2.1
Requires-Dist: pyquaternion>=0.9.9
Requires-Dist: scipy>=1.13.1
Description-Content-Type: text/markdown

# TrakCoreLib
TrakCoreLib provides the building blocks that tracking applications commonly
need, grouped into four modules:

- **`trakcorelib.images`** — reading, manipulating and saving sequences of images;
- **`trakcorelib.calibration`** — modelling cameras with DLT calibrations:
  projecting 3D points into a view, and reconstructing 3D points from several
  views;
- **`trakcorelib.tracking`** — detecting moving objects and linking their 3D
  positions into tracks over time;
- **`trakcorelib.rotations`** — small helpers for rotations and the vectors they
  act on.

A typical multi-camera pipeline reads image sequences, detects the moving objects
in them, and uses a calibration to reconstruct and track those objects in 3D. The
sections below walk through each part — jump to the one you need. Everything shown
in a section is importable from that section's module (for example
`from trakcorelib.calibration import MultiViewCalibration`).

## Installation
We recommend creating a virtual environment to install packages in. For example:
```bash
python -m venv .venv
. .venv/bin/activate
```

Alternatively, you can use tools like [`uv`](https://docs.astral.sh/uv/) to
automatically manage your Python project.

### Installing with `pip`
Install the `trakcorelib` package from PyPI with:
```bash
pip install trakcorelib
```

### Local editable installation
Navigate to your local TrakCoreLib directory and install the package:
```bash
pip install -e .
```
The `-e` flag makes the package "editable", ensuring that edits you make to
the TrakCoreLib directory will be reflected in your own package.

## Image examples

### Reading a single image as 8-bit grayscale
To read an image as a [Pillow](https://hugovk-pillow.readthedocs.io/en/stable/index.html)
`Image` object, use:

```python
from trakcorelib.images import read_image_as_8bit_grayscale

image = read_image_as_8bit_grayscale("/path/to/image.tif")
```

For images with a bit depth higher than 8, different conversion modes to 8-bits
exist, for example, to use the lowest 8 bits:

```python
image = read_image_as_8bit_grayscale("/path/to/image.tif", convert_mode="low")
```

### Reading an image sequence from a directory
Use the `DirectoryImageSequence` class for reading an image sequence from a
directory; this represents a collection of images on disk. Images from this
sequence can be conveniently read, causing them to be cached such that the next
read does not need to touch the disk. Each image also has an identifier, which
can be any string.

To read an image sequence from a directory with a [glob](https://en.wikipedia.org/wiki/Glob_(programming)) pattern, use for example:
```python
sequence = DirectoryImageSequence.from_glob("/path/to/images", "cam001*.tif")
```
Here, a default image identifier is created for each image by finding the
substrings at the start and end of all filenames, and using the remainder as an
identifier. For example, the filenames "cam001_0005.tif", "cam001_0010.tif",
"cam001_0015.tif", all start with "cam001_00", and all end with ".tif", so the
remaining identifiers are: "05", "10", and "15".

Alternatively, an image sequence can be read from a directory with a [regular
expression](https://docs.python.org/3/library/re.html). This regular expression
should have a single group that contains the identifier. For example:
```python
sequence = DirectoryImageSequence.from_regex("/path/to/images", "cam001_(\d{4}).tif")
```
This will result in a sequence with the identifiers "0005", "0010", and "0015";
note that the leading "00" is preserved in this case, which may be useful in
some cases.

### Retrieving images from a sequence
Images can be easily retrieved from an image sequence, in the same way for all
types of images sequences.

Images can be retrieved by their index:
```python
image = sequence[10]
```

Or by their identifier:
```python
image = sequence.by_identifier("0010")
```

Furthermore, a sequence can be iterated over:
```python
for image in sequence:
    # Do something with the image.
    ...
```

### Saving an image to disk
An image sequence can be saved to disk, where each image is written to the same
directory.

For example, using a default filename pattern:
```python
sequence.save("/path/to/dest")
```
This will save the contents of the sequence as TIFF-images to
"image_<identifier>.tif", for example: "/path/to/dest/image_0005.tif",
"/path/to/dest/image_0010.tif", ...

The filename pattern and image format can also be specified. The filename
pattern should contain a single placeholder `{}`, which will be replaced by the
image identifier.
```python
sequence.save("/path/to/dest", filename_pattern="cam001_{}.jpg", image_format="JPEG")
```
Which will result in JPEG-images: "/path/to/dest/cam001_0005.jpg",
"/path/to/dest/cam001_0010.jpg", ...

### Creating and using an in-memory image sequence
Image sequences can also be created without any association with files on disk.
So-called "in-memory" sequences can be manipulated at will. Creating a
`MemoryImageSequence` can be done in several ways.

From an existing disk image sequence:
```python
disk_sequence = DirectoryImageSequence.from_glob("/path/to/images", "cam001*.tif")
memory_sequence = disk_sequence.to_memory_image_sequence()
```

Or a new sequence without any images:
```python
sequence = MemoryImageSequence.empty()
```

Or a new sequence with black (i.e. all pixels 0) images:
```python
sequence = MemoryImageSequence.new_8bit_grayscale(20, (1920, 1080))
```
This creates a sequence with 20 8-bit grayscale images, with a size of
1920 x 1080 pixels.

In-memory image sequences behave like lists, images can be set, inserted,
appended:
```python
sequence = MemoryImageSequence.new_8bit_grayscale(20, (1920, 1080))
sequence[10] = image
sequence.insert(3, image)
sequence.append(image)
```

Also, individual images in a `MemoryImageSequence` can be modified, unlike
disk image sequences: with a disk image sequence you always get a _copy_ of the
image, while a memory image sequence gives you a _reference_. For example:
```python
sequence = MemoryImageSequence.new_8bit_grayscale(20, (1920, 1080))

sequence[10].paste(image_to_be_pasted)
# The image at index 10 has now been modified.
```

### Applying image operations
Several common image operations (crop, resize, stitch, adjust brightness /
contrast / sharpness, ...) live in `trakcorelib.images.operations`.

They can be applied to a single image (i.e. a Pillow `Image` object), for
example:
```python
from trakcorelib.images.operations import adjust_brightness

adjusted = adjust_brightness(image, 1.5)
```

An operation can also be applied to a whole image sequence with `apply` (exported
from `trakcorelib.images`), which returns a `MemoryImageSequence` with the
operation applied to every image:
```python
from trakcorelib.images import apply
from trakcorelib.images.operations import adjust_brightness

adjusted = apply(sequence, adjust_brightness, 1.5)
```

## DLT calibration examples
TrakCoreLib creates and handles Direct Linear Transformation (DLT) calibrations —
the camera model that projects a 3D world point into a camera's 2D image and,
from two or more views, reconstructs a 3D point from where it appears in each. The
mathematics is documented in the Python modules; everything below is importable
from `trakcorelib.calibration`.

If you already have a calibration, start with *Loading a multi-view calibration*.
If you still need to make one, skip to *Estimating a calibration from digitised
points*.

### Loading a multi-view calibration
A multi-camera setup is described by a `MultiViewCalibration`: one DLT per view,
each with a name (`"view1"`, `"view2"`, ... by default). The usual way to get one
is to load a CSV you saved earlier — and it can be saved back the same way:
```python
calib = MultiViewCalibration.from_csv("/path/to/file.csv")
calib.to_csv("/path/to/other_file.csv")
```
If you already hold the raw 11 DLT coefficients of each view (for example from an
older file format), build a calibration straight from them, without constructing
each `DltCameraCalibration` yourself:
```python
calib = MultiViewCalibration.from_coefficients([coefs_cam1, coefs_cam2, coefs_cam3])
```
Or group per-view `DltCameraCalibration` objects directly, optionally naming the
views:
```python
calib = MultiViewCalibration([dlt1, dlt2, dlt3], ["cam1", "cam2", "cam3"])
```
The individual views and their names are available by index:
```python
dlt_cam1 = calib[0]
name_cam1 = calib.names[0]
```

### Reconstructing a 3D point from several views
Given where a point appears in each view — an N x 2 array with one row per view —
reconstruct its 3D position:
```python
image_points = [
    [10, 20],   # where the point is in view 1
    [30, 40],   # ... in view 2
    [50, 60],   # ... in view 3
]
x, y, z = calib.reconstruct_point_3d(image_points)
```
If you would rather not build a calibration object first, the module-level
`reconstruct_3d` and `project_to_2d` take either a calibration *or* the raw DLT
coefficients and construct what they need:
```python
from trakcorelib.calibration import project_to_2d, reconstruct_3d

x, y, z = reconstruct_3d(per_view_coefficients, image_points)   # several views -> 3D
uv = project_to_2d(dlt_coefficients, object_points)             # one camera, N x 3 -> N x 2
```

### Working with a single camera's DLT
Each view of a multi-view calibration — and any calibration you estimate for one
camera — is a `DltCameraCalibration`. Project 3D object points into its 2D image:
```python
image_points = dlt.project_to_2d(object_points)   # N x 3 -> N x 2
```
Check its accuracy against known correspondences, and read out its matrix,
coefficients or camera properties:
```python
error = dlt.compute_reprojection_rms_error(object_points, image_points)

matrix = dlt.projection_matrix
coefs = dlt.coefficients          # the 11 independent DLT coefficients
properties = dlt.compute_camera_properties()
intrinsic, extrinsic = properties.intrinsic, properties.extrinsic
```

### Estimating a calibration from digitised points
If you do not have a calibration yet, you make one by photographing an object of
known geometry from every view and digitising where each of its points lands in
each view. Read those two CSV files into a `CalibrationPoints`:
```python
points = read_calibration_points(
    "object_points.csv",   # columns x, y, z
    "image_points.csv",    # columns <view>_X, <view>_Y per view
    scale=0.001,           # e.g. read millimetres as metres
    image_height=1024,     # flip y when the points were digitised top-down (MATLAB)
)
```
A point that was not digitised in a view is left out for that view; a point that
turns out to be digitised badly can be dropped from every view by its number:
```python
points = points.without([3, 17])
```
Estimate the calibration from the points, then judge it with the per-view
reprojection error, or a leave-one-out error that flags a point worth
re-digitising (one whose absence lowers the error markedly):
```python
calibration = estimate_multiview_dlt(points)                    # classic DLT
calibration = estimate_multiview_dlt(points, do_optimise=True)  # orthogonal rotation matrix

errors = compute_reprojection_rms_errors(calibration, points)
identifiers, errors_without_each = compute_leave_one_out_errors(points, view_index=0)
```

### Estimating a single DLT from point arrays *(advanced)*
When you already have matched object/image point arrays in memory (rather than the
digitised-point CSVs above), you can estimate one camera's DLT directly. Three
methods are available:

- **classic** — a linear least-squares solve; historically the most common, but
  the resulting camera orientation matrix may not be perfectly orthogonal (i.e.
  not a valid rotation matrix);
- **optimisation** — fits the intrinsic and extrinsic camera properties directly,
  so the orientation matrix is guaranteed orthogonal; slightly more expensive but
  still near-instant;
- **modified (MDLT)** — Hatze's method; starts from the classic DLT and adds a
  non-linear constraint so the eleven coefficients correspond to a genuine,
  orthogonal camera ("homogeneous" DLT coefficients). Use it when downstream code
  expects constraint-satisfying coefficients (e.g. the DLTdv lineage).

```python
# object_points: N x 3, image_points: N x 2
dlt_classic = estimate_dlt_classic(object_points, image_points)
dlt_optimised = estimate_dlt_optimisation(object_points, image_points)
dlt_modified = estimate_dlt_modified(object_points, image_points)
```
The optimisation method can also hold part of the camera fixed — keep the
intrinsic (principal point, focal lengths) or the extrinsic (location,
orientation) and fit only the rest:
```python
fixed_intrinsic = IntrinsicProperties(
    u_principal=512, v_principal=512, focal_length_u=100, focal_length_v=100,
)
fixed_extrinsic = ExtrinsicProperties(
    coords=[0.1, 0.2, 0.3],
    rotation=pyquaternion.Quaternion([1.0, 2.0, 3.0, 4.0]).normalised,
)

with_fixed_intrinsic = estimate_dlt_optimisation(
    object_points, image_points, fixed_intrinsic=fixed_intrinsic,
)
with_fixed_extrinsic = estimate_dlt_optimisation(
    object_points, image_points, fixed_extrinsic=fixed_extrinsic,
)
```

### Calibrating a plane
When the calibrated object is planar (a 2D DLT), use `PlanarDltCalibration`. A
plane is a homography, so a single view already determines an object point on it —
no second view is needed:
```python
calibration = estimate_planar_dlt(object_points, image_points)   # object points are N x 2
image_points = calibration.project_to_2d(object_points)
object_point = calibration.reconstruct_point_2d([12, 34])
```
Several planar views can be grouped in a `MultiViewPlanarCalibration`, which
combines them to average out their error, exactly like `MultiViewCalibration`
does for a volume.

## Point tracking examples
TrakCoreLib can detect moving objects in a sequence of images and reconstruct
their 3D positions and tracks across several views. The detection uses OpenCV
(a required dependency).

### Detecting moving objects
`BackgroundBlobDetector` estimates a background from a sequence of grayscale
images, subtracts it, and detects blobs in what moved. It returns the blobs of
each image, and writes nothing:
```python
from trakcorelib.tracking import BackgroundBlobDetector

detector = BackgroundBlobDetector()   # or pass BackgroundSubtractorSettings / BlobDetectorSettings
detections = detector.detect(images)  # images: a sequence of grayscale arrays

for frame_blobs in detections:
    for blob in frame_blobs:
        print(blob.x, blob.y, blob.area)
```

### Reconstructing 3D points and tracks
Reconstructing tracks is two steps: reconstruct the 3D points of each frame from
the per-view 2D detections, then link those points into tracks over time.

First, `MultiViewReconstructor` turns the per-view 2D detections into 3D points.
Give it the detections of each view — their x, y and the frame each belongs to:
```python
from trakcorelib.tracking import MultiViewReconstructor, ViewDetections

reconstructor = MultiViewReconstructor(calibration)   # a MultiViewCalibration

detections_per_view = [
    ViewDetections(x=xs_cam1, y=ys_cam1, frame=frames_cam1),
    ViewDetections(x=xs_cam2, y=ys_cam2, frame=frames_cam2),
    ViewDetections(x=xs_cam3, y=ys_cam3, frame=frames_cam3),
]

objects_by_frame = reconstructor.reconstruct_objects(detections_per_view)
```
`objects_by_frame` maps each frame to the `ReconstructedPoint`s found in it (with
their x, y, z and reprojection error). `ReconstructorSettings` tunes the matching.

Then link those points into tracks. The recommended linker is `KalmanTracker`: it
predicts each object's motion with a constant-velocity Kalman filter, matches all
objects of a frame at once, and coasts over short gaps, so it keeps identities
through crossings and occlusions:
```python
from trakcorelib.tracking import KalmanTracker, KalmanTrackerSettings

tracker = KalmanTracker()                    # or KalmanTracker(KalmanTrackerSettings(...))
tracks = tracker.track(objects_by_frame)
```
Each `Track` holds the path of one object over time. `KalmanTrackerSettings` tunes
the filter with a few physical scalars (`process_noise`, `measurement_noise`,
`gate_distance`, `max_gap`, `min_length`).

A simpler greedy linker, `MultiViewReconstructor.stitch_tracks`, is also
available. It is easier to reason about but loses identities where objects cross,
so prefer `KalmanTracker` when several objects are tracked at once:
```python
tracks = reconstructor.stitch_tracks(objects_by_frame)   # simpler, greedy alternative
```

## Rotation examples
`trakcorelib.rotations` has helpers for rotations and the vectors they act on,
for example the angle between two vectors, the rotation taking one onto another,
and converting any common description of a rotation to a quaternion:
```python
from trakcorelib.rotations import angle_between_vectors, rotation_between_vectors, to_quaternion

angle = angle_between_vectors([1, 0, 0], [0, 1, 0])     # 90 (degrees)
rotation = rotation_between_vectors([1, 0, 0], [0, 0, 1])
quaternion = to_quaternion([90, 0, 0])                  # from Euler angles, a matrix, ...
```
