Metadata-Version: 2.4
Name: vcti-fieldset
Version: 1.4.0
Summary: Named array container with lazy expression evaluation, scoped metadata, and pluggable key mapping
Author: Visual Collaboration Technologies Inc.
License: Proprietary
Project-URL: Homepage, https://github.com/VCollabOrg/vcti-python-fieldset
Project-URL: Repository, https://github.com/VCollabOrg/vcti-python-fieldset
Project-URL: Documentation, https://github.com/VCollabOrg/vcti-python-fieldset#readme
Project-URL: Changelog, https://github.com/VCollabOrg/vcti-python-fieldset/blob/main/CHANGELOG.md
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: numexpr>=2.8
Requires-Dist: vcti-cache>=1.0.0
Provides-Extra: dataframe
Requires-Dist: pandas>=2.0; extra == "dataframe"
Provides-Extra: datanode
Requires-Dist: vcti-datanode>=2.0.0; extra == "datanode"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: pandas>=2.0; extra == "test"
Requires-Dist: vcti-datanode>=2.0.0; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff; extra == "lint"
Provides-Extra: typecheck
Requires-Dist: mypy; extra == "typecheck"
Requires-Dist: pandas-stubs; extra == "typecheck"
Requires-Dist: vcti-datanode>=2.0.0; extra == "typecheck"
Provides-Extra: benchmark
Requires-Dist: pytest-benchmark; extra == "benchmark"
Provides-Extra: docs
Requires-Dist: sphinx; extra == "docs"
Requires-Dist: furo; extra == "docs"
Requires-Dist: myst-parser; extra == "docs"
Requires-Dist: vcti-datanode>=2.0.0; extra == "docs"
Dynamic: license-file

# Field Set

A lightweight container for named columnar fields with lazy NumExpr expressions, scoped metadata, and pluggable field sources.

## Overview

FieldSet holds named, row-aligned columnar fields and computed expressions over
them. Each field is obtained through a *field source* — an eager in-memory numpy
array by default, or a lazy provider such as a computed array, a memory-mapped
file, or a [vcti-datanode](https://pypi.org/project/vcti-datanode/) `DataNode` —
so large or deferred data is loaded only when it is actually read. Expressions
like `"strain = stress / youngs_modulus"` are parsed and evaluated lazily via
NumExpr and cached, avoiding intermediate arrays. Row operations —
`fs.rows().where(...).sort_by(...).head(...).select()` — filter, sort, slice, and
group rows lazily into a new FieldSet, folding the chain into one index instead
of copying a table per step. Scoped metadata keeps system settings (components)
and user attributes (units, labels) in separate namespaces,
and an optional `[dataframe]` extra exports to pandas with MultiIndex columns.
The required core stays small (numpy, numexpr, vcti-cache); heavier integrations
are optional extras.

## Installation

```bash
pip install vcti-fieldset>=1.4.0
```

For pandas DataFrame support:

```bash
pip install vcti-fieldset[dataframe]>=1.4.0
```

For binding [vcti-datanode](https://pypi.org/project/vcti-datanode/) `DataNode`s as fields:

```bash
pip install vcti-fieldset[datanode]>=1.4.0
```

### In `pyproject.toml` dependencies

```toml
dependencies = [
    "vcti-fieldset>=1.4.0",
]

# or, with DataFrame support:
dependencies = [
    "vcti-fieldset[dataframe]>=1.4.0",
]
```

---

## Quick Start

```python
import numpy as np
from vcti.fieldset import FieldSet

# Create from named arrays
fs = FieldSet(
    stress=np.array([100.0, 200.0, 150.0]),
    displacement=np.array([0.1, 0.2, 0.15]),
)

# Column operations — expressions compute new columns (lazy)
fs.add_expression("strain = stress / 200000")
fs.get_values("strain")  # array([0.0005, 0.001, 0.00075])

# Row operations — filter/sort/slice rows into a new FieldSet
high = fs.rows().where("stress > 120").sort_by("stress", descending=True).select()
high["stress"]  # array([200., 150.])

# Metadata — user attributes and system settings
fs.set_property("units", "MPa", field="stress")
fs.set_property("units", "mm", field="displacement")

# Components for multi-dimensional fields
fs = FieldSet(velocity=np.array([[1, 2, 3], [4, 5, 6]]))
fs.set_components("velocity", ["x", "y", "z"])

# Generate pandas DataFrame with MultiIndex columns
df = fs.create_dataframe()
```

---

## Core API

### FieldSet

| Method | Description |
|--------|-------------|
| `add_data(*args, **kwargs)` | Add structured or named arrays (atomic) |
| `add_field(name, source)` | Add a field from a `FieldSource` (e.g. a lazy provider) |
| `get_values(name)` / `fs[name]` | Get field array or evaluate expression |
| `add_expression(expr)` | Register lazy expression (e.g., "c = a + b") |
| `rows()` | Deferred row-op builder: `.where().sort_by().head().slice().select()` / `.group_by()` |
| `materialize(name)` | Convert expression to permanent field |
| `remove_field(name)` | Remove a stored field |
| `remove_expression(name)` | Remove a registered expression |
| `list_fields()` | List all fields and expressions |
| `load_from_npz(path)` | Load arrays from .npz file |
| `set_property(name, value, field)` | Set user attribute or system setting |
| `get_property(name, field, default)` | Get metadata value |
| `set_components(field, components)` | Set component names for a field |
| `get_components(field)` | Get component names |
| `create_dataframe(fields)` | Generate pandas DataFrame (requires `[dataframe]`) |
| `shape` | (rows, total_fields) tuple |
| `name in fs` | Check if field or expression exists |
| `for name in fs` | Iterate over field and expression names |
| `copy.copy(fs)` / `copy.deepcopy(fs)` | Shallow / deep copy |

### Metadata

Pluggable key-value storage with hierarchical key resolution:

| Class | Key format | Use case |
|-------|-----------|----------|
| `DefaultKeyMapper` | `a.b.c` (separator-joined) | Simple paths |
| `ScopedKeyMapper` | `system.col.setting` / `user.col.attr` | System + user scopes |
| `ConfigKeyMapper` | `col.setting` | System-only config |

---

## Dependencies

- [numpy](https://numpy.org/) (>=1.24)
- [numexpr](https://github.com/pydata/numexpr) (>=2.8)
- [vcti-cache](https://pypi.org/project/vcti-cache/) (>=1.0.0) — ObjectCache for expression results

### Optional

- [pandas](https://pandas.pydata.org/) (>=2.0) — required for `create_dataframe()`, install via `vcti-fieldset[dataframe]`
