Metadata-Version: 2.4
Name: vcti-array-display
Version: 2.0.0
Summary: Presentation layer for NumPy structured arrays — column filtering, dtype flattening, enum mapping, array slicing, and adapters
Author: Visual Collaboration Technologies Inc.
Requires-Python: <3.15,>=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: vcti-nputils>=1.0.0
Provides-Extra: pandas
Requires-Dist: pandas>=2.0; extra == "pandas"
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: pytest-cov; extra == "test"
Requires-Dist: pandas>=2.0; extra == "test"
Provides-Extra: lint
Requires-Dist: ruff; extra == "lint"
Provides-Extra: typecheck
Requires-Dist: mypy; extra == "typecheck"
Requires-Dist: pandas-stubs; extra == "typecheck"
Dynamic: license-file

# Array Display

Presentation layer for NumPy arrays — structured, 1-D, or 2-D.

ArrayDisplay wraps a NumPy array reference (no copy) and provides five
capabilities for controlling how the data is presented.  Plain 1-D and
2-D arrays are reinterpreted as structured via zero-copy ``np.ndarray.view()``
under the hood.

1. **Column filtering** — hide columns by name, regex, or predicate
2. **Dtype flattening** — expand vector/matrix fields into scalar columns with custom component names
3. **Enum mapping** — lazily map integer ID columns to human-readable names
4. **Array slicing** — create lightweight index arrays for bounded access
5. **Adapters** — render as text table or pandas DataFrame on bounded slices

## Installation

```bash
pip install vcti-array-display>=2.0.0
```

With pandas support (optional):

```bash
pip install vcti-array-display[pandas]>=2.0.0
```

---

## 1. Column Filtering

Hide unwanted columns using any combination of exact names, regex patterns,
and callable predicates.

```python
import re
import numpy as np
from vcti.arraydisplay import ArrayDisplay, FILLER_COLUMNS, LENGTH_COLUMNS, VOID_COLUMNS

dt = np.dtype([
    ('id', 'i4'), ('f0', 'V4'), ('label', 'U20'),
    ('label_len', 'i4'), ('value', 'f8'), ('f3', 'V8'),
])
arr = np.zeros(10, dtype=dt)

# Pre-built patterns for C++ interop noise
view = ArrayDisplay(arr, exclude_columns=[FILLER_COLUMNS, LENGTH_COLUMNS])
view.view_columns  # ['id', 'label', 'value']

# Or exclude by dtype — catches all void padding regardless of name
view = ArrayDisplay(arr, exclude_columns=[VOID_COLUMNS, LENGTH_COLUMNS])

# Mix exact names, regex, and callables
view = ArrayDisplay(arr, exclude_columns=[
    FILLER_COLUMNS,                        # regex: ^f\d+$
    LENGTH_COLUMNS,                        # regex: _len$
    "debug_flag",                          # exact name
    re.compile(r"^tmp_"),                  # custom regex
    lambda name, dtype: dtype.kind == 'V', # by dtype
])
```

No magic defaults — `ArrayDisplay(arr)` shows all columns.  Pre-built patterns
`FILLER_COLUMNS`, `LENGTH_COLUMNS`, and `VOID_COLUMNS` are opt-in.
Callables receive `(name, dtype)` for filtering by type, shape, or both.

## 2. Dtype Flattening

Expand vector and matrix fields into individual scalar columns, with
optional user-defined component names.

```python
dt = np.dtype([('id', 'i4'), ('position', 'f8', (3,))])
arr = np.zeros(5, dtype=dt)

# Default numeric suffixes
view = ArrayDisplay(arr, flatten_dtype=True)
view.view_columns  # ['id', 'position_0', 'position_1', 'position_2']

# Custom component names
view = ArrayDisplay(arr, flatten_dtype=True,
                 component_names={'position': ['x', 'y', 'z']})
view.view_columns  # ['id', 'position_x', 'position_y', 'position_z']

# Component grouping is tracked for consumers (e.g., header spanning)
view.field_components
# {'position': ['position_x', 'position_y', 'position_z']}

# Rename after construction
view.set_component_names('position', ['lat', 'lon', 'alt'])
```

> **Note:** Default flattened names (e.g., `position_0`) are generated by
> `vcti-nputils` and may truncate long field names.  Use `component_names`
> to ensure predictable, readable column names regardless of the defaults.

## 3. Enum Mapping

Map integer ID columns to human-readable names.  The mapping is stored as
a recipe and resolved lazily — only when presenting a bounded slice.

```python
dt = np.dtype([('id', 'i4'), ('element_type', 'i4'), ('value', 'f8')])
arr = np.array([(1, 1, 10.5), (2, 2, 20.3), (3, 1, 15.0)], dtype=dt)
view = ArrayDisplay(arr)

view.add_name_columns({
    'element_type_name': ('element_type', {1: 'QUAD', 2: 'HEX'}),
})
view.view_columns  # ['id', 'element_type_name', 'value']

# The mapping is NOT materialized yet — it's a recipe:
view.name_columns
# {'element_type_name': ('element_type', {1: 'QUAD', 2: 'HEX'})}
```

## 4. Array Slicing

Create lightweight index arrays for bounded access.  The index array is
tiny regardless of the underlying array size.

```python
idx = view.to_slice(head=20)                       # first 20 rows
idx = view.to_slice(tail=10)                       # last 10 rows
idx = view.to_slice(head=10, tail=5)               # first 10 + last 5
idx = view.to_slice(mask=arr['value'] > 50)        # boolean filter
idx = view.to_slice(indices=np.array([0, 100, 999]))  # explicit

view.array[idx]                                    # sliced data
```

## 5. Adapters

### Text table (pure numpy — no extra dependencies)

```python
print(view.to_table(head=10, tail=5))
# id | element_type_name | value
# ---+-------------------+------
#  1 | QUAD              |  10.5
#  2 | HEX               |  20.3
# ...
# [1000 rows x 3 columns]
```

### pandas DataFrame (optional dependency)

```python
df = view.to_dataframe()                   # full array
df = view.to_dataframe(head=100)           # first 100 rows
df = view.to_dataframe(resolve_names=False)  # raw ID columns
```

Enum names are materialized using `pd.Categorical` for memory efficiency.

### Jupyter notebooks

ArrayDisplay provides `_repr_html_()` — displays a bounded row window with
enum names resolved automatically.

---

## Performance

Designed for large CAE arrays (millions of rows, GBs of data):

- **No array copy** — wraps a reference to the original numpy array
- **Lazy enum resolution** — resolved only on the displayed slice
- **Index arrays** — `to_slice()` returns kilobytes regardless of array size
- **Bounded presentation** — `to_table()` and `to_dataframe(head=N)` never touch the full array
- **`pd.Categorical`** — ~100x less memory than string columns for enum values

---

## API Summary

**Cost column:**
- **Cheap** — metadata only; no iteration over rows.
- **Moderate** — O(displayed rows × cols) or O(N) one-off; safe for bounded slices.
- **Heavy** — O(N × C) with allocation; avoid on large arrays unless you need it.

| Method | Pillar | Cost | Description |
|--------|--------|------|-------------|
| `set_view_columns(...)` | Filtering | Cheap | Configure visible columns |
| `include_view_columns(cols)` | Filtering | Cheap | Add columns to the view |
| `exclude_view_columns(cols)` | Filtering | Cheap | Remove columns from the view |
| `replace_view_columns(mapping)` | Filtering | Cheap | Rename columns in the view |
| `set_array(arr, ...)` | Flattening | Cheap | Set array; `flatten_dtype=True` uses zero-copy view |
| `set_component_names(field, names)` | Flattening | Cheap | Rename flattened components |
| `add_name_columns(mappings)` | Enum mapping | Cheap | Register lazy enum recipes |
| `to_slice(head=H, tail=T)` | Slicing | Cheap | O(H + T) index array |
| `to_slice(mask=M)` | Slicing | Moderate | O(N) boolean scan |
| `to_table(head=H, tail=T)` | Adapter | Moderate | O((H + T) × cols) |
| `to_dataframe(head=H)` | Adapter | Moderate | O(H × cols) |
| `to_dataframe()` (no bounds) | Adapter | **Heavy** | O(N × C) pandas materialization |
| `copy()` | Misc | Cheap | Shared array, config deep-copied |

See [docs/performance.md](docs/performance.md) for full complexity analysis.

> **Note on `to_dataframe()`:** Without `head`/`tail`/`indices` bounds,
> this method copies the full array into pandas — pandas cannot share
> memory across heterogeneous dtypes, so expect ~2× memory usage during
> conversion.  Use the bounded form `view.to_dataframe(head=100)` for
> inspection and analysis; reserve the unbounded form for genuine
> full-array export (parquet / CSV / etc).

## Plain 1-D and 2-D Arrays

Plain arrays are accepted too — they're reinterpreted as structured via
``np.ndarray.view()`` (zero-copy for C-contiguous input).

```python
# 1-D — single column named "value" by default
view = ArrayDisplay(np.array([1.5, 2.0, 3.5]))
view.view_columns  # ['value']

# 1-D with explicit name
view = ArrayDisplay(np.array([1.5, 2.0, 3.5]), field_names=['temperature'])

# 2-D — auto-named col_0, col_1, ...
arr = np.random.rand(1000, 3)
view = ArrayDisplay(arr)
view.view_columns  # ['col_0', 'col_1', 'col_2']

# 2-D with explicit names (the common case)
view = ArrayDisplay(arr, field_names=['x', 'y', 'z'])

# 2-D with custom prefix
view = ArrayDisplay(arr, field_name_prefix='comp')
view.view_columns  # ['comp_0', 'comp_1', 'comp_2']
```

Non-contiguous 2-D arrays raise ``ValueError`` — call
``np.ascontiguousarray()`` first if you accept the copy cost.
``ndim > 2`` is rejected explicitly; reshape first.

## Examples

See [examples/full_pipeline.py](examples/full_pipeline.py) for a complete
end-to-end script demonstrating all five pillars.

## Dependencies

- [numpy](https://numpy.org/) (>=1.24) — required
- [vcti-nputils](https://pypi.org/project/vcti-nputils/) (>=1.0.0) — required
- [pandas](https://pandas.pydata.org/) (>=2.0) — optional, for `to_dataframe()` and Jupyter display
