Metadata-Version: 2.4
Name: fid-tools
Version: 0.1.0
Summary: Functional Information Decomposition - analyze how inputs contribute to predicting outputs
Author-email: Clifford Bohm <cliff.bohm@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Clifford Bohm
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Keywords: information-theory,mutual-information,synergy,information-decomposition,entropy
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20
Requires-Dist: scipy>=1.7
Requires-Dist: matplotlib>=3.4
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Dynamic: license-file

# Functional Information Decomposition (FID) Library - User Manual

## Table of Contents
1. [Overview](#overview)
2. [Installation & Setup](#installation--setup)
3. [Core Concepts](#core-concepts)
4. [Quick Start](#quick-start)
5. [Working with TPMs](#working-with-tpms)
6. [FID Analysis](#fid-analysis)
7. [Handling Missing Data](#handling-missing-data)
8. [Visualization](#visualization)
9. [API Reference](#api-reference)
10. [Examples](#examples)
11. [Limitations & Warnings](#limitations--warnings)

---

## Overview

**FID** (Functional Information Decomposition) is a library for analyzing how multiple input variables contribute to predicting an output, using information-theoretic concepts.

### What It Does
- **Quantifies independent information**: How mucdh each input alone predicts the output
- **Measures synergy**: Information only available from joint inputs
- **Computes solo synergy**: How much each input contributes when combined with all others
- **Handles incomplete data**: Intelligently manages missing input patterns through probabilistic sampling

### Key Features
- Build TPMs (Transition Probability Matrices) from raw data
- Compute FID decompositions on complete or incomplete data
- Sample completions using multiple strategies (Dirichlet, grid, deterministic, edges)
- Visualize results with scatter plots, clouds, and uncertainty bounds
- Flexible handling of categorical/discrete variables with arbitrary names and symbols

---

## Installation & Setup

### Prerequisites
- Python 3.7+
- NumPy, SciPy, Matplotlib

### Install Dependencies
```bash
pip install -r requirements.txt
```

### Import the Library
```python
from fid import TPM, entropy, display_fid, plot_fid_clouds
```

---

## Core Concepts

### TPM (Transition Probability Matrix)
A matrix representing the relationship between inputs and an output:
- **Rows**: All possible combinations of input patterns (exhaustive enumeration)
- **Columns**: Output symbols/states
- **Values**: Probabilities P(Y|X) for conditional, or P(X,Y) for joint

### RN_TPM (Row-Normalized TPM)
Each row sums to 1.0, representing **conditional probability P(Y|X_i)**
- Answers: "Given this input pattern, what's the probability of each output?"

### MN_TPM (Matrix-Normalized TPM)
The entire matrix sums to 1.0, representing **joint probability P(X,Y)**
- Answers: "What's the probability of seeing this input-output pair?"
- Computed as: `MN_TPM = RN_TPM / n_rows` (assuming uniform input distribution)

### Information-Theoretic Metrics

**Entropy H(Y)**
- Measure of output uncertainty
- Higher entropy = more uniform output distribution

**Conditional Entropy H(Y|X)**
- Average uncertainty in output given the inputs
- Lower is better (inputs explain more)

**Mutual Information I(X;Y)**
- How much knowing X reduces uncertainty about Y
- Equals: `H(Y) - H(Y|X)`

**Independent Information**
- How much a single input X_i predicts Y by itself

**Synergy**
- Information only available from the joint state of all inputs
- Equals: `Total MI - Sum of Independents`

**Solo Synergy**
- How much input X_i contributes when combined with all other inputs
- Accounts for interactions and non-linear effects

---

## Quick Start

### Example 1: Build and Analyze a Simple TPM

```python
import numpy as np
from fid import TPM, display_fid

# Create data: one input perfectly predicts output, the other doesn't
input_A = ['a', 'a', 'b', 'b', 'a', 'b']
input_B = [0, 1, 0, 1, 0, 1]
output = ['X', 'Y', 'X', 'Y', 'X', 'Y']

# Build TPM from data
tpm = TPM.from_data(
    data=[input_A, input_B, output],
    input_names=['A', 'B'],
    output_name='Z'
)

# Display the TPM
tpm.display()

# Compute FID
fid_result = tpm.fid()

# Display results
display_fid(fid_result)
```

**What to expect:**
- Input B perfectly predicts output (B=0 always → X, B=1 always → Y)
- Input A does not predict output
- So: B has high independent information (~1.0 bits), A has none
- Synergy should be ~0.0 (no joint effects needed)

### Example 2: Handle Incomplete Data

```python
# Data with a missing input pattern (a,a)
input_1 = ['a', 'a', 'b', 'b']
input_2 = [0, 1, 0, 1]
output = ['X', 'Y', 'X', 'Y']

tpm = TPM.from_data(
    data=[input_1, input_2, output],
    input_names=['A', 'B'],
    output_name='Z'
)

# Check missing rows
print(f"Missing patterns: {len(tpm.missing_rows)}")

# Set a prior for missing pattern (a,0)
tpm.set_row(('a', 0), {'X': 0.8, 'Y': 0.2})

# Now compute FID on neutral (uniform) completion
result = tpm.neutral_completion_fid()
display_fid(result)
```

---

## Working with TPMs

### Creating a TPM

```python
tpm = TPM.from_data(
    data=[input_1_seq, input_2_seq, output_seq],
    input_names=['Input1', 'Input2'],
    output_name='Output'
)
```
- Automatically builds all possible patterns
- Counts observed transitions to compute P(Y|X)
- Marks patterns never seen as "missing"

### Inspecting a TPM

```python
# Display the full TPM
tpm.display()

# Get input names
inputs = tpm.get_input_names()
print(inputs)  # ['A', 'B']

# Get symbols for each input
input_symbols = tpm.get_input_symbols()
print(input_symbols)  # {'A': ['a', 'b'], 'B': [0, 1]}

# Get output symbols
output_symbols = tpm.get_output_symbols()
print(output_symbols)  # ['X', 'Y']

# Check for missing rows
print(f"Missing: {len(tpm.missing_rows)} / {tpm.n_rows}")
```

### Modifying a TPM

#### Add/Extend Output Symbols
```python
# Add new output symbol
tpm.extend_symbols(['Z'])
tpm.display()  # New column automatically filled with 0.0 for complete rows
```

#### Set a Row (Prior)
Set a specific input pattern to a known probability distribution:

```python
# Set pattern (a, 0) to probability {'X': 0.8, 'Y': 0.2}
tpm.set_row(('a', 0), {'X': 0.8, 'Y': 0.2})

# Internally, probabilities are normalized
# After refresh, RN_TPM[pattern_idx] = [0.8, 0.2]
```

#### Set a Row (Restriction)
Restrict which outputs are allowed for a pattern, but keep it incomplete:

```python
# Pattern (a, 0) can only produce 'X' (but exact probability unknown)
tpm.set_row(('a', 0), ['X'])

# Row stays in missing_rows; FID requires all rows complete
# Useful for constraints in completion sampling
```

#### Clear a Row
```python
# Reset pattern (a, 0) to unknown (all NaN)
tpm.clear_row(('a', 0))
```

#### Refresh After Changes
```python
# Automatically called by set_row(), clear_row(), extend_symbols()
# But can be called manually:
tpm.refresh_TPMs()

# This recomputes:
# - RN_TPM normalization (each row sums to 1)
# - MN_TPM from RN_TPM (joint distribution)
```

---

## FID Analysis

### Single-Point FID

For a **complete TPM** (no missing rows), compute exact FID:

```python
result = tpm.fid()

# Result keys:
# - 'total_information' (float): I(X;Y)
# - 'independents' (dict): {label -> float} for each input
# - 'synergy' (float): residual joint information
# - 'solo_synergies' (dict): {label -> float} for each input
# - 'output_entropy' (float): H(Y)
# - 'conditional_entropy' (float): H(Y|X)
# - 'input_labels' (list): formatted variable names

print(f"Total Info: {result['total_information']:.4f} bits")
print(f"Synergy: {result['synergy']:.4f} bits")
for label, val in result['independents'].items():
    print(f"  {label}: {val:.4f} bits")
```

### Neutral Completion FID

For **incomplete TPMs**, fill missing rows uniformly and compute FID:

```python
result = tpm.neutral_completion_fid()

# Returns same structure as fid()
# Missing rows filled with P(Y) = uniform over allowed outputs
```

---

## Handling Missing Data

The library provides multiple strategies for sampling **completions** of missing rows.

### Motivation
When data is sparse, not all input patterns are observed. FID requires a complete TPM. Different completion strategies yield different (but related) FID results. Sampling many completions gives uncertainty bounds.

### Strategy 1: Dirichlet Cloud (Recommended)

Sample missing rows from a **Dirichlet distribution** (soft, probabilistic):

```python
cloud = tpm.fid_cloud(
    method='dirichlet',
    n_samples=10000,
    alpha=1.0
)

# cloud keys:
# - 'total_information' (array of shape (n_samples,))
# - 'independents' (dict of arrays)
# - 'synergy' (array)
# - 'solo_synergies' (dict of arrays)
# - 'output_entropy', 'conditional_entropy' (arrays)
# - 'n_completions' (int): 10000

# Get summary statistics
import numpy as np
mean_synergy = np.mean(cloud['synergy'])
std_synergy = np.std(cloud['synergy'])
print(f"Synergy: {mean_synergy:.4f} ± {std_synergy:.4f}")
```

**Alpha parameter:**
- `alpha = 1.0`: Uniform (neutral, default)
- `alpha < 1.0`: Sparse distributions (peaked)
- `alpha > 1.0`: Uniform distributions

### Strategy 2: Deterministic Cloud

Sample missing rows from **discrete delta distributions** (hard one-hot):

```python
cloud = tpm.fid_deterministic(n_samples=5000)

# Each missing row is assigned to a single output with 100% probability
# Result: all outputs represented equally
```

**Use when:**
- You want extreme/boundary cases
- Grid is computationally feasible

### Strategy 3: Grid-Based Cloud

Sample missing rows from a **structured grid** (balanced, parametric):

```python
cloud = tpm.fid_grid(
    steps=5,      # discretization level
    n_samples=10000
)

# Automatically subsamples if grid is larger than n_samples
# Deterministic and reproducible
```

**Use when:**
- You want systematic coverage
- Comparability across runs

### Strategy 4: Edges Cloud

Sample missing rows along **interpolated paths** between deterministic completions:

```python
cloud = tpm.fid_edges(n_samples=5000)

# Samples midpoints between pairs of deterministic completions
# Explores the "surface" of the completion space
```

**Use when:**
- You want smooth transitions
- Interested in boundaries

### Comparing Completions

```python
# Compute multiple clouds
cloud_dirichlet = tpm.fid_cloud(n_samples=5000, alpha=1.0)
cloud_deterministic = tpm.fid_deterministic(n_samples=5000)

# Display with bounds
from fid import display_fid_with_bounds
neutral = tpm.neutral_completion_fid()
display_fid_with_bounds(neutral, [cloud_dirichlet, cloud_deterministic])
```

---

## Visualization

### Simple Scatter Plot

```python
from fid import plot_fid_clouds

neutral = tpm.neutral_completion_fid()
cloud = tpm.fid_cloud(n_samples=10000)

plot_fid_clouds(
    base_fid=neutral,
    fid_clouds=[cloud],
    names=['Dirichlet'],
    colors=['blue'],
    alphas=[0.3],
    x_metric='synergy',
    y_metric='total_information',
    filename='plot1.png'
)
```

### 3D Scatter Plot

```python
from fid import plot_fid_clouds_3d

plot_fid_clouds_3d(
    base_fid=neutral,
    fid_clouds=[cloud],
    names=['Dirichlet'],
    colors=['blue'],
    alphas=[0.3],
    x_metric='synergy',
    y_metric='total_information',
    z_metric='output_entropy',
    filename='plot_3d.png'
)
```

### Relationship Grid (Advanced)

```python
from fid import plot_fid_relationships

plot_fid_relationships(
    base_fid=neutral,
    fid_clouds=[cloud],
    names=['Dirichlet'],
    colors=['blue'],
    alphas=[0.3],
    y_axis='total_information',
    share_axis='common',  # all plots share axes
    filename='relationships.png'
)
```

This creates a matrix of scatter plots showing:
- **Top row**: Synergy and Total Information vs. y-axis
- **Remaining rows** (one per input):
  - Independent vs. y-axis
  - Solo Synergy vs. y-axis
  - Loss (Independent + Solo Synergy) vs. y-axis
  - Solo Synergy vs. Independent (2D relationship)

### Retrieving Metrics from Clouds

```python
from fid import get_metric

# Get a specific metric from a cloud
synergy_values = get_metric(cloud, 'synergy')
x0_independent = get_metric(cloud, 'X0_independent')
x1_solo_synergy = get_metric(cloud, 'X1_solo_synergy')
```

---

## API Reference

### TPM Class

#### Constructors

**`TPM.from_data(data, input_names=None, output_name='Y')`**
- Build TPM from sequences
- **Args:**
  - `data` (list): [input_1_seq, input_2_seq, ..., output_seq]
  - `input_names` (list, optional): Names for inputs
  - `output_name` (str): Name for output
- **Returns:** TPM instance

#### Display Methods

**`tpm.display()`**
- Print formatted TPM with all patterns and probabilities

**`tpm.get_input_names()`**
- **Returns:** List of input variable names

**`tpm.get_input_symbols()`**
- **Returns:** Dict mapping input names to symbol lists

**`tpm.get_output_symbols()`**
- **Returns:** List of output symbols

#### Modification Methods

**`tpm.set_row(pattern, symbols)`**
- Set prior or restriction for a row
- **Args:**
  - `pattern` (tuple): Input pattern using visible symbols
  - `symbols` (list or dict):
    - List: restriction to allowed outputs
    - Dict: prior probabilities {symbol -> weight}

**`tpm.clear_row(pattern)`**
- Reset row to missing (all NaN)

**`tpm.extend_symbols(new_symbols)`**
- Add new output symbols to TPM

**`tpm.refresh_TPMs()`**
- Recompute RN_TPM and MN_TPM after manual changes

#### Analysis Methods

**`tpm.fid()`**
- Compute FID for complete TPM
- **Returns:** FID result dict
- **Raises:** ValueError if missing rows exist

**`tpm.neutral_completion_fid()`**
- Compute FID with uniform completion of missing rows
- **Returns:** FID result dict

**`tpm.fid_cloud(method='dirichlet', n_samples=50000, alpha=1.0)`**
- Sample completions from Dirichlet distribution
- **Returns:** Cloud dict with arrays

**`tpm.fid_deterministic(n_samples)`**
- Sample deterministic completions
- **Returns:** Cloud dict with arrays

**`tpm.fid_grid(steps=None, n_samples=...)`**
- Sample from grid of probability compositions
- **Returns:** Cloud dict with arrays

**`tpm.fid_edges(n_samples)`**
- Sample interpolations between deterministic completions
- **Returns:** Cloud dict with arrays

#### Utility Methods

**`tpm.marginalize(indices_to_keep)`**
- Compute marginal distributions for subset of inputs
- **Args:** List of input indices
- **Returns:** Dict mapping partial patterns to output distributions

**`tpm.output_entropy()`**
- **Returns:** H(Y) as float

**`tpm.conditional_entropy()`**
- **Returns:** H(Y|X) as float

**`tpm.mutual_information()`**
- **Returns:** I(X;Y) as float

---

### Module-Level Functions

**`entropy(probs)`**
- Compute entropy of probability distribution (vectorized)
- **Args:** NumPy array of shape (..., n)
- **Returns:** Array of shape (...)

**`display_fid(fid_result)`**
- Print formatted FID report

**`display_fid_with_bounds(base_fid, clouds)`**
- Print FID with uncertainty bounds from multiple completions

**`plot_fid_clouds(base_fid, fid_clouds, names, colors, alphas, x_metric, y_metric, ...)`**
- 2D scatter plot with base assumption overlay

**`plot_fid_clouds_3d(base_fid, fid_clouds, names, colors, alphas, x_metric, y_metric, z_metric, ...)`**
- 3D scatter plot

**`plot_fid_relationships(base_fid, fid_clouds, names, colors, alphas, y_axis, ...)`**
- Relationship grid (matrix of scatter plots)

**`get_metric(cloud, metric_name)`**
- Extract a metric from a cloud dict
- Handles special naming: `X0_independent`, `X1_solo_synergy`, etc.

---

## Examples

### Example 1: Boolean Function Analysis

```python
from fid import TPM, display_fid

# XOR function: output = X1 XOR X2
x1 = [0, 0, 1, 1, 0, 0, 1, 1]
x2 = [0, 1, 0, 1, 0, 1, 0, 1]
y = [0, 1, 1, 0, 0, 1, 1, 0]

tpm = TPM.from_data([x1, x2, y], input_names=['X1', 'X2'], output_name='Y')
result = tpm.fid()
display_fid(result)

# Expected: High synergy, low independents (neither input alone predicts output)
```

### Example 2: Redundant Inputs

```python
# Two identical inputs predicting output (all 4 patterns observed)
x1 = [0, 0, 1, 1, 0, 0, 1, 1]
x2 = [0, 0, 1, 1, 0, 0, 1, 1]  # Same as x1
y = [0, 0, 1, 1, 0, 0, 1, 1]   # Same as x1 and x2

tpm = TPM.from_data([x1, x2, y], input_names=['X1', 'X2'], output_name='Y')
result = tpm.fid()

# Expected: Each input has ~1.0 bits independent info, very low synergy
# (both inputs are redundant—each alone fully predicts output)
```

### Example 3: Sparse Data with Completion Sampling

```python
# Only 2 out of 4 patterns observed
x1 = [0, 0, 1, 1]
x2 = [0, 1, 0, 1]
y = [0, 1, 1, 0]  # Missing patterns: (0,0) and (1,1)

tpm = TPM.from_data([x1, x2, y], input_names=['X1', 'X2'], output_name='Y')

# Get baseline
neutral = tpm.neutral_completion_fid()
print(f"Neutral synergy: {neutral['synergy']:.4f}")

# Sample completions
cloud = tpm.fid_cloud(n_samples=5000)

# Analyze uncertainty
import numpy as np
mean = np.mean(cloud['synergy'])
std = np.std(cloud['synergy'])
print(f"Cloud synergy: {mean:.4f} ± {std:.4f}")
```

### Example 4: Multi-Input System with Visualization

```python
from fid import TPM, plot_fid_relationships
import numpy as np

# 3 inputs, 2 outputs
x1 = np.random.choice(['a', 'b'], 100)
x2 = np.random.choice([0, 1], 100)
x3 = np.random.choice(['low', 'high'], 100)
y = np.random.choice(['yes', 'no'], 100)

tpm = TPM.from_data(
    [x1, x2, x3, y],
    input_names=['Feature1', 'Feature2', 'Feature3'],
    output_name='Outcome'
)

# Handle missing rows with prior knowledge
for (f1, f2, f3) in [('a', 0, 'low'), ('b', 1, 'high')]:
    if len(tpm.missing_rows) > 0:
        tpm.set_row((f1, f2, f3), {'yes': 0.7, 'no': 0.3})

neutral = tpm.neutral_completion_fid()
cloud = tpm.fid_cloud(n_samples=5000, alpha=2.0)

# Visualize relationships
plot_fid_relationships(
    base_fid=neutral,
    fid_clouds=[cloud],
    names=['Completion Cloud'],
    colors=['steelblue'],
    alphas=[0.2],
    y_axis='total_information',
    share_axis='common'
)
```

---

## Limitations & Warnings

### Combinatorial Explosion

⚠️ **Critical Limitation:** The library enumerates ALL possible input patterns upfront.

- 3 inputs × 3 values each = 27 patterns
- 5 inputs × 3 values each = 243 patterns
- 10 inputs × 3 values each = 59,049 patterns
- 20 inputs × 3 values each = 3.5 billion patterns ❌

**Recommendation:** For sparse data (>10 inputs or high cardinality), this library will struggle. Consider:
- Reducing dimensionality beforehand
- Using a sparse-friendly alternative for very high dimensions
- Accepting that dense enumeration is necessary for exact FID calculations

### Missing Rows and Completeness

⚠️ **FID requires all rows to be complete.** Options:
1. `fid()` - Works only if no missing rows
2. `neutral_completion_fid()` - Fills missing rows uniformly
3. `fid_cloud()` etc. - Samples many completions and returns distributions

**Choose based on your problem:**
- All data observed? → Use `fid()`
- Sparse data with no prior? → Use `fid_cloud()` + `neutral_completion_fid()`
- Domain knowledge about missing patterns? → Use `set_row()` + `neutral_completion_fid()`

### Normalization Assumptions

The library assumes **uniform input distribution** P(X) = 1/n_rows when computing MN_TPM.

If your observed data has non-uniform pattern frequencies, consider:
- Resampling or weighting your data upfront
- Using `from_data()` which properly counts observed patterns

### Numerical Precision

- Small probabilities (< 1e-12) are clipped to avoid log(0)
- Very sparse distributions may have numerical issues
- Use `np.clip()` if you see NaN/inf in results

### Reproducibility

- Sampling methods (`fid_cloud`, etc.) use random number generation
- Set `np.random.seed()` before calls if reproducibility is needed

```python
np.random.seed(42)
cloud = tpm.fid_cloud(n_samples=5000)
```

### Performance Considerations

- **Bottleneck:** Computing marginalizations for each FID sample
- **For large n_samples (>100k):** Memory usage grows ~10GB+
- **Recommendation:** Batch clouds or reduce n_samples if memory-constrained

---

## FAQ

**Q: What's the difference between `fid()` and `neutral_completion_fid()`?**

A: `fid()` requires a complete TPM (raises error if missing rows). `neutral_completion_fid()` fills missing rows with uniform distributions and returns a single FID result.

**Q: Can I compare FID results across different datasets?**

A: Partially. FID values are relative to the datasets' entropies. Comparing synergy or independents directly is only valid if output entropies are similar.

**Q: How do I know if my completion is "good"?**

A: Sample many completions and check the bounds. Wide bounds = uncertain. Narrow bounds = robust. Compare across strategies (Dirichlet, deterministic, grid, edges) for consistency.

**Q: What if I have ordered/continuous variables?**

A: Discretize them first (bin into categories), then use the library.

**Q: Can I export the TPM to CSV?**

A: Use `tpm.RN_TPM`, `tpm.input_info`, and `tpm.output_info` to extract the data for export.

**Q: How do I interpret "synergy" intuitively?**

A: Synergy is the "magic" that only happens when inputs are jointly considered. Example: XOR (each input alone is useless; together they perfectly predict output).

---

## Version & Author Notes

- **Version:** 2.0 (Retooled)
- **Status:** Active development
- **Key Recent Fix:** NaN handling in `refresh_TPMs()` — missing rows no longer contaminate MN_TPM

For issues or contributions, refer to the repository documentation.

---

**Happy Information Decomposing! 📊**
