Metadata-Version: 2.4
Name: intuitiveness
Version: 0.2.3
Summary: Transform raw datasets into purpose-built data through descent-ascent cycles
Author-email: Arthur Sarazin <arthur.sarazin@etu-iepg.fr>
License: MIT
Project-URL: Homepage, https://github.com/ArthurSrz/intuitiveness
Project-URL: Repository, https://github.com/ArthurSrz/intuitiveness
Project-URL: Documentation, https://pypi.org/project/intuitiveness/
Project-URL: Bug Tracker, https://github.com/ArthurSrz/intuitiveness/issues
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas
Requires-Dist: networkx
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: scikit-learn
Requires-Dist: openai>=1.0.0
Requires-Dist: requests
Provides-Extra: graph
Requires-Dist: neo4j>=5.0.0; extra == "graph"
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"
Provides-Extra: app
Requires-Dist: streamlit>=1.28.0; extra == "app"
Requires-Dist: streamlit-agraph>=0.0.45; extra == "app"
Requires-Dist: matplotlib; extra == "app"
Provides-Extra: all
Requires-Dist: intuitiveness[app,graph,postgres]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: behave; extra == "dev"
Dynamic: license-file

# Intuitiveness

[![DOI](https://zenodo.org/badge/685140191.svg)](https://zenodo.org/badge/latestdoi/685140191)
[![PyPI](https://img.shields.io/pypi/v/intuitiveness)](https://pypi.org/project/intuitiveness/)

<p align="center">
  <img src="https://raw.githubusercontent.com/ArthurSrz/intuitiveness/main/frontend/public/intuitiveness_mark.svg" alt="Intuitiveness" width="120"/>
</p>

<p align="center">
  <strong>Intuitiveness as the Next Stage of Dataset Design</strong><br>
  <em>Granularity and complexity for tabular data</em>
</p>

---

## What is Intuitiveness?

**Intuitiveness** is a methodology and Python package for transforming raw, complex datasets into purpose-built data that directly answers your questions.

The method works through a **descent-ascent cycle**:
- **Descent** (L4 -> L0): Strip away complexity to find the core truth
- **Ascent** (L0 -> L3): Rebuild with YOUR intent, adding only relevant dimensions

### The 5 Levels of Abstraction

| Level | Name | Description |
|-------|------|-------------|
| **L4** | Raw Dataset | Original tabular data (one or more CSV sources) |
| **L3** | Entity Graph | Knowledge graph of relationships between entities |
| **L2** | Domain Categories | Grouped by semantic domains |
| **L1** | Feature Vector | Unified numeric representation |
| **L0** | Core Datum | Single atomic value (the truth) |

## Installation

```bash
pip install intuitiveness
```

With optional extras:

```bash
pip install intuitiveness[graph]      # Neo4j/Memgraph knowledge graphs
pip install intuitiveness[postgres]   # Durable session store on PostgreSQL
pip install intuitiveness[all]        # Everything
```

## Quick Start

```python
from intuitiveness import Level4Dataset, NavigationSession

# Load your data
import pandas as pd
tables = {"population.csv": pd.read_csv("population.csv")}

# Create a session and start navigating
l4 = Level4Dataset(tables)
session = NavigationSession(l4)
print(f"Level: {session.current_level.value}")  # 4

# Descend through abstraction levels
session.descend()  # L4 -> L3 (entity graph)
session.descend()  # L3 -> L2 (domain categories)
session.descend()  # L2 -> L1 (feature vector)
session.descend()  # L1 -> L0 (core datum)

# Ascend back with new intent
session.ascend()   # L0 -> L1 (enriched vector)
session.ascend()   # L1 -> L2 (new dimensions)
session.ascend()   # L2 -> L3 (linked graph)
```

## Features

- **Descent-Ascent Engine**: Navigate 5 abstraction levels with full lineage tracking
- **Navigation Tree**: Branch, time-travel, and compare alternative data redesigns
- **MCP Integration**: Search data.gouv.fr and World Bank Data360 via Model Context Protocol
- **AI-Assisted Transitions**: LLM suggests entity joins, domain categories, aggregations, and linkages
- **Session Persistence**: Save/restore navigation sessions (file or PostgreSQL backend)
- **Complexity Measurement**: Quantify dataset complexity across abstraction levels

## Web Application

The full interactive experience is available as a deployed web app with a React frontend and FastAPI backend. See the [repository](https://github.com/ArthurSrz/intuitiveness) for deployment instructions.

## Scientific Article

This work is documented in a peer-reviewed research paper:

> Sarazin, A., Mourey, M. (2026). *Intuitiveness as the Next Stage of Dataset Design: granularity and complexity for tabular data*. Harvard Data Science Review (under review).

- **PDF**: [scientific_article/Intuitiveness.pdf](scientific_article/Intuitiveness.pdf)

## Acknowledgments

Part of the [Dataflow](https://dataflow.hypotheses.org/) research project, funded by:
- **Datactivist**
- **UNESCO Chair in AI and Data Science for Society**

---

<p align="center">
  <sub>Designed by Arthur Sarazin & Mathis Mourey</sub>
</p>
