Metadata-Version: 2.4
Name: data-analysis-framework
Version: 2.0.0
Summary: AI-powered analysis framework for structured data files and databases - part of the unified analysis framework suite
Home-page: https://github.com/rdwj/data-analysis-framework
Author: Wes Jackson
Author-email: AI Building Blocks <wjackson@redhat.com>
License: MIT
Project-URL: Homepage, https://github.com/rdwj/data-analysis-framework
Project-URL: Repository, https://github.com/rdwj/data-analysis-framework
Project-URL: Issues, https://github.com/rdwj/data-analysis-framework/issues
Project-URL: Documentation, https://github.com/rdwj/data-analysis-framework/blob/main/README.md
Keywords: data-analysis,ai,ml,structured-data,database,excel,csv,json,semantic-search,business-intelligence
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Database
Classifier: Topic :: Office/Business
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pandas>=1.5.0
Requires-Dist: numpy>=1.21.0
Requires-Dist: openpyxl>=3.0.0
Requires-Dist: pyarrow>=8.0.0
Requires-Dist: sqlalchemy>=1.4.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: toml>=0.10.2
Requires-Dist: analysis-framework-base>=1.0.0
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.8; extra == "dev"
Requires-Dist: mypy>=0.800; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=3.0; extra == "docs"
Requires-Dist: sphinx_rtd_theme>=0.5; extra == "docs"
Provides-Extra: database
Requires-Dist: psycopg2-binary>=2.9.0; extra == "database"
Requires-Dist: pymongo>=4.0.0; extra == "database"
Provides-Extra: advanced
Requires-Dist: scikit-learn>=1.1.0; extra == "advanced"
Requires-Dist: scipy>=1.8.0; extra == "advanced"
Provides-Extra: visualization
Requires-Dist: matplotlib>=3.5.0; extra == "visualization"
Requires-Dist: seaborn>=0.11.0; extra == "visualization"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Data Analysis Framework

**Version 2.0.0** - Part of the unified analysis framework suite

## 📈 Purpose

Specialized framework for analyzing **structured data** (spreadsheets, databases, configuration files) with AI-powered pattern detection and safe agent query capabilities.

**Important:** This framework focuses on **structured data access via natural language queries**, not document chunking. For document processing, see the complementary frameworks below.

## 📦 Supported Formats

### Spreadsheets & Tables
- **Excel**: XLSX, XLS with multiple sheets
- **CSV/TSV**: Delimiter detection and parsing
- **Apache Parquet**: Columnar data analysis
- **JSON**: Nested and flat structure analysis
- **JSONL**: Line-delimited JSON streams

### Configuration Data
- **YAML**: Configuration files and data serialization
- **TOML**: Configuration file analysis
- **INI**: Legacy configuration parsing
- **Environment Files**: .env variable analysis

### Database Exports
- **SQL Dumps**: Schema and data analysis
- **SQLite**: Database file inspection
- **Database Connection**: Live data analysis

## 🤖 AI Integration Features

- **Schema Detection**: Automatic column type inference
- **Pattern Analysis**: Anomaly and trend detection
- **Data Quality Assessment**: Missing values, duplicates, outliers
- **Relationship Discovery**: Cross-table dependencies
- **Business Logic Extraction**: Rules and constraints
- **Predictive Insights**: Forecasting and recommendations

## 🚀 Quick Start

```python
from data_analysis_framework import DataAnalyzer

analyzer = DataAnalyzer()
result = analyzer.analyze("sales_data.xlsx")

print(f"Data Type: {result.document_type.type_name}")
print(f"Schema: {result.analysis.schema_info}")
print(f"Quality Score: {result.analysis.quality_metrics['overall_score']}")
print(f"AI Insights: {result.analysis.ai_insights}")
```

## 🔄 Unified Interface Support

This framework now supports the unified interface standard, providing consistent access patterns across all analysis frameworks:

```python
import data_analysis_framework as daf

# Use the unified interface
result = daf.analyze_unified("sales_data.csv")

# All access patterns work consistently
doc_type = result['document_type']        # Dict access ✓
doc_type = result.document_type           # Attribute access ✓
doc_type = result.get('document_type')    # get() method ✓
as_dict = result.to_dict()                # Full dict conversion ✓

# Works the same across all frameworks
print(f"Framework: {result.framework}")   # 'data-analysis-framework'
print(f"Type: {result.document_type}")    # 'CSV Data'
print(f"Confidence: {result.confidence}")  # Quality-based confidence
print(f"AI opportunities: {result.ai_opportunities}")
```

The unified interface ensures compatibility when switching between frameworks or using multiple frameworks together.

## 🏗️ Status

**🚧 Active Development** - Core functionality implemented, v2.0.0 adopts unified framework interfaces

## 🌐 Framework Suite

This framework is part of a unified suite of analysis frameworks, each optimized for different data types:

### Document Processing Frameworks (Chunking-Based)
These frameworks chunk documents for RAG/LLM consumption:

- **[xml-analysis-framework](https://github.com/rdwj/xml-analysis-framework)** - XML document analysis with 29+ specialized handlers (SCAP, Maven, Spring, etc.)
- **[docling-analysis-framework](https://github.com/rdwj/docling-analysis-framework)** - Office documents, PDFs, and images using IBM Docling
- **[document-analysis-framework](https://github.com/rdwj/document-analysis-framework)** - General document processing and analysis

### Data Access Framework (Query-Based)
This framework provides safe AI agent access to structured data:

- **[data-analysis-framework](https://github.com/rdwj/data-analysis-framework)** (this framework) - Structured data via natural language queries

### Shared Foundation
- **[analysis-framework-base](https://github.com/rdwj/analysis-framework-base)** - Common interfaces and models for all frameworks

### Key Differences

| Framework Type | Use Case | AI Integration | Output |
|---------------|----------|----------------|---------|
| **Document Frameworks** | "Chunk this manual for search" | RAG, semantic search | Text chunks for embeddings |
| **Data Framework** | "Show customers with revenue > $10M" | Natural language queries | Query results and insights |

### When to Use What

- **Processing documents?** Use xml/docling/document frameworks to chunk content for vector search
- **Querying databases/spreadsheets?** Use data-analysis-framework for safe AI agent access
- **Both?** Combine them! Document frameworks for knowledge + data framework for operational queries

See [CHUNKING_DECISION.md](CHUNKING_DECISION.md) for detailed explanation of this framework's query-based approach.

## 📝 What's New in v2.0.0

- ✅ Adopted `analysis-framework-base` for unified interfaces
- ✅ Inherits from `BaseAnalyzer` for consistent API across frameworks
- ✅ Implements `UnifiedAnalysisResult` for standard result format
- ✅ Added `get_supported_formats()` method for format discovery
- ✅ 100% backward compatible - all existing code works unchanged
- ℹ️ Does not implement `BaseChunker` - uses query-based paradigm instead (see [CHUNKING_DECISION.md](CHUNKING_DECISION.md))
