Metadata-Version: 2.4
Name: dva-lab
Version: 0.3.0
Summary: A cloud-synchronized Python package for Data Visualization & Analytics (DVA) and Machine Learning (ML) experiments in Jupyter Notebook and Python.
Author-email: Anand Chowdari <anandchowdari@example.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/AnandChowdari/dva-lab
Project-URL: Repository, https://github.com/AnandChowdari/dva-lab.git
Project-URL: Issues, https://github.com/AnandChowdari/dva-lab/issues
Keywords: data-visualization,analytics,machine-learning,ml,scikit-learn,clustering,regression,classification,apriori,arima,dva,dva-lab,jupyter,matplotlib,seaborn,powerbi,olap,sql,dax,education
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Visualization
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Framework :: Jupyter
Classifier: Framework :: IPython
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: viz
Requires-Dist: matplotlib>=3.5.0; extra == "viz"
Requires-Dist: numpy>=1.20.0; extra == "viz"
Requires-Dist: seaborn>=0.12.0; extra == "viz"
Requires-Dist: scipy>=1.7.0; extra == "viz"
Requires-Dist: statsmodels>=0.13.0; extra == "viz"
Provides-Extra: ml
Requires-Dist: scikit-learn>=1.0.0; extra == "ml"
Requires-Dist: pandas>=1.3.0; extra == "ml"
Requires-Dist: numpy>=1.20.0; extra == "ml"
Requires-Dist: matplotlib>=3.5.0; extra == "ml"
Requires-Dist: seaborn>=0.12.0; extra == "ml"
Requires-Dist: scipy>=1.7.0; extra == "ml"
Requires-Dist: statsmodels>=0.13.0; extra == "ml"
Provides-Extra: full
Requires-Dist: dva-lab[viz]; extra == "full"
Requires-Dist: dva-lab[ml]; extra == "full"
Requires-Dist: pandas>=1.3.0; extra == "full"
Requires-Dist: ipython>=8.0.0; extra == "full"
Provides-Extra: test
Requires-Dist: pytest>=7.0.0; extra == "test"
Requires-Dist: matplotlib>=3.5.0; extra == "test"
Requires-Dist: numpy>=1.20.0; extra == "test"
Requires-Dist: seaborn>=0.12.0; extra == "test"
Requires-Dist: scipy>=1.7.0; extra == "test"
Requires-Dist: statsmodels>=0.13.0; extra == "test"
Requires-Dist: pandas>=1.3.0; extra == "test"
Requires-Dist: scikit-learn>=1.0.0; extra == "test"
Requires-Dist: build; extra == "test"
Requires-Dist: twine; extra == "test"
Dynamic: license-file

# 📊 `dva-lab` — Data Visualization, Analytics & Machine Learning Laboratory Library

[![PyPI Version](https://img.shields.io/pypi/v/dva-lab.svg)](https://pypi.org/project/dva-lab/)
[![Python Version](https://img.shields.io/badge/python-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Jupyter Compatible](https://img.shields.io/badge/Jupyter-Notebook%20%7C%20Lab%20%7C%20Colab-orange.svg)](https://jupyter.org/)
[![Tests](https://img.shields.io/badge/tests-86%20passed-brightgreen.svg)]()

> **Cloud-synchronized, personal code library and execution engine for Data Visualization and Analytics (DVA) and Machine Learning (ML) experiments.**  
> Retrieve, study, copy, and run all **12 DVA laboratory experiments** and all **9 Machine Learning laboratory experiments** seamlessly in **Jupyter Notebook**, **JupyterLab**, **Google Colab**, or the **Terminal** with `pip install dva-lab`.

---

## 📑 Table of Contents

- [1. Overview & Architecture](#1-overview--architecture)
- [2. Installation](#2-installation)
- [3. Quick Start Workflow](#3-quick-start-workflow)
- [4. Jupyter Notebook & IPython Magics](#4-jupyter-notebook--ipython-magics)
- [5. Machine Learning (ML) Laboratory Guide](#5-machine-learning-ml-laboratory-guide)
  - [Ways to Access ML Codes in Jupyter](#ways-to-access-ml-codes-in-jupyter)
  - [Complete 9-Experiment ML Catalog](#complete-9-experiment-ml-catalog)
- [6. Command-Line Interface (CLI)](#6-command-line-interface-cli)
- [7. Complete 12-Experiment DVA Catalog](#7-complete-12-experiment-dva-catalog)
- [8. Execution Guidelines](#8-execution-guidelines)
  - [SQL / OLAP Experiments (1 & 2)](#sql--olap-experiments-1--2)
  - [Python Visualization Experiments (3 to 9)](#python-visualization-experiments-3-to-9)
  - [Power BI & DAX Experiments (10 to 12)](#power-bi--dax-experiments-10-to-12)
- [9. Testing in a Clean Virtual Environment](#9-testing-in-a-clean-virtual-environment)
- [10. Publishing Guide (TestPyPI & Production PyPI)](#10-publishing-guide-testpypi--production-pypi)
- [11. License](#11-license)

---

## 1. Overview & Architecture

`dva-lab` decouples laboratory code management and exam preparation from local machine environments:
- **Zero Local Git Required**: Retrieves experiment files and metadata from GitHub over standard HTTPS or runs directly from bundled offline caches.
- **Offline Resilient & Self-Contained**: Automatically falls back to bundled authoritative experiment definitions and datasets (`student_performance.csv`, `Salary_Data.csv`, `house_price.csv`, `movies.csv`, `SAHeart.csv`, `salary.csv`, `Mall_Customers.csv`, `store.csv`, `sales_data.csv`).
- **Direct Jupyter Cell Injection**: `%load_code <id>` or `%load_ml <id>` replaces active notebook cell contents with clean code ready to run or edit.
- **Automated Dataset Extraction**: `ml.setup()` unpacks all required CSV files into your working folder so `pd.read_csv(...)` never throws `FileNotFoundError`.
- **System Clipboard Integration**: `ml.copy(n)` copies experiment code to your OS clipboard so you can paste (`Ctrl+V`) into any portal or notebook.
- **One-Click Full Notebook Generator**: `ml.create_notebook()` generates a complete `.ipynb` file populated with all 9 ML questions.

```text
                    ┌──────────────────────────────────────────────┐
                    │               GitHub Storage                 │
                    │      github.com/AnandChowdari/dva-lab        │
                    │                                              │
                    │  codes/ (12 DVA experiments)                 │
                    │  codes/ml/ (9 ML experiments)                │
                    │  data/ (13 bundled datasets)                 │
                    └──────────────────────┬───────────────────────┘
                                           │
                                           │ HTTPS (raw.githubusercontent.com)
                                           ▼
                    ┌──────────────────────────────────────────────┐
                    │             dva-lab (Package)                │
                    │    Import: import dva_lab                    │
                    │    ML Module: import dva_lab.ml as ml        │
                    │    (with intelligent offline fallback)       │
                    └──────────────────────┬───────────────────────┘
                                           │
                                     pip install
                                           │
                 ┌──────────────────────────┼──────────────────────────┐
                 ▼                          ▼                          ▼
           Google Colab              Jupyter Notebook           Command Line (CLI)
       %pip install dva-lab        import dva_lab as dva        dva-lab ml list
       import dva_lab.ml as ml     %load_ml 1                   dva-lab ml get 1 -o q1.py
       ml.setup()                  ml.setup()                   dva-lab ml notebook
       ml.run(1)                   ml.run(1)                    dva-lab ml setup
```

---

## 2. Installation

### From PyPI (Production)
```bash
pip install dva-lab
```

In Jupyter Notebook or Google Colab:
```python
%pip install dva-lab
```

### With All Machine Learning & Visualization Dependencies:
```bash
pip install "dva-lab[full]"
```

### From Source (Local Development)
```bash
git clone https://github.com/AnandChowdari/dva-lab.git
cd dva-lab
pip install -e .
```

---

## 3. Quick Start Workflow

```python
import dva_lab as dva
import dva_lab.ml as ml

# 1. Unpack all ML lab datasets into the active working folder
ml.setup()

# 2. View all 9 ML questions in a styled table
ml.list()

# 3. View the full manual (aim, dataset, code, expected output)
ml.show(2)

# 4. Fetch syntax-highlighted code
code = ml.code(2)

# 5. Run the experiment directly (plots and metrics render inline)
ml.run(2)

# 6. Copy code to clipboard to paste (Ctrl+V) anywhere
ml.copy(2)
```

---

## 4. Jupyter Notebook & IPython Magics

When `dva_lab` is imported, it automatically registers IPython line magics:

### Machine Learning Magics

| Magic Command | Description | Example |
|---|---|---|
| `%load_ml <id>` | **Replaces cell with code** for question `<id>` | `%load_ml 1` |
| `%ml <id>` | Fast shortcut for `%load_ml` | `%ml 5` |
| `%show_ml <id>` | Displays rich Markdown documentation with Aim, Dataset & Code | `%show_ml 2` |
| `%list_ml` | Renders interactive HTML table of all 9 ML questions | `%list_ml` |
| `%run_ml <id>` | Executes ML experiment inline | `%run_ml 7` |
| `%setup_ml` | Unpacks all ML CSV datasets into active folder | `%setup_ml` |
| `%copy_ml <id>` | Copies question code to system clipboard | `%copy_ml 1` |

### DVA Lab Magics

| Magic Command | Description | Example |
|---|---|---|
| `%load_code <id>` | Injects DVA code directly into active cell (also supports `%load_code ml 1`) | `%load_code 3` |
| `%show_experiment <id>` | Displays DVA experiment Markdown documentation | `%show_experiment 1` |
| `%list_experiments` | Renders HTML table of all 12 DVA experiments | `%list_experiments` |
| `%run_experiment <id>` | Executes DVA experiment inline | `%run_experiment 3` |

---

## 5. Machine Learning (ML) Laboratory Guide

### Ways to Access ML Codes in Jupyter

Students often need different ways to access, copy, run, or export laboratory codes during practical exams and assignments. `dva-lab` provides 6 distinct methods:

#### Method 1: Instant Cell Injection via `%load_ml` *(Recommended in Jupyter)*
Type `%load_ml 1` (or `%ml 1`) into an empty notebook cell and run it. The cell is instantly replaced with the clean, runnable Python code:
```python
%load_ml 1
```

#### Method 2: Python API (`import dva_lab.ml as ml`)
```python
from dva_lab import ml

# Display table of all questions
ml.list()

# Inspect code with Markdown syntax highlighting
ml.code(1)

# Run experiment directly
ml.run(1)

# Or use instant shortcuts:
ml.q1()     # runs Question 1
ml.code1()  # returns Question 1 code
```

#### Method 3: One-Command Dataset Unpacking (`ml.setup()` or `%setup_ml`)
Avoid `FileNotFoundError: No such file or directory` forever. With a single command:
```python
ml.setup()
```
All required CSV files (`student_performance.csv`, `Salary_Data.csv`, `house_price.csv`, `movies.csv`, `SAHeart.csv`, `salary.csv`, `Mall_Customers.csv`, `store.csv`, etc.) are unpacked directly into your active folder.

#### Method 4: One-Click Complete Notebook Generator (`ml.create_notebook()`)
Generate an entire, pre-populated Jupyter Notebook containing all 9 ML questions formatted in separate cells:
```python
ml.create_notebook("ML_Lab_Solutions.ipynb")
```
Open `ML_Lab_Solutions.ipynb` in Jupyter Notebook, JupyterLab, or VS Code and run any cell immediately!

#### Method 5: Instant Clipboard Copy (`ml.copy(n)` or `%copy_ml n`)
Copy the clean code directly to your operating system clipboard:
```python
ml.copy(1)
```
Then simply press `Ctrl + V` in any cell, text editor, or test portal.

#### Method 6: Command Line Interface (`dva-lab ml ...`)
Work from the terminal without opening Python:
```bash
dva-lab ml list
dva-lab ml get 1 -o question1.py
dva-lab ml get all
dva-lab ml setup
dva-lab ml notebook
```

---

### Complete 9-Experiment ML Catalog

| # | Question Title | Category | Dataset | Core Algorithm / Focus |
|:---:|---|---|---|---|
| **Q1** | Student Performance Data Preparation & Grade Analysis | `Data Preparation & EDA` | `student_performance.csv` | Crosstab gender analysis, mark totals, letter grades (A+ to F), and scholarship eligibility |
| **Q2** | Salary Prediction using Simple Linear Regression | `Supervised Learning` | `Salary_Data.csv` | OLS Regression, slope/intercept, 5-yr projection, R², MAE, MSE, RMSE, regression line plot |
| **Q3** | House Price Prediction & Residual Diagnostic Analysis | `Supervised Learning` | `house_price.csv` | Multiple linear regression, sample house pricing, Residuals vs Fitted, Normal P-P plot, Histogram |
| **Q4** | Movie Genre Association Rule Mining with Apriori | `Unsupervised Learning` | `movies.csv` | Transaction extraction, frequent 1/2-itemsets (Support ≥ 0.02), Strong Rules (Conf ≥ 0.30, Lift > 1), Bar plot |
| **Q5** | Heart Disease Prediction using Decision Tree Classifier | `Supervised Learning` | `SAHeart.csv` | DecisionTreeClassifier (max_depth=3), Classification Report, Confusion Matrix heatmap, ROC-AUC curve |
| **Q6** | Batch Gradient Descent Optimization from Scratch | `Optimization` | `salary.csv` | Feature normalization, mathematical partial derivatives (dw, db), 1000 iterations, Cost curve plot |
| **Q7** | Customer Segmentation using K-Means Clustering (K=2) | `Unsupervised Learning` | `Mall_Customers.csv` | StandardScaler normalization, KMeans (K=2), 2D Income vs Spending scatter plot, cluster records |
| **Q8** | Customer Segmentation using Agglomerative Clustering | `Unsupervised Learning` | `Mall_Customers.csv` | Ward linkage hierarchical tree, truncated dendrogram plot, AgglomerativeClustering, cluster scatter |
| **Q9** | Time-Series Demand Forecasting using ARIMA | `Time-Series Analysis` | `store.csv` | Datetime index, ARIMA(1,1,1) model, 10-period future demand forecast, AIC/BIC parameters, curve plot |

---

## 6. Command-Line Interface (CLI)

`dva-lab` provides a command-line interface for DVA and ML tasks:

```bash
# === DVA Commands ===
dva-lab list                        # List all 12 DVA experiments
dva-lab show 3                      # Show manual for DVA experiment 3
dva-lab get 3 -o exp3.py            # Download DVA experiment 3 code
dva-lab run 3                       # Run DVA experiment 3
dva-lab search histogram            # Search experiments

# === Machine Learning Commands ===
dva-lab ml list                     # List all 9 ML experiments
dva-lab ml show 1                   # Display manual for ML Question 1
dva-lab ml get 1 -o q1.py           # Save ML Question 1 code to file
dva-lab ml get all                  # Export all 9 ML question files (question1.py - question9.py)
dva-lab ml run 1                    # Execute ML Question 1
dva-lab ml setup                    # Copy all ML CSV datasets to current folder
dva-lab ml copy 1                   # Copy ML Question 1 code to clipboard
dva-lab ml notebook                 # Generate ML_Lab_Solutions.ipynb notebook

# === Package Configuration ===
dva-lab info                        # Display library and configuration details
```

---

## 7. Complete 12-Experiment DVA Catalog

| # | Title | Environment / Type | Methodology & Dependencies | Focus |
|:---:|---|---|---|---|
| **1** | OLAP Operations Using Materialized Views | `PostgreSQL 12+` (SQL) | PostgreSQL, Materialized Views, CUBE | Roll Up, Slice, Dice, Pivot |
| **2** | OLAP Operations Based on Star Schema | `PostgreSQL 12+` (SQL) | Fact & Dimension tables, CUBE | Star schema joins, Drill Down |
| **3** | Vertical and Horizontal Bar Charts | `Python 3.10+` | Matplotlib, NumPy | `plt.bar`, `plt.barh`, categorical comparison |
| **4** | Grouped and Stacked Bar Charts | `Python 3.10+` | Matplotlib, NumPy | Clustered offsets, stacked bottoms |
| **5** | Single & Multiple Variable Histograms | `Python 3.10+` | Matplotlib, NumPy | Frequency bins, transparency overlays |
| **6** | Single & Multiple Variable Density Plots | `Python 3.10+` | Seaborn, Matplotlib | KDE probability curves (`sns.kdeplot`) |
| **7** | Empirical Cumulative Distribution Function | `Python 3.10+` | Statsmodels, Matplotlib | Non-parametric ECDF step function |
| **8** | Q-Q Plot to Compare Two Datasets | `Python 3.10+` | SciPy, Matplotlib | Normal probability quantiles (`probplot`) |
| **9** | Scatter Plot Using Sample Data | `Python 3.10+` | Matplotlib, NumPy | Bivariate correlation, alpha blending |
| **10** | Power BI Sales Data Visualization | `Power BI Desktop` | Power BI Desktop, `sales_data.csv` | Clustered column, Pie, Line, Slicers |
| **11** | Power BI Calculations & DAX Measures | `Power BI Desktop` | DAX, Power BI Desktop | `Total Cost`, `Profit`, `CALCULATE`, `ALL` |
| **12** | Power BI Conditional & Logical Analysis | `Power BI Desktop` | DAX, Power BI Desktop | `IF`, `AND`, `OR`, Dynamic tiers |

---

## 8. Execution Guidelines

### SQL / OLAP Experiments (1 & 2)
Experiments 1 and 2 define complete PostgreSQL DDL and OLAP queries. Calling `dva_lab.run_experiment(1)` displays the verified SQL script and instructions. If you have an active database connection (e.g. `psycopg2` or `SQLAlchemy`), pass it directly: `dva_lab.run_experiment(1, connection=conn)`.

### Python Visualization Experiments (3 to 9)
All 7 Python plotting experiments run directly in Jupyter or Python scripts with inline rendering.

### Power BI & DAX Experiments (10 to 12)
Power BI Desktop is a Windows visual application. `dva-lab` provides step-by-step instructions, exact DAX formulas, bundled CSV data (`dva_lab.load_sales_data()`), and a pure Python simulation verifying all DAX calculations.

---

## 9. Testing in a Clean Virtual Environment

### Step 1: Run the Complete Test Suite
```bash
pytest -v
```
*(All 86 tests pass across DVA, ML, datasets, CLI, Jupyter magics, and remote retrieval).*

### Step 2: Build Wheel and Source Distribution
```bash
python -m build
```

---

## 10. Publishing Guide (TestPyPI & Production PyPI)

### Prerequisites
```bash
pip install twine
```

### Staging: Publish to TestPyPI
```bash
python -m twine upload --repository testpypi dist/*
```

### Production: Publish to PyPI
```bash
python -m twine upload dist/*
```

---

## 11. License

This project is licensed under the [MIT License](LICENSE).
