Metadata-Version: 2.4
Name: bigdata-practical-toolkit
Version: 1.0.1
Summary: A CLI toolkit for Big Data Analytics Laboratory Practicals (12 Experiments)
Author: Big Data Lab
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: pandas>=1.5.0
Requires-Dist: numpy>=1.23.0
Requires-Dist: matplotlib>=3.6.0
Requires-Dist: seaborn>=0.12.0
Requires-Dist: scikit-learn>=1.1.0
Requires-Dist: openpyxl>=3.0.0
Requires-Dist: colorama>=0.4.6
Requires-Dist: scipy>=1.9.0
Requires-Dist: pymongo>=4.0.0
Requires-Dist: mongomock>=4.0.0
Dynamic: author
Dynamic: requires-python

# 📊 Big Data Analytics Practical Toolkit

> A professional, installable Python CLI package for **Big Data Analytics Laboratory Practicals**.  
> After installation, simply type `bigdata` in the terminal to launch the application.

---

## 🚀 Features

- ✅ **12 Complete Lab Experiments** (Descriptive Stats → ML → Profiling)
- ✅ **Native Windows File Picker** – no manual path typing
- ✅ **Coloured Terminal UI** with Colorama
- ✅ **Auto-saves results + plots** to `Output/` folder
- ✅ **Supports CSV, Excel, TXT, JSON**
- ✅ **Beginner-friendly** comments throughout
- ✅ **Extensible** – add Experiment 13 in one file + one menu line

---

## 📋 Experiment List

| # | Experiment | Key Techniques |
|---|-----------|----------------|
| 1 | Descriptive Statistics & EDA | mean, std, skewness, kurtosis, histograms |
| 2 | Data Cleaning & Preprocessing | imputation, encoding, normalisation |
| 3 | Data Visualization | bar, pie, line charts |
| 4 | Correlation Analysis & Heatmap | Pearson correlation, pair plot |
| 5 | Linear Regression | MAE, RMSE, R², residuals |
| 6 | Classification – Decision Tree & KNN | accuracy, confusion matrix |
| 7 | Clustering – K-Means | elbow method, PCA 2D plot |
| 8 | Dimensionality Reduction – PCA | scree plot, loadings heatmap |
| 9 | Frequency Distribution & Histograms | KDE, cumulative frequency |
| 10 | Outlier Detection (IQR & Z-Score) | box plots, z-score chart |
| 11 | Time Series Analysis | rolling mean, trend, forecast |
| 12 | Dataset Profiling & Summary Report | full data profile |

---

## 🛠️ Installation

### Step 1 – Clone or Download

```bash
# If you have the folder, navigate to it:
cd bigdata-practical-toolkit
```

### Step 2 – (Recommended) Create a Virtual Environment

```bash
python -m venv venv
venv\Scripts\activate          # Windows
```

### Step 3 – Install the Package

```bash
pip install .
```

This automatically installs all dependencies:
`pandas`, `numpy`, `matplotlib`, `seaborn`, `scikit-learn`, `openpyxl`, `colorama`, `scipy`

---

## ▶️ Running the Application

After installation:

```bash
bigdata
```

The application will:
1. Show the main menu
2. Ask you to pick an experiment
3. Open a **native file picker** for your dataset
4. Run the experiment automatically
5. Save results and plots to an `Output/` folder
6. Return to the main menu

---

## 📁 Project Structure

```
bigdata-practical-toolkit/
│
├── bigdata_practical_toolkit/       # Main package
│   ├── __init__.py
│   ├── cli.py                       # Entry point (bigdata command)
│   ├── menu.py                      # Coloured menu display
│   │
│   ├── experiments/                 # 12 experiment modules
│   │   ├── exp1.py  – Descriptive Statistics
│   │   ├── exp2.py  – Data Cleaning
│   │   ├── exp3.py  – Visualization
│   │   ├── exp4.py  – Correlation
│   │   ├── exp5.py  – Linear Regression
│   │   ├── exp6.py  – Classification
│   │   ├── exp7.py  – K-Means Clustering
│   │   ├── exp8.py  – PCA
│   │   ├── exp9.py  – Frequency Distribution
│   │   ├── exp10.py – Outlier Detection
│   │   ├── exp11.py – Time Series
│   │   └── exp12.py – Dataset Profiling
│   │
│   └── utils/                       # Shared utilities
│       ├── loader.py                # File picker + DataFrame loader
│       ├── validator.py             # Dataset validation
│       ├── plotting.py              # Common plot helpers
│       └── save_output.py           # Output folder management
│
├── datasets/                        # Sample datasets
│   ├── sample_employees.csv
│   └── sample_sales.csv
│
├── setup.py
├── pyproject.toml
├── requirements.txt
└── README.md
```

---

## 📂 Output Files

Every experiment saves its results inside:

```
Output/
  exp1_descriptive_stats_20240101_120000/
    statistics.txt
    histogram_Age.png
    histogram_Salary.png
    boxplot_all.png
```

---

## 🧪 Sample Datasets

Two sample datasets are included in the `datasets/` folder:

| File | Description | Best For |
|------|-------------|----------|
| `sample_employees.csv` | 40 employee records (Age, Salary, Performance, etc.) | Exp 1–10, 12 |
| `sample_sales.csv` | 40 sales records with Date column | Exp 3, 9, 11 |

---

## ➕ Adding a New Experiment (Exp 13+)

1. Create `bigdata_practical_toolkit/experiments/exp13.py`
2. Define a `run(df)` function inside it
3. Add one line in `menu.py` → `EXPERIMENTS` list
4. Add one line in `cli.py` → `EXPERIMENT_MAP` dict
5. Import `exp13` in `cli.py`

That's it! No other changes needed.

---

## 📦 Dependencies

| Package | Purpose |
|---------|---------|
| `pandas` | Data loading and manipulation |
| `numpy` | Numeric computing |
| `matplotlib` | Base plotting library |
| `seaborn` | Statistical visualisations |
| `scikit-learn` | ML algorithms |
| `openpyxl` | Excel file support |
| `colorama` | Coloured terminal output |
| `scipy` | Statistical functions (Z-score) |

---

## 🎓 Intended Audience

Undergraduate students in **Big Data Analytics**, **Data Science**, or **Machine Learning** laboratory courses.

---

## 📄 License

MIT License – free to use, modify, and distribute for educational purposes.
