Metadata-Version: 2.5
Name: tangra-dataforge
Version: 0.1.0a1
Summary: Python toolkit for data collection, cleaning, analysis, and investigative journalism.
Author: Irena Popova
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: matplotlib>=3.8
Requires-Dist: openpyxl>=3.1
Requires-Dist: pandas>=2.0
Requires-Dist: requests>=2.31
Provides-Extra: analytics
Requires-Dist: duckdb>=1.1; extra == 'analytics'
Requires-Dist: pyarrow>=17.0; extra == 'analytics'
Requires-Dist: scipy>=1.13; extra == 'analytics'
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: mypy>=1.14; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Provides-Extra: scraping
Requires-Dist: beautifulsoup4>=4.12; extra == 'scraping'
Requires-Dist: httpx>=0.27; extra == 'scraping'
Requires-Dist: lxml>=5.0; extra == 'scraping'
Requires-Dist: playwright>=1.50; extra == 'scraping'
Requires-Dist: scrapy>=2.12; extra == 'scraping'
Provides-Extra: visualization
Requires-Dist: geopandas>=1.0; extra == 'visualization'
Requires-Dist: plotly>=5.24; extra == 'visualization'
Description-Content-Type: text/markdown

# TANGRA DataForge

**Open-Source Python Toolkit for Data Engineering, Analysis & Investigative Journalism**

TANGRA DataForge is an open-source Python project focused on collecting, processing, analyzing, and visualizing data. It is designed for data journalism, investigative research, and reproducible data analysis.

The project aims to transform raw data from public datasets, websites, and APIs into structured, verifiable, and meaningful information.

## Project Goals

- Collect data from public APIs, websites, CSV, JSON, and Excel files.
- Extract structured information through web scraping.
- Clean, validate, and transform datasets using Python and Pandas.
- Perform statistical analysis and identify patterns, trends, and anomalies.
- Create informative visualizations and charts.
- Support reproducible investigations with transparent methodologies and documented sources.

## Technology Stack

- **Python** — Core programming language
- **Pandas** — Data manipulation and analysis
- **NumPy** — Numerical computing
- **Beautiful Soup** — HTML parsing and web scraping
- **Requests** — HTTP requests and API integration
- **Matplotlib / Plotly** — Data visualization
- **pytest** — Automated testing
- **Git / GitHub** — Version control and collaboration

These technologies represent the planned toolkit. Dependencies will be introduced as features are implemented.

## Planned Architecture

```text
tangra-dataforge/
├── src/
│   └── tangra_dataforge/
│       ├── __init__.py
│       ├── collect.py
│       ├── clean.py
│       ├── analyze.py
│       └── visualize.py
├── investigations/
├── examples/
├── tests/
├── pyproject.toml
├── README.md
└── .gitignore
```

## Data Journalism & Research

TANGRA DataForge is being developed to support data-driven investigations into economic, social, political, and technological developments.

Planned research applications include:

- Economic indicators and regional inequality
- Housing affordability and income development
- Labour markets and workforce shortages
- Education and professional qualifications
- Technology, digital transformation, and society

Investigations will prioritize publicly accessible data, source attribution, transparent analysis, and reproducible results.

## Development Status

**Early development — project initialization**

The toolkit is currently being established. Its first development phase will focus on Python/Pandas data processing, dataset validation, and reproducible analysis workflows.

The project is not yet available as an installable Python package.

## Roadmap

- [ ] Establish Python package structure
- [ ] Implement CSV and JSON data loading
- [ ] Develop Pandas-based cleaning functions
- [ ] Add data validation and descriptive statistics
- [ ] Integrate API data collection
- [ ] Implement web scraping components
- [ ] Add visualization utilities
- [ ] Create a reproducible journalism case study
- [ ] Add automated tests and documentation
- [ ] Publish an initial release on PyPI

## Responsible Data Collection

Data collection functionality will be developed with attention to applicable laws, data protection, website access policies, rate limits, and source attribution.

## Author

**Irena Popova**

Independent Researcher | Python & Full-Stack Developer | Author & Publisher

[Website](https://www.irena-popova.com) | [GitHub](https://github.com/irenapopova)

## License

An open-source license will be added before the first public software release.
