Metadata-Version: 2.3
Name: pathlib-requests-nemkov
Version: 0.1.1
Summary: 
Author: roBoot-LDT
Author-email: maksim.nemkov.ldt@gmail.com
Requires-Python: >=3.13
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: aiofiles (==24.1.0)
Requires-Dist: aiohappyeyeballs (==2.6.1)
Requires-Dist: aiohttp (==3.11.18)
Requires-Dist: aiohttp (>=3.11.18,<4.0.0)
Requires-Dist: aiosignal (==1.3.2)
Requires-Dist: attrs (==25.3.0)
Requires-Dist: certifi (==2025.4.26)
Requires-Dist: charset-normalizer (==3.4.2)
Requires-Dist: dotenv (==0.9.9)
Requires-Dist: frozenlist (==1.6.0)
Requires-Dist: idna (==3.10)
Requires-Dist: multidict (==6.4.3)
Requires-Dist: propcache (==0.3.1)
Requires-Dist: python-dotenv (==1.1.0)
Requires-Dist: requests (==2.32.3)
Requires-Dist: urllib3 (==2.4.0)
Requires-Dist: yarl (==1.20.0)
Description-Content-Type: text/markdown

# Async API Data Fetcher

![Python](https://img.shields.io/badge/python-3.8%2B-blue)
![aiohttp](https://img.shields.io/badge/aiohttp-3.8%2B-green)
![Poetry](https://img.shields.io/badge/packaging-poetry-ff69b4)
![MIT License](https://img.shields.io/badge/license-MIT-blue)

A high-performance asynchronous solution for fetching and organizing user data from REST APIs.

## Features

- **Asynchronous Architecture**: Uses `aiohttp` for concurrent API requests and `aiofiles` for non-blocking file operations
- **Structured Data Organization**:
  - Creates user-specific directories
  - Stores posts as individual JSON files
- **Configuration Management**:
  - Environment variables via `.env`
  - Automatic config validation
- **Efficient Resource Usage**:
  - Single reusable HTTP session
  - Proper resource cleanup
- **Error Handling**:
  - Basic data validation
  - HTTP status code checking

## Prerequisites

- Python 3.8+
- Poetry (for dependency management)

## Installation

1. Clone the repository:
   ```bash
   git clone https://github.com/roBoot-LDT/pathlib-requests.git
   cd pathlib-requests
   ```

2. Install dependencies:
   ```bash
   poetry install
   ```

3. Create `.env` file:
   ```ini
   API_URL=https://jsonplaceholder.typicode.com/
   USERS_ENDPOINT=users
   POSTS_ENDPOINT=posts
   DATA_DIR=data
   TIMEOUT=5
   ```

## Usage

Run the main script:
```bash
poetry run python main.py
```

### Expected Output Structure
```
data/
└── user_posts/
    ├── 1/
    │   ├── post_1.json
    │   └── post_2.json
    └── 2/
        ├── post_3.json
        └── post_4.json
```

## Configuration

| Environment Variable | Default | Description |
|----------------------|---------|-------------|
| `API_URL` | Required | Base API URL |
| `USERS_ENDPOINT` | `users` | Users endpoint path |
| `POSTS_ENDPOINT` | `posts` | Posts endpoint path |
| `DATA_DIR` | `data` | Root data directory |
| `TIMEOUT` | `5` | Request timeout in seconds |

## Architecture

```mermaid
classDiagram
    class AppConfig{
        +data_dir: Path
        +user_dir: Path
        +api_url: str
        +timeout: int
        +_find_dotenv()
        +_get_project_root()
    }
    
    class Data{
        +save_data()
        +_async_save_data()
    }
    
    class Organizer{
        +fetch_all()
        +_fetch_data()
        +close()
    }
    
    AppConfig --> Data : provides config
    AppConfig --> Organizer : provides config
    Organizer --> Data : saves data
```

## Performance

Typical execution times:
- Fetch 10 users with posts: ~1.2s
- Fetch 100 users with posts: ~3.5s

## License

Distributed under the MIT License. See LICENSE for more information.

## Contributing

1. Fork the project
2. Create your feature branch
3. Commit your changes
4. Push to the branch
5. Open a pull request

## TODO

- [ ] Add retry mechanism for failed requests
- [ ] Implement comprehensive logging
- [ ] Add unit tests
- [ ] Support paginated API responses

Made with ❤️ by Nemkov
