Metadata-Version: 2.4
Name: threaded_map_reduce
Version: 0.1.0
Summary: A threaded version of map reduce
Project-URL: Homepage, https://github.com/JoseBlanca/threaded-map-reduce
Author-email: Jose Blanca <jblanca@upv.es>
License-Expression: MIT
License-File: LICENSE
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.13
Description-Content-Type: text/markdown

# threaded-map-reduce

A Python library that implements **map**, **unordered map**, and **map-reduce** using threads.

This library is intented to be performant for CPU-bound tasks.
The map implementation has been tested to be much more performant than the map method of the [futures.ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor) class of the standar library.

---

## Features

* **Parallel map** with deterministic order: `map`
* **Parallel unordered map** for maximum throughput: `map_unordered`
* **Parallel map-reduce**: `map_reduce`
* It groups items into chunks to reduce the parallelization overhead
* No external dependencies

---

## Installation

```bash
pip install threaded-map-reduce
```

(Or, if you use `uv`:)

```bash
uv pip install threaded-map-reduce
```

---

## Quick Start

### 1. Parallel map (ordered)

```python
from threaded_map_reduce import map

def square(x):
    return x * x

nums = range(1000)
result = list(threaded_map(square, nums, num_computing_threads=4, chunk_size=100))
print(result[-10:])
```

---

### 2. Parallel map (unordered)

Faster, but order is not preserved:

```python
from threaded_map_reduce import map_unordered

nums = range(1000)
result = list(map_unordered(square, nums, num_computing_threads=4, chunk_size=100))
print(sorted(result))
print(result[-10:])
```

---

### 3. Parallel map-reduce

Useful for reductions such as sums, counts, or any associative operation.

```python
from operator import add
from threaded_map_reduce import map_reduce

def square(x):
    return x * x

nums = range(0, 1000)
result = map_reduce(square, add, nums,
                    num_computing_threads=4,
                    chunk_size=100)
print(result)
```

---

## API Summary

### `threaded_map(map_fn, items, num_computing_threads, chunk_size)`

Runs `map_fn` over every item in parallel and yields results keeping input order.

### `map_unordered(map_fn, items, num_computing_threads, chunk_size)`

Same as above, but yields items in any order.

### `map_reduce(map_fn, reduce_fn, items, num_computing_threads, chunk_size)`

Maps items in parallel, reduces mapped chunks using the provided reducer function, and returns a single result.

---

## License

MIT License.
