Metadata-Version: 2.4
Name: ndgrad
Version: 0.1.0
Summary: An educational tensor and scalar automatic differentiation engine.
Author: xaanton
License-Expression: MIT
Keywords: autograd,automatic differentiation,backpropagation,numpy,education
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Education
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.1
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Provides-Extra: oracle
Requires-Dist: torch; extra == "oracle"
Dynamic: license-file

# ndgrad

A small automatic differentiation engine for NumPy arrays, written for learning.

> **Status:** educational project in early development. The API may change
> between versions.

ndgrad follows the design of
[micrograd](https://github.com/karpathy/micrograd): every operation records a
function that computes its local gradient, and `backward()` applies them in
reverse topological order. Unlike micrograd, the main engine works on whole
NumPy arrays with broadcasting. A micrograd-style scalar engine is included as
`ndgrad.scalar`.

## Installation

```bash
pip install ndgrad
```

Requires Python 3.10 or newer and NumPy 2.1 or newer.

## Example

Fit a linear model with plain gradient descent:

```python
import numpy as np
from ndgrad import Tensor

rng = np.random.default_rng(0)
x = rng.normal(size=(100, 3))
y = x @ np.array([[2.0], [-1.0], [0.5]]) + 0.3

w = Tensor(np.zeros((3, 1)))
b = Tensor(np.zeros((1, 1)))

for step in range(200):
    loss = ((Tensor(x) @ w + b - y) ** 2).mean()
    loss.backward()
    w.data -= 0.1 * w.grad
    b.data -= 0.1 * b.grad

print(w.data.ravel(), b.data.ravel())  # [ 2.  -1.   0.5] [0.3]
```

The scalar engine works on single numbers:

```python
from ndgrad.scalar import Value

a, b = Value(2.0), Value(-3.0)
c = (a * b + a ** 2).tanh()
c.backward()
print(a.grad, b.grad)  # ≈ 0.0707 0.1413
```

## What is supported

`ndgrad.Tensor` stores float64 data and supports:

- arithmetic with NumPy-style broadcasting: `+`, `-`, `*`, `/` and unary `-`;
  `**` with a numeric exponent
- matrix multiplication of 2-D tensors with `@`
- reductions: `sum` and `mean`, over all dimensions or selected ones, with
  `keepdim`
- elementwise functions: `exp`, `log`, `sigmoid`, `tanh`, `relu`, `clip`
- shape operations: `reshape`, `transpose`, `.mT`, and `.T` for 2-D tensors
- NumPy functions on tensors: `np.add`, `np.subtract`, `np.multiply`,
  `np.divide`, `np.negative`, `np.power`, `np.matmul`, `np.exp`, `np.log` and
  `np.tanh` return tensors
- `backward()` on a scalar result, or `backward(gradient)` with an explicit
  upstream gradient
- `no_grad()`, a context manager that skips recording the graph

`ndgrad.scalar.Value` supports `+`, `-`, `*`, `/`, `**` with a numeric
exponent, `log`, `sigmoid`, `tanh`, `relu`, `clip` and comparisons, and has its
own `no_grad()`.

## Limitations

- CPU only, float64 only, plain NumPy: the code favours readability over speed.
- `@` works on 2-D tensors only; there is no batched matrix multiplication yet.
- There is no indexing, slicing, concatenation, `max` or `softmax` yet.
- There are no layers, optimizers or initializers. The example above updates
  parameters by hand.
- `backward()` overwrites gradients instead of accumulating them across calls.
- The graph is traversed recursively, so very deep graphs, such as a recurrent
  network unrolled over about a thousand steps, exceed Python's recursion limit.

## Testing

The test suite compares forward values and gradients with PyTorch on random
inputs and on edge cases such as NaN, infinities and broadcasting. PyTorch is
used only for testing; ndgrad itself depends only on NumPy.

## License

MIT. See [LICENSE](LICENSE).
