Metadata-Version: 2.5
Name: eventlog-pro
Version: 0.2.1
Summary: A tiny, dependency-free structured event log — pure Python (SQLite/PostgreSQL/MySQL/JSONL) or Django, one API, one table shape.
Project-URL: Homepage, https://github.com/latingate/eventlog-pro
Project-URL: Source, https://github.com/latingate/eventlog-pro
Project-URL: Issues, https://github.com/latingate/eventlog-pro/issues
Project-URL: Changelog, https://github.com/latingate/eventlog-pro/blob/main/CHANGELOG.md
Author-email: Gal Sarig <dev@peltransport.com>
License-Expression: MIT
License-File: LICENSE
Keywords: audit,django,eventlog,logging,mysql,postgresql,sqlite
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: Django
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Framework :: Django :: 5.2
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: System :: Logging
Classifier: Typing :: Typed
Requires-Python: >=3.10
Provides-Extra: all
Requires-Dist: django>=4.2; extra == 'all'
Requires-Dist: psycopg[binary]>=3.1; extra == 'all'
Requires-Dist: pymysql>=1.1; extra == 'all'
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: django>=4.2; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: psycopg[binary]>=3.1; extra == 'dev'
Requires-Dist: pymysql>=1.1; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest-django>=4.8; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: django
Requires-Dist: django>=4.2; extra == 'django'
Provides-Extra: mysql
Requires-Dist: pymysql>=1.1; extra == 'mysql'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.1; extra == 'postgres'
Description-Content-Type: text/markdown

# eventlog-pro

A small structured event log: one `log_event()` call, one twelve-column row, two
interchangeable modes.

- **Pure Python** — the default install has **zero dependencies**. Writes to
  SQLite, PostgreSQL or MySQL/MariaDB, chosen by a DSN. SQLite needs no server
  and no dependency, so "I don't want to run a database" is already covered.
- **Django** — the app, model, migrations and admin, routed through the ORM.

Both modes write the **same table shape**, so one database can be read by
either. This is not a replacement for stdlib `logging`: it records structured
business events you will query later, not lines of text you will grep.

```python
import eventlog_pro
from eventlog_pro import log_event

eventlog_pro.configure(dsn="postgresql://user:pw@db/app")

log_event(
    app="api",
    category="webhook",
    sub_category="zoho",
    event_type="error",
    event_code="SIGNATURE_MISMATCH",
    entity=customer,                  # or "INV-1234", or ("pel", "customer", 7)
    remarks="Invalid webhook signature",
    data={"path": request.path, "ip": request.META.get("REMOTE_ADDR")},
    created_by="system",
)
```

## Install

```bash
pip install eventlog-pro                 # SQLite, no dependencies
pip install "eventlog-pro[django]"       # the Django app
pip install "eventlog-pro[postgres]"     # psycopg 3
pip install "eventlog-pro[mysql]"        # PyMySQL
pip install "eventlog-pro[all]"          # everything
```

## Requirements

The base install has **no dependencies** and needs only the standard library.
Each extra adds exactly one.

| | Required | Tested in CI | Also verified by hand |
|---|---|---|---|
| Python | 3.10+ | 3.10, 3.11, 3.12, 3.13 | — |
| Django (`[django]`) | 4.2+ | 4.2, 5.2 | 5.2.17, 6.1 |
| PostgreSQL (`[postgres]`) | 9.5+ server, `psycopg` 3.1+ | `postgres:16` | 18.3 |
| MySQL / MariaDB (`[mysql]`) | MySQL 5.7+ / MariaDB 10.2+, `PyMySQL` 1.1+ | `mysql:8` | — |
| SQLite | the `sqlite3` bundled with your Python | via the full suite | — |

Notes:

- **PostgreSQL** — `psycopg2` is accepted as a fallback if your deployment
  already has it; both speak DB-API 2.0 and libpq URLs. The server minimum is
  9.5 because the DDL uses `jsonb` and `CREATE TABLE IF NOT EXISTS`; the `id`
  column is emitted as `GENERATED BY DEFAULT AS IDENTITY`, matching what Django
  4.1+ generates, which needs **PostgreSQL 10+**.
- **MySQL** — `mysqlclient` (`MySQLdb`) is accepted as a fallback. The minimum
  is set by the native `json` column type: MySQL 5.7 or MariaDB 10.2.
- **SQLite** — needs the JSON1 extension for the `data` column's `CHECK
  (JSON_VALID(...))` constraint, which mirrors Django's own SQLite `JSONField`.
  JSON1 has shipped in CPython's bundled SQLite for years; if you build Python
  against a custom SQLite, compile it in.
- **Django** — the `[django]` extra is `Django>=4.2` and does not pin an upper
  bound, so a fresh install resolves to the newest Django available. Pin it
  yourself if you need to match a deployment.

## Two modes

### Pure Python

```python
import eventlog_pro
from eventlog_pro import log_event

eventlog_pro.configure(dsn="sqlite:///./eventlog-pro.db")   # once, at startup

event = log_event(app="api", category="system", event_code="STARTUP")
print(event.id)
```

Or configure nothing in code and set `EVENTLOG_DSN` in the environment.

The two import styles are deliberate. `log_event` is imported by name because it
appears at every call site, and `configure` is left qualified because it is
called once and `configure(...)` alone says nothing about *what* is being
configured in a file that sets up several libraries.

### Django

```python
# settings.py
INSTALLED_APPS = [
    ...,
    "eventlog_pro.contrib.django",
]

EVENTLOG_PRO = {
    "TABLE": "eventlog_eventlog",
    "DATABASE_ALIAS": "default",
    "ADMIN_ENABLED": True,
    "ADMIN_READONLY": True,      # add/change disabled; delete still allowed
    "ADMIN_SEARCH_DATA": True,   # searching the JSON column: see Limitations
    "ADMIN_LIST_PER_PAGE": 50,
    "RAISE_ON_ERROR": True,
    "DEFAULT_APP": "",
}
```

```bash
python manage.py migrate eventlog_pro
```

Then log events from anywhere in the project. There is no `configure()` call —
putting the app in `INSTALLED_APPS` is the configuration, and `AppConfig.ready()`
points the package at the `DATABASE_ALIAS` you chose:

```python
from eventlog_pro import log_event

event = log_event(app="api", category="system", event_code="STARTUP")
print(event.id)
```

`log_event()` is the *same function* in both modes. In Django mode it returns
the `EventLog` model instance (matching what `EventLog.objects.create(...)`
returned before); in pure mode, an `Event` dataclass. Both expose `.id`, `.app`,
`.event_code`, `.data` and `.created_at`.

**Mode selection is explicit — never autodetected.** In precedence order:
`configure(backend="django")`, then a `django://<alias>` DSN, then
`AppConfig.ready()`, which only fires because you put the app in
`INSTALLED_APPS`.

## DSN formats

| DSN | Backend | Extra |
|---|---|---|
| `sqlite:///./eventlog-pro.db` · `sqlite:////abs/path.db` · `sqlite://:memory:` | SQLite | none |
| `postgresql://u:pw@host:5432/db` · `postgres://…` | PostgreSQL | `[postgres]` |
| `mysql://u:pw@host:3306/db` · `mariadb://…` | MySQL/MariaDB | `[mysql]` |
| `jsonl:///./events.jsonl` | JSON Lines — **export only**, see below | none |
| `memory://` | in-process list, for tests | none |
| `null://` | accepts and discards everything | none |
| `django://` · `django://<alias>` | Django ORM | `[django]` |

Query parameters: `?table=` overrides the table name in any backend, so one
environment variable can configure a whole deployment. SQLite also takes
`?timeout=` and `?journal_mode=`; MySQL takes `?charset=`, `?connect_timeout=`,
`?unix_socket=` and `?ssl_disabled=`; PostgreSQL passes every other parameter
straight through to libpq (`?sslmode=require`, `?application_name=…`).

Three slashes means a relative path, four means absolute — the SQLAlchemy
convention.

### Which one to use

**If you do not want to run a database server, use `sqlite://`.** It is in the
standard library, adds no dependency, needs no server, and supports the entire
API. It is the default for exactly that reason.

**`jsonl://` is not a substitute for it and is not recommended for general
use.** It is an export format for one job: writing a file that something else —
Fluent Bit, Vector, Loki, `logrotate` plus S3 — picks up and owns. Because it is
a flat append-only file rather than a database, `id` is always `None`, every
`event_query()` is a full file scan, and `delete_events()` raises rather than
rewrite the file. Retention is rotation, not deletion. Pick it only when the
file leaving the process is the actual requirement — see
[docs/features/jsonl-backend.md](https://github.com/latingate/eventlog-pro/blob/main/docs/features/jsonl-backend.md).

`memory://` and `null://` are for tests and for switching logging off; neither
survives the process.

## API

### `log_event(**kwargs) -> Event`

**Raises** on failure. That is today's behaviour at every existing call site,
and a logger that silently returns `None` is how you discover in month three
that nothing was recorded.

| Parameter | Type | Notes |
|---|---|---|
| `app` | `str` | Source system. Arbitrary text, max 100, dots allowed, unvalidated. Falls back to `default_app`. |
| `category` | `str` | Required. Main grouping. |
| `event_code` | `str` | Required. Stable machine-readable code. |
| `event_type` | `str` | Free text: `"error"`, `"info"`, `"warning"`, … |
| `sub_category` | `str` | Optional secondary grouping. |
| `entity` | any | See below. |
| `remarks` | `str` | Unbounded text. |
| `data` | `dict \| list \| None` | JSON payload; `None` is stored as `{}`. |
| `created_by` | `str \| None` | Username, email or process name. |
| `entity_app` / `entity_model` / `entity_id` | `str` | Set the entity columns directly, bypassing `entity=`. |

Every `varchar(100)` value is silently **truncated** to fit rather than
rejected, and `data` is serialised with `default=str`, so a stray datetime never
takes down the caller.

### `log_event_safe(**kwargs) -> Event | None`

Never raises. Logs the traceback to the `eventlog_pro` stdlib logger and returns
`None`. **This is the one to call from webhooks and signal handlers.**

`KeyboardInterrupt` and `SystemExit` are never swallowed, in either function.

### `event_query(**filters) -> list[Event]`

Reads events back, in any mode, without caring which backend is configured.
**Raises** on failure — the kill switch below is a *write*-path switch, and a
read that quietly returned `[]` would hide the problem instead of reporting it.

```python
from datetime import date
from eventlog_pro import event_query

event_query(app="api", event_code="RECEIVED")          # newest 100 first
event_query(from_created_at=date(2026, 8, 1))          # everything this month
event_query(data="INV-1234")                           # mentions this invoice
event_query(order_by=["category", ("created_at", "DESC")], limit=None)
```

| Parameter | Type | Notes |
|---|---|---|
| `id` | `int \| None` | Exact primary key. |
| `created_at` | `datetime \| date \| None` | A `datetime` matches that instant; a **`date` matches the whole UTC day**. Cannot be combined with the range arguments. |
| `from_created_at` | `datetime \| date \| None` | Inclusive lower bound. A `date` means `00:00` that day. |
| `to_created_at` | `datetime \| date \| None` | Upper bound, inclusive as a `datetime`; a **`date` means "through the end of that day"**. |
| `created_by`, `app`, `category`, `sub_category`, `event_code`, `event_type`, `entity_app`, `entity_model`, `entity_id`, `remarks` | `str \| None` | Exact match. `None` is "not filtered"; `""` is a real filter matching the empty column. |
| `data` | `str \| None` | **Substring, not equality** — see below. |
| `order_by` | `str \| tuple \| sequence \| None` | `"category"`, `"-created_at"`, `("category", "ASC")`, or a sequence mixing those. Position is sort priority. Defaults to newest first. |
| `limit` | `int \| None` | **Defaults to 100.** Pass `limit=None` for every match. |

`data` is **the one argument that does not mean equality.** Byte-exact JSON
comparison is useless in practice; finding the event that mentions an invoice
number is not. So `data="INV-1234"` matches any row whose stored JSON contains
that text, in a key or a value. Three things to know:

- Passing a `dict` raises `TypeError` rather than matching nothing — `data=` is
  a search string, so pass the value you are looking for, not the payload.
- It is **case-sensitive on SQLite and PostgreSQL, case-insensitive on MySQL**,
  whose default collation says so.
- Single tokens behave the same on every backend; anything spanning JSON
  punctuation does not, because PostgreSQL's `jsonb` reorders keys and
  normalises whitespace on the way out. It is also an unindexed scan, so pair
  it with `app=` or a date range on a large table.

Results are always `Event` objects — in Django mode too, unlike `log_event()`,
which returns the model instance.

### `delete_events(**filters) -> int`

Takes exactly the filters `event_query()` takes and returns how many rows went.
Retention, in one line:

```python
from datetime import date, timedelta
from eventlog_pro import delete_events

delete_events(to_created_at=date.today() - timedelta(days=90))
delete_events(to_created_at=date.today() - timedelta(days=90), limit=10_000)
```

**A bare `delete_events()` raises.** At least one real filter is required —
`limit` and `order_by` do not count, because they choose *which* rows, not
whether a row matches. An audit log that can empty itself by accident is a
different product.

**`event_query()` caps at 100 and this does not**, so the obvious
check-then-delete pair compares a 100-row preview against an unbounded delete.
To preview a delete exactly, pass the same arguments to both:

```python
doomed = event_query(to_created_at=cutoff, limit=None)   # limit=None matters
assert delete_events(to_created_at=cutoff) == len(doomed)
```

With `limit` set, the **oldest** matching rows go first — the retention case —
unless `order_by` says otherwise. That path runs two statements (select the ids,
then delete them), because `DELETE ... LIMIT` is MySQL-only, so a row inserted
between the two is not deleted.

`jsonl://` raises `BackendError` here and always will: the file is append-only,
and deleting rows would mean rewriting it whole. Rotate the file instead, or use
a backend that can delete — see
[docs/features/jsonl-backend.md](https://github.com/latingate/eventlog-pro/blob/main/docs/features/jsonl-backend.md).

### Kill switch

`configure(raise_on_error=False)` or `EVENTLOG_SILENT=1` makes `log_event()`
behave like `log_event_safe()`, so ops can defuse a misconfigured logger without
a deploy. `EVENTLOG_DSN=null://` turns logging off entirely.

### Configuration

| Setting | `configure()` | Env var | `EVENTLOG_PRO` key | Default |
|---|---|---|---|---|
| DSN | `dsn` | `EVENTLOG_DSN` | — | `sqlite:///./eventlog-pro.db` |
| Table | `table` | `EVENTLOG_TABLE` | `TABLE` | `eventlog_eventlog` |
| Backend override | `backend` | `EVENTLOG_BACKEND` | — | `None` |
| Raise on error | `raise_on_error` | `EVENTLOG_SILENT` (inverted) | `RAISE_ON_ERROR` | `True` |
| Create the table | `auto_create_table` | `EVENTLOG_AUTO_CREATE_TABLE` | — | `True` |
| Default `app` | `default_app` | `EVENTLOG_DEFAULT_APP` | `DEFAULT_APP` | `""` |

Precedence: explicit `configure()` → environment → defaults. Nothing connects at
import time; the first `log_event()` resolves the backend and runs
`CREATE TABLE IF NOT EXISTS` once per process. Re-configuring closes the live
backend, and `reset()` tears everything down — both safe in tests.

If nothing is configured anywhere, the package logs a one-time warning naming
the `eventlog-pro.db` file it is about to create. If an `events.db` from before
0.2.0 is in the same directory, it says so too — that file is left alone, and
`configure(dsn="sqlite:///./events.db")` keeps using it.

### `.env` files

Supported, with no extra dependency here: the environment is read on first use,
not at import, so anything that populates `os.environ` before the first
`log_event()` is picked up — including `load_dotenv()` called after
`import eventlog_pro`.

```python
from dotenv import load_dotenv      # pip install python-dotenv
load_dotenv()

from eventlog_pro import log_event
log_event(category="webhook", event_code="OK")
```

```ini
# .env
EVENTLOG_DSN=postgresql://user:pw@localhost:5432/events
EVENTLOG_DEFAULT_APP=auto.pel
EVENTLOG_SILENT=1
```

Loading the file is deliberately left to the application. This package depends
on nothing, and a library that reads files from the working directory and
mutates `os.environ` would affect every other library in the process. Django
projects normally load `.env` in `manage.py`/`wsgi.py` already; that is enough,
and `EVENTLOG_PRO` in `settings.py` covers the rest.

### `entity=`

`resolve_entity()` tries, in order: `None`; an explicit `entity_*` kwarg; an
`__eventlog_entity__()` method returning a 3-tuple or dict; a duck-typed Django
model (`_meta.app_label`, `_meta.model_name`, `pk`); a dict with
`entity_app`/`entity_model`/`entity_id` or `app`/`model`/`id`; a 3-element
tuple or list; a generic object (module, class name, first of
`pk`/`id`/`uuid`/`slug`); and finally any scalar, so `entity="INV-1234"` just
works.

**It never raises.** A broken `__eventlog_entity__` or an exploding
`__getattr__` degrades to `("", "", "")`.

### Custom backends

```python
from eventlog_pro import Backend, register_backend

class RedisBackend(Backend):
    schemes = ("redis",)

    def write(self, event):
        ...
        return event

register_backend("redis", RedisBackend)          # or "my_pkg.backends:RedisBackend"
```

Packages can also advertise backends through the `eventlog_pro.backends` entry
point group.

### Exceptions

`EventLogError` is the base. `ConfigurationError` covers bad DSNs, invalid table
names and missing drivers (the message always names the extra to install);
`UnknownSchemeError` is a subclass of it. `BackendError` means the store refused
the write, with the driver's exception kept as `__cause__`.

## The schema

Twelve columns plus `id`, defined once and built identically by both modes:
`created_at`, `created_by`, `app`, `category`, `sub_category`, `event_code`,
`event_type`, `entity_app`, `entity_model`, `entity_id`, `remarks`, `data`.

| | SQLite | PostgreSQL | MySQL |
|---|---|---|---|
| `id` | `integer … AUTOINCREMENT` | `bigint … GENERATED BY DEFAULT AS IDENTITY` | `bigint AUTO_INCREMENT` |
| `created_at` | `datetime` | `timestamp with time zone` | `datetime(6)` |
| char columns | `varchar(100)` | `varchar(100)` | `varchar(100)` |
| `remarks` | `text` | `text` | `longtext` |
| `data` | `text` + `JSON_VALID` check | `jsonb` | `json` |

Plus three indexes: `(created_at DESC)`, `(app, category, event_code)` and
`(entity_app, entity_model, entity_id)`.

**Datetime storage.** With `USE_TZ=True`, Django stores SQLite and MySQL
datetimes as UTC with the tzinfo stripped and a space separator — no `T`, no
`+00:00`. The core backends write exactly that, which is what lets both modes
read each other's rows and keeps the admin's `date_hierarchy` working. The test
suite builds the table both ways and compares the DDL character for character.

## Upgrading from an in-repo `eventlog` app

**Back up first**, and rehearse against a copy of production.

```bash
# INSTALLED_APPS: 'eventlog.apps.EventlogConfig' -> 'eventlog_pro.contrib.django'
python manage.py migrate eventlog zero --fake      # drop the old history, keep the table
python manage.py migrate eventlog_pro --fake-initial
python manage.py migrate eventlog_pro              # applies 0002_add_indexes
```

`0001` is adopted, `0002` really runs, and the table is never dropped in either
direction. If the table was created by the **core backends** it already has the
indexes, so fake both instead: `python manage.py migrate eventlog_pro --fake`.

On a large table the three `CREATE INDEX` statements take a lock proportional to
row count; on PostgreSQL, create them with `CREATE INDEX CONCURRENTLY` by hand
and fake `0002`.

## Limitations

- The admin searches the JSON `data` column by default — a full-table `LIKE`
  scan that no index helps. Set `ADMIN_SEARCH_DATA = False` past ~1M rows.
- No pooling, no batching, no async in 0.1. One connection per thread, held
  open; point the DSN at pgbouncer, or use `django://`.
- `jsonl://` is an export format, not a database, and is not recommended unless
  you are shipping the file somewhere: `id` stays `None`, every read is a full
  file scan with `limit` applied only afterwards, and `delete_events()` raises
  rather than rewrite the file. Use `sqlite://` if you just want no server.
- `event_query()` caps at 100 rows unless you pass `limit`, while
  `delete_events()` never caps. Pass `limit=None` when using one to preview the
  other.
- A limited `delete_events()` is two statements, not one — a row inserted
  between them is not deleted.
- `data=` is a text scan that no index helps, and its case-sensitivity follows
  the backend's collation.
- Changing `TABLE` after the app has loaded does not move the table or generate
  a rename; the `eventlog_pro.W001` check reports the drift.

See [CHANGELOG.md](CHANGELOG.md) for the full list of deliberate deviations from
the app this package replaced.

## Development

```bash
pip install -e ".[dev]"
pytest                      # Postgres/MySQL tests skip themselves
ruff check . && ruff format --check . && mypy

# integration tests, against throwaway containers
docker run -d --rm -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=evp -p 55432:5432 postgres:16-alpine
docker run -d --rm -e MYSQL_ROOT_PASSWORD=secret -e MYSQL_DATABASE=evp -p 33306:3306 mysql:8
EVENTLOG_TEST_POSTGRES_DSN=postgresql://postgres:secret@localhost:55432/evp \
EVENTLOG_TEST_MYSQL_DSN=mysql://root:secret@localhost:33306/evp pytest
```

## License

MIT — see [LICENSE](LICENSE).
