Metadata-Version: 2.5
Name: elven-logs-interceptor-python
Version: 0.1.19
Summary: Production-grade logs interceptor for Python with Loki transport, resilience, batching, and framework integrations.
Author: Elven Observability
License: MIT
Keywords: clean-architecture,interceptor,logging,logs,loki,observability,resilience
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: System :: Logging
Requires-Python: >=3.10
Requires-Dist: httpx<0.29.0,>=0.24.1
Requires-Dist: typing-extensions>=4.12.0
Provides-Extra: all
Requires-Dist: brotli>=1.1.0; extra == 'all'
Requires-Dist: celery>=5.4.0; extra == 'all'
Requires-Dist: django>=4.2; extra == 'all'
Requires-Dist: fastapi>=0.100.0; extra == 'all'
Requires-Dist: flask>=3.0.0; extra == 'all'
Requires-Dist: loguru>=0.7.2; extra == 'all'
Requires-Dist: opentelemetry-api<2.0.0,>=1.45.1; extra == 'all'
Requires-Dist: orjson>=3.10.0; extra == 'all'
Requires-Dist: protobuf>=5.0.0; extra == 'all'
Requires-Dist: python-snappy>=0.7.1; extra == 'all'
Requires-Dist: starlette>=0.27.0; extra == 'all'
Requires-Dist: structlog>=24.0.0; extra == 'all'
Provides-Extra: celery
Requires-Dist: celery>=5.4.0; extra == 'celery'
Provides-Extra: dev
Requires-Dist: build>=1.2.2; extra == 'dev'
Requires-Dist: mypy>=1.11.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
Requires-Dist: pytest>=8.3.0; extra == 'dev'
Requires-Dist: ruff>=0.6.0; extra == 'dev'
Requires-Dist: twine>=5.1.1; extra == 'dev'
Provides-Extra: django
Requires-Dist: django>=4.2; extra == 'django'
Provides-Extra: fastapi
Requires-Dist: fastapi>=0.100.0; extra == 'fastapi'
Requires-Dist: starlette>=0.27.0; extra == 'fastapi'
Provides-Extra: flask
Requires-Dist: flask>=3.0.0; extra == 'flask'
Provides-Extra: loguru
Requires-Dist: loguru>=0.7.2; extra == 'loguru'
Provides-Extra: otel
Requires-Dist: opentelemetry-api<2.0.0,>=1.45.1; extra == 'otel'
Provides-Extra: perf
Requires-Dist: brotli>=1.1.0; extra == 'perf'
Requires-Dist: orjson>=3.10.0; extra == 'perf'
Requires-Dist: protobuf>=5.0.0; extra == 'perf'
Requires-Dist: python-snappy>=0.7.1; extra == 'perf'
Provides-Extra: structlog
Requires-Dist: structlog>=24.0.0; extra == 'structlog'
Description-Content-Type: text/markdown

# elven-logs-interceptor-python

High-performance, production-ready log interceptor for Python with Loki transport, batching, compression, circuit breaker, DLQ, and framework integrations.

## Installation

```bash
pip install elven-logs-interceptor-python
```

With all extras:

```bash
pip install "elven-logs-interceptor-python[all]"
```

### OpenTelemetry Security

Version 0.1.18 raises the optional `otel` and `all` extras to
`opentelemetry-api>=1.45.1,<2.0.0`, excluding the vulnerable parser described in
[CVE-2026-114603](https://github.com/open-telemetry/opentelemetry-python/security/advisories/GHSA-xcq7-p3pf-p88h).
The base installation still does not require OpenTelemetry. Logging APIs and
framework minimum versions are unchanged. Existing application lockfiles must
be regenerated to receive the security update.

### Resource Guardrails

Version 0.1.19 fixes quadratic matching in built-in message redaction without
disabling sanitization. Custom user-supplied regular expressions must still be
reviewed for their own runtime cost.

`LOGS_BUFFER_MAX_MEMORY_MB` (default 50 MiB) now budgets estimated retained
entry memory across the built-in buffer, queued batches, and in-flight sends.
Structured dictionaries and sequences are included; cyclic references are
handled and entries exceeding 10,000 inspected values are rejected. This is
not a cap on process RSS: Python allocation, serialization, transport buffers,
custom object internals, and an optional DLQ have separate overhead.

Pending batches are limited to `LOGS_MAX_CONCURRENT_FLUSHES`, in addition to
that many active sends. Immediate flush timers are coalesced and automatic
flushes do not create threads waiting for network completion. When admission
or queue limits are exceeded, logs are dropped and counted in
`get_metrics()["dropped_by_backpressure"]`. This intentionally favors bounded
resource use over lossless delivery during a slow or unavailable export path.
Explicit `flush()` still waits for admitted sends and propagates its send error.

## Quick Start

```python
from logs_interceptor import init, logger

init(
    {
        "appName": "billing-service",
        "interceptConsole": True,
        "transport": {
            "url": "https://loki.example.com/loki/api/v1/push",
            "tenantId": "tenant-a",
            "authToken": "token",
            "compression": "gzip",
        },
    }
)

logger.info("service started", {"port": 3000})
```

## Logging compatibility

The logger accepts both the Elven structured style and common Python
`logging` call patterns. This makes migration safer when legacy code replaces
`logging.getLogger(__name__)` with the Elven proxy.

```python
logger.info("payload created", {"request_id": "req-1"})
logger.info("payload created: %s", payload)
logger.info("payload created: %(id)s", {"id": payload_id})
logger.info("payload created: {}", payload_id)
logger.info("payload created: %s", payload_id, extra={"request_id": "req-1"})
logger.exception("failed to create payload: %s", payload_id)
```

When intercepting the standard Python `logging` module, `extra={...}` is
captured under `context.extra`:

```python
logging.getLogger(__name__).info(
    "payload created",
    extra={"payload": payload},
)
```

Supported aliases: `warning(...)`, `exception(...)` and `critical(...)`.

Sensitive fields such as `cpf`, `cnpj`, `cpfCnpj`, `documento`,
`numeroDocumento`, `taxId`, `apiKey`, `clientSecret`, `senha`, `password`,
`token`, `authorization`, `cartao/cartão`, `chavePix` and related credential
fields are redacted by default. In messages, the library redacts only the
sensitive fragment/value and preserves the rest of the log. In structured
context, sensitive keys redact the full value. The key matcher normalizes
common `snake_case`, `camelCase`, `kebab-case`, dotted variants, and accents.
Set `LOGS_FILTER_SANITIZE=false`, `LOGS_SANITIZE=false`, or
`UO_LOGS_SANITIZE=false` only when raw sensitive data is explicitly required
and approved.

## Environment Variables

The package supports the Python `LOGS_*` configuration surface aligned with
the JS v3 design where applicable.

Required for direct Loki mode:

- `LOGS_URL`
- `LOGS_TENANT`
- `LOGS_APP_NAME`

For collector-first mode, set `LOGS_EXPORTER=otlp` (or `collector`) and use
`OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`. If the explicit logs endpoint is missing,
the library derives `/v1/logs` from `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`,
`OTEL_EXPORTER_OTLP_METRICS_ENDPOINT`, or `OTEL_EXPORTER_OTLP_ENDPOINT`.
In this mode `LOGS_URL`, `LOKI_URL`, `LOGS_TENANT` and `LOGS_TOKEN` are not
required by the library. `LOGS_URL` is intentionally ignored in collector mode
to avoid accidentally posting OTLP logs to a direct Loki `/loki/api/v1/push`
endpoint. Put tenant or auth headers in
`OTEL_EXPORTER_OTLP_HEADERS`/`LOGS_OTLP_HEADERS` if your collector gateway
requires them.

Core:

- `LOGS_TOKEN`
- `LOGS_APP_VERSION`
- `LOGS_ENVIRONMENT`

Transport:

- `LOGS_EXPORTER` (`loki|otlp`, aliases: `collector`, `otel`)
- `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` (collector logs endpoint, e.g. `http://collector:4318/v1/logs`)
- `LOGS_OTLP_HEADERS` (comma-separated headers, e.g. `x-tenant=team-a`)
- `LOGS_COMPRESSION` (`none|gzip|brotli|snappy`)
- `LOGS_COMPRESSION_LEVEL`
- `LOGS_COMPRESSION_THRESHOLD`
- `LOGS_USE_WORKERS`
- `LOGS_MAX_WORKERS`
- `LOGS_WORKER_TIMEOUT`
- `LOGS_CONNECTION_POOLING`
- `LOGS_MAX_SOCKETS`
- `LOGS_ENABLE_STRUCTURED_METADATA` (`false` by default; enable only when Loki allows `limits_config.allow_structured_metadata`)
- `LOGS_TIMEOUT`
- `LOGS_MAX_RETRIES`
- `LOGS_RETRY_DELAY`

Buffer:

- `LOGS_BUFFER_MAX_SIZE`
- `LOGS_BUFFER_FLUSH_INTERVAL`
- `LOGS_BUFFER_MAX_MEMORY_MB`
- `LOGS_BUFFER_MAX_AGE`
- `LOGS_BUFFER_AUTO_FLUSH`

Filter:

- `LOGS_FILTER_LEVELS`
- `LOGS_FILTER_SAMPLING_RATE`
- `LOGS_FILTER_SANITIZE` (aliases: `LOGS_SANITIZE`, `UO_LOGS_SANITIZE`)
- `LOGS_FILTER_MAX_MESSAGE_LENGTH`
- `LOGS_FILTER_EXCLUDE_LOGGER_PREFIXES` (comma-separated)

Circuit Breaker:

- `LOGS_CIRCUIT_BREAKER_ENABLED`
- `LOGS_CIRCUIT_BREAKER_FAILURE_THRESHOLD`
- `LOGS_CIRCUIT_BREAKER_RESET_TIMEOUT`
- `LOGS_CIRCUIT_BREAKER_HALF_OPEN_REQUESTS`

DLQ:

- `LOGS_DLQ_ENABLED`
- `LOGS_DLQ_TYPE` (`memory|file`)
- `LOGS_DLQ_MAX_SIZE`
- `LOGS_DLQ_MAX_RETRIES`
- `LOGS_DLQ_BASE_PATH`

Runtime:

- `LOGS_MAX_CONCURRENT_FLUSHES`
- `LOGS_INTERCEPT_CONSOLE`
- `LOGS_PRESERVE_ORIGINAL_CONSOLE`
- `LOGS_ENABLE_METRICS`
- `LOGS_ENABLE_HEALTH_CHECK`
- `LOGS_DEBUG`
- `LOGS_SILENT_ERRORS`
- `LOGS_ENABLED`
- `LOGS_AUTO_INIT`
- `LOGS_ENABLE_EXPERIMENTAL_PROTOBUF` (optional, enables experimental snappy/protobuf transport path)

Labels:

- Prefix `LOGS_LABEL_*` (example: `LOGS_LABEL_SERVICE=billing`)

## Public API

- `init(config)`
- `get_logger()`
- `is_initialized()`
- `destroy()`
- `adestroy()`
- `logger` proxy with `debug/info/warn/warning/error/exception/fatal/critical/log/track_event/flush/aflush/get_metrics/get_health/destroy/adestroy/with_context/with_context_async`

Python import remains:

```python
import logs_interceptor
```

## Integrations

- Python `logging` (`LoggingHandler`)
- FastAPI / Starlette (`FastAPIMiddleware`)
- Django (`DjangoMiddleware`)
- Flask (`FlaskExtension`)
- Celery (`CelerySignals`)
- structlog (`StructlogProcessor`)
- loguru (`LoguruSink`)

## Auto Init

Set `LOGS_AUTO_INIT=true` and import the package.

Preload mode:

```bash
python -m logs_interceptor.preload
```

## Development

```bash
pip install -e ".[dev]"
ruff check .
mypy src
pytest
```

## Deploy / Publish

Local publish script:

```bash
./scripts/publish.sh --repository testpypi --dry-run
./scripts/publish.sh --repository testpypi
./scripts/publish.sh --repository pypi
```

Token environment variables used by default:

- `TEST_PYPI_API_TOKEN` for `--repository testpypi`
- `PYPI_API_TOKEN` for `--repository pypi`

Optional script flags:

- `--skip-checks` (skip lint/type/test)
- `--skip-build` (skip build/twine check)
- `--token-env CUSTOM_VAR` (custom token env var name)
- `--no-skip-existing` (upload fails if version already exists)

Makefile shortcuts:

```bash
make qa
make publish-dry-run
make publish-testpypi
make publish-pypi
```

GitHub Actions publish workflow:

- File: `.github/workflows/publish.yml`
- Trigger: manual (`workflow_dispatch`)
- Inputs: `repository = testpypi | pypi`
- Required secrets:
  - `TEST_PYPI_API_TOKEN`
  - `PYPI_API_TOKEN`

## License

MIT
