Metadata-Version: 2.4
Name: sumo-logger-otl
Version: 1.1.0
Summary: One-call OpenTelemetry setup for logs, traces, and metrics, shipped to Sumo Logic
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: opentelemetry-api>=1.44.0
Requires-Dist: opentelemetry-sdk>=1.44.0
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.44.0
Requires-Dist: opentelemetry-instrumentation-logging>=0.65b0
Provides-Extra: flask
Requires-Dist: opentelemetry-instrumentation-flask>=0.65b0; extra == "flask"
Provides-Extra: fastapi
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.65b0; extra == "fastapi"
Provides-Extra: django
Requires-Dist: opentelemetry-instrumentation-django>=0.65b0; extra == "django"
Provides-Extra: requests
Requires-Dist: opentelemetry-instrumentation-requests>=0.65b0; extra == "requests"
Provides-Extra: httpx
Requires-Dist: opentelemetry-instrumentation-httpx>=0.65b0; extra == "httpx"
Provides-Extra: postgres
Requires-Dist: opentelemetry-instrumentation-psycopg2>=0.65b0; extra == "postgres"
Requires-Dist: opentelemetry-instrumentation-asyncpg>=0.65b0; extra == "postgres"
Provides-Extra: redis
Requires-Dist: opentelemetry-instrumentation-redis>=0.65b0; extra == "redis"
Provides-Extra: all
Requires-Dist: opentelemetry-instrumentation-flask>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-django>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-requests>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-httpx>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-psycopg2>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-asyncpg>=0.65b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-redis>=0.65b0; extra == "all"
Dynamic: license-file

# sumo_logger_otl

One-call OpenTelemetry setup for logs, traces, and metrics, shipped straight to Sumo Logic. Call `setup_observability()` once at startup, then use a normal Python `logger` — tracing, metrics, and library instrumentation (Flask, requests, Postgres, Redis, ...) all wire themselves up automatically in the background.

```python
from sumo_logger_otl import setup_observability

logger = setup_observability(
    service_name="application-b",
    environment="production",
    presigned_url="https://endpoint4.collection.sumologic.com/receiver/v1/otlp/<your-token>",
)

logger.info("hello world!")
```

That's the whole setup. Everything below is what you get on top of it, and how to use it.

---

## Table of contents

- [Install](#install)
- [Quickstart](#quickstart)
- [Core concept: logs, traces, and how they connect](#core-concept-logs-traces-and-how-they-connect)
- [`setup_observability()` reference](#setup_observability-reference)
- [Automatic library instrumentation](#automatic-library-instrumentation)
- [Web apps: Flask / FastAPI / Django](#web-apps-flask--fastapi--django)
- [`@traced()` — tracing plain functions and scripts](#traced--tracing-plain-functions-and-scripts)
- [Distributed tracing across two services](#distributed-tracing-across-two-services)
- [`request_id` — your own correlation ID](#request_id--your-own-correlation-id)
- [Console output](#console-output)
- [Short-lived scripts: flushing before exit](#short-lived-scripts-flushing-before-exit)
- [Custom metrics](#custom-metrics)
- [Error handling in web apps](#error-handling-in-web-apps)
- [Full API reference](#full-api-reference)
- [Troubleshooting](#troubleshooting)
- [Design notes](#design-notes)

---

## Install

```bash
pip install -r requirements.txt
```

Then drop the `sumo_logger_otl/` folder into your project (or install it as a local package — see [`pyproject.toml`](./pyproject.toml)).

Instrumentation for specific libraries (Flask, requests, Postgres, Redis, ...) is picked up automatically **if the corresponding `opentelemetry-instrumentation-*` package is installed** — see [Automatic library instrumentation](#automatic-library-instrumentation) for the full list and how to add more.

---

## Quickstart

```python
from sumo_logger_otl import setup_observability

logger = setup_observability(
    service_name="my-service",
    environment="production",
    presigned_url="https://endpoint4.collection.sumologic.com/receiver/v1/otlp/<your-token>",
)

logger.info("service started")
```

Call `setup_observability()` **once**, as early as possible in your app's startup — before creating a Flask/FastAPI app, before your first outbound HTTP call. It returns a fully-configured `logging.Logger`; that's the only thing you need to hold onto. Tracing, metrics, and instrumentation are all running in the background as a side effect of this one call.

`presigned_url` is the single URL Sumo Logic gives you for an OTLP collection endpoint — token included. The package parses it, extracts the token, and builds the three correct per-signal endpoints (`/v1/traces`, `/v1/logs`, `/v1/metrics`) for you. You never need to construct these URLs yourself.

---

## Core concept: logs, traces, and how they connect

This is the one thing worth understanding up front, because it explains almost every "why isn't X showing up" question.

**A `trace_id`/`span_id` only appears on a log line if that log line runs *while a span is active*.** Nothing is automatic here by magic — it's genuinely tied to whether your code is currently "inside" a traced operation:

```python
logger.info("no span is open here - trace_id will be empty")

with tracer.start_as_current_span("my-operation"):
    logger.info("this IS inside a span - trace_id will be filled in")

logger.info("span has closed - trace_id is empty again")
```

You get this "for free" (no manual span code needed) in two situations:

1. **Inside a web request handler**, once you've called `instrument(app)` — see [Web apps](#web-apps-flask--fastapi--django).
2. **Inside a function decorated with `@traced()`** — see [`@traced()`](#traced--tracing-plain-functions-and-scripts).

Everywhere else — module-level code, background threads, anything not wrapped in a span — logs will correctly show an empty `trace_id`. That's not a bug; there's genuinely no traced operation for that log to belong to.

---

## `setup_observability()` reference

```python
setup_observability(
    service_name=None,          # e.g. "application-b" - shows up as service.name in Sumo
    environment=None,           # e.g. "production", "staging"
    log_level=logging.INFO,
    enable_metrics=True,
    instrument_http=True,       # auto-instrument requests + httpx
    instrument_db=False,        # auto-instrument psycopg / psycopg2 / asyncpg
    instrument_cache=False,     # auto-instrument redis
    presigned_url=None,         # RECOMMENDED - see below
    otlp_endpoint=None,         # manual override, all 3 signals
    traces_endpoint=None,       # manual override, traces only
    logs_endpoint=None,         # manual override, logs only
    metrics_endpoint=None,      # manual override, metrics only
    headers=None,               # manual header override
    console_output=False,       # print logs to stdout too?
)
```

Returns a `logging.Logger` — the only thing you need to keep.

### Using `presigned_url` (recommended)

```python
setup_observability(
    service_name="application-b",
    environment="production",
    presigned_url="https://endpoint4.collection.sumologic.com/receiver/v1/otlp/<token>",
)
```

Pass the single, full URL Sumo Logic gives you. The token is extracted and sent as an `x-sumo-token` header; the three per-signal endpoints are built from the base path. You don't need to build any URL yourself, or know the `/v1/traces` / `/v1/logs` / `/v1/metrics` suffix convention.

### Manual endpoint overrides

For anything other than the presigned-URL case — e.g. a self-hosted OTel Collector:

```python
setup_observability(
    service_name="application-b",
    environment="production",
    otlp_endpoint="http://otel-collector:4318",  # base URL, /v1/<signal> appended automatically
)
```

Or override one signal individually:

```python
setup_observability(
    service_name="application-b",
    presigned_url="https://.../otlp/<token>",
    logs_endpoint="https://some-other-destination/v1/logs",  # wins over presigned_url for logs only
)
```

`otlp_endpoint`/`traces_endpoint`/`logs_endpoint`/`metrics_endpoint` are used **exactly as given, no path appended** — pass the complete URL.

### Environment variable fallback

If you call `setup_observability()` with no arguments at all (or omit specific ones), it falls back to the standard OTel environment variables:

| Variable | Purpose |
|---|---|
| `OTEL_SERVICE_NAME` | Default `service_name` |
| `OTEL_ENVIRONMENT` / `ENVIRONMENT` | Default `environment` |
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Base collector URL (per-signal suffix appended) |
| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` / `_LOGS_ENDPOINT` / `_METRICS_ENDPOINT` | Explicit per-signal URL (no suffix appended) |
| `OTEL_EXPORTER_OTLP_HEADERS` | Auth headers, if not passed in code |

---

## Automatic library instrumentation

`instrument_http=True` (the default) auto-instruments **outbound** `requests` and `httpx` calls — every call becomes its own span automatically, with zero code changes at the call site:

```python
logger = setup_observability(service_name="worker", presigned_url=URL)

requests.get("https://example.com")  # <- automatically becomes a traced span
```

Two more flags cover common backends:

```python
setup_observability(
    ...,
    instrument_db=True,      # psycopg / psycopg2 / asyncpg - whichever is installed
    instrument_cache=True,   # redis
)
```

### How it works (and why it's zero-maintenance)

Instrumentation is **fully dynamic** — there is no hardcoded list of libraries in this package. Every installed `opentelemetry-instrumentation-*` package registers itself under the standard `opentelemetry_instrumentor` entry point group (the same mechanism OpenTelemetry's own official auto-instrumentation agent uses). `instrument(app)` discovers whatever's installed at runtime and wires it up:

```bash
pip install opentelemetry-instrumentation-celery
```

No code change needed anywhere in this package — Celery instrumentation is picked up automatically the next time `instrument()` runs.

### Checking what actually got instrumented

```python
result = instrument(app)
print(result)
# {'enabled': ['flask', 'requests', 'logging'], 'skipped': ['django'], 'failed': []}
```

- **`enabled`** — actually instrumented.
- **`skipped`** — the OTel instrumentation package is installed, but the *target* library isn't (or is an incompatible version). Not an error.
- **`failed`** — instrumentation was attempted and raised an exception.

For the *why* behind a skip or failure:

```python
from sumo_logger_otl import get_instrumentation_status
print(get_instrumentation_status())
# {'enabled': [...], 'skipped': {'django': 'DependencyConflict: requested "django>=2.0" but found "None"'}, 'failed': {...}}
```

If a library you expect shows up in `skipped`, it almost always means the OTel *instrumentation* package for it isn't installed yet:

```bash
pip install opentelemetry-instrumentation-flask
```

---

## Web apps: Flask / FastAPI / Django

Call `instrument(app)` once, right after creating your app object:

```python
from flask import Flask
from sumo_logger_otl import setup_observability, instrument

logger = setup_observability(service_name="application-b", presigned_url=URL)

app = Flask(__name__)
instrument(app)   # every route now gets its own span + HTTP duration metric automatically

@app.route("/ping")
def ping():
    logger.info("ping")   # <- automatically has trace_id/span_id, no extra code needed
    return {"message": "pong"}
```

Same pattern for FastAPI and Django — `instrument(app)` handles the framework-specific wiring for whichever one is installed. No manual span code needed inside any route.

> **Requires the matching OTel instrumentation package to be installed** (`opentelemetry-instrumentation-flask`, `-fastapi`, or `-django`) — see [Automatic library instrumentation](#automatic-library-instrumentation) if a framework shows up in `skipped`.

---

## `@traced()` — tracing plain functions and scripts

For anything **outside** a web request — scripts, background jobs, queue consumers, CLI tools — wrap the top-level unit of work in `@traced()` instead of writing a manual `with tracer.start_as_current_span(...)` block:

```python
from sumo_logger_otl import setup_observability, traced

logger = setup_observability(service_name="worker", presigned_url=URL)

@traced()
def process_order(order_id):
    logger.info("starting")          # <- has trace_id
    charge_customer(order_id)        # <- anything this calls, too
    logger.info("done")              # <- same trace_id throughout

process_order(123)
```

Works on both sync and async functions:

```python
@traced("custom-span-name")
async def handle_message(msg):
    ...
```

One `@traced()` at the top of a logical operation is enough — everything it calls, synchronously, inherits the same span via context propagation. You don't need to decorate every helper function individually.

---

## Distributed tracing across two services

If service A calls service B over HTTP, both services' spans link into **one trace automatically** — no manual propagation code required, as long as both sides use this package (or any OTel-instrumented stack):

**Service A** (the caller):
```python
import requests
from sumo_logger_otl import setup_observability, traced

logger = setup_observability(service_name="application-a", presigned_url=URL_A)

@traced()
def call_service_b():
    logger.info("calling service B")
    # `requests` is auto-instrumented: this call creates its own span
    # AND injects a traceparent header carrying the trace context
    response = requests.get("http://service-b/ping")
    logger.info(f"got {response.status_code}")

call_service_b()
```

**Service B** (the callee):
```python
from flask import Flask
from sumo_logger_otl import setup_observability, instrument

logger = setup_observability(service_name="application-b", presigned_url=URL_B)

app = Flask(__name__)
instrument(app)   # extracts the incoming traceparent header automatically

@app.route("/ping")
def ping():
    logger.info("ping")   # <- same trace_id as service A's call
    return {"message": "pong"}
```

**Why this works with no extra code:** OpenTelemetry uses a global propagator (W3C `traceparent` header) by default. The `requests` instrumentation on A's side injects it; the Flask instrumentation on B's side extracts it. Same `trace_id` on both sides, automatically.

`presigned_url` can be **different per service** (different Sumo tokens/sources) — logs land in separate sources, but traces still merge into one, as long as both tokens belong to the same Sumo Logic organization/account.

### Handling errors from the remote service

If B returns a 5xx, don't assume the response body is JSON — Flask's default error page is HTML:

```python
response = requests.get("http://service-b/ping", timeout=10)

if response.status_code >= 400:
    logger.error(f"service-b returned {response.status_code}: {response.text[:200]}")
else:
    logger.info(f"got {response.json()}")
```

See [Error handling in web apps](#error-handling-in-web-apps) for making sure B always logs *something* even when it crashes.

---

## `request_id` — your own correlation ID

Separate from OTel's `trace_id`, `request_id` is a plain string you control — useful for correlating against your own external ID (an order ID, a request ID from an API gateway, etc.):

```python
from sumo_logger_otl import set_request_id, get_request_id

set_request_id("order-12345")           # or call with no args for an auto-generated UUID
logger.info("processing")               # <- this log's request_id attribute is "order-12345"

current = get_request_id()
```

It's backed by a `contextvars.ContextVar`, so it's correctly isolated per async task / thread context, the same way OTel's own span context is.

---

## Console output

By default, logs are **not** printed to stdout — only exported to Sumo. Turn console printing on:

```python
setup_observability(..., console_output=True)
```

When enabled, console lines look like:
```
2026-08-26 15:36:50,624 INFO [sumo_logger] [trace_id=01dc337e...] [span_id=4b59a693...] [request_id=order-12345] processing
```

---

## Short-lived scripts: flushing before exit

Logs, traces, and metrics all export in the background on a batch timer — metrics specifically export only once every **60 seconds**. A script that runs and exits in milliseconds can lose everything if it doesn't force a flush first:

```python
from sumo_logger_otl import setup_observability, flush_logging, shutdown_tracing, shutdown_metrics

logger = setup_observability(service_name="one-off-script", presigned_url=URL)

try:
    logger.info("doing the thing")
    # ... your script logic ...

finally:
    flush_logging()
    shutdown_tracing()
    shutdown_metrics()
```

**You don't need this in a long-running service** (a web app, a worker that keeps running) — the batch exporters flush on their own in the background as the process keeps running. Call `shutdown_tracing()`/`shutdown_metrics()` once, at actual process exit (e.g. a signal handler), not sprinkled through request-handling code.

---

## Custom metrics

`setup_observability()` wires up the metrics *pipeline*, but doesn't record anything on its own — metrics need at least one explicit instrument:

```python
from sumo_logger_otl import get_meter

meter = get_meter("application-b")
request_counter = meter.create_counter("orders.processed", description="orders successfully processed")

request_counter.add(1, {"status": "ok"})
```

HTTP/DB/cache instrumentation (`instrument_http`, `instrument_db`, `instrument_cache`) also emit their own duration metrics automatically — no code needed for those.

---

## Error handling in web apps

An exception raised **before** any `logger` call in a route produces Flask's bare default error page — and zero log lines, since nothing ever called `logger.info()`/`logger.error()`. Add a global error handler so every request logs *something*, success or crash:

```python
from flask import request, jsonify

@app.errorhandler(Exception)
def handle_unhandled_exception(exc):
    logger.error(
        f"unhandled exception on {request.path}: {exc}",
        exc_info=True,   # includes the full traceback as an attribute
    )
    return jsonify({"error": "internal server error"}), 500
```

This runs **inside** the request's already-open span (Flask's instrumentation opens it before dispatching to your view), so `trace_id`/`span_id` still populate correctly on this log line — it's often the *only* log line a crashed request produces otherwise.

---

## Full API reference

```python
from sumo_logger_otl import (
    setup_observability,   # one-call setup - see above

    # Logging
    setup_logging, get_logger, flush_logging, shutdown_logging,

    # Tracing
    setup_tracing, get_tracer, shutdown_tracing, traced,

    # Metrics
    setup_metrics, get_meter, shutdown_metrics,

    # Instrumentation
    instrument, uninstrument, get_instrumentation_status,

    # Request correlation ID
    set_request_id, get_request_id,

    # Pre-built (but unconfigured until setup_observability runs)
    logger, tracer,
)
```

| Function | Purpose |
|---|---|
| `setup_observability(...)` | One-call setup for logs + traces + metrics + instrumentation. Returns the logger. |
| `traced(name=None)` | Decorator - wraps a function in a span (sync or async). |
| `instrument(app=None)` | Discover + enable installed OTel instrumentors. Call once at startup; safe to call again with an app object later. Returns `{"enabled": [...], "skipped": [...], "failed": [...]}`. |
| `get_instrumentation_status()` | Same info as `instrument()`'s return, plus the *reason* for each skip/failure. |
| `set_request_id(id=None)` | Set a correlation ID for the current context (auto-generates a UUID if omitted). |
| `get_request_id()` | Read the current context's request_id. |
| `get_logger()` / `get_tracer(name)` / `get_meter(name)` | Access the already-configured logger/tracer/meter directly. |
| `flush_logging()` | Force-flush buffered logs. Use before exit in short scripts. |
| `shutdown_logging()` / `shutdown_tracing()` / `shutdown_metrics()` | Flush and tear down. Use once, at real process exit. |
| `uninstrument()` | Reverse global (non-app-specific) instrumentation. Mainly useful in tests. |

---

## Troubleshooting

**`trace_id` is empty on a log line.**
The log ran outside any active span. See [Core concept](#core-concept-logs-traces-and-how-they-connect) — wrap the operation in `@traced()`, or check you called `instrument(app)` for a web route.

**A library shows up in `instrument()`'s `skipped` list.**
The OTel instrumentation package for it isn't installed. Run `pip install opentelemetry-instrumentation-<library>` and check `get_instrumentation_status()` for the exact dependency conflict.

**A web framework instrumentor is `enabled`, but routes still show no `trace_id`.**
Double-check `instrument(app)` was actually called with your live `app` object (not `instrument()` with no arguments) — app-specific instrumentors (Flask, FastAPI, Django) only wire up a specific instance, not future ones.

**A request to another service returns 500 with no matching log on that service's side.**
See [Error handling in web apps](#error-handling-in-web-apps) — the remote route likely crashed before any `logger` call ran. Add a global error handler.

**Metrics never show up, even though logs/traces work.**
Metrics batch-export every 60 seconds. In a short-lived script, call `flush_logging()` / `shutdown_metrics()` before exit (see [Short-lived scripts](#short-lived-scripts-flushing-before-exit)); a metric also needs at least one explicit `counter.add(...)`/`histogram.record(...)` call somewhere — the pipeline being configured doesn't create data on its own.

**`request_id` isn't showing up in Sumo, only in local console output.**
Make sure you're on the current version of `logging_handler.py` — `request_id` is included in the OTLP attribute set sent to Sumo (not just the console formatter).

---

## Design notes

- **Idempotent setup** — calling `setup_observability()`/`setup_logging()`/`setup_tracing()`/`setup_metrics()` more than once is safe; each returns the already-initialized instance rather than re-initializing.
- **Fully dynamic instrumentation** — no hardcoded per-library registry; new `opentelemetry-instrumentation-*` packages are picked up automatically via the standard `opentelemetry_instrumentor` entry point group, the same mechanism OpenTelemetry's own auto-instrumentation agent uses.
- **Failure-isolated** — one instrumentor failing to load/attach never prevents the others, or the app itself, from starting.
- **`trace_id`/`span_id` are duplicated into log attributes**, not just the OTLP `LogRecord`'s dedicated correlation fields — this is specifically because Sumo Logic's Log Search surfaces `attributes` but not those dedicated fields directly, so without this duplication, trace correlation would be invisible in Sumo's UI even though it's technically present on the wire.
