Metadata-Version: 2.4
Name: baltimore-patapsco
Version: 0.20.0
Summary: Shared Django runtime foundations for Baltimore civic applications
Author: Mayor's Office of Performance and Innovation
License-Expression: MIT
License-File: LICENSE
Requires-Dist: asgiref>=3.9,<4.0
Requires-Dist: django>=6.0,<6.1
Requires-Dist: djangorestframework>=3.16,<4.0
Requires-Dist: drf-spectacular>=0.28,<0.31
Requires-Dist: pyyaml>=6.0.3,<7.0
Requires-Dist: sentry-sdk[django]>=2.64.0,<3.0 ; extra == 'observability'
Requires-Dist: dj-database-url>=3.0,<4.0 ; extra == 'runtime'
Requires-Dist: django-environ>=0.12,<0.15 ; extra == 'runtime'
Requires-Dist: whitenoise>=6.9,<7.0 ; extra == 'runtime'
Requires-Dist: django-axes>=8.3.1,<9.0 ; extra == 'signin'
Requires-Dist: pytest>=9.1.1,<10.0 ; extra == 'testing'
Requires-Dist: pytest-django>=4.12.0,<5.0 ; extra == 'testing'
Requires-Python: >=3.13
Provides-Extra: observability
Provides-Extra: runtime
Provides-Extra: signin
Provides-Extra: testing
Description-Content-Type: text/markdown

# baltimore-patapsco (Python)

Reusable Django runtime primitives for Baltimore civic applications.

```python
INSTALLED_APPS += ["baltimore.patapsco"]

REST_FRAMEWORK = {
    "EXCEPTION_HANDLER": "baltimore.patapsco.api.exception_handler",
}

urlpatterns = [path("", include("baltimore.patapsco.urls"))]
```

Place `RequestContextMiddleware` early in `MIDDLEWARE` — after
`SecurityMiddleware` (and WhiteNoise when present), before session, auth, and
anything that can short-circuit a response — so every later middleware, view,
and log line sees the bound request ID. The reference app shows the canonical
order:

```python
MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "whitenoise.middleware.WhiteNoiseMiddleware",
    "baltimore.patapsco.observability.RequestContextMiddleware",
    # sessions, common, csrf, auth, messages, clickjacking...
]
```

An error that carries data, such as a 412 rebase, declares it, and the handler
sends it in the canonical envelope with its headers:

```python
from baltimore.patapsco.api import PublicAPIError

raise PublicAPIError(
    status=412,
    code="precondition_failed",
    message="This record changed since you opened it.",
    details={"current_etag": etag},
    headers={"ETag": etag},
)
```

The status must be a 4xx, and the code must match `ERROR_CODE_PATTERN`.
The details must render as strict JSON, so NaN and Infinity are refused;
they are kept as they will render, so a later change to the caller's own
list or dict can't reach the response. The headers can't include one the
platform sets, such as `X-Request-ID` or `Set-Cookie`. A mistake raises
`ValueError` where the error is declared.

## Idempotent uploads

`baltimore.patapsco.idempotency` is optional. Add it to `INSTALLED_APPS` (it
brings one table) and decorate the DRF view methods a client may resend. A
repeated `Idempotency-Key` from the same user then gets the first answer
instead of a second record:

```python
from baltimore.patapsco.idempotency import idempotent


class PhotoUploadView(APIView):
    @idempotent
    def post(self, request): ...
```

`idempotency_settings(window=timedelta(hours=24))` sets the window, and
`manage.py patapsco_idempotency_cleanup` deletes expired records. The SDK's
upload queue sends the key. See the
[uploads guide](../../docs/use/uploads.md).

## Stable health and readiness views

Applications that mount or extend the operational views directly use the public
facade:

```python
from baltimore.patapsco.api import HealthView, ReadinessView
```

`HealthView` is process-only liveness. `ReadinessView` runs the established
database check and returns its public-safe 200 or 503 status document. Both use
no authentication classes and `AllowAny`, and both remain ordinary DRF
`APIView` classes with `as_view()`. Both stay outside `ATOMIC_REQUESTS` on every
database, subclasses included, so an outage is answered by the view, not a 500. A database failure is reported once per
worker as it begins, at error level with the exception; later failed probes log
`readiness.database_unavailable` at warning level with only the exception type,
and the first healthy probe logs `readiness.database_recovered`. A probe every
few seconds during an outage therefore sends one Sentry event, not one per probe.

`HealthView` and `DiagnosticsView` render their errors in the Patapsco envelope
their OpenAPI declares, whatever the app's `REST_FRAMEWORK["EXCEPTION_HANDLER"]`
is. A subclass may override `get_exception_handler()` and then owns its schema.

An application may subclass either class, override `get()`, and call
`super().get(request)` before composing app-owned public-safe checks. The parent
response status, request ID, and database result remain the starting contract;
the application owns an explicit serializer and OpenAPI annotation for any
extended payload.

The previous `baltimore.patapsco.api.views` imports resolve to the same class
objects for compatibility. `DiagnosticsView` remains available only from that
module and through the packaged URL configuration; it is intentionally absent
from `baltimore.patapsco.api.__all__` because its staff/public policy is a
separate operational surface. The settings-free package root exports neither
view class.

`api_error_response()` and the DRF exception handler force every 5xx response to
the public `internal_error` code, generic message, and no details. Application
callers cannot opt a server failure out of that safety boundary. An unexpected
exception also rolls back every active database transaction managed by Django's
`ATOMIC_REQUESTS`, so a failed request cannot commit partial writes. Applications
retain ownership of explicitly managed transactions.

Use `build_logging_config()` for the shared JSON/pretty log shape. Modules are
incrementally adoptable; installing the package does not enable auth, email, or
Sentry by itself. Structured log context is copied and recursively redacted,
including credentials nested in mappings and sequences. Credential keys are
matched across case and separator conventions, including `clientAPIKey` and
`X-API-Key`; repeated safe context is preserved and cycles are cut safely.
A Django or DRF request object anywhere in a record, such as the `request` extra
Django adds to every `django.request` 4xx/5xx record, renders as
`{"method": ..., "path": ...}`, never as its repr, so a request object never
carries its query string into a log. Lines other loggers write are their own:
`runserver`'s `django.server` request line still prints the full URL.
`configure_sentry()` renders request objects in Sentry extras and log
breadcrumbs the same way.

Bind a record's identity once it is known and every later line of the request,
including `request.complete`, carries it under `context`:

```python
from baltimore.patapsco.observability import bind_log_context

bind_log_context(case_id=case.public_id)
```

Bind identifiers, never resident data: logs redact credentials, not civic
contact or location fields. `None` values are ignored, an explicit record
`extra` wins over a bound value, and outside a request the binding persists
unless the work runs inside `request_context()`. Bound values do not tag Sentry
events.

## File downloads

`stream_csv()` streams a CSV export that a client can always tell is incomplete,
and `content_disposition()` names a file safely for every client:

```python
from baltimore.patapsco.api import stream_csv


class CaseExportView(APIView):
    def get(self, request):
        rows = cases_for(request).values_list("number", "opened", "status").iterator()
        return stream_csv(request, rows, header=["Case", "Opened", "Status"], filename="cases.csv")
```

A download holds one 64 Ki-character chunk in memory whatever its row count,
under WSGI and ASGI. A failure before the first chunk is the view's ordinary
error response. A failure after it is logged as `download.stream_failed` with
the request ID and re-raised, so the server aborts the transfer instead of
ending it. `max_rows=` answers 413 with the error envelope (`export_too_large`,
`details.max_rows`) before anything is sent, and otherwise sends the file with a
Content-Length. `bom=True` adds the byte order mark Excel needs for non-ASCII
text. A request that carries `X-Download-End-Marker`, as `downloadFile()` in
`@city-of-baltimore/patapsco` sends, gets its token echoed and appended to the
body, so a cut is visible even where HTTP framing hides it (runserver, or a
proxy that speaks HTTP/1.0 to gunicorn). Call it from a synchronous view: it
reads the first rows before returning.

A workbook, or any export that isn't a CSV, answers an oversized request the
same way: `capped_rows(rows, max_rows=N)` returns at most N rows for the app's
XLSX library to write, or raises `ExportTooLarge`, the same 413 from a DRF view;
a plain Django view returns `export_too_large_response(N)`. Answer the written
file with `file_response(content, filename=..., content_type=XLSX_CONTENT_TYPE)`
(a PDF the same way): it carries `stream_csv`'s download headers, the safe
`Content-Disposition`, `Cache-Control: no-store` and
`X-Content-Type-Options: nosniff`, from one place.

Every cell of the header and rows goes through `spreadsheet_safe_cell()`, with
no opt-out, so no cell opens in a spreadsheet as a formula. Text that starts
with TAB, CR or LF, or whose first character after whitespace is `=`, `+`, `-`,
`@` or a full-width form of one, gets a leading `'`. Typed numbers and text
that is exactly a negative decimal (`-12.5`, not `-0500`) are left alone.
Vertical tab and form feed become a space, and the other C0 controls except
TAB, LF and CR are removed, so a pasted control never fails an export. Call it
yourself for a CSV written anywhere else, such as an import report on disk:

```python
from baltimore.patapsco.api import spreadsheet_safe_cell

writer.writerow([spreadsheet_safe_cell(value) for value in row])
```

For an XLSX text cell, `spreadsheet_safe_text()` gives the same text with the
controls scrubbed and no `'`. The writer's string type is the protection
(openpyxl `data_type = "s"`, xlsxwriter `write_string`); a `'` prefix never is,
because XLSX stores it as visible text:

```python
from baltimore.patapsco.api import spreadsheet_safe_text

cell = sheet.cell(row=row, column=column, value=spreadsheet_safe_text(value))
cell.data_type = "s"
```

The [wire contract](../../docs/reference/wire-contract.md#spreadsheet-safe-cells)
states the whole policy. The TypeScript SDK's `spreadsheetSafeText`,
`spreadsheetSafeCell` and `buildCsv` apply it in the browser, tested against the
same vectors.

`spreadsheet_cell_vectors()` returns those vectors from the installed package,
so an app's own export tests run every one through its real writer instead of
copying them (app consistency standard §8.2). It needs no Django settings. Each
vector is a frozen `SpreadsheetCellVector` with `case`, `input`, the XLSX `text`
and the CSV `output`:

```python
from baltimore.patapsco.contracts import spreadsheet_cell_vectors


@pytest.mark.parametrize("vector", spreadsheet_cell_vectors(), ids=lambda v: v.case)
def test_the_export_is_spreadsheet_safe(vector):
    assert exported_csv_cell(vector.input) == vector.output
```

`content_disposition(filename, disposition="attachment")` adds what Django's
header lacks: an ASCII `filename="..."` fallback beside `filename*` for
non-ASCII names, and a name reduced to its final path component, without
control or bidi-control characters, and cut to 255 UTF-8 bytes.

## Session lifecycle

An application that wants one session policy applies it in settings, installs
the middleware after `AuthenticationMiddleware`, and authenticates its API
with Patapsco's `SessionAuthentication`:

```python
from datetime import timedelta

from baltimore.patapsco.sessions import session_settings

globals().update(session_settings(idle_timeout=timedelta(hours=1)))
MIDDLEWARE += ["baltimore.patapsco.sessions.middleware.SessionActivityMiddleware"]
REST_FRAMEWORK["DEFAULT_AUTHENTICATION_CLASSES"] = [
    "baltimore.patapsco.sessions.authentication.SessionAuthentication",
]
```

Only requests a person starts renew the session. Health, readiness, the status
read and views marked `@session_passive` never do. A signed-out API request
gets 401 with a `WWW-Authenticate` challenge. `baltimore.patapsco.urls` then
mounts `GET /api/session/status`, `POST /api/session/renew` and
`POST /api/session/end` ("Sign in again now").
[Session lifecycle](../../docs/use/sessions.md) is the adoption sequence, and
the [wire contract](../../docs/reference/wire-contract.md#session-lifecycle) is
the exact behaviour. Nothing changes until `session_settings()` is applied.

## Send email

```python
from baltimore.patapsco.mail import email_settings
from baltimore.patapsco.observability.runtime import runtime_environment

globals().update(
    email_settings(
        production=runtime_environment() == "production",
        from_email="Baltimore OPI <no-reply@baltimorecity.gov>",
    )
)
```

Apps send through Django's email API. Production sends through SendGrid's SMTP
relay with `SENDGRID_API_KEY` from the environment; anywhere else mail is
printed, or, with `PATAPSCO_EMAIL_ALLOWLIST`, sent only to the addresses and
domains listed. A key alone never turns sending on, and the allowlist is
refused in production. `assert_sends_through_the_relay` is the assertion each
app's suite shares. [Send email](../../docs/use/mail.md) is the adoption
sequence, including the sender authentication City IT sets up.

## Passwords and reset

```python
from baltimore.patapsco.auth import password_settings

globals().update(password_settings(proxies=1))
```

Django's four validators with a minimum of 15 characters, one-hour reset links
and the reset throttle, which names a client through the `proxies` the app
declares in front of it. An app that offers self-service reset mounts
`baltimore.patapsco.auth.urls` (or, on Wagtail, sets
`WAGTAILADMIN_USER_PASSWORD_RESET_FORM`), and runs the shared
`PasswordResetVectors` from the `testing` extra in its own suite. An imported
account with no password gets its first one through an activation link
(`send_activation`, `manage.py send_activation_links`), never through
self-service reset.
[Passwords](../../docs/use/passwords.md) is the adoption sequence.

## Optimistic concurrency

```python
from baltimore.patapsco.api import ETagField, OptimisticConcurrencyMixin


class PlanViewSet(
    OptimisticConcurrencyMixin, viewsets.ModelViewSet
): ...  # the serializer declares etag = ETagField()
```

A read answers `ETag` (and each list row its `etag`); a write must send
`If-Match`, is compared with the locked row and saved by compare-and-swap
(`claim_revision`), and a stale one answers 412 `precondition_failed` with
`details.current_etag`, a missing one 428. Service code uses
`check_if_match` and `claim_revision`. [Save and autosave](../../docs/use/autosave.md)
is the adoption sequence.

## Branded error pages

```python
handler404 = "baltimore.patapsco.pages.page_not_found"
handler500 = "baltimore.patapsco.pages.server_error"
```

With `page_settings(product_name=...)` in settings, a server-rendered site's
404 and 500 are the City's page with the Reference ID, the value of
`X-Request-ID`. `too_many_requests(request, retry_after=...)` answers a lockout
the same way, with `Retry-After`. A request that prefers JSON keeps the
canonical envelope, and the 500 renders with no database and no context
processor. [Error pages](../../docs/use/pages.md) is the adoption sequence.

## Audit events

```python
from baltimore.patapsco.audit import acting_as, audit, SystemActor, target_of

audit("data.export", outcome="succeeded", target=target_of(report), rows=340)
with acting_as(SystemActor("importer")):
    audit("access.override", outcome="succeeded", reason="backfill")
```

One event shape for who did which sensitive action, in the structured log by
default. Installing `"baltimore.patapsco.audit"` adds an append-only table
(`PATAPSCO_AUDIT_SINKS = ["log", "database"]`, with
`PATAPSCO_AUDIT_RETENTION_DAYS`) and records sign-ins and sign-outs.
[Audit events](../../docs/use/audit.md) is the adoption sequence.

## Local sign-in

```python
globals().update(sign_in_settings())  # from baltimore.patapsco.auth.sign_in
path("", include("baltimore.patapsco.auth.sign_in_urls"))
```

With the `signin` extra, `POST /api/session/` signs in and `DELETE` signs out.
Every refusal answers one 400 `sign_in_failed`, and five failures for one name
from one address lock that pair for fifteen minutes (django-axes, fixed
window, proxy-aware). `SignInVectors` in the `testing` extra proves it against
the app's routes. [Local sign-in](../../docs/use/sign-in.md) is the adoption
sequence.

## Sentry privacy defaults

Install the `observability` extra and call the shared initializer only when a DSN
is configured:

```python
from baltimore.patapsco.observability import configure_sentry

configure_sentry(
    environment="production",
    release="my-app@1.2.3",
    traces_sample_rate=0.1,
)
```

`configure_sentry()` is deny-by-default: it disables automatic PII, request-body
capture, and stack-frame local variables. Its event pipeline also removes request
cookies and query strings, strips query/fragment data from request URLs and the
`Referer` request header, reduces
breadcrumb, span, and trace data URLs (`url`, `http.url`, `url.full`) to origin
plus path and removes `http.query`, `http.fragment`, `url.query`, and
`url.fragment` from them, and recursively redacts the shared credential-key fragments, Sentry's credential/session
denylist, and common civic contact and precise-location fields. This protection
also applies to transaction events. Safe operational fields such as request IDs,
routes, and HTTP methods remain available.

`traces_sampler` optionally accepts a callable with the SDK sampling-context
dictionary and a return rate from zero to one. It decides whether a trace is
recorded at creation, before event processing; it takes precedence over
`traces_sample_rate` and the parent decision. Apps own route exclusions and
whether to honor `context["parent_sampled"]`. Omitting the callback preserves
SDK defaults. Callback exceptions retain SDK behavior and can propagate, so app
samplers must handle their expected failure cases. Initialization failures still
return `False` without exposing configuration in logs.

Pass `dsn=""` to explicitly disable reporting even when `SENTRY_DSN` exists;
omitting the argument or passing `None` permits that environment fallback.
For domain enrichment, register a public SDK global event processor once, after
successful initialization. It runs before the shared scrubber and final privacy
hooks for errors and transactions. Retain stronger app policies there; do not
import private Patapsco filters or replace the client's privacy options.

Redaction is key-based. Applications must not attach sensitive values under
misleading keys or place resident data in exception messages. Product-specific
fields still require a privacy review before being added to Sentry context.

## Configuration surface

Everything the library reads from the environment or Django settings:

| Name                          | Kind           | Default               | Meaning                                                                                                                                                       |
| ----------------------------- | -------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `APP_ENV`                     | env var        | `local`               | Environment label in diagnostics and Sentry events; overridden by `PATAPSCO_ENVIRONMENT` when set                                                             |
| `APP_RELEASE`                 | env var        | —                     | Release identifier in diagnostics and Sentry events; overridden by `PATAPSCO_RELEASE` when set                                                                |
| `APP_LOG_FORMAT`              | env var        | `json`                | `json` or `pretty` output from `build_logging_config()`                                                                                                       |
| `APP_LOG_LEVEL`               | env var        | `INFO`                | Root log level from `build_logging_config()`                                                                                                                  |
| `SENTRY_DSN`                  | env var        | —                     | Enables `configure_sentry()`; absent means Sentry stays off                                                                                                   |
| `PATAPSCO_SERVICE_NAME`       | Django setting | project name          | Service label in the diagnostics payload                                                                                                                      |
| `PATAPSCO_ENVIRONMENT`        | Django setting | `APP_ENV` env var     | Overrides the diagnostics `environment` field via Django settings instead of the process environment (a real `override_settings` test seam)                   |
| `PATAPSCO_RELEASE`            | Django setting | `APP_RELEASE` env var | Overrides the diagnostics `release` field via Django settings instead of the process environment (a real `override_settings` test seam)                       |
| `PATAPSCO_DIAGNOSTICS_PUBLIC` | Django setting | `False`               | **Security-relevant:** flips `/api/diagnostics` from `IsAdminUser` to `AllowAny`. Leave `False` unless the deployment deliberately publishes runtime metadata |
| `PATAPSCO_SESSIONS`           | Django setting | unset                 | The session policy `session_settings()` writes; setting it mounts the session endpoints and lets `SessionActivityMiddleware` load. Do not write it by hand    |
