Metadata-Version: 2.4
Name: scalio-platform
Version: 0.30.0
Summary: Shared building blocks of the Scalio apps: job leases, schema migrations, health, error reports, app links, setup wizard state, roles and permissions, factory reset, audit log, jobs, mail, encrypted file attachments, the demo mode and the AI module
License: Proprietary
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.27
Requires-Dist: pymongo>=4.7
Provides-Extra: files
Requires-Dist: cryptography>=42; extra == "files"
Provides-Extra: files-azure
Requires-Dist: cryptography>=42; extra == "files-azure"
Requires-Dist: azure-storage-blob>=12.19; extra == "files-azure"
Requires-Dist: azure-identity>=1.15; extra == "files-azure"
Provides-Extra: files-s3
Requires-Dist: cryptography>=42; extra == "files-s3"
Requires-Dist: boto3>=1.34; extra == "files-s3"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: cryptography>=42; extra == "dev"
Requires-Dist: boto3>=1.34; extra == "dev"
Requires-Dist: moto[s3]>=5; extra == "dev"
Requires-Dist: fastapi>=0.110; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Requires-Dist: mongomock-motor>=0.0.29; extra == "dev"
Requires-Dist: ruff>=0.17.3; extra == "dev"
Requires-Dist: pyright==1.1.414; extra == "dev"

# scalio-platform

Shared building blocks of the Scalio apps (Nexus, Compass, Citadel, Foundry, Rosetta, Pulse, Beacon, Cockpit, ID,
Monitor). Before this package each app kept its own copy of these files (architecture review C2/C4, ADR 0058 in
scalio-docs). Nothing in this package is secret.

| Module | What it does |
| --- | --- |
| `scalio_platform.job_lease` | Runs a background loop on one instance at a time (lease in `db.job_leases`). During an update two instances run side by side; without the lease both would act. |
| `scalio_platform.migrations` | Schema migrations for MongoDB: ordered steps, each applied once, version in `db.meta` (`schema_version`). |
| `scalio_platform.health` | The answer of `GET /api/health` that Scalio Monitor reads (`ok` / `degraded` / `down`, leases of the loops). |
| `scalio_platform.error_report` | Sends unexpected errors to Scalio Monitor, only when `ERROR_REPORT_URL` and `ERROR_REPORT_TOKEN` are set. Since 0.23.0 also browser errors: `client_errors_router(require_user)` gives `POST /api/client-errors` (signed-in people only, always 204, short fields without query, at most 20 a minute per process, kind `browser`); the frontend sends with `reportClientErrors` from @scalio-cloud/ui. Since 0.30.0 (Monitor M4) `configure(url, token)` sets the token of a self-hosted installation at runtime (switch «Fehler an Scalio melden», token from Compass via scalio-connect `CompassMonitor`), `configure("", "")` switches off, `state()` answers `{on, from_env, url}`; the environment always wins. |
| `scalio_platform.app_links` | Links and features of the product switcher (`SCALIO_APP_LINKS`, `SCALIO_STAGE`, `SAAS_MODE`). |
| `scalio_platform.roles` | Permission catalog of the app, built-in and own roles (`app_roles`), `require_perm` for routes, shared role API (`roles_router`) for `RoleAdmin` in scalio-ui. |
| `scalio_platform.tenant_jobs` | Runs one job per tenant side by side (`for_each_tenant`): at most `TENANT_JOBS_PARALLEL` at once (default 4), a time budget per tenant (`TENANT_JOBS_TIMEOUT`, default 900 s), a failing tenant never stops the others; for MSP instances with many customers (ADR 0063 `mandant/M11`). |
| `scalio_platform.setup_wizard` | State of the Einrichtungsassistent (`GET/PUT /api/setup/wizard`, ADR 0060): `wizard_state`, `save_wizard`, `parse_update`. |
| `scalio_platform.audit` | Protokoll → Aktionen (ADR 0070): one format in `db.audit_log` (`record`, `target`, `diff`), TTL 365 days, `audit_router` (`GET /api/audit` with filters, cursor and counts per chip, `GET /api/audit/export.csv`), `migrate` for old formats, `uncovered` for the route guard test. |
| `scalio_platform.jobs` | Protokoll → Jobs (ADR 0070): `Registry` of the background jobs, `run` records each run (`db.job_runs`, TTL 90 days; a job returning None, 0, False, an empty list or a dict without summary and counts only updates `db.job_state`), «Jetzt ausführen» through `db.job_requests` and `serve_requests` in the worker, `QueueSource` for the app's queue, `jobs_router`. |
| `scalio_platform.worker` | The background worker of an app (`python -m worker`, architecture review AR5): `Worker(db, app, [Job(name, every, fn, lease=True)], registry=..., prepare=...)` runs each job in its own loop, logs an error without stopping, holds a lease per job (`lease=False` for the process's own state), records registered jobs and serves «Jetzt ausführen»; `worker.start()` stops on SIGINT/SIGTERM. |
| `scalio_platform.maintenance` | Wartungssperre für den Import einer Sicherung (ADR 0069 W5, since 0.14.0): `enter(db, kind)` sets the lock in `db.maintenance` and waits until no job runs, `leave(db, result)` ends it; `run_guard(db, name)` around each job run (skips while locked, `Worker` does it by default, `maintenance=False` turns it off); `Gate` middleware answers 423 for `/api` except health, version and the import progress. |
| `scalio_platform.jwt_secret` | Signing secret for session tokens (since 0.16.0, security review M5): `is_weak()` (shorter than 32, published sample values); `SigningSecret().ensure(db)` at startup uses a random secret of the installation in `db.meta` when `JWT_SECRET` is weak (warning in the log, everybody signs in once more), `current()` signs, `all()` adds the strong `JWT_SECRET_PREVIOUS` values. Not for keys that encrypt stored data. |
| `scalio_platform.session` | Session-Cookie und CSRF-Schutz (since 0.15.0): `set_session_cookie`/`clear_session_cookie` (SameSite=Lax, HttpOnly, Secure, gleiche Attribute beim Löschen); `OriginCheck` middleware refuses POST/PUT/PATCH/DELETE with the session cookie from another origin (403); requests without the cookie, `exempt` paths and `allowed` origins (e.g. `http://localhost:3000`) pass. |
| `scalio_platform.net` | Client-Adresse und Rate-Limit (since 0.17.0): `client_ip(request, trusted_hops=1)` nimmt den letzten X-Forwarded-For-Eintrag (den unser Proxy anhängt), nie den ersten; ohne Header die direkte Gegenstelle. `RateLimit(limit, window)` zählt pro Schlüssel im Speicher eines Prozesses (`hit(key)` False = zu viele), höchstens `max_keys` Schlüssel. `host_problem(host, what=…)` (since 0.20.0) lehnt Server mit interner Adresse ab (Loopback, Link-Local/Metadaten immer; private Netze ausser mit `MAIL_ALLOW_PRIVATE_HOSTS=true`); `mail` prüft so den SMTP-Server beim Speichern und beim Senden. |
| `scalio_platform.mongo_auth` | MongoDB-Anmeldung (since 0.26.0, aus Nexus übernommen): `client_kwargs()` ist leer ohne `MONGO_AUTH` (Self-hosted und Docker: Benutzer und Passwort in `MONGO_URL`, nichts ändert sich); mit `MONGO_AUTH=azure-managed-identity` meldet sich der Client mit einem Token der Managed Identity bei Azure DocumentDB an (MONGODB-OIDC, `AZURE_CLIENT_ID`, `MONGO_OIDC_RESOURCE`). Die Azure-Engine setzt diese Variablen. Aufruf: `AsyncIOMotorClient(url, **mongo_auth.client_kwargs())`. |
| `scalio_platform.files` | Verschlüsselte Dateianhänge in MongoDB (since 0.18.0, Extra `files`): `FileStore(get_db, seal=, unseal=, collection=)` speichert ein Dokument pro Datei, AES-256-GCM mit eigenem Schlüssel pro Datei, dieser Schlüssel versiegelt mit der Funktion der App (Backup-Import versiegelt ihn neu); `put`/`list`/`meta`/`get`/`delete`/`delete_owner`, Aufbewahrung ab 0.22.0 mit `retention_cutoff(months)` (`RETENTION_MONTHS`, 0 = behalten), `owners()` und `delete_owners(owners)`, eigene Felder der App in `extra` (z.B. intern), Grenzen 10 MB pro Datei und 50 pro Objekt (`FileError.status` 413/409/422). `download(meta, data)` liefert immer als Anhang mit nosniff, CSP sandbox und no-store, damit HTML oder SVG nie im Origin der App läuft. |
| `scalio_platform.blobs` | Speicherort der Dateianhänge (since 0.27.0, BL-213): `blobs.from_env()` liest `FILES_STORE` = `mongo` (Standard, Bytes im Dokument), `fs` (Ordner `FILES_FS_ROOT`, Self-hosted), `azure` (`FILES_AZURE_ACCOUNT_URL` mit Managed Identity oder `FILES_AZURE_CONNECTION_STRING`, `FILES_AZURE_CONTAINER`; Extra `files-azure`) oder `s3` (`FILES_S3_BUCKET`, `FILES_S3_PREFIX`, `FILES_S3_ENDPOINT`, `FILES_S3_REGION`, Zugang über die üblichen AWS-Variablen; Extra `files-s3`). `FileStore(..., blobs=)` legt dann nur noch den verschlüsselten Inhalt dort ab, Schlüssel und Metadaten bleiben in MongoDB. `migrate(source=, target=)` verschiebt in jede Richtung (kopieren, zurücklesen, umschalten, alte Kopie löschen; wieder startbar), `usage()` zählt pro Speicher, `backup_export`/`backup_import` sind Hooks für scalio-backup, damit die Sicherung die Dateien immer enthält. |
| `scalio_platform.demo` | Demo-Modus (since 0.24.0, scalio-docs ADR 0097 `stage/7`): `enabled()` only with `DEMO_MODE=true` and `SCALIO_STAGE` demo or test; `Dataset.load` reads the app's `demo_seed.json` (accounts per role, collections); `reseed` empties and fills the owned collections and creates the accounts through the app's `make_user`; `reset_if_due` in the worker resets once a night (03:00 Europe/Zurich, one claim in `db.demo_state`); `login_info` gives the login page the demo accounts; `blocked(action)` the reason (de/en) for password change, deleting demo accounts, settings, factory reset, mail, license, MFA, self-registration, tenant connection; `Guard` middleware refuses writes on listed paths (403); since 0.25.0 `exclusive(db, load)` loads the dataset in only one process at a time (lock in `db.demo_state`, taken over after 10 minutes) and `claim_slot(db)` lets the API start claim the reset slot (moved here from Citadel, Beacon and Rosetta). |
| `scalio_platform.ai` | KI für alle Apps (since 0.29.0, scalio-docs ADR 0104): Anbieter OpenAI (auch kompatibler Endpunkt), Anthropic, Gemini, Ollama über httpx mit dem Schlüssel des Kunden; zwei Modelle pro Anbieter, `FAST` (Zusammenfassen, Erklären, Zuordnen, Berichtsfilter) und `STRONG` (Massnahmenplan, Hilfe), beide einstellbar (`models`, `strongModels`), leer oder ein früherer Standard = aktueller Standard (`DEFAULT_MODELS`); `Assistant(get_db).complete(cfg, user, system, prompt, tier=...)` prüft das Tageslimit pro Person (`dailyTokenLimitPerUser`, 429 `daily_limit`), ruft auf und zählt Tokens (nur Zahlen, nie Fragen oder Antworten; Sammlungen `ai_usage`, `ai_usage_users` wie Nexus); `merge_settings`, `public_settings` (Schlüssel nie zurück), `Usage.summary`/`reset`. |
| `scalio_platform.mail` | E-Mail (ADR 0069 W5): sending through SMTP or Microsoft 365 (secrets sealed by the app, never returned), outbox `mail_outbox` with retries (`flush` in the worker), templates per mail kind and language with placeholders (`MailKind`, own texts in `settings` {id: "mail_templates"}), `router` for «E-Mail-Versand» and «E-Mail-Vorlagen» in scalio-ui 0.29.0; free mails `queue` and the provider `tenant` (`graph_token`) since 0.10.0, several tenants (`tenants`) and the script for the send right in Exchange (`send-rights`) since 0.12.0; since 0.11.0 (N22) no SMTP user name or password without encryption (`security: none`), also not when sending an older setting. Since 0.19.0 (ADR 0090) the default from above: `defaults` (Compass via scalio-connect `MailDefaults`, the installation via `env_defaults`, both with `first_default`), provider «default» («Vorgabe verwenden», the choice of new instances) and «Über Scalio» through `relay`. Since 0.28.0 attachments (`Attachment`, at most 10 MB together, Graph 3 MB), several recipients, copies, Reply-To and a sender name per mail in `send_now` and `queue` (`send_now` returns the Message-ID), «Über Scalio» with attachments only when `relay` takes them; an own store instead of MongoDB (`store=`, same methods as `MongoStore`, Kontor: PostgreSQL). |

## Use in an app

```
scalio-platform==0.1.0            # backend/requirements.txt, exact version
```

```python
from scalio_platform import error_report, health, job_lease
from scalio_platform.migrations import BASELINE, run_migrations

MIGRATIONS = BASELINE                      # append (2, migrate_002_...) and so on, never edit past steps

async def startup():
    await run_migrations(db, MIGRATIONS)   # before the background loops
    for name, loop in LOOPS.items():
        asyncio.create_task(job_lease.run_exclusive(db, name, loop))

error_report.install(app, "citadel", build_info)
code, body = await health.health_report(db, build_info(), lease_jobs=sorted(LOOPS))

# probe of the provisioning engine (SaaS on the Docker engine), header X-Provision-Health-Secret
code, body = await health.provision_report(db, build_info(), secret=os.environ.get("PROVISION_HEALTH_SECRET", ""),
                                           provided=request.headers.get("x-provision-health-secret", ""))
# 0.9.0 (AR5): busy=<async () -> int> reports running work an update would break; the engine waits while > 0
```

A migration step must be safe to run twice: two instances can start at the same moment.

### Zurücksetzen (factory reset, ADR 0064 / 0069)

```python
from scalio_platform.factory_reset import factory_reset_router

async def after_wipe(remove_license: bool):
    if remove_license:
        await licensing.runtime.refresh(db)      # free edition right away
    await login_policy.announce_setup_code()     # new setup code in the log

app.include_router(factory_reset_router(
    get_db=lambda: db, current_user=require_admin,
    password_hash=lambda u: u.get("password_hash"), verify_password=auth.verify_password,
    has_second_factor=lambda u: bool(u.get("mfa_enabled")), verify_second_factor=check_mfa_code,
    keep_meta=(f"{PRODUCT.code}_installation", "schema_version"), license_meta=(PRODUCT.meta_id,),
    saas_mode=saas_mode, after_wipe=after_wipe))
```

`GET /api/admin/factory-reset` answers `{available, password, second_factor}` (SaaS: not available, the customer
portal resets it); `POST` takes `{confirm: "RESET", password, code, remove_license}`. Since 0.11.0 (N21, security review
2026-10-27) the second factor is required: `has_second_factor(user)` and `verify_second_factor(user, code)` are
mandatory, `verify_second_factor` checks a TOTP or backup code and consumes it (last step stored, backup code removed).
An admin without a second factor gets `second_factor_missing`, a wrong code `second_factor_wrong`; the password is
checked as well when the account has one. `allowed(user)` narrows who may (Foundry: platform owner),
`error(code)` maps the codes `reset_in_portal`, `reset_confirm_wrong`, `current_password_wrong`, `second_factor_missing`, `second_factor_wrong` to the app's own errors.
The frontend is `FactoryReset` from `@scalio-cloud/ui` (0.32.0 with the code field).

## Develop and release

- `pip install -e ".[dev]"`, then `pytest -q` and `ruff check .` (both run in CI).
- Release: bump `pyproject.toml` and `__version__`, open a scalio-docs PR with `python3 tools/changelog.py --repo scalio-platform release <version>` (changelog in scalio-docs `changelog/scalio-platform/`, ADR 0066), merge, then «Freigeben» in the Scalio
  Cockpit (workflow `publish.yml`: PyPI with trusted publishing, tag and GitHub release). Tags without `v`.
- A change of behaviour changes every app on its next upgrade: say so in the PR and the CHANGELOG.

### Protokoll: Aktionen und Jobs (ADR 0070)

```python
from scalio_platform import audit, jobs
from scalio_platform.audit import Category

CATEGORIES = (Category("access", "Zugriff", "Access", ("user.", "role.")),
              Category("license", "Lizenz", "License", ("license.",), always=True))

await audit.record(db, "user.create", user, category="access", target=audit.target("user", u["id"], u["email"]),
                   detail=f"Konto angelegt · {u['email']}", request=request)

app.include_router(audit.audit_router(get_db=lambda: db, viewer=require_perm(CATALOG, me, get_db, "audit.view"),
                                      categories=CATEGORIES, tenant_scope=lambda u: None), prefix="/api")

# worker
REGISTRY = jobs.Registry([jobs.Job("mail_intake", "Mail-Eingang", "Mail intake", "Jede Minute", "Every minute")])
await jobs.run(db, "mail_intake", mail_intake)            # inside the loop; return None when nothing happened
await jobs.serve_requests(db, REGISTRY, HANDLERS)         # own loop every few seconds («Jetzt ausführen»)

# API
app.include_router(jobs.jobs_router(registry=REGISTRY, get_db=lambda: db, viewer=view, runner=run_perm,
                                    queue=MyQueue(), audit=log), prefix="/api")
```

Call `audit.ensure_indexes(db)` and `jobs.ensure_indexes(db)` at startup. Never write passwords, tokens or contents
into `detail`, `changes` or a job summary (`changes` drops fields named like pass/secret/token/key).

### E-Mail (ADR 0069 W5)

```python
from scalio_platform import mail

KINDS = (mail.MailKind("resolved", ("Ticket erledigt", "Ticket done"),
                       ("An die meldende Person, wenn ein Ticket erledigt ist", "To the reporter when a ticket is done"),
                       subject={"de": "[{key}] Erledigt: {title}", "en": "[{key}] Done: {title}"},
                       body={"de": "Guten Tag {name}\n\n...\n\n{sender}", "en": "Hello {name}\n\n...\n\n{sender}"},
                       tokens=("key", "title", "name", "link"), sample={"key": "IT-12", "title": "Drucker", ...},
                       switchable=True),)

mailer = mail.Mailer(get_db=lambda: db, app_name="Scalio Beacon", kinds=KINDS,
                     seal=secret_box.seal, unseal=secret_box.unseal,      # the app's own encryption
                     collection="meta",                                   # default "settings"
                     override=scalio_selftest.mail.smtp_settings,         # test mailbox on test instances
                     allowed=scalio_selftest.mail.allowed, base_url=lambda: os.environ.get("APP_PUBLIC_URL", ""))
app.include_router(mailer.router(admin=require_admin, audit=log), prefix="/api")   # /api/mail/...

# ADR 0090: one default for all apps of a customer. Self-hosted: MAIL_* from mail.env; SaaS: Compass (scalio-connect).
mailer = mail.Mailer(..., defaults=mail.first_default(mail.env_defaults, compass_mail.defaults),
                     relay=compass_mail.relay)   # «Über Scalio», only when Compass allows it for the customer

await mailer.send(person["email"], "resolved", {"key": t["key"], ...}, lang=person.get("language"), ref=t["id"])
await jobs.run(db, "mail_outbox", mailer.flush)        # worker, every 30 seconds; call mailer.ensure_indexes() once
```

Every template knows `{app_name}`, `{app_url}` and `{sender}` (sender name of the settings, else the app name). A
template may be plain text or HTML (from the editor); in HTML the values are escaped. The built-in kind `test` is the
test mail. `send` returns False when mail is off, the kind is switched off, or `allowed` refuses the address. `check(subject, html)` may refuse a mail with ValueError (Foundry: no forms, only https links); it runs on saving or testing a template and before a mail goes into the outbox.

Since 0.10.0: `queue(to, subject, body, ref=...)` puts a free mail into the same outbox (own subject and text, plain
text or HTML, no template), for mails whose text the customer writes elsewhere (Nexus workflow steps); it returns
False like `send` and runs `check`. An app with its own connection to the customer's tenant passes
`graph_token=` (async, returns a Graph token or raises `MailError`): the settings then offer the provider `tenant`
(«Microsoft 365 über die Tenant-Verbindung», scalio-ui 0.31.0), which needs only the sending mailbox `graph.sender`
and no second app secret. `settings()` lists the offered providers in `providers`.

Since 0.12.0: an app connected to several tenants (Citadel, Foundry, Pulse) also passes `tenants=` (async, returns
`[{"id": Entra tenant ID, "name", "client_id"}]`); the administrator then picks the tenant that sends
(`graph.tenant_id`, taken without asking when there is only one) and the mailer calls `graph_token(tenant_id)`.
`settings()` lists `tenants` (ID and name only) and `send_rights`. `GET /mail/send-rights` (and `send_rights_script`)
writes the PowerShell that gives the send right in Exchange Online with «RBAC for Applications», limited to the sending
mailbox (management scope + role «Application Mail.Send»), for the provider `graph` and for `tenant` with `tenants`; the
app registration then needs no Mail.Send permission in Entra ID. The script can run again and checks every value.
