scalio-platform 0.1.0__tar.gz

scalio_platform-0.1.0/PKG-INFO

@@ -0,0 +1,59 @@
+Metadata-Version: 2.4
+Name: scalio-platform
+Version: 0.1.0
+Summary: Shared building blocks of the Scalio apps: job leases, schema migrations, health, error reports, app links
+License: Proprietary
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: httpx>=0.27
+Requires-Dist: pymongo>=4.6
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: fastapi>=0.110; extra == "dev"
+Requires-Dist: httpx>=0.27; extra == "dev"
+Requires-Dist: mongomock-motor>=0.0.29; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+
+# scalio-platform
+
+Shared building blocks of the Scalio apps (Nexus, Compass, Citadel, Foundry, Rosetta, Pulse, Beacon, Cockpit, ID,
+Monitor). Before this package each app kept its own copy of these files (architecture review C2/C4, ADR 0058 in
+scalio-docs). Nothing in this package is secret.
+
+| Module | What it does |
+| --- | --- |
+| `scalio_platform.job_lease` | Runs a background loop on one instance at a time (lease in `db.job_leases`). During an update two instances run side by side; without the lease both would act. |
+| `scalio_platform.migrations` | Schema migrations for MongoDB: ordered steps, each applied once, version in `db.meta` (`schema_version`). |
+| `scalio_platform.health` | The answer of `GET /api/health` that Scalio Monitor reads (`ok` / `degraded` / `down`, leases of the loops). |
+| `scalio_platform.error_report` | Sends unexpected errors to Scalio Monitor, only when `ERROR_REPORT_URL` and `ERROR_REPORT_TOKEN` are set. |
+| `scalio_platform.app_links` | Links and features of the product switcher (`SCALIO_APP_LINKS`, `SCALIO_STAGE`, `SAAS_MODE`). |
+
+## Use in an app
+
+```
+scalio-platform==0.1.0            # backend/requirements.txt, exact version
+```
+
+```python
+from scalio_platform import error_report, health, job_lease
+from scalio_platform.migrations import BASELINE, run_migrations
+
+MIGRATIONS = BASELINE                      # append (2, migrate_002_...) and so on, never edit past steps
+
+async def startup():
+    await run_migrations(db, MIGRATIONS)   # before the background loops
+    for name, loop in LOOPS.items():
+        asyncio.create_task(job_lease.run_exclusive(db, name, loop))
+
+error_report.install(app, "citadel", build_info)
+code, body = await health.health_report(db, build_info(), lease_jobs=sorted(LOOPS))
+```
+
+A migration step must be safe to run twice: two instances can start at the same moment.
+
+## Develop and release
+
+- `pip install -e ".[dev]"`, then `pytest -q` and `ruff check .` (both run in CI).
+- Release: bump `pyproject.toml` and `__version__`, add the CHANGELOG entry, merge, then «Freigeben» in the Scalio
+  Cockpit (workflow `publish.yml`: PyPI with trusted publishing, tag and GitHub release). Tags without `v`.
+- A change of behaviour changes every app on its next upgrade: say so in the PR and the CHANGELOG.

scalio_platform-0.1.0/README.md

@@ -0,0 +1,43 @@
+# scalio-platform
+
+Shared building blocks of the Scalio apps (Nexus, Compass, Citadel, Foundry, Rosetta, Pulse, Beacon, Cockpit, ID,
+Monitor). Before this package each app kept its own copy of these files (architecture review C2/C4, ADR 0058 in
+scalio-docs). Nothing in this package is secret.
+
+| Module | What it does |
+| --- | --- |
+| `scalio_platform.job_lease` | Runs a background loop on one instance at a time (lease in `db.job_leases`). During an update two instances run side by side; without the lease both would act. |
+| `scalio_platform.migrations` | Schema migrations for MongoDB: ordered steps, each applied once, version in `db.meta` (`schema_version`). |
+| `scalio_platform.health` | The answer of `GET /api/health` that Scalio Monitor reads (`ok` / `degraded` / `down`, leases of the loops). |
+| `scalio_platform.error_report` | Sends unexpected errors to Scalio Monitor, only when `ERROR_REPORT_URL` and `ERROR_REPORT_TOKEN` are set. |
+| `scalio_platform.app_links` | Links and features of the product switcher (`SCALIO_APP_LINKS`, `SCALIO_STAGE`, `SAAS_MODE`). |
+
+## Use in an app
+
+```
+scalio-platform==0.1.0            # backend/requirements.txt, exact version
+```
+
+```python
+from scalio_platform import error_report, health, job_lease
+from scalio_platform.migrations import BASELINE, run_migrations
+
+MIGRATIONS = BASELINE                      # append (2, migrate_002_...) and so on, never edit past steps
+
+async def startup():
+    await run_migrations(db, MIGRATIONS)   # before the background loops
+    for name, loop in LOOPS.items():
+        asyncio.create_task(job_lease.run_exclusive(db, name, loop))
+
+error_report.install(app, "citadel", build_info)
+code, body = await health.health_report(db, build_info(), lease_jobs=sorted(LOOPS))
+```
+
+A migration step must be safe to run twice: two instances can start at the same moment.
+
+## Develop and release
+
+- `pip install -e ".[dev]"`, then `pytest -q` and `ruff check .` (both run in CI).
+- Release: bump `pyproject.toml` and `__version__`, add the CHANGELOG entry, merge, then «Freigeben» in the Scalio
+  Cockpit (workflow `publish.yml`: PyPI with trusted publishing, tag and GitHub release). Tags without `v`.
+- A change of behaviour changes every app on its next upgrade: say so in the PR and the CHANGELOG.

scalio_platform-0.1.0/pyproject.toml

@@ -0,0 +1,29 @@
+[build-system]
+requires = ["setuptools>=69"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "scalio-platform"
+version = "0.1.0"
+description = "Shared building blocks of the Scalio apps: job leases, schema migrations, health, error reports, app links"
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "Proprietary" }
+dependencies = ["httpx>=0.27", "pymongo>=4.6"]
+
+[project.optional-dependencies]
+dev = ["pytest>=8", "fastapi>=0.110", "httpx>=0.27", "mongomock-motor>=0.0.29", "ruff>=0.6"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.ruff]
+line-length = 120
+target-version = "py311"
+
+[tool.ruff.lint]
+select = ["E", "F", "W", "I", "B"]
+ignore = ["B008"]

scalio_platform-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

scalio_platform-0.1.0/src/scalio_platform/__init__.py

@@ -0,0 +1,6 @@
+"""Shared building blocks of the Scalio apps (architecture review C2/C4, ADR 0058).
+
+Modules: job_lease (one instance per background loop), migrations (schema version and steps), health
+(GET /api/health), error_report (errors to Scalio Monitor), app_links (product switcher).
+"""
+__version__ = "0.1.0"

scalio_platform-0.1.0/src/scalio_platform/app_links.py

@@ -0,0 +1,64 @@
+"""Links of the product switcher: where the other Scalio apps of this installation run.
+
+SCALIO_APP_LINKS lists them per instance, e.g.
+  SCALIO_APP_LINKS=nexus=https://dev.scalionexus.com,rosetta=https://dev.scaliorosetta.com
+Apps without a link stay greyed out in the switcher. Module of the shared package scalio-platform (formerly a
+copy in every app with the switcher).
+
+Which Scalio functions this instance shows depends on how it runs:
+  SCALIO_STAGE=dev|test|demo  our own instances (set in scalio-infra); empty = production
+  SAAS_MODE=true              a SaaS customer instance (set by the provisioning engine)
+A self-hosted customer sets neither, so the switcher stays hidden there (we cannot know which other apps that
+customer has licensed). Its support button shows while the license maintenance runs (ADR 0022, T11): the caller
+passes that in (support.py).
+"""
+import os
+from urllib.parse import urlsplit
+
+APPS = ("nexus", "rosetta", "foundry", "citadel", "beacon", "pulse")
+STAGES = ("dev", "test", "demo")
+_TRUE = ("1", "true", "yes", "on")
+
+
+def parse_app_links(raw: str | None) -> dict:
+    """Known app keys with an http(s) URL; anything else is ignored."""
+    links = {}
+    for part in (raw or "").split(","):
+        key, sep, url = part.partition("=")
+        key, url = key.strip().lower(), url.strip().rstrip("/")
+        if not sep or key not in APPS:
+            continue
+        parts = urlsplit(url)
+        if parts.scheme in ("http", "https") and parts.netloc and not parts.username and not parts.password:
+            links[key] = url
+    return links
+
+
+def stage() -> str:
+    """dev, test or demo for our own instances, "prod" otherwise (also for unknown values)."""
+    value = (os.environ.get("SCALIO_STAGE") or "").strip().lower()
+    return value if value in STAGES else "prod"
+
+
+def saas_mode() -> bool:
+    return (os.environ.get("SAAS_MODE") or "").strip().lower() in _TRUE
+
+
+def features(self_hosted_support: bool = False) -> dict:
+    """Scalio functions shown by this instance. Demo shows the switcher but takes no support tickets.
+    self_hosted_support: a self-hosted customer's maintenance runs (decided by the caller from the license)."""
+    current, saas = stage(), saas_mode()
+    return {
+        "switcher": saas or current in STAGES,
+        "support": saas or current in ("dev", "test") or (current == "prod" and bool(self_hosted_support)),
+    }
+
+
+def app_links(support: bool | None = None) -> dict:
+    """Public answer of GET /api/apps. Links only where the switcher is shown. support: the caller's answer
+    whether the support button shows (support.shown(): maintenance, hidden by an admin)."""
+    shown = features()
+    if support is not None:
+        shown["support"] = bool(support)
+    links = parse_app_links(os.environ.get("SCALIO_APP_LINKS")) if shown["switcher"] else {}
+    return {"links": links, "stage": stage(), "saas": saas_mode(), "features": shown}

scalio_platform-0.1.0/src/scalio_platform/error_report.py

@@ -0,0 +1,179 @@
+"""Report unexpected errors to Scalio Monitor (monitoring.scaliocloud.com, POST /ingest/error).
+
+Module of the shared package scalio-platform (formerly a copy in every app); protocol in scalio-monitor docs/ERRORS.md.
+Off unless ERROR_REPORT_URL and ERROR_REPORT_TOKEN are set, so self-hosted customers send nothing unless they
+opt in (decision M4). Sent are only: type, message, stack, route template, method, status, version, commit.
+Never request bodies, headers, query strings, cookies or user data; the monitor cleans the text again.
+
+    from scalio_platform import error_report
+    error_report.install(app, "nexus", build_info)        # after app = FastAPI(...)
+    await error_report.report_exception(exc, kind="job", path="lifecycle_task")   # background loops
+
+The report runs in the background with a short timeout: a slow or missing monitor never slows a request.
+The same error at the same place is sent at most once per minute per process.
+The groups the monitor answered are kept in memory (`recent_groups()`): a support ticket lists them with a link to
+the monitor (ADR 0022, T9; `support_router(..., errors=error_report.errors_for_support)`).
+"""
+import asyncio
+import logging
+import os
+import re
+import time
+import traceback
+from datetime import datetime, timezone
+
+import httpx
+
+log = logging.getLogger("error_report")
+TIMEOUT = 5.0
+REPEAT_SECONDS = 60
+MAX_STACK = 8000
+MAX_GROUPS = 50
+_FINGERPRINT = re.compile(r"^[0-9a-f]{8,64}$")
+_ID = re.compile(r"^(?:\d+|[0-9a-f]{24}|[0-9a-f-]{36}|[\w-]{20,})$", re.I)  # numbers, ObjectIds, UUIDs, tokens
+
+
+class Reporter:
+    def __init__(self, app_name: str, build_info=None, url: str | None = None, token: str | None = None,
+                 transport: httpx.AsyncBaseTransport | None = None, clock=time.monotonic):
+        self.app_name = app_name
+        self.build_info = build_info or (lambda: {})
+        self.url = (url if url is not None else os.environ.get("ERROR_REPORT_URL", "")).strip().rstrip("/")
+        self.token = (token if token is not None else os.environ.get("ERROR_REPORT_TOKEN", "")).strip()
+        self.transport, self.clock = transport, clock
+        self._client: httpx.AsyncClient | None = None
+        self._last: dict = {}
+        self._tasks: set = set()
+        self._groups: dict = {}
+
+    @property
+    def enabled(self) -> bool:
+        return bool(self.url and self.token)
+
+    def payload(self, exc: BaseException, kind: str, path: str = "", method: str = "",
+                status: int | None = None) -> dict:
+        try:
+            build = self.build_info() or {}
+        except Exception:  # noqa: BLE001 - a broken version lookup must not hide the error
+            build = {}
+        stack = "".join(traceback.format_exception(type(exc), exc, exc.__traceback__))[-MAX_STACK:]
+        return {"app": self.app_name, "kind": kind, "type": type(exc).__name__, "message": str(exc)[:2000],
+                "stack": stack, "path": path, "method": method, "status": status,
+                "version": str(build.get("version") or ""), "commit": str(build.get("commit") or "")}
+
+    def _repeat(self, body: dict) -> bool:
+        tb = body["stack"].strip().splitlines()
+        key = (body["type"], body["path"], tb[-3] if len(tb) >= 3 else body["message"][:100])
+        now = self.clock()
+        if now - self._last.get(key, -REPEAT_SECONDS) < REPEAT_SECONDS:
+            return True
+        self._last[key] = now
+        if len(self._last) > 500:
+            self._last = {k: v for k, v in self._last.items() if now - v < REPEAT_SECONDS}
+        return False
+
+    async def send(self, body: dict) -> bool:
+        if self._client is None:
+            self._client = httpx.AsyncClient(transport=self.transport, timeout=TIMEOUT)
+        try:
+            r = await self._client.post(f"{self.url}/ingest/error", json=body,
+                                        headers={"Authorization": f"Bearer {self.token}"})
+        except httpx.HTTPError as exc:
+            log.warning("error report not sent: %s", type(exc).__name__)  # never log the token
+            return False
+        if r.status_code != 200:
+            return False
+        self._remember(body, r)
+        return True
+
+    def _remember(self, body: dict, response: httpx.Response) -> None:
+        try:
+            fp = str((response.json() or {}).get("group") or "")
+        except ValueError:
+            return
+        if not _FINGERPRINT.match(fp):
+            return
+        old = self._groups.pop(fp, {})
+        self._groups[fp] = {"fingerprint": fp, "type": body["type"], "route": body["path"] or body["kind"],
+                            "time": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+                            "count": old.get("count", 0) + 1, "monitor": self.url}
+        while len(self._groups) > MAX_GROUPS:
+            self._groups.pop(next(iter(self._groups)))
+
+    def recent_groups(self, limit: int = 10) -> list:
+        """Last error groups this process reported, oldest first (no messages, no stack)."""
+        return [dict(g) for g in list(self._groups.values())[-limit:]]
+
+    def submit(self, exc: BaseException, kind: str = "backend", path: str = "", method: str = "",
+               status: int | None = None):
+        """Queue a report in the background. Returns the task, or None when off or repeated."""
+        if not self.enabled:
+            return None
+        body = self.payload(exc, kind, path, method, status)
+        if self._repeat(body):
+            return None
+        task = asyncio.get_running_loop().create_task(self.send(body))
+        self._tasks.add(task)
+        task.add_done_callback(self._tasks.discard)
+        return task
+
+    async def drain(self):
+        """Wait for queued reports (tests, shutdown)."""
+        if self._tasks:
+            await asyncio.gather(*list(self._tasks), return_exceptions=True)
+
+
+def route_template(scope: dict) -> str:
+    route = scope.get("route")
+    if route is not None and getattr(route, "path", None):
+        return route.path
+    parts = [("{id}" if _ID.match(p) else p) for p in (scope.get("path") or "").split("/")]
+    return "/".join(parts)[:300]
+
+
+class ErrorReportMiddleware:
+    """Pure ASGI middleware: reports exceptions no handler caught (the 500s), then lets them go on."""
+
+    def __init__(self, app, reporter: Reporter):
+        self.app, self.reporter = app, reporter
+
+    async def __call__(self, scope, receive, send):
+        if scope["type"] != "http":
+            return await self.app(scope, receive, send)
+        try:
+            await self.app(scope, receive, send)
+        except Exception as exc:
+            try:
+                self.reporter.submit(exc, "backend", route_template(scope), scope.get("method", ""), 500)
+            except Exception:  # noqa: BLE001 - reporting must never change the answer
+                log.warning("error report failed")
+            raise
+
+
+_reporter: Reporter | None = None
+
+
+def install(app, app_name: str, build_info=None, **kw) -> Reporter:
+    """Report the unhandled errors of this FastAPI app. Off without ERROR_REPORT_URL/ERROR_REPORT_TOKEN."""
+    global _reporter
+    _reporter = Reporter(app_name, build_info, **kw)
+    app.add_middleware(ErrorReportMiddleware, reporter=_reporter)
+    if _reporter.enabled:
+        log.info("error reports to %s are on", _reporter.url)
+    return _reporter
+
+
+async def report_exception(exc: BaseException, kind: str = "job", path: str = "") -> None:
+    """For background loops: report an exception that would otherwise only reach the log."""
+    if _reporter is not None:
+        _reporter.submit(exc, kind, path)
+
+
+def recent_groups(limit: int = 10) -> list:
+    """Last error groups reported by this process: fingerprint, type, route, time, count, monitor."""
+    return _reporter.recent_groups(limit) if _reporter is not None else []
+
+
+async def errors_for_support(_user=None) -> list:
+    """For the support ticket's diagnostic block (scalio-connect support_router errors=...)."""
+    return recent_groups()

scalio_platform-0.1.0/src/scalio_platform/health.py

@@ -0,0 +1,62 @@
+"""GET /api/health: the same answer in every Scalio app, read by the monitoring dashboard
+(monitoring.scaliocloud.com, concept https://claude.ai/artifact/Ls47WPEtKL2LQfEiMroJFH).
+
+    {"status": "ok" | "degraded" | "down", "version": "...", "commit": "...", "db": "ok" | "error" | "none",
+     "jobs": {"<loop>": {"ok": true, "last_seen": "<ISO time>"}}}
+
+HTTP 200 for ok and degraded, 503 for down (database not answering). degraded = a background loop that should
+run has no valid lease (job_leases, see job_lease.py), so nobody runs it. No login, no secrets, no counts of
+customers or users, no host names. db "none" = the app runs without a database (cockpit without MONGO_URL).
+Module of the shared package scalio-platform (formerly a copy in every app).
+"""
+import asyncio
+from datetime import UTC, datetime
+
+DB_TIMEOUT = 2.0  # seconds; the dashboard checks every minute and must not hang on a dead database
+
+
+def _aware(value):
+    if isinstance(value, datetime) and value.tzinfo is None:
+        return value.replace(tzinfo=UTC)  # pymongo returns naive UTC
+    return value if isinstance(value, datetime) else None
+
+
+async def _db_ok(db, timeout: float) -> bool:
+    try:
+        await asyncio.wait_for(db.command("ping"), timeout)
+        return True
+    except Exception:  # noqa: BLE001 - any failure means "down" for the dashboard
+        return False
+
+
+async def _lease_jobs(db, expected, now) -> dict:
+    found = {}
+    async for d in db["job_leases"].find({"_id": {"$in": list(expected)}}, {"until": 1, "renewedAt": 1}):
+        found[d["_id"]] = d
+    jobs = {}
+    for name in sorted(expected):
+        d = found.get(name) or {}
+        until, seen = _aware(d.get("until")), _aware(d.get("renewedAt"))
+        jobs[name] = {"ok": bool(until and until >= now), "last_seen": seen.isoformat() if seen else None}
+    return jobs
+
+
+async def health_report(db, build: dict, lease_jobs=(), timeout: float = DB_TIMEOUT, now=None):
+    """(HTTP status, body) for GET /api/health. `lease_jobs` = names of the loops that run under a job lease."""
+    now = now or datetime.now(UTC)
+    body = {"status": "ok", "version": build.get("version", ""), "commit": build.get("commit", ""),
+            "db": "ok", "jobs": {}}
+    if db is None:
+        body["db"] = "none"
+        return 200, body
+    if not await _db_ok(db, timeout):
+        body.update(status="down", db="error")
+        return 503, body
+    if lease_jobs:
+        try:
+            body["jobs"] = await asyncio.wait_for(_lease_jobs(db, lease_jobs, now), timeout)
+        except Exception:  # noqa: BLE001
+            body["jobs"] = {}
+    if any(not j["ok"] for j in body["jobs"].values()):
+        body["status"] = "degraded"
+    return 200, body

scalio_platform-0.1.0/src/scalio_platform/job_lease.py

@@ -0,0 +1,99 @@
+"""Run each background loop on one backend instance only.
+
+During an update two backends or workers can run side by side for a moment (compose recreate, Container Apps
+revision swap). Without a lock both would run the same job twice (act in a tenant, bill, send mail). Each loop
+therefore runs only while its instance holds a lease in
+db.job_leases ({_id: loop name, owner, until}). The holder renews it every RENEW seconds; another instance
+waits and takes over once the lease has expired (TTL seconds after the holder stopped renewing). A graceful
+stop releases the lease at once.
+
+Architecture review C1: https://claude.ai/artifact/2GWNZiXnnFVo5RTRnwm1Zu
+"""
+import asyncio
+import logging
+import os
+import socket
+import uuid
+from datetime import datetime, timedelta, timezone
+
+from pymongo import ReturnDocument
+from pymongo.errors import DuplicateKeyError
+
+logger = logging.getLogger("scalio_platform.lease")
+
+COLLECTION = "job_leases"
+OWNER = f"{socket.gethostname()}:{os.getpid()}:{uuid.uuid4().hex[:8]}"
+TTL = 60       # seconds a lease stays valid without renewal
+RENEW = 20     # holder renews this often
+WAIT = 15      # a waiting instance retries this often
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+async def acquire(db, name: str, owner: str = OWNER, ttl: float = TTL) -> bool:
+    """Take or renew the lease `name`. True when `owner` holds it afterwards."""
+    now = _now()
+    try:
+        doc = await db[COLLECTION].find_one_and_update(
+            {"_id": name, "$or": [{"owner": owner}, {"until": {"$lt": now}}]},
+            {"$set": {"owner": owner, "until": now + timedelta(seconds=ttl), "renewedAt": now}},
+            upsert=True, return_document=ReturnDocument.AFTER)
+    except DuplicateKeyError:  # someone else holds a valid lease
+        return False
+    return bool(doc) and doc.get("owner") == owner
+
+
+async def release(db, name: str, owner: str = OWNER):
+    await db[COLLECTION].delete_one({"_id": name, "owner": owner})
+
+
+async def run_exclusive(db, name: str, loop_factory, owner: str = OWNER,
+                        ttl: float = TTL, renew: float = RENEW, wait: float = WAIT):
+    """Run `loop_factory()` only while holding the lease `name`.
+
+    Waits for the lease, starts the loop, renews the lease while the loop runs, and cancels the loop when the
+    lease is lost (another instance took it over). If the database cannot be reached for a renewal, the loop
+    keeps running until the lease it already holds would expire, so a short outage does not stop the work.
+    """
+    task = None
+    try:
+        while True:
+            held_until = None
+            while held_until is None:
+                try:
+                    if await acquire(db, name, owner, ttl):
+                        held_until = _now() + timedelta(seconds=ttl)
+                        break
+                except Exception as e:  # noqa: BLE001
+                    logger.warning(f"lease {name}: acquire failed: {e}")
+                await asyncio.sleep(wait)
+
+            task = asyncio.create_task(loop_factory())
+            while True:
+                done, _ = await asyncio.wait({task}, timeout=renew)
+                if done:
+                    return task.result()  # the loop ended by itself (or raised)
+                try:
+                    if await acquire(db, name, owner, ttl):
+                        held_until = _now() + timedelta(seconds=ttl)
+                        continue
+                    logger.warning(f"lease {name}: taken over by another instance, stopping the loop here")
+                except Exception as e:  # noqa: BLE001
+                    if _now() + timedelta(seconds=renew) < held_until:
+                        logger.warning(f"lease {name}: renewal failed, keeping the loop for now: {e}")
+                        continue
+                    logger.warning(f"lease {name}: renewal failed and the lease expires, stopping the loop: {e}")
+                task.cancel()
+                await asyncio.gather(task, return_exceptions=True)
+                task = None
+                break
+    finally:
+        if task is not None and not task.done():
+            task.cancel()
+            await asyncio.gather(task, return_exceptions=True)
+        try:
+            await release(db, name, owner)
+        except Exception as e:  # noqa: BLE001
+            logger.info(f"lease {name}: release failed, it expires on its own: {e}")

scalio_platform-0.1.0/src/scalio_platform/migrations.py

@@ -0,0 +1,76 @@
+"""Schema migrations for MongoDB: ordered steps, each applied once, with the schema version stored in the database.
+
+Taken from scalio-nexus `backend/migrations.py` (architecture review C4). The version lives in
+`db.meta` ({id: "schema_version", version, appliedAt}), the same document Nexus already writes, so Nexus keeps its
+history. Export/import (scalio-backup) can read it with `schema_version(db)` on both sides.
+
+    from scalio_platform.migrations import BASELINE, run_migrations
+
+    async def migrate_002_rename_field(db):
+        await db.items.update_many({"old": {"$exists": True}}, {"$rename": {"old": "new"}})
+
+    MIGRATIONS = BASELINE + [(2, migrate_002_rename_field)]
+    await run_migrations(db, MIGRATIONS)          # at startup, before the background loops
+
+Rules: versions strictly increasing, append only, never edit or reorder a past step. Every step must be safe to run
+again (idempotent): during an update two instances can start at the same moment and both run it.
+A database with a newer version than the image knows (an older image after a rollback) is left as it is; the
+call logs a warning and returns the stored version.
+"""
+import logging
+from datetime import datetime, timezone
+
+logger = logging.getLogger("scalio_platform.migrations")
+
+DOC_ID = "schema_version"
+
+
+async def migrate_001_baseline(db):
+    """Baseline: marks a database as managed by the runner. Changes nothing."""
+    return
+
+
+BASELINE = [(1, migrate_001_baseline)]
+
+
+def _checked(migrations) -> list:
+    steps = list(migrations)
+    versions = [v for v, _ in steps]
+    if any(not isinstance(v, int) or v < 1 for v in versions):
+        raise ValueError("migration versions must be positive integers")
+    if versions != sorted(versions) or len(set(versions)) != len(versions):
+        raise ValueError("migration versions must be strictly increasing (append only)")
+    return steps
+
+
+async def schema_version(db, collection: str = "meta") -> int:
+    """The schema version stored in the database, 0 for a database the runner never touched."""
+    doc = await db[collection].find_one({"id": DOC_ID}, {"_id": 0, "version": 1}) or {}
+    return int(doc.get("version", 0))
+
+
+async def run_migrations(db, migrations, collection: str = "meta", log: logging.Logger | None = None) -> int:
+    """Apply every step whose version is above the stored one, in order, storing the version after each step.
+    Returns the schema version afterwards. A failing step raises; the version stays at the last good step."""
+    log = log or logger
+    steps = _checked(migrations)
+    current = await schema_version(db, collection)
+    latest = steps[-1][0] if steps else 0
+    if current > latest:
+        log.warning("database schema_version %s is newer than this build knows (%s); nothing migrated",
+                    current, latest)
+        return current
+    applied = current
+    for version, step in steps:
+        if version <= current:
+            continue
+        log.info("running DB migration %s: %s", version, getattr(step, "__name__", "step"))
+        await step(db)
+        await db[collection].update_one(  # $max: a second instance that is already further never goes back
+            {"id": DOC_ID},
+            {"$max": {"version": version}, "$set": {"appliedAt": datetime.now(timezone.utc).isoformat()}},
+            upsert=True)
+        applied = version
+    if applied != current:
+        log.info("DB migrations complete, schema_version now %s", applied)
+    return applied

scalio_platform-0.1.0/src/scalio_platform.egg-info/PKG-INFO

@@ -0,0 +1,59 @@
+Metadata-Version: 2.4
+Name: scalio-platform
+Version: 0.1.0
+Summary: Shared building blocks of the Scalio apps: job leases, schema migrations, health, error reports, app links
+License: Proprietary
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: httpx>=0.27
+Requires-Dist: pymongo>=4.6
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: fastapi>=0.110; extra == "dev"
+Requires-Dist: httpx>=0.27; extra == "dev"
+Requires-Dist: mongomock-motor>=0.0.29; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+
+# scalio-platform
+
+Shared building blocks of the Scalio apps (Nexus, Compass, Citadel, Foundry, Rosetta, Pulse, Beacon, Cockpit, ID,
+Monitor). Before this package each app kept its own copy of these files (architecture review C2/C4, ADR 0058 in
+scalio-docs). Nothing in this package is secret.
+
+| Module | What it does |
+| --- | --- |
+| `scalio_platform.job_lease` | Runs a background loop on one instance at a time (lease in `db.job_leases`). During an update two instances run side by side; without the lease both would act. |
+| `scalio_platform.migrations` | Schema migrations for MongoDB: ordered steps, each applied once, version in `db.meta` (`schema_version`). |
+| `scalio_platform.health` | The answer of `GET /api/health` that Scalio Monitor reads (`ok` / `degraded` / `down`, leases of the loops). |
+| `scalio_platform.error_report` | Sends unexpected errors to Scalio Monitor, only when `ERROR_REPORT_URL` and `ERROR_REPORT_TOKEN` are set. |
+| `scalio_platform.app_links` | Links and features of the product switcher (`SCALIO_APP_LINKS`, `SCALIO_STAGE`, `SAAS_MODE`). |
+
+## Use in an app
+
+```
+scalio-platform==0.1.0            # backend/requirements.txt, exact version
+```
+
+```python
+from scalio_platform import error_report, health, job_lease
+from scalio_platform.migrations import BASELINE, run_migrations
+
+MIGRATIONS = BASELINE                      # append (2, migrate_002_...) and so on, never edit past steps
+
+async def startup():
+    await run_migrations(db, MIGRATIONS)   # before the background loops
+    for name, loop in LOOPS.items():
+        asyncio.create_task(job_lease.run_exclusive(db, name, loop))
+
+error_report.install(app, "citadel", build_info)
+code, body = await health.health_report(db, build_info(), lease_jobs=sorted(LOOPS))
+```
+
+A migration step must be safe to run twice: two instances can start at the same moment.
+
+## Develop and release
+
+- `pip install -e ".[dev]"`, then `pytest -q` and `ruff check .` (both run in CI).
+- Release: bump `pyproject.toml` and `__version__`, add the CHANGELOG entry, merge, then «Freigeben» in the Scalio
+  Cockpit (workflow `publish.yml`: PyPI with trusted publishing, tag and GitHub release). Tags without `v`.
+- A change of behaviour changes every app on its next upgrade: say so in the PR and the CHANGELOG.

scalio_platform-0.1.0/src/scalio_platform.egg-info/SOURCES.txt

@@ -0,0 +1,19 @@
+README.md
+pyproject.toml
+src/scalio_platform/__init__.py
+src/scalio_platform/app_links.py
+src/scalio_platform/error_report.py
+src/scalio_platform/health.py
+src/scalio_platform/job_lease.py
+src/scalio_platform/migrations.py
+src/scalio_platform.egg-info/PKG-INFO
+src/scalio_platform.egg-info/SOURCES.txt
+src/scalio_platform.egg-info/dependency_links.txt
+src/scalio_platform.egg-info/requires.txt
+src/scalio_platform.egg-info/top_level.txt
+tests/test_app_links.py
+tests/test_error_report.py
+tests/test_health.py
+tests/test_job_lease.py
+tests/test_migrations.py
+tests/test_texts.py

scalio_platform-0.1.0/src/scalio_platform.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

scalio_platform-0.1.0/src/scalio_platform.egg-info/requires.txt

@@ -0,0 +1,9 @@
+httpx>=0.27
+pymongo>=4.6
+
+[dev]
+pytest>=8
+fastapi>=0.110
+httpx>=0.27
+mongomock-motor>=0.0.29
+ruff>=0.6

scalio_platform-0.1.0/src/scalio_platform.egg-info/top_level.txt

@@ -0,0 +1 @@
+scalio_platform

scalio_platform-0.1.0/tests/test_app_links.py

@@ -0,0 +1,83 @@
+"""Product switcher links (SCALIO_APP_LINKS)."""
+from scalio_platform.app_links import app_links, parse_app_links
+
+
+def test_known_apps_with_https_links():
+    raw = "nexus=https://dev.scalionexus.com/, rosetta = https://dev.scaliorosetta.com ,citadel=http://localhost:3001"
+    assert parse_app_links(raw) == {
+        "nexus": "https://dev.scalionexus.com",
+        "rosetta": "https://dev.scaliorosetta.com",
+        "citadel": "http://localhost:3001",
+    }
+
+
+def test_unknown_apps_and_bad_urls_are_ignored():
+    raw = "compass=https://portal.scaliocloud.com,foundry=javascript:alert(1),nexus=,rosetta,citadel=https://u:p@x.com,pulse=ftp://x"
+    assert parse_app_links(raw) == {}
+
+
+def test_empty_or_missing_setting(monkeypatch):
+    assert parse_app_links("") == {}
+    assert parse_app_links(None) == {}
+    monkeypatch.delenv("SCALIO_APP_LINKS", raising=False)
+    monkeypatch.setenv("SCALIO_STAGE", "dev")
+    assert app_links()["links"] == {}
+    monkeypatch.setenv("SCALIO_APP_LINKS", "foundry=https://dev.scaliofoundry.com")
+    assert app_links()["links"] == {"foundry": "https://dev.scaliofoundry.com"}
+
+
+def _mode(monkeypatch, stage=None, saas=None, links="nexus=https://dev.scalionexus.com"):
+    for key, value in (("SCALIO_STAGE", stage), ("SAAS_MODE", saas), ("SCALIO_APP_LINKS", links)):
+        if value is None:
+            monkeypatch.delenv(key, raising=False)
+        else:
+            monkeypatch.setenv(key, value)
+    return app_links()
+
+
+def test_self_hosted_customer_shows_neither_switcher_nor_support(monkeypatch):
+    # Nothing set: a customer's own installation. Links are not even sent.
+    assert _mode(monkeypatch) == {"links": {}, "stage": "prod", "saas": False,
+                                  "features": {"switcher": False, "support": False}}
+
+
+def test_dev_and_test_show_switcher_and_support(monkeypatch):
+    for stage in ("dev", " TEST "):
+        answer = _mode(monkeypatch, stage=stage)
+        assert answer["stage"] == stage.strip().lower()
+        assert answer["features"] == {"switcher": True, "support": True}
+        assert answer["links"] == {"nexus": "https://dev.scalionexus.com"}
+
+
+def test_demo_shows_switcher_without_support(monkeypatch):
+    answer = _mode(monkeypatch, stage="demo")
+    assert answer["stage"] == "demo"
+    assert answer["features"] == {"switcher": True, "support": False}
+
+
+def test_saas_shows_both_in_production(monkeypatch):
+    answer = _mode(monkeypatch, saas="true")
+    assert answer["stage"] == "prod" and answer["saas"] is True
+    assert answer["features"] == {"switcher": True, "support": True}
+
+
+def test_unknown_stage_counts_as_production(monkeypatch):
+    answer = _mode(monkeypatch, stage="staging")
+    assert answer["stage"] == "prod"
+    assert answer["features"] == {"switcher": False, "support": False}
+
+
+def test_self_hosted_support_follows_the_callers_answer(monkeypatch):
+    """ADR 0022, T11: a self-hosted customer shows support while maintenance runs; support.py decides."""
+    from scalio_platform.app_links import features
+    _mode(monkeypatch)
+    assert features()["support"] is False
+    assert features(self_hosted_support=True)["support"] is True
+    assert app_links(support=True)["features"]["support"] is True
+    _mode(monkeypatch, stage="demo")
+    assert features(self_hosted_support=True)["support"] is False
+
+
+def test_beacon_link_is_kept():
+    """Scalio Beacon is in the switcher (scalio-ui 0.15.0), so its link must not be dropped."""
+    assert parse_app_links("beacon=https://dev.scaliobeacon.com") == {"beacon": "https://dev.scaliobeacon.com"}

scalio_platform-0.1.0/tests/test_error_report.py

@@ -0,0 +1,161 @@
+"""error_report.py: unexpected errors go to Scalio Monitor, only when configured, never with request data.
+Module scalio_platform.error_report (formerly a copy in every Scalio Python app)."""
+import asyncio
+import json
+
+import httpx
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+
+from scalio_platform import error_report
+
+
+class Monitor:
+    def __init__(self, status=200):
+        self.sent, self.status = [], status
+
+    def handler(self, request: httpx.Request) -> httpx.Response:
+        self.sent.append((str(request.url), request.headers.get("authorization"), json.loads(request.content)))
+        return httpx.Response(self.status, json={"ok": True})
+
+
+TOKEN = "tok-123"  # fake
+
+
+def make(monitor, url="https://mon.test", token=None, clock=None):
+    token = TOKEN if token is None else token
+    app = FastAPI()
+
+    @app.get("/api/users/{user_id}")
+    async def boom(user_id: str):
+        raise KeyError(f"user {user_id}")
+
+    @app.get("/ok")
+    async def ok():
+        return {"ok": True}
+
+    kw = {"clock": clock} if clock else {}
+    rep = error_report.install(app, "nexus", lambda: {"version": "1.2.5", "commit": "abc"}, url=url, token=token,
+                               transport=httpx.MockTransport(monitor.handler), **kw)
+    client = TestClient(app, raise_server_exceptions=False)
+    client.__enter__()  # keeps one event loop, so the background report can be awaited
+    return client, rep
+
+
+def call(client, rep, path, **kw):
+    r = client.get(path, **kw)
+    client.portal.call(rep.drain)
+    return r
+
+
+def teardown_function():
+    error_report._reporter = None
+
+
+def test_unhandled_error_is_reported_without_request_data():
+    mon = Monitor()
+    client, rep = make(mon)
+    r = call(client, rep, "/api/users/u-42?token=secret-q", headers={"Cookie": "sid=cookie-secret",
+                                                                       "Authorization": "Bearer user-secret"})
+    assert r.status_code == 500
+    assert len(mon.sent) == 1
+    url, auth, body = mon.sent[0]
+    assert url == "https://mon.test/ingest/error" and auth == "Bearer tok-123"
+    assert body["app"] == "nexus" and body["kind"] == "backend" and body["type"] == "KeyError"
+    assert body["path"] == "/api/users/{user_id}" and body["method"] == "GET" and body["status"] == 500
+    assert body["version"] == "1.2.5" and body["commit"] == "abc" and "boom" in body["stack"]
+    raw = json.dumps(body)
+    for secret in ("secret-q", "cookie-secret", "user-secret"):
+        assert secret not in raw
+
+
+def test_off_without_url_or_token_and_nothing_for_good_requests():
+    mon = Monitor()
+    client, rep = make(mon, token="")
+    assert not rep.enabled
+    call(client, rep, "/api/users/1")
+    client2, rep2 = make(mon)
+    assert call(client2, rep2, "/ok").status_code == 200
+    assert mon.sent == []
+
+
+def test_same_error_once_per_minute():
+    now = [1000.0]
+    mon = Monitor()
+    client, rep = make(mon, clock=lambda: now[0])
+    call(client, rep, "/api/users/1")
+    call(client, rep, "/api/users/2")
+    assert len(mon.sent) == 1
+    now[0] += 61
+    call(client, rep, "/api/users/3")
+    assert len(mon.sent) == 2
+
+
+def test_monitor_down_does_not_change_the_answer(caplog):
+    def down(request):
+        raise httpx.ConnectError("no route", request=request)
+    app = FastAPI()
+
+    @app.get("/x")
+    async def x():
+        raise ValueError("x")
+    rep = error_report.install(app, "nexus", url="https://mon.test", token="tok-123",
+                               transport=httpx.MockTransport(down))
+    with TestClient(app, raise_server_exceptions=False) as client:
+        assert client.get("/x").status_code == 500
+        client.portal.call(rep.drain)
+    assert "tok-123" not in caplog.text
+
+
+def test_route_template_without_a_route():
+    path = "/api/items/123/sub/0123456789abcdef01234567"
+    assert error_report.route_template({"path": path}) == "/api/items/{id}/sub/{id}"
+
+
+def test_background_job_report():
+    mon = Monitor()
+    rep = error_report.Reporter("nexus", url="https://mon.test", token="t", transport=httpx.MockTransport(mon.handler))
+    error_report._reporter = rep
+
+    async def run():
+        try:
+            raise RuntimeError("loop broke")
+        except RuntimeError as exc:
+            await error_report.report_exception(exc, path="lifecycle_task")
+        await rep.drain()
+    asyncio.run(run())
+    assert mon.sent[0][2]["kind"] == "job" and mon.sent[0][2]["path"] == "lifecycle_task"
+
+
+class GroupingMonitor(Monitor):
+    """Answers like Scalio Monitor: the fingerprint of the group the report landed in."""
+
+    def handler(self, request: httpx.Request) -> httpx.Response:
+        super().handler(request)
+        body = json.loads(request.content)
+        return httpx.Response(200, json={"ok": True, "group": "0123456789abcdef" if body["type"] == "KeyError"
+                                         else "fedcba9876543210", "new": True})
+
+
+def test_reported_groups_are_kept_for_support_tickets():
+    """A support ticket lists the last error groups of the instance with a link to the monitor (ADR 0022, T9)."""
+    mon = GroupingMonitor()
+    t = [0.0]
+    client, rep = make(mon, clock=lambda: t[0])
+    call(client, rep, "/api/users/1")
+    t[0] += 120  # past the repeat window
+    call(client, rep, "/api/users/2")
+    groups = error_report.recent_groups()
+    assert len(groups) == 1
+    g = groups[0]
+    assert g["fingerprint"] == "0123456789abcdef" and g["type"] == "KeyError" and g["count"] == 2
+    assert g["route"] == "/api/users/{user_id}" and g["monitor"] == "https://mon.test" and g["time"]
+    assert asyncio.run(error_report.errors_for_support(None)) == groups
+
+
+def test_no_groups_without_reporter_or_with_a_bad_answer():
+    assert error_report.recent_groups() == []
+    mon = Monitor()  # answers without "group"
+    client, rep = make(mon)
+    call(client, rep, "/api/users/1")
+    assert error_report.recent_groups() == []

scalio_platform-0.1.0/tests/test_health.py

@@ -0,0 +1,71 @@
+"""GET /api/health (health.py): the answer the monitoring dashboard reads. Offline (mongomock)."""
+import asyncio
+from datetime import datetime, timedelta, timezone
+
+from mongomock_motor import AsyncMongoMockClient
+
+from scalio_platform import health as H
+
+BUILD = {"version": "1.2.5", "commit": "e93339f", "build_date": "2026-10-27", "service": "backend"}
+NOW = datetime(2026, 10, 27, 12, 0, tzinfo=timezone.utc)
+
+
+def run(coro):
+    return asyncio.run(coro)
+
+
+def test_ok_with_running_jobs():
+    async def go():
+        db = AsyncMongoMockClient()["health_test"]
+        await db.job_leases.insert_one({"_id": "workflow_task", "until": NOW + timedelta(seconds=40),
+                                        "renewedAt": NOW - timedelta(seconds=20), "owner": "a"})
+        return await H.health_report(db, BUILD, lease_jobs=["workflow_task"], now=NOW)
+    code, body = run(go())
+    assert code == 200
+    assert body == {"status": "ok", "version": "1.2.5", "commit": "e93339f", "db": "ok",
+                    "jobs": {"workflow_task": {"ok": True, "last_seen": "2026-10-27T11:59:40+00:00"}}}
+
+
+def test_degraded_when_a_lease_expired_or_is_missing():
+    async def go():
+        db = AsyncMongoMockClient()["health_test"]
+        await db.job_leases.insert_one({"_id": "pam_worker_task", "until": NOW - timedelta(minutes=5),
+                                        "renewedAt": NOW - timedelta(minutes=6)})
+        return await H.health_report(db, BUILD, lease_jobs=["pam_worker_task", "hr_task"], now=NOW)
+    code, body = run(go())
+    assert code == 200 and body["status"] == "degraded"
+    assert body["jobs"]["pam_worker_task"]["ok"] is False
+    assert body["jobs"]["hr_task"] == {"ok": False, "last_seen": None}
+
+
+def test_old_lease_names_do_not_count():
+    async def go():
+        db = AsyncMongoMockClient()["health_test"]
+        await db.job_leases.insert_one({"_id": "removed_loop", "until": NOW - timedelta(days=3)})
+        return await H.health_report(db, BUILD, lease_jobs=[], now=NOW)
+    code, body = run(go())
+    assert code == 200 and body["status"] == "ok" and body["jobs"] == {}
+
+
+class DeadDb:
+    async def command(self, _name):
+        await asyncio.sleep(10)
+
+
+def test_down_with_503_when_the_database_does_not_answer():
+    code, body = run(H.health_report(DeadDb(), BUILD, lease_jobs=["x"], timeout=0.05, now=NOW))
+    assert code == 503
+    assert body == {"status": "down", "version": "1.2.5", "commit": "e93339f", "db": "error", "jobs": {}}
+
+
+def test_no_secrets_or_counts_in_the_answer():
+    async def go():
+        db = AsyncMongoMockClient()["health_test"]
+        return await H.health_report(db, BUILD, now=NOW)
+    _code, body = run(go())
+    assert set(body) == {"status", "version", "commit", "db", "jobs"}
+
+
+def test_app_without_database():
+    code, body = run(H.health_report(None, BUILD, now=NOW))
+    assert code == 200 and body["status"] == "ok" and body["db"] == "none"

scalio_platform-0.1.0/tests/test_job_lease.py

@@ -0,0 +1,129 @@
+"""Background loops run on one backend instance at a time (job_lease.py, architecture review C1).
+Offline (mongomock).
+"""
+import asyncio
+from datetime import datetime, timedelta, timezone
+
+from mongomock_motor import AsyncMongoMockClient
+
+from scalio_platform import job_lease as L
+
+
+def fresh_db():
+    return AsyncMongoMockClient()["lease_test"]
+
+
+def test_only_one_owner_holds_a_lease():
+    async def run():
+        db = fresh_db()
+        assert await L.acquire(db, "workflow_task", "a")
+        assert not await L.acquire(db, "workflow_task", "b")
+        assert await L.acquire(db, "workflow_task", "a")          # renewal by the holder
+        assert await L.acquire(db, "other_task", "b")             # leases are per loop
+    asyncio.run(run())
+
+
+def test_expired_lease_is_taken_over_and_release_frees_it():
+    async def run():
+        db = fresh_db()
+        assert await L.acquire(db, "pam_worker_task", "a")
+        await db.job_leases.update_one({"_id": "pam_worker_task"},
+                                       {"$set": {"until": datetime.now(timezone.utc) - timedelta(seconds=1)}})
+        assert await L.acquire(db, "pam_worker_task", "b")
+        await L.release(db, "pam_worker_task", "a")              # not the holder: no effect
+        assert not await L.acquire(db, "pam_worker_task", "a")
+        await L.release(db, "pam_worker_task", "b")
+        assert await L.acquire(db, "pam_worker_task", "a")
+    asyncio.run(run())
+
+
+def test_two_instances_never_run_the_same_loop_together():
+    """Instance A and B start the same loop side by side (rolling update): only one runs it at any time;
+    when A stops, B takes over."""
+    async def run():
+        db = fresh_db()
+        running, max_running, ran = set(), [0], []
+
+        def make(owner):
+            async def loop():
+                running.add(owner)
+                ran.append(owner)
+                max_running[0] = max(max_running[0], len(running))
+                try:
+                    while True:
+                        await asyncio.sleep(0.01)
+                finally:
+                    running.discard(owner)
+            return loop
+
+        kw = {"ttl": 0.3, "renew": 0.05, "wait": 0.05}
+        a = asyncio.create_task(L.run_exclusive(db, "lifecycle_task", make("a"), owner="a", **kw))
+        await asyncio.sleep(0.05)
+        b = asyncio.create_task(L.run_exclusive(db, "lifecycle_task", make("b"), owner="b", **kw))
+        await asyncio.sleep(0.4)
+        assert running == {"a"} and max_running[0] == 1
+        a.cancel()                          # graceful stop of A releases the lease
+        await asyncio.gather(a, return_exceptions=True)
+        await asyncio.sleep(0.2)
+        assert running == {"b"} and ran == ["a", "b"] and max_running[0] == 1
+        b.cancel()
+        await asyncio.gather(b, return_exceptions=True)
+        assert await db.job_leases.count_documents({}) == 0
+    asyncio.run(run())
+
+
+def test_loop_stops_when_the_lease_is_taken_over():
+    async def run():
+        db = fresh_db()
+        stopped = asyncio.Event()
+
+        async def loop():
+            try:
+                while True:
+                    await asyncio.sleep(0.01)
+            finally:
+                stopped.set()
+
+        t = asyncio.create_task(L.run_exclusive(db, "hr_runtime_task", loop, owner="a", ttl=5, renew=0.05, wait=10))
+        await asyncio.sleep(0.05)
+        until = datetime.now(timezone.utc) + timedelta(seconds=60)
+        await db.job_leases.update_one({"_id": "hr_runtime_task"}, {"$set": {"owner": "b", "until": until}})
+        await asyncio.wait_for(stopped.wait(), timeout=1)
+        t.cancel()
+        await asyncio.gather(t, return_exceptions=True)
+        assert (await db.job_leases.find_one({"_id": "hr_runtime_task"}))["owner"] == "b"
+    asyncio.run(run())
+
+
+def test_short_database_outage_keeps_the_loop_running():
+    async def run():
+        db = fresh_db()
+        ticks = []
+
+        async def loop():
+            while True:
+                ticks.append(1)
+                await asyncio.sleep(0.01)
+
+        real = L.acquire
+        calls = {"n": 0}
+
+        async def flaky(*a, **k):
+            calls["n"] += 1
+            if calls["n"] in (2, 3):
+                raise RuntimeError("mongo unreachable")
+            return await real(*a, **k)
+
+        L.acquire = flaky
+        try:
+            t = asyncio.create_task(L.run_exclusive(db, "exo_task", loop, owner="a", ttl=1, renew=0.05, wait=0.05))
+            await asyncio.sleep(0.3)
+            assert not t.done()
+            before = len(ticks)
+            await asyncio.sleep(0.05)
+            assert len(ticks) > before          # still running after two failed renewals
+            t.cancel()
+            await asyncio.gather(t, return_exceptions=True)
+        finally:
+            L.acquire = real
+    asyncio.run(run())

scalio_platform-0.1.0/tests/test_migrations.py

@@ -0,0 +1,91 @@
+"""Schema migrations (scalio_platform.migrations): each step once, version in db.meta, safe with two instances."""
+import asyncio
+
+import pytest
+from mongomock_motor import AsyncMongoMockClient
+
+from scalio_platform import migrations as M
+
+
+def fresh_db():
+    return AsyncMongoMockClient()["migrations_test"]
+
+
+def test_new_database_runs_every_step_once_and_stores_the_version():
+    async def run():
+        db, calls = fresh_db(), []
+
+        async def m2(db):
+            calls.append(2)
+            await db.items.update_many({}, {"$rename": {"old": "new"}})
+
+        async def m3(db):
+            calls.append(3)
+
+        await db.items.insert_one({"old": 1})
+        steps = M.BASELINE + [(2, m2), (3, m3)]
+        assert await M.schema_version(db) == 0
+        assert await M.run_migrations(db, steps) == 3
+        assert calls == [2, 3]
+        assert await db.items.find_one({}, {"_id": 0}) == {"new": 1}
+        assert await M.run_migrations(db, steps) == 3          # second start: nothing runs again
+        assert calls == [2, 3]
+        doc = await db.meta.find_one({"id": "schema_version"})
+        assert doc["version"] == 3 and doc["appliedAt"]
+    asyncio.run(run())
+
+
+def test_nexus_history_is_kept():
+    """Nexus already stored version 1 in the same document: only newer steps run."""
+    async def run():
+        db, calls = fresh_db(), []
+        await db.meta.insert_one({"id": "schema_version", "version": 1, "appliedAt": "2026-10-27T00:00:00+00:00"})
+
+        async def m2(db):
+            calls.append(2)
+
+        assert await M.run_migrations(db, M.BASELINE + [(2, m2)]) == 2
+        assert calls == [2]
+    asyncio.run(run())
+
+
+def test_failing_step_keeps_the_last_good_version():
+    async def run():
+        db = fresh_db()
+
+        async def broken(db):
+            raise RuntimeError("boom")
+
+        with pytest.raises(RuntimeError):
+            await M.run_migrations(db, M.BASELINE + [(2, broken)])
+        assert await M.schema_version(db) == 1
+    asyncio.run(run())
+
+
+def test_newer_database_is_left_alone():
+    async def run():
+        db = fresh_db()
+        await db.meta.insert_one({"id": "schema_version", "version": 7})
+        assert await M.run_migrations(db, M.BASELINE) == 7
+        assert await M.schema_version(db) == 7
+    asyncio.run(run())
+
+
+def test_version_never_goes_back_when_two_instances_race():
+    async def run():
+        db = fresh_db()
+
+        async def m2(db):
+            await asyncio.sleep(0.01)
+
+        steps = M.BASELINE + [(2, m2)]
+        await asyncio.gather(M.run_migrations(db, steps), M.run_migrations(db, steps))
+        assert await M.schema_version(db) == 2
+        assert await db.meta.count_documents({"id": "schema_version"}) == 1
+    asyncio.run(run())
+
+
+@pytest.mark.parametrize("steps", [[(2, None), (1, None)], [(1, None), (1, None)], [(0, None)]])
+def test_bad_step_lists_are_refused(steps):
+    with pytest.raises(ValueError):
+        asyncio.run(M.run_migrations(fresh_db(), steps))

scalio_platform-0.1.0/tests/test_texts.py

@@ -0,0 +1,10 @@
+"""German text is written with "ss", never the sharp s (project rule)."""
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+
+
+def test_no_sharp_s():
+    sharp = "ß"
+    for p in list(ROOT.glob("src/**/*.py")) + list(ROOT.glob("*.md")):
+        assert sharp not in p.read_text(encoding="utf-8"), p