mobius-operations-trace-py 1.0.0__tar.gz

mobius_operations_trace_py-1.0.0/PKG-INFO

@@ -0,0 +1,82 @@
+Metadata-Version: 2.4
+Name: mobius-operations-trace-py
+Version: 1.0.0
+Summary: Request-flow reconstruction and HUMAN_API replay for Mobius Python (FastAPI) services, built on the span/parent-span action-log fields written by mobius-tracer-py and mobius-auditlog-py.
+Author: Mobius Platform
+Keywords: tracing,replay,request-flow,fastapi,mobius
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: pydantic>=2.0
+Requires-Dist: httpx>=0.24
+Requires-Dist: fastapi>=0.100
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
+Requires-Dist: httpx>=0.24; extra == "dev"
+
+# mobius-operations-trace-py
+
+Request-flow reconstruction and `HUMAN_API` replay for Mobius Python (FastAPI)
+services. Python port of `platform-artifact-lifecycle-lib` +
+`platform-artifact-lifecycle-web` (`MPH-926` / `MPH-928`), verified against
+the Java source (`OperationTraceServiceImpl`, `LifecycleOperationsExceptionHandler`).
+
+Relies on action-log rows carrying `spanid` / `parentservicespanid` /
+`requestid` / `httpmethod` / `httpurl` / `requesterid` / `actionsource`, written
+by [mobius-tracer-py](../mobius-tracer-py) +
+[mobius-auditlog-py](../mobius-auditlog-py).
+
+## What it does
+
+- `get_request_flow(txnid, requestid)` — queries action logs (either
+  identifier alone is enough) and rebuilds the causal call tree (`HUMAN_API`
+  entry point first, then each service hop it triggered, in call order) from
+  the `spanid` / `parentservicespanid` fields.
+- `replay_request(txnid, requestid)` — finds the originating `HUMAN_API` call,
+  exchanges the original requester's identity for a bearer token, and
+  re-issues that exact HTTP call (method/URL/body).
+
+Both are exposed as FastAPI endpoints:
+
+- `GET /v1/operations/request-flows?txnid=...&requestid=...`
+- `POST /v1/operations/request-replays?txnid=...&requestid=...`
+
+Error handling mirrors the Java REST layer:
+
+- Internal failures (missing identifiers, PI lookup failure, missing
+  `HUMAN_API` fields, replay connectivity failure) → HTTP 400 with the same
+  envelope `LifecycleOperationsExceptionHandler` emits (`timestamp`, `origin`,
+  `errorCode`, `errorMessage`, `httpStatusCode`, `actionsRequired`).
+- A replayed target's own response — even a 4xx/5xx — is forwarded verbatim
+  instead of raised, with the outer HTTP status set to match the target's.
+
+## Usage
+
+This package has no hard dependency on a specific action-log store or IAM
+client — you supply small adapters:
+
+```python
+from mobius_operations_trace import setup_operations_trace
+
+setup_operations_trace(
+    app,
+    token_provider=my_token_provider,  # TokenProvider protocol
+    instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+    schema_id="action-log-schema-id",
+)
+```
+
+`instances_list_url` must contain a literal `{schemaId}` placeholder — PI
+takes the schema id as a URL path variable, not a body field (matches
+`lifecycle.operations.trace.instances-list-url` on the Java side). Pass
+`action_log_source` instead of `instances_list_url`/`schema_id` to bypass the
+built-in PI-backed source entirely.
+
+`TokenProvider` has a single method, `retrieve_token_from_tenant_id(tenant_id)`
+— used both to authenticate the PI query (with the fixed
+`PI_SERVICE_TENANT_ID`) and to authenticate a replay as the original requester
+(with the `HUMAN_API` row's `requesterid`), matching the Java side's single
+`IamVaultUtils.retrieveTokenFromTenantId` method.
+
+See `mobius_operations_trace.pi_client.ActionLogSource` and
+`mobius_operations_trace.tokens.TokenProvider` for the full adapter contracts.

mobius_operations_trace_py-1.0.0/README.md

@@ -0,0 +1,66 @@
+# mobius-operations-trace-py
+
+Request-flow reconstruction and `HUMAN_API` replay for Mobius Python (FastAPI)
+services. Python port of `platform-artifact-lifecycle-lib` +
+`platform-artifact-lifecycle-web` (`MPH-926` / `MPH-928`), verified against
+the Java source (`OperationTraceServiceImpl`, `LifecycleOperationsExceptionHandler`).
+
+Relies on action-log rows carrying `spanid` / `parentservicespanid` /
+`requestid` / `httpmethod` / `httpurl` / `requesterid` / `actionsource`, written
+by [mobius-tracer-py](../mobius-tracer-py) +
+[mobius-auditlog-py](../mobius-auditlog-py).
+
+## What it does
+
+- `get_request_flow(txnid, requestid)` — queries action logs (either
+  identifier alone is enough) and rebuilds the causal call tree (`HUMAN_API`
+  entry point first, then each service hop it triggered, in call order) from
+  the `spanid` / `parentservicespanid` fields.
+- `replay_request(txnid, requestid)` — finds the originating `HUMAN_API` call,
+  exchanges the original requester's identity for a bearer token, and
+  re-issues that exact HTTP call (method/URL/body).
+
+Both are exposed as FastAPI endpoints:
+
+- `GET /v1/operations/request-flows?txnid=...&requestid=...`
+- `POST /v1/operations/request-replays?txnid=...&requestid=...`
+
+Error handling mirrors the Java REST layer:
+
+- Internal failures (missing identifiers, PI lookup failure, missing
+  `HUMAN_API` fields, replay connectivity failure) → HTTP 400 with the same
+  envelope `LifecycleOperationsExceptionHandler` emits (`timestamp`, `origin`,
+  `errorCode`, `errorMessage`, `httpStatusCode`, `actionsRequired`).
+- A replayed target's own response — even a 4xx/5xx — is forwarded verbatim
+  instead of raised, with the outer HTTP status set to match the target's.
+
+## Usage
+
+This package has no hard dependency on a specific action-log store or IAM
+client — you supply small adapters:
+
+```python
+from mobius_operations_trace import setup_operations_trace
+
+setup_operations_trace(
+    app,
+    token_provider=my_token_provider,  # TokenProvider protocol
+    instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+    schema_id="action-log-schema-id",
+)
+```
+
+`instances_list_url` must contain a literal `{schemaId}` placeholder — PI
+takes the schema id as a URL path variable, not a body field (matches
+`lifecycle.operations.trace.instances-list-url` on the Java side). Pass
+`action_log_source` instead of `instances_list_url`/`schema_id` to bypass the
+built-in PI-backed source entirely.
+
+`TokenProvider` has a single method, `retrieve_token_from_tenant_id(tenant_id)`
+— used both to authenticate the PI query (with the fixed
+`PI_SERVICE_TENANT_ID`) and to authenticate a replay as the original requester
+(with the `HUMAN_API` row's `requesterid`), matching the Java side's single
+`IamVaultUtils.retrieveTokenFromTenantId` method.
+
+See `mobius_operations_trace.pi_client.ActionLogSource` and
+`mobius_operations_trace.tokens.TokenProvider` for the full adapter contracts.

mobius_operations_trace_py-1.0.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mobius-operations-trace-py"
+version = "1.0.0"
+description = "Request-flow reconstruction and HUMAN_API replay for Mobius Python (FastAPI) services, built on the span/parent-span action-log fields written by mobius-tracer-py and mobius-auditlog-py."
+readme = "README.md"
+requires-python = ">=3.10"
+authors = [{ name = "Mobius Platform" }]
+keywords = ["tracing", "replay", "request-flow", "fastapi", "mobius"]
+dependencies = [
+    "pydantic>=2.0",
+    "httpx>=0.24",
+    "fastapi>=0.100",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.4",
+    "pytest-asyncio>=0.21",
+    "httpx>=0.24",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+mobius_operations_trace = ["py.typed"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+asyncio_mode = "auto"

mobius_operations_trace_py-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/__init__.py

@@ -0,0 +1,32 @@
+"""Request-flow reconstruction and HUMAN_API replay for Mobius Python services.
+
+Python port of ``platform-artifact-lifecycle-lib`` (core logic) and
+``platform-artifact-lifecycle-web`` (REST exposure): given a ``txnid`` /
+``requestid``, rebuild the causal call tree from action-log
+``spanid`` / ``parentservicespanid`` fields, and replay the originating
+``HUMAN_API`` call.
+"""
+from __future__ import annotations
+
+from .bootstrap import setup_operations_trace
+from .errors import DEFAULT_ORIGIN, OperationTraceError, error_envelope
+from .models import PI_SERVICE_TENANT_ID, RequestReplayResponse
+from .pi_client import ActionLogSource, PiActionLogSource
+from .service import OperationTraceService, order_by_span_flow
+from .tokens import TokenProvider
+
+__version__ = "1.0.0"
+
+__all__ = [
+    "setup_operations_trace",
+    "OperationTraceService",
+    "OperationTraceError",
+    "error_envelope",
+    "DEFAULT_ORIGIN",
+    "RequestReplayResponse",
+    "ActionLogSource",
+    "PiActionLogSource",
+    "TokenProvider",
+    "PI_SERVICE_TENANT_ID",
+    "order_by_span_flow",
+]

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/bootstrap.py

@@ -0,0 +1,58 @@
+"""One-call setup for FastAPI services."""
+from __future__ import annotations
+
+from typing import Any, Optional
+
+import httpx
+
+from .errors import DEFAULT_ORIGIN
+from .models import PI_SERVICE_TENANT_ID
+from .pi_client import ActionLogSource, PiActionLogSource
+from .router import router, set_origin, set_service
+from .service import OperationTraceService
+from .tokens import TokenProvider
+
+
+def setup_operations_trace(
+    app: Any,
+    *,
+    token_provider: TokenProvider,
+    action_log_source: Optional[ActionLogSource] = None,
+    instances_list_url: Optional[str] = None,
+    schema_id: Optional[str] = None,
+    service_tenant_id: str = PI_SERVICE_TENANT_ID,
+    origin: str = DEFAULT_ORIGIN,
+    http_client: Optional[httpx.Client] = None,
+) -> OperationTraceService:
+    """Wire request-flow/replay into a FastAPI ``app``.
+
+    Supply either ``action_log_source`` directly, or ``instances_list_url`` +
+    ``schema_id`` to use the default PI-backed source
+    (:class:`~mobius_operations_trace.pi_client.PiActionLogSource`).
+
+    ``instances_list_url`` must contain a literal ``{schemaId}`` placeholder,
+    matching ``lifecycle.operations.trace.instances-list-url`` on the Java
+    side (e.g. ``http://pi-service/v2.0/schemas/{schemaId}/instances/list``).
+    """
+    if action_log_source is None:
+        if not instances_list_url or not schema_id:
+            raise ValueError(
+                "setup_operations_trace requires either action_log_source, "
+                "or both instances_list_url and schema_id"
+            )
+        action_log_source = PiActionLogSource(
+            instances_list_url=instances_list_url,
+            schema_id=schema_id,
+            token_provider=token_provider,
+            service_tenant_id=service_tenant_id,
+        )
+
+    service = OperationTraceService(
+        action_log_source=action_log_source,
+        token_provider=token_provider,
+        http_client=http_client,
+    )
+    set_service(service)
+    set_origin(origin)
+    app.include_router(router)
+    return service

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/errors.py

@@ -0,0 +1,34 @@
+"""Errors raised by request-flow reconstruction and replay."""
+from __future__ import annotations
+
+from datetime import datetime, timezone
+from typing import Any, Dict
+
+DEFAULT_ORIGIN = "mobius-lifecycle-operations"
+
+
+class OperationTraceError(Exception):
+    """Raised for any failure in fetching/reconstructing/replaying a request.
+
+    Mirrors the Java ``LifecycleException`` used for the same failure paths:
+    missing identifiers, PI lookup failure, connectivity issues, and missing
+    required fields on the ``HUMAN_API`` row. Every one of these maps to a
+    generic HTTP 400 on the Java side (``LifecycleOperationsExceptionHandler``)
+    — unlike a replay target's own response, which is forwarded verbatim
+    rather than raised as an exception.
+    """
+
+
+def error_envelope(exc: OperationTraceError, *, origin: str = DEFAULT_ORIGIN) -> Dict[str, Any]:
+    """Build the exact JSON body ``LifecycleOperationsExceptionHandler`` emits
+    for a ``LifecycleException``, for parity with the Java REST layer."""
+    return {
+        "timestamp": datetime.now(timezone.utc).isoformat(timespec="milliseconds").replace(
+            "+00:00", "Z"
+        ),
+        "origin": origin,
+        "errorCode": "LIFECYCLE_ERROR",
+        "errorMessage": str(exc),
+        "httpStatusCode": "400",
+        "actionsRequired": [],
+    }

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/models.py

@@ -0,0 +1,56 @@
+"""Data shapes for request-flow reconstruction and replay.
+
+Action-log rows are kept as plain ``dict``s (mirroring the Java
+``List<Map<String, Object>>``) rather than a strict model, since the PI
+action-log schema can carry additional fields beyond the ones this package
+reads.
+"""
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+from pydantic import BaseModel
+
+# Field names read off an action-log row, verified against
+# OperationTraceServiceImpl.java's own constants — all lowercase, no
+# separators. PI's ingestion/storage layer folds the camelCase
+# ActionLogCreateRequest fields (`spanId`, `httpUrl`, ...) mobius-acl-lib
+# publishes down to these lowercase columns; that folding happens in PI
+# itself, not in either Java repo.
+FIELD_SPAN_ID = "spanid"
+FIELD_PARENT_SPAN_ID = "parentservicespanid"
+FIELD_REQUEST_ID = "requestid"
+FIELD_TIMESTAMP = "timestamp"  # epoch milliseconds (System.currentTimeMillis())
+FIELD_HTTP_METHOD = "httpmethod"
+FIELD_HTTP_URL = "httpurl"  # flat/top-level, not nested under any envelope
+FIELD_ACTION_SOURCE = "actionsource"
+FIELD_REQUESTER_ID = "requesterid"
+FIELD_REQUEST_OBJECT = "requestobject"  # flat/top-level
+
+HUMAN_API = "HUMAN_API"
+
+ActionLogRow = Dict[str, Any]
+
+# HTTP methods that carry a body when replayed.
+BODY_METHODS = frozenset({"POST", "PUT", "PATCH", "DELETE"})
+
+# --- PI "instances list" query contract (OperationTraceServiceImpl.java) ---
+
+# Filter keys sent in the PI query body — distinct from any row field name;
+# these are never read back off a returned row.
+FILTER_KEY_TXNID = "txnid"
+FILTER_KEY_REQUESTID = "requestid"
+
+REQUEST_KEY_DB_TYPE = "dbType"  # the one camelCase key in the PI request body
+REQUEST_KEY_FILTER = "filter"
+DB_TYPE_TIDB = "TIDB"
+
+# Fixed platform service-tenant id used to authenticate the PI query call —
+# the same shared constant mobius-auditlog-py defaults to for its tenant id.
+PI_SERVICE_TENANT_ID = "2cf76e5f-26ad-4f2c-bccc-f4bc1e7bfb64"
+
+
+class RequestReplayResponse(BaseModel):
+    request_id: Optional[str] = None
+    status_code: Optional[int] = None
+    response: Any = None

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/pi_client.py

@@ -0,0 +1,101 @@
+"""Action-log lookup.
+
+Defines the ``ActionLogSource`` contract (so tests / alternate stores can
+substitute their own) and a default HTTP implementation that queries the PI
+"instances list" API for the action-log schema, mirroring
+``OperationTraceServiceImpl.fetchActionLogs`` on the Java side.
+
+Verified contract (see ``OperationTraceServiceImpl.java`` / its test fixture):
+- ``schemaId`` is a URL *path* variable — the configured
+  ``instances-list-url`` contains a literal ``{schemaId}`` placeholder that
+  gets expanded, e.g. ``http://pi-service/v2.0/schemas/{schemaId}/instances/list``.
+- Request body is exactly ``{"dbType": "TIDB", "filter": {...}}`` — no
+  ``schemaId`` in the body. ``filter`` only includes whichever of
+  ``txnid`` / ``requestid`` are non-blank.
+- Response is a **bare JSON array** of row dicts, never wrapped in an
+  envelope. An empty array means "not found".
+"""
+from __future__ import annotations
+
+import logging
+from typing import Dict, List, Optional, Protocol, runtime_checkable
+
+import httpx
+
+from .errors import OperationTraceError
+from .models import (
+    DB_TYPE_TIDB,
+    FILTER_KEY_REQUESTID,
+    FILTER_KEY_TXNID,
+    PI_SERVICE_TENANT_ID,
+    REQUEST_KEY_DB_TYPE,
+    REQUEST_KEY_FILTER,
+    ActionLogRow,
+)
+from .tokens import TokenProvider
+
+log = logging.getLogger(__name__)
+
+
+@runtime_checkable
+class ActionLogSource(Protocol):
+    def fetch_action_logs(
+        self, txnid: Optional[str], requestid: Optional[str]
+    ) -> List[ActionLogRow]:
+        ...
+
+
+class PiActionLogSource:
+    """Default ``ActionLogSource``: queries PI over HTTP."""
+
+    def __init__(
+        self,
+        *,
+        instances_list_url: str,
+        schema_id: str,
+        token_provider: TokenProvider,
+        service_tenant_id: str = PI_SERVICE_TENANT_ID,
+        db_type: str = DB_TYPE_TIDB,
+        client: Optional[httpx.Client] = None,
+        timeout: float = 10.0,
+    ) -> None:
+        self._url = instances_list_url.replace("{schemaId}", schema_id)
+        self._tokens = token_provider
+        self._service_tenant_id = service_tenant_id
+        self._db_type = db_type
+        self._client = client or httpx.Client(timeout=timeout)
+
+    def fetch_action_logs(
+        self, txnid: Optional[str], requestid: Optional[str]
+    ) -> List[ActionLogRow]:
+        filter_: Dict[str, str] = {}
+        if txnid:
+            filter_[FILTER_KEY_TXNID] = txnid
+        if requestid:
+            filter_[FILTER_KEY_REQUESTID] = requestid
+
+        payload = {REQUEST_KEY_DB_TYPE: self._db_type, REQUEST_KEY_FILTER: filter_}
+
+        try:
+            token = self._tokens.retrieve_token_from_tenant_id(self._service_tenant_id)
+            resp = self._client.post(
+                self._url,
+                json=payload,
+                headers={"Authorization": f"Bearer {token}"},
+            )
+            resp.raise_for_status()
+        except httpx.HTTPError as exc:
+            log.error(
+                "PI action-log lookup failed for txnid=%s requestid=%s", txnid, requestid,
+                exc_info=True,
+            )
+            raise OperationTraceError("Unable to fetch request action logs") from exc
+
+        body = resp.json()
+        if not isinstance(body, list):
+            log.error(
+                "Unexpected PI response shape for txnid=%s requestid=%s: %r",
+                txnid, requestid, type(body),
+            )
+            raise OperationTraceError("Unable to fetch request action logs")
+        return body

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/router.py

@@ -0,0 +1,65 @@
+"""FastAPI REST exposure.
+
+Python port of ``IOperationController`` / ``OperationControllerImpl``:
+
+- ``GET /v1/operations/request-flows?txnid=...&requestid=...``
+- ``POST /v1/operations/request-replays?txnid=...&requestid=...``
+
+Both delegate straight to the configured :class:`OperationTraceService`.
+
+Error handling mirrors ``LifecycleOperationsExceptionHandler``: any
+:class:`OperationTraceError` (missing identifiers, PI lookup failure, missing
+``HUMAN_API`` fields, replay connectivity failure) maps to a generic HTTP 400
+with the Java error envelope. A replayed target's own response — even a 4xx/5xx
+— is forwarded verbatim instead, with the outer HTTP status set to match.
+"""
+from __future__ import annotations
+
+from typing import Optional
+
+from fastapi import APIRouter
+from fastapi.responses import JSONResponse
+
+from .errors import DEFAULT_ORIGIN, OperationTraceError, error_envelope
+from .service import OperationTraceService
+
+router = APIRouter(prefix="/v1/operations", tags=["operations"])
+
+_default_service: Optional[OperationTraceService] = None
+_origin: str = DEFAULT_ORIGIN
+
+
+def set_service(service: Optional[OperationTraceService]) -> None:
+    global _default_service
+    _default_service = service
+
+
+def set_origin(origin: str) -> None:
+    global _origin
+    _origin = origin
+
+
+def get_service() -> OperationTraceService:
+    if _default_service is None:
+        raise RuntimeError(
+            "mobius-operations-trace: no OperationTraceService configured. "
+            "Call setup_operations_trace(app, ...) first."
+        )
+    return _default_service
+
+
+@router.get("/request-flows", response_model=None)
+def get_request_flows(txnid: Optional[str] = None, requestid: Optional[str] = None):
+    try:
+        return get_service().get_request_flow(txnid, requestid)
+    except OperationTraceError as exc:
+        return JSONResponse(status_code=400, content=error_envelope(exc, origin=_origin))
+
+
+@router.post("/request-replays", response_model=None)
+def post_request_replays(txnid: Optional[str] = None, requestid: Optional[str] = None):
+    try:
+        result = get_service().replay_request(txnid, requestid)
+    except OperationTraceError as exc:
+        return JSONResponse(status_code=400, content=error_envelope(exc, origin=_origin))
+    return JSONResponse(status_code=result.status_code or 200, content=result.model_dump())

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/service.py

@@ -0,0 +1,171 @@
+"""Request-flow reconstruction and replay.
+
+Python port of ``OperationTraceServiceImpl``:
+
+- :func:`order_by_span_flow` rebuilds the causal call tree from the flat list
+  of action-log rows using ``spanid`` / ``parentservicespanid`` — the
+  ``HUMAN_API`` row(s) first, then each service hop in call order, orphans
+  appended at the end.
+- :class:`OperationTraceService` wraps that with the PI lookup
+  (``get_request_flow``) and re-issuing the original ``HUMAN_API`` call as the
+  original requester (``replay_request``).
+"""
+from __future__ import annotations
+
+import logging
+from typing import Any, Dict, List, Optional
+
+import httpx
+
+from .errors import OperationTraceError
+from .models import (
+    BODY_METHODS,
+    FIELD_ACTION_SOURCE,
+    FIELD_HTTP_METHOD,
+    FIELD_HTTP_URL,
+    FIELD_PARENT_SPAN_ID,
+    FIELD_REQUEST_OBJECT,
+    FIELD_REQUESTER_ID,
+    FIELD_SPAN_ID,
+    FIELD_TIMESTAMP,
+    HUMAN_API,
+    ActionLogRow,
+    RequestReplayResponse,
+)
+from .pi_client import ActionLogSource
+from .tokens import TokenProvider
+
+log = logging.getLogger(__name__)
+
+
+def _timestamp_ms(row: ActionLogRow) -> int:
+    """``timestamp`` is epoch milliseconds (``System.currentTimeMillis()`` on
+    the write side); tolerate it arriving as a number or a numeric string."""
+    value = row.get(FIELD_TIMESTAMP)
+    if isinstance(value, (int, float)):
+        return int(value)
+    try:
+        return int(value)  # type: ignore[arg-type]
+    except (TypeError, ValueError):
+        return 0
+
+
+def order_by_span_flow(rows: List[ActionLogRow]) -> List[ActionLogRow]:
+    indexed = list(enumerate(rows))
+
+    children_by_parent: Dict[Any, List[tuple]] = {}
+    for idx, row in indexed:
+        parent = row.get(FIELD_PARENT_SPAN_ID)
+        children_by_parent.setdefault(parent, []).append((idx, row))
+    for children in children_by_parent.values():
+        children.sort(key=lambda pair: (_timestamp_ms(pair[1]), pair[0]))
+
+    roots = [
+        (idx, row)
+        for idx, row in indexed
+        if str(row.get(FIELD_ACTION_SOURCE, "")).upper() == HUMAN_API
+    ]
+    if not roots:
+        roots = [(idx, row) for idx, row in indexed if not row.get(FIELD_PARENT_SPAN_ID)]
+    roots.sort(key=lambda pair: (_timestamp_ms(pair[1]), pair[0]))
+
+    ordered: List[ActionLogRow] = []
+    visited: set = set()
+
+    def visit(idx: int, row: ActionLogRow) -> None:
+        if idx in visited:
+            return
+        visited.add(idx)
+        ordered.append(row)
+        for child_idx, child_row in children_by_parent.get(row.get(FIELD_SPAN_ID), []):
+            visit(child_idx, child_row)
+
+    for idx, row in roots:
+        visit(idx, row)
+
+    orphans = [(idx, row) for idx, row in indexed if idx not in visited]
+    orphans.sort(key=lambda pair: (_timestamp_ms(pair[1]), pair[0]))
+    ordered.extend(row for _, row in orphans)
+    return ordered
+
+
+class OperationTraceService:
+    def __init__(
+        self,
+        *,
+        action_log_source: ActionLogSource,
+        token_provider: TokenProvider,
+        http_client: Optional[httpx.Client] = None,
+    ) -> None:
+        self._logs = action_log_source
+        self._tokens = token_provider
+        self._client = http_client or httpx.Client(timeout=30.0)
+
+    def get_request_flow(
+        self, txnid: Optional[str], requestid: Optional[str]
+    ) -> List[ActionLogRow]:
+        # PI's filter accepts either identifier alone; only reject when both
+        # are blank (mirrors OperationTraceServiceImpl's own filter-building).
+        if not txnid and not requestid:
+            raise OperationTraceError("At least one of txnid or requestid is required")
+
+        rows = self._logs.fetch_action_logs(txnid, requestid)
+        if not rows:
+            raise OperationTraceError(
+                f"No action logs found for txnid={txnid}, requestid={requestid}"
+            )
+        return order_by_span_flow(rows)
+
+    def replay_request(
+        self, txnid: Optional[str], requestid: Optional[str]
+    ) -> RequestReplayResponse:
+        ordered = self.get_request_flow(txnid, requestid)
+        human_api_row = next(
+            (
+                row
+                for row in ordered
+                if str(row.get(FIELD_ACTION_SOURCE, "")).upper() == HUMAN_API
+            ),
+            None,
+        )
+        if human_api_row is None:
+            raise OperationTraceError(
+                f"No HUMAN_API entry point found for txnid={txnid}, requestid={requestid}"
+            )
+
+        requester_id = human_api_row.get(FIELD_REQUESTER_ID)
+        if not requester_id:
+            raise OperationTraceError("HUMAN_API action-log row is missing requesterid")
+
+        method = human_api_row.get(FIELD_HTTP_METHOD)
+        url = human_api_row.get(FIELD_HTTP_URL)
+        if not method or not url:
+            raise OperationTraceError("HUMAN_API action-log row is missing httpmethod/httpurl")
+
+        try:
+            token = self._tokens.retrieve_token_from_tenant_id(requester_id)
+        except Exception as exc:  # noqa: BLE001 - surfaced as a descriptive trace error
+            raise OperationTraceError(
+                f"Failed to exchange token for requester {requester_id}"
+            ) from exc
+
+        kwargs: Dict[str, Any] = {"headers": {"Authorization": f"Bearer {token}"}}
+        if method.upper() in BODY_METHODS:
+            kwargs["json"] = human_api_row.get(FIELD_REQUEST_OBJECT)
+
+        # A non-2xx response from the replayed call is NOT an error here - it
+        # is forwarded verbatim (status + body), mirroring the Java side's
+        # RestClientResponseException handling for the target call. Only a
+        # genuine transport/connectivity failure raises OperationTraceError.
+        try:
+            resp = self._client.request(method.upper(), url, **kwargs)
+        except httpx.HTTPError as exc:
+            log.error("Replay call failed for requestid=%s", requestid, exc_info=True)
+            raise OperationTraceError(f"Failed to replay request {requestid}") from exc
+
+        try:
+            body = resp.json()
+        except ValueError:
+            body = resp.text
+
+        return RequestReplayResponse(request_id=requestid, status_code=resp.status_code, response=body)

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace/tokens.py

@@ -0,0 +1,26 @@
+"""Token acquisition contract.
+
+This package has no hard dependency on a specific IAM/vault client (mirrors
+the Java side's ``IamVaultUtils``) — each service supplies a small adapter.
+
+Verified against the Java source: there is exactly one method,
+``retrieveTokenFromTenantId(tenantId)``, used identically for both the fixed
+service tenant (to authenticate the PI query) and a row's ``requesterid``
+(to authenticate a replay as the original requester) — no other parameters
+(requester type, separate tenant id) are involved.
+"""
+from __future__ import annotations
+
+from typing import Protocol, runtime_checkable
+
+
+@runtime_checkable
+class TokenProvider(Protocol):
+    def retrieve_token_from_tenant_id(self, tenant_id: str) -> str:
+        """Bearer token for the given tenant id.
+
+        Called with :data:`~mobius_operations_trace.models.PI_SERVICE_TENANT_ID`
+        to authenticate the PI query, and with a ``HUMAN_API`` row's
+        ``requesterid`` to authenticate a replay as that requester.
+        """
+        ...

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace_py.egg-info/PKG-INFO

@@ -0,0 +1,82 @@
+Metadata-Version: 2.4
+Name: mobius-operations-trace-py
+Version: 1.0.0
+Summary: Request-flow reconstruction and HUMAN_API replay for Mobius Python (FastAPI) services, built on the span/parent-span action-log fields written by mobius-tracer-py and mobius-auditlog-py.
+Author: Mobius Platform
+Keywords: tracing,replay,request-flow,fastapi,mobius
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: pydantic>=2.0
+Requires-Dist: httpx>=0.24
+Requires-Dist: fastapi>=0.100
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
+Requires-Dist: httpx>=0.24; extra == "dev"
+
+# mobius-operations-trace-py
+
+Request-flow reconstruction and `HUMAN_API` replay for Mobius Python (FastAPI)
+services. Python port of `platform-artifact-lifecycle-lib` +
+`platform-artifact-lifecycle-web` (`MPH-926` / `MPH-928`), verified against
+the Java source (`OperationTraceServiceImpl`, `LifecycleOperationsExceptionHandler`).
+
+Relies on action-log rows carrying `spanid` / `parentservicespanid` /
+`requestid` / `httpmethod` / `httpurl` / `requesterid` / `actionsource`, written
+by [mobius-tracer-py](../mobius-tracer-py) +
+[mobius-auditlog-py](../mobius-auditlog-py).
+
+## What it does
+
+- `get_request_flow(txnid, requestid)` — queries action logs (either
+  identifier alone is enough) and rebuilds the causal call tree (`HUMAN_API`
+  entry point first, then each service hop it triggered, in call order) from
+  the `spanid` / `parentservicespanid` fields.
+- `replay_request(txnid, requestid)` — finds the originating `HUMAN_API` call,
+  exchanges the original requester's identity for a bearer token, and
+  re-issues that exact HTTP call (method/URL/body).
+
+Both are exposed as FastAPI endpoints:
+
+- `GET /v1/operations/request-flows?txnid=...&requestid=...`
+- `POST /v1/operations/request-replays?txnid=...&requestid=...`
+
+Error handling mirrors the Java REST layer:
+
+- Internal failures (missing identifiers, PI lookup failure, missing
+  `HUMAN_API` fields, replay connectivity failure) → HTTP 400 with the same
+  envelope `LifecycleOperationsExceptionHandler` emits (`timestamp`, `origin`,
+  `errorCode`, `errorMessage`, `httpStatusCode`, `actionsRequired`).
+- A replayed target's own response — even a 4xx/5xx — is forwarded verbatim
+  instead of raised, with the outer HTTP status set to match the target's.
+
+## Usage
+
+This package has no hard dependency on a specific action-log store or IAM
+client — you supply small adapters:
+
+```python
+from mobius_operations_trace import setup_operations_trace
+
+setup_operations_trace(
+    app,
+    token_provider=my_token_provider,  # TokenProvider protocol
+    instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+    schema_id="action-log-schema-id",
+)
+```
+
+`instances_list_url` must contain a literal `{schemaId}` placeholder — PI
+takes the schema id as a URL path variable, not a body field (matches
+`lifecycle.operations.trace.instances-list-url` on the Java side). Pass
+`action_log_source` instead of `instances_list_url`/`schema_id` to bypass the
+built-in PI-backed source entirely.
+
+`TokenProvider` has a single method, `retrieve_token_from_tenant_id(tenant_id)`
+— used both to authenticate the PI query (with the fixed
+`PI_SERVICE_TENANT_ID`) and to authenticate a replay as the original requester
+(with the `HUMAN_API` row's `requesterid`), matching the Java side's single
+`IamVaultUtils.retrieveTokenFromTenantId` method.
+
+See `mobius_operations_trace.pi_client.ActionLogSource` and
+`mobius_operations_trace.tokens.TokenProvider` for the full adapter contracts.

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace_py.egg-info/SOURCES.txt

@@ -0,0 +1,18 @@
+README.md
+pyproject.toml
+src/mobius_operations_trace/__init__.py
+src/mobius_operations_trace/bootstrap.py
+src/mobius_operations_trace/errors.py
+src/mobius_operations_trace/models.py
+src/mobius_operations_trace/pi_client.py
+src/mobius_operations_trace/py.typed
+src/mobius_operations_trace/router.py
+src/mobius_operations_trace/service.py
+src/mobius_operations_trace/tokens.py
+src/mobius_operations_trace_py.egg-info/PKG-INFO
+src/mobius_operations_trace_py.egg-info/SOURCES.txt
+src/mobius_operations_trace_py.egg-info/dependency_links.txt
+src/mobius_operations_trace_py.egg-info/requires.txt
+src/mobius_operations_trace_py.egg-info/top_level.txt
+tests/test_operations_trace.py
+tests/test_pi_client.py

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace_py.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace_py.egg-info/requires.txt

@@ -0,0 +1,8 @@
+pydantic>=2.0
+httpx>=0.24
+fastapi>=0.100
+
+[dev]
+pytest>=7.4
+pytest-asyncio>=0.21
+httpx>=0.24

mobius_operations_trace_py-1.0.0/src/mobius_operations_trace_py.egg-info/top_level.txt

@@ -0,0 +1 @@
+mobius_operations_trace

mobius_operations_trace_py-1.0.0/tests/test_operations_trace.py

@@ -0,0 +1,277 @@
+from __future__ import annotations
+
+import pytest
+from fastapi import FastAPI
+from starlette.testclient import TestClient
+
+from mobius_operations_trace import (
+    OperationTraceError,
+    OperationTraceService,
+    RequestReplayResponse,
+    order_by_span_flow,
+    setup_operations_trace,
+)
+from mobius_operations_trace.models import HUMAN_API
+
+
+def _row(**kw):
+    base = {
+        "requestid": "req-1",
+        "timestamp": 0,
+    }
+    base.update(kw)
+    return base
+
+
+def test_order_by_span_flow_reconstructs_call_tree():
+    rows = [
+        _row(spanid="child-2", parentservicespanid="root", actionsource="SERVICE", timestamp=2000),
+        _row(spanid="root", parentservicespanid=None, actionsource=HUMAN_API, timestamp=1000),
+        _row(spanid="child-1", parentservicespanid="root", actionsource="SERVICE", timestamp=1500),
+        _row(spanid="grandchild", parentservicespanid="child-1", actionsource="SERVICE", timestamp=1600),
+    ]
+
+    ordered = order_by_span_flow(rows)
+
+    assert [row["spanid"] for row in ordered] == ["root", "child-1", "grandchild", "child-2"]
+
+
+def test_order_by_span_flow_tolerates_numeric_string_timestamps():
+    rows = [
+        _row(spanid="root", parentservicespanid=None, actionsource=HUMAN_API, timestamp="1000"),
+        _row(spanid="child", parentservicespanid="root", actionsource="SERVICE", timestamp="2000"),
+    ]
+
+    ordered = order_by_span_flow(rows)
+
+    assert [row["spanid"] for row in ordered] == ["root", "child"]
+
+
+def test_order_by_span_flow_falls_back_to_no_parent_when_no_human_api_root():
+    rows = [
+        _row(spanid="a", parentservicespanid=None, actionsource="SERVICE", timestamp=1000),
+        _row(spanid="b", parentservicespanid="a", actionsource="SERVICE", timestamp=2000),
+    ]
+
+    ordered = order_by_span_flow(rows)
+
+    assert [row["spanid"] for row in ordered] == ["a", "b"]
+
+
+def test_order_by_span_flow_appends_orphans_at_end():
+    rows = [
+        _row(spanid="root", parentservicespanid=None, actionsource=HUMAN_API, timestamp=1000),
+        _row(spanid="orphan", parentservicespanid="missing-parent", actionsource="SERVICE", timestamp=2000),
+    ]
+
+    ordered = order_by_span_flow(rows)
+
+    assert [row["spanid"] for row in ordered] == ["root", "orphan"]
+
+
+class _FakeActionLogSource:
+    def __init__(self, rows):
+        self.rows = rows
+        self.calls = []
+
+    def fetch_action_logs(self, txnid, requestid):
+        self.calls.append((txnid, requestid))
+        return self.rows
+
+
+class _FakeTokenProvider:
+    def retrieve_token_from_tenant_id(self, tenant_id):
+        return f"token-for-{tenant_id}"
+
+
+def test_get_request_flow_requires_at_least_one_identifier():
+    service = OperationTraceService(
+        action_log_source=_FakeActionLogSource([]), token_provider=_FakeTokenProvider()
+    )
+    with pytest.raises(OperationTraceError):
+        service.get_request_flow(None, None)
+
+
+def test_get_request_flow_allows_a_single_identifier():
+    source = _FakeActionLogSource([_row(spanid="root", actionsource=HUMAN_API)])
+    service = OperationTraceService(action_log_source=source, token_provider=_FakeTokenProvider())
+
+    service.get_request_flow("txn-1", None)
+
+    assert source.calls == [("txn-1", None)]
+
+
+def test_get_request_flow_raises_when_no_rows_found():
+    service = OperationTraceService(
+        action_log_source=_FakeActionLogSource([]), token_provider=_FakeTokenProvider()
+    )
+    with pytest.raises(OperationTraceError):
+        service.get_request_flow("txn-1", "req-1")
+
+
+def test_replay_request_reissues_the_human_api_call(monkeypatch):
+    human_row = _row(
+        spanid="root",
+        parentservicespanid=None,
+        actionsource=HUMAN_API,
+        requesterid="user-42",
+        httpmethod="POST",
+        httpurl="https://svc.internal/v1/widgets",
+        requestobject={"name": "widget"},
+    )
+    service = OperationTraceService(
+        action_log_source=_FakeActionLogSource([human_row]),
+        token_provider=_FakeTokenProvider(),
+    )
+
+    calls = {}
+
+    class _FakeResponse:
+        status_code = 201
+
+        def json(self):
+            return {"id": "widget-1"}
+
+    def fake_request(method, url, **kwargs):
+        calls["method"] = method
+        calls["url"] = url
+        calls["kwargs"] = kwargs
+        return _FakeResponse()
+
+    monkeypatch.setattr(service._client, "request", fake_request)
+
+    result = service.replay_request("txn-1", "req-1")
+
+    assert isinstance(result, RequestReplayResponse)
+    assert result.status_code == 201
+    assert result.response == {"id": "widget-1"}
+    assert calls["method"] == "POST"
+    assert calls["url"] == "https://svc.internal/v1/widgets"
+    assert calls["kwargs"]["headers"]["Authorization"] == "Bearer token-for-user-42"
+    assert calls["kwargs"]["json"] == {"name": "widget"}
+
+
+def test_replay_request_forwards_target_error_response_verbatim(monkeypatch):
+    """A non-2xx from the replayed target is forwarded as-is, not raised."""
+    human_row = _row(
+        spanid="root",
+        parentservicespanid=None,
+        actionsource=HUMAN_API,
+        requesterid="user-42",
+        httpmethod="GET",
+        httpurl="https://svc.internal/v1/widgets/missing",
+    )
+    service = OperationTraceService(
+        action_log_source=_FakeActionLogSource([human_row]),
+        token_provider=_FakeTokenProvider(),
+    )
+
+    class _FakeErrorResponse:
+        status_code = 404
+
+        def json(self):
+            return {"error": "not found"}
+
+    monkeypatch.setattr(
+        service._client, "request", lambda method, url, **kw: _FakeErrorResponse()
+    )
+
+    result = service.replay_request("txn-1", "req-1")
+
+    assert result.status_code == 404
+    assert result.response == {"error": "not found"}
+
+
+def test_replay_request_raises_when_human_api_row_missing_url():
+    human_row = _row(
+        spanid="root",
+        parentservicespanid=None,
+        actionsource=HUMAN_API,
+        requesterid="user-42",
+    )
+    service = OperationTraceService(
+        action_log_source=_FakeActionLogSource([human_row]),
+        token_provider=_FakeTokenProvider(),
+    )
+    with pytest.raises(OperationTraceError):
+        service.replay_request("txn-1", "req-1")
+
+
+def test_router_exposes_request_flows_and_replays():
+    human_row = _row(
+        spanid="root",
+        parentservicespanid=None,
+        actionsource=HUMAN_API,
+        requesterid="user-42",
+        httpmethod="GET",
+        httpurl="https://svc.internal/v1/widgets/1",
+    )
+
+    app = FastAPI()
+    setup_operations_trace(
+        app,
+        action_log_source=_FakeActionLogSource([human_row]),
+        token_provider=_FakeTokenProvider(),
+    )
+    client = TestClient(app)
+
+    flow_resp = client.get(
+        "/v1/operations/request-flows", params={"txnid": "txn-1", "requestid": "req-1"}
+    )
+    assert flow_resp.status_code == 200
+    assert flow_resp.json()[0]["spanid"] == "root"
+
+
+def test_router_returns_java_parity_error_envelope_on_operation_trace_error():
+    app = FastAPI()
+    setup_operations_trace(
+        app,
+        action_log_source=_FakeActionLogSource([]),
+        token_provider=_FakeTokenProvider(),
+    )
+    client = TestClient(app)
+
+    resp = client.get(
+        "/v1/operations/request-flows", params={"txnid": "txn-1", "requestid": "req-1"}
+    )
+    assert resp.status_code == 400
+    body = resp.json()
+    assert body["origin"] == "mobius-lifecycle-operations"
+    assert body["errorCode"] == "LIFECYCLE_ERROR"
+    assert body["httpStatusCode"] == "400"
+    assert body["actionsRequired"] == []
+    assert "timestamp" in body and "errorMessage" in body
+
+
+def test_router_replay_sets_outer_status_to_target_status(monkeypatch):
+    human_row = _row(
+        spanid="root",
+        parentservicespanid=None,
+        actionsource=HUMAN_API,
+        requesterid="user-42",
+        httpmethod="GET",
+        httpurl="https://svc.internal/v1/widgets/missing",
+    )
+    app = FastAPI()
+    service = setup_operations_trace(
+        app,
+        action_log_source=_FakeActionLogSource([human_row]),
+        token_provider=_FakeTokenProvider(),
+    )
+
+    class _FakeErrorResponse:
+        status_code = 404
+
+        def json(self):
+            return {"error": "not found"}
+
+    monkeypatch.setattr(
+        service._client, "request", lambda method, url, **kw: _FakeErrorResponse()
+    )
+    client = TestClient(app)
+
+    resp = client.post(
+        "/v1/operations/request-replays", params={"txnid": "txn-1", "requestid": "req-1"}
+    )
+    assert resp.status_code == 404
+    assert resp.json()["response"] == {"error": "not found"}

mobius_operations_trace_py-1.0.0/tests/test_pi_client.py

@@ -0,0 +1,130 @@
+from __future__ import annotations
+
+import httpx
+import pytest
+
+from mobius_operations_trace import OperationTraceError, PI_SERVICE_TENANT_ID
+from mobius_operations_trace.pi_client import PiActionLogSource
+
+
+class _FakeTokenProvider:
+    def __init__(self):
+        self.seen_tenant_ids = []
+
+    def retrieve_token_from_tenant_id(self, tenant_id):
+        self.seen_tenant_ids.append(tenant_id)
+        return f"token-for-{tenant_id}"
+
+
+def _client_returning(json_body, status_code=200):
+    def handler(request: httpx.Request) -> httpx.Response:
+        return httpx.Response(status_code, json=json_body, request=request)
+
+    return httpx.Client(transport=httpx.MockTransport(handler))
+
+
+def test_schema_id_is_interpolated_into_the_url_path_not_the_body():
+    seen = {}
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        import json as _json
+
+        seen["url"] = str(request.url)
+        seen["payload"] = _json.loads(request.content)
+        return httpx.Response(200, json=[])
+
+    client = httpx.Client(transport=httpx.MockTransport(handler))
+    tokens = _FakeTokenProvider()
+    source = PiActionLogSource(
+        instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+        schema_id="action-log-schema",
+        token_provider=tokens,
+        client=client,
+    )
+
+    source.fetch_action_logs("txn-1", "req-1")
+
+    assert seen["url"] == "http://pi-service/v2.0/schemas/action-log-schema/instances/list"
+    assert seen["payload"] == {
+        "dbType": "TIDB",
+        "filter": {"txnid": "txn-1", "requestid": "req-1"},
+    }
+    assert "schemaId" not in seen["payload"]
+
+
+def test_filter_only_includes_non_blank_identifiers():
+    captured = {}
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        import json as _json
+
+        captured["payload"] = _json.loads(request.content)
+        return httpx.Response(200, json=[])
+
+    client = httpx.Client(transport=httpx.MockTransport(handler))
+    source = PiActionLogSource(
+        instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+        schema_id="action-log-schema",
+        token_provider=_FakeTokenProvider(),
+        client=client,
+    )
+
+    source.fetch_action_logs("txn-1", None)
+
+    assert captured["payload"]["filter"] == {"txnid": "txn-1"}
+
+
+def test_uses_service_tenant_id_for_authentication_token():
+    tokens = _FakeTokenProvider()
+    client = _client_returning([])
+    source = PiActionLogSource(
+        instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+        schema_id="action-log-schema",
+        token_provider=tokens,
+        client=client,
+    )
+
+    source.fetch_action_logs("txn-1", "req-1")
+
+    assert tokens.seen_tenant_ids == [PI_SERVICE_TENANT_ID]
+
+
+def test_bare_array_response_is_returned_directly():
+    rows = [{"spanid": "root"}]
+    client = _client_returning(rows)
+    source = PiActionLogSource(
+        instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+        schema_id="action-log-schema",
+        token_provider=_FakeTokenProvider(),
+        client=client,
+    )
+
+    result = source.fetch_action_logs("txn-1", "req-1")
+
+    assert result == rows
+
+
+def test_non_array_response_raises_operation_trace_error():
+    client = _client_returning({"data": [{"spanid": "root"}]})
+    source = PiActionLogSource(
+        instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+        schema_id="action-log-schema",
+        token_provider=_FakeTokenProvider(),
+        client=client,
+    )
+
+    with pytest.raises(OperationTraceError):
+        source.fetch_action_logs("txn-1", "req-1")
+
+
+def test_http_error_raises_generic_operation_trace_error():
+    client = _client_returning({"error": "boom"}, status_code=500)
+    source = PiActionLogSource(
+        instances_list_url="http://pi-service/v2.0/schemas/{schemaId}/instances/list",
+        schema_id="action-log-schema",
+        token_provider=_FakeTokenProvider(),
+        client=client,
+    )
+
+    with pytest.raises(OperationTraceError):
+        source.fetch_action_logs("txn-1", "req-1")