mobius-logging-py 1.0.0__tar.gz

mobius_logging_py-1.0.0/PKG-INFO

@@ -0,0 +1,211 @@
+Metadata-Version: 2.4
+Name: mobius-logging-py
+Version: 1.0.0
+Summary: Standardized logging context, observation, masking and Kafka schema events for Mobius Python (FastAPI) services.
+Author: Mobius Platform
+Keywords: logging,observability,fastapi,mobius,kafka
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: py-kafka-producer-client>=0.1.7
+Provides-Extra: otel
+Requires-Dist: opentelemetry-api>=1.20; extra == "otel"
+Provides-Extra: fastapi
+Requires-Dist: starlette>=0.27; extra == "fastapi"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: starlette>=0.27; extra == "dev"
+Requires-Dist: httpx>=0.24; extra == "dev"
+Requires-Dist: opentelemetry-api>=1.20; extra == "dev"
+
+# Mobius Logging for Python
+
+Python port of the Java Mobius logging library. It standardizes logging context
+for Mobius **FastAPI / Starlette** services: a contextvars-based context store
+(the analog of SLF4J MDC), a request middleware, an `@observed` method
+decorator, sensitive-data masking, OpenTelemetry trace sync, JSON logging, and
+a Kafka schema-event producer.
+
+Schema constants (Kafka topic `construct-data-1`, schema id, tenant id, and
+`schema_version=mobius.log.v1`) are **identical to the Java library**, so events
+from Python services land in the same pipeline.
+
+Requires Python 3.10+.
+
+## Install
+
+```bash
+pip install mobius-logging-py
+
+# or directly from git
+pip install "mobius-logging-py @ git+https://<your-git-host>/mobius-logging-py.git@v1.0.0"
+```
+
+The import package is `mobius_logging` (e.g. `from mobius_logging import setup_logging`).
+
+Optional extras: `mobius-logging-py[otel]` (OpenTelemetry trace sync),
+`mobius-logging-py[fastapi]` (Starlette middleware).
+
+## Quick start
+
+```python
+from fastapi import FastAPI
+from mobius_logging import setup_logging, observed, LogType
+from py_kafka_producer_client import Producer  # your internal client
+
+app = FastAPI()
+
+setup_logging(
+    app,
+    service_name="orders-service",
+    profile="prod",                       # json logs for staging/prod/production
+    sensitive_keys=["session_id", "refresh_token"],
+    kafka_producer=Producer(...),         # wrapped in PyKafkaProducerClientSender
+)
+
+@app.post("/orders")
+@observed(event_type="order.create", log_type=LogType.AUDIT, resource_type="order")
+async def create_order(order_id: str):
+    ...
+```
+
+`setup_logging` is the one-call equivalent of Java's Spring auto-configuration
+(Python has no auto-config, so wiring is explicit). It configures logging,
+registers custom sensitive keys, builds the schema producer, and adds the
+request middleware.
+
+## Java → Python mapping
+
+| Java Mobius logging library | Python (`mobius_logging`) |
+| --- | --- |
+| SLF4J `MDC` | `contextvars.ContextVar` (`context.py`) |
+| `PlatformLogContext` | `mobius_logging.context` functions |
+| `PlatformLogFields` | `mobius_logging.fields` |
+| `LogType` enum | `LogType` enum |
+| `PlatformLoggingFilter` (servlet) | `PlatformLoggingMiddleware` (ASGI) |
+| `@PlatformObserved` + AOP aspect | `@observed` decorator |
+| `KafkaLogProducer` (`KafkaTemplate`) | `KafkaLogProducer` + `LogEventSender` |
+| `PlatformKafka*Context` | `producer_headers` / `apply_consumer_record` |
+| `SensitiveDataMasker` | `mobius_logging.masking` |
+| `LogbackConfigurator` | `configure_platform_logging` |
+| Spring auto-config classes | `setup_logging` (explicit) |
+
+Out of scope for v1 (present in Java): JDBC/JPA, MongoDB, and Camunda logging.
+
+## Context API
+
+```python
+from mobius_logging import context, LogType
+
+context.init()                       # sets schema_version
+context.tenant_id("tenant-123")
+context.log_type(LogType.AUDIT)
+context.resource_type("order")
+context.resource_id("order-789")
+context.field("custom_key", "value")
+context.ensure_correlation_id()      # generates one if missing
+context.sync_trace_from_opentelemetry()
+
+snap = context.snapshot()
+try:
+    context.put("event_type", "order.approved")
+finally:
+    context.restore(snap)
+```
+
+> Note: Java's `ensureCorrelationId()` has an inverted condition and only
+> regenerates when one already exists. This port fixes that — it generates a
+> correlation id when none is present.
+
+## `@observed`
+
+Works on sync and async functions. Adds `log_type`, `event_type`,
+`resource_type`, `class_name`, `method_name`, `outcome`, `method_duration_ms`,
+and `exception_type`/`exception_message` on failure. After completion it ships
+the context to the schema producer (if one is registered), then restores the
+pre-call context snapshot (Java clears MDC instead).
+
+```python
+@observed(event_type="order.create", resource_type="order")
+def create_order(order_id: str): ...
+```
+
+## Kafka
+
+### Schema event producer
+
+The producer depends only on a small protocol, so the library has no hard Kafka
+dependency. It matches the `py-kafka-producer-client` (>=0.1.7) signature — the
+event is a dict and `topic` is keyword-only:
+
+```python
+class LogEventSender(Protocol):
+    def send(self, value: dict, *, topic: str) -> Any: ...
+```
+
+For the internal `py-kafka-producer-client`, pass the client to
+`setup_logging(kafka_producer=...)` or wrap it directly:
+
+```python
+from mobius_logging import KafkaLogProducer, PyKafkaProducerClientSender, set_log_producer
+
+producer = KafkaLogProducer(PyKafkaProducerClientSender(client), "orders-service")
+set_log_producer(producer)
+```
+
+`PyKafkaProducerClientSender` calls `client.send(value, topic=...)`, passing the
+`DataIngestionOperation` envelope as a dict (the client serializes it). To use a
+different client, pass any object exposing `send(value, *, topic)` as
+`kafka_sender=`.
+
+Sending runs on a background thread pool (best-effort, never raises, never
+blocks the request) — the analog of the Java `@Async` method.
+
+### Context propagation
+
+```python
+from mobius_logging import producer_headers, apply_consumer_record
+
+# producer side
+client.send(topic, value, headers=producer_headers())
+
+# consumer side
+apply_consumer_record(record.headers(), topic=record.topic(),
+                      partition=record.partition(), offset=record.offset())
+```
+
+Propagated header fields: `trace_id`, `span_id`, `correlation_id`,
+`causation_id`, `tenant_id`. Consumer also sets `kafka_topic`,
+`kafka_partition`, `kafka_offset`.
+
+## Logging output
+
+`configure_platform_logging(service_name, profile)` installs a root handler that
+injects the current context onto every record. Profiles `staging`/`prod`/
+`production` emit JSON; others use a readable text pattern. A daily-rotating
+file handler writes to `logs/<service>.log` (30 days retained).
+
+In JSON mode every context field is included automatically. For the text
+pattern, `txn_id`, `tenant_id`, and `trace_id` are surfaced inline.
+
+## Sensitive data masking
+
+```python
+from mobius_logging import mask, mask_map, init_custom_keys
+
+init_custom_keys(["session_id", "refresh_token"])
+mask("authorization", token)      # masked
+mask_map(context.copy())          # masks all sensitive keys in a dict
+```
+
+Default sensitive keys: `password`, `token`, `secret`, `authorization`,
+`cookie`, `apikey`, `privatekey`, `kubeconfig`. Matching is case-insensitive.
+Masking keeps the last 4 characters of generic values and partially masks
+emails — identical behaviour to the Java masker.
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pytest
+```

mobius_logging_py-1.0.0/README.md

@@ -0,0 +1,192 @@
+# Mobius Logging for Python
+
+Python port of the Java Mobius logging library. It standardizes logging context
+for Mobius **FastAPI / Starlette** services: a contextvars-based context store
+(the analog of SLF4J MDC), a request middleware, an `@observed` method
+decorator, sensitive-data masking, OpenTelemetry trace sync, JSON logging, and
+a Kafka schema-event producer.
+
+Schema constants (Kafka topic `construct-data-1`, schema id, tenant id, and
+`schema_version=mobius.log.v1`) are **identical to the Java library**, so events
+from Python services land in the same pipeline.
+
+Requires Python 3.10+.
+
+## Install
+
+```bash
+pip install mobius-logging-py
+
+# or directly from git
+pip install "mobius-logging-py @ git+https://<your-git-host>/mobius-logging-py.git@v1.0.0"
+```
+
+The import package is `mobius_logging` (e.g. `from mobius_logging import setup_logging`).
+
+Optional extras: `mobius-logging-py[otel]` (OpenTelemetry trace sync),
+`mobius-logging-py[fastapi]` (Starlette middleware).
+
+## Quick start
+
+```python
+from fastapi import FastAPI
+from mobius_logging import setup_logging, observed, LogType
+from py_kafka_producer_client import Producer  # your internal client
+
+app = FastAPI()
+
+setup_logging(
+    app,
+    service_name="orders-service",
+    profile="prod",                       # json logs for staging/prod/production
+    sensitive_keys=["session_id", "refresh_token"],
+    kafka_producer=Producer(...),         # wrapped in PyKafkaProducerClientSender
+)
+
+@app.post("/orders")
+@observed(event_type="order.create", log_type=LogType.AUDIT, resource_type="order")
+async def create_order(order_id: str):
+    ...
+```
+
+`setup_logging` is the one-call equivalent of Java's Spring auto-configuration
+(Python has no auto-config, so wiring is explicit). It configures logging,
+registers custom sensitive keys, builds the schema producer, and adds the
+request middleware.
+
+## Java → Python mapping
+
+| Java Mobius logging library | Python (`mobius_logging`) |
+| --- | --- |
+| SLF4J `MDC` | `contextvars.ContextVar` (`context.py`) |
+| `PlatformLogContext` | `mobius_logging.context` functions |
+| `PlatformLogFields` | `mobius_logging.fields` |
+| `LogType` enum | `LogType` enum |
+| `PlatformLoggingFilter` (servlet) | `PlatformLoggingMiddleware` (ASGI) |
+| `@PlatformObserved` + AOP aspect | `@observed` decorator |
+| `KafkaLogProducer` (`KafkaTemplate`) | `KafkaLogProducer` + `LogEventSender` |
+| `PlatformKafka*Context` | `producer_headers` / `apply_consumer_record` |
+| `SensitiveDataMasker` | `mobius_logging.masking` |
+| `LogbackConfigurator` | `configure_platform_logging` |
+| Spring auto-config classes | `setup_logging` (explicit) |
+
+Out of scope for v1 (present in Java): JDBC/JPA, MongoDB, and Camunda logging.
+
+## Context API
+
+```python
+from mobius_logging import context, LogType
+
+context.init()                       # sets schema_version
+context.tenant_id("tenant-123")
+context.log_type(LogType.AUDIT)
+context.resource_type("order")
+context.resource_id("order-789")
+context.field("custom_key", "value")
+context.ensure_correlation_id()      # generates one if missing
+context.sync_trace_from_opentelemetry()
+
+snap = context.snapshot()
+try:
+    context.put("event_type", "order.approved")
+finally:
+    context.restore(snap)
+```
+
+> Note: Java's `ensureCorrelationId()` has an inverted condition and only
+> regenerates when one already exists. This port fixes that — it generates a
+> correlation id when none is present.
+
+## `@observed`
+
+Works on sync and async functions. Adds `log_type`, `event_type`,
+`resource_type`, `class_name`, `method_name`, `outcome`, `method_duration_ms`,
+and `exception_type`/`exception_message` on failure. After completion it ships
+the context to the schema producer (if one is registered), then restores the
+pre-call context snapshot (Java clears MDC instead).
+
+```python
+@observed(event_type="order.create", resource_type="order")
+def create_order(order_id: str): ...
+```
+
+## Kafka
+
+### Schema event producer
+
+The producer depends only on a small protocol, so the library has no hard Kafka
+dependency. It matches the `py-kafka-producer-client` (>=0.1.7) signature — the
+event is a dict and `topic` is keyword-only:
+
+```python
+class LogEventSender(Protocol):
+    def send(self, value: dict, *, topic: str) -> Any: ...
+```
+
+For the internal `py-kafka-producer-client`, pass the client to
+`setup_logging(kafka_producer=...)` or wrap it directly:
+
+```python
+from mobius_logging import KafkaLogProducer, PyKafkaProducerClientSender, set_log_producer
+
+producer = KafkaLogProducer(PyKafkaProducerClientSender(client), "orders-service")
+set_log_producer(producer)
+```
+
+`PyKafkaProducerClientSender` calls `client.send(value, topic=...)`, passing the
+`DataIngestionOperation` envelope as a dict (the client serializes it). To use a
+different client, pass any object exposing `send(value, *, topic)` as
+`kafka_sender=`.
+
+Sending runs on a background thread pool (best-effort, never raises, never
+blocks the request) — the analog of the Java `@Async` method.
+
+### Context propagation
+
+```python
+from mobius_logging import producer_headers, apply_consumer_record
+
+# producer side
+client.send(topic, value, headers=producer_headers())
+
+# consumer side
+apply_consumer_record(record.headers(), topic=record.topic(),
+                      partition=record.partition(), offset=record.offset())
+```
+
+Propagated header fields: `trace_id`, `span_id`, `correlation_id`,
+`causation_id`, `tenant_id`. Consumer also sets `kafka_topic`,
+`kafka_partition`, `kafka_offset`.
+
+## Logging output
+
+`configure_platform_logging(service_name, profile)` installs a root handler that
+injects the current context onto every record. Profiles `staging`/`prod`/
+`production` emit JSON; others use a readable text pattern. A daily-rotating
+file handler writes to `logs/<service>.log` (30 days retained).
+
+In JSON mode every context field is included automatically. For the text
+pattern, `txn_id`, `tenant_id`, and `trace_id` are surfaced inline.
+
+## Sensitive data masking
+
+```python
+from mobius_logging import mask, mask_map, init_custom_keys
+
+init_custom_keys(["session_id", "refresh_token"])
+mask("authorization", token)      # masked
+mask_map(context.copy())          # masks all sensitive keys in a dict
+```
+
+Default sensitive keys: `password`, `token`, `secret`, `authorization`,
+`cookie`, `apikey`, `privatekey`, `kubeconfig`. Matching is case-insensitive.
+Masking keeps the last 4 characters of generic values and partially masks
+emails — identical behaviour to the Java masker.
+
+## Development
+
+```bash
+python -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+pytest
+```

mobius_logging_py-1.0.0/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mobius-logging-py"
+version = "1.0.0"
+description = "Standardized logging context, observation, masking and Kafka schema events for Mobius Python (FastAPI) services."
+readme = "README.md"
+requires-python = ">=3.10"
+authors = [{ name = "Mobius Platform" }]
+keywords = ["logging", "observability", "fastapi", "mobius", "kafka"]
+dependencies = [
+    # Schema-event producer used by @observed. Resolved from the internal index.
+    "py-kafka-producer-client>=0.1.7",
+]
+
+[project.optional-dependencies]
+# OpenTelemetry trace-id sync (sync_trace_from_opentelemetry is a no-op without it)
+otel = ["opentelemetry-api>=1.20"]
+# Web middleware target
+fastapi = ["starlette>=0.27"]
+# Dev / test tooling
+dev = [
+    "pytest>=7.4",
+    "starlette>=0.27",
+    "httpx>=0.24",
+    "opentelemetry-api>=1.20",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+mobius_logging = ["py.typed"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

mobius_logging_py-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

mobius_logging_py-1.0.0/src/mobius_logging/__init__.py

@@ -0,0 +1,80 @@
+"""Mobius Logging for Python.
+
+Python port of the Java Mobius logging library: standardized logging context,
+method observation, sensitive-data masking, OpenTelemetry trace sync, JSON
+logging and a Kafka schema-event producer for FastAPI / Starlette services.
+"""
+from __future__ import annotations
+
+from . import context, fields
+from .context import (
+    clear,
+    correlation_id,
+    ensure_correlation_id,
+    field,
+    get,
+    init,
+    log_type,
+    put,
+    requester_type,
+    resource_id,
+    resource_type,
+    restore,
+    snapshot,
+    sync_trace_from_opentelemetry,
+    tenant_id,
+    transaction_id,
+)
+from .kafka_context import apply_consumer_record, producer_headers
+from .kafka_producer import (
+    KafkaLogProducer,
+    LogEventSender,
+    PyKafkaProducerClientSender,
+    get_log_producer,
+    set_log_producer,
+)
+from .log_type import LogType
+from .logging_config import configure_platform_logging
+from .masking import init_custom_keys, mask, mask_map
+from .middleware import PlatformLoggingMiddleware
+from .observed import observed
+from .bootstrap import setup_logging
+
+__version__ = "1.0.0"
+
+__all__ = [
+    "context",
+    "fields",
+    "LogType",
+    "PlatformLoggingMiddleware",
+    "observed",
+    "setup_logging",
+    "configure_platform_logging",
+    "KafkaLogProducer",
+    "LogEventSender",
+    "PyKafkaProducerClientSender",
+    "set_log_producer",
+    "get_log_producer",
+    "producer_headers",
+    "apply_consumer_record",
+    "init_custom_keys",
+    "mask",
+    "mask_map",
+    # context functions
+    "put",
+    "get",
+    "field",
+    "init",
+    "clear",
+    "snapshot",
+    "restore",
+    "log_type",
+    "tenant_id",
+    "resource_id",
+    "resource_type",
+    "correlation_id",
+    "transaction_id",
+    "requester_type",
+    "ensure_correlation_id",
+    "sync_trace_from_opentelemetry",
+]

mobius_logging_py-1.0.0/src/mobius_logging/bootstrap.py

@@ -0,0 +1,120 @@
+"""One-call bootstrap for FastAPI services.
+
+Wires logging configuration, sensitive-key registration, the schema producer
+and the request middleware in a single call - the rough equivalent of the
+Java auto-configuration classes (which Spring applies automatically; Python
+has no auto-config, so this is explicit).
+
+Example::
+
+    from kafka_producer_client import KafkaProducerConfig
+    from mobius_logging import setup_logging
+
+    setup_logging(
+        app,
+        service_name="orders-service",
+        profile="prod",
+        sensitive_keys=["session_id", "refresh_token"],
+        kafka_config=KafkaProducerConfig(
+            bootstrap_servers="broker:9092",
+            default_topic="construct-data-1",
+        ),
+    )
+"""
+from __future__ import annotations
+
+import logging
+from typing import Any, Iterable, Optional
+
+from . import masking
+from .kafka_producer import (
+    KafkaLogProducer,
+    LogEventSender,
+    PyKafkaProducerClientSender,
+    set_log_producer,
+)
+from .logging_config import configure_platform_logging
+from .middleware import PlatformLoggingMiddleware
+
+log = logging.getLogger(__name__)
+
+
+def setup_logging(
+    app: Optional[Any] = None,
+    *,
+    service_name: str = "unknown-service",
+    profile: Optional[str] = None,
+    sensitive_keys: Optional[Iterable[str]] = None,
+    kafka_config: Optional[Any] = None,
+    kafka_producer: Optional[Any] = None,
+    kafka_sender: Optional[LogEventSender] = None,
+    enable_file: bool = True,
+) -> Optional[KafkaLogProducer]:
+    """Configure the library and (optionally) register FastAPI middleware.
+
+    The schema producer is resolved in this order:
+
+    1. ``kafka_sender`` - any object with ``send(value, *, topic)``;
+    2. ``kafka_producer`` - a ``py-kafka-producer-client`` ``KafkaProducerClient``
+       instance, wrapped automatically;
+    3. ``kafka_config`` - a ``KafkaProducerConfig``; the client's singleton is
+       configured and used (the default integration);
+    4. the already-configured ``py-kafka-producer-client`` singleton, if the app
+       called ``configure_kafka_producer(...)`` itself.
+
+    If none resolve, ``@observed`` simply skips schema publishing.
+
+    Returns the registered :class:`KafkaLogProducer`, if any.
+    """
+    configure_platform_logging(service_name, profile, enable_file=enable_file)
+    masking.init_custom_keys(sensitive_keys)
+
+    sender = _resolve_sender(kafka_sender, kafka_producer, kafka_config)
+
+    producer: Optional[KafkaLogProducer] = None
+    if sender is not None:
+        producer = KafkaLogProducer(sender, service_name)
+        set_log_producer(producer)
+    else:
+        log.warning(
+            "mobius-logging: no Kafka producer resolved; @observed will not "
+            "publish schema events. Pass kafka_config/kafka_producer/kafka_sender."
+        )
+
+    if app is not None:
+        app.add_middleware(PlatformLoggingMiddleware, service_name=service_name)
+
+    return producer
+
+
+def _resolve_sender(
+    kafka_sender: Optional[LogEventSender],
+    kafka_producer: Optional[Any],
+    kafka_config: Optional[Any],
+) -> Optional[LogEventSender]:
+    if kafka_sender is not None:
+        return kafka_sender
+    if kafka_producer is not None:
+        return PyKafkaProducerClientSender(kafka_producer)
+
+    # Default: integrate with the internal py-kafka-producer-client singleton.
+    try:
+        from kafka_producer_client.action_logger import (
+            configure_kafka_producer,
+            get_kafka_producer,
+        )
+    except ImportError:
+        log.warning("mobius-logging: py-kafka-producer-client not installed.")
+        return None
+
+    try:
+        if kafka_config is not None:
+            configure_kafka_producer(kafka_config)
+        return PyKafkaProducerClientSender(get_kafka_producer())
+    except Exception:  # noqa: BLE001 - producer is optional, never crash setup
+        log.warning(
+            "mobius-logging: could not initialize py-kafka-producer-client "
+            "(was configure_kafka_producer/kafka_config provided?).",
+            exc_info=True,
+        )
+        return None

mobius_logging_py-1.0.0/src/mobius_logging/constants.py

@@ -0,0 +1,32 @@
+"""Platform constants. Mirrors the Java ``PlatformConstant``.
+
+The Kafka topic, schema id and tenant id are kept identical to the Java
+library on purpose: Python services emit into the same schema pipeline.
+"""
+from __future__ import annotations
+
+import re
+
+SCHEMA_VERSION = "mobius.log.v1"
+
+# Masking
+EMAIL_RE = re.compile(r"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,6}$")
+# Mask every word char that is followed by at least 4 more word chars,
+# i.e. keep the last 4 characters visible. Matches the Java MASKING_REGEX.
+MASKING_RE = re.compile(r"\w(?=\w{4})")
+MASKING_PATTERN = "*****"
+AT_SYMBOL = "@"
+STAR = "*"
+
+# Kafka schema pipeline (must match the Java library)
+TOPIC = "construct-data-1"
+TENANT_ID = "2cf76e5f-26ad-4f2c-bccc-f4bc1e7bfb64"
+SCHEMA_ID = "6a2f9b55f7827435c6ff87c4"
+
+# Legacy -> canonical key remapping applied before shipping to the schema.
+SANITIZE_KEYS = {
+    "agentId": "agent_id",
+    "userId": "user_id",
+    "tenantId": "tenant_id",
+    "txnId": "txn_id",
+}