streamgate 3.0.0__tar.gz → 3.1.0__tar.gz

{streamgate-3.0.0 → streamgate-3.1.0}/CHANGELOG.md

@@ -5,6 +5,49 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.1.0] - 2026-09-14
+
+### Added
+
+- **`JsonFormatter`** — a built-in, out-of-the-box stdlib `logging.Formatter`
+  that renders every record as one JSON line, restoring the structured-field
+  visibility 2.x hosts got for free from loguru's auto-configured JSON sink.
+  3.0.0 kept emitting all structured fields (as flat `LogRecord` extras) but
+  stdlib's default formatter never renders them — a batch consumer logged just
+  the bare `batch_handle_success` event name, hiding `batch_size` /
+  `duration_ms`. Non-breaking; import from `streamgate` or `streamgate.obs`
+  and wire it in three lines:
+
+  ```python
+  import logging
+  from streamgate import JsonFormatter
+
+  handler = logging.StreamHandler()        # stderr, same destination as 2.x
+  handler.setFormatter(JsonFormatter())
+  logging.basicConfig(level=logging.INFO, handlers=[handler])
+  ```
+
+  Output layout per record: `timestamp` (UTC ISO-8601 with `Z` suffix, taken
+  from the `LogRecord` clock, not format time), `level`, `logger` (hierarchical
+  namespace — new versus 2.x), `event` (the stable event-name contract; host
+  `%s`-style messages render identically), then every flat extra field
+  (`batch_size`, `duration_ms`, `partition`, `offset`, ...), plus `error`
+  (formatted traceback) when a record carries `exc_info`. Serialization: one
+  `json.dumps` per record, `ensure_ascii=False`, `default=str` fallback.
+  The formatter is a generic component — any logger's records render correctly
+  through it (no extras ⇒ base fields only), and it never raises on
+  `LogRecord` reserved-attribute collisions (host-side extras are skipped;
+  rendering-side tolerance, the emission-side `emit()` fail-fast is unchanged).
+  Attach the handler to the root logger for unified host + streamgate JSON
+  logs, or to `logging.getLogger("streamgate")` for streamgate-only.
+- `streamgate.obs.logging.RESERVED_RECORD_ATTRS` (also re-exported from
+  `streamgate.obs`) — the reserved-attribute set as a single shared constant,
+  used by `emit()` for fail-fast and by `JsonFormatter` for skip-tolerance
+  (previously privately defined per side).
+- All `examples/` now configure logging through the built-in `JsonFormatter`,
+  so copy-pasting an example yields field-visible JSON logs out of the box
+  (previously `basicConfig(format=...)` rendered no extras).
+
 ## [3.0.0] - 2026-09-14
 
 ### BREAKING — loguru removed, logging switched to stdlib `logging`

{streamgate-3.0.0 → streamgate-3.1.0}/CONFIGURATION.md

@@ -29,6 +29,27 @@ streamgate emits logs through the standard-library `logging` module and ships
 `streamgate.*` logger tree; level, handlers and formats are configured by the
 host application (e.g. `logging.getLogger("streamgate")`).
 
+For structured output, attach the built-in `JsonFormatter`
+(`from streamgate import JsonFormatter`, also exported from
+`streamgate.obs`): one JSON line per record with `timestamp` (UTC
+ISO-8601, `Z` suffix) / `level` / `logger` / `event` base fields, every flat
+extra field, and `error` when the record carries `exc_info`:
+
+```python
+import logging
+from streamgate import JsonFormatter
+
+handler = logging.StreamHandler()      # stderr
+handler.setFormatter(JsonFormatter())
+logging.basicConfig(level=logging.INFO, handlers=[handler])
+```
+
+Rendering-side notes: the formatter is generic (any logger's records render
+correctly — records without extras produce just the base fields) and never
+raises on `LogRecord` reserved-attribute collisions (host-side extras are
+skipped; only the library's own `emit()` fails fast on them). The reserved
+attribute set is shared as `streamgate.obs.logging.RESERVED_RECORD_ATTRS`.
+
 ---
 
 ## Consumer (flat arguments)

{streamgate-3.0.0 → streamgate-3.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: streamgate
-Version: 3.0.0
+Version: 3.1.0
 Summary: A pure-core Kafka data pipeline framework: conditional admission, reliable delivery, and one obvious consume-loop outlet via a typed handler.
 Project-URL: Homepage, https://github.com/pwg-code/streamgate
 Project-URL: Repository, https://github.com/pwg-code/streamgate
@@ -67,7 +67,7 @@ aiokafka gives you **transport**; streamgate gives you the **operating semantics
 
 A complete produce → Kafka → consume → SQLite round trip that runs on a **bare install** (stdlib outlet, zero extra packages) — [`examples/pure_pipeline/`](examples/pure_pipeline/):
 
-> Logging note: streamgate emits via stdlib `logging` and never configures it. Without host configuration only WARNING+ shows (stdlib `lastResort`); add one line `logging.basicConfig(level=logging.INFO, ...)` at your entry point to see INFO logs (the examples all do this — see [Logging](#logging)).
+> Logging note: streamgate emits via stdlib `logging` and never configures it. Without host configuration only WARNING+ shows (stdlib `lastResort`); wire the built-in [`JsonFormatter`](#logging) at your entry point (3 lines, above) to see INFO logs with every structured field rendered as JSON (the examples all do this — see [Logging](#logging)).
 
 ```python
 import asyncio
@@ -233,16 +233,25 @@ There are exactly two sources of truth. **Required settings are true required co
 
 ## Logging
 
-streamgate logs through the standard-library `logging` module — **the library emits records and never configures logging** (no handlers added, no levels changed at import). Every logger lives under the `streamgate` namespace (`streamgate.ingest.producer`, `streamgate.consumer.loop`, ...), so the whole tree is controlled by the host:
+streamgate logs through the standard-library `logging` module — **the library emits records and never configures logging** (no handlers added, no levels changed at import). Every logger lives under the `streamgate` namespace (`streamgate.ingest.producer`, `streamgate.consumer.loop`, ...), so the whole tree is controlled by the host. Event names (`batch_handle_start`, `poison_message_skipped`, ...) are a stable public contract; structured fields travel as flat `LogRecord` extras.
+
+### JSON output: built-in `JsonFormatter`
+
+stdlib's default formatter does **not** render `LogRecord` extras — with `basicConfig(format="%(message)s")` a batch consumer logs just `batch_handle_success`, hiding `batch_size` / `duration_ms`. Use the built-in `JsonFormatter` to make every structured field visible (this is what all examples do):
 
 ```python
 import logging
 
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
-logging.getLogger("streamgate").setLevel(logging.WARNING)   # or any subtree: silence INFO noise
+from streamgate import JsonFormatter   # equivalently: from streamgate.obs import JsonFormatter
+
+handler = logging.StreamHandler()      # stderr — same destination as 2.x
+handler.setFormatter(JsonFormatter())
+logging.basicConfig(level=logging.INFO, handlers=[handler])
 ```
 
-Unconfigured hosts see WARNING and above only (stdlib `lastResort` on stderr). Event names (`batch_handle_start`, `poison_message_skipped`, ...) are a stable public contract; structured fields travel as flat `LogRecord` extras. For JSON output, attach any stdlib-compatible JSON formatter of your choice.
+Each record renders as one JSON line: `timestamp` (UTC ISO-8601 with `Z` suffix, taken from the `LogRecord` clock), `level`, `logger` (the hierarchical namespace — a 3.0+ extra versus 2.x), `event` (the stable event name; host `%s`-style messages render the same way), then every flat extra field (`batch_size`, `duration_ms`, `partition`, `offset`, ...), plus `error` (formatted traceback) when a record carries `exc_info`. The formatter is a generic component — any logger renders correctly through it: records without extras produce just the base fields, and reserved-attribute collisions are skipped, never raised.
+
+Attach the handler to the root logger as above and host + streamgate logs share one JSON format; attach the same way to `logging.getLogger("streamgate")` instead and only the streamgate tree is formatted. Level control is orthogonal: `logging.getLogger("streamgate").setLevel(logging.WARNING)` silences INFO noise regardless of formatter. Unconfigured hosts still see WARNING+ only (stdlib `lastResort` on stderr).
 
 ## Examples
 

{streamgate-3.0.0 → streamgate-3.1.0}/README.md

@@ -32,7 +32,7 @@ aiokafka gives you **transport**; streamgate gives you the **operating semantics
 
 A complete produce → Kafka → consume → SQLite round trip that runs on a **bare install** (stdlib outlet, zero extra packages) — [`examples/pure_pipeline/`](examples/pure_pipeline/):
 
-> Logging note: streamgate emits via stdlib `logging` and never configures it. Without host configuration only WARNING+ shows (stdlib `lastResort`); add one line `logging.basicConfig(level=logging.INFO, ...)` at your entry point to see INFO logs (the examples all do this — see [Logging](#logging)).
+> Logging note: streamgate emits via stdlib `logging` and never configures it. Without host configuration only WARNING+ shows (stdlib `lastResort`); wire the built-in [`JsonFormatter`](#logging) at your entry point (3 lines, above) to see INFO logs with every structured field rendered as JSON (the examples all do this — see [Logging](#logging)).
 
 ```python
 import asyncio
@@ -198,16 +198,25 @@ There are exactly two sources of truth. **Required settings are true required co
 
 ## Logging
 
-streamgate logs through the standard-library `logging` module — **the library emits records and never configures logging** (no handlers added, no levels changed at import). Every logger lives under the `streamgate` namespace (`streamgate.ingest.producer`, `streamgate.consumer.loop`, ...), so the whole tree is controlled by the host:
+streamgate logs through the standard-library `logging` module — **the library emits records and never configures logging** (no handlers added, no levels changed at import). Every logger lives under the `streamgate` namespace (`streamgate.ingest.producer`, `streamgate.consumer.loop`, ...), so the whole tree is controlled by the host. Event names (`batch_handle_start`, `poison_message_skipped`, ...) are a stable public contract; structured fields travel as flat `LogRecord` extras.
+
+### JSON output: built-in `JsonFormatter`
+
+stdlib's default formatter does **not** render `LogRecord` extras — with `basicConfig(format="%(message)s")` a batch consumer logs just `batch_handle_success`, hiding `batch_size` / `duration_ms`. Use the built-in `JsonFormatter` to make every structured field visible (this is what all examples do):
 
 ```python
 import logging
 
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
-logging.getLogger("streamgate").setLevel(logging.WARNING)   # or any subtree: silence INFO noise
+from streamgate import JsonFormatter   # equivalently: from streamgate.obs import JsonFormatter
+
+handler = logging.StreamHandler()      # stderr — same destination as 2.x
+handler.setFormatter(JsonFormatter())
+logging.basicConfig(level=logging.INFO, handlers=[handler])
 ```
 
-Unconfigured hosts see WARNING and above only (stdlib `lastResort` on stderr). Event names (`batch_handle_start`, `poison_message_skipped`, ...) are a stable public contract; structured fields travel as flat `LogRecord` extras. For JSON output, attach any stdlib-compatible JSON formatter of your choice.
+Each record renders as one JSON line: `timestamp` (UTC ISO-8601 with `Z` suffix, taken from the `LogRecord` clock), `level`, `logger` (the hierarchical namespace — a 3.0+ extra versus 2.x), `event` (the stable event name; host `%s`-style messages render the same way), then every flat extra field (`batch_size`, `duration_ms`, `partition`, `offset`, ...), plus `error` (formatted traceback) when a record carries `exc_info`. The formatter is a generic component — any logger renders correctly through it: records without extras produce just the base fields, and reserved-attribute collisions are skipped, never raised.
+
+Attach the handler to the root logger as above and host + streamgate logs share one JSON format; attach the same way to `logging.getLogger("streamgate")` instead and only the streamgate tree is formatted. Level control is orthogonal: `logging.getLogger("streamgate").setLevel(logging.WARNING)` silences INFO noise regardless of formatter. Unconfigured hosts still see WARNING+ only (stdlib `lastResort` on stderr).
 
 ## Examples
 

{streamgate-3.0.0 → streamgate-3.1.0}/README.zh-CN.md

@@ -372,16 +372,25 @@ pip install "streamgate[http]"     # + HTTP 探活背压
 
 ## 日志
 
-streamgate 走标准库 `logging`——**库只发日志记录、永不配置日志**（import 时不增删任何 handler、不改任何级别）。所有 logger 挂在 `streamgate` 命名空间下（`streamgate.ingest.producer`、`streamgate.consumer.loop` 等），宿主一行整树控制：
+streamgate 走标准库 `logging`——**库只发日志记录、永不配置日志**（import 时不增删任何 handler、不改任何级别）。所有 logger 挂在 `streamgate` 命名空间下（`streamgate.ingest.producer`、`streamgate.consumer.loop` 等），宿主整树控制。事件名（`batch_handle_start`、`poison_message_skipped` 等）是稳定的对外契约；结构化字段以 `LogRecord` extra 平铺。
+
+### JSON 输出：内置 `JsonFormatter`
+
+stdlib 默认 Formatter **不渲染** `LogRecord` extra——用 `basicConfig(format="%(message)s")` 时，批量消费只会打出裸事件名 `batch_handle_success`，看不到 `batch_size` / `duration_ms`。接内置 `JsonFormatter` 即可让全部结构化字段可见（所有示例均如此）：
 
 ```python
 import logging
 
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s")
-logging.getLogger("streamgate").setLevel(logging.WARNING)   # 或任意子树：关掉 INFO 噪音
+from streamgate import JsonFormatter   # 等价：from streamgate.obs import JsonFormatter
+
+handler = logging.StreamHandler()      # stderr，与 2.x 输出目标一致
+handler.setFormatter(JsonFormatter())
+logging.basicConfig(level=logging.INFO, handlers=[handler])
 ```
 
-未配置日志的宿主只能看到 WARNING 及以上（stdlib `lastResort` 输出到 stderr）。事件名（`batch_handle_start`、`poison_message_skipped` 等）是稳定的对外契约；结构化字段以 `LogRecord` extra 平铺。需要 JSON 输出时，接任意 stdlib 兼容的 JSON Formatter 即可。
+每条日志渲染为一行 JSON：`timestamp`（UTC ISO8601，`Z` 后缀，取自 LogRecord 时钟）、`level`、`logger`（层级命名空间，2.x 没有的新字段）、`event`（稳定事件名；宿主 `%s` 风格日志同样适用），其余键为全部平铺的结构化字段（`batch_size`、`duration_ms`、`partition`、`offset` 等），记录携带 `exc_info` 时附 `error`（格式化堆栈）。Formatter 是通用组件——任何 logger 经它渲染都正确：无 extra 的记录只输出基础字段，与 LogRecord 保留属性撞键时直接跳过、不抛错。
+
+按上面方式挂 root logger，宿主与 streamgate 日志统一为 JSON 格式；同样三行挂 `logging.getLogger("streamgate")` 则仅 streamgate 树生效。级别控制与 Formatter 正交：`logging.getLogger("streamgate").setLevel(logging.WARNING)` 可关掉 INFO 噪音。未配置日志的宿主仍只能看到 WARNING 及以上（stdlib `lastResort` 输出到 stderr）。
 
 ## 架构
 

{streamgate-3.0.0 → streamgate-3.1.0}/examples/http_probe/consume.py

@@ -13,7 +13,7 @@ import os
 
 from health_server import serve_consumer_health
 
-from streamgate import ConsumeContext, Consumer, JsonObject
+from streamgate import ConsumeContext, Consumer, JsonFormatter, JsonObject
 
 
 async def print_orders(
@@ -24,9 +24,9 @@ async def print_orders(
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     consumer = Consumer(
         bootstrap_servers=os.environ.get("KAFKA__BOOTSTRAP_SERVERS", "localhost:29092"),
         topic=os.environ.get("KAFKA__TOPIC", "orders"),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/http_probe/produce.py

@@ -15,7 +15,7 @@ import os
 
 from models import OrderIn
 
-from streamgate import BackpressureConfig, Producer, ProducerOptions
+from streamgate import BackpressureConfig, JsonFormatter, Producer, ProducerOptions
 from streamgate.contrib.http_probe import HttpProbeSignal
 
 
@@ -38,9 +38,9 @@ def backpressure_config() -> BackpressureConfig:
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     producer = Producer(
         os.environ.get("KAFKA__BOOTSTRAP_SERVERS", "localhost:29092"),
         topic=os.environ.get("KAFKA__TOPIC", "orders"),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/prod_pipeline/consume.py

@@ -19,7 +19,7 @@ from models import Order
 from sqlalchemy.ext.asyncio import create_async_engine
 from sqlmodel import SQLModel
 
-from streamgate import ConsumerOptions
+from streamgate import ConsumerOptions, JsonFormatter
 from streamgate.contrib.mssql_upsert import MssqlConsumer, Upsert
 
 
@@ -38,9 +38,9 @@ async def init_db() -> None:
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     await init_db()
     consumer = MssqlConsumer(
         db=db_conn(),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/prod_pipeline/produce.py

@@ -15,7 +15,13 @@ import os
 
 from models import OrderIn
 
-from streamgate import BackpressureConfig, DedupOptions, Producer, ProducerOptions
+from streamgate import (
+    BackpressureConfig,
+    DedupOptions,
+    JsonFormatter,
+    Producer,
+    ProducerOptions,
+)
 from streamgate.contrib.http_probe import HttpProbeSignal
 from streamgate.contrib.redis_dedup import (
     RedisConfig,
@@ -59,9 +65,9 @@ def backpressure_config() -> BackpressureConfig:
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     producer = Producer(
         os.environ.get("KAFKA__BOOTSTRAP_SERVERS", "localhost:9092"),
         topic=os.environ.get("KAFKA__TOPIC", "orders"),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/pure_pipeline/consume.py

@@ -18,6 +18,7 @@ from streamgate import (
     Consumer,
     ConsumerOptions,
     DlqOptions,
+    JsonFormatter,
     JsonObject,
 )
 
@@ -58,9 +59,9 @@ async def handle_orders(
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     consumer = Consumer(
         bootstrap_servers=os.environ.get("KAFKA__BOOTSTRAP_SERVERS", "localhost:29092"),
         topic=os.environ.get("KAFKA__TOPIC", "orders"),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/pure_pipeline/produce.py

@@ -14,7 +14,7 @@ import os
 
 from models import OrderIn
 
-from streamgate import DedupOptions, Producer, ProducerOptions
+from streamgate import DedupOptions, JsonFormatter, Producer, ProducerOptions
 
 
 def bootstrap_servers() -> str:
@@ -23,9 +23,9 @@ def bootstrap_servers() -> str:
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     producer = Producer(
         bootstrap_servers(),
         topic=os.environ.get("KAFKA__TOPIC", "orders"),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/redis_admission/produce.py

@@ -15,7 +15,7 @@ import os
 
 from models import OrderIn
 
-from streamgate import DedupOptions, Producer, ProducerOptions
+from streamgate import DedupOptions, JsonFormatter, Producer, ProducerOptions
 from streamgate.contrib.redis_dedup import (
     RedisConfig,
     RedisDedupCache,
@@ -41,9 +41,9 @@ def build_carrier() -> RedisDedupCarrier[OrderIn]:
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     carrier = build_carrier()
     producer = Producer(
         os.environ.get("KAFKA__BOOTSTRAP_SERVERS", "localhost:29092"),

{streamgate-3.0.0 → streamgate-3.1.0}/examples/sqlite_upsert/consume.py

@@ -13,14 +13,14 @@ import os
 
 from models import Order
 
-from streamgate import ConsumerOptions
+from streamgate import ConsumerOptions, JsonFormatter
 from streamgate.contrib.sqlite_upsert import SqliteConsumer, Upsert
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     consumer = SqliteConsumer(
         db=os.environ.get("DB_CONN", "sqlite+aiosqlite:///./data/streamgate.db"),
         upserts=[Upsert(model=Order, keys=["order_id"])],

{streamgate-3.0.0 → streamgate-3.1.0}/examples/sqlite_upsert/produce.py

@@ -10,13 +10,13 @@ import os
 
 from models import OrderIn
 
-from streamgate import Producer, ProducerOptions
+from streamgate import JsonFormatter, Producer, ProducerOptions
 
 
 async def main() -> None:
-    logging.basicConfig(
-        level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s %(message)s"
-    )
+    handler = logging.StreamHandler()
+    handler.setFormatter(JsonFormatter())
+    logging.basicConfig(level=logging.INFO, handlers=[handler])
     producer = Producer(
         os.environ.get("KAFKA__BOOTSTRAP_SERVERS", "localhost:29092"),
         topic=os.environ.get("KAFKA__TOPIC", "orders"),

{streamgate-3.0.0 → streamgate-3.1.0}/src/streamgate/__init__.py

@@ -52,6 +52,7 @@ from streamgate.ingest.producer import (
     Producer,
     ProducerOptions,
 )
+from streamgate.obs import JsonFormatter
 from streamgate.obs.metrics import LoggingMetricsSink, MetricsSink
 from streamgate.protocols import (
     AllowAllSignal,
@@ -85,7 +86,7 @@ from streamgate.resilience.health import (
 from streamgate.transport.codec import JsonEnvelopeCodec
 from streamgate.transport.kafka import KafkaConsumerService
 
-__version__ = "3.0.0"
+__version__ = "3.1.0"
 
 __all__ = [
     "AllowAllSignal",
@@ -115,6 +116,7 @@ __all__ = [
     "InMemoryDedupCarrier",
     "JsonObject",
     "JsonEnvelopeCodec",
+    "JsonFormatter",
     "KafkaConfig",
     "KafkaConsumerService",
     "LoggingMetricsSink",

streamgate-3.1.0/src/streamgate/obs/__init__.py

@@ -0,0 +1,9 @@
+"""可观测性出口：结构化日志发射层（logging）与宿主侧 JSON 渲染（formatter）。"""
+
+from streamgate.obs.formatter import JsonFormatter
+from streamgate.obs.logging import RESERVED_RECORD_ATTRS
+
+__all__ = [
+    "JsonFormatter",
+    "RESERVED_RECORD_ATTRS",
+]

streamgate-3.1.0/src/streamgate/obs/formatter.py

@@ -0,0 +1,57 @@
+"""内置 JSON Formatter：宿主开箱即用的结构化日志渲染。
+
+3.0.0 移除 loguru 后，``emit()`` 经 ``extra=fields`` 发出的结构化字段仍在
+LogRecord 上，但 stdlib 默认 Formatter 不渲染 extra——字段"看不见"。
+本模块补齐渲染侧：每条日志渲染为一行 JSON（基础字段 + extra 平铺 +
+exc_info 兜底）。定位是通用组件：任何 logger（含宿主自身的）经它渲染
+都正确；本模块同样不做任何 logging 全局配置（挂载方式归宿主）。
+"""
+
+import json
+import logging
+from datetime import datetime, timezone
+from typing import Final
+
+from streamgate.obs.logging import RESERVED_RECORD_ATTRS
+
+_BASE_FIELDS: Final[frozenset[str]] = frozenset(
+    {"timestamp", "level", "logger", "event"}
+)
+
+
+def _iso_utc(epoch_seconds: float) -> str:
+    """LogRecord 时间戳 → UTC ISO8601（毫秒精度，Z 后缀）。
+
+    必须取自 LogRecord 时钟（record.created）而非 format 时刻，避免漂移。
+    """
+    moment = datetime.fromtimestamp(epoch_seconds, tz=timezone.utc)
+    return moment.isoformat(timespec="milliseconds").replace("+00:00", "Z")
+
+
+class JsonFormatter(logging.Formatter):
+    """stdlib ``logging.Formatter`` 子类：每条日志渲染为一行 JSON。
+
+    字段布局：``timestamp``（UTC ISO8601，Z 后缀）/ ``level`` /
+    ``logger``（层级命名空间）/ ``event``（``record.getMessage()``，
+    兼容宿主 ``%s`` 风格日志），其余键从 ``record.__dict__`` 剔除保留
+    属性后平铺（即 ``emit(extra=...)`` 注入的全部结构化字段）；
+    ``record.exc_info`` 存在时附 ``error``（formatException 输出）。
+    序列化：``ensure_ascii=False``（中文原样）、``default=str`` 兜底。
+    与发射侧 emit 的 fail-fast 不同层：遇保留属性撞键直接跳过、不抛错
+    （渲染侧必须容错——宿主自己的 extra 不受 emit 保护）。
+    """
+
+    def format(self, record: logging.LogRecord) -> str:
+        entry: dict[str, object] = {
+            "timestamp": _iso_utc(record.created),
+            "level": record.levelname,
+            "logger": record.name,
+            "event": record.getMessage(),
+        }
+        for key, value in record.__dict__.items():
+            if key in RESERVED_RECORD_ATTRS or key in _BASE_FIELDS:
+                continue
+            entry[key] = value
+        if record.exc_info is not None:
+            entry["error"] = self.formatException(record.exc_info)
+        return json.dumps(entry, ensure_ascii=False, default=str)

{streamgate-3.0.0 → streamgate-3.1.0}/src/streamgate/obs/logging.py

@@ -6,6 +6,8 @@
 ``emit`` 是内部发射辅助：保持调用点单行形态，并统一防御
 （结构化字段名与 LogRecord 保留属性冲突时 fail-fast）。
 本模块不触碰任何 logging 全局状态（无 handler 增删、无级别修改）。
+``RESERVED_RECORD_ATTRS`` 是保留属性集合的唯一来源：发射侧据此 fail-fast，
+渲染侧（formatter.JsonFormatter）据此做"跳过"容错，两处禁止各自定义。
 """
 
 import logging
@@ -20,14 +22,14 @@ _LEVELS: Final[dict[str, int]] = {
 
 # LogRecord 实例属性 + 格式化期追加属性（asctime/message/exc_text）：
 # extra 携带这些键会在 stdlib 内部炸 KeyError，这里提前 fail-fast 并给出可读错误
-_RESERVED: Final[frozenset[str]] = frozenset(
+RESERVED_RECORD_ATTRS: Final[frozenset[str]] = frozenset(
     logging.LogRecord("", 0, "", 0, "", (), None).__dict__
 ) | {"asctime", "exc_text", "message", "taskName"}
 
 
 def emit(logger: logging.Logger, level: str, event: str, /, **fields: object) -> None:
     """发射一条结构化日志记录：event 为事件名（对外契约，勿改语义），fields 平铺进 extra。"""
-    conflicts = fields.keys() & _RESERVED
+    conflicts = fields.keys() & RESERVED_RECORD_ATTRS
     if conflicts:
         names = ", ".join(sorted(conflicts))
         raise ValueError(