hannah-grpc-lib 0.4.0__tar.gz

hannah_grpc_lib-0.4.0/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: hannah-grpc-lib
+Version: 0.4.0
+Summary: Shared gRPC client code for Hannah components (log shipping to the Hannah log collector)
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: grpcio>=1.84.0
+Requires-Dist: hannah-proto>=4.5.0
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+
+# hannah-grpc-lib (Python)
+
+Shared gRPC client code for Hannah components. Until 0.3.1 this package was called
+`hannah-logging` (import `hannah_logging`); that package gets no further updates.
+
+## Logging (`hannah_grpc.logging`)
+
+Ships the logs of a Hannah component to the Hannah log collector, so a user can download
+one archive of all components' logs from the WebUI.
+
+The library adds one `logging.Handler`. Your existing handlers (stdout, journald, syslog)
+keep working unchanged.
+
+## Installation
+
+```sh
+pip install hannah-grpc-lib
+```
+
+The library relies on the `hannah-proto` version the component already uses, because
+every call to Hannah Core has to carry the matching protocol version.
+
+## Usage
+
+```python
+import logging
+import hannah_grpc.logging as hannah_logging   # alias it: `from hannah_grpc import logging` would shadow the stdlib
+
+logging.basicConfig(level=logging.INFO)
+
+# As early as possible: everything logged from here on is buffered.
+shipping = hannah_logging.install(
+    "telegram",
+    version=__version__,
+    secrets=[config.bot_token],                         # masked wherever they appear
+    logger_categories={"hannah.stt": hannah_logging.TRANSCRIPT},
+)
+
+# Once the config is loaded:
+shipping.connect(
+    hannah_address="localhost:50051",                   # discovery via Hannah Core
+    collector_address=None,                             # optional static fallback
+)
+```
+
+### Setting the collector address directly
+
+A component that knows the collector address itself (Hannah Core, which announces it)
+passes it in instead of subscribing to discovery:
+
+```python
+shipping.set_collector_address("192.168.1.10:50061")   # None withdraws it
+```
+
+It starts shipping if `connect` hasn't been called yet, takes precedence over the static
+`collector_address`, and a new address moves the stream.
+
+### Categories
+
+Each line is tagged `GENERAL`, `TRANSCRIPT` (speech transcripts, user utterances) or
+`METADATA` (room and device names, presence). Exports can leave out the last two, and
+the WebUI does so by default.
+
+- By logger name: `logger_categories={"hannah.stt": TRANSCRIPT}` also covers child
+  loggers such as `hannah.stt.whisper`.
+- Per call: `log.info("heard: %s", text, extra={hannah_logging.CATEGORY_ATTR: hannah_logging.TRANSCRIPT})`
+
+### Secrets
+
+Before a line leaves the process, the library masks:
+
+- values of keys such as `password`, `token`, `api_key`, `secret`, `psk`
+- `Bearer` tokens, JWTs, credentials in URLs (`mqtt://user:pw@host`), PEM private keys
+- Telegram bot tokens and well-known token prefixes (`glpat-`, `ghp_`, …)
+- every value passed as `secrets=[…]` or later through `shipping.add_secret(…)`
+
+Extra regexes go in `secret_patterns=[…]`. The component's own handlers still see the
+unmasked line.
+
+## Behaviour
+
+- **Buffer:** 4 MiB by default (`max_buffer_bytes`). When it is full, the oldest lines are
+  dropped and reported to the collector as a gap. While no collector is known, the
+  buffer simply keeps running as a ring.
+- **Timestamps** are taken when a line is logged, not when it is sent.
+- **Instance name:** tells several instances of a component apart in the export. Without
+  `instance`, the library uses the `HANNAH_LOG_INSTANCE` environment variable, else the
+  hostname. In a container whose hostname is just its ID (a new one with every recreate),
+  it uses `container` instead. Running the same component in several containers? Set
+  `HANNAH_LOG_INSTANCE`, or `hostname:` in Compose.
+- **Discovery:** subscribes to Hannah Core's infrastructure announcements and follows the
+  log collector when it moves. If Core is unreachable, the last known collector is kept.
+  The static `collector_address` is used while none is announced.
+- **Never blocks the component:** sending runs on its own threads. Connection problems
+  are logged once to the `hannah_grpc.logging` logger (not shipped) and retried with backoff.
+- **Shutdown:** at exit, the library tries for up to 2 seconds to send what is still
+  buffered (`shipping.close(timeout=…)`).

hannah_grpc_lib-0.4.0/README.md

@@ -0,0 +1,97 @@
+# hannah-grpc-lib (Python)
+
+Shared gRPC client code for Hannah components. Until 0.3.1 this package was called
+`hannah-logging` (import `hannah_logging`); that package gets no further updates.
+
+## Logging (`hannah_grpc.logging`)
+
+Ships the logs of a Hannah component to the Hannah log collector, so a user can download
+one archive of all components' logs from the WebUI.
+
+The library adds one `logging.Handler`. Your existing handlers (stdout, journald, syslog)
+keep working unchanged.
+
+## Installation
+
+```sh
+pip install hannah-grpc-lib
+```
+
+The library relies on the `hannah-proto` version the component already uses, because
+every call to Hannah Core has to carry the matching protocol version.
+
+## Usage
+
+```python
+import logging
+import hannah_grpc.logging as hannah_logging   # alias it: `from hannah_grpc import logging` would shadow the stdlib
+
+logging.basicConfig(level=logging.INFO)
+
+# As early as possible: everything logged from here on is buffered.
+shipping = hannah_logging.install(
+    "telegram",
+    version=__version__,
+    secrets=[config.bot_token],                         # masked wherever they appear
+    logger_categories={"hannah.stt": hannah_logging.TRANSCRIPT},
+)
+
+# Once the config is loaded:
+shipping.connect(
+    hannah_address="localhost:50051",                   # discovery via Hannah Core
+    collector_address=None,                             # optional static fallback
+)
+```
+
+### Setting the collector address directly
+
+A component that knows the collector address itself (Hannah Core, which announces it)
+passes it in instead of subscribing to discovery:
+
+```python
+shipping.set_collector_address("192.168.1.10:50061")   # None withdraws it
+```
+
+It starts shipping if `connect` hasn't been called yet, takes precedence over the static
+`collector_address`, and a new address moves the stream.
+
+### Categories
+
+Each line is tagged `GENERAL`, `TRANSCRIPT` (speech transcripts, user utterances) or
+`METADATA` (room and device names, presence). Exports can leave out the last two, and
+the WebUI does so by default.
+
+- By logger name: `logger_categories={"hannah.stt": TRANSCRIPT}` also covers child
+  loggers such as `hannah.stt.whisper`.
+- Per call: `log.info("heard: %s", text, extra={hannah_logging.CATEGORY_ATTR: hannah_logging.TRANSCRIPT})`
+
+### Secrets
+
+Before a line leaves the process, the library masks:
+
+- values of keys such as `password`, `token`, `api_key`, `secret`, `psk`
+- `Bearer` tokens, JWTs, credentials in URLs (`mqtt://user:pw@host`), PEM private keys
+- Telegram bot tokens and well-known token prefixes (`glpat-`, `ghp_`, …)
+- every value passed as `secrets=[…]` or later through `shipping.add_secret(…)`
+
+Extra regexes go in `secret_patterns=[…]`. The component's own handlers still see the
+unmasked line.
+
+## Behaviour
+
+- **Buffer:** 4 MiB by default (`max_buffer_bytes`). When it is full, the oldest lines are
+  dropped and reported to the collector as a gap. While no collector is known, the
+  buffer simply keeps running as a ring.
+- **Timestamps** are taken when a line is logged, not when it is sent.
+- **Instance name:** tells several instances of a component apart in the export. Without
+  `instance`, the library uses the `HANNAH_LOG_INSTANCE` environment variable, else the
+  hostname. In a container whose hostname is just its ID (a new one with every recreate),
+  it uses `container` instead. Running the same component in several containers? Set
+  `HANNAH_LOG_INSTANCE`, or `hostname:` in Compose.
+- **Discovery:** subscribes to Hannah Core's infrastructure announcements and follows the
+  log collector when it moves. If Core is unreachable, the last known collector is kept.
+  The static `collector_address` is used while none is announced.
+- **Never blocks the component:** sending runs on its own threads. Connection problems
+  are logged once to the `hannah_grpc.logging` logger (not shipped) and retried with backoff.
+- **Shutdown:** at exit, the library tries for up to 2 seconds to send what is still
+  buffered (`shipping.close(timeout=…)`).

hannah_grpc_lib-0.4.0/hannah_grpc/__init__.py

@@ -0,0 +1,7 @@
+"""Shared gRPC client code for Hannah components.
+
+    import hannah_grpc.logging as hannah_logging   # ships the component's logs
+
+Import the subpackages under an alias: `from hannah_grpc import logging` would shadow
+the standard library's `logging` in your module.
+"""

hannah_grpc_lib-0.4.0/hannah_grpc/logging/__init__.py

@@ -0,0 +1,155 @@
+"""Ships the logs of a Hannah component to the Hannah log collector.
+
+    import hannah_grpc.logging as hannah_logging
+
+    shipping = hannah_logging.install("core", version=__version__)   # as early as possible
+    ...
+    shipping.connect(hannah_address="localhost:50051")                # once the config is loaded
+
+Everything logged from `install` on is buffered in memory (bounded), and sent to the
+collector as soon as one is known — announced by Hannah Core, or a static address.
+The host's own handlers (stdout, journald, syslog) are not touched.
+"""
+from __future__ import annotations
+
+import atexit
+import logging
+import threading
+from typing import Iterable, Mapping, Optional
+
+from hannah_proto import logging_pb2
+
+from .buffer import LogBuffer
+from .handler import CATEGORY_ATTR, GENERAL, METADATA, TRANSCRIPT, CollectorHandler
+from .instance import resolve_instance
+from .secrets import SecretFilter
+from .shipper import Discovery, Shipper
+
+__all__ = [
+    "install", "LogShipping", "CollectorHandler", "SecretFilter",
+    "GENERAL", "TRANSCRIPT", "METADATA", "CATEGORY_ATTR",
+]
+
+DEFAULT_MAX_BUFFER_BYTES = 4 * 1024 * 1024
+
+
+class LogShipping:
+    """Handle returned by `install`. Owns the handler and the background threads."""
+
+    def __init__(
+        self,
+        component: str,
+        version: str,
+        instance: str,
+        handler: CollectorHandler,
+        buffer: LogBuffer,
+        secret_filter: SecretFilter,
+        target_logger: logging.Logger,
+    ):
+        self.handler = handler
+        self._hello = logging_pb2.ShipHello(component=component, instance=instance, version=version)
+        self._buffer = buffer
+        self._secret_filter = secret_filter
+        self._target_logger = target_logger
+        self._shipper: Optional[Shipper] = None
+        self._discovery: Optional[Discovery] = None
+        self._closed = False
+        self._lock = threading.Lock()
+
+    def connect(self, hannah_address: Optional[str] = None, collector_address: Optional[str] = None) -> None:
+        """Starts shipping. `hannah_address` enables discovery via Hannah Core;
+        `collector_address` is the fallback while none is announced. Only the
+        first call has an effect."""
+        with self._lock:
+            if self._shipper is not None or self._closed:
+                return
+            self._shipper = Shipper(self._buffer, self._hello, collector_address or None)
+            self._shipper.start()
+            if hannah_address:
+                self._discovery = Discovery(hannah_address, self._shipper.set_discovered)
+                self._discovery.start()
+
+    def set_collector_address(self, address: Optional[str]) -> None:
+        """Sets the collector address directly, for a component that knows it without
+        discovery (Hannah Core itself). Treated like an address announced by Core: it
+        takes precedence over the static fallback, and a change moves the stream.
+        None withdraws it. Starts shipping if `connect` hasn't been called yet."""
+        self.connect()
+        with self._lock:
+            shipper = self._shipper
+        if shipper is not None:
+            shipper.set_discovered(address or None)
+
+    def add_secret(self, secret: str) -> None:
+        """Masks this literal value in everything logged from now on (e.g. a token
+        that only becomes known after `install`)."""
+        self._secret_filter.add_literal(secret)
+
+    def close(self, timeout: float = 2.0) -> None:
+        """Detaches the handler and gives the shipper up to `timeout` seconds to
+        send what is still buffered."""
+        self._target_logger.removeHandler(self.handler)
+        with self._lock:
+            discovery, shipper = self._discovery, self._shipper
+            self._discovery = self._shipper = None
+            self._closed = True
+        if discovery is not None:
+            discovery.stop()
+        if shipper is not None:
+            shipper.stop()
+            shipper.join(timeout)
+
+    @property
+    def collector_address(self) -> Optional[str]:
+        """The collector address currently in use — announced by Hannah Core, else the
+        static fallback. None while none is known or before `connect`."""
+        with self._lock:
+            shipper = self._shipper
+        return shipper.target() if shipper is not None else None
+
+
+def install(
+    component: str,
+    *,
+    version: str = "",
+    instance: Optional[str] = None,
+    hannah_address: Optional[str] = None,
+    collector_address: Optional[str] = None,
+    secrets: Iterable[str] = (),
+    secret_patterns: Iterable[str] = (),
+    logger_categories: Optional[Mapping[str, int]] = None,
+    level: int = logging.NOTSET,
+    max_buffer_bytes: int = DEFAULT_MAX_BUFFER_BYTES,
+    logger: Optional[logging.Logger] = None,
+) -> LogShipping:
+    """Attaches a CollectorHandler to `logger` (default: the root logger) and starts
+    buffering. Pass `hannah_address`/`collector_address` here or later via
+    `LogShipping.connect`.
+
+    component          e.g. "core", "telegram" — the name shown in the export
+    instance           distinguishes several instances; defaults to $HANNAH_LOG_INSTANCE,
+                       else the hostname ("container" if that is just a container ID)
+    secrets            literal values to mask (the component's own tokens/passwords)
+    secret_patterns    extra regexes to mask, on top of the built-in ones
+    logger_categories  logger name prefix -> category, e.g. {"hannah.stt": TRANSCRIPT};
+                       a single call can override it with extra={CATEGORY_ATTR: ...}
+    """
+    buffer = LogBuffer(max_buffer_bytes)
+    secret_filter = SecretFilter(secrets, secret_patterns)
+    handler = CollectorHandler(buffer, secret_filter, logger_categories, level)
+    target_logger = logger or logging.getLogger()
+    target_logger.addHandler(handler)
+
+    shipping = LogShipping(
+        component=component,
+        version=version,
+        instance=resolve_instance(instance),
+        handler=handler,
+        buffer=buffer,
+        secret_filter=secret_filter,
+        target_logger=target_logger,
+    )
+    atexit.register(shipping.close)
+    if hannah_address or collector_address:
+        shipping.connect(hannah_address, collector_address)
+    return shipping

hannah_grpc_lib-0.4.0/hannah_grpc/logging/buffer.py

@@ -0,0 +1,84 @@
+"""Bounded in-memory buffer between the logging handler and the shipper thread."""
+from __future__ import annotations
+
+import collections
+import threading
+from dataclasses import dataclass
+from typing import Deque, List, Optional, Tuple
+
+# Rough per-entry overhead on top of message + logger name, so a flood of empty
+# messages still counts against the limit.
+_ENTRY_OVERHEAD = 64
+
+
+@dataclass(frozen=True)
+class Entry:
+    timestamp_ms: int
+    level: int      # hannah_proto LogLevel value
+    logger: str
+    message: str
+    category: int   # hannah_proto LogCategory value
+
+    def size(self) -> int:
+        return len(self.message) + len(self.logger) + _ENTRY_OVERHEAD
+
+
+@dataclass
+class Gap:
+    dropped: int
+    from_ms: int
+    to_ms: int
+
+
+class LogBuffer:
+    """Ring buffer limited by size. When full, the oldest entries are dropped and
+    recorded as a gap, which is shipped before the remaining entries.
+
+    `put` never blocks the logging thread beyond a short lock."""
+
+    def __init__(self, max_bytes: int):
+        self._max_bytes = max_bytes
+        self._entries: Deque[Entry] = collections.deque()
+        self._bytes = 0
+        self._gap: Optional[Gap] = None
+        self._cond = threading.Condition()
+
+    def put(self, entry: Entry) -> None:
+        with self._cond:
+            self._entries.append(entry)
+            self._bytes += entry.size()
+            while self._bytes > self._max_bytes and self._entries:
+                self._drop_oldest()
+            self._cond.notify()
+
+    def _drop_oldest(self) -> None:
+        old = self._entries.popleft()
+        self._bytes -= old.size()
+        if self._gap is None:
+            self._gap = Gap(dropped=1, from_ms=old.timestamp_ms, to_ms=old.timestamp_ms)
+        else:
+            self._gap.dropped += 1
+            self._gap.to_ms = max(self._gap.to_ms, old.timestamp_ms)
+
+    def take(self, max_items: int, timeout: Optional[float]) -> Tuple[Optional[Gap], List[Entry]]:
+        """Waits up to `timeout` for data, then removes and returns the pending gap
+        (if any) and up to `max_items` entries, oldest first."""
+        with self._cond:
+            if not self._entries and self._gap is None:
+                self._cond.wait(timeout)
+            gap, self._gap = self._gap, None
+            items: List[Entry] = []
+            while self._entries and len(items) < max_items:
+                entry = self._entries.popleft()
+                self._bytes -= entry.size()
+                items.append(entry)
+            return gap, items
+
+    def wake(self) -> None:
+        """Wakes a waiting `take`, e.g. to react to a new collector address or shutdown."""
+        with self._cond:
+            self._cond.notify_all()
+
+    def __len__(self) -> int:
+        with self._cond:
+            return len(self._entries)

hannah_grpc_lib-0.4.0/hannah_grpc/logging/handler.py

@@ -0,0 +1,75 @@
+"""logging.Handler that feeds the buffer. Runs in addition to the host's own handlers."""
+from __future__ import annotations
+
+import logging
+from typing import Mapping, Optional
+
+from hannah_proto import logging_pb2
+
+from .buffer import Entry, LogBuffer
+from .secrets import SecretFilter
+from .shipper import OWN_THREADS
+
+GENERAL = logging_pb2.LOG_CATEGORY_GENERAL
+TRANSCRIPT = logging_pb2.LOG_CATEGORY_TRANSCRIPT
+METADATA = logging_pb2.LOG_CATEGORY_METADATA
+
+# Name of the `extra=` key that tags a single log call, e.g.
+# log.info("heard: %s", text, extra={CATEGORY_ATTR: TRANSCRIPT})
+CATEGORY_ATTR = "log_category"
+
+
+def to_log_level(levelno: int) -> int:
+    if levelno >= logging.CRITICAL:
+        return logging_pb2.LOG_LEVEL_CRITICAL
+    if levelno >= logging.ERROR:
+        return logging_pb2.LOG_LEVEL_ERROR
+    if levelno >= logging.WARNING:
+        return logging_pb2.LOG_LEVEL_WARNING
+    if levelno >= logging.INFO:
+        return logging_pb2.LOG_LEVEL_INFO
+    return logging_pb2.LOG_LEVEL_DEBUG
+
+
+class CollectorHandler(logging.Handler):
+    """Formats, filters and buffers each record. The timestamp is the record's
+    creation time, so entries buffered before the collector was reachable keep
+    their original time."""
+
+    def __init__(
+        self,
+        buffer: LogBuffer,
+        secret_filter: SecretFilter,
+        logger_categories: Optional[Mapping[str, int]] = None,
+        level: int = logging.NOTSET,
+    ):
+        super().__init__(level)
+        self._buffer = buffer
+        self._secret_filter = secret_filter
+        # Longest prefix first, so "hannah.stt.transcript" wins over "hannah.stt".
+        self._logger_categories = sorted((logger_categories or {}).items(), key=lambda kv: len(kv[0]), reverse=True)
+        self.setFormatter(logging.Formatter("%(message)s"))
+
+    def emit(self, record: logging.LogRecord) -> None:
+        if record.thread in OWN_THREADS:
+            return
+        try:
+            message = self._secret_filter(self.format(record))
+            self._buffer.put(Entry(
+                timestamp_ms=int(record.created * 1000),
+                level=to_log_level(record.levelno),
+                logger=record.name,
+                message=message,
+                category=self._category(record),
+            ))
+        except Exception:
+            self.handleError(record)
+
+    def _category(self, record: logging.LogRecord) -> int:
+        explicit = getattr(record, CATEGORY_ATTR, None)
+        if explicit:
+            return int(explicit)
+        for prefix, category in self._logger_categories:
+            if record.name == prefix or record.name.startswith(prefix + "."):
+                return category
+        return GENERAL

hannah_grpc_lib-0.4.0/hannah_grpc/logging/instance.py

@@ -0,0 +1,33 @@
+"""Resolves the instance name a component reports to the collector."""
+from __future__ import annotations
+
+import os
+import re
+import socket
+from typing import Optional
+
+INSTANCE_ENV = "HANNAH_LOG_INSTANCE"
+CONTAINER_INSTANCE = "container"
+
+# Docker and Podman drop one of these files into every container.
+_CONTAINER_MARKERS = ("/.dockerenv", "/run/.containerenv")
+# Without an explicit hostname, the container gets its short (12) or full (64) ID.
+_CONTAINER_ID = re.compile(r"[0-9a-f]{12}|[0-9a-f]{64}")
+
+
+def resolve_instance(explicit: Optional[str] = None) -> str:
+    """The explicit value, else $HANNAH_LOG_INSTANCE, else "container" when the hostname
+    is just a container ID (it changes with every recreate), else the hostname."""
+    if explicit:
+        return explicit
+    from_env = os.environ.get(INSTANCE_ENV, "").strip()
+    if from_env:
+        return from_env
+    host = socket.gethostname()
+    if _CONTAINER_ID.fullmatch(host) and _in_container():
+        return CONTAINER_INSTANCE
+    return host
+
+
+def _in_container() -> bool:
+    return any(os.path.exists(p) for p in _CONTAINER_MARKERS)

hannah_grpc_lib-0.4.0/hannah_grpc/logging/secrets.py

@@ -0,0 +1,69 @@
+"""Masks secrets in a log message before it leaves the process."""
+from __future__ import annotations
+
+import re
+from typing import Iterable, List, Pattern, Tuple
+
+MASK = "***"
+
+# Key names whose value is treated as secret in "key=value", "key: value" and JSON-ish "key": "value".
+_SECRET_KEYS = (
+    r"password|passwd|pwd|passphrase|secret|client[_-]?secret|token|access[_-]?token|refresh[_-]?token"
+    r"|api[_-]?key|apikey|access[_-]?key|private[_-]?key|psk|auth|authorization|credentials?"
+)
+
+_BUILTIN_PATTERNS: List[Tuple[Pattern[str], str]] = [
+    # PEM blocks (private keys, certificates with keys).
+    (re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----.*?-----END [A-Z ]*PRIVATE KEY-----", re.S), MASK),
+    # "Authorization: Bearer abc", "Bearer abc"
+    (re.compile(r"(?i)\b(bearer|basic)\s+[A-Za-z0-9._~+/=-]+"), r"\1 " + MASK),
+    # key=value / key: value / "key": "value" — the key has to *end* in a secret word,
+    # so "bot_token=…" is masked but "tokens=512" or "token_count=3" is not.
+    (
+        re.compile(
+            r"""(?ix)
+            (?P<key>["']?[\w.-]*(?:""" + _SECRET_KEYS + r""")["']?)
+            (?P<sep>\s*[:=]\s*)
+            (?P<value>"[^"]*"|'[^']*'|[^\s,;&)}\]]+)
+            """
+        ),
+        lambda m: m.group("key") + m.group("sep") + _quoted_mask(m.group("value")),
+    ),
+    # Credentials in URLs: scheme://user:password@host
+    (re.compile(r"(?i)\b([a-z][a-z0-9+.-]*://[^/\s:@]+):[^/\s@]+@"), r"\1:" + MASK + "@"),
+    # JWTs
+    (re.compile(r"\beyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+"), MASK),
+    # Telegram bot tokens
+    (re.compile(r"\b\d{8,10}:[A-Za-z0-9_-]{35}\b"), MASK),
+    # Well-known token prefixes (GitLab, GitHub, npm, PyPI, Slack, OpenAI-style)
+    (re.compile(r"\b(?:glpat|gldt|glrt|ghp|gho|ghu|ghs|ghr|github_pat|npm|pypi|xox[abpr]|sk)[-_][A-Za-z0-9_-]{16,}"), MASK),
+]
+
+
+def _quoted_mask(value: str) -> str:
+    if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
+        return value[0] + MASK + value[0]
+    return MASK
+
+
+class SecretFilter:
+    """Masks well-known secret shapes plus any literal values the component knows are secret
+    (its own tokens and passwords from its config — the most reliable way to catch them)."""
+
+    def __init__(self, literals: Iterable[str] = (), patterns: Iterable[str] = ()):
+        # Longest first, so a secret that contains another one is masked as a whole.
+        self._literals = sorted({s for s in literals if s and len(s) >= 4}, key=len, reverse=True)
+        self._patterns = list(_BUILTIN_PATTERNS) + [(re.compile(p), MASK) for p in patterns]
+
+    def add_literal(self, secret: str) -> None:
+        if secret and len(secret) >= 4 and secret not in self._literals:
+            self._literals.append(secret)
+            self._literals.sort(key=len, reverse=True)
+
+    def __call__(self, message: str) -> str:
+        for literal in self._literals:
+            if literal in message:
+                message = message.replace(literal, MASK)
+        for pattern, replacement in self._patterns:
+            message = pattern.sub(replacement, message)
+        return message