log-foundry 0.10.2.dev61__tar.gz → 0.10.2.dev63__tar.gz

{log_foundry-0.10.2.dev61 → log_foundry-0.10.2.dev63}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: log-foundry
-Version: 0.10.2.dev61
+Version: 0.10.2.dev63
 Summary: Generate logs for your console and JSON events for downstream consumption.
 License-Expression: MIT
 License-File: LICENSE
@@ -951,6 +951,8 @@ if (
 
 `closing_sinks` is deliberately not a term here: it is briefly non-zero during a perfectly healthy
 sink swap, so a single reading is not a fault. Watch it over time instead — see the table below.
+`inherited_sink` is not a term either, for a different reason: it reports a *state* the process is
+in rather than a loss it took, and in a prefork deployment it is `True` on every worker by design.
 
 They tell you different things, and they want different responses:
 
@@ -963,6 +965,7 @@ They tell you different things, and they want different responses:
 | `sink.failed` | The sink attempted delivery and could not confirm it — abandoned requests, partially-failed batches, responses it could not adjudicate. | Fix the destination. |
 | `retired` + `submitted_after_shutdown` | `shutdown()` was called and the process **kept logging**. Those events are queued where nothing will drain them — total loss, for as long as the process runs. | Use `flush()`, not `shutdown()`, in a process that logs again. This is the serverless mistake below. |
 | `incomplete_swaps` | A late `configure(sink=...)` could not confirm the previous sink was drained. The swap took effect; that sink was left open and some queued events may have gone to the new one. | Investigate the previous sink — it was hung or failing. Configure the sink before the first log where you can. |
+| `inherited_sink` | This process is delivering to a sink it **inherited across a `fork`** and may not release, so it will not be closed here. Not a loss and not an alert term. | Nothing, usually. It explains a handle still open after `shutdown()`, and tells you a deployment shares one sink across a fork at all. `True` for a shared `StdoutSink` too, whose `close()` only flushes — so a `True` is not by itself evidence that anything is held. If you want the child to own its transport, build the sink in the worker process (see Forking). |
 | `closing_sinks` | Swapped-out sinks inside `close()` **right now** — a live gauge, not a counter, and the only field that falls as well as rises. Non-zero on a single read is normal during a swap. | Nothing, unless it stays non-zero. That means a destination is stuck in `close()` and will not release its resources. |
 
 `h.sink` is a `SinkLosses(dropped, failed)` or `None` — `None` when no worker exists yet, or when

{log_foundry-0.10.2.dev61 → log_foundry-0.10.2.dev63}/README.md

@@ -915,6 +915,8 @@ if (
 
 `closing_sinks` is deliberately not a term here: it is briefly non-zero during a perfectly healthy
 sink swap, so a single reading is not a fault. Watch it over time instead — see the table below.
+`inherited_sink` is not a term either, for a different reason: it reports a *state* the process is
+in rather than a loss it took, and in a prefork deployment it is `True` on every worker by design.
 
 They tell you different things, and they want different responses:
 
@@ -927,6 +929,7 @@ They tell you different things, and they want different responses:
 | `sink.failed` | The sink attempted delivery and could not confirm it — abandoned requests, partially-failed batches, responses it could not adjudicate. | Fix the destination. |
 | `retired` + `submitted_after_shutdown` | `shutdown()` was called and the process **kept logging**. Those events are queued where nothing will drain them — total loss, for as long as the process runs. | Use `flush()`, not `shutdown()`, in a process that logs again. This is the serverless mistake below. |
 | `incomplete_swaps` | A late `configure(sink=...)` could not confirm the previous sink was drained. The swap took effect; that sink was left open and some queued events may have gone to the new one. | Investigate the previous sink — it was hung or failing. Configure the sink before the first log where you can. |
+| `inherited_sink` | This process is delivering to a sink it **inherited across a `fork`** and may not release, so it will not be closed here. Not a loss and not an alert term. | Nothing, usually. It explains a handle still open after `shutdown()`, and tells you a deployment shares one sink across a fork at all. `True` for a shared `StdoutSink` too, whose `close()` only flushes — so a `True` is not by itself evidence that anything is held. If you want the child to own its transport, build the sink in the worker process (see Forking). |
 | `closing_sinks` | Swapped-out sinks inside `close()` **right now** — a live gauge, not a counter, and the only field that falls as well as rises. Non-zero on a single read is normal during a swap. | Nothing, unless it stays non-zero. That means a destination is stuck in `close()` and will not release its resources. |
 
 `h.sink` is a `SinkLosses(dropped, failed)` or `None` — `None` when no worker exists yet, or when

{log_foundry-0.10.2.dev61 → log_foundry-0.10.2.dev63}/pyproject.toml

@@ -20,7 +20,7 @@ dependencies = [
 ]
 
 # Optional features. Install with: pip install log-foundry[aws]
-version = "0.10.2.dev61"
+version = "0.10.2.dev63"
 
 [project.optional-dependencies]
 aws = ["boto3>=1.43.61"]              # SQSSink, SNSSink, KinesisSink, FirehoseSink

{log_foundry-0.10.2.dev61 → log_foundry-0.10.2.dev63}/src/log_foundry/_fork.py

@@ -37,6 +37,36 @@ and all three would have to import this module for it to reach them; instead the
 it, so this module imports nothing that imports it and there is no cycle to break later.
 """
 
+_FORK_SKIP = ("reacquired_in_child",)
+"""This module's own opt-out from the walk below (:data:`_SKIP_ATTRIBUTE`).
+
+:data:`reacquired_in_child` holds **sinks**, so without this the next fork's walk enters the
+previous fork's roster and re-collects every sink in it — including ones ``configure()``
+superseded in between. Measured: a child's walk collected a sink two ``configure()`` calls out
+of date, its hook was called, and the reclaim that follows then stamped it for the grandchild's
+pid, so a sink no descendant ever acquired read ``releasable``. A ratchet, not a one-off: once a
+hook-implementing sink enters a roster, every descendant's walk finds it again.
+
+Exactly the hazard ``_lifecycle._owned`` already declares an opt-out for, reached by adding a
+second global that holds sinks. The rebind below is the belt to this brace.
+"""
+
+reacquired_in_child: list[Any] = []
+"""Sinks whose re-acquisition hook returned normally in this child (SPEC-042 FR-005).
+
+Published rather than acted on, because what it means is an *ownership* claim and ownership is
+``_lifecycle``'s: a sink that returns from the hook holds a transport of its own, so a child may
+release it.
+
+Rebound — never mutated in place — **before the repair walk runs**, so a second fork cannot
+inherit the first one's roster even if the opt-out above were removed. The first version of this
+claimed the rebind happened "at the start of every child repair" while doing it in the middle,
+after the walk; both halves were false and nothing tested either.
+
+This module still imports nothing but ``_diag`` (SPEC-039 FR-006); the reader registers a
+handler and reads this, which is the same inversion :data:`_child_handlers` exists for.
+"""
+
 _installed = False
 """Whether :func:`install` has already registered the child handler in this process.
 
@@ -330,8 +360,12 @@ def _fresh_primitive(value: Any, memo: dict[int, Any], keepalive: list[Any]) ->
     return fresh
 
 
-_DISCARD_HOOK = "discard_buffered_after_fork"
-"""The optional sink member a forked child asks to strand the parent's pending bytes.
+_REACQUIRE_HOOK = "reacquire_after_fork"
+"""The optional sink member a forked child asks to take its transport over.
+
+Named for the larger half of what it does (SPEC-042 FR-005): stranding the parent's pending
+bytes is one consequence, and re-acquiring the transport is the step — a sink that returns from
+it has claimed the transport as this process's own, which is what makes releasing it safe.
 
 Probed by name, as ``losses()`` and ``log_foundry_stop_signal`` are, so no existing sink stops
 satisfying ``Sink`` by not having it. ``sinks/base.py`` states the contract; this is the only
@@ -339,7 +373,7 @@ place the name is read.
 """
 
 
-def _offers_discard(holder: object) -> bool:
+def _offers_reacquire(holder: object) -> bool:
     """Whether this object carries FR-004's buffer hook, without letting the question raise.
 
     Every other read the walk makes is guarded individually, and this one has to be too: an
@@ -361,11 +395,48 @@ def _offers_discard(holder: object) -> bool:
       None.
     """
     try:
-        return callable(getattr(holder, _DISCARD_HOOK, None))
+        return callable(getattr(holder, _REACQUIRE_HOOK, None))
     except Exception:
         return False
 
 
+_SKIP_ATTRIBUTE = "_FORK_SKIP"
+"""A module's opt-out: attribute names the walk must not read or descend into.
+
+**Bookkeeping that pins objects is not live state to repair**, and the difference is invisible
+to a walk that only sees a container. ``_lifecycle._owned`` is the case: SPEC-042 FR-001 requires
+it to hold a **strong** reference to every sink this process ever acquired, so without an opt-out
+the walk reaches sinks the process abandoned several ``configure()`` calls ago — replacing their
+locks, which is merely wasteful, and calling their fork hooks, which is not. Measured: a forked
+child announced a buffer-discard failure for a sink that had been superseded, and a ``FileSink``
+in that state would have its file reopened on every fork for the life of the process.
+
+Declared by the module that owns the state rather than listed here, which keeps this module's
+rule that it imports nothing but ``_diag`` (SPEC-039 FR-006). Nothing is lost by skipping: a sink
+that is still *live* is reached through the config and the worker, which are not opted out.
+"""
+
+
+def _skipped_names(holder: object) -> frozenset[str]:
+    """Returns the attribute names a holder has opted out of the walk.
+
+    Args:
+      holder: The module, class or instance the walk is about to read.
+
+    Returns:
+      The opted-out names, empty for anything that declares none.
+
+    Raises:
+      None. A malformed or unreadable declaration opts nothing out, so the walk does more work
+        rather than less — the safe direction for a repair whose absence is a hang.
+    """
+    try:
+        declared = getattr(holder, _SKIP_ATTRIBUTE, None)
+        return frozenset(declared) if declared else frozenset()
+    except Exception:
+        return frozenset()
+
+
 def _reinit_primitives() -> list[Any]:
     """Replaces every lock and event this package owns, wherever the walk reaches one.
 
@@ -391,7 +462,7 @@ def _reinit_primitives() -> list[Any]:
       None.
 
     Returns:
-      The owned instances carrying :data:`_DISCARD_HOOK`, in the order the walk reached them.
+      The owned instances carrying :data:`_REACQUIRE_HOOK`, in the order the walk reached them.
 
     Raises:
       None.
@@ -415,9 +486,12 @@ def _reinit_primitives() -> list[Any]:
         if not isinstance(holder, types.ModuleType | type):
             if not _is_owned(holder):
                 continue
-            if _offers_discard(holder):
+            if _offers_reacquire(holder):
                 buffered.append(holder)
+        skip = _skipped_names(holder)
         for name, value in _namespace_items(holder):
+            if name in skip:
+                continue
             fresh = _fresh_primitive(value, memo, keepalive)
             if fresh is not None:
                 _assign(holder, name, fresh)
@@ -426,19 +500,25 @@ def _reinit_primitives() -> list[Any]:
     return buffered
 
 
-def _discard_buffers(holders: list[Any]) -> None:
-    """Asks each sink that owns a buffered stream to strand what it inherited (FR-004).
+def _reacquire_transports(holders: list[Any]) -> list[Any]:
+    """Asks each sink that owns a buffered stream to re-acquire it in this child (FR-004).
 
     A fork landing inside ``emit`` — after the write loop, before the flush — leaves the child
     holding the parent's unflushed bytes, which both processes then write: measured, the event
     at the fork point appeared on disk twice. Without a ``before`` handler there is nowhere to
-    empty the buffer from (FR-001), so the child throws its copy away instead.
+    empty the buffer from (FR-001), so the child re-opens instead, which strands the copy.
+
+    **Which hooks returned normally is published rather than acted on** (SPEC-042 FR-005). A
+    sink that returns from the hook has claimed the transport as this process's own, which is an
+    ownership fact — but ownership lives in ``_lifecycle``, and this module imports nothing but
+    ``_diag``. So the roster is handed back and the handler ``_lifecycle`` registers re-stamps
+    it.
 
     Args:
       holders: What :func:`_reinit_primitives` collected on its walk.
 
     Returns:
-      None.
+      The holders whose hook returned normally, in the order they were called.
 
     Raises:
       None. One sink's failure is absorbed separately from the rest, so a hook that cannot
@@ -446,15 +526,19 @@ def _discard_buffers(holders: list[Any]) -> None:
         sink its repair — and the consequence is named, because an absorbed failure is
         invisible apart from that line.
     """
+    reacquired: list[Any] = []
     for holder in holders:
         try:
-            getattr(holder, _DISCARD_HOOK)()
+            getattr(holder, _REACQUIRE_HOOK)()
+            reacquired.append(holder)
         except Exception as exc:
             _diag.absorbed(
-                "discarding an inherited buffer after a fork",
+                "re-acquiring a transport after a fork",
                 exc,
-                f"{type(holder).__name__} may write the parent's pending bytes again",
+                f"{type(holder).__name__} may write the parent's pending bytes again, and this "
+                f"child will not release it",
             )
+    return reacquired
 
 
 def register_child_handler(fn: Callable[[], None]) -> None:
@@ -515,12 +599,14 @@ def _reinit_after_fork() -> None:
         registers another simply causes it to run — and :func:`register_child_handler` is the
         only writer, appending only.
     """
+    global reacquired_in_child
+    reacquired_in_child = []
     buffered: list[Any] = []
     try:
         buffered = _reinit_primitives()
     except Exception as exc:
         _diag.absorbed("repairing the library after a fork", exc, "this child may block or lose")
-    _discard_buffers(buffered)
+    reacquired_in_child = _reacquire_transports(buffered)
     for handler in _child_handlers:
         try:
             handler()