log-foundry 0.10.2.dev62__tar.gz → 0.10.2.dev63__tar.gz

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: log-foundry
-Version: 0.10.2.dev62
+Version: 0.10.2.dev63
 Summary: Generate logs for your console and JSON events for downstream consumption.
 License-Expression: MIT
 License-File: LICENSE
@@ -951,6 +951,8 @@ if (
 
 `closing_sinks` is deliberately not a term here: it is briefly non-zero during a perfectly healthy
 sink swap, so a single reading is not a fault. Watch it over time instead — see the table below.
+`inherited_sink` is not a term either, for a different reason: it reports a *state* the process is
+in rather than a loss it took, and in a prefork deployment it is `True` on every worker by design.
 
 They tell you different things, and they want different responses:
 
@@ -963,6 +965,7 @@ They tell you different things, and they want different responses:
 | `sink.failed` | The sink attempted delivery and could not confirm it — abandoned requests, partially-failed batches, responses it could not adjudicate. | Fix the destination. |
 | `retired` + `submitted_after_shutdown` | `shutdown()` was called and the process **kept logging**. Those events are queued where nothing will drain them — total loss, for as long as the process runs. | Use `flush()`, not `shutdown()`, in a process that logs again. This is the serverless mistake below. |
 | `incomplete_swaps` | A late `configure(sink=...)` could not confirm the previous sink was drained. The swap took effect; that sink was left open and some queued events may have gone to the new one. | Investigate the previous sink — it was hung or failing. Configure the sink before the first log where you can. |
+| `inherited_sink` | This process is delivering to a sink it **inherited across a `fork`** and may not release, so it will not be closed here. Not a loss and not an alert term. | Nothing, usually. It explains a handle still open after `shutdown()`, and tells you a deployment shares one sink across a fork at all. `True` for a shared `StdoutSink` too, whose `close()` only flushes — so a `True` is not by itself evidence that anything is held. If you want the child to own its transport, build the sink in the worker process (see Forking). |
 | `closing_sinks` | Swapped-out sinks inside `close()` **right now** — a live gauge, not a counter, and the only field that falls as well as rises. Non-zero on a single read is normal during a swap. | Nothing, unless it stays non-zero. That means a destination is stuck in `close()` and will not release its resources. |
 
 `h.sink` is a `SinkLosses(dropped, failed)` or `None` — `None` when no worker exists yet, or when

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/README.md

@@ -915,6 +915,8 @@ if (
 
 `closing_sinks` is deliberately not a term here: it is briefly non-zero during a perfectly healthy
 sink swap, so a single reading is not a fault. Watch it over time instead — see the table below.
+`inherited_sink` is not a term either, for a different reason: it reports a *state* the process is
+in rather than a loss it took, and in a prefork deployment it is `True` on every worker by design.
 
 They tell you different things, and they want different responses:
 
@@ -927,6 +929,7 @@ They tell you different things, and they want different responses:
 | `sink.failed` | The sink attempted delivery and could not confirm it — abandoned requests, partially-failed batches, responses it could not adjudicate. | Fix the destination. |
 | `retired` + `submitted_after_shutdown` | `shutdown()` was called and the process **kept logging**. Those events are queued where nothing will drain them — total loss, for as long as the process runs. | Use `flush()`, not `shutdown()`, in a process that logs again. This is the serverless mistake below. |
 | `incomplete_swaps` | A late `configure(sink=...)` could not confirm the previous sink was drained. The swap took effect; that sink was left open and some queued events may have gone to the new one. | Investigate the previous sink — it was hung or failing. Configure the sink before the first log where you can. |
+| `inherited_sink` | This process is delivering to a sink it **inherited across a `fork`** and may not release, so it will not be closed here. Not a loss and not an alert term. | Nothing, usually. It explains a handle still open after `shutdown()`, and tells you a deployment shares one sink across a fork at all. `True` for a shared `StdoutSink` too, whose `close()` only flushes — so a `True` is not by itself evidence that anything is held. If you want the child to own its transport, build the sink in the worker process (see Forking). |
 | `closing_sinks` | Swapped-out sinks inside `close()` **right now** — a live gauge, not a counter, and the only field that falls as well as rises. Non-zero on a single read is normal during a swap. | Nothing, unless it stays non-zero. That means a destination is stuck in `close()` and will not release its resources. |
 
 `h.sink` is a `SinkLosses(dropped, failed)` or `None` — `None` when no worker exists yet, or when

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/pyproject.toml

@@ -20,7 +20,7 @@ dependencies = [
 ]
 
 # Optional features. Install with: pip install log-foundry[aws]
-version = "0.10.2.dev62"
+version = "0.10.2.dev63"
 
 [project.optional-dependencies]
 aws = ["boto3>=1.43.61"]              # SQSSink, SNSSink, KinesisSink, FirehoseSink

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/src/log_foundry/_fork.py

@@ -37,6 +37,36 @@ and all three would have to import this module for it to reach them; instead the
 it, so this module imports nothing that imports it and there is no cycle to break later.
 """
 
+_FORK_SKIP = ("reacquired_in_child",)
+"""This module's own opt-out from the walk below (:data:`_SKIP_ATTRIBUTE`).
+
+:data:`reacquired_in_child` holds **sinks**, so without this the next fork's walk enters the
+previous fork's roster and re-collects every sink in it — including ones ``configure()``
+superseded in between. Measured: a child's walk collected a sink two ``configure()`` calls out
+of date, its hook was called, and the reclaim that follows then stamped it for the grandchild's
+pid, so a sink no descendant ever acquired read ``releasable``. A ratchet, not a one-off: once a
+hook-implementing sink enters a roster, every descendant's walk finds it again.
+
+Exactly the hazard ``_lifecycle._owned`` already declares an opt-out for, reached by adding a
+second global that holds sinks. The rebind below is the belt to this brace.
+"""
+
+reacquired_in_child: list[Any] = []
+"""Sinks whose re-acquisition hook returned normally in this child (SPEC-042 FR-005).
+
+Published rather than acted on, because what it means is an *ownership* claim and ownership is
+``_lifecycle``'s: a sink that returns from the hook holds a transport of its own, so a child may
+release it.
+
+Rebound — never mutated in place — **before the repair walk runs**, so a second fork cannot
+inherit the first one's roster even if the opt-out above were removed. The first version of this
+claimed the rebind happened "at the start of every child repair" while doing it in the middle,
+after the walk; both halves were false and nothing tested either.
+
+This module still imports nothing but ``_diag`` (SPEC-039 FR-006); the reader registers a
+handler and reads this, which is the same inversion :data:`_child_handlers` exists for.
+"""
+
 _installed = False
 """Whether :func:`install` has already registered the child handler in this process.
 
@@ -330,8 +360,12 @@ def _fresh_primitive(value: Any, memo: dict[int, Any], keepalive: list[Any]) ->
     return fresh
 
 
-_DISCARD_HOOK = "discard_buffered_after_fork"
-"""The optional sink member a forked child asks to strand the parent's pending bytes.
+_REACQUIRE_HOOK = "reacquire_after_fork"
+"""The optional sink member a forked child asks to take its transport over.
+
+Named for the larger half of what it does (SPEC-042 FR-005): stranding the parent's pending
+bytes is one consequence, and re-acquiring the transport is the step — a sink that returns from
+it has claimed the transport as this process's own, which is what makes releasing it safe.
 
 Probed by name, as ``losses()`` and ``log_foundry_stop_signal`` are, so no existing sink stops
 satisfying ``Sink`` by not having it. ``sinks/base.py`` states the contract; this is the only
@@ -339,7 +373,7 @@ place the name is read.
 """
 
 
-def _offers_discard(holder: object) -> bool:
+def _offers_reacquire(holder: object) -> bool:
     """Whether this object carries FR-004's buffer hook, without letting the question raise.
 
     Every other read the walk makes is guarded individually, and this one has to be too: an
@@ -361,7 +395,7 @@ def _offers_discard(holder: object) -> bool:
       None.
     """
     try:
-        return callable(getattr(holder, _DISCARD_HOOK, None))
+        return callable(getattr(holder, _REACQUIRE_HOOK, None))
     except Exception:
         return False
 
@@ -428,7 +462,7 @@ def _reinit_primitives() -> list[Any]:
       None.
 
     Returns:
-      The owned instances carrying :data:`_DISCARD_HOOK`, in the order the walk reached them.
+      The owned instances carrying :data:`_REACQUIRE_HOOK`, in the order the walk reached them.
 
     Raises:
       None.
@@ -452,7 +486,7 @@ def _reinit_primitives() -> list[Any]:
         if not isinstance(holder, types.ModuleType | type):
             if not _is_owned(holder):
                 continue
-            if _offers_discard(holder):
+            if _offers_reacquire(holder):
                 buffered.append(holder)
         skip = _skipped_names(holder)
         for name, value in _namespace_items(holder):
@@ -466,19 +500,25 @@ def _reinit_primitives() -> list[Any]:
     return buffered
 
 
-def _discard_buffers(holders: list[Any]) -> None:
-    """Asks each sink that owns a buffered stream to strand what it inherited (FR-004).
+def _reacquire_transports(holders: list[Any]) -> list[Any]:
+    """Asks each sink that owns a buffered stream to re-acquire it in this child (FR-004).
 
     A fork landing inside ``emit`` — after the write loop, before the flush — leaves the child
     holding the parent's unflushed bytes, which both processes then write: measured, the event
     at the fork point appeared on disk twice. Without a ``before`` handler there is nowhere to
-    empty the buffer from (FR-001), so the child throws its copy away instead.
+    empty the buffer from (FR-001), so the child re-opens instead, which strands the copy.
+
+    **Which hooks returned normally is published rather than acted on** (SPEC-042 FR-005). A
+    sink that returns from the hook has claimed the transport as this process's own, which is an
+    ownership fact — but ownership lives in ``_lifecycle``, and this module imports nothing but
+    ``_diag``. So the roster is handed back and the handler ``_lifecycle`` registers re-stamps
+    it.
 
     Args:
       holders: What :func:`_reinit_primitives` collected on its walk.
 
     Returns:
-      None.
+      The holders whose hook returned normally, in the order they were called.
 
     Raises:
       None. One sink's failure is absorbed separately from the rest, so a hook that cannot
@@ -486,15 +526,19 @@ def _discard_buffers(holders: list[Any]) -> None:
         sink its repair — and the consequence is named, because an absorbed failure is
         invisible apart from that line.
     """
+    reacquired: list[Any] = []
     for holder in holders:
         try:
-            getattr(holder, _DISCARD_HOOK)()
+            getattr(holder, _REACQUIRE_HOOK)()
+            reacquired.append(holder)
         except Exception as exc:
             _diag.absorbed(
-                "discarding an inherited buffer after a fork",
+                "re-acquiring a transport after a fork",
                 exc,
-                f"{type(holder).__name__} may write the parent's pending bytes again",
+                f"{type(holder).__name__} may write the parent's pending bytes again, and this "
+                f"child will not release it",
             )
+    return reacquired
 
 
 def register_child_handler(fn: Callable[[], None]) -> None:
@@ -555,12 +599,14 @@ def _reinit_after_fork() -> None:
         registers another simply causes it to run — and :func:`register_child_handler` is the
         only writer, appending only.
     """
+    global reacquired_in_child
+    reacquired_in_child = []
     buffered: list[Any] = []
     try:
         buffered = _reinit_primitives()
     except Exception as exc:
         _diag.absorbed("repairing the library after a fork", exc, "this child may block or lose")
-    _discard_buffers(buffered)
+    reacquired_in_child = _reacquire_transports(buffered)
     for handler in _child_handlers:
         try:
             handler()

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/src/log_foundry/_lifecycle.py

@@ -347,7 +347,12 @@ def _mark_inherited() -> None:
         destructive close. **That covers less than it appears to**: every read the walk makes is
         already absorbed one level down in ``_fork``, which returns empty and announces, so the
         common failure is a *partial* walk that raises nothing and leaves the flag clear. Sinks
-        it did not reach are then unrecorded rather than marked. The outer guard is for a fault
+        it did not reach are then unrecorded rather than marked. **The reclaim below runs
+        either way**: a sink that returned from its hook provably holds its own transport, and a
+        walk that failed is no reason to refuse it forever — which is the outcome
+        :func:`reclaim`'s own docstring says a ``setdefault`` would wrongly cause. The ceiling
+        path already reached it; the exception path did not, and the two differed silently.
+        The outer guard is for a fault
         in this function's own frame — resolving the roots, or an object whose ``__class__``
         property raises, which makes ``isinstance`` raise here. Not a hostile *metaclass*: a
         value's ``__instancecheck__`` is never consulted, since ``Sink``'s own ``_ProtocolMeta``
@@ -384,6 +389,9 @@ def _mark_inherited() -> None:
             for _name, value in _fork._namespace_items(holder):
                 if _fork._is_container(value) or _may_be_a_sink(value):
                     stack.append(value)
+        with _owned_lock:
+            for inherited in found:
+                _owned.setdefault(id(inherited), (_FOREIGN, inherited))
     except Exception as exc:
         _marking_failed = True
         _diag.absorbed(
@@ -391,10 +399,37 @@ def _mark_inherited() -> None:
             exc,
             "this child will refuse to close any sink it has no record of",
         )
-        return
+    for reacquired in _fork.reacquired_in_child:
+        reclaim(reacquired)
+
+
+def reclaim(sink: object) -> None:
+    """Records that a sink re-acquired its transport in this process (SPEC-042 FR-005).
+
+    The one write that **overrides** an existing record, and it has to be: :func:`_mark_inherited`
+    has already stamped everything inherited ``_FOREIGN`` by the time the hook roster is read, so
+    a ``setdefault`` here would leave a sink that provably holds its own descriptor refused
+    forever.
+
+    **It re-stamps the sink that re-acquired, and nothing above it** (FR-005 AC-8). A child
+    inheriting ``MultiSink(FileSink, FileSink)`` re-stamps the two children — only they implement
+    the hook — while the wrapper keeps the parent's mark and stays refused, which leaves the
+    re-acquired children reachable only through a wrapper nothing will release. That is a leak
+    and loses nothing, since ``FileSink.emit`` flushes at the end of every batch, but it is
+    stated rather than discovered.
+
+    Args:
+      sink: The sink whose hook returned normally.
+
+    Returns:
+      None.
+
+    Raises:
+      None.
+    """
+    pid = os.getpid()
     with _owned_lock:
-        for inherited in found:
-            _owned.setdefault(id(inherited), (_FOREIGN, inherited))
+        _owned[id(sink)] = (pid, sink)
 
 
 def stamp(sink: object) -> None:

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/src/log_foundry/decorator.py

@@ -12,7 +12,7 @@ from time import monotonic
 from typing import TYPE_CHECKING, Any, TypeVar, cast, overload
 
 from log_foundry import _diag, _fork, _lifecycle, context
-from log_foundry.config import _ensure_sink
+from log_foundry.config import _ensure_sink, _live_config
 from log_foundry.ids import (
     is_valid_span_id,
     is_valid_trace_id,
@@ -728,6 +728,47 @@ def _flush_worker(timeout: float | None = 5.0) -> FlushResult:
         return FlushResult(ok=False, reason="thread-died")
 
 
+def _delivering_to_an_inherited_sink() -> bool:
+    """Whether the sink this process last installed for delivery is one it may not release.
+
+    Answerable with **no worker**, which is what makes it truthful in a process that only ever
+    logs outside a span — the same refusal :func:`_worker_health` already makes for ``retired``,
+    and for the same reason: standing up a thread to answer ``health()`` is forbidden.
+
+    The three candidates are asked in delivery order: the worker's sink if a worker exists,
+    else the sink an orphan emit reached, else the configured one. SPEC-033's measured
+    disagreement is worker-versus-config, which the **first** term already covers — and the
+    second cannot currently produce a distinct answer, because with no worker ``_orphan_sink``
+    is either ``None`` or the same object the config holds (``_swap_sink`` re-points both, and
+    where it returns early the record is ``None``). It is kept rather than removed, and recorded
+    rather than tested: every other orphan-path read in this module treats ``_orphan_sink`` as
+    the authority on which sink an emit reached, and dropping it here would make one site read
+    the config while its neighbours read the record — resting on a property of ``_swap_sink``'s
+    current body that nothing states as an invariant. A test would have to arrange an
+    unreachable state by hand, which is the fixture-built vacuity this repo keeps finding.
+
+    With no sink resolved at all there is nothing installed and nothing inherited, so the answer
+    is ``False`` rather than a guess.
+
+    Args:
+      None.
+
+    Returns:
+      Whether that one sink carries another process's ownership record.
+
+    Raises:
+      None. ``health()`` is a diagnostic and must not be the reason a caller fails; an
+        unanswerable question reports ``False``, the same direction as a process that never
+        forked.
+    """
+    try:
+        worker = _worker
+        sink = worker.sink if worker is not None else (_orphan_sink or _live_config().sink)
+        return sink is not None and not _lifecycle.releasable(sink)
+    except Exception:
+        return False
+
+
 def _worker_health() -> Health:
     """Snapshots the process worker's counters, or zeros if none was ever created.
 
@@ -772,6 +813,7 @@ def _worker_health() -> Health:
             failed_batches=0,
             retired=_orphan_retired,
             closing_sinks=_lifecycle.closing_count(),
+            inherited_sink=_delivering_to_an_inherited_sink(),
         )
     health = worker.health()
     if _orphan_retired and not health.retired:

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/src/log_foundry/sinks/base.py

@@ -90,13 +90,31 @@ class Sink(Protocol):
     accident, and the cost is one verbose attribute on the sinks that want interruptibility.
 
     A fifth is optional in the same way and exists for one event only:
-    ``discard_buffered_after_fork() -> None``. A sink that owns a **buffered** stream — one it
+    ``reacquire_after_fork() -> None``. A sink that owns a **buffered** stream — one it
     opened itself, rather than the process's ``sys.stdout`` — inherits the parent's unflushed
-    bytes in a forked child, and both processes then write them (SPEC-039 FR-004). Defining this
-    method is how a sink says it can strand what it inherited; a sink holding no buffer of its
-    own needs nothing here. It is *called* rather than assigned, which is why it carries no
-    ``log_foundry_`` prefix: the collision the fourth member's name guards against is an
-    attribute the library writes onto an object it does not own.
+    bytes in a forked child, and both processes then write them (SPEC-039 FR-004). It is
+    *called* rather than assigned, which is why it carries no ``log_foundry_`` prefix: the
+    collision the fourth member's name guards against is an attribute the library writes onto an
+    object it does not own.
+
+    **The contract has two halves and the name says the larger one** (SPEC-042 FR-005). Strand
+    what you inherited — the parent's pending bytes are not yours to write — *and*, by returning
+    normally, **claim the transport as this process's own**. ``FileSink`` satisfies both in one
+    step, because reopening the path is what strands the buffer: measured, parent and child then
+    hold different descriptors. That second half is what the library acts on, so a sink that
+    merely dropped a buffer without re-acquiring would satisfy a name describing only the
+    discard — which this member carried until SPEC-042 — while making a destructive close look
+    safe. A sink that does **not** implement this keeps whatever it inherited, so a child will
+    not release it; one that raises is treated the same way, since a failed re-acquisition is
+    not a claim.
+
+    **Inheriting the hook claims the whole object, so a subclass that adds a transport must
+    override it.** ``class MySink(FileSink)`` that also holds a socket inherits a hook which
+    re-acquires only the *file*, returns normally, and thereby tells the library the child owns
+    everything — after which the child's ``close()`` runs against the parent's connection.
+    Measured. Returning normally is the claim; the library cannot check what was actually
+    re-acquired, which is why the obligation sits here. A subclass that cannot honour it should
+    define the member to raise, which is refused and therefore safe.
 
     **Which sinks are actually asked is narrower than "whichever define it", and the boundary is
     the same one every fork repair has.** The child's repair walks this package's own objects —

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/src/log_foundry/sinks/file.py

@@ -124,8 +124,8 @@ class FileSink:
                 self._stream.write(json.dumps(event) + "\n")
             self._stream.flush()
 
-    def discard_buffered_after_fork(self) -> None:
-        """Throws away the parent's unflushed bytes in a forked child (SPEC-039 FR-004).
+    def reacquire_after_fork(self) -> None:
+        """Re-opens the file so this child holds its own descriptor (SPEC-039 FR-004, SPEC-042).
 
         ``emit`` writes a whole batch into a **buffered** stream and flushes once at the end, so
         a fork landing inside it leaves both processes holding the same pending bytes and both
@@ -138,6 +138,11 @@ class FileSink:
         taking it could only wait on a holder that cannot exist. A hook that blocks here blocks a
         child that has not yet returned from ``fork``, where no watchdog can reach it.
 
+        A **closed** sink returns without re-acquiring anything, which is a trivially true claim
+        rather than an empty one: there is no transport left to hold, so nothing a later close
+        could destroy. Stated because ``sinks/base.py`` says returning normally *is* the claim,
+        and this is the one shipped sink that can return having done nothing.
+
         Args:
           None.
 
@@ -290,10 +295,10 @@ class RotatingFileSink:
                 self._size += data
             self._stream.flush()
 
-    def discard_buffered_after_fork(self) -> None:
-        """Throws away the parent's unflushed bytes in a forked child (SPEC-039 FR-004).
+    def reacquire_after_fork(self) -> None:
+        """Re-opens the file so this child holds its own descriptor (SPEC-039 FR-004, SPEC-042).
 
-        Identical to :meth:`FileSink.discard_buffered_after_fork` and measured on this class
+        Identical to :meth:`FileSink.reacquire_after_fork` and measured on this class
         too, because the window is the same one: a whole batch written into a buffered stream
         and flushed once at the end.
 

{log_foundry-0.10.2.dev62 → log_foundry-0.10.2.dev63}/src/log_foundry/worker.py

@@ -128,6 +128,22 @@ class Health:
         still holds its resources. It is deliberately a live read rather than a count of expired
         joins: a slow close and a stuck one are indistinguishable at the moment a join expires,
         and SPEC-028 reverted a design that guessed.
+      inherited_sink: Whether the sink this process **last installed for delivery** is one it
+        may not release — one it inherited across a ``fork`` (SPEC-042 FR-004). "Last
+        installed", not "would deliver to now": after ``shutdown()`` the process delivers
+        nowhere, and reporting ``True`` there is the point rather than a wrinkle, since an
+        inherited sink left open at exit is exactly what this explains. **The referent is one
+        object, named
+        here because SPEC-033 measured three candidates disagreeing**: the worker's sink if a
+        worker exists, else the sink the orphan path recorded, else the configured one. It
+        describes that object and *not* the graph beneath it, so a child that wraps an
+        inherited sink in a ``MultiSink`` of its own reads ``False`` here while the wrapper's
+        child is still refused — stated because the opposite reading is the natural one.
+        It is a **state, not a fault**, and deliberately not a term in the documented alert
+        idiom, which is the call ``closing_sinks`` got. It explains a handle still open after
+        ``shutdown()``, and it is the signal that a deployment shares a sink across a fork at
+        all. ``True`` for a shared ``StdoutSink`` too, whose ``close()`` only flushes — so a
+        ``True`` is not by itself evidence that anything is held.
     """
 
     queued: int
@@ -139,6 +155,7 @@ class Health:
     submitted_after_shutdown: int = 0
     incomplete_swaps: int = 0
     closing_sinks: int = 0
+    inherited_sink: bool = False
 
 
 class _FlushMarker:
@@ -543,6 +560,7 @@ class Worker:
             submitted_after_shutdown=submitted_after_shutdown,
             incomplete_swaps=incomplete_swaps,
             closing_sinks=_lifecycle.closing_count(),
+            inherited_sink=not _lifecycle.releasable(self.sink),
         )
 
     def _sink_losses(self) -> SinkLosses | None: