log-foundry 0.10.2.dev67__tar.gz → 0.10.2.dev69__tar.gz

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: log-foundry
-Version: 0.10.2.dev67
+Version: 0.10.2.dev69
 Summary: Generate logs for your console and JSON events for downstream consumption.
 License-Expression: MIT
 License-File: LICENSE
@@ -233,6 +233,15 @@ is FIFO, everything submitted before the call is necessarily ahead of that marke
 exactly why the guarantee is "events submitted before this call", and why concurrent
 submissions from other threads may or may not be included.
 
+**`flush()` also sweeps the spans that are still open**, so an in-span event does not have to wait
+for its span to close to be delivered. The span stays open and usable afterwards: its events go
+now and its `span.end` arrives later, in its own batch. Two consequences worth knowing. Boundary
+events swept this way carry the baggage as of the **flush** rather than as of the close, since
+that completion has to happen before they leave. And the sweep reaches only the **calling
+context's** spans — `contextvars` offers no way to enumerate another thread's or task's context,
+so a `flush()` in a handler that fanned out to tasks does not reach what those tasks have
+buffered; their events arrive when their own spans close.
+
 ## Usage
 
 ### `configure(...)`
@@ -413,15 +422,23 @@ def enqueue_check(location: str) -> None:
 
 ```python
 @lf.trace
-def handler(event, context):
+def _handler(event, context):
     lf.continue_trace(event.get("traceparent"), baggage=event.get("baggage"))
     lf.info("inspecting")          # same trace_id as the producer; parent is its span
+    return inspect(event)
+
+
+def handler(event, context):       # the entry point, deliberately not decorated
     try:
-        return inspect(event)
+        return _handler(event, context)
     finally:
-        lf.flush()
+        lf.flush()                 # the span has closed, so its events are drained
 ```
 
+`flush()` goes **outside** the traced function, not in its `finally`. An in-span event lives on the
+span until the span *closes*, and `flush()` drains the queue — so a `flush()` inside the span has
+nothing to drain yet.
+
 | Call | Does |
 |---|---|
 | `continue_trace(traceparent=None, *, trace_id=None, parent_span_id=None, baggage=None)` | Adopt an inbound context. Returns a `ContinueResult`: truthy if adopted, else falsy with `reason` of `"nothing-supplied"` or `"rejected"`. The verdict is about the **trace context** — `baggage=` is merged independently and does not make it truthy. Never raises. |
@@ -953,6 +970,7 @@ if (
     h.dropped or h.failed_batches or h.stopped_reason or h.incomplete_swaps
     or (h.sink and (h.sink.dropped or h.sink.failed))
     or (h.retired and h.submitted_after_shutdown)
+    or h.orphan_lost or h.in_span_lost
 ):
     ...  # logs were silently lost — worth an alert
 ```
@@ -974,8 +992,14 @@ They tell you different things, and they want different responses:
 | `retired` + `submitted_after_shutdown` | `shutdown()` was called and the process **kept logging**. Those events are queued where nothing will drain them — total loss, for as long as the process runs. | Use `flush()`, not `shutdown()`, in a process that logs again. This is the serverless mistake below. |
 | `incomplete_swaps` | A late `configure(sink=...)` could not confirm the previous sink was drained. The swap took effect; that sink was left open and some queued events may have gone to the new one. | Investigate the previous sink — it was hung or failing. Configure the sink before the first log where you can. |
 | `inherited_sink` | This process is delivering to a sink it **inherited across a `fork`** and may not release, so it will not be closed here. Not a loss and not an alert term. | Nothing, usually. It explains a handle still open after `shutdown()`, and tells you a deployment shares one sink across a fork at all. `True` for a shared `StdoutSink` too, whose `close()` only flushes — so a `True` is not by itself evidence that anything is held. If you want the child to own its transport, build the sink in the worker process (see Forking). |
+| `orphan_lost` | An event logged **with no active span** never reached the sink. That call emits on your own thread with no worker behind it, so no other field here can carry it — it is not a batch, there was no retry, and there may be no worker at all. Covers a sink that failed to *construct* as well as one that raised. | Fix the destination, or the data. The stderr line names the exception type. If a process logs this way at all, this is the field to alert on: nothing else describes that path. |
+| `in_span_lost` | An event logged **inside a span** could not be built — a value that could not be turned into an event. Always the data, never the destination: the in-span path cannot fail at delivery, which is `failed_batches`. | Fix the call site. Passing a non-string message (an exception object, say) is the common cause. |
 | `closing_sinks` | Swapped-out sinks inside `close()` **right now** — a live gauge, not a counter, and the only field that falls as well as rises. Non-zero on a single read is normal during a swap. | Nothing, unless it stays non-zero. That means a destination is stuck in `close()` and will not release its resources. |
 
+`orphan_lost` and `in_span_lost` are deliberately two fields and their sum is deliberately not
+reported. They aggregate different failure populations — one can mean the destination *or* the
+data, the other can only mean the data — so a single number would hide which fix applies.
+
 `h.sink` is a `SinkLosses(dropped, failed)` or `None` — `None` when no worker exists yet, or when
 the configured sink reports nothing (`losses()` is optional). Note the two `dropped` fields count
 different things: the worker's is backpressure at *its* queue, the sink's is an event that never
@@ -996,14 +1020,21 @@ logged, and after a clean `shutdown()`, so a plain truthiness check is safe. Wit
 thread showed up only indirectly, as `dropped` climbing once the queue filled — the wrong signal,
 pointing at the wrong fix.
 
+`orphan_lost` climbing is on its own a reason to look: unlike `dropped`, it is never
+backpressure and never transient. Each increment is one event that reached no destination, and on
+a process that logs only outside a span it is the *only* field that can say so.
+
 `retired` is deliberately **not** alerted on by itself. A process that shuts down and then stops
 logging is doing the right thing; it is the *pair* — retired, and still being handed events — that
 means every log line since the shutdown has gone nowhere. That state used to read as perfectly
 healthy: `stopped_reason` is `None` after a clean shutdown, and the queue simply grows.
 
-`retired` is also the one field reported for a process that has **no worker at all**. A program
-that only ever calls `info()`/`error()` outside a span emits synchronously and builds no background
-worker, so every other field describes something that does not exist and reads zero. Its
+`retired`, `orphan_lost` and `in_span_lost` are the fields reported for a process that has **no
+worker at all**. A program that only ever calls `info()`/`error()` outside a span emits
+synchronously and builds no background worker, so the rest describe something that does not exist
+and read zero — which is why that path needs counters of its own. Until it had them, such a process
+reported `queued=0 dropped=0 failed_batches=0 stopped_reason=None` over total, permanent loss, and
+the only thing that said otherwise was a line on stderr. Its
 `shutdown()` still closes the sink, exactly once and without starting a thread, and `retired` reads
 `True` afterwards rather than staying vacuously `False`. `submitted_after_shutdown` stays `0` there
 by design: a later level call is *refused* at the closed sink and announced on stderr — if the sink
@@ -1124,19 +1155,28 @@ from log_foundry.sinks.sqs import SQSSink
 lf.configure(service="billing-api", env="prod", sink=SQSSink(queue_url=QUEUE_URL))
 
 @lf.trace
-def handler(event, context):
+def _handler(event, context):
     lf.info("received", records=len(event["Records"]))
+    return do_work(event)
+
+
+def handler(event, context):
+    # NOT decorated, so the span closes when `_handler` returns and its events reach the queue
+    # before `flush()` runs. A `flush()` *inside* the traced function has nothing to drain yet.
     try:
-        return do_work(event)
+        return _handler(event, context)
     finally:
         # In `finally`: the failed invocation is the one worth logging. NEVER shutdown() here —
         # the worker does not come back, and every later invocation on this warm container
         # would log nothing.
         drained = lf.flush()
         h = lf.health()
-        if not drained or h.failed_batches or h.dropped or h.stopped_reason or h.retired:
+        if (not drained or h.failed_batches or h.dropped or h.stopped_reason or h.retired
+                or h.orphan_lost or h.in_span_lost):
             # `drained` covers this invocation's tail; the counters cover anything the worker
             # lost earlier — a batch its own interval trigger already gave up on, for instance.
+            # `orphan_lost`/`in_span_lost` cover the two paths no worker field can describe: a
+            # log made outside any span, and an event that could not be built.
             # `h.retired` catches the mistake above: inside a handler it can only mean something
             # called shutdown(), and from here on this container logs nothing.
             # Emitting this through your platform's own logger keeps it outside the pipeline

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/README.md

@@ -197,6 +197,15 @@ is FIFO, everything submitted before the call is necessarily ahead of that marke
 exactly why the guarantee is "events submitted before this call", and why concurrent
 submissions from other threads may or may not be included.
 
+**`flush()` also sweeps the spans that are still open**, so an in-span event does not have to wait
+for its span to close to be delivered. The span stays open and usable afterwards: its events go
+now and its `span.end` arrives later, in its own batch. Two consequences worth knowing. Boundary
+events swept this way carry the baggage as of the **flush** rather than as of the close, since
+that completion has to happen before they leave. And the sweep reaches only the **calling
+context's** spans — `contextvars` offers no way to enumerate another thread's or task's context,
+so a `flush()` in a handler that fanned out to tasks does not reach what those tasks have
+buffered; their events arrive when their own spans close.
+
 ## Usage
 
 ### `configure(...)`
@@ -377,15 +386,23 @@ def enqueue_check(location: str) -> None:
 
 ```python
 @lf.trace
-def handler(event, context):
+def _handler(event, context):
     lf.continue_trace(event.get("traceparent"), baggage=event.get("baggage"))
     lf.info("inspecting")          # same trace_id as the producer; parent is its span
+    return inspect(event)
+
+
+def handler(event, context):       # the entry point, deliberately not decorated
     try:
-        return inspect(event)
+        return _handler(event, context)
     finally:
-        lf.flush()
+        lf.flush()                 # the span has closed, so its events are drained
 ```
 
+`flush()` goes **outside** the traced function, not in its `finally`. An in-span event lives on the
+span until the span *closes*, and `flush()` drains the queue — so a `flush()` inside the span has
+nothing to drain yet.
+
 | Call | Does |
 |---|---|
 | `continue_trace(traceparent=None, *, trace_id=None, parent_span_id=None, baggage=None)` | Adopt an inbound context. Returns a `ContinueResult`: truthy if adopted, else falsy with `reason` of `"nothing-supplied"` or `"rejected"`. The verdict is about the **trace context** — `baggage=` is merged independently and does not make it truthy. Never raises. |
@@ -917,6 +934,7 @@ if (
     h.dropped or h.failed_batches or h.stopped_reason or h.incomplete_swaps
     or (h.sink and (h.sink.dropped or h.sink.failed))
     or (h.retired and h.submitted_after_shutdown)
+    or h.orphan_lost or h.in_span_lost
 ):
     ...  # logs were silently lost — worth an alert
 ```
@@ -938,8 +956,14 @@ They tell you different things, and they want different responses:
 | `retired` + `submitted_after_shutdown` | `shutdown()` was called and the process **kept logging**. Those events are queued where nothing will drain them — total loss, for as long as the process runs. | Use `flush()`, not `shutdown()`, in a process that logs again. This is the serverless mistake below. |
 | `incomplete_swaps` | A late `configure(sink=...)` could not confirm the previous sink was drained. The swap took effect; that sink was left open and some queued events may have gone to the new one. | Investigate the previous sink — it was hung or failing. Configure the sink before the first log where you can. |
 | `inherited_sink` | This process is delivering to a sink it **inherited across a `fork`** and may not release, so it will not be closed here. Not a loss and not an alert term. | Nothing, usually. It explains a handle still open after `shutdown()`, and tells you a deployment shares one sink across a fork at all. `True` for a shared `StdoutSink` too, whose `close()` only flushes — so a `True` is not by itself evidence that anything is held. If you want the child to own its transport, build the sink in the worker process (see Forking). |
+| `orphan_lost` | An event logged **with no active span** never reached the sink. That call emits on your own thread with no worker behind it, so no other field here can carry it — it is not a batch, there was no retry, and there may be no worker at all. Covers a sink that failed to *construct* as well as one that raised. | Fix the destination, or the data. The stderr line names the exception type. If a process logs this way at all, this is the field to alert on: nothing else describes that path. |
+| `in_span_lost` | An event logged **inside a span** could not be built — a value that could not be turned into an event. Always the data, never the destination: the in-span path cannot fail at delivery, which is `failed_batches`. | Fix the call site. Passing a non-string message (an exception object, say) is the common cause. |
 | `closing_sinks` | Swapped-out sinks inside `close()` **right now** — a live gauge, not a counter, and the only field that falls as well as rises. Non-zero on a single read is normal during a swap. | Nothing, unless it stays non-zero. That means a destination is stuck in `close()` and will not release its resources. |
 
+`orphan_lost` and `in_span_lost` are deliberately two fields and their sum is deliberately not
+reported. They aggregate different failure populations — one can mean the destination *or* the
+data, the other can only mean the data — so a single number would hide which fix applies.
+
 `h.sink` is a `SinkLosses(dropped, failed)` or `None` — `None` when no worker exists yet, or when
 the configured sink reports nothing (`losses()` is optional). Note the two `dropped` fields count
 different things: the worker's is backpressure at *its* queue, the sink's is an event that never
@@ -960,14 +984,21 @@ logged, and after a clean `shutdown()`, so a plain truthiness check is safe. Wit
 thread showed up only indirectly, as `dropped` climbing once the queue filled — the wrong signal,
 pointing at the wrong fix.
 
+`orphan_lost` climbing is on its own a reason to look: unlike `dropped`, it is never
+backpressure and never transient. Each increment is one event that reached no destination, and on
+a process that logs only outside a span it is the *only* field that can say so.
+
 `retired` is deliberately **not** alerted on by itself. A process that shuts down and then stops
 logging is doing the right thing; it is the *pair* — retired, and still being handed events — that
 means every log line since the shutdown has gone nowhere. That state used to read as perfectly
 healthy: `stopped_reason` is `None` after a clean shutdown, and the queue simply grows.
 
-`retired` is also the one field reported for a process that has **no worker at all**. A program
-that only ever calls `info()`/`error()` outside a span emits synchronously and builds no background
-worker, so every other field describes something that does not exist and reads zero. Its
+`retired`, `orphan_lost` and `in_span_lost` are the fields reported for a process that has **no
+worker at all**. A program that only ever calls `info()`/`error()` outside a span emits
+synchronously and builds no background worker, so the rest describe something that does not exist
+and read zero — which is why that path needs counters of its own. Until it had them, such a process
+reported `queued=0 dropped=0 failed_batches=0 stopped_reason=None` over total, permanent loss, and
+the only thing that said otherwise was a line on stderr. Its
 `shutdown()` still closes the sink, exactly once and without starting a thread, and `retired` reads
 `True` afterwards rather than staying vacuously `False`. `submitted_after_shutdown` stays `0` there
 by design: a later level call is *refused* at the closed sink and announced on stderr — if the sink
@@ -1088,19 +1119,28 @@ from log_foundry.sinks.sqs import SQSSink
 lf.configure(service="billing-api", env="prod", sink=SQSSink(queue_url=QUEUE_URL))
 
 @lf.trace
-def handler(event, context):
+def _handler(event, context):
     lf.info("received", records=len(event["Records"]))
+    return do_work(event)
+
+
+def handler(event, context):
+    # NOT decorated, so the span closes when `_handler` returns and its events reach the queue
+    # before `flush()` runs. A `flush()` *inside* the traced function has nothing to drain yet.
     try:
-        return do_work(event)
+        return _handler(event, context)
     finally:
         # In `finally`: the failed invocation is the one worth logging. NEVER shutdown() here —
         # the worker does not come back, and every later invocation on this warm container
         # would log nothing.
         drained = lf.flush()
         h = lf.health()
-        if not drained or h.failed_batches or h.dropped or h.stopped_reason or h.retired:
+        if (not drained or h.failed_batches or h.dropped or h.stopped_reason or h.retired
+                or h.orphan_lost or h.in_span_lost):
             # `drained` covers this invocation's tail; the counters cover anything the worker
             # lost earlier — a batch its own interval trigger already gave up on, for instance.
+            # `orphan_lost`/`in_span_lost` cover the two paths no worker field can describe: a
+            # log made outside any span, and an event that could not be built.
             # `h.retired` catches the mistake above: inside a handler it can only mean something
             # called shutdown(), and from here on this container logs nothing.
             # Emitting this through your platform's own logger keeps it outside the pipeline

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/pyproject.toml

@@ -20,7 +20,7 @@ dependencies = [
 ]
 
 # Optional features. Install with: pip install log-foundry[aws]
-version = "0.10.2.dev67"
+version = "0.10.2.dev69"
 
 [project.optional-dependencies]
 aws = ["boto3>=1.43.61"]              # SQSSink, SNSSink, KinesisSink, FirehoseSink

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/__init__.py

@@ -72,9 +72,17 @@ def health() -> Health:
         h = log_foundry.health()
         if h.dropped or h.failed_batches or h.stopped_reason or h.incomplete_swaps or (
             h.sink and (h.sink.dropped or h.sink.failed)
-        ) or (h.retired and h.submitted_after_shutdown):
+        ) or (h.retired and h.submitted_after_shutdown) or h.orphan_lost or h.in_span_lost:
             ...  # raise an alert; logs were silently lost
 
+    ``orphan_lost`` and ``in_span_lost`` are the two terms that do **not** describe the worker
+    (SPEC-036 FR-003). A level call made with no active span emits on the caller's own thread, and
+    an event that cannot be *built* never reaches a queue at all — so every other field here
+    describes machinery those two losses never touched, and a process that only logs outside a
+    span read all zeros over total loss until they existed. They stay separate because one can
+    mean the destination or the data and the other can only mean the data; their sum is a number
+    nobody can act on.
+
     ``retired`` alone is not a fault — a process that shuts down and then stops logging is
     doing the right thing, which is why it is paired with the count rather than alerted on.
     ``closing_sinks`` is deliberately absent for the same kind of reason: it is briefly non-zero

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/api.py

@@ -8,7 +8,7 @@ from log_foundry import _diag, context
 from log_foundry.config import _ensure_sink
 from log_foundry.console import ConsoleWriter
 from log_foundry.context import set_baggage
-from log_foundry.decorator import _note_orphan_emit
+from log_foundry.decorator import _note_in_span_loss, _note_orphan_emit, _note_orphan_loss
 from log_foundry.ids import new_span_id, new_trace_id
 from log_foundry.model import Span, build_event
 
@@ -71,6 +71,30 @@ def _log(level: str, message: str, echo: bool, fields: dict[str, object]) -> Non
     A guard that reads "this cannot fail" is a guard that stops being true when the code under
     it changes, and this one had already stopped.
 
+    **An append to a span that has already closed takes the orphan route** (SPEC-036 FR-004).
+    ``contextvars`` copies the same ``Span`` object into every task created inside a span, so a
+    fire-and-forget ``create_task`` can log after its parent returned — and since ``_flush``
+    detaches at submit, that append lands in a buffer nothing will ever emit. This is the only
+    place that can notice, because nothing in the library reads a span again after it closes; a
+    post-hoc check of the buffer has no observer. A span that has closed is not a span, so the
+    event becomes a fresh one-event span and inherits the orphan path's accounting entire —
+    delivered on success, ``orphan_lost`` on failure. The flag is read before the event is built
+    and the append happens after, so a thread *sharing* the span can still slip between the two and
+    strand its event in the post-swap list; that window is recorded in ``architecture.md`` §13
+    rather than closed, since closing it costs a per-span lock on the hottest path. That is why FR-004 adds no ``Health`` field
+    of its own, and why the destination is named here rather than left implied.
+
+    The cost is stated rather than hidden: the event gets a **fresh** ``trace_id``, so it leaves
+    its trace, not merely its span. ``contextvars`` cannot deliver the alternative once the span
+    is gone, and the choice is against losing the event outright.
+
+    **Both losses are counted, and counted apart** (SPEC-036 FR-003). Each guard records before it
+    announces, so the stderr line SPEC-025 already writes is unchanged and this adds a counter
+    rather than a second announcement. The orphan increment sits in the ``except`` rather than
+    after ``sink.emit``, which is what makes a sink that fails to *construct* count too. They stay
+    two fields because the populations differ: this branch can fail at the destination, the
+    in-span branch can only fail at the data.
+
     The echo runs after the emit, so a closed or redirected stream never costs the event itself.
 
     Args:
@@ -89,11 +113,12 @@ def _log(level: str, message: str, echo: bool, fields: dict[str, object]) -> Non
     baggage = context._live_baggage()
     span = context.current_span()
     event: dict[str, object] | None = None
-    if span is not None:
+    if span is not None and not span.closed:
         try:
             event = build_event(span, level, message, fields=fields, baggage=baggage)
             span.events.append(event)
         except Exception as exc:
+            _note_in_span_loss()
             _diag.absorbed("building an in-span log", exc, "the event was lost")
     else:
         try:
@@ -109,6 +134,7 @@ def _log(level: str, message: str, echo: bool, fields: dict[str, object]) -> Non
             _note_orphan_emit(sink)
             sink.emit([event])
         except Exception as exc:
+            _note_orphan_loss()
             _diag.absorbed("emitting an orphan log", exc, "the event was lost")
     if echo and event is not None:
         try:

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/context.py

@@ -55,6 +55,27 @@ def current_span() -> Span | None:
     return stack[-1] if stack else None
 
 
+def _live_span_stack() -> tuple[Span, ...]:
+    """Returns every span open on this context, outermost first (SPEC-036 FR-001).
+
+    The library's non-copying read, named as :func:`_live_baggage` is and distinguished from the
+    public accessors for the same reason (SPEC-034 FR-003): a caller gets a copy, the library
+    reads the live object. There is nothing to copy here in any case — the stack is a tuple, so
+    handing it out cannot let anyone mutate it. :func:`current_span` returns only the innermost,
+    which is what an event needs and not what a sweep does.
+
+    Args:
+      None.
+
+    Returns:
+      Every span open on this context, outermost first. Empty when none is.
+
+    Raises:
+      None.
+    """
+    return _span_stack.get()
+
+
 def push_span(span: Span) -> contextvars.Token[tuple[Span, ...]]:
     """Pushes a span onto the stack.
 

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/decorator.py

@@ -60,6 +60,26 @@ shutdown's event has every backoff collapsed to zero.
 """
 _orphan_retired = False
 
+_sweep_lock = threading.Lock()
+"""Serializes the span sweep, so two threads cannot deliver one span's buffer twice.
+
+The detach is a load and a store, and ``contextvars`` copies the same ``Span`` object into every
+task and into any ``copy_context()`` thread, so the two-reader case is ordinary rather than exotic
+(SPEC-036 FR-001 AC-10). A flush is not a hot path; a sink lock it is not competing with.
+"""
+_loss_lock = threading.Lock()
+"""Guards the two loss counters, and deliberately not ``_worker_lock`` (SPEC-036 FR-003 AC-5).
+
+SPEC-028's ordering rule: a counter takes its own lock, because the orphan path runs on arbitrary
+application threads and ``_worker_lock`` is held across ``Worker(_ensure_sink())`` in
+:func:`_get_worker` — a blocking build a counter increment must never queue behind. It cannot
+deadlock here either: the increment sits in ``api._log``'s ``except``, where
+:func:`_note_orphan_emit` has already released ``_worker_lock`` and the propagating exception has
+released any sink lock.
+"""
+_orphan_lost = 0
+_in_span_lost = 0
+
 F = TypeVar("F", bound=Callable[..., Any])
 
 
@@ -184,6 +204,14 @@ def continue_trace(
         _diag.rejected("parent_span_id given with no trace_id to join", parent_span_id)
         announced = True
 
+    if adopted is not None and _current_span_was_swept():
+        _diag.rejected(
+            "the current span has already been flushed; trace context refused",
+            traceparent if traceparent is not None else str(trace_id),
+        )
+        adopted = None
+        announced = True
+
     if adopted is not None:
         context.set_adopted_context(*adopted)
         _reparent_current_span(*adopted)
@@ -201,6 +229,32 @@ def continue_trace(
     return ContinueResult(ok=False, reason="rejected" if announced else "nothing-supplied")
 
 
+def _current_span_was_swept() -> bool:
+    """Reports whether an in-span ``flush()`` has already shipped this span's events.
+
+    Read from ``context.current_span()`` — what :func:`_reparent_current_span` itself reads —
+    **and only when that span is a root**, which is the other half of that function's own guard.
+    A swept *child* is not a reason to refuse: the re-parent would have returned early on it and
+    rewritten nothing, so a refusal there prevents no corruption. It would still be wrong to
+    refuse — the two guards must agree, or the refusal fires where the thing it guards does not
+    run — though the adoption it spares reaches less than it appears to: SPEC-024 clears the
+    adopted context at the **root** span's close, so one made inside a child does not survive to
+    the next root span either. ``continue_trace``'s documented placement on the entry
+    point's first line is untouched either way: nothing has been swept that early.
+
+    Args:
+      None.
+
+    Returns:
+      Whether the current span is a root that has been swept.
+
+    Raises:
+      None.
+    """
+    span = context.current_span()
+    return span is not None and span.parent_span_id is None and span.swept
+
+
 def _reparent_current_span(trace_id: str, parent_span_id: str | None) -> None:
     """Moves an already-open root span into the adopted trace, events included.
 
@@ -699,33 +753,191 @@ def _adopt_declined_swap(new_sink: Sink) -> None:
         _orphan_sink = new_sink
 
 
+def _sweep_open_spans() -> None:
+    """Hands the worker every event buffered on an open span in this context (SPEC-036 FR-001).
+
+    An in-span event lives on ``span.events`` until the span *closes*, and ``Worker.flush``
+    drains the *queue* — so a ``flush()`` called inside a ``@trace``d function, which is where
+    the README's serverless recipe put it, had by construction nothing to drain. Measured: zero
+    of two events delivered, every counter clean, and ``FlushResult`` reporting ``reason=None``.
+
+    The span stays **open**: its events go now and its ``span.end`` arrives later in its own
+    batch. Closing and reopening was rejected — it would emit a ``span.end`` the function never
+    reached, with a fabricated ``duration_ms`` and ``status``.
+
+    Two things must happen before the events leave, and both are why this is not a one-liner.
+    The boundary events are backfilled **first**, because SPEC-015 completes them at close by
+    iterating ``span.events`` and a swept buffer would ship ``span.start`` with ``fields={}`` —
+    the very defect that spec exists to fix, recreated by any in-span flush. They therefore carry
+    the baggage as of the flush rather than as of the close, which is a real semantic change and
+    the alternative is mutating an event the worker already owns (SPEC-028). And the buffer is
+    **detached by swap**, never cleared: ``clear()`` empties the same list object the worker was
+    handed.
+
+    The worker is created when there is something to submit, and **resolved before the buffer is
+    detached**. That ordering is the whole of the difference between a lost batch and a delivered
+    one: ``_get_worker`` can raise — it ends in ``Thread.start()`` — and a detach that has already
+    happened leaves the events in a discarded local while the span reads empty and ``flush()``
+    reports success. Measured with the failure injected: 3 of 4 events destroyed, every counter
+    zero, on a span that was still open and would have delivered them at its close.
+    ``Worker.submit`` raises nothing, so once it is reached the batch is safe. Creating the worker
+    at all narrows SPEC-013's refusal rather than contradicting it — that exists so an *empty*
+    flush does not stand up a thread, and a sweep that found buffered events is not an empty
+    flush. A cold-start Lambda flushing before it returns is exactly this case: the worker is
+    built when the first span *closes*, so inside the first traced call there is none.
+
+    Concurrent sweeps are serialized on ``_sweep_lock``. The detach is a load and a store with a
+    real gap between them, and two threads sharing one ``Span`` — which ``contextvars`` makes
+    ordinary — can both read the same buffer and deliver it twice: measured, all 8 events
+    duplicated with the window held open, and 9 of 25 runs with only a GIL yield between them.
+    Rarely preempted on today's build is not a guarantee, and the floor is ``>=3.12`` where a
+    free-threading build removes even that. A flush is not a hot path, so a single lock is the
+    right cost.
+
+    **The detach stays one statement, and the two orderings above are not in tension.** A draft
+    hoisted the load to the top of the loop so a test could park on it — which put
+    ``_get_worker()``, and therefore ``Thread.start()``, *inside* the load-to-store gap: measured,
+    a sweep racing a close then delivered the whole batch twice, two ``span.end`` events among
+    them, in 67 of 100 unforced trials against 0 before.
+
+    One statement makes that gap **narrow, not closed**, and the difference matters. It compiles
+    to ``LOAD_ATTR … STORE_ATTR`` with no ``CALL`` between, so CPython's eval breaker never runs
+    there and today's GIL cannot switch inside it — 0 of 500 unforced trials. Forced with an
+    opcode-level preemption it reproduces 10 of 10, and a free-threaded build removes the
+    accident entirely while ``requires-python`` has no upper bound. So :func:`_flush` takes this
+    same lock rather than relying on the width of a window: that is the *detach-vs-detach* race,
+    and a process-global lock is the right instrument for it. The **append** race
+    (``api._log`` versus a detach) is a different window needing a per-span lock, and
+    ``architecture.md`` §13 declines it on cost.
+
+    It reaches only the calling context's spans. ``contextvars`` offers no way to enumerate
+    another thread's or task's context, so a ``flush()`` in a handler that fanned out does not
+    reach what those tasks buffered.
+
+    Args:
+      None.
+
+    Returns:
+      None.
+
+    Raises:
+      Exception: Whatever building the worker raises. :func:`_flush_worker` guards it, because a
+        flush is the call most likely to be made in a ``finally``.
+    """
+    with _sweep_lock:
+        for span in context._live_span_stack():
+            if not span.events:
+                span.swept = True
+                continue
+            worker = _get_worker()
+            backfill_baggage(span, context._live_baggage())
+            span.swept = True
+            buffered, span.events = span.events, []
+            worker.submit(buffered)
+
+
 def _flush_worker(timeout: float | None = 5.0) -> FlushResult:
     """Drains the process worker without retiring it, backing ``flush()`` (SPEC-013 FR-003).
 
-    This deliberately does not call :func:`_get_worker`: a process that never logged has
-    nothing to drain, and building a worker — with the thread and ``atexit`` registration
-    that brings — in order to flush nothing would be pure cost.
+    ~~This deliberately does not call :func:`_get_worker`~~ — narrowed by SPEC-036 FR-001. The
+    refusal still holds for an *empty* flush: a process that never logged has nothing to drain,
+    and building a worker — with the thread and ``atexit`` registration that brings — in order to
+    flush nothing would be pure cost. What changed is that :func:`_sweep_open_spans`, which runs
+    first, does build one when it finds buffered events on an open span, because submitting them
+    into a worker that does not exist delivers nothing and still reports success.
 
     Args:
       timeout: Seconds to wait for the drain, or ``None`` to wait indefinitely.
 
     Returns:
       A :class:`FlushResult`, truthy when everything outstanding was delivered and when no
-      worker exists — a process that never logged has nothing to drain, so it has lost
-      nothing.
+      worker exists — a process that never logged has nothing to drain, so it has lost nothing.
+      A sweep that could not hand its buffers over reports ``"abandoned"``, the existing token
+      for "this call did not deliver them" (SPEC-036 FR-001): the events are still on their open
+      spans and their close may yet carry them, but the caller asked *now*, and on the
+      cold-start path this exists for there may be no close — reporting success there is the
+      exact shape the spec was written to remove. The drain still runs, so whatever was
+      submitted before the failure is not held back by it.
 
     Raises:
       None. A flush is the call most likely to be made in a ``finally``, so the library must
         never be the reason a caller's function fails; a failure is reported by the return
         value instead (FR-003).
     """
+    swept = True
+    try:
+        _sweep_open_spans()
+    except Exception as exc:
+        _diag.absorbed("sweeping open spans for a flush", exc, "buffered events were not swept")
+        swept = False
     worker = _worker
     if worker is None:
-        return FlushResult(ok=True)
+        return FlushResult(ok=True) if swept else FlushResult(ok=False, reason="abandoned")
     try:
-        return worker.flush(timeout)
+        if swept:
+            return worker.flush(timeout)
+        worker.flush(timeout)
     except Exception:
         return FlushResult(ok=False, reason="thread-died")
+    return FlushResult(ok=False, reason="abandoned")
+
+
+def _note_orphan_loss() -> None:
+    """Counts one event lost on the synchronous path (SPEC-036 FR-003).
+
+    Called from ``api._log``'s orphan ``except``, which wraps the span construction, the event
+    build, ``_ensure_sink`` and the emit — so a sink that failed to *construct* is counted here
+    too, which an increment placed after ``sink.emit`` would miss.
+
+    Args:
+      None.
+
+    Returns:
+      None.
+
+    Raises:
+      None.
+    """
+    global _orphan_lost
+    with _loss_lock:
+        _orphan_lost += 1
+
+
+def _note_in_span_loss() -> None:
+    """Counts one event lost while being built inside a span (SPEC-036 FR-003).
+
+    Separate from :func:`_note_orphan_loss` because the two aggregate different failure
+    populations: this path cannot fail at ``emit``, so a non-zero count here always means the
+    data, never the destination.
+
+    Args:
+      None.
+
+    Returns:
+      None.
+
+    Raises:
+      None.
+    """
+    global _in_span_lost
+    with _loss_lock:
+        _in_span_lost += 1
+
+
+def _read_losses() -> tuple[int, int]:
+    """Reads both loss counters under the lock they are written under.
+
+    Args:
+      None.
+
+    Returns:
+      The orphan and in-span loss counts, in that order.
+
+    Raises:
+      None.
+    """
+    with _loss_lock:
+        return _orphan_lost, _in_span_lost
 
 
 def _delivering_to_an_inherited_sink() -> bool:
@@ -786,6 +998,12 @@ def _worker_health() -> Health:
     defines that count as submissions queued where nothing will drain them, and a later orphan
     log is refused at the closed sink and announced instead. The two are not the same claim.
 
+    The two loss counters are synthesized on **both** branches, for the reason ``retired`` is on
+    one: they describe the caller's own path, not the worker's, and ``Worker`` cannot report them
+    because it does not know they exist — ``worker.py`` imports nothing from this module, and the
+    reverse read would be a cycle. A process that only ever logged outside a span has no worker
+    and is exactly the process whose loss they exist to show (SPEC-036 FR-003 AC-7).
+
     The synthesis also survives a worker built *after* that shutdown, which is why it is an
     ``or`` rather than a fallback. An orphan-only ``shutdown()`` leaves ``_worker`` unset, so a
     later ``@trace`` constructs a fresh worker whose own ``retired`` is ``False`` — and reading
@@ -806,6 +1024,7 @@ def _worker_health() -> Health:
       None.
     """
     worker = _worker
+    orphan_lost, in_span_lost = _read_losses()
     if worker is None:
         return Health(
             queued=0,
@@ -814,11 +1033,14 @@ def _worker_health() -> Health:
             retired=_orphan_retired,
             closing_sinks=_lifecycle.closing_count(),
             inherited_sink=_delivering_to_an_inherited_sink(),
+            orphan_lost=orphan_lost,
+            in_span_lost=in_span_lost,
         )
     health = worker.health()
-    if _orphan_retired and not health.retired:
-        return replace(health, retired=True)
-    return health
+    retired = _orphan_retired or health.retired
+    return replace(
+        health, retired=retired, orphan_lost=orphan_lost, in_span_lost=in_span_lost
+    )
 
 
 def _flush(span: Span) -> None:
@@ -828,6 +1050,24 @@ def _flush(span: Span) -> None:
     ``_ensure_sink``, so a zero-config ``@trace`` still falls back to ``StdoutSink`` rather
     than crashing.
 
+    **The buffer is detached, not handed over** (SPEC-036 FR-004). ``submit`` used to take the
+    live list, so a task outliving its parent span appended to a list the worker already owned:
+    under the flush interval the event was delivered but ordered after ``span.end``, and over it
+    silently lost — a pure race on a timer. The swap gives the worker the old list and leaves the
+    span a fresh one, so the outcome stops depending on timing. A copy would do the same and
+    allocates a second list per span on the hottest path in the library; the swap is free.
+
+    The late append is now landing in a buffer nothing will emit, which is *also* loss — that
+    half is ``api._log``'s, keyed on :attr:`Span.closed`.
+
+    It takes ``_sweep_lock`` for the detach, because :func:`_sweep_open_spans` performs the same
+    detach on the same attribute and a span can be swept and closed concurrently — measured, the
+    whole batch delivered twice with two ``span.end`` events among them. The hold covers one
+    statement and not the submit; ``Worker.submit`` is a ``put_nowait`` that never blocks, and the
+    cost of the lock on the traced path measured within noise (+0.4% single-threaded, +0.9% across
+    eight threads, 20,000 spans each). The **append** window this does not close is a different
+    one, needs a per-span lock, and is declined in ``architecture.md`` §13.
+
     Args:
       span: The finished span whose buffered events are submitted.
 
@@ -837,7 +1077,9 @@ def _flush(span: Span) -> None:
     Raises:
       Exception: Whatever creating the worker or submitting raises; :func:`_end` is the guard.
     """
-    _get_worker().submit(span.events)
+    with _sweep_lock:
+        events, span.events = span.events, []
+    _get_worker().submit(events)
 
 
 type _SpanScope = tuple[
@@ -947,6 +1189,12 @@ def _close_span(span: Span, status: str, exc: BaseException | None) -> None:
     baggage known at their construction, empty for ``span.start``, so SPEC-015 completes them
     from the baggage live now, while the batch is still ours.
 
+    ``closed`` is set **before** the flush, not after (SPEC-036 FR-004). ``_flush`` can raise —
+    building the worker, or the queue path — and :func:`_end`'s guard absorbs it, so setting the
+    flag afterwards would leave it ``False`` on exactly the runs where a later append most needs
+    the orphan route. ``sinks/base.py`` settles the general form of this: set it before releasing
+    anything.
+
     Args:
       span: The span being closed.
       status: The span outcome, ``"ok"`` or ``"error"``.
@@ -961,6 +1209,7 @@ def _close_span(span: Span, status: str, exc: BaseException | None) -> None:
     """
     span.events.append(end_event(span, status, exc))
     backfill_baggage(span, context._live_baggage())
+    span.closed = True
     _flush(span)
 
 

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/model.py

@@ -30,6 +30,24 @@ class Span:
     ``start_ts`` is a ``time.monotonic()`` reading rather than wall-clock, so
     ``duration_ms`` can never go negative across a clock change. ``defaults`` are per-
     decorator field overrides, and ``events`` is the queue flushed together at span end.
+
+    ``closed`` marks a span whose events have already been handed to the worker (SPEC-036
+    FR-004). ``contextvars`` copies the *same* ``Span`` object into every task created inside a
+    span, so a fire-and-forget ``create_task`` can outlive its parent and append to a buffer
+    nothing will emit again. It is read by ``api._log`` at append time, which is the only place
+    that can notice: nothing in the library looks at a span after ``_close_span`` returns.
+
+    ``swept`` marks a span whose buffered events an in-span ``flush()`` has already handed to the
+    worker (SPEC-036 FR-001). Nothing in the delivery path needs it — the sweep is correct without
+    it — but ``continue_trace`` does: ``_reparent_current_span`` adopts a context by rewriting the
+    events still *buffered* on the open root span, and swept events have left that buffer, so an
+    adoption after a sweep would leave one span carrying two trace ids. That is the SPEC-024
+    category, wrong data rather than lost data, so the flag lets the adoption refuse instead.
+
+    The guarantee is **single-threaded**. The flag is not a synchronization primitive:
+    ``continue_trace`` reads it and then re-parents across an ordinary function call, so a sweep
+    arriving in that gap still splits the span. It is set before the detach so a concurrent reader
+    errs toward refusing, and the residual is recorded in ``architecture.md`` §13.
     """
 
     trace_id: str
@@ -39,6 +57,8 @@ class Span:
     start_ts: float
     defaults: dict[str, object] = field(default_factory=dict)
     events: list[dict[str, object]] = field(default_factory=list)
+    closed: bool = False
+    swept: bool = False
 
 
 def _iso_now() -> str:

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/sinks/multi.py

@@ -52,12 +52,21 @@ class MultiSink:
         """
         self._sinks = sinks
         self.failed = 0
+        self._silent_failed = 0
         self._counter_lock = threading.Lock()
         self._stop_signal: threading.Event | None = None
 
     def emit(self, batch: list[dict[str, object]]) -> None:
         """Forwards the batch to every child in construction order, isolating failures (FR-002).
 
+        A failing child that reports **nothing** of its own is counted here, in events, because
+        otherwise it is invisible: ``losses()`` sums only children with a ``losses()``, so a
+        destination that has delivered nothing since the process started reported zero loss
+        forever (SPEC-036 FR-005). Which children are silent is decided by ``read_losses`` per
+        call, the same probe the aggregate uses, so a child that gains a ``losses()`` later moves
+        categories on its own — at the cost that the aggregate can then *fall*, which is why the
+        counter is per child rather than a single total.
+
         Partial success stays isolated: a retry there would re-deliver the batch to the children
         that already took it, and duplicates are worse than the one failure already counted on
         ``failed`` and written to stderr. Total failure delivered nothing, so it has no
@@ -84,8 +93,11 @@ class MultiSink:
             try:
                 sink.emit(batch)
             except Exception as err:
+                silent = read_losses(sink) is None
                 with self._counter_lock:
                     self.failed += 1
+                    if silent:
+                        self._silent_failed += len(batch)
                 if first_error is None:
                     first_error = err
                 _diag.absorbed(
@@ -147,18 +159,32 @@ class MultiSink:
         """Sums the children's losses so a fan-out reports the whole tree (SPEC-026 FR-002).
 
         Nesting is handled for free, since a child ``MultiSink`` is just another sink with a
-        ``losses()``. ``MultiSink.failed`` is deliberately absent from the total: it counts
+        ``losses()``. ~~``MultiSink.failed`` is deliberately absent from the total: it counts
         child calls that raised, not events, so adding it to a per-event figure would produce a
-        number with no unit, and the children already report their own loss in events.
+        number with no unit, and the children already report their own loss in events.~~ —
+        superseded in part by SPEC-036 FR-005. ``failed`` is still absent, and still for that
+        reason. What was wrong was concluding that the fan-out therefore reports nothing: a child
+        with no ``losses()`` contributed nothing to the total, so a permanently dead destination
+        was invisible with ``health()`` reading all zeros. ``_silent_failed`` is the same loss in
+        the **right unit** — events from failing children that report nothing — so a reporting
+        child is still left to report itself and nothing is counted twice.
+
+        The figure is a **total**, not a breakdown: it cannot say which child is failing. The
+        per-batch stderr line names the class (:meth:`emit`), which is where that lives.
+
+        ``close()`` deliberately does not move ``_silent_failed``. Its failure path has no batch,
+        so there is no event count to add, and a ``+1`` there would mix units in exactly the way
+        that keeps ``failed`` out of this sum. A client buffer lost to a failed close is of
+        unknown size, and an invented number is worse than an absent one.
 
         Args:
           None.
 
         Returns:
-          The summed losses, or ``None`` when no child reported anything, including an empty
-          fan-out. FR-003 separates "reports nothing" from "reports no loss", and a tree of
-          silent children has not given a clean bill of health; one reporting child is enough to
-          make the total meaningful, since the silent ones contribute zero.
+          The summed losses, or ``None`` when no child reported anything **and** no silent child
+          has lost a batch. FR-003 separates "reports nothing" from "reports no loss", and a tree
+          of silent children that has lost nothing has still not given a clean bill of health —
+          but one that *has* lost something now says so, which is the whole of FR-005.
 
         Raises:
           None. A child without ``losses()`` contributes zero, and a child whose accessor raises
@@ -167,11 +193,13 @@ class MultiSink:
         """
         children = [read_losses(sink) for sink in self._sinks]
         reported = [child for child in children if child is not None]
-        if not reported:
+        with self._counter_lock:
+            silent_failed = self._silent_failed
+        if not reported and not silent_failed:
             return None
         return SinkLosses(
             dropped=sum(child.dropped for child in reported),
-            failed=sum(child.failed for child in reported),
+            failed=sum(child.failed for child in reported) + silent_failed,
         )
 
     def close(self) -> None:

{log_foundry-0.10.2.dev67 → log_foundry-0.10.2.dev69}/src/log_foundry/worker.py

@@ -144,6 +144,23 @@ class Health:
         ``shutdown()``, and it is the signal that a deployment shares a sink across a fork at
         all. ``True`` for a shared ``StdoutSink`` too, whose ``close()`` only flushes — so a
         ``True`` is not by itself evidence that anything is held.
+      orphan_lost: Events lost on the **synchronous** path — a level call made with no active
+        span, which emits on the caller's own thread with no worker between it and the sink
+        (SPEC-036 FR-003). Until this field existed that loss was counted nowhere: ``health()``
+        describes a worker, and this path has none, so a process logging only this way read all
+        zeros over total loss. Not ``failed_batches``, which means batches a worker abandoned
+        after spending a retry budget and kept running past; there is no batch, no retry and no
+        worker here. Not ``SinkLosses`` either — the sink did not absorb anything, it raised,
+        which is what SPEC-026 requires of it. It covers everything inside the orphan guard, a
+        sink that failed to *construct* included, so it climbing means **the destination or the
+        data**.
+      in_span_lost: Events lost while being built *inside* a span (SPEC-037 AC-5c, deferred to
+        SPEC-036 FR-003 so the pair was designed together). The in-span path cannot lose an
+        event at ``emit`` — that is ``failed_batches`` — so this climbing means **the data**,
+        always: a value that could not be built into an event. Two fields rather than one
+        because they aggregate different failure populations and so fail SPEC-026's test,
+        *would one number hide which fix applies*. Their sum is deliberately not reported: with
+        different populations it is a number nobody can act on.
     """
 
     queued: int
@@ -156,6 +173,8 @@ class Health:
     incomplete_swaps: int = 0
     closing_sinks: int = 0
     inherited_sink: bool = False
+    orphan_lost: int = 0
+    in_span_lost: int = 0
 
 
 class _FlushMarker: