by-framework 0.2.2.dev13__tar.gz → 0.2.2.dev15__tar.gz

by_framework-0.2.2.dev15/CHANGELOG.md

@@ -0,0 +1,30 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Fixed
+- A resumed agent can read its own request metadata again. Previously,
+  `command.header.metadata` after a suspend (`ask_user`, or a `call_agent`
+  hop) held only the metadata of the message that woke the agent up, and
+  everything it had originally been dispatched with was gone. It is now the
+  original dispatch metadata with the waking message's merged on top (the
+  waking message wins on key collisions). Handlers that treated the resumed
+  header's metadata as "only what this hop just sent" will now see additional
+  keys. Per-hop trace fields (`trace_parent_span_id`,
+  `framework_parent_span_id`, `langfuse_parent_observation_id`) still
+  describe the current hop and are not restored from the record. Executions
+  recorded before this change are unaffected — they degrade to the previous
+  behaviour rather than erroring.
+
+### Added
+- Standardized Open Source governance files (CONTRIBUTING, CODE_OF_CONDUCT, SECURITY).
+- GitHub Issue and Pull Request templates.
+- Dependabot configuration for dependency monitoring.
+
+## [0.2.0] - 2026-05-13
+### Initial release

{by_framework-0.2.2.dev13 → by_framework-0.2.2.dev15}/CLAUDE.md

@@ -151,6 +151,22 @@ Pre-commit hooks are configured in `.pre-commit-config.yaml` and run isort, ruff
   plugin/trace integration point must degrade to a no-op (logged) rather
   than raising.
 
+- **A rule enforced only behind an optional dependency is not enforced.** If
+  the check short-circuits when a package is missing, then whatever controls
+  the environment controls the rule — and a green run stops meaning anything.
+  Incident: `tests/metrics/test_catalog.py` asserts every exported metric has a
+  `metrics/catalog.py` entry, but returned early when `prometheus-client` was
+  absent, while `make test` synced with `--extra dev` and dropped the root
+  `observability` extra. The assertion ran against an empty registry and
+  reported success for months; four uncatalogued metrics landed on main with CI
+  green (#142, then the CI fix). Correct form: enforce the rule where the
+  optional dependency cannot reach it — here `scripts/check-metric-catalog.sh`
+  reads the source, so **every new Prometheus `Counter`/`Gauge`/`Histogram` in
+  `src/` needs a `catalog.py` entry** regardless of what is installed — and
+  make the dependent test `pytest.skip()` rather than `return`, so degrading is
+  at least visible. Test commands must sync the extras the tests exercise
+  (`make test` uses `--all-extras`); narrowing them silently deletes coverage.
+
 ## Iron rules
 
 <!-- No process rules (test invocation, release steps) have caused an
@@ -166,6 +182,7 @@ Pre-commit hooks are configured in `.pre-commit-config.yaml` and run isort, ruff
 | Worker deployment/production-readiness — README's 部署 section, `__main__.py` CLI flags, `run_worker()`'s signature, or shutdown/signal handling | `docs/architecture/production-deployment.md` |
 | Worker readiness/health-check endpoint (`WorkerHealthServer`, `/readyz`, `--health-port`) — building it, or touching anything that changes what "ready" means | `docs/architecture/worker-readiness-endpoint.md` |
 | suspend/resume liveness — the wait index, its idempotency gate, or the sweep that resolves callers whose reply never arrives | `docs/architecture/suspend-resume-liveness.md` |
+| Langfuse tracing state held in worker memory — `_active_workflows`, trace-output uploads, or anything that queues/retains per-task telemetry | `docs/architecture/langfuse-tracing-memory.md` |
 
 ## Maintaining this map
 

{by_framework-0.2.2.dev13 → by_framework-0.2.2.dev15}/Makefile

@@ -37,11 +37,22 @@ list-projects:
 		echo $$project; \
 	done
 
+# All ten projects share the workspace venv, so syncing them in a loop makes
+# each one PRUNE the previous one's dependencies — the loop used to end with a
+# venv holding only the last project's deps (and it uninstalled prometheus-client
+# on the second iteration). --all-packages resolves the whole workspace once
+# instead. The loop survives only for an explicit PROJECTS override, where
+# narrowing the venv is what the caller asked for.
 install:
-	@for project in $(PROJECTS); do \
-		echo "==> Syncing $$project"; \
-		(cd $$project && uv sync --all-extras); \
-	done
+	@if [ "$(strip $(PROJECTS))" = "$(strip $(PYTHON_PROJECTS))" ]; then \
+		echo "==> Syncing workspace (all packages, all extras)"; \
+		uv sync --all-extras --all-packages; \
+	else \
+		for project in $(PROJECTS); do \
+			echo "==> Syncing $$project (narrowed: PROJECTS override)"; \
+			(cd $$project && uv sync --all-extras); \
+		done; \
+	fi
 
 format:
 	@PROJECTS="$(PROJECTS)" ./scripts/python_quality.sh format $(FILES)
@@ -80,6 +91,18 @@ lint-changed:
 		$(MAKE) lint PROJECTS="$(PROJECTS)" FILES="$$changed_files"; \
 	fi
 
+# --all-extras, not --extra dev: `uv run` only ever adds what it is asked for,
+# it never restores an extra it was not told about. So once `install`'s old
+# per-project loop had uninstalled prometheus-client, `--extra dev` left it
+# uninstalled and the root project's whole `observability` extra stayed missing
+# for the entire test run. tests/metrics then ran against a build with no
+# Prometheus registry, where the metric catalog check short-circuits and reports
+# success without inspecting anything. State what the tests need, rather than
+# inheriting whatever `install` happened to leave behind.
+#
+# Deliberately per-project rather than --all-packages: each project must be
+# testable with only its own declared dependencies, or a package that quietly
+# leans on a sibling's deps would still pass here and break on install.
 test:
 	@set -e; for project in $(PROJECTS); do \
 		if ! find "$$project/tests" -type f \( -name 'test_*.py' -o -name '*_test.py' \) -print -quit 2>/dev/null | grep -q .; then \
@@ -87,7 +110,7 @@ test:
 			continue; \
 		fi; \
 		echo "==> Testing $$project"; \
-		(cd $$project && uv run --extra dev pytest); \
+		(cd $$project && uv run --all-extras pytest); \
 	done
 
 clean:

{by_framework-0.2.2.dev13 → by_framework-0.2.2.dev15}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: by-framework
-Version: 0.2.2.dev13
+Version: 0.2.2.dev15
 Summary: 分布式 Agent 调度框架
 License-File: LICENSE
 Requires-Python: >=3.12

{by_framework-0.2.2.dev13 → by_framework-0.2.2.dev15}/docs/architecture/KEY_FILES.md

@@ -260,7 +260,13 @@ Entry anatomy:
   implementations fall back to minting a fresh execution_id. Root-dispatch
   trace writes (`_write_trace_root_start/_end`) must only fire when
   `not parent_message_id` — firing them on every `call_agent` hop would
-  duplicate trace roots.
+  duplicate trace roots. `initialize_execution()`'s payload carries the
+  dispatch `metadata`: a root agent suspends (an `ask_user` round is the
+  common case) and comes back on a `ResumeCommand` carrying the *callee's*
+  metadata, so the record is the only place its own request metadata still
+  exists for `worker.py`'s inbound restore to find. `runner.py` records the
+  same field independently on first pickup, which is what makes this work
+  for dispatchers that don't write it (TS/Java callers, a bare `xadd`).
 
 - `src/by_framework/worker/runner.py` — `WorkerRunner`, the consume loop:
   `XREADGROUP` fetch, command dispatch, resume/suspend bookkeeping, denylist
@@ -290,7 +296,16 @@ Entry anatomy:
   suspended caller persists as `WAITING_AGENT`/`WAITING_USER` precisely to
   keep that inference true; reusing QUEUED for any post-pickup state silently
   makes a resume re-derive its identity from the message header instead of
-  the record. Every `ResumeCommand` passes the `core/wait_gate.py`
+  the record. On a **non-resume** pickup it also writes `header.metadata`
+  onto the record (as an `update_execution_status` kwarg, or in the
+  `save_execution` fallback payload): `header.metadata` *is* the dispatch
+  metadata by definition, so recording it here — rather than trusting
+  whoever sent the message to have done it — is what makes
+  `worker.py`'s inbound restore work for a client root dispatch and for a
+  TS/Java caller, neither of which writes the field. The `ResumeCommand`
+  guard is load-bearing: a resume writing its own waking metadata over the
+  record would destroy the stored original that the restore exists to
+  recover, and nothing else keeps a copy. Every `ResumeCommand` passes the `core/wait_gate.py`
   idempotency gate *here*, before the execution lookup — and being upstream
   of `GatewayWorker` is what also puts it before Task Group join, whose
   `HINCRBY completed` a duplicate would push past `total` and aggregate a
@@ -353,6 +368,22 @@ Entry anatomy:
   transient hop overwrite the caller's own data instead of being layered
   under `task_result.metadata` like `_enqueue_agent_return()`'s merge
   already does correctly.
+  That is the OUTBOUND direction. The INBOUND one — what this execution's own
+  handler reads — is `_restore_inbound_metadata()`, and it is deliberately a
+  different rule: the original dispatch metadata from the same snapshot is
+  the base and the waking message's metadata is **merged on top** (waking
+  message wins collisions), because this agent *is* the addressee of that
+  message. Without it, everything a handler was dispatched with disappears
+  the moment it first suspends. Two constraints keep the directions from
+  contaminating each other: the restore must build a NEW command
+  (`prepare_command_for_processing()` returns the *same* object for the base
+  worker, so an in-place header mutation would rewrite `raw_command`'s and
+  leak the merge into the reply), and `_resolve_reply_command()` must keep
+  taking `raw_command`. The shared merge rule, including which framework
+  trace keys are excluded from the restored base, lives in
+  `_resume_metadata.py` — one copy, because the last time these two files
+  each carried their own version of a resume rule the processor was left
+  behind for a whole commit.
   The success path must NOT reply while `context._is_suspended` — a suspended
   execution has no result, and forwarding the value the handler returned in
   order to unwind both wakes the caller early and consumes the single reply it
@@ -431,8 +462,22 @@ Entry anatomy:
   to no execution. `_resolve_reply_header()` must read a resumed execution's
   caller from the execution record, not from the resume header (which names
   the sub-agent), while excluding `CLIENT_SOURCE_AGENT_TYPE` for the reason
-  spelled out under `worker.py`; and no callback may be sent while
-  `context._is_suspended` (with the same terminal-status exception).
+  spelled out under `worker.py`; it restores **four** fields from that record
+  — `source_agent_type`, `parent_message_id`, `task_group_id` and
+  `metadata` — the last one as a full replacement, never merged with the
+  waking message's own metadata, exactly as `worker.py`'s
+  `_resolve_reply_command()` does (restoring only the three routing fields is
+  the drift this entry warns about: it shipped that way and leaked the waking
+  hop's metadata to the caller); `_restore_inbound_metadata()` mirrors
+  `worker.py`'s inbound merge for what the *handler* reads, and must run even
+  when `_resolve_reply_header()` returned `None` — a client-dispatched root
+  is owed no reply but still has its own metadata to get back, so the two
+  must not share a short-circuit. `_load_execution_snapshot()` is the single
+  registry read feeding both directions; being fail-soft there is what keeps
+  a registry error a degrade rather than a lost reply. The reply is built
+  from `raw_command`, never the inbound-restored one. And no callback may be
+  sent while `context._is_suspended` (with the same terminal-status
+  exception).
   Being a *second* entry point for replies, it carries the same
   `core/wait_gate.py` gate as `runner.py` — a gate on one of two doors is not
   a gate, and an ungated reply here would both wake a resolved caller and
@@ -441,6 +486,25 @@ Entry anatomy:
   `call_agents`' queued stand-ins on the same terms as `worker.py` — after
   the handler returns, never when it raised.
 
+- `src/by_framework/worker/_resume_metadata.py` — the one copy of the
+  **inbound** resume-metadata rule, shared by `worker.py` and `processor.py`
+  so the two cannot drift again. `merge_resume_metadata()` puts the
+  execution's original dispatch metadata underneath the waking message's
+  (waking message wins collisions) — the opposite of the outbound
+  replacement, because a resumed handler *is* the addressee of the message
+  that woke it. `FRAMEWORK_HOP_METADATA_KEYS` (`trace_parent_span_id`,
+  `framework_parent_span_id`, `langfuse_parent_observation_id`) are dropped
+  from the restored base: the framework injects them per dispatch, so a
+  stored copy describes the hop that *created* the execution, and
+  `context.py`'s `_resolve_call_langfuse_parent_id()` reads that key straight
+  off the current command — restoring it would parent a post-resume call to
+  an observation from before the suspend. The filter belongs here, at read
+  time, and not at write time: the outbound path deliberately restores the
+  stored dict verbatim, trace keys included, to stay identical to the
+  non-suspended reply. A missing/absent stored value degrades to the waking
+  message's metadata unchanged, which is the pre-restore behaviour, so old
+  records and cross-SDK callers need no migration or version gate.
+
 - `src/by_framework/core/registry.py` — `WorkerRegistry`: Redis-backed worker
   membership/heartbeat/execution-state, admin lifecycle, locking primitives.
   `mark_execution_finished()` must stamp `finished_at` only

by_framework-0.2.2.dev15/docs/architecture/langfuse-tracing-memory.md

@@ -0,0 +1,130 @@
+# Langfuse tracing: what the worker holds, and for how long
+
+## What this covers
+
+`libs/by-framework-trace-langfuse` is the only plugin that keeps **live
+per-task state in worker memory across hook calls**. Everything else in the
+trace layer is either fire-and-forget (`RedisSpanExporter`) or bounded by the
+SDK (`LangfuseSpanProcessor` wraps OTel's `BatchSpanProcessor`, whose queue
+tops out at 2048 spans and drops the overflow).
+
+This doc exists because three separate unbounded-growth bugs shipped in that
+plugin and none of them were visible from reading a single function. They
+shared one root cause, stated below, and the fixes are only correct as long
+as that framing holds.
+
+## The rule everything follows from
+
+**Langfuse reporting is best-effort telemetry, not business data.** A slow or
+unreachable Langfuse must cost trace fidelity — never worker memory, never
+worker shutdown, never a shared executor.
+
+Every degraded path therefore **sheds load**: drop the work, count it, log it
+sparsely. The failure mode to avoid is the intuitive one — "queue it, we'll
+send it when Langfuse recovers" — because a worker completing tasks faster
+than Langfuse accepts them turns that queue into an OOM with the whole
+serialized task output pinned in each entry.
+
+This is a stricter rule than the repo-wide fail-soft invariant in `CLAUDE.md`.
+Fail-soft says *don't raise into the task path*. This says *don't accumulate
+in the task path either* — a plugin that swallows every exception can still
+kill the worker by growing.
+
+## The three things the plugin holds
+
+### 1. `_active_workflows` — durable workflow spans
+
+An `agent.workflow` observation spans the **logical** task, which outlives any
+single worker execution: `ask_user` and `call_agent` suspend the execution and
+a later `RESUME` reattaches to it (see `suspend-resume-liveness.md`). So the
+handle cannot be closed when the first execution returns — it lives in an
+`OrderedDict` keyed by `(session_id, message_id)` until the task reaches a
+terminal status.
+
+Three things can end an entry, and all three must exist:
+
+| Exit | Trigger |
+|---|---|
+| Normal | terminal `on_task_complete` / `on_task_error` / `on_task_cancel` |
+| TTL | `_expire_active_workflows()`, for a suspend whose resume never came |
+| LRU | `_evict_active_workflows_if_needed()`, at `max_active_workflows` |
+
+**Invariant — LRU order equals expiry order.** `_expire_active_workflows()`
+scans from the head and stops at the first live entry, which is only valid
+because every write goes through `_track_active_workflow()`, which refreshes
+`expires_at` *and* moves the entry to the tail. A future write that bypasses
+that helper silently turns expiry into a no-op for everything behind it.
+
+**Ownership is claimed at registration, not at the end of the hook.**
+`on_task_start` registers the workflow and immediately sets
+`LANGFUSE_WORKFLOW_OBSERVATION_ATTR` on the context, because the tracer calls
+that follow it can raise — and `PluginRegistry._execute_hook()` swallows that.
+`_end_workflow_observation()` additionally falls back to the
+`(session_id, message_id)` key when the context attribute is missing. Both
+halves are load-bearing: without the fallback, a mid-hook SDK error orphaned
+100% of entries until LRU eviction, each one a live span that would never
+export.
+
+For the same reason `on_task_error` / `on_task_cancel` must **not** return
+early when the context has no agent/worker observations. The workflow entry
+can outlive them, and bailing is what stranded it.
+
+### 2. `_pending_trace_output_updates` — in-flight trace-output uploads
+
+`update_trace_output` is a *synchronous* HTTP POST — it patches the trace-list
+`output` field, which the OTel span pipeline does not carry. It runs on the
+plugin's **own** `ThreadPoolExecutor`, never the asyncio default executor:
+that pool is shared with the rest of the framework's blocking calls, and a
+stalled Langfuse would starve unrelated work.
+
+The bound is enforced **before submitting**, not by trimming the tracking set.
+Dropping only the future leaves the work item — and the payload it closes
+over — sitting in the executor queue, which is exactly the growth being
+guarded. At the limit the upload is discarded and counted.
+
+`on_worker_shutdown` waits on the remainder with a deadline and abandons
+whatever misses it. The discard callback catches `BaseException`, not
+`Exception`: shutdown cancels the stragglers, and `CancelledError` is a
+`BaseException`, so a narrower catch dumps one traceback per abandoned upload.
+
+### 3. Serialized output
+
+`_serialize_value` deep-copies the whole task result. The end-of-task path
+serializes **once**, at the top of `on_task_complete`; `_end_observation`,
+`_end_workflow_observation` and `_update_trace_output` all take an
+already-serialized value and must not re-serialize it. Their docstrings state
+this precondition — it is a contract, not an optimization, because copy #3 is
+what an in-flight upload pins.
+
+## Tunables
+
+All have safe defaults; unset behaves no worse than the built-in values.
+Explicit constructor kwargs always win over the environment, per `CLAUDE.md`'s
+"explicitly-passed kwarg" invariant.
+
+| Env | Default | What it bounds |
+|---|---|---|
+| `BYAI_LANGFUSE_MAX_ACTIVE_WORKFLOWS` | 10000 | live workflow spans |
+| `BYAI_LANGFUSE_WORKFLOW_TTL_SECONDS` | 3600 | how long a suspended workflow is held |
+| `BYAI_LANGFUSE_MAX_PENDING_TRACE_OUTPUTS` | 128 | in-flight trace-output uploads |
+| `BYAI_LANGFUSE_SHUTDOWN_FLUSH_TIMEOUT_SECONDS` | 5 | shutdown flush deadline |
+
+## Accepted trade-off
+
+A workflow reclaimed by TTL or LRU whose resume arrives later gets a *new*
+observation built from the same deterministic span id
+(`str_to_uint64(f"{execution_anchor}:agent.workflow")`), so Langfuse sees a
+duplicate span id. This is pre-existing LRU behaviour, not new; the 1-hour
+default TTL keeps it rare. Trace-output drops under load are likewise
+deliberate — the trace and its observations still ship via the SDK's own
+batch pipeline, so only the trace-list `output` preview is lost.
+
+## One more thing that is not a leak
+
+`build_langchain_callback` subclasses the Langfuse `CallbackHandler` to stop
+LangChain runs being promoted to trace roots. That subclass is cached per base
+class in a `WeakKeyDictionary`. Minting one per call was *not* a hard leak —
+the classes are reclaimable — but it ran once per task, and type objects are
+only collected by the generational GC while every creation invalidates
+CPython's type method cache. Per-callback state lives on the instance
+(`_by_framework_metadata`), so one shared subclass is safe.