@zerwiz/ymir 0.1.4 → 0.1.7

package/.agents/backend/fm-trace-context-lib.sh

@@ -0,0 +1,227 @@
+# shellcheck shell=bash
+# Native W3C trace-context propagation for firstmate spawns (default-off).
+#
+# When enabled, firstmate resolves one W3C `traceparent` carrier for a task,
+# injects it into the agent's pane shell as the TRACEPARENT environment variable
+# before launch (bin/fm-spawn.sh, alongside GOTMPDIR, so it reaches every spawn
+# backend and every harness for ship, scout, and secondmate spawns), and records
+# the identical value as `traceparent=` in state/<id>.meta. Because the injected
+# carrier and the recorded carrier are the same string, an observer that reads
+# the metadata sees exactly the identity the child received - no collector,
+# storage, UI, or vendor coupling.
+#
+# TRACEPARENT here is a firstmate CONVENTION that carries a W3C-formatted
+# traceparent value in the process environment. W3C Trace Context standardizes
+# the `traceparent` HTTP header, not an environment variable, and OpenTelemetry
+# SDKs do NOT read TRACEPARENT from the environment automatically. A downstream
+# observer or instrumentation must explicitly read this env value (or the meta
+# field); this library parents no SDK span by itself.
+#
+# Identity is per TASK, and each task is its own trace boundary: the carrier is
+# minted as a fresh random root on the task's first spawn and REUSED verbatim
+# from the meta on relaunch/recovery, so a task keeps one stable logical
+# identity across restarts. The spawning process's own ambient TRACEPARENT is
+# the agent identity it received at ITS launch, never a parent for new tasks: a
+# persistent supervisor routes many unrelated tasks from one long-lived
+# environment, and adopting its carrier would merge every routed task into one
+# ever-growing trace instead of one trace per task.
+#
+# Usage: . bin/fm-trace-context-lib.sh
+#
+# Public entry points:
+#   fm_trace_context_session_start <config-dir> <effective-state-file>
+#     Resolves config/trace-context plus FM_TRACE_CONTEXT once and atomically
+#     writes the normalized on/off decision bound to the locked home session.
+#   fm_trace_context_session_effective <effective-state-file>
+#     Echoes the normalized frozen decision only when its session binding matches
+#     the current lock, defaulting to off when the state is absent, stale, or invalid.
+#   fm_trace_context_resolve <config-dir> <meta-file>
+#     Echoes the traceparent to inject AND record, or nothing when the
+#     capability is off or when entropy or self-validation fails. It ALWAYS
+#     returns 0: telemetry is omitted safely and never aborts the spawn. The
+#     task's recorded carrier wins so recovery keeps identity; otherwise a
+#     fresh root is minted, never derived from this process's environment.
+#
+# Enablement (see docs/configuration.md for the schema):
+#   config/trace-context   presence flag under the home's config dir enables it.
+#   FM_TRACE_CONTEXT        env override: 1/on/true/yes enables, any other
+#                           non-empty value disables, and unset OR empty defers
+#                           to the file.
+#   Each locked home session resolves these inputs once into
+#   state/.trace-context-effective. The record is atomically published through a
+#   same-directory temporary file and bound to state/.lock; a failed publication
+#   cannot reactivate a stale on decision. Every spawn reads only that frozen
+#   on/off value, so later config and environment edits take effect only after a
+#   new home session starts.
+#   At launch, the primary propagates config/trace-context into the secondmate
+#   home (FM_INHERITABLE_CONFIG in bin/fm-config-inherit-lib.sh) and passes its
+#   frozen on/off decision into the new process as a non-empty FM_TRACE_CONTEXT
+#   value in the launch prefix (bin/fm-spawn.sh). The Secondmate freezes that
+#   inherited decision when its own home session starts.
+#   A REMOTE secondmate route resolves here too, in the PARENT process that owns
+#   that task's meta: fm-spawn's spawn_remote_secondmate resolves the carrier,
+#   hands it to the configured host through fm-spawn's --traceparent, and records
+#   the carrier the remote endpoint reports back. Only the pane export moves
+#   hosts; identity, enablement, and the per-task boundary do not.
+#
+# Wire shape: version 00 only, "00-<32 hex trace>-<16 hex span>-<2 hex flags>",
+# with the trace id and span id never all-zero (W3C rejects both). New roots use
+# RANDOM ids from /dev/urandom. The root's `01` (sampled) flag records a
+# sampling DECISION that downstream parent-based samplers honor; it does not
+# guarantee any collector stores a span, and firstmate emits no spans itself.
+#
+# Security / trust boundary. This feature adds no OTEL_* variables, no
+# tracestate, no arbitrary environment injection, and no configurable or
+# arbitrary command execution. It DOES run the fixed local utilities `od` and
+# `tr` (resolved from PATH) to read a few bytes of entropy - a small local
+# pipeline with no configured provider, network, or watchdog, and no hard latency
+# guarantee; any resolver failure that returns omits the carrier without aborting
+# the spawn. Carrier-delivery failure also omits telemetry and continues when the
+# backend clears its input; if the backend reports that partial input could not be
+# cleared, fm-spawn refuses to append the launch command. Every carrier this lib
+# yields is either a firstmate-MINTED random root that reads no prompt, path,
+# task prose, credential, or arbitrary environment key, or the same task's
+# previously recorded carrier reused verbatim from its own meta. Ambient
+# TRACEPARENT is never read, so no caller-controlled bytes enter a new carrier.
+#
+# Root / recovery semantics (the trace boundary is each task):
+#   recovery - a valid traceparent already recorded in the meta file is reused
+#              verbatim, so a relaunched or recovered task keeps one stable
+#              identity across restarts.
+#   root     - any other spawn mints a fresh random trace id, fresh span id, and
+#              sampled flags (01), beginning a new trace: one per task. The
+#              spawning process's ambient TRACEPARENT is never adopted, so a
+#              persistent supervisor's environment cannot chain its unrelated
+#              routed tasks into one trace.
+
+# Strict W3C traceparent validator: version 00, 32-hex trace id, 16-hex span id,
+# 2-hex flags, with neither id all-zero. The regex lives in a variable because
+# bash 3.2 only honors an unquoted right-hand side for =~.
+fm_trace_context_valid() {  # <traceparent>
+  local tp=$1
+  local re='^00-[0-9a-f]{32}-[0-9a-f]{16}-[0-9a-f]{2}$'
+  [[ $tp =~ $re ]] || return 1
+  [ "${tp:3:32}" = "00000000000000000000000000000000" ] && return 1
+  [ "${tp:36:16}" = "0000000000000000" ] && return 1
+  return 0
+}
+
+# Echo <byte-count> random bytes as lowercase hex, or echo nothing and return 1
+# on any entropy failure (unreadable source, short read, non-hex). -v stops od
+# from collapsing repeated byte lines to '*'; the explicit length and charset
+# checks turn a masked pipeline failure into a clean omission upstream.
+fm_trace_context_hex() {  # <byte-count>
+  local bytes=$1 hex
+  hex=$(LC_ALL=C od -An -v -tx1 -N "$bytes" /dev/urandom 2>/dev/null | tr -d ' \n') || return 1
+  case "$hex" in
+    '' | *[!0-9a-f]*) return 1 ;;
+  esac
+  [ "${#hex}" -eq "$((bytes * 2))" ] || return 1
+  printf '%s' "$hex"
+}
+
+# True when the capability is enabled for this home. The env override wins so a
+# spawn can be forced on or off without touching the file; otherwise the
+# presence of config/trace-context decides, and its absence is the default-off.
+fm_trace_context_enabled() {  # <config-dir>
+  local config_dir=$1 v
+  # A non-empty value is an explicit override; unset OR empty defers to the file
+  # (the conventional "empty is like unset" behavior).
+  if [ -n "${FM_TRACE_CONTEXT:-}" ]; then
+    v=$(printf '%s' "$FM_TRACE_CONTEXT" | tr '[:upper:]' '[:lower:]')
+    case "$v" in
+      1 | on | true | yes) return 0 ;;
+      *) return 1 ;;
+    esac
+  fi
+  [ -f "$config_dir/trace-context" ]
+}
+
+# Echo the lock pid that owns the effective-state file's home, or fail when the
+# adjacent session lock is absent or malformed. Binding the decision to this
+# token makes a prior session's record inactive even if publication cannot
+# replace or remove that stale file.
+fm_trace_context_session_lock() {  # <effective-state-file>
+  local effective_file=$1 state_dir lock_pid
+  state_dir=${effective_file%/*}
+  [ "$state_dir" = "$effective_file" ] && state_dir=.
+  # Grouped so the stderr redirect is in place BEFORE the input redirect is
+  # attempted: an absent lock is an ordinary silent "not locked" answer, and a
+  # trailing 2>/dev/null on the bare read would still leak the open failure.
+  { IFS= read -r lock_pid < "$state_dir/.lock"; } 2>/dev/null || return 1
+  case "$lock_pid" in
+    '' | *[!0-9]*) return 1 ;;
+  esac
+  [ "$lock_pid" -gt 1 ] || return 1
+  printf '%s' "$lock_pid"
+}
+
+fm_trace_context_session_start() {  # <config-dir> <effective-state-file>
+  local config_dir=$1 effective_file=$2 value=off lock_pid tmp
+  lock_pid=$(fm_trace_context_session_lock "$effective_file") || {
+    rm -f "$effective_file" 2>/dev/null || true
+    return 0
+  }
+  fm_trace_context_enabled "$config_dir" && value=on
+  tmp=$(mktemp "$effective_file.tmp.XXXXXX" 2>/dev/null) || {
+    rm -f "$effective_file" 2>/dev/null || true
+    return 0
+  }
+  if ! printf '%s %s\n' "$lock_pid" "$value" > "$tmp" 2>/dev/null \
+    || ! mv -f "$tmp" "$effective_file" 2>/dev/null; then
+    rm -f "$tmp" 2>/dev/null || true
+    rm -f "$effective_file" 2>/dev/null || true
+  fi
+  return 0
+}
+
+fm_trace_context_session_effective() {  # <effective-state-file>
+  local effective_file=$1 current_lock recorded_lock='' value='' extra=''
+  current_lock=$(fm_trace_context_session_lock "$effective_file") || {
+    printf '%s' off
+    return 0
+  }
+  if [ -f "$effective_file" ] && [ ! -L "$effective_file" ]; then
+    IFS=' ' read -r recorded_lock value extra < "$effective_file" 2>/dev/null || true
+  fi
+  if [ "$recorded_lock" = "$current_lock" ] && [ "$value" = on ] && [ -z "$extra" ]; then
+    printf '%s' on
+  else
+    printf '%s' off
+  fi
+}
+
+# Echo any traceparent already recorded in <meta-file>, else nothing. Used for
+# the recovery path so a relaunch reuses the first spawn's identity.
+fm_trace_context_recorded() {  # <meta-file>
+  local meta=$1 line
+  [ -f "$meta" ] || return 0
+  line=$(grep '^traceparent=' "$meta" 2>/dev/null | head -n1) || return 0
+  printf '%s' "${line#traceparent=}"
+}
+
+# Mint a fresh sampled root traceparent. Echo nothing and return 1 on entropy
+# or validation failure so the caller can omit telemetry.
+fm_trace_context_mint() {
+  local trace span tp
+  trace=$(fm_trace_context_hex 16) || return 1
+  span=$(fm_trace_context_hex 8) || return 1
+  tp="00-$trace-$span-01"
+  fm_trace_context_valid "$tp" || return 1
+  printf '%s' "$tp"
+}
+
+# Public entry point. Echo the single carrier to inject and record, or nothing.
+# Always returns 0 so a spawn is never aborted by a telemetry decision. The
+# recorded value wins so recovery keeps one task identity; otherwise a fresh
+# root is minted, never derived from this process's environment.
+fm_trace_context_resolve() {  # <config-dir> <meta-file>
+  local config_dir=$1 meta=$2 existing
+  fm_trace_context_enabled "$config_dir" || return 0
+  existing=$(fm_trace_context_recorded "$meta")
+  if fm_trace_context_valid "$existing"; then
+    printf '%s' "$existing"
+    return 0
+  fi
+  fm_trace_context_mint || return 0
+}

package/.agents/backend/fm-transition-lib.sh

@@ -0,0 +1,103 @@
+#!/usr/bin/env bash
+# Shared, backend-neutral agent-state transition shape and supervision policy.
+#
+# This library owns TWO contracts, deliberately backend-independent so any
+# push-capable session backend (herdr today, others later) reuses them instead
+# of re-deriving a private, per-status escalation hack:
+#
+#   1. The NORMALIZED TRANSITION RECORD - the ONE shape every backend's event
+#      stream is normalized into before any policy runs. A single TAB-separated
+#      line:
+#          <pane_id>\t<workspace_id>\t<from_status>\t<to_status>\t<agent>
+#      Only `to_status` is authoritative for the policy below; the other fields
+#      are identity/telemetry and MAY be empty when a backend cannot supply
+#      them. `from_status` in particular is empty for backends whose event
+#      carries only the new status (herdr's `pane.agent_status_changed` does
+#      not report the previous status, and its stream is edge-triggered, so
+#      each `to_status` IS itself a fresh edge); it exists in the shape for
+#      backends that DO report the prior state and for future edge diagnostics.
+#      Statuses use the shared agent-state vocabulary
+#      (idle|working|blocked|done|unknown), the same enum herdr's `agent get`
+#      and `pane.agent_status_changed` report.
+#
+#   2. The STATUS -> ACTION POLICY TABLE (fm_transition_policy) - the SINGLE
+#      OWNER of the mapping from a normalized `to_status` to the supervision
+#      action a consumer must take. Every consumer READS this table; no
+#      consumer re-encodes the mapping. Adding or changing a status's action is
+#      a one-line edit here, and it changes every backend at once.
+#
+# The split is what keeps the escalation general rather than a herdr blocked
+# hack: a backend contributes only a wire->record normalizer and a stream
+# reader; the shape and the policy are shared. See bin/backends/herdr.sh
+# (fm_backend_herdr_wait_transition) for the herdr producer and bin/fm-watch.sh
+# (the watcher's event-wait splice) for the consumer.
+
+# Field separator for the normalized record. A literal TAB; every field is
+# scrubbed of TAB/newline by the producer so the record is exactly five fields.
+FM_TRANSITION_FIELD_SEP=$'\t'
+
+# fm_transition_record: THE constructor for a normalized transition record.
+# Both a backend's stream normalizer and its level-reconcile read MUST build
+# records through this one function, so the record's field order and separator
+# have a single owner. Fields are TAB/newline-scrubbed here.
+fm_transition_record() {  # <pane_id> <workspace_id> <from_status> <to_status> <agent>
+  local pane_id ws from to agent
+  pane_id=$(fm_transition_clean_field "${1:-}")
+  ws=$(fm_transition_clean_field "${2:-}")
+  from=$(fm_transition_clean_field "${3:-}")
+  to=$(fm_transition_clean_field "${4:-}")
+  agent=$(fm_transition_clean_field "${5:-}")
+  printf '%s\t%s\t%s\t%s\t%s' "$pane_id" "$ws" "$from" "$to" "$agent"
+}
+
+# fm_transition_clean_field: collapse any TAB/CR/LF in a field value to spaces
+# so a stray control char can never desync the fixed five-field record.
+fm_transition_clean_field() {  # <value>
+  printf '%s' "${1:-}" | LC_ALL=C tr '\t\r\n' '   '
+}
+
+# Field accessors (1-based), so consumers never hardcode the column layout.
+fm_transition_field() {  # <record> <n>
+  printf '%s' "$1" | cut -d"$FM_TRANSITION_FIELD_SEP" -f"$2"
+}
+
+fm_transition_pane_id()      { fm_transition_field "$1" 1; }
+fm_transition_workspace_id() { fm_transition_field "$1" 2; }
+fm_transition_from_status()  { fm_transition_field "$1" 3; }
+fm_transition_to_status()    { fm_transition_field "$1" 4; }
+fm_transition_agent()        { fm_transition_field "$1" 5; }
+
+# fm_transition_policy: THE single-owner status -> supervision-action table.
+# Given a normalized `to_status`, print exactly one action token:
+#
+#   actionable - escalate to the supervisor IMMEDIATELY (a fresh edge here is a
+#                durable wake now). `blocked` is the only immediately-actionable
+#                status today: herdr reports it precisely when a harness is
+#                waiting on the human (a permission/trust dialog, an interactive
+#                menu, a wedged prompt) - the cases that write no status file
+#                and otherwise sit until the stale-pane wedge timer.
+#   absorb     - do NOT wake, but CLEAR this pane's per-pane escalation dedupe
+#                marker so a later `->blocked` edge re-escalates. `working`
+#                (a crew resumed/started a turn) is the clearing edge.
+#   defer      - do NOTHING on the fast path; leave it to the existing
+#                status/turn-end completion semantics and the poll backstop.
+#                `idle`/`done` blip transiently between tool calls, so
+#                fast-pathing them would be a false-positive firehose - they are
+#                already covered by the debounced signal/stale machinery.
+#   fallback   - the status is unknown/unrecognized: fall back to polling for
+#                this pane (the permanent fail-closed backstop), taking no fast
+#                action from an ambiguous read.
+#
+# Consumers act on `actionable`, mutate dedupe state on `absorb`, and ignore
+# `defer`/`fallback` on the fast path. Subscribing to ALL statuses (not just
+# `blocked`) is deliberate: `working`/`idle`/`done` carry the dedupe-clear and
+# reconnect/level-reconcile state; only THIS policy makes `blocked` the sole
+# immediate action.
+fm_transition_policy() {  # <to_status> -> actionable|absorb|defer|fallback
+  case "$1" in
+    blocked) printf 'actionable' ;;
+    working) printf 'absorb' ;;
+    idle|done) printf 'defer' ;;
+    *) printf 'fallback' ;;
+  esac
+}