@llblab/pi-state-flow 0.15.0 → 0.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -8,18 +8,19 @@ The extension owns durable memory while enabled. Global semantic memory is alway
8
8
 
9
9
  ## Composition
10
10
 
11
- `index.ts` is the public export and extension composition boundary. Independent modules under `lib/` own one concern each and are mirrored by tests:
11
+ `index.ts` is the minimal public export boundary. `lib/extension.ts` is the Pi lifecycle composition root: it wires configuration and domain capabilities into commands, tools, event subscriptions, and handlers while delegating imperative mechanics to their owning modules. Independent modules under `lib/` own one concern each and are mirrored by tests:
12
12
 
13
13
  - `state`, `json`: semantic shape, validation, recursive overlay and deletion.
14
14
  - `temporal`, `history`: causal boundaries, checkpoint/tail folding and hot history.
15
- - `durable`, `storage`, `git`: exact files, CAS publication, Git commits/restoration, and owned push processes.
16
- - `snapshot`, `session`, `runtime`, `recovery`, `episode`: Pi branch/runtime lifecycle.
15
+ - `durable`, `storage`, `git`: exact files, CAS publication, Git commits/restoration, and Git transport.
16
+ - `snapshot`, `session`, `runtime`, `recovery`, `episode`: Pi branch/runtime lifecycle, branch traversal, and passive-boundary interpretation.
17
17
  - `transition`, `terminal`, `context`: inference barriers, turn resolution, passive projection, and response reconciliation.
18
18
  - `artifact`, `acquisition`, `maintenance`, `skills`, `rehydration`: source routing and compilation.
19
19
  - `memory`: external promotion records and memory diagnostics.
20
20
  - `continuation`: native-header discovery, runtime-provenance inspection, deterministic recommendation, and host startup precedence.
21
- - `publication`: remote policy, durable CAS queue/store, cross-process leases, and attempt outcomes.
22
- - `status`, `telegram`, `extension`: operator projection, the optional fail-open pi-telegram presentation adapter, and Pi adapter wiring.
21
+ - `publication`: remote policy, durable CAS queue/store, cross-process leases, worker lifecycle, generation fencing, and attempt outcomes.
22
+ - `protocol`, `logging`: model/tool presentation and bounded diagnostic persistence.
23
+ - `status`, `telegram`, `extension`: operator projection, the optional fail-open pi-telegram presentation adapter, and high-level Pi adapter wiring.
23
24
 
24
25
  ## Semantic state
25
26
 
@@ -61,17 +62,17 @@ checkpoint.json + patches.jsonl
61
62
 
62
63
  The checkpoint is an older anchored materialization. The tail contains at most seven effective patches. On overflow, the oldest tail patch folds into the checkpoint before the new patch is appended.
63
64
 
64
- `state[n]`, `state.global[n]`, `state.cwd[n]` and `state.session[n]` resolve the same nth previous causal boundary. They are not independent per-scope patch counters. Pre-origin history is unavailable rather than empty.
65
+ `effective[n]`, `global[n]`, `cwd[n]`, and `session[n]` resolve the same nth previous causal boundary. They are not independent per-scope patch counters. The retired top-level `state` segment is rejected; pre-origin history is unavailable rather than empty.
65
66
 
66
67
  A final-only `patch_state({"final":true})` call changes only ephemeral terminal eligibility and creates no identity, commit, or history step. A changed accepted response is runtime-owned semantic state and advances history.
67
68
 
68
69
  ## Pi lifecycle
69
70
 
70
- `patch_state` is the sole mutation tool. It validates any supplied global/CWD/session patches against one causal basis and publishes them as one atomic transition, then acts as an inference barrier. Pi executes no sibling tools from the same assistant response; the next inference sees rematerialized `state[0]`.
71
+ `patch_state` is the sole mutation tool. It validates any supplied global/CWD/session patches against one causal basis and publishes them as one atomic transition, then acts as an inference barrier. Pi executes no sibling tools from the same assistant response; the next inference sees the rematerialized current effective state.
71
72
 
72
73
  Tool preflight follows Pi's public `getLeafEntry()` / `getEntry(parentId)` links to the nearest assistant containing the current call ID. It inspects that response's complete tool batch without constructing the whole branch or caching a batch across calls/selections. Foreign custom entries and earlier sibling results remain in the native trace. A missing call ID still searches the selected ancestry and preserves the existing unmatched-call behavior; this is not an unconditional constant-time guarantee. See [measured traversal evidence](performance.md#tool-preflight-parent-traversal).
73
74
 
74
- `read_state` reads one cached effective or scoped projection at offsets zero through seven. It never publishes or advances history.
75
+ `read_state` reads one cached effective or scoped projection at current index zero or a retained causal index one through seven. It never publishes or advances history.
75
76
 
76
77
  Every enabled assistant iteration starts terminal-ineligible. Only a successful `patch_state` call containing `final:true` latches eligibility for the next accepted `turn_end`; the call may atomically include global, CWD, and session patches. Eligibility does not stop later reasoning, tools, or patches. If terminal prose arrives before eligibility, State Flow preserves that draft and reconciles it into runtime-owned `response` at `turn_end`, then starts at most two same-run fallback turns whose only purpose is the `final:true` patch. The same path covers an eligible draft whose final validation fails after a later acquisition. Fallback turns never become the response: a successful `final:true` commits its patches and closes resolution with the preserved answer intact, while two failed fallbacks close the iteration with the preserved answer and current state plus one bounded warning and a finalization diagnostic. Failed patch calls do not consume the budget. A following legal patch remains possible, and only an accepted ordinary answer is reconciled into runtime-owned `response` at `turn_end`. State Flow no longer parses `state_flow` or generic HTML comments; historical comments are ordinary text and other extensions retain their own comment handling.
77
78
 
@@ -175,9 +176,11 @@ Compilation is routing, not a substitute for source text. Full source is read on
175
176
 
176
177
  Skills are CWD artifacts with stricter compilation: `kind: "skill"` and a non-empty compilation describing applicability, constraints and failure conditions. Their source bodies do not persist in state. Matching provenance proves source-version consistency, not semantic fidelity, truth, or higher instruction authority.
177
178
 
178
- ## Memory curation and promotion
179
+ ## Operational guidance, memory curation, and promotion
179
180
 
180
- The optional packaged `state-flow-memory` Skill performs bounded explicit audits, scope narrowing, contradiction cleanup and external handoffs. It is not part of ordinary retention or background maintenance. Curation compiles a read Skill at CWD before accumulating global compilation obligations, writes and separately reads a migration destination before source deletion, then verifies the changed scope and effective overlay. Simultaneously pending CWD/global acquisitions must be compiled together in one atomic `patch_state` call. Destination write, readback, and source deletion remain separate migration steps so accepted-copy verification is not skipped.
181
+ The packaged Skills deliberately separate two responsibilities. `state-flow-guide` is the on-demand operational reference for concrete read, patch, inheritance, acquisition, finalization, and recovery questions; it does not initiate memory audits or unsolicited cleanup. `state-flow-memory` performs one bounded explicit or phase-boundary curation over stale knowledge, commitments, continuation, ownership, and external handoffs; it is not part of routine turns or background maintenance.
182
+
183
+ Curation compiles a read Skill at CWD before dependent work, writes and separately reads a migration destination before source deletion, then verifies the changed owner and effective overlay. Simultaneously pending CWD/global acquisitions must be compiled together in one atomic `patch_state` call. Destination write, readback, and source deletion remain separate migration steps so accepted-copy verification is not skipped.
181
184
 
182
185
  External promotion remains a semantic two-phase handoff, not a memory-owner mode. Optional global `working.memory_promotions` entries record `pending`, `accepted`, `failed` or `unknown` status plus owner. Accepted records additionally require destination pointer and revision. Failed or uncertain promotion preserves the State Flow candidate; the only accepted copy is never deleted.
183
186
 
@@ -204,7 +207,7 @@ Both [tested Pi SDKs](compatibility.md) choose or create `SessionManager` before
204
207
  {"session":{"intents":{"next":"Verify the corrected behavior"}},"final":true}
205
208
  ```
206
209
 
207
- `intents` is the hot plane for active commitments, not requirements, observations, alternatives, or completed plans. Removing an intent does not remove its consequences or any referenced state. A conventional `{"$ref":"cwd.lazy.plan"}` value remains ordinary JSON and is followed explicitly with `read_state`; it creates no dependency, hydration, execution, or completion semantics.
210
+ `intents` is the hot plane for active commitments, not requirements, observations, alternatives, or completed plans. Removing an intent does not remove its consequences or any referenced state. Semantic-state references use either the optional structured `{"$ref":"cwd.lazy.plan"}` convention or `$` immediately followed by one valid `read_state` path inside ordinary text, for example `$effective.lazy.memory[7]`. The text prefix distinguishes references from incidental path-like prose and leaves a deterministic seam for possible future parsing. Resource paths, document locators, URIs, Skill identities, and agent identities retain their native syntax. State Flow stores all forms as ordinary JSON and currently does not parse or validate targets. The agent resolves a relevant locator explicitly through `read_state` or the appropriate external tool; presence alone creates no authority, existence proof, dependency, hydration, execution, or completion semantics. Reference repair is reactive: the agent never scans or resolves references merely to test them. Only after one requested value path is missing does the query domain perform one bounded reverse lookup over current model-patchable semantic planes for exact structured `$ref` or `$path` matches. When matches exist, `read_state` returns the explicit diagnostic sentinel `{value:null, hint:[{type:"dangling-reference", message, paths}]}`. `hint` is a top-level sibling rather than state data; its message asks for reconciliation and `paths` contains at most three runtime-verified current owning addresses. The null sentinel is never returned alone for this case. Keys, patch, and multi-path reads keep all-or-error semantics, while no durable match retains the ordinary missing-path error. A match establishes durable semantic provenance, not staleness; no match does not prove invention. The agent may then inspect ownership and patch a proven stale source without discarding surrounding meaning. Effective absence does not establish ownership, and unavailable history, external inaccessibility, or transient read failure does not prove a broken reference.
208
211
 
209
212
  `read_state` accepts one unified path. `effective == effective[0]` is the current effective materialization; `global == global[0]` (and CWD/session equivalents) selects that scope at the same composed causal boundary. `global.patches == global.patches[0]` reads the latest accepted retained global patch, with higher patch indices walking only that scope's retained accepted patches. Indices are bounded to zero through seven; unavailable pre-origin or pre-tail history is an error. Resolver aliases are not literal JSON containers. Reads stay cached and create no Git query, publication, checkpoint append, or semantic step.
210
213
 
@@ -4,7 +4,7 @@ This matrix records exact tested dependency stacks, not the version of an operat
4
4
 
5
5
  ## Current release candidate
6
6
 
7
- The 0.12.0 candidate passes `npm run validate` on the repository-local 0.84.4 dependency graph: typecheck, import check, and 451/451 tests. Its focused path/config/interactive-rendering cohort passes 41/41, including current-as-index-zero `read_state` aliases, composed-lineage scoped reads, retained scope-patch reads, legacy `{offset, scope}` compatibility, and both visible/default and suppressed `patch_state` argument rendering. A live enabled host also resolves current and historical legacy reads; the host must reload the candidate before its model-facing tool schema can expose the new `path` input. The 0.85.1 full-suite result below belongs to the earlier source checkpoint and has not been repeated for this candidate.
7
+ The 0.16.0 candidate passes `npm run validate` on the repository-local 0.84.4 dependency graph: build, typecheck, import check, package dry run, and 444/444 tests. Its focused context/Skill/invariant cohort passes 43/43, including discovery of the separate operational and memory-curation Skills and release-package inventory checks for both compiled Skill paths. Current and historical reads use semantic `path`/`paths`; the retired top-level `state` segment and legacy top-level `offset`/`scope` inputs are rejected. The 0.85.1 full-suite evidence below belongs to the earlier recorded source checkpoint and has not been repeated for this candidate.
8
8
 
9
9
  ## Tested matrix
10
10
 
@@ -46,13 +46,21 @@ global | CWD | session
46
46
  └── lazy
47
47
  ```
48
48
 
49
- `artifacts`, `contract`, `working`, `intents`, and `response` remain hot. `intents` may keep compact active direction while pointing to large supporting detail in `lazy` with an ordinary `{"$ref":"cwd.lazy.plan"}` value. References are read explicitly and never auto-hydrated, rewritten, executed, or completed. `lazy` differs only in projection policy:
49
+ `artifacts`, `contract`, `working`, `intents`, and `response` remain hot. `intents` may keep compact active direction while referring to large supporting detail in `lazy`. `lazy` differs only in projection policy:
50
50
 
51
51
  - It is canonical semantic JSON, validated and versioned with its owning scope.
52
52
  - It is excluded from the ordinary baseline effective-state body.
53
53
  - It becomes model-visible only through bounded baseline navigation hints or explicit `read_state` output.
54
54
  - Reading it does not mutate state, freshness, usage metadata, history, or future context.
55
55
 
56
+ ### Semantic references
57
+
58
+ A reference is semantic content, not a runtime type. The optional `{"$ref":"cwd.lazy.plan"}` object provides the structured state-reference form. Inside any ordinary string or paragraph, a semantic-state reference uses `$` immediately followed by one valid `read_state` path, for example `$effective.lazy.memory[7]`. The prefix separates a deliberate reference from incidental path-like text and provides a deterministic seam if code-based parsing is ever justified. File paths, document sections, URIs, artifact locators, Skill identities, and agent identities retain their native syntax.
59
+
60
+ State Flow preserves all forms exactly as ordinary JSON. It does not scan prose, index targets, validate existence, rewrite relative locators, or infer authority, dependency, hydration, execution, or completion. When a reference matters, the agent resolves it explicitly with `read_state` for semantic paths or the appropriate external read/tool for other resources. A locator supports retrieval but does not replace content required for the current decision.
61
+
62
+ Reference repair is reactive, not a maintenance scan. The agent does not enumerate, audit, or resolve references merely to test them. Only after one requested `read_state` value path is missing does State Flow perform one bounded reverse lookup over current model-patchable semantic planes for exact structured `$ref` and `$path` matches. If found, the tool returns `{value:null, hint:[{type:"dangling-reference", message, paths}]}`. `hint` is explicit top-level metadata rather than state data; its action message asks for reconciliation and its path array contains at most three runtime-verified current owners. The null sentinel is never returned alone for this case. Keys, patch, and multi-path reads keep ordinary all-or-error semantics; no durable match retains the missing-path error and does not prove the agent invented the path. The agent may then reconcile a proven stale owning value while preserving surrounding meaning. This applies equally to `$ref` objects and contextual references in prose. Effective-state absence alone does not identify the owner, and unavailable history, inaccessible external resources, or transient read failure do not prove that a durable reference is broken.
63
+
56
64
  Valid lazy values include:
57
65
 
58
66
  ```json
@@ -106,7 +114,7 @@ Active obligations, current constraints, unresolved next actions, and facts requ
106
114
 
107
115
  `projection` defaults to `value`. A batch uses one projection for every path, evaluates every path against one captured state view, and returns results in request order. Duplicate paths remain duplicate results. If any path is invalid, the whole read fails; there is no mixed partial result.
108
116
 
109
- The legacy single `path` form remains first-class rather than mere compatibility syntax. Existing `offset`/`scope` input may remain temporarily during migration but cannot combine with `path` or `paths`.
117
+ The single `path` form is first-class. The retired top-level `offset` and `scope` inputs are rejected; history and ownership belong in the semantic path itself, such as `cwd[1].lazy.memory`.
110
118
 
111
119
  ### Response shape
112
120
 
@@ -118,13 +126,13 @@ There are three projections:
118
126
  | `keys` | `{ "meta": ..., "keys": ... }` | Minimal structural facts followed by immediate keys |
119
127
  | `patch` | `{ "patch": ... }` | Historical semantic patch at the selected boundary |
120
128
 
121
- A single-path request returns one payload. A multi-path request returns positionally aligned arrays under the same projection fields.
129
+ A single-path request returns one payload. A multi-path request returns positionally aligned arrays under the same projection fields. One missing value path with exact current durable references instead returns `{ "value": null, "hint": [{ "type": "dangling-reference", "message": "Reconcile the verified current values that reference this path.", "paths": ["cwd.working.note"] }] }`; this explicit sentinel is diagnostic metadata, not semantic state.
122
130
 
123
- `value` deliberately mirrors the effective-state snapshot injected at iteration start: it is semantic state without revision, provenance, range, transport, or storage fields. `patch` likewise contains only the selected semantic patch. Only structural discovery earns `meta`, and `keys` remains the final and most valuable field in that response.
131
+ `value` otherwise deliberately mirrors the effective-state snapshot injected at iteration start: it is semantic state without revision, provenance, range, transport, or storage fields. `patch` likewise contains only the selected semantic patch. Only structural discovery earns `meta`, and `keys` remains the final and most valuable field in that response.
124
132
 
125
133
  The response does not repeat the requested path or projection and does not return an internal revision. Runtime owns revision selection, locking, CAS, and publication; the model cannot improve correctness by echoing that machinery.
126
134
 
127
- Errors use the normal tool-error channel rather than successful JSON containing an `error` field.
135
+ Errors use the normal tool-error channel rather than successful JSON containing an `error` field. The explicit dangling-reference sentinel above is the sole missing-path exception; keys, patch, multi-path, and unmatched value reads still fail.
128
136
 
129
137
  ### Path and range model
130
138
 
package/docs/usage.md CHANGED
@@ -20,7 +20,7 @@ State Flow does not undo tool effects. After interruption or returning to an old
20
20
 
21
21
  Native fork replacement copies the source session checkpoint, retained patch tail and matching provenance into the new session's own storage. Global/CWD streams and provenance stay current and unchanged. An earlier fork selection copies that point's private state, not the parent's later private work. Parent data/history remain intact; selected enablement is retained, so a stopped source does not become enabled automatically.
22
22
 
23
- The child starts at step zero and a new temporal origin. Its copied tail is preserved, but pre-origin records are not seven past aligned `state[n]` boundaries. The child's own transitions build its hot window; owned checkpoints support normal reload/resume. Parent Stop projection is not inherited, including after child reload.
23
+ The child starts at step zero and a new temporal origin. Its copied tail is preserved, but pre-origin records are not seven past aligned causal boundaries addressable through `effective[n]` or scoped paths. The child's own transitions build its hot window; owned checkpoints support normal reload/resume. Parent Stop projection is not inherited, including after child reload.
24
24
 
25
25
  Initial copying requires a native fork start event, a regular canonical direct-parent session file, matching CWD/identity, a readable temporal source and an unused child namespace. Missing/unsafe evidence or CAS conflicts leave the copy unavailable rather than importing unrelated or newer private state. Explicit Start can retry an unaccepted copy in the same loaded fork after the cause is corrected.
26
26
 
@@ -144,4 +144,4 @@ The agent should use sufficient materialized knowledge before rereading files. R
144
144
 
145
145
  Knowledge discovery finds regular lowercase `*.md` beneath its configured root, hashes opaque bytes, and skips symlinks. Only confirmed missing Markdown paths within an available root are pruned; external/non-Markdown artifacts and state under a missing whole root are preserved. An unavailable root makes freshness unknown in status. Successful reads of stale ordinary candidates require same-path global compilation; Skill reads require CWD compilation. The runtime owns provenance: model patches cannot write or delete individual freshness fields, including legacy spellings. Existing legacy entries remain readable and semantically editable. Missing freshness evidence means unknown-but-usable, not proof that a source was acquired.
146
146
 
147
- The optional `state-flow-memory` Skill handles explicit bounded curation, required feature/release/project phase-boundary reconciliation, and external promotion. Ordinary handoffs clean only touched and obviously stale visible branches; the Skill supplies the fuller scoped migration procedure when a phase boundary or explicit request earns it. It is not a background maintenance loop. Promotion must verify the destination before removing the only accepted source copy. Artifact/compiler details and model-tool contracts belong in the [architecture](architecture.md#artifact-routing).
147
+ The packaged `state-flow-guide` Skill answers concrete operational questions about reads, patches, inheritance, acquisition, finalization, and recovery without initiating cleanup. The separate `state-flow-memory` Skill handles explicit bounded curation, feature/release/project phase-boundary reconciliation, and external promotion. Ordinary handoffs clean only touched and obviously stale visible branches; neither Skill is a background maintenance loop. Promotion must verify the destination before removing the only accepted source copy. Artifact/compiler details and model-tool contracts belong in the [architecture](architecture.md#artifact-routing).
package/lib/extension.ts CHANGED
@@ -4,27 +4,24 @@ import { StringEnum, Type } from "@earendil-works/pi-ai";
4
4
  import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
5
5
  import { getAgentDir } from "@earendil-works/pi-coding-agent";
6
6
  import { Text } from "@earendil-works/pi-tui";
7
- import { assistantToolCallCount, finalizedAssistantResponse, stateFlowProtocol } from "./protocol.ts";
7
+ import { assistantToolCallCount, finalizedAssistantResponse, formatPatchStateArguments, normalizePatchStateArguments, separatedFailure, separatedOutput, stateFlowProtocol } from "./protocol.ts";
8
8
  import { createPassiveContinuation, currentRunTrajectory, lazyNavigationHint, passiveContinuationMessages, runtimeContextMessage, syntheticUser, VALIDATION_MESSAGE_TYPE, withoutPrivateValidation, type PassiveContinuation } from "./context.ts";
9
9
  import { ArtifactReadTracker } from "./acquisition.ts";
10
10
  import { loadStateFlowConfig } from "./config.ts";
11
11
  import { createStateFlowTelegramAdapter, type StateFlowTelegramControlResult, type StateFlowTelegramLoader } from "./telegram.ts";
12
- import { isAbsolute, relative, resolve, sep } from "node:path";
12
+ import { isAbsolute, resolve } from "node:path";
13
13
  import { SkillReadTracker } from "./skills.ts";
14
14
  import { emptySnapshot, migrationFailure, persistableSnapshot, RevisionUnavailableError, type Snapshot } from "./snapshot.ts";
15
15
  import { readNativeSessionHeader } from "./continuation.ts";
16
16
  import { MissingSessionRuntimeError, SharedScopeRemovalConflictError, TemporalRuntime, type RuntimePublication } from "./runtime.ts";
17
17
  import { emptyState, overlayStates, projectStateForModel, type AtomicScopePatches, type MaterializedState, type ScopedStates, type StateScope } from "./state.ts";
18
18
  import { commitScopedTransition, stageAtomicScopePatches, stageScopedTransition, validateFinalEligibility, type StagedScopedTransition } from "./transition.ts";
19
- import { discoverSnapshotData, hasPriorConversation, isNewSession, SNAPSHOT_ENTRY_TYPE } from "./session.ts";
19
+ import { discoverSnapshotData, findAssistantToolBatch, findPassiveStopBoundary, hasPriorConversation, isNewSession, retainsPhysicalSessionProjection, SNAPSHOT_ENTRY_TYPE } from "./session.ts";
20
20
  import { compactStatus, detailedStatus, STATUS_KEY, type PendingPublicationDiagnostic, type StatusDiagnostics } from "./status.ts";
21
21
  import { completeRun, prepareRun, resumeEpisode, startEpisode, stopEpisode } from "./episode.ts";
22
22
  import { recoverSnapshot } from "./recovery.ts";
23
23
  import type { RehydrationPhase } from "./rehydration.ts";
24
- import { resolveRemotePublicationPolicy, serializeRemotePublicationPolicyDocument } from "./publication.ts";
25
- import { coalescePublicationTarget, createPublicationQueue, type PublicationQueueState } from "./publication.ts";
26
- import { acquirePublicationWorkerLease, loadPublicationQueue, publicationQueuePath, removePublicationQueue, savePublicationQueue } from "./publication.ts";
27
- import { runPublicationWorker } from "./publication.ts";
24
+ import { loadPublicationQueue, publicationQueuePath, PublicationWorkerController, resolveRemotePublicationPolicy, serializeRemotePublicationPolicyDocument, type PublicationQueueState } from "./publication.ts";
28
25
  import { getKnowledgeRoot, GlobalMarkdownDiscovery } from "./discovery.ts";
29
26
  import { canonicalJson, isObject, sameJson } from "./json.ts";
30
27
  import { readProjectedState, readStatePath } from "./query.ts";
@@ -35,7 +32,7 @@ import {
35
32
  type SessionAddress,
36
33
  } from "./durable.ts";
37
34
  import { projectRecentTransitionsWithLimit, RECENT_TRANSITION_LIMIT } from "./history.ts";
38
- import { appendStateFlowDiagnostic, projectDiagnosticContent, stateFlowLogPath, type StateFlowDiagnosticCategory } from "./logging.ts";
35
+ import { StateFlowDiagnosticWriter, stateFlowLogPath, type DiagnosticExtras, type StateFlowDiagnosticCategory } from "./logging.ts";
39
36
  import { isGitCommitAncestor, pushGitCommit, pushGitTarget, resolveGitPushDestination } from "./git.ts";
40
37
  import {
41
38
  ORDINARY_ARTIFACT_COMPILER,
@@ -54,64 +51,13 @@ export interface StateFlowExtensionOptions {
54
51
  passive?: { bootstrap?: boolean; tools?: boolean };
55
52
  }
56
53
 
54
+ export { formatPatchStateArguments, normalizePatchStateArguments };
55
+
57
56
  export const PATCH_STATE_TOOL_NAME = "patch_state";
58
57
  export const READ_STATE_TOOL_NAME = "read_state";
59
58
  export const MAX_FALLBACK_ATTEMPTS: number = 2;
60
59
  const PASSIVE_STOP_ENTRY_TYPE = "state-flow-passive-stop";
61
60
  const PUBLICATION_SHUTDOWN_WAIT_MS = 2_000;
62
- const PATCH_DISPLAY_SECTION_KEYS = new Set([
63
- "global",
64
- "cwd",
65
- "session",
66
- "artifacts",
67
- "contract",
68
- "working",
69
- "response",
70
- "final",
71
- ]);
72
-
73
- /** Keep successful patch JSON valid while separating adjacent scopes and memory sections visually. */
74
- export function formatPatchStateArguments(args: unknown): string {
75
- const seenAtIndent = new Set<number>();
76
- return JSON.stringify(args, null, 2).split("\n").flatMap((line) => {
77
- const indent = line.length - line.trimStart().length;
78
- const match = /^(\s+)"([^"]+)":/.exec(line);
79
- if (match === null || !PATCH_DISPLAY_SECTION_KEYS.has(match[2])) {
80
- for (const seenIndent of seenAtIndent) {
81
- if (seenIndent > indent) seenAtIndent.delete(seenIndent);
82
- }
83
- return [line];
84
- }
85
- const separator = seenAtIndent.has(indent) ? [""] : [];
86
- seenAtIndent.add(indent);
87
- return [...separator, line];
88
- }).join("\n");
89
- }
90
-
91
- /** Normalize a bounded compatibility superset without advertising aliases in the model-facing contract. */
92
- export function normalizePatchStateArguments(args: unknown): any {
93
- if (!isObject(args) || !Object.hasOwn(args, "final")) return args;
94
- const value = args.final;
95
- let final: boolean;
96
- if (typeof value === "boolean") final = value;
97
- else if (value === 1) final = true;
98
- else if (value === 0) final = false;
99
- else if (typeof value === "string" && value.trim().toLowerCase() === "true") final = true;
100
- else if (typeof value === "string" && value.trim().toLowerCase() === "false") final = false;
101
- else return args;
102
- return { ...args, final };
103
- }
104
-
105
- /** Keep tool output visually separated from its heading with exactly one leading newline. */
106
- function separatedOutput(text: string): string {
107
- return `\n${text.replace(/^\n+/, "")}`;
108
- }
109
-
110
- /** Keep a failed tool invocation visually separated from its rendered error without changing error semantics. */
111
- function separatedFailure(error: unknown): Error {
112
- const message = error instanceof Error ? error.message : String(error);
113
- return new Error(separatedOutput(message), error instanceof Error ? { cause: error } : undefined);
114
- }
115
61
 
116
62
  export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowExtensionOptions = {}): void {
117
63
  const agentDir = options.agentDir ?? getAgentDir();
@@ -145,15 +91,22 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
145
91
  let pendingPublication: PendingPublicationDiagnostic | undefined;
146
92
  let rehydrationPhase: RehydrationPhase | undefined;
147
93
  let turnPublicationTarget: string | undefined;
148
- const activePublicationWorkers = new Map<string, { controller: AbortController; done: Promise<void> }>();
149
- let publicationStopped = false;
150
94
  let publicationShutdown: Promise<void> | undefined;
151
95
  const repositoryRoot = resolve(options.repositoryRoot ?? config.directory);
96
+ const diagnosticWriter = new StateFlowDiagnosticWriter(config.logging, stateFlowLogPath(agentDir), repositoryRoot, (message) => activeContext?.ui.notify(message, "warning"));
97
+ const publicationWorker = new PublicationWorkerController({
98
+ resolveDestination: () => resolveGitPushDestination(repositoryRoot),
99
+ isAncestor: (ancestor, descendant) => isGitCommitAncestor(repositoryRoot, ancestor, descendant),
100
+ push: (destination, target, signal) => pushGitTarget(repositoryRoot, destination, target, signal),
101
+ onDiverged: (dropped, target) => activeContext && recordDiagnostic(
102
+ `Retired publication queue target ${dropped.target} after a journal lineage rewrite; retargeting to ${target}`,
103
+ "publication-conflict", activeContext,
104
+ ),
105
+ });
152
106
  const skillReads = new SkillReadTracker();
153
107
  const artifactReads = new ArtifactReadTracker();
154
108
  const globalMarkdown = new GlobalMarkdownDiscovery(options.knowledgeRoot ?? getKnowledgeRoot(agentDir));
155
109
  let artifactInvalidations: ArtifactInvalidationRequest[] = [];
156
- let loggingWarningReported = false;
157
110
  let telegramStartPending = false;
158
111
 
159
112
  function sessionAddress(ctx: ExtensionContext): SessionAddress {
@@ -206,27 +159,6 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
206
159
  artifactReads.clear();
207
160
  }
208
161
 
209
- function passiveStopBoundary(ctx: ExtensionContext): { at: number; from?: number } | undefined {
210
- for (const entry of [...ctx.sessionManager.getBranch()].reverse()) {
211
- try {
212
- if (entry?.type !== "custom" || entry.customType !== PASSIVE_STOP_ENTRY_TYPE) continue;
213
- const { at, from, reset, owner } = (entry.data as { at?: unknown; from?: unknown; reset?: unknown; owner?: unknown } | undefined) ?? {};
214
- if (reset === true && owner === ctx.sessionManager.getSessionId()) return undefined;
215
- if (typeof at === "number" && Number.isSafeInteger(at) && at >= 0) return {
216
- at,
217
- ...(typeof from === "number" && Number.isSafeInteger(from) && from >= 0 ? { from } : {}),
218
- };
219
- } catch {
220
- // A hostile unrelated branch entry cannot manufacture or suppress a valid marker.
221
- }
222
- }
223
- return undefined;
224
- }
225
-
226
- function retainsPhysicalSessionProjection(reason: unknown): boolean {
227
- return reason === undefined || reason === "startup" || reason === "reload" || reason === "resume";
228
- }
229
-
230
162
  function deferArtifactRefresh(): void {
231
163
  artifactInvalidations = [];
232
164
  artifactReads.setCandidates([]);
@@ -289,21 +221,6 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
289
221
  : active.filter((name) => !owned.includes(name)));
290
222
  }
291
223
 
292
- function assistantToolBatch(ctx: ExtensionContext, toolCallId: string): string[] | undefined {
293
- for (let cursor = ctx.sessionManager.getLeafEntry(); cursor; cursor = cursor.parentId ? ctx.sessionManager.getEntry(cursor.parentId) : undefined) {
294
- const entry = cursor as { type?: unknown; message?: { role?: unknown; content?: unknown } };
295
- if (entry.type !== "message" || entry.message?.role !== "assistant" || !Array.isArray(entry.message.content)) continue;
296
- const calls = entry.message.content.filter((block): block is { type: "toolCall"; id: string; name: string } => {
297
- return typeof block === "object" && block !== null
298
- && (block as { type?: unknown }).type === "toolCall"
299
- && typeof (block as { id?: unknown }).id === "string"
300
- && typeof (block as { name?: unknown }).name === "string";
301
- });
302
- if (calls.some(({ id }) => id === toolCallId)) return calls.map(({ name }) => name);
303
- }
304
- return undefined;
305
- }
306
-
307
224
  function recordPublication(publication: RuntimePublication, ctx: ExtensionContext): void {
308
225
  const revision = publication.revision ?? publication.commit;
309
226
  if (revision !== undefined) snapshot.meta.durableBase = revision;
@@ -330,8 +247,8 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
330
247
  if (mode !== "turn-end" || target === undefined || !/^[0-9a-f]{40,64}$/.test(target)) return;
331
248
  turnPublicationTarget = target;
332
249
  try {
333
- enqueueTurnPublication(ctx);
334
- launchPublicationWorker();
250
+ enqueueTurnPublication();
251
+ publicationWorker.launch();
335
252
  } catch (error) {
336
253
  ctx.ui.notify(
337
254
  `State Flow accepted the local commit; remote publication is deferred: ${error instanceof Error ? error.message : String(error)}`,
@@ -372,95 +289,16 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
372
289
  return true;
373
290
  }
374
291
 
375
- function enqueueTurnPublication(ctx: ExtensionContext): void {
292
+ function enqueueTurnPublication(): void {
376
293
  const target = turnPublicationTarget;
377
294
  turnPublicationTarget = undefined;
378
- if (!target) return;
379
- const destination = resolveGitPushDestination(repositoryRoot);
380
- if (!destination) return;
381
- const path = publicationQueuePath(destination);
382
- const previous = loadPublicationQueue(path);
383
- const next = previous
384
- ? coalescePublicationTarget(previous, destination, target, (ancestor, descendant) => isGitCommitAncestor(repositoryRoot, ancestor, descendant), {
385
- onDivergedLineage: (dropped) => recordDiagnostic(
386
- `Retired publication queue target ${dropped.target} after a journal lineage rewrite; retargeting to ${target}`,
387
- "publication-conflict", ctx,
388
- ),
389
- })
390
- : createPublicationQueue(destination, target);
391
- savePublicationQueue(path, next, previous);
392
- }
393
-
394
- function launchPublicationWorker(): void {
395
- if (publicationStopped) return;
396
- let destination: ReturnType<typeof resolveGitPushDestination>;
397
- try {
398
- destination = resolveGitPushDestination(repositoryRoot);
399
- } catch (error) {
400
- if (error instanceof Error && /ENOENT/.test(error.message)) return;
401
- throw error;
402
- }
403
- if (!destination) return;
404
- const path = publicationQueuePath(destination);
405
- if (activePublicationWorkers.has(path)) return;
406
- let queued: PublicationQueueState | undefined;
407
- let lease: ReturnType<typeof acquirePublicationWorkerLease>;
408
- try {
409
- queued = loadPublicationQueue(path);
410
- if (!queued) return;
411
- lease = acquirePublicationWorkerLease(path);
412
- } catch {
413
- return;
414
- }
415
- if (!lease) return;
416
- const controller = new AbortController();
417
- const done = runPublicationWorker(
418
- queued,
419
- ({ target }) => pushGitTarget(repositoryRoot, destination, target, controller.signal),
420
- () => loadPublicationQueue(path) ?? queued,
421
- (ancestor, descendant) => isGitCommitAncestor(repositoryRoot, ancestor, descendant),
422
- ).then((result) => {
423
- if (publicationStopped) return; // Late outcomes belong to an unconfirmed queue, not the replacement generation.
424
- const current = loadPublicationQueue(path);
425
- if (!current) return;
426
- if (result.next === undefined) {
427
- if (current.target === result.attempted.target) removePublicationQueue(path, current);
428
- return;
429
- }
430
- if (current.target === result.attempted.target || result.next.target === current.target) {
431
- savePublicationQueue(path, result.next, current);
432
- }
433
- }).catch(() => {
434
- // Queue/CAS truth remains durable; status and a later activation expose retry.
435
- }).finally(() => {
436
- activePublicationWorkers.delete(path);
437
- try {
438
- lease.release();
439
- if (!publicationStopped && loadPublicationQueue(path)?.status === "pending") launchPublicationWorker();
440
- } catch {
441
- // Failed lease cleanup or malformed persistence stays inert until retry or repair.
442
- }
443
- });
444
- activePublicationWorkers.set(path, { controller, done });
295
+ if (target) publicationWorker.enqueue(target);
445
296
  }
446
297
 
447
298
  function shutdownPublicationWorkers(ctx: ExtensionContext): Promise<void> {
448
- publicationStopped = true;
449
- return publicationShutdown ??= (async () => {
450
- const workers = [...activePublicationWorkers.values()];
451
- for (const { controller } of workers) controller.abort();
452
- if (workers.length === 0) return;
453
- let timeout: ReturnType<typeof setTimeout> | undefined;
454
- try {
455
- const completed = await Promise.race([
456
- Promise.all(workers.map(({ done }) => done)).then(() => true),
457
- new Promise<false>((resolve) => { timeout = setTimeout(() => resolve(false), PUBLICATION_SHUTDOWN_WAIT_MS); }),
458
- ]);
459
- if (!completed) ctx.ui.notify(`State Flow push cleanup is unconfirmed after ${PUBLICATION_SHUTDOWN_WAIT_MS}ms; worker leases remain held until child exit.`, "warning");
460
- } finally {
461
- clearTimeout(timeout);
462
- }
463
- })();
299
+ return publicationShutdown ??= publicationWorker.shutdown(PUBLICATION_SHUTDOWN_WAIT_MS).then((completed) => {
300
+ if (!completed) ctx.ui.notify(`State Flow push cleanup is unconfirmed after ${PUBLICATION_SHUTDOWN_WAIT_MS}ms; worker leases remain held until child exit.`, "warning");
301
+ });
464
302
  }
465
303
 
466
304
  function retryPendingPush(ctx: ExtensionContext): void {
@@ -472,8 +310,8 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
472
310
  if (mode === "turn-end") {
473
311
  turnPublicationTarget = target;
474
312
  try {
475
- enqueueTurnPublication(ctx);
476
- launchPublicationWorker();
313
+ enqueueTurnPublication();
314
+ publicationWorker.launch();
477
315
  } catch (error) {
478
316
  ctx.ui.notify(`State Flow retained local state; asynchronous publication recovery is deferred: ${error instanceof Error ? error.message : String(error)}`, "warning");
479
317
  }
@@ -681,7 +519,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
681
519
  ctx.ui.notify(`State Flow restored disabled: ${snapshot.meta.validation.error}`, "error");
682
520
  }
683
521
  if (retainsPhysicalSessionProjection(sessionStartReason) && runtime?.view) {
684
- const boundary = passiveStopBoundary(ctx);
522
+ const boundary = findPassiveStopBoundary(ctx.sessionManager.getBranch(), ctx.sessionManager.getSessionId(), PASSIVE_STOP_ENTRY_TYPE);
685
523
  if (boundary !== undefined) {
686
524
  const continuation = createPassiveContinuation(
687
525
  projectStateForModel(overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session)),
@@ -705,41 +543,8 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
705
543
  updateUi(ctx);
706
544
  }
707
545
 
708
- interface DiagnosticExtras {
709
- content?: unknown;
710
- input?: unknown;
711
- tool?: string;
712
- toolCallId?: string;
713
- resolutionAttempt?: number;
714
- terminalEligible?: boolean;
715
- }
716
-
717
546
  function recordDiagnostic(error: string, category: StateFlowDiagnosticCategory, ctx: ExtensionContext, extras: DiagnosticExtras = {}): void {
718
- if (!config.logging) return;
719
- try {
720
- const path = stateFlowLogPath(agentDir);
721
- const fromRepository = relative(repositoryRoot, path);
722
- if (fromRepository === "" || (!isAbsolute(fromRepository) && fromRepository !== ".." && !fromRepository.startsWith(`..${sep}`))) {
723
- throw new Error("diagnostic path overlaps the State Flow repository");
724
- }
725
- appendStateFlowDiagnostic(path, {
726
- at: new Date().toISOString(),
727
- sessionId: sessionAddress(ctx).id,
728
- cwd: resolve(ctx.cwd),
729
- category,
730
- error,
731
- ...(extras.content === undefined ? {} : { content: projectDiagnosticContent(extras.content) }),
732
- ...(extras.input === undefined ? {} : { input: extras.input }),
733
- ...(extras.tool === undefined ? {} : { tool: extras.tool }),
734
- ...(extras.toolCallId === undefined ? {} : { toolCallId: extras.toolCallId }),
735
- ...(extras.resolutionAttempt === undefined ? {} : { resolutionAttempt: extras.resolutionAttempt }),
736
- ...(extras.terminalEligible === undefined ? {} : { terminalEligible: extras.terminalEligible }),
737
- });
738
- } catch (failure) {
739
- if (loggingWarningReported) return;
740
- loggingWarningReported = true;
741
- ctx.ui.notify(`State Flow could not write diagnostics: ${failure instanceof Error ? failure.message : String(failure)}`, "warning");
742
- }
547
+ diagnosticWriter.record(sessionAddress(ctx).id, ctx.cwd, error, category, extras);
743
548
  }
744
549
 
745
550
  function continueFallbackResolution(lastWarning: boolean): void {
@@ -1193,7 +998,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
1193
998
 
1194
999
  pi.on("tool_call", (event, ctx) => {
1195
1000
  if (!snapshot.config.enabled) return;
1196
- const batch = assistantToolBatch(ctx, event.toolCallId);
1001
+ const batch = findAssistantToolBatch(ctx.sessionManager, event.toolCallId);
1197
1002
  const patchCalls = batch?.filter((name) => name === PATCH_STATE_TOOL_NAME).length ?? 0;
1198
1003
  if (patchCalls > 0) {
1199
1004
  if (patchCalls !== 1) {
@@ -1264,8 +1069,8 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
1264
1069
  responseCommitted = true;
1265
1070
  bootstrapContinuation = undefined;
1266
1071
  rehydrationPhase = "step";
1267
- enqueueTurnPublication(ctx);
1268
- if (snapshot.meta.remotePublication?.mode === "turn-end") launchPublicationWorker();
1072
+ enqueueTurnPublication();
1073
+ if (snapshot.meta.remotePublication?.mode === "turn-end") publicationWorker.launch();
1269
1074
  completedRunAccepted = !wasBootstrap;
1270
1075
  } catch (error) {
1271
1076
  if (!responseCommitted && specification !== undefined) snapshot.meta.specification = specification;
@@ -1292,7 +1097,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
1292
1097
  telegramStartPending = false;
1293
1098
  startStateFlow(ctx);
1294
1099
  }
1295
- if (snapshot.meta.remotePublication?.mode === "turn-end") launchPublicationWorker();
1100
+ if (snapshot.meta.remotePublication?.mode === "turn-end") publicationWorker.launch();
1296
1101
  if (!completedRunAccepted || compactionStopped || !snapshot.config.enabled || snapshot.meta.bootstrap || resolutionPending
1297
1102
  || compactionInFlight || !ctx.isIdle() || ctx.hasPendingMessages() || !snapshot.meta.durableBase
1298
1103
  || !shouldRequestStateFlowCompaction(ctx.getContextUsage())) return;
@@ -1314,7 +1119,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
1314
1119
  rehydrationPhase = event.reason === "resume" ? "resume-bootstrap" : "new-bootstrap";
1315
1120
  restoreActiveBranch(ctx, event.reason);
1316
1121
  retryPendingPush(ctx);
1317
- if (snapshot.meta.remotePublication?.mode === "turn-end") launchPublicationWorker();
1122
+ if (snapshot.meta.remotePublication?.mode === "turn-end") publicationWorker.launch();
1318
1123
  updateUi(ctx);
1319
1124
  void telegram.ensure();
1320
1125
  });
package/lib/logging.ts CHANGED
@@ -1,6 +1,6 @@
1
1
  // Domain: opt-in diagnostic capture for rejected State Flow resolutions.
2
2
  import { appendFileSync, mkdirSync } from "node:fs";
3
- import { dirname, join } from "node:path";
3
+ import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
4
4
  import { isObject } from "./json.ts";
5
5
 
6
6
  export type StateFlowDiagnosticCategory = "invalid-patch" | "publication-conflict" | "terminal-pending" | "finalization";
@@ -45,3 +45,55 @@ export function appendStateFlowDiagnostic(path: string, record: StateFlowDiagnos
45
45
  mkdirSync(dirname(path), { recursive: true });
46
46
  appendFileSync(path, `${JSON.stringify(record)}\n`, { encoding: "utf8", mode: 0o600 });
47
47
  }
48
+
49
+ export interface DiagnosticExtras {
50
+ content?: unknown;
51
+ input?: unknown;
52
+ tool?: string;
53
+ toolCallId?: string;
54
+ resolutionAttempt?: number;
55
+ terminalEligible?: boolean;
56
+ }
57
+
58
+ /** Own diagnostic path safety, projection, persistence, and one-shot failure reporting. */
59
+ export class StateFlowDiagnosticWriter {
60
+ private warningReported = false;
61
+ private readonly enabled: boolean;
62
+ private readonly path: string;
63
+ private readonly repositoryRoot: string;
64
+ private readonly notify: (message: string) => void;
65
+
66
+ constructor(enabled: boolean, path: string, repositoryRoot: string, notify: (message: string) => void) {
67
+ this.enabled = enabled;
68
+ this.path = path;
69
+ this.repositoryRoot = repositoryRoot;
70
+ this.notify = notify;
71
+ }
72
+
73
+ record(sessionId: string, cwd: string, error: string, category: StateFlowDiagnosticCategory, extras: DiagnosticExtras = {}): void {
74
+ if (!this.enabled) return;
75
+ try {
76
+ const fromRepository = relative(this.repositoryRoot, this.path);
77
+ if (fromRepository === "" || (!isAbsolute(fromRepository) && fromRepository !== ".." && !fromRepository.startsWith(`..${sep}`))) {
78
+ throw new Error("diagnostic path overlaps the State Flow repository");
79
+ }
80
+ appendStateFlowDiagnostic(this.path, {
81
+ at: new Date().toISOString(),
82
+ sessionId,
83
+ cwd: resolve(cwd),
84
+ category,
85
+ error,
86
+ ...(extras.content === undefined ? {} : { content: projectDiagnosticContent(extras.content) }),
87
+ ...(extras.input === undefined ? {} : { input: extras.input }),
88
+ ...(extras.tool === undefined ? {} : { tool: extras.tool }),
89
+ ...(extras.toolCallId === undefined ? {} : { toolCallId: extras.toolCallId }),
90
+ ...(extras.resolutionAttempt === undefined ? {} : { resolutionAttempt: extras.resolutionAttempt }),
91
+ ...(extras.terminalEligible === undefined ? {} : { terminalEligible: extras.terminalEligible }),
92
+ });
93
+ } catch (failure) {
94
+ if (this.warningReported) return;
95
+ this.warningReported = true;
96
+ this.notify(`State Flow could not write diagnostics: ${failure instanceof Error ? failure.message : String(failure)}`);
97
+ }
98
+ }
99
+ }