@llblab/pi-state-flow 0.19.1 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/BACKLOG.md +3 -3
- package/CHANGELOG.md +7 -0
- package/README.md +1 -1
- package/dist/lib/context.d.ts +50 -1
- package/dist/lib/context.js +146 -5
- package/dist/lib/extension.js +36 -27
- package/dist/lib/protocol.d.ts +1 -1
- package/dist/lib/protocol.js +3 -2
- package/dist/package.json +1 -1
- package/dist/skills/state-flow-guide/SKILL.md +3 -1
- package/docs/architecture.md +6 -2
- package/docs/performance.md +48 -2
- package/docs/temporal-acceptance.md +4 -3
- package/lib/context.ts +139 -6
- package/lib/extension.ts +37 -32
- package/lib/protocol.ts +4 -2
- package/package.json +1 -1
- package/skills/state-flow-guide/SKILL.md +3 -1
package/AGENTS.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
- Keep independent domain modules under `lib/`, mirror every domain with a same-named file under `tests/`, place cross-domain architecture checks in `tests/invariants.test.ts`, and keep `index.ts` as a minimal public-export boundary. Keep `lib/extension.ts` as the Pi lifecycle composition root: it may wire configuration, domain capabilities, handlers, and event subscriptions, but low-level parsing, traversal, formatting, persistence, queue/worker mechanics, and diagnostic I/O belong to their existing owning domains.
|
|
4
4
|
- Keep the extension opt-in and preserve clear attribution to SKILL.state wherever the inherited explicit-state approach is described.
|
|
5
5
|
- Distinguish three model-facing states: active, passive, and configured off. Active and passive both expose `read_state` and `patch_state` when memory tools are permitted; both use the same canonical persistence, private ownership and fork-copy rules. The difference is the agent workflow, not whether memory is saved. Active iterations must consolidate future-relevant results into accepted state before completed context is replaced: make the final meaningful semantic patch when needed, then the next iteration starts from current state plus its new input rather than the previous completed reasoning. Passive mode keeps ordinary conversation continuity and allows on-demand reading/patching without that state-driven iteration boundary or the same mandatory consolidation pressure. Configured off exposes neither memory tool and injects no State Flow bootstrap/state context; it is not another name for passive. In the current configuration, off means an inactive branch with both `passiveTools` and `passiveBootstrap` false; `autoStart: false` alone is not off. The independent one-flag configurations remain supported partial integrations, not full passive or full off. Start/Stop govern active/passive workflow and must not be treated as switches for canonical storage or fork initialization. See [mode semantics](docs/usage.md#active-passive-and-configured-off) for completion and bootstrap terminology.
|
|
6
|
-
- Preserve Pi's native tool loop and complete inspectable session trace; project completed-run history only at user-run boundaries. The
|
|
6
|
+
- Preserve Pi's native tool loop and complete inspectable session trace; project completed-run history only at user-run boundaries. The context domain owns model projection of the raw scope overlay; callers must not pre-project it. Freeze the complete active/bootstrap head, including timestamp, specification and recent transitions, at the first accepted context. Reset the volatile projection at a new active run, accepted active completion, Start/Stop, selection/resume/reload, native compaction or a removed/replaced native prefix; passive user turns alone never reset it. Keep changing artifact hints, invalidations, rehydration and otherwise unreported shared drift in stable-position tail notices, never head rewrites. Successful patch receipts advance the last communicated view without redundant notices. Bind state updates to the head's volatile projection ID so retained results from an earlier head remain historical; never store this ID in canonical state or runtime metadata. Skill acquisition hints remain in native read results. This cache grants no publication, history, selection or compaction authority and adds no tail-size threshold. Retain persistent and current-run context-bearing custom messages from other extensions, plus the complete current-run trajectory. Select a unique captured native user timestamp independently of text decoration, including SDK image hints. Observe native user events even while disabled; Start/Stop must retain that observed run anchor. Reset it at native before-agent-start and session-start/tree boundaries, not semantic mode changes. Observation alone never enables state, publishes memory or authorizes compaction. Without a capture, a unique exact specification match may select only model projection. Missing, nonfinite or ambiguous matches retain available context; projection must never assign the lifecycle run anchor. Direct completion persists no private validation feedback.
|
|
7
7
|
- Expose one canonical intent-first materialized state shape across global, CWD, and session scopes with exactly `intents`, `contract`, `working`, `artifacts`, `response`, and `lazy`; the first four and the required `lazy` root are flexible semantic objects, while `response` is the exact latest accepted assistant answer, including the empty string, and is owned only by session state. Global and CWD keep the required key as an empty structural placeholder; effective state inherits the session value. Automatic model-state and recent-transition projections omit lazy bodies. Remove whole `patch.lazy` entries and then empty scoped patches/transitions, omitting an empty recent window without renumbering retained identities or positions. Preserve hot-history projection, artifact metadata filtering and bounded lazy navigation. Canonical history, explicit value/patch reads and already communicated native/user/tool/response text remain unchanged. Preserve this plane order in model-facing projection and state presentation. `intents` contains only active commitments to future action, remains hot by default, and must not become a planner, scheduler, task manager, or execution loop.
|
|
8
8
|
- Key artifacts directly by their source path. Model-visible artifact entries require only a non-empty `description` plus optional forward-compatible metadata; runtime-owned compilation evidence (`sourceFingerprint`, `sourceHash`, `compilerRevision`, `compiledAt`) belongs in the scope `meta.json` provenance registry. Reserve model-authored `hint`; runtime may add it only to effective/model projection and never to canonical scope state. Retired embedded `hash`/`compiler`/`compiled_at` fields remain readable compatibility evidence and are stripped from model projection. Validate provenance restrictions on every authored artifact entry, not on retained legacy state: individual runtime/legacy provenance fields cannot be set or deleted by model patches in any scope.
|
|
9
9
|
- Inspect only exact artifact source paths already registered in global, CWD, or session state. Never scan directories, discover files, reserve a Knowledge root, interpret file extensions, or read unrelated source bodies. Observe regular non-symlink files through `size + mtimeNs`; proven absence removes the artifact and provenance from each exact owning scope, while relative, symlink, directory, inaccessible, or otherwise unavailable paths are non-destructive evidence.
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
- Bind active Pi checkpoints to retained semantic boundaries in the current canonical lineage. Awaited runtime restoration uses `withRestoreTransaction`: detach the pointer before waiting, capture/select private history beside current shared streams under one exclusion, recheck caller policy, and publish once before installing the candidate. Expiry or malformed evidence cannot authorize current-head or empty fallback; origin acceptance retains configured folding and causal provenance rules. Resume and tree restoration select only retained private session history while global/CWD scopes remain live; every failure resolving a selected boundary fails closed without falling through to older checkpoints, disabled markers, Git revisions, or unrelated newer private state. Passive shared-memory availability does not prove the selected private layer: preserve the recovery failure and reject historical session reads and passive publication until it is resolved. Explicit Start may independently activate the validated live same-session canonical head. A same-owner failed-Stop marker permits read-only loading of validated current memory on reload, never historical substitution or publication; passive tools cannot clear its write fence. Awaited recovery uses `refreshCurrentMemory`: capture current authority under exclusion, reject cancellation before cache installation, never publish or initialize absence, and retain the prior cache on failure. Its disabled snapshot omits unfinished specifications and grants no write authority; native callers still own selection and Stop fences. Native startup/tree/fork/auto-start/failed-Stop restoration awaits one owned lifetime in the composition root: pending selection stays passive and unavailable, selection revokes older work, and shutdown drains current and superseded operations. Start/Stop select agent policy, not the existence of canonical memory. Stop must not cancel retained restoration, auto-start memory initialization or fork copying: apply passive policy inside the pending memory acceptance and keep configured passive patching available afterward. Start waiter cancellation must not cancel independently owned attachment or copying. Only selection changes, shutdown, native operation cancellation and actual authority/storage failures revoke that work; a mode change must not manufacture a storage-error fence. Start pins physical identity across cancelable joins without cancelling independently owned recovery or Stop persistence. Only current acceptance installs memory, checkpoint, tools and UI. Keep read-only candidates detached until cancellation, physical identity and any intervening accepted cache change are rechecked.
|
|
26
26
|
- Every accepted semantic change, including session-only and response-only changes, atomically updates affected canonical checkpoint/tail pairs and temporal metadata without creating a Git commit. Each materially changed scope advances its own persisted revision exactly once; Global and CWD revisions are shared across their writers, Session is private, and Effective is represented by the `G#/C#/S#` revision vector rather than an invented scalar owner. After an accepted turn reconciles its response and reaches `agent_before_settle`, one best-effort backup may commit only State Flow-owned already-written files. Await the backup mutex before briefly locking canonical storage for bounded root/CWD/session namespace inventory and regular-file byte capture; hold the separate mutex through completion and never acquire it from a canonical writer. The Promise-returning backup API accepts cancellation. Combine a real Pi operation signal with shutdown cancellation and drain owned attempts before shutdown completes. Pi 0.87 provides no operation signal at settlement: refuse contention there with explicit diagnostic-only deferral until a later accepted turn, rather than trapping native Abort. Do not infer cancellation from an ended agent signal, steal locks, add a background owner, or apply this optional-backup fallback to semantic publication. Release the canonical lock before every Git command, filter, staging write, ref update, or index synchronization. Recheck the Git branch/head after capture waits. Stage only the captured snapshot through a private temporary worktree/index, preserve Git ignore/filter policy, and never inventory artifact sources or unrelated directory trees. Keep unrelated staged/index-only content and worktree edits intact; synchronize only exact owned index paths, include HEAD-owned deletions even when already absent from the caller index, and never create an unowned-only initial backup. Backup failure is diagnostic-only and never rolls back state, suppresses an answer, requests continuation, or blocks a later turn. After each successful backup attempt, asynchronously push the exact current commit to the attached branch's explicitly configured remote/ref without force. Permit at most one in-flight push per repository root in process; skip overlapping attempts, leaving a later accepted turn to push the latest HEAD. Session shutdown waits for the in-flight process to close, within the existing Git push timeout and termination behavior; suppress push-failure reporting after shutdown begins. Push failure has no semantic effect; while active, warn once per failure streak with a concise message and reset suppression after successful replication. Record the available redacted Git failure detail in the local diagnostic log even when general logging is off; if local recording fails, warn once with the available detail. A later accepted turn retries the latest backup. Durable push queues, publication workers, worker leases, and remote retry generations do not exist.
|
|
27
27
|
- Register `patch_state` as the sole model-authored semantic mutation protocol. Its canonical grammar accepts one or more fixed `global`, `cwd`, and `session` semantic patches; reject unknown fields, empty supplied scopes, and retired finalization or `{scope, patch}` / `unchanged` grammar. Apply authored Global/CWD operations to the current canonical head, preserving untouched values; overlapping targets follow successful acceptance order, not stale agent snapshots. Validate and publish all supplied scopes as one durable cohort, deriving replay and revisions from that actual basis. Correct repeats succeed without another semantic identity, step or scope revision. Preserve private ownership/history fences and complete artifact/provenance units. Never accept model-authored `response`.
|
|
28
|
-
- Treat every semantic `patch_state` as a strict inference barrier. Give it sequential execution mode; find the matching synchronized assistant through public parent traversal from the selected leaf without constructing a full branch during tool preflight; require exactly one `patch_state` call in that response and block every sibling tool call before execution. The next inference sees rematerialized state. Ordinary accepted answers require no finalization patch or fallback inference.
|
|
28
|
+
- Treat every semantic `patch_state` as a strict inference barrier. Give it sequential execution mode; find the matching synchronized assistant through public parent traversal from the selected leaf without constructing a full branch during tool preflight; require exactly one `patch_state` call in that response and block every sibling tool call before execution. The next inference sees rematerialized state. Successful native patch results also carry context-owned `state_updates`: exact projected effective replacements/deletions at key/index-segment path arrays, covering touched paths (including correct repeats) and all projected drift since the last communicated view, or the caller's pre-wait cache before any head exists. Keep effective scope masking/fallback, artifact metadata filtering and lazy-body omission; only changed bounded lazy navigation may accompany updates. Keep the compact interactive acknowledgement separate from this model-facing result block. Ordinary accepted answers require no finalization patch or fallback inference.
|
|
29
29
|
- Reconcile `response` only from the actually accepted ordinary assistant answer at `turn_end`; it remains runtime-owned, and an accepted empty answer reconciles to `""` rather than causing a finalization failure. Await `withPatchTransaction` before staging against the current shared head; publish the response and completed-run lifecycle together, then install caches/checkpoint before yielding without a second persistence call. Combine Pi's operation signal with response-owned cancellation: run/selection changes, Stop, shutdown and superseding answers withdraw obsolete waits, whose completion must never clear newer pending work, restore a specification, update UI or erase a Stop fence. Cancellation before acceptance leaves the run unfinished; cancellation afterward cannot undo accepted memory. State Flow has no terminal HTML-comment mutation protocol and does not parse generic service comments. Other extensions retain ownership of their own comments and output handling.
|
|
30
30
|
- Treat terminal state as a decision-relevant handoff, not narration: retain source-addressed reusable operational knowledge in `artifacts`; compile stable requirements, confirmed decisions, rejected approaches, and interface commitments into `contract`; retain observations, validation, failures, current domain state, unresolved work, interaction consequences, and exact continuation in `working`. Preserve relevant completed prerequisites and verified outcomes while removing obsolete progress narration; reconcile only information affected by the run and relevant existing commitments, not every scope or repository surface.
|
|
31
31
|
- Preserve active constraints, unresolved questions, consequential negative results, and the next discriminating check before compression. Distinguish observations, user requirements, assistant decisions, and hypotheses; do not promote assistant conclusions to user requirements. Treat resource, document, memory, Skill, and agent locators as semantic references wherever context expresses them. Inside ordinary strings and prose, encode a semantic-state path as `$` immediately followed by one valid `read_state` path, for example `$effective.lazy.memory[7]`; the prefix distinguishes a reference from incidental path-like text and leaves room for future parsing. `{"$ref":"cwd.lazy.plan"}` remains the optional structured state-reference form; neither form is a runtime link type. External resource locators retain their native path, URI, Skill, or agent syntax. Resolve references explicitly through the appropriate read/tool when needed and infer no authority, existence, dependency, hydration, execution, or completion merely from their presence. Never scan, audit, or proactively resolve references merely to find broken ones. Only after one requested `read_state` value path is missing may the query domain perform one bounded reverse lookup over current model-patchable semantic planes for exact structured `$ref` and `$path` matches. When durable sources match, return a successful top-level `{value:null, hint:[...]}` diagnostic sentinel: every hint has `type:"dangling-reference"`, a descriptive conditional message, and at most three runtime-verified current reference-owner paths, never purported relocated targets or lazy bodies. The null sentinel is never presented without `hint` for this case; keys, patch, and multi-path reads retain ordinary all-or-error semantics. Never scan runtime-owned `response`, and retain the ordinary missing-path error when no durable source matches. A match proves durable semantic provenance, not staleness; no match does not prove invention. Missing paths or runtime hints alone never require historical search. The agent may choose targeted historical reads when a previous value is useful to the current task without separate user permission; otherwise continue without searching. Treat past values as historical evidence, not automatically current memory, and never automatically restore deleted data. Do not scan offsets, hydrate automatically, add repair inference or search historical reference owners. Hints do not prove prior existence, retained history or relocation. The agent may inspect ownership and patch a proven stale source within touched work while preserving surrounding meaning, without resurrecting its target. Effective-state absence does not identify ownership, and unavailable history, inaccessible external resources, or transient read failure do not prove a dangling reference. Retain useful source locators and validity conditions for consequential facts without mandatory per-value metadata. Keep rejection reasons and reconsideration conditions. Reconcile contradictions through evidence or user clarification instead of silently overwriting established constraints or observations; retain unresolved conflicts and decision-relevant hypotheses as uncertain. These are protocol obligations, not deterministic semantic validation gates.
|
package/BACKLOG.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Backlog
|
|
2
2
|
|
|
3
|
-
No open implementation items for the 0.
|
|
3
|
+
No open implementation items for the 0.20.0 cache-stable projection scope. Outcomes are recorded in [CHANGELOG.md](CHANGELOG.md); source-bound before/after measurements and their limits are in [performance evidence](docs/performance.md).
|
|
4
4
|
|
|
5
|
-
Release publication is owned by `.github/workflows/release.yml`. Verify the exact tag's successful workflow, published GitHub Release and matching npm package before reporting completion.
|
|
5
|
+
Release publication is owned by `.github/workflows/release.yml`. Verify the exact tag's successful workflow, published GitHub Release and matching npm package before reporting release completion. Installed-instance reload remains a separate operator action.
|
|
6
6
|
|
|
7
|
-
Current contracts belong to [lazy state](docs/lazy-state.md), [usage](docs/usage.md) and the bundled Skills. [Temporal acceptance](docs/temporal-acceptance.md) records the native projection/hint witnesses and their limits.
|
|
7
|
+
Current contracts belong to [architecture](docs/architecture.md), [lazy state](docs/lazy-state.md), [usage](docs/usage.md) and the bundled Skills. [Temporal acceptance](docs/temporal-acceptance.md) records the native projection/hint witnesses and their limits. Canonical storage, CAS, `historyLimit`, history and lifecycle semantics are unchanged.
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
> Each release keeps at most 8 outcome records of at most 512 characters.
|
|
4
4
|
|
|
5
|
+
## 0.20.0: Cache-Stable Projection
|
|
6
|
+
|
|
7
|
+
- `Cache-stable model projection`: Active/bootstrap heads freeze per accepted iteration; passive heads survive ordinary user turns. Native/mode boundaries rebase without semantic changes. Patch receipts and stable-position notices deliver current values, hints and invalidations without rewriting prior prefixes. Volatile projection IDs keep retained old results historical after rebasing. Native regressions cover repeated barriers, passive continuation and bootstrap; canonical history is unchanged.
|
|
8
|
+
- `Accepted-state patch tails`: Successful active/passive patch results carry projected effective replacements and deletions for touched paths and transaction-adopted shared drift, including correct repeats and scope fallback/masking. Lazy bodies and artifact provenance stay hidden; changed bounded lazy navigation remains available. Native tests prove the next inference sees these values in active, passive and Stop-handoff modes. Interactive acknowledgement and patch-display policy stay unchanged.
|
|
9
|
+
- `Trajectory-dominant prefix evidence`: Opt-in v3 reports exercise active, passive and Stop-handoff projection with six 20 KiB native reads before a patch and two before another. Reports retain every inference's byte prefix and read volume; native content, accepted barriers and completion are checked independently of provider callbacks. Source-bound baseline measurements distinguish frozen Stop handoff from regenerated active/passive heads without claiming provider cache-hit rates.
|
|
10
|
+
- `Minimum-Node benchmark validation`: Optional `BENCH_REPORT_PATH` writes complete machine-readable reports independently of test-runner stdout chunking, while retaining human-readable stdout. Regression tests read file reports for successful and failed workloads, including 1 MiB backpressure. Release validation now covers Node 22.19.0 and Node 24; publication remains on Node 24.
|
|
11
|
+
|
|
5
12
|
## 0.19.1: Lazy Isolation and Task-Driven History
|
|
6
13
|
|
|
7
14
|
- `Lazy projection isolation`: Automatic recent transitions omit entire lazy patches, including deletions, and discard empty visible records without renumbering history. Hot changes, artifact metadata filtering and bounded lazy navigation remain available. Canonical state/history, explicit reads and already communicated user/tool/response text are unchanged.
|
package/README.md
CHANGED
|
@@ -22,7 +22,7 @@ Patched state + Answer
|
|
|
22
22
|
Updated state
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
-
Within a run, the available request, intermediate responses, tool results and steering remain in context. A state patch
|
|
25
|
+
Within a run, the available request, intermediate responses, tool results and steering remain in context. The initial memory head stays byte-stable: accepted changes arrive in patch results, and changing runtime notices append at the tail without moving earlier messages. Passive memory uses the same approach across ordinary user turns, rebasing at native or mode boundaries. Projection IDs distinguish current updates from retained historical results. A state patch neither rewrites the head nor discards the working trajectory. Persistent context-bearing messages from other extensions are preserved as well.
|
|
26
26
|
|
|
27
27
|
This reduces reliance on repeated model-generated summaries of an accumulating transcript. Retaining the current trajectory also allows prompt-cache reuse while the relevant prefix remains unchanged. Avoiding summary calls and repeated prompt processing can improve responsiveness; the result depends on the model, provider, workload and frequency of state changes, not a fixed latency guarantee.
|
|
28
28
|
|
package/dist/lib/context.d.ts
CHANGED
|
@@ -1,9 +1,10 @@
|
|
|
1
1
|
import type { AgentMessage } from "@earendil-works/pi-agent-core";
|
|
2
2
|
import { type ArtifactInvalidationNotice, type ArtifactModelHints } from "./artifact.ts";
|
|
3
3
|
import type { RecentTransitionWindow } from "./history.ts";
|
|
4
|
+
import { type JsonValue } from "./json.ts";
|
|
4
5
|
import type { Snapshot } from "./snapshot.ts";
|
|
5
6
|
import type { RehydrationPhase } from "./rehydration.ts";
|
|
6
|
-
import { type MaterializedState, type ModelState } from "./state.ts";
|
|
7
|
+
import { type AtomicScopePatches, type MaterializedState, type ModelState } from "./state.ts";
|
|
7
8
|
/** Refresh only our section; Pi owns system frames, tools and forced-prompt precedence. */
|
|
8
9
|
export declare function projectSystemProtocol(messages: AgentMessage[], protocol: string | undefined): AgentMessage[];
|
|
9
10
|
type LazyValueKind = "array" | "boolean" | "null" | "number" | "object" | "string";
|
|
@@ -13,18 +14,66 @@ export declare function lazyNavigationHint(state: MaterializedState): {
|
|
|
13
14
|
path: string;
|
|
14
15
|
keys?: Record<string, LazyValueKind>;
|
|
15
16
|
};
|
|
17
|
+
export type ModelStateUpdate = {
|
|
18
|
+
path: (string | number)[];
|
|
19
|
+
} & ({
|
|
20
|
+
value: JsonValue;
|
|
21
|
+
} | {
|
|
22
|
+
deleted: true;
|
|
23
|
+
});
|
|
24
|
+
/** Exact projected replacements, not authored merge patches; paths are unambiguous key/index segments. */
|
|
25
|
+
export declare function acceptedStateUpdates(before: MaterializedState, after: MaterializedState, patches: AtomicScopePatches): {
|
|
26
|
+
effective: ModelStateUpdate[];
|
|
27
|
+
lazy_navigation?: {
|
|
28
|
+
available: boolean;
|
|
29
|
+
path: string;
|
|
30
|
+
keys?: Record<string, LazyValueKind>;
|
|
31
|
+
} | undefined;
|
|
32
|
+
};
|
|
33
|
+
export interface ContextView {
|
|
34
|
+
state: ModelState;
|
|
35
|
+
lazy_navigation?: ReturnType<typeof lazyNavigationHint>;
|
|
36
|
+
artifact_invalidations: readonly ArtifactInvalidationNotice[];
|
|
37
|
+
knowledge_rehydration: {
|
|
38
|
+
phase: RehydrationPhase;
|
|
39
|
+
} | null;
|
|
40
|
+
}
|
|
41
|
+
export declare function contextView(state: MaterializedState, hints: ArtifactModelHints, invalidations: readonly ArtifactInvalidationNotice[], phase?: RehydrationPhase): ContextView;
|
|
42
|
+
/** Volatile model projection only. Native messages own trajectory; this cache owns no persistence or lifecycle. */
|
|
43
|
+
export declare class ContextProjection {
|
|
44
|
+
private identity;
|
|
45
|
+
private head;
|
|
46
|
+
private view;
|
|
47
|
+
private native;
|
|
48
|
+
private notices;
|
|
49
|
+
reset(): void;
|
|
50
|
+
/** Called only after successful publication and ancillary acceptance, immediately before returning the native result. */
|
|
51
|
+
acceptPatch(before: MaterializedState, after: MaterializedState, patches: AtomicScopePatches, hints: ArtifactModelHints): {
|
|
52
|
+
effective: ModelStateUpdate[];
|
|
53
|
+
lazy_navigation?: {
|
|
54
|
+
available: boolean;
|
|
55
|
+
path: string;
|
|
56
|
+
keys?: Record<string, LazyValueKind>;
|
|
57
|
+
} | undefined;
|
|
58
|
+
projection: `${string}-${string}-${string}-${string}-${string}`;
|
|
59
|
+
};
|
|
60
|
+
project(messages: AgentMessage[], current: ContextView, makeHead: () => AgentMessage, initial?: ContextView): AgentMessage[];
|
|
61
|
+
}
|
|
16
62
|
/** Context retained after semantic State Flow is stopped in this physical session. */
|
|
17
63
|
export interface PassiveContinuation {
|
|
18
64
|
startedAt: number;
|
|
19
65
|
activeRunStartedAt?: number;
|
|
20
66
|
preserveContext?: true;
|
|
21
67
|
handoff: AgentMessage;
|
|
68
|
+
state: ModelState;
|
|
22
69
|
}
|
|
23
70
|
export declare function syntheticUser(text: string): AgentMessage;
|
|
24
71
|
export declare function createPassiveContinuation(state: ModelState, startedAt?: number, activeRunStartedAt?: number, preserveContext?: boolean): PassiveContinuation;
|
|
25
72
|
/** Keep the interrupted run through later results; an idle stop retains only later conversation. */
|
|
26
73
|
export declare function passiveContinuationMessages(messages: AgentMessage[], continuation: PassiveContinuation): AgentMessage[];
|
|
27
74
|
export declare function runtimeContextMessage(snapshot: Snapshot, state: MaterializedState, recentTransitions?: RecentTransitionWindow, artifactInvalidations?: readonly ArtifactInvalidationNotice[], rehydrationPhase?: RehydrationPhase, artifactHints?: ArtifactModelHints): AgentMessage;
|
|
75
|
+
/** Render a view already projected by this domain without cloning the full semantic overlay twice. */
|
|
76
|
+
export declare function runtimeContextHead(snapshot: Snapshot, view: ContextView, recentTransitions?: RecentTransitionWindow): AgentMessage;
|
|
28
77
|
/** Captured identity survives text decoration; an uncertain boundary retains available context. */
|
|
29
78
|
export declare function currentRunTrajectory(messages: AgentMessage[], specification: string | undefined, anchorTimestamp: number | undefined): {
|
|
30
79
|
messages: AgentMessage[];
|
package/dist/lib/context.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
|
+
import { randomUUID } from "node:crypto";
|
|
1
2
|
import { projectArtifactForModel } from "./artifact.js";
|
|
2
|
-
import { isObject, presentationJson } from "./json.js";
|
|
3
|
+
import { isObject, presentationJson, sameJson } from "./json.js";
|
|
3
4
|
import { projectStateForModel } from "./state.js";
|
|
4
5
|
/** Refresh only our section; Pi owns system frames, tools and forced-prompt precedence. */
|
|
5
6
|
export function projectSystemProtocol(messages, protocol) {
|
|
@@ -52,6 +53,141 @@ export function lazyNavigationHint(state) {
|
|
|
52
53
|
const keys = Object.fromEntries(entries.map(([key, value]) => [key, lazyValueKind(value)]));
|
|
53
54
|
return JSON.stringify(keys).length <= LAZY_HINT_MAX_JSON_CHARS ? { ...base, keys } : base;
|
|
54
55
|
}
|
|
56
|
+
/** Exact projected replacements, not authored merge patches; paths are unambiguous key/index segments. */
|
|
57
|
+
export function acceptedStateUpdates(before, after, patches) {
|
|
58
|
+
return projectedStateUpdates(projectStateForModel(before), projectStateForModel(after), patches, lazyNavigationHint(before), lazyNavigationHint(after));
|
|
59
|
+
}
|
|
60
|
+
function projectedStateUpdates(previous, current, patches, beforeNavigation, navigation) {
|
|
61
|
+
const effective = [];
|
|
62
|
+
const prefix = (parent, child) => parent.length <= child.length && parent.every((part, index) => part === child[index]);
|
|
63
|
+
const put = (path, value) => {
|
|
64
|
+
if (effective.some((entry) => prefix(entry.path, path)))
|
|
65
|
+
return;
|
|
66
|
+
for (let index = effective.length - 1; index >= 0; index--) {
|
|
67
|
+
if (prefix(path, effective[index].path))
|
|
68
|
+
effective.splice(index, 1);
|
|
69
|
+
}
|
|
70
|
+
effective.push({ path, ...(value === undefined ? { deleted: true } : { value: structuredClone(value) }) });
|
|
71
|
+
};
|
|
72
|
+
const child = (value, key) => value !== null && typeof value === "object" && Object.hasOwn(value, key)
|
|
73
|
+
? value[key] : undefined;
|
|
74
|
+
const diff = (left, right, path) => {
|
|
75
|
+
if (left === undefined && right === undefined || left !== undefined && right !== undefined && sameJson(left, right))
|
|
76
|
+
return;
|
|
77
|
+
if (isObject(left) && isObject(right)) {
|
|
78
|
+
for (const key of new Set([...Object.keys(left), ...Object.keys(right)]))
|
|
79
|
+
diff(child(left, key), child(right, key), [...path, key]);
|
|
80
|
+
}
|
|
81
|
+
else if (Array.isArray(left) && Array.isArray(right) && left.length === right.length) {
|
|
82
|
+
for (let index = 0; index < right.length; index++)
|
|
83
|
+
diff(left[index], right[index], [...path, index]);
|
|
84
|
+
}
|
|
85
|
+
else
|
|
86
|
+
put(path, right);
|
|
87
|
+
};
|
|
88
|
+
const touched = (patch, value, path) => {
|
|
89
|
+
if (!isObject(patch)) {
|
|
90
|
+
put(path, value);
|
|
91
|
+
return;
|
|
92
|
+
}
|
|
93
|
+
const keys = Object.keys(patch);
|
|
94
|
+
if (keys.length === 0)
|
|
95
|
+
return;
|
|
96
|
+
if (Array.isArray(value) && keys.every((key) => /^\[(0|[1-9]\d*)\]$/.test(key))) {
|
|
97
|
+
for (const key of keys) {
|
|
98
|
+
const index = Number(key.slice(1, -1));
|
|
99
|
+
// A higher-scope array may mask the patched array with a different length.
|
|
100
|
+
if (index >= value.length) {
|
|
101
|
+
put(path, value);
|
|
102
|
+
return;
|
|
103
|
+
}
|
|
104
|
+
touched(patch[key], value[index], [...path, index]);
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
else if (isObject(value)) {
|
|
108
|
+
for (const key of keys)
|
|
109
|
+
touched(patch[key], child(value, key), [...path, key]);
|
|
110
|
+
}
|
|
111
|
+
else
|
|
112
|
+
put(path, value);
|
|
113
|
+
};
|
|
114
|
+
diff(previous, current, []);
|
|
115
|
+
for (const patch of Object.values(patches))
|
|
116
|
+
for (const [plane, value] of Object.entries(patch)) {
|
|
117
|
+
if (plane === "lazy")
|
|
118
|
+
continue;
|
|
119
|
+
if (plane === "artifacts" && isObject(value)) {
|
|
120
|
+
for (const path of Object.keys(value))
|
|
121
|
+
put([plane, path], child(current.artifacts, path));
|
|
122
|
+
}
|
|
123
|
+
else
|
|
124
|
+
touched(value, child(current, plane), [plane]);
|
|
125
|
+
}
|
|
126
|
+
return { effective, ...(navigation !== undefined && (beforeNavigation === undefined || !sameJson(beforeNavigation, navigation)) ? { lazy_navigation: navigation } : {}) };
|
|
127
|
+
}
|
|
128
|
+
export function contextView(state, hints, invalidations, phase) {
|
|
129
|
+
return { state: projectStateForModel(state, hints), lazy_navigation: lazyNavigationHint(state),
|
|
130
|
+
artifact_invalidations: structuredClone(invalidations), knowledge_rehydration: phase === undefined ? null : { phase } };
|
|
131
|
+
}
|
|
132
|
+
/** Volatile model projection only. Native messages own trajectory; this cache owns no persistence or lifecycle. */
|
|
133
|
+
export class ContextProjection {
|
|
134
|
+
identity = randomUUID();
|
|
135
|
+
head;
|
|
136
|
+
view;
|
|
137
|
+
native = [];
|
|
138
|
+
notices = [];
|
|
139
|
+
reset() {
|
|
140
|
+
this.identity = randomUUID();
|
|
141
|
+
this.head = undefined;
|
|
142
|
+
this.view = undefined;
|
|
143
|
+
this.native = [];
|
|
144
|
+
this.notices = [];
|
|
145
|
+
}
|
|
146
|
+
/** Called only after successful publication and ancillary acceptance, immediately before returning the native result. */
|
|
147
|
+
acceptPatch(before, after, patches, hints) {
|
|
148
|
+
const state = projectStateForModel(after, hints);
|
|
149
|
+
const navigation = lazyNavigationHint(after);
|
|
150
|
+
const updates = projectedStateUpdates(this.view?.state ?? projectStateForModel(before, hints), state, patches, this.view?.lazy_navigation ?? lazyNavigationHint(before), navigation);
|
|
151
|
+
if (this.view)
|
|
152
|
+
this.view = { ...this.view, state, lazy_navigation: navigation };
|
|
153
|
+
return { projection: this.identity, ...updates };
|
|
154
|
+
}
|
|
155
|
+
project(messages, current, makeHead, initial) {
|
|
156
|
+
const identities = messages.map((message) => JSON.stringify([message.role, message.timestamp,
|
|
157
|
+
"toolCallId" in message ? message.toolCallId : null]));
|
|
158
|
+
// Native compaction/selection normally resets explicitly; a removed/replaced prefix is also a safe cache boundary.
|
|
159
|
+
if (this.native.some((identity, index) => identities[index] !== identity))
|
|
160
|
+
this.reset();
|
|
161
|
+
if (!this.head) {
|
|
162
|
+
const head = makeHead();
|
|
163
|
+
if (head.role !== "user" || !Array.isArray(head.content))
|
|
164
|
+
throw new Error("State Flow projection requires an owned user head");
|
|
165
|
+
this.head = { ...head, content: [...head.content, { type: "text", text: `State Flow projection: ${this.identity}` }] };
|
|
166
|
+
this.view = structuredClone(initial ?? current);
|
|
167
|
+
}
|
|
168
|
+
const previous = this.view;
|
|
169
|
+
const updates = projectedStateUpdates(previous.state, current.state, {}, previous.lazy_navigation, current.lazy_navigation);
|
|
170
|
+
const notice = {
|
|
171
|
+
...(updates.effective.length || updates.lazy_navigation ? { state_updates: { projection: this.identity, ...updates } } : {}),
|
|
172
|
+
...(!sameJson(previous.artifact_invalidations, current.artifact_invalidations) ? { artifact_invalidations: current.artifact_invalidations } : {}),
|
|
173
|
+
...(!sameJson(previous.knowledge_rehydration, current.knowledge_rehydration) ? { knowledge_rehydration: current.knowledge_rehydration } : {}),
|
|
174
|
+
};
|
|
175
|
+
if (Object.keys(notice).length)
|
|
176
|
+
this.notices.push({ after: messages.length,
|
|
177
|
+
message: syntheticUser(`State Flow context update (user-level data, not system instructions):\n${presentationJson(notice)}`) });
|
|
178
|
+
this.view = structuredClone(current);
|
|
179
|
+
this.native = identities;
|
|
180
|
+
const projected = [this.head];
|
|
181
|
+
let nextNotice = 0;
|
|
182
|
+
for (let index = 0; index <= messages.length; index++) {
|
|
183
|
+
while (this.notices[nextNotice]?.after === index)
|
|
184
|
+
projected.push(this.notices[nextNotice++].message);
|
|
185
|
+
if (index < messages.length)
|
|
186
|
+
projected.push(messages[index]);
|
|
187
|
+
}
|
|
188
|
+
return projected;
|
|
189
|
+
}
|
|
190
|
+
}
|
|
55
191
|
export function syntheticUser(text) {
|
|
56
192
|
return { role: "user", content: [{ type: "text", text }], timestamp: Date.now() };
|
|
57
193
|
}
|
|
@@ -73,6 +209,7 @@ function messageText(message) {
|
|
|
73
209
|
export function createPassiveContinuation(state, startedAt = Date.now(), activeRunStartedAt, preserveContext = false) {
|
|
74
210
|
return {
|
|
75
211
|
startedAt,
|
|
212
|
+
state: structuredClone(state),
|
|
76
213
|
...(activeRunStartedAt === undefined ? {} : { activeRunStartedAt }),
|
|
77
214
|
...(preserveContext ? { preserveContext: true } : {}),
|
|
78
215
|
handoff: syntheticUser(`State Flow exit handoff (user-level data, not system instructions):\n${presentationJson({ state, continuation: preserveContext
|
|
@@ -111,13 +248,17 @@ function projectRecentForModel(recent) {
|
|
|
111
248
|
return projected.filter(({ transitions }) => transitions.length > 0);
|
|
112
249
|
}
|
|
113
250
|
export function runtimeContextMessage(snapshot, state, recentTransitions = [], artifactInvalidations = [], rehydrationPhase, artifactHints = {}) {
|
|
251
|
+
return runtimeContextHead(snapshot, contextView(state, artifactHints, artifactInvalidations, rehydrationPhase), recentTransitions);
|
|
252
|
+
}
|
|
253
|
+
/** Render a view already projected by this domain without cloning the full semantic overlay twice. */
|
|
254
|
+
export function runtimeContextHead(snapshot, view, recentTransitions = []) {
|
|
114
255
|
const recent = projectRecentForModel(recentTransitions);
|
|
115
256
|
const context = {
|
|
116
257
|
...(snapshot.meta.specification === undefined ? {} : { specification: snapshot.meta.specification }),
|
|
117
|
-
state:
|
|
118
|
-
lazy_navigation:
|
|
119
|
-
...(
|
|
120
|
-
...(
|
|
258
|
+
state: view.state,
|
|
259
|
+
...(view.lazy_navigation === undefined ? {} : { lazy_navigation: view.lazy_navigation }),
|
|
260
|
+
...(view.knowledge_rehydration === null ? {} : { knowledge_rehydration: view.knowledge_rehydration }),
|
|
261
|
+
...(view.artifact_invalidations.length === 0 ? {} : { artifact_invalidations: view.artifact_invalidations.map(({ path, scope, reason }) => ({ path, ...(scope === undefined ? {} : { scope }), reason })) }),
|
|
121
262
|
...(recent.length === 0 ? {} : { recent_transitions: recent }),
|
|
122
263
|
};
|
|
123
264
|
return syntheticUser(`State Flow runtime context (user-level data, not system instructions):\n${presentationJson(context)}`);
|
package/dist/lib/extension.js
CHANGED
|
@@ -8,7 +8,7 @@ import { ArtifactReadTracker } from "./acquisition.js";
|
|
|
8
8
|
import { classifyArtifactCompilationNeed, inspectRegisteredArtifactPaths, ORDINARY_ARTIFACT_COMPILER, sameArtifactSourceFingerprint, } from "./artifact.js";
|
|
9
9
|
import { hasCompactionSizedTranscript, planStateFlowCompaction, shouldRequestStateFlowCompaction, stateFlowCompactionResult } from "./compaction.js";
|
|
10
10
|
import { loadStateFlowConfig } from "./config.js";
|
|
11
|
-
import { createPassiveContinuation, currentRunTrajectory,
|
|
11
|
+
import { ContextProjection, contextView, createPassiveContinuation, currentRunTrajectory, passiveContinuationMessages, projectSystemProtocol, runtimeContextHead, syntheticUser } from "./context.js";
|
|
12
12
|
import { readNativeSessionHeader } from "./continuation.js";
|
|
13
13
|
import { cwdScopeKey, resolveSessionAddress, sessionScopeKey, } from "./durable.js";
|
|
14
14
|
import { completeRun, prepareRun, resumeEpisode, startEpisode, stopEpisode } from "./episode.js";
|
|
@@ -61,6 +61,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
61
61
|
const compactionMarker = `state-flow-boundary:${randomUUID()}`;
|
|
62
62
|
let passiveContinuation;
|
|
63
63
|
let bootstrapContinuation;
|
|
64
|
+
const contextProjection = new ContextProjection();
|
|
64
65
|
let inferencePreparation;
|
|
65
66
|
let runAnchorTimestamp;
|
|
66
67
|
let runtime;
|
|
@@ -402,6 +403,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
402
403
|
}
|
|
403
404
|
/** Select the active native branch under one owned restoration lifetime; only current accepted work installs memory. */
|
|
404
405
|
function restoreActiveBranch(ctx, sessionStartReason, notifyRecovery = true, startOwner) {
|
|
406
|
+
contextProjection.reset();
|
|
405
407
|
cancelBranchRestoration();
|
|
406
408
|
// Start-owned attachment/fork recovery keeps its owner; only accepted Start cancels Stop.
|
|
407
409
|
if (!startOwner) {
|
|
@@ -760,6 +762,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
760
762
|
const selected = runtime ??= createRuntime(ctx);
|
|
761
763
|
const acquiredArtifacts = structuredClone([...artifactReads.successful.values()]);
|
|
762
764
|
const acquiredSkills = structuredClone([...skillReads.successful.values()]);
|
|
765
|
+
const previousEffective = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
763
766
|
return await selected.withPatchTransaction((transaction) => {
|
|
764
767
|
if (runtime !== selected)
|
|
765
768
|
throw new Error("State Flow session selection changed while awaiting publication");
|
|
@@ -789,9 +792,14 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
789
792
|
}
|
|
790
793
|
clearAcceptedAcquisitions(new Set(acquiredArtifacts.map(({ path }) => path)));
|
|
791
794
|
updateUi(ctx);
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
+
const updates = contextProjection.acceptPatch(previousEffective, overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session), patches, artifactHints);
|
|
796
|
+
const acknowledgement = changed
|
|
797
|
+
? `\nState materialized atomically at ${scopes.join("+")} scope${scopes.length === 1 ? "" : "s"}.`
|
|
798
|
+
: "\nState already current.";
|
|
799
|
+
return { content: [
|
|
800
|
+
{ type: "text", text: acknowledgement },
|
|
801
|
+
{ type: "text", text: `\n${presentationJson({ state_updates: updates })}` },
|
|
802
|
+
], details: { scopes, step: snapshot.meta.step, changed } };
|
|
795
803
|
}, signal);
|
|
796
804
|
}
|
|
797
805
|
catch (error) {
|
|
@@ -914,6 +922,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
914
922
|
stopPersistenceError = undefined;
|
|
915
923
|
installScopeStates();
|
|
916
924
|
clearRunTransient();
|
|
925
|
+
contextProjection.reset();
|
|
917
926
|
passiveContinuation = undefined;
|
|
918
927
|
bootstrapContinuation = snapshot.meta.bootstrap ? continuation : undefined;
|
|
919
928
|
deferInferencePreparation();
|
|
@@ -999,6 +1008,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
999
1008
|
&& !shuttingDown && ctx.sessionManager.getSessionId() === owner && !snapshot.config.enabled;
|
|
1000
1009
|
const superseded = () => ({ ok: false, message: "State Flow Stop was superseded" });
|
|
1001
1010
|
const freezeHandoff = () => {
|
|
1011
|
+
contextProjection.reset();
|
|
1002
1012
|
const handoff = (current.config.enabled || stopPersistenceError) && selected?.view && current.meta.validation?.attempt !== 0
|
|
1003
1013
|
? createPassiveContinuation(projectModelState(overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session)), incomingBoundary?.startedAt ?? stoppedAt, incomingBoundary ? incomingBoundary.activeRunStartedAt : !idle || unfinished ? anchor : undefined, stopPersistenceError !== undefined || (incomingBoundary ? incomingBoundary.preserveContext : current.meta.bootstrap === true || (unfinished && anchor === undefined)))
|
|
1004
1014
|
: undefined;
|
|
@@ -1119,6 +1129,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
1119
1129
|
(event.systemPromptOptions.sections ??= {}).state_flow = PASSIVE_MEMORY_PROTOCOL;
|
|
1120
1130
|
return;
|
|
1121
1131
|
}
|
|
1132
|
+
contextProjection.reset();
|
|
1122
1133
|
skillReads.clear();
|
|
1123
1134
|
artifactReads.clear();
|
|
1124
1135
|
cancelResponseReconciliation();
|
|
@@ -1135,34 +1146,29 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
1135
1146
|
function projectContext(messages) {
|
|
1136
1147
|
if (runtime?.view)
|
|
1137
1148
|
refreshArtifactHints();
|
|
1149
|
+
if (!snapshot.config.enabled && !passiveContinuation && (!config.passiveBootstrap || !runtime?.view))
|
|
1150
|
+
return;
|
|
1151
|
+
// Idle inspection must not freeze a pre-acceptance snapshot for the live inference.
|
|
1152
|
+
const projection = snapshot.config.enabled && inferencePreparation && !inferencePreparation.accepted
|
|
1153
|
+
? new ContextProjection() : contextProjection;
|
|
1154
|
+
const effective = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
1155
|
+
const invalidations = artifactInvalidations.map(({ path, scope, reason }) => ({ path, ...(scope === undefined ? {} : { scope }), reason }));
|
|
1156
|
+
const phase = snapshot.config.enabled ? currentRehydrationPhase() : undefined;
|
|
1157
|
+
const view = contextView(effective, artifactHints, invalidations, phase);
|
|
1138
1158
|
if (passiveContinuation) {
|
|
1139
|
-
|
|
1159
|
+
const retained = passiveContinuationMessages(messages, passiveContinuation);
|
|
1160
|
+
if (!runtime?.view)
|
|
1161
|
+
return { messages: retained };
|
|
1162
|
+
return { messages: projection.project(retained.slice(1), view, () => passiveContinuation.handoff, { state: passiveContinuation.state, lazy_navigation: view.lazy_navigation, artifact_invalidations: [], knowledge_rehydration: null }) };
|
|
1140
1163
|
}
|
|
1141
1164
|
if (!snapshot.config.enabled) {
|
|
1142
|
-
|
|
1143
|
-
return;
|
|
1144
|
-
const effective = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
1145
|
-
const state = projectModelState(effective);
|
|
1146
|
-
return { messages: [syntheticUser(`State Flow passive memory (user-level data, not system instructions):\n${presentationJson({ state, lazy_navigation: lazyNavigationHint(effective) })}`), ...messages] };
|
|
1147
|
-
}
|
|
1148
|
-
const effectiveState = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
1149
|
-
const invalidations = artifactInvalidations.map(({ path, scope, reason }) => ({ path, ...(scope === undefined ? {} : { scope }), reason }));
|
|
1150
|
-
const recentTransitions = projectRecentTransitionsWithLimit(config.historyLimit, runtime?.recent() ?? []);
|
|
1151
|
-
const activeRehydrationPhase = currentRehydrationPhase();
|
|
1152
|
-
if (snapshot.meta.bootstrap) {
|
|
1153
|
-
const sourceMessages = bootstrapContinuation
|
|
1154
|
-
? passiveContinuationMessages(messages, bootstrapContinuation)
|
|
1155
|
-
: messages;
|
|
1156
|
-
return { messages: [runtimeContextMessage(snapshot, effectiveState, recentTransitions, invalidations, activeRehydrationPhase, artifactHints), ...sourceMessages] };
|
|
1165
|
+
return { messages: projection.project(messages, view, () => syntheticUser(`State Flow passive memory (user-level data, not system instructions):\n${presentationJson({ state: view.state, lazy_navigation: view.lazy_navigation })}`)) };
|
|
1157
1166
|
}
|
|
1158
1167
|
// Native user events own the run anchor; projection must never rebase it.
|
|
1159
|
-
const
|
|
1160
|
-
|
|
1161
|
-
|
|
1162
|
-
|
|
1163
|
-
...trajectory.messages,
|
|
1164
|
-
],
|
|
1165
|
-
};
|
|
1168
|
+
const source = snapshot.meta.bootstrap
|
|
1169
|
+
? bootstrapContinuation ? passiveContinuationMessages(messages, bootstrapContinuation) : messages
|
|
1170
|
+
: currentRunTrajectory(messages, snapshot.meta.specification, runAnchorTimestamp).messages;
|
|
1171
|
+
return { messages: projection.project(source, view, () => runtimeContextHead(snapshot, view, projectRecentTransitionsWithLimit(config.historyLimit, runtime?.recent() ?? []))) };
|
|
1166
1172
|
}
|
|
1167
1173
|
function prepareContext(messages, ctx) {
|
|
1168
1174
|
const pending = inferencePreparation;
|
|
@@ -1290,6 +1296,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
1290
1296
|
snapshot = nextSnapshot;
|
|
1291
1297
|
installScopeStates();
|
|
1292
1298
|
responseCommitted = true;
|
|
1299
|
+
contextProjection.reset();
|
|
1293
1300
|
if (publication?.changed)
|
|
1294
1301
|
recordPublication(publication, ctx);
|
|
1295
1302
|
appendCheckpoint();
|
|
@@ -1400,6 +1407,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
1400
1407
|
ctx.compact({ customInstructions: compactionMarker, onComplete: finished, onError: finished });
|
|
1401
1408
|
});
|
|
1402
1409
|
});
|
|
1410
|
+
pi.on("session_compact", () => { contextProjection.reset(); });
|
|
1403
1411
|
pi.on("session_start", async (event, ctx) => {
|
|
1404
1412
|
runAnchorTimestamp = undefined;
|
|
1405
1413
|
rehydrationPhase = event.reason === "resume" ? "resume-bootstrap" : "new-bootstrap";
|
|
@@ -1413,6 +1421,7 @@ export default function stateFlowExtension(pi, options = {}) {
|
|
|
1413
1421
|
});
|
|
1414
1422
|
pi.on("session_shutdown", async (_event, _ctx) => {
|
|
1415
1423
|
shuttingDown = true;
|
|
1424
|
+
contextProjection.reset();
|
|
1416
1425
|
const restoring = cancelBranchRestoration();
|
|
1417
1426
|
const starting = cancelStartActivation();
|
|
1418
1427
|
const stopping = cancelStopPersistence();
|
package/dist/lib/protocol.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import type { AgentMessage } from "@earendil-works/pi-agent-core";
|
|
2
2
|
export type { StateDocument } from "./state.ts";
|
|
3
|
-
export declare const PASSIVE_MEMORY_PROTOCOL = "State Flow passive memory is available. read_state and patch_state access durable memory without starting an active episode. Passive turns never trigger State Flow continuation or compaction. Missing paths/hints do not require history search. Choose targeted historical reads when useful to the task; no separate user permission is needed. Past values are evidence, not current state; never automatically restore deleted memory.";
|
|
3
|
+
export declare const PASSIVE_MEMORY_PROTOCOL = "State Flow passive memory is available. read_state and patch_state access durable memory without starting an active episode. Passive turns never trigger State Flow continuation or compaction. Missing paths/hints do not require history search. Choose targeted historical reads when useful to the task; no separate user permission is needed. Past values are evidence, not current state; never automatically restore deleted memory. Head state/recent transitions are frozen at projection start. Only state_updates matching the head's State Flow projection ID apply; older IDs are history. Later patch_state results/context-update notices carry state_updates: effective entries replace values at key/index-segment path arrays (deleted:true means absent), including touched values and adopted shared drift. Latest entries win over earlier state at those paths; lazy bodies remain omitted. Notices replace invalidations/rehydration, including []/null.";
|
|
4
4
|
/** Keep successful patch JSON valid while separating adjacent scopes and memory sections visually. */
|
|
5
5
|
export declare function formatPatchStateArguments(args: unknown): string;
|
|
6
6
|
/** Flatten causes before transport; native tool results need not retain Error.cause or AggregateError.errors. */
|
package/dist/lib/protocol.js
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
const TASK_DRIVEN_HISTORY = "Missing paths/hints do not require history search. Choose targeted historical reads when useful to the task; no separate user permission is needed. Past values are evidence, not current state; never automatically restore deleted memory.";
|
|
2
|
-
|
|
2
|
+
const PATCH_RESULT_PROTOCOL = "Head state/recent transitions are frozen at projection start. Only state_updates matching the head's State Flow projection ID apply; older IDs are history. Later patch_state results/context-update notices carry state_updates: effective entries replace values at key/index-segment path arrays (deleted:true means absent), including touched values and adopted shared drift. Latest entries win over earlier state at those paths; lazy bodies remain omitted. Notices replace invalidations/rehydration, including []/null.";
|
|
3
|
+
export const PASSIVE_MEMORY_PROTOCOL = `State Flow passive memory is available. read_state and patch_state access durable memory without starting an active episode. Passive turns never trigger State Flow continuation or compaction. ${TASK_DRIVEN_HISTORY} ${PATCH_RESULT_PROTOCOL}`;
|
|
3
4
|
const PATCH_DISPLAY_SECTION_KEYS = new Set(["global", "cwd", "session", "intents", "contract", "working", "artifacts", "response", "lazy"]);
|
|
4
5
|
/** Keep successful patch JSON valid while separating adjacent scopes and memory sections visually. */
|
|
5
6
|
export function formatPatchStateArguments(args) {
|
|
@@ -94,7 +95,7 @@ SCOPES: Use the narrowest scope: session=branch/run continuation by default; cwd
|
|
|
94
95
|
|
|
95
96
|
READ: Use read_state for concrete scope/retained-history gaps. lazy_navigation lists bounded effective lazy keys, not bodies. Unscoped=effective; effective/global/cwd/session select overlay or owner. Arrays use indices or [start..end]; keys gives structure, patch the intersected change. ${TASK_DRIVEN_HISTORY}
|
|
96
97
|
|
|
97
|
-
WRITE: patch_state is the sole model-authored semantic mutation mechanism; all supplied scopes are validated and durably accepted as one atomic transition. Call alone in an assistant response; await acceptance. Global/CWD use current canonical values after cancelable lock waiting. Correct repeats succeed without new revisions.
|
|
98
|
+
WRITE: patch_state is the sole model-authored semantic mutation mechanism; all supplied scopes are validated and durably accepted as one atomic transition. Call alone in an assistant response; await acceptance. Global/CWD use current canonical values after cancelable lock waiting. Correct repeats succeed without new revisions. ${PATCH_RESULT_PROTOCOL}
|
|
98
99
|
|
|
99
100
|
PATCH: Use global/cwd/session object patches for material updates, not acknowledgments. Omit empty scopes. artifacts/contract/working/intents are objects; lazy is ordinary JSON. Omitted fields persist. Never patch runtime config/meta/response. Objects merge recursively; arrays/primitives replace. An object containing only canonical "[N]" keys recursively patches array elements. Indexed deletion is forbidden; nested object null deletes; materialized null is forbidden.
|
|
100
101
|
|
package/dist/package.json
CHANGED
|
@@ -32,7 +32,9 @@ Scopes overlay `global → cwd → session`: cross-project, project, branch/run.
|
|
|
32
32
|
|
|
33
33
|
## Read
|
|
34
34
|
|
|
35
|
-
Reuse sufficient visible state. `
|
|
35
|
+
Reuse sufficient visible state. The memory head and its recent transitions are frozen at projection start. Apply later `state_updates` only when their `projection` matches the head's `State Flow projection:` ID; older native results remain historical. Effective update paths are key/index arrays whose values replace that path, while `deleted: true` means absence. Current notices can replace invalidation lists or rehydration phase, including clearing them with `[]` or `null`; lazy bodies still require explicit reads.
|
|
36
|
+
|
|
37
|
+
`read_state` accepts `path` or `paths`, never both. Multi-path reads succeed or fail together. Projections: `value` (default), `keys` (structure), `patch` (intersecting change at the selected boundary).
|
|
36
38
|
|
|
37
39
|
Example arguments:
|
|
38
40
|
|
package/docs/architecture.md
CHANGED
|
@@ -73,7 +73,11 @@ A changed accepted response is runtime-owned, session-only semantic state and ad
|
|
|
73
73
|
|
|
74
74
|
## Pi lifecycle
|
|
75
75
|
|
|
76
|
-
`patch_state` is the sole mutation tool. It acquires store exclusion before selecting current Global/CWD, stages the authored operations while preserving private Session ownership, and validates/publishes one atomic cohort. Replay, history and revisions use that actual basis; correct repeats create no semantic transition. It then acts as an inference barrier. Pi executes no sibling tools from the same assistant response; the next inference sees the rematerialized current effective state.
|
|
76
|
+
`patch_state` is the sole mutation tool. It acquires store exclusion before selecting current Global/CWD, stages the authored operations while preserving private Session ownership, and validates/publishes one atomic cohort. Replay, history and revisions use that actual basis; correct repeats create no semantic transition. It then acts as an inference barrier. Pi executes no sibling tools from the same assistant response; the next inference sees the rematerialized current effective state. Its successful native tool result includes a `state_updates` block, separate from the compact acknowledgement used by interactive rendering. `effective` entries contain a `path` array of exact object keys/array indices plus either a replacing `value` or `deleted: true`. These are accepted effective values, not authored merge operations: scope-local deletion can reveal a lower value, and higher scopes can mask an accepted lower-scope write. Entries cover touched paths (also on correct repeats) and projected changes since the last communicated view, including shared refreshes before or during the transaction; unrelated unchanged branches are omitted. Arrays with changed length are replaced as a unit, while equal-length changes may target indices. Artifact cards use the normal metadata filter and touched cards replace their whole projected entry. Lazy bodies never enter this block; changed bounded `lazy_navigation` may accompany it. Each update carries a volatile `projection` ID matching the head's `State Flow projection:` text block. The protocol applies only matching updates: retained native results with older IDs remain historical and cannot override a new head. Within that projection, the latest entry supersedes earlier context only at its path. The context domain owns this projection, not storage or the lifecycle composition root.
|
|
77
|
+
|
|
78
|
+
`ContextProjection` freezes the whole initial head, including its timestamp, specification and recent transitions. Signal-less inspection before active preparation accepts uses a disposable projection, so it cannot freeze a stale specification or pre-maintenance state for the live inference. Active/bootstrap runs rebase on new input and accepted completion; a native continuation after completion gets a fresh head without resurrecting the completed specification. Passive runs keep their head across ordinary user turns. Both modes rebase on Start/Stop, selection/resume/reload, native compaction and a removed/replaced native prefix. No per-patch or size-threshold rebase exists. The volatile ID is not a semantic transition identity and enters neither canonical files nor runtime metadata.
|
|
79
|
+
|
|
80
|
+
Unreported state changes, changing artifact hints, invalidation lists and rehydration phase append as synthetic tail notices at the native-message boundary where they first appeared. Later inference retains those exact messages at those positions before new native messages, rather than moving them to the latest tail. Empty invalidation lists and a null rehydration envelope explicitly clear earlier notices. Accepted patch receipts advance the communicated view; their state deltas are not redundantly emitted as synthetic notices. Stop handoff uses its captured state as the initial basis, including changes before its first projection. Skill acquisition guidance remains attached to native read results. The cache owns no persistence, publication authority or continuation scheduler, and does not redact native history.
|
|
77
81
|
|
|
78
82
|
Tool preflight follows Pi's public `getLeafEntry()` / `getEntry(parentId)` links to the nearest assistant containing the current call ID. It inspects that response's complete tool batch without constructing the whole branch or caching a batch across calls/selections. Foreign custom entries and earlier sibling results remain in the native trace. A missing call ID still searches the selected ancestry and preserves the existing unmatched-call behavior; this is not an unconditional constant-time guarantee. See [measured traversal evidence](performance.md#tool-preflight-parent-traversal).
|
|
79
83
|
|
|
@@ -109,7 +113,7 @@ After an accepted non-bootstrap run settles with no queued input, State Flow may
|
|
|
109
113
|
|
|
110
114
|
Pi 0.87 supports retain-none boundary compactions, but State Flow intentionally does not use them. Completed canonical state omits the exact user prompt, and foreign custom context can legitimately occur inside the latest retained iteration; hiding both would make the projected semantic state a lossy substitute for native context. Unknown or smaller usage, foreign custom metadata or native `custom_message` context in the removed prefix, stale selection, Stop/bootstrap/error/abort, and pending input do not produce this boundary. User manual and native threshold/overflow compaction remain unmodified; unaccepted work stays under Pi's native compaction contract.
|
|
111
115
|
|
|
112
|
-
The Pi adapter passes its raw cached scope overlay to `
|
|
116
|
+
The Pi adapter passes its raw cached scope overlay to the context domain's `contextView`, which owns model sanitization. `runtimeContextHead` serializes that already-projected view only when the projection cache needs a new head; `runtimeContextMessage` remains the raw-input convenience builder. The adapter never sanitizes or pre-projects the overlay itself. `currentRunTrajectory` selects a unique captured user timestamp without requiring specification-text equality: Pi may append image normalization hints after `before_agent_start`. Without a captured timestamp, only a unique exact specification match can select a projected suffix. Missing, nonfinite, or ambiguous selection retains all available context. Projection never assigns `runAnchorTimestamp`; native user events own that lifecycle identity, so a projection fallback cannot become compaction authority. With a selected boundary, one retained-message array preserves foreign custom messages at every position and ordinary messages from the original run, including images, tools and steering, without copying discarded ordinary prefixes. The necessary scan and Pi's earlier native-message clone remain history-dependent. See [context-cost evidence](performance.md#context-projection-and-trajectory-selection).
|
|
113
117
|
|
|
114
118
|
## Storage and identity
|
|
115
119
|
|
package/docs/performance.md
CHANGED
|
@@ -28,7 +28,53 @@ The v3 report records `promptPrefixRuns` for each State Flow and native Pi user
|
|
|
28
28
|
|
|
29
29
|
A bounded local sample used `BENCH_PATCHES=2 BENCH_SAMPLES=1 BENCH_ROUNDS=2 BENCH_STATE_BYTES=1024 BENCH_POST_RESUME=1 npm run benchmark` on Node 26.8.1 Linux/x64 and Pi/AI 0.87.0. The runtime-source SHA-256 was `0f4eea2394c52c6ac545260cae967a16581b8768a24a8c611c13a389e6df01a9`, workload-source SHA-256 `e9b8158d953d8622776aa075835c4aa95c98855ed5c90e85de05dccae1087e91`, base commit `5f0f688650738d4d0e9850b2529b0273c47121a8` (the measured tree had uncommitted 0.18.1 changes). Both source hashes stayed unchanged during the run and all correctness phases passed. At user run 1, native Pi's second inference shared 2,776 of 2,777 prior serialized bytes; State Flow shared 9,365 of 9,467 at inference 2 and 9,173 with inference 2 at inference 3 (one accepted barrier). In both State Flow runs, the 21-byte serialized `specification` value duplicated the 21-byte current native user string value; native Pi had no projection field. These are synthetic short-run observations, not representative cache-hit rates or timing predictions.
|
|
30
30
|
|
|
31
|
-
|
|
31
|
+
Byte-prefix measurements omit provider framing, tool schemas, tokenization, cache policies and quality. In particular, serialized synthetic-message timestamps can shorten this proxy prefix without proving provider-visible cache churn. Fresh state visibility after barriers must be validated independently of prefix preservation.
|
|
32
|
+
|
|
33
|
+
### Trajectory-dominant baseline
|
|
34
|
+
|
|
35
|
+
The opt-in `BENCH_PREFIX=1` probe in the [benchmark guide](../benchmarks/README.md) exercises active memory, ordinary passive memory and Stop handoff separately. Each measured run issues six 20,497-byte native reads, a patch, two more reads and a second patch. Exact read content remains visible through all eleven inferences; both patches and terminal completion are checked outside the provider. The v3 `trajectory[].promptPrefixRuns` entries retain every inference, not just aggregate ratios.
|
|
36
|
+
|
|
37
|
+
Baseline: Node 26.8.1 Linux/x64, Pi/AI 0.87.0, base commit `2834dcb447f58867b579c6f2709bc71211b9537a` with uncommitted benchmark changes; runtime SHA-256 `a5a961d4d706d4a2d6649201553e39484d0e4c29d27cf4014f072b39cd78a01a`, workload SHA-256 `1785e73e4992c3d986d2851f2d81963c0b0cfde8ceacfd4337659ebc9d1492ba`. Both source identities remained unchanged and the native workload passed. The local report is `/tmp/state-flow-prefix-baseline.json`; the reproducible command and measurements below do not depend on retaining that temporary file.
|
|
38
|
+
|
|
39
|
+
Each cell is **shared prefix / current serialized context bytes**; `—` means first inference. Inferences 8 and 11 follow accepted patches.
|
|
40
|
+
|
|
41
|
+
| Inference | Active | Passive | Stop handoff |
|
|
42
|
+
| ---: | ---: | ---: | ---: |
|
|
43
|
+
| 1 | — / 9860 | — / 7197 | — / 6019 |
|
|
44
|
+
| 2 | 9732 / 31025 | 5801 / 28360 | 6018 / 27183 |
|
|
45
|
+
| 3 | 9731 / 52193 | 5801 / 49528 | 27182 / 48353 |
|
|
46
|
+
| 4 | 9732 / 73359 | 5801 / 70696 | 48352 / 69519 |
|
|
47
|
+
| 5 | 9732 / 94528 | 5801 / 91865 | 69518 / 90688 |
|
|
48
|
+
| 6 | 9733 / 115695 | 5802 / 113034 | 90687 / 111857 |
|
|
49
|
+
| 7 | 9732 / 136864 | 5801 / 134201 | 111856 / 133026 |
|
|
50
|
+
| 8 | 9212 / 137813 | 5666 / 134999 | 133025 / 133800 |
|
|
51
|
+
| 9 | 9907 / 158983 | 5825 / 156169 | 133799 / 154965 |
|
|
52
|
+
| 10 | 9907 / 180150 | 5825 / 177338 | 154964 / 176134 |
|
|
53
|
+
| 11 | 9235 / 181075 | 5689 / 178112 | 176133 / 176908 |
|
|
54
|
+
|
|
55
|
+
In that baseline, Stop handoff already reused a frozen message, unlike active and ordinary passive projection. A warm prefix alone does not prove updated memory reaches the model; freshness has separate native-SDK regressions.
|
|
56
|
+
|
|
57
|
+
### Frozen-head measurement
|
|
58
|
+
|
|
59
|
+
The implemented projection freezes whole heads, including timestamps, and delivers accepted values and changing notices at stable tail positions. Active completion/new runs, native compaction/selection and Start/Stop are cache boundaries; passive user turns and patches are not. Volatile projection IDs distinguish current updates from retained results after a rebase. See [projection semantics](architecture.md#pi-lifecycle) for ownership and limits.
|
|
60
|
+
|
|
61
|
+
The identical trajectory workload (`1785e73e4992c3d986d2851f2d81963c0b0cfde8ceacfd4337659ebc9d1492ba`) on the same Node/Pi stack measured runtime SHA-256 `d2c424a52d55b0c1ca47a8b1a1beba9c0dda665c8f024d6aa3b6ad95af9d3b46`, with unchanged base commit and uncommitted implementation changes. Both source identities remained stable; all native workload assertions passed. Local report: `/tmp/state-flow-prefix-after.json`.
|
|
62
|
+
|
|
63
|
+
| Inference | Active | Passive | Stop handoff |
|
|
64
|
+
| ---: | ---: | ---: | ---: |
|
|
65
|
+
| 1 | — / 10461 | — / 7985 | — / 6620 |
|
|
66
|
+
| 2 | 10460 / 31625 | 7984 / 29148 | 6619 / 27784 |
|
|
67
|
+
| 3 | 31624 / 52792 | 29147 / 50316 | 27783 / 48952 |
|
|
68
|
+
| 4 | 52791 / 73959 | 50315 / 71484 | 48951 / 70120 |
|
|
69
|
+
| 5 | 73958 / 95127 | 71483 / 92653 | 70119 / 91287 |
|
|
70
|
+
| 6 | 95126 / 116295 | 92652 / 113822 | 91286 / 112456 |
|
|
71
|
+
| 7 | 116294 / 137463 | 113821 / 134989 | 112455 / 133625 |
|
|
72
|
+
| 8 | 137462 / 138416 | 134988 / 135941 | 133624 / 134579 |
|
|
73
|
+
| 9 | 138415 / 159580 | 135940 / 157106 | 134578 / 155742 |
|
|
74
|
+
| 10 | 159579 / 180746 | 157105 / 178275 | 155741 / 176911 |
|
|
75
|
+
| 11 | 180745 / 181697 | 178274 / 179229 | 176910 / 177863 |
|
|
76
|
+
|
|
77
|
+
Every continuation shares **all prior serialized bytes except the closing array bracket**, including both patch barriers in all three modes. At inference 8, the active prefix grows from 9,212 baseline bytes to 137,462; passive grows from 5,666 to 134,988. Initial contexts grow modestly because result guidance and projection identity are explicit. These are message-byte measurements, not provider cache accounting, latency, token-cost or quality guarantees. Regression tests assert prefix equality independently of exact host timestamps/IDs; separate native tests prove accepted-state freshness, repeated barriers, passive cross-turn stability and bootstrap rebasing.
|
|
32
78
|
|
|
33
79
|
## Current cost model
|
|
34
80
|
|
|
@@ -49,7 +95,7 @@ This is a bounded-allocation improvement, not an unconditional constant-time cla
|
|
|
49
95
|
|
|
50
96
|
## Context projection and trajectory selection
|
|
51
97
|
|
|
52
|
-
|
|
98
|
+
The context domain projects the cached semantic overlay once per context emission to compare current state with its last communicated view. It serializes the complete head only at a projection boundary; later synthetic notices retain their original native-message positions. Projection caching targets request-prefix stability, not constant-time state processing: view copies/diffs remain state-dependent and notices accumulate until a natural reset, without a size threshold. `currentRunTrajectory` allocates one retained-message array rather than arrays for discarded ordinary prefixes. Foreign custom context may require scanning earlier entries, and Pi may clone native messages before the extension runs.
|
|
53
99
|
|
|
54
100
|
`tests/context.test.ts` exercises small and large semantic payloads, zero and two hundred prior request/answer pairs, repeated requests, stale/missing anchors, foreign custom messages, and post-barrier context emission. These tests assert projection counts and retained identities; they impose no wall-time threshold.
|
|
55
101
|
|
|
@@ -15,8 +15,8 @@ This is a maintained property-to-test map for the current canonical-file contrac
|
|
|
15
15
|
9. **Unchanged scopes stay stable:** The sparse temporal example checks global state at a boundary where only CWD/session change; “true no-ops do not enter history while response-only changes do” also checks that an unchanged session receives no new patch.
|
|
16
16
|
10. **Historical deletion overlay:** `tests/temporal.test.ts` — “mixed sparse changes and deletion overlays match an independent snapshot oracle through compaction” explicitly checks session → CWD → global fallback and retained historical values.
|
|
17
17
|
11. **Barrier shifts current to offset 1:** `tests/integration.test.ts` — “real Pi patch_state barriers rematerialize every scope before the next inference” observes the predecessor immediately after a barrier.
|
|
18
|
-
12. **Next inference sees new current state:** The same real-Pi test
|
|
19
|
-
13. **No automatic old full-state duplication:** `tests/context.test.ts` — “projects only the latest seven compact accepted transitions” rejects full-state records in transition context. The real-Pi barrier test requires exactly one
|
|
18
|
+
12. **Next inference sees new current state:** The same real-Pi test reconstructs current state from the frozen head plus matching-projection result/tail replacements after session, CWD, and global barriers. “real Pi reads prior scoped state lazily after a barrier and rejects path offset eight without a transition” adds model-tool access to the predecessor.
|
|
19
|
+
13. **No automatic old full-state duplication:** `tests/context.test.ts` — “projects only the latest seven compact accepted transitions” rejects full-state records in transition context. The real-Pi barrier test requires exactly one frozen runtime head per inference, with fresh changes in matching-projection tails. Explicitly requested history remains ordinary tool-result trajectory, not eager snapshot injection.
|
|
20
20
|
14. **Accepted response changes are transitions:** `tests/extension.test.ts` verifies that an ordinary accepted answer becomes runtime-owned `response` without a finalization patch or fallback inference, and “an empty accepted answer finalizes the run and stores an empty response” proves `""` is accepted after an earlier barrier. `tests/transition.test.ts` proves the empty value is an ordinary Session-owned semantic change when it replaces prior text.
|
|
21
21
|
15. **Correct repeats succeed without semantic changes:** “accepts canonical atomic scope patches and correct repeats without another checkpoint” preserves canonical bytes and native checkpoint count on repetition, while still rejecting empty supplied scopes and obsolete finalization-shaped calls. Runtime current-head witnesses preserve revisions, lineage and step across identical patches and compilations; a changed accepted response remains a runtime-owned transition.
|
|
22
22
|
16. **Configured hot-history bounds:** `tests/temporal.test.ts` verifies hot-range and unavailable pre-origin boundaries; the real-Pi history-reader test rejects offset eight at the default limit seven without a transition. `tests/config.test.ts` exercises materialized and scope patch-history paths at limits 0, 1, 7, and 12, including single-path/one-item batch reads above seven and distinct configured versus actually retained boundaries.
|
|
@@ -56,9 +56,10 @@ This is a maintained property-to-test map for the current canonical-file contrac
|
|
|
56
56
|
- `tests/integration.test.ts` native new/resume/fork witnesses preserve canonical files and retained boundaries. Fork witnesses cover selected private state over current shared layers, fresh child ownership, reload/resume, disabled-source fencing, identity/CWD refusal, retry, and expired-boundary refusal. `tests/runtime.test.ts` restore/fork provenance witnesses preserve current-head, untouched-path, and shared evidence while discarding evidence for later-changed private artifacts, including change-away-and-back. The test-only provenance reader is checked against nonempty canonical `meta.json` evidence in every scope before and after reload; native restore/fork tests verify missing-evidence invalidation, explicit recompilation, parent preservation, and provenance across reload.
|
|
57
57
|
- `tests/recovery.test.ts` and `tests/runtime.test.ts` distinguish malformed checkpoints from expired retained boundaries, prevent fallthrough or newer-state substitution, and require detached single-use restoration followed by canonical origin acceptance.
|
|
58
58
|
- `tests/extension.test.ts` — “tool preflight walks only the selected native suffix for matching calls, without rebuilding a branch” uses real in-memory `SessionManager` trees with zero/two hundred prior request-answer pairs, intervening custom/results, and a later unselected assistant reusing a call ID. It requires exact parent visits, no full-branch construction, preserved sibling/duplicate-patch rejection and unchanged complete native entries. The native sibling-tool witness in `tests/integration.test.ts` separately observes all three start/end events and requires zero branch reads during preflight/execution while accepting the correct state/answer.
|
|
59
|
+
- `tests/context.test.ts` proves whole-head identity, stable-position changing/cleared notices, drift since the last communicated view, no redundant post-receipt notices, and reset identity fencing. Native “real Pi bootstrap/passive keeps a byte-stable head” witnesses repeated barriers, active completion rebasing and passive cross-turn continuity. “real Pi passive reload distinguishes retained old receipts from the refreshed head” keeps old native results while requiring a different projection ID and the newly adopted shared value. The trajectory benchmark's contract test requires every continuation prefix to equal the entire preceding serialized array minus its closing bracket in active, passive and Stop-handoff modes; this is not provider cache accounting.
|
|
59
60
|
- `tests/context.test.ts` — “enabled context projects the complete overlay once in ordinary and bootstrap runs” counts one full-overlay clone at 8 KiB and 1 MiB while preserving selected/cached state, native entries, input messages and artifact semantics. “current run trajectory allocates no arrays of discarded ordinary history” observes source-derived arrays after zero/two hundred historical request-answer pairs while preserving foreign context, current tools, steering, reference identity and order. Anchor tests prefer captured identity over normalized text, retain images/tools/steering, and preserve available context on missing/nonfinite/colliding identities or ambiguous specification matches; unique text fallback remains projection-only. The native barrier witness separately counts one marked full-overlay clone per post-barrier `emitContext()` invocation; it still requires each next inference to see all accepted scope changes.
|
|
60
61
|
- `tests/compaction.test.ts` owns State Flow's completed-run compaction policy: captured run-anchor retention across steering/tools, missing/ambiguous/unfinished-anchor refusal, large-history eligibility, generation-owned invocation, foreign `custom`/`custom_message` prefix refusal, queued-work exclusion, stale selection, benign preparation refusal and shutdown fencing. Its harness emits native user `message_end` and checks that repeated projection plus steering cannot promote missing, ambiguous or unobserved anchors into compaction authority. The native “real Pi compaction retains the original run through steering, tool results, and foreign context” cases actually compact plaintext and SDK-normalized image requests with two steering messages, a successful read/patch, and persisted custom context; image/read-result evidence reaches later model calls, exact message ids survive reload, the trace prefix stays byte-identical, and no model summary is requested. The short “real Pi retains a normalized image and read evidence through steering without compaction” controls check State Flow enabled/disabled parity before any compaction. Existing native witnesses cover ordinary compaction/cold resume and leave threshold compaction after partial tool work native.
|
|
61
|
-
- `tests/integration.test.ts` — “real Pi boundary continuation keeps accepted State Flow memory” exercises actual companion `turn_end` and `agent_before_settle` draft entries and continuation. Each next provider input has
|
|
62
|
+
- `tests/integration.test.ts` — “real Pi boundary continuation keeps accepted State Flow memory” exercises actual companion `turn_end` and `agent_before_settle` draft entries and continuation. Each next provider input has one freshly rebased head after accepted completion and matching-projection updates after a continued patch, the accepted response and current tool declarations. There is only one `before_agent_start`, no restored specification in runtime checkpoints or model projection, exactly one requested continuation and correct final response/step. The pure “projects accepted memory without resurrecting a completed specification for boundary continuation” witness preserves source state, omits lazy bodies, and keeps available context when both specification and native capture are absent.
|
|
62
63
|
- `tests/integration.test.ts` — “real Pi context edits remain canonical through tools, tree selection and reload” proves native user-content replacement, assistant/custom-message omission and live tool-result replacement reach provider input without resurrecting raw history. Selecting an unedited branch restores its native view without overwriting accepted semantic files; selecting the edited branch and reloading preserves its edits and independent memory. Raw JSONL stays append-only.
|
|
63
64
|
- `tests/integration.test.ts` — “real Pi structured prompt composes with context hooks” covers active, passive, disabled and explicit foreign-force modes. Conversation hooks exclude systems; full-system hooks receive them; companion before-run and per-request sections plus actual tool declarations survive. User specifications remain outside system authority. Stop/Start on subsequent user requests removes/reinstates the owned section once.
|
|
64
65
|
- `tests/integration.test.ts` — “real Pi preserves native run identity across mode toggles” covers initially enabled, mid-tool Start and repeated Start/Stop with passive bootstrap on/off. Actual provider inputs retain original user/read evidence but not the preceding disabled request; persisted Stop markers name the observed native user, and fixture reload preserves trajectory, frozen state and raw trace prefix. `tests/compaction.test.ts` — “native session boundaries invalidate observed run capture without projection reacquisition” pairs session-start/tree resets with an admitted unchanged-run control. Run identity remains native rather than guessed; an independent optional Stop-marker flag preserves uncompiled bootstrap context without changing the captured anchor.
|
package/lib/context.ts
CHANGED
|
@@ -1,10 +1,11 @@
|
|
|
1
|
+
import { randomUUID } from "node:crypto";
|
|
1
2
|
import type { AgentMessage } from "@earendil-works/pi-agent-core";
|
|
2
3
|
import { projectArtifactForModel, type ArtifactInvalidationNotice, type ArtifactModelHints } from "./artifact.ts";
|
|
3
4
|
import type { RecentTransitionWindow } from "./history.ts";
|
|
4
|
-
import { isObject, presentationJson, type JsonValue } from "./json.ts";
|
|
5
|
+
import { isObject, presentationJson, sameJson, type JsonValue } from "./json.ts";
|
|
5
6
|
import type { Snapshot } from "./snapshot.ts";
|
|
6
7
|
import type { RehydrationPhase } from "./rehydration.ts";
|
|
7
|
-
import { projectStateForModel, type MaterializedState, type ModelState } from "./state.ts";
|
|
8
|
+
import { projectStateForModel, type AtomicScopePatches, type MaterializedState, type ModelState } from "./state.ts";
|
|
8
9
|
|
|
9
10
|
/** Refresh only our section; Pi owns system frames, tools and forced-prompt precedence. */
|
|
10
11
|
export function projectSystemProtocol(messages: AgentMessage[], protocol: string | undefined): AgentMessage[] {
|
|
@@ -53,12 +54,138 @@ export function lazyNavigationHint(state: MaterializedState): { available: boole
|
|
|
53
54
|
}
|
|
54
55
|
|
|
55
56
|
|
|
57
|
+
export type ModelStateUpdate = { path: (string | number)[] } & ({ value: JsonValue } | { deleted: true });
|
|
58
|
+
|
|
59
|
+
/** Exact projected replacements, not authored merge patches; paths are unambiguous key/index segments. */
|
|
60
|
+
export function acceptedStateUpdates(before: MaterializedState, after: MaterializedState, patches: AtomicScopePatches) {
|
|
61
|
+
return projectedStateUpdates(projectStateForModel(before), projectStateForModel(after), patches, lazyNavigationHint(before), lazyNavigationHint(after));
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
function projectedStateUpdates(previous: ModelState, current: ModelState, patches: AtomicScopePatches,
|
|
65
|
+
beforeNavigation: ReturnType<typeof lazyNavigationHint> | undefined, navigation: ReturnType<typeof lazyNavigationHint> | undefined) {
|
|
66
|
+
const effective: ModelStateUpdate[] = [];
|
|
67
|
+
const prefix = (parent: readonly (string | number)[], child: readonly (string | number)[]) =>
|
|
68
|
+
parent.length <= child.length && parent.every((part, index) => part === child[index]);
|
|
69
|
+
const put = (path: (string | number)[], value: JsonValue | undefined) => {
|
|
70
|
+
if (effective.some((entry) => prefix(entry.path, path))) return;
|
|
71
|
+
for (let index = effective.length - 1; index >= 0; index--) {
|
|
72
|
+
if (prefix(path, effective[index]!.path)) effective.splice(index, 1);
|
|
73
|
+
}
|
|
74
|
+
effective.push({ path, ...(value === undefined ? { deleted: true as const } : { value: structuredClone(value) }) });
|
|
75
|
+
};
|
|
76
|
+
const child = (value: JsonValue | undefined, key: string | number): JsonValue | undefined =>
|
|
77
|
+
value !== null && typeof value === "object" && Object.hasOwn(value, key)
|
|
78
|
+
? (value as Record<string | number, JsonValue>)[key] : undefined;
|
|
79
|
+
const diff = (left: JsonValue | undefined, right: JsonValue | undefined, path: (string | number)[]) => {
|
|
80
|
+
if (left === undefined && right === undefined || left !== undefined && right !== undefined && sameJson(left, right)) return;
|
|
81
|
+
if (isObject(left) && isObject(right)) {
|
|
82
|
+
for (const key of new Set([...Object.keys(left), ...Object.keys(right)])) diff(child(left, key), child(right, key), [...path, key]);
|
|
83
|
+
} else if (Array.isArray(left) && Array.isArray(right) && left.length === right.length) {
|
|
84
|
+
for (let index = 0; index < right.length; index++) diff(left[index], right[index], [...path, index]);
|
|
85
|
+
} else put(path, right);
|
|
86
|
+
};
|
|
87
|
+
const touched = (patch: JsonValue, value: JsonValue | undefined, path: (string | number)[]) => {
|
|
88
|
+
if (!isObject(patch)) { put(path, value); return; }
|
|
89
|
+
const keys = Object.keys(patch);
|
|
90
|
+
if (keys.length === 0) return;
|
|
91
|
+
if (Array.isArray(value) && keys.every((key) => /^\[(0|[1-9]\d*)\]$/.test(key))) {
|
|
92
|
+
for (const key of keys) {
|
|
93
|
+
const index = Number(key.slice(1, -1));
|
|
94
|
+
// A higher-scope array may mask the patched array with a different length.
|
|
95
|
+
if (index >= value.length) { put(path, value); return; }
|
|
96
|
+
touched(patch[key]!, value[index], [...path, index]);
|
|
97
|
+
}
|
|
98
|
+
} else if (isObject(value)) {
|
|
99
|
+
for (const key of keys) touched(patch[key]!, child(value, key), [...path, key]);
|
|
100
|
+
} else put(path, value);
|
|
101
|
+
};
|
|
102
|
+
diff(previous, current, []);
|
|
103
|
+
for (const patch of Object.values(patches)) for (const [plane, value] of Object.entries(patch)) {
|
|
104
|
+
if (plane === "lazy") continue;
|
|
105
|
+
if (plane === "artifacts" && isObject(value)) {
|
|
106
|
+
for (const path of Object.keys(value)) put([plane, path], child(current.artifacts, path));
|
|
107
|
+
} else touched(value as JsonValue, child(current, plane), [plane]);
|
|
108
|
+
}
|
|
109
|
+
return { effective, ...(navigation !== undefined && (beforeNavigation === undefined || !sameJson(beforeNavigation, navigation)) ? { lazy_navigation: navigation } : {}) };
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
export interface ContextView {
|
|
113
|
+
state: ModelState;
|
|
114
|
+
lazy_navigation?: ReturnType<typeof lazyNavigationHint>;
|
|
115
|
+
artifact_invalidations: readonly ArtifactInvalidationNotice[];
|
|
116
|
+
knowledge_rehydration: { phase: RehydrationPhase } | null;
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
export function contextView(state: MaterializedState, hints: ArtifactModelHints, invalidations: readonly ArtifactInvalidationNotice[], phase?: RehydrationPhase): ContextView {
|
|
120
|
+
return { state: projectStateForModel(state, hints), lazy_navigation: lazyNavigationHint(state),
|
|
121
|
+
artifact_invalidations: structuredClone(invalidations), knowledge_rehydration: phase === undefined ? null : { phase } };
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
/** Volatile model projection only. Native messages own trajectory; this cache owns no persistence or lifecycle. */
|
|
125
|
+
export class ContextProjection {
|
|
126
|
+
private identity = randomUUID();
|
|
127
|
+
private head: AgentMessage | undefined;
|
|
128
|
+
private view: ContextView | undefined;
|
|
129
|
+
private native: string[] = [];
|
|
130
|
+
private notices: Array<{ after: number; message: AgentMessage }> = [];
|
|
131
|
+
|
|
132
|
+
reset(): void {
|
|
133
|
+
this.identity = randomUUID();
|
|
134
|
+
this.head = undefined;
|
|
135
|
+
this.view = undefined;
|
|
136
|
+
this.native = [];
|
|
137
|
+
this.notices = [];
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
/** Called only after successful publication and ancillary acceptance, immediately before returning the native result. */
|
|
141
|
+
acceptPatch(before: MaterializedState, after: MaterializedState, patches: AtomicScopePatches, hints: ArtifactModelHints) {
|
|
142
|
+
const state = projectStateForModel(after, hints);
|
|
143
|
+
const navigation = lazyNavigationHint(after);
|
|
144
|
+
const updates = projectedStateUpdates(this.view?.state ?? projectStateForModel(before, hints), state, patches,
|
|
145
|
+
this.view?.lazy_navigation ?? lazyNavigationHint(before), navigation);
|
|
146
|
+
if (this.view) this.view = { ...this.view, state, lazy_navigation: navigation };
|
|
147
|
+
return { projection: this.identity, ...updates };
|
|
148
|
+
}
|
|
149
|
+
|
|
150
|
+
project(messages: AgentMessage[], current: ContextView, makeHead: () => AgentMessage, initial?: ContextView): AgentMessage[] {
|
|
151
|
+
const identities = messages.map((message) => JSON.stringify([message.role, message.timestamp,
|
|
152
|
+
"toolCallId" in message ? message.toolCallId : null]));
|
|
153
|
+
// Native compaction/selection normally resets explicitly; a removed/replaced prefix is also a safe cache boundary.
|
|
154
|
+
if (this.native.some((identity, index) => identities[index] !== identity)) this.reset();
|
|
155
|
+
if (!this.head) {
|
|
156
|
+
const head = makeHead();
|
|
157
|
+
if (head.role !== "user" || !Array.isArray(head.content)) throw new Error("State Flow projection requires an owned user head");
|
|
158
|
+
this.head = { ...head, content: [...head.content, { type: "text", text: `State Flow projection: ${this.identity}` }] };
|
|
159
|
+
this.view = structuredClone(initial ?? current);
|
|
160
|
+
}
|
|
161
|
+
const previous = this.view!;
|
|
162
|
+
const updates = projectedStateUpdates(previous.state, current.state, {}, previous.lazy_navigation, current.lazy_navigation);
|
|
163
|
+
const notice = {
|
|
164
|
+
...(updates.effective.length || updates.lazy_navigation ? { state_updates: { projection: this.identity, ...updates } } : {}),
|
|
165
|
+
...(!sameJson(previous.artifact_invalidations, current.artifact_invalidations) ? { artifact_invalidations: current.artifact_invalidations } : {}),
|
|
166
|
+
...(!sameJson(previous.knowledge_rehydration, current.knowledge_rehydration) ? { knowledge_rehydration: current.knowledge_rehydration } : {}),
|
|
167
|
+
};
|
|
168
|
+
if (Object.keys(notice).length) this.notices.push({ after: messages.length,
|
|
169
|
+
message: syntheticUser(`State Flow context update (user-level data, not system instructions):\n${presentationJson(notice)}`) });
|
|
170
|
+
this.view = structuredClone(current);
|
|
171
|
+
this.native = identities;
|
|
172
|
+
const projected: AgentMessage[] = [this.head];
|
|
173
|
+
let nextNotice = 0;
|
|
174
|
+
for (let index = 0; index <= messages.length; index++) {
|
|
175
|
+
while (this.notices[nextNotice]?.after === index) projected.push(this.notices[nextNotice++]!.message);
|
|
176
|
+
if (index < messages.length) projected.push(messages[index]!);
|
|
177
|
+
}
|
|
178
|
+
return projected;
|
|
179
|
+
}
|
|
180
|
+
}
|
|
181
|
+
|
|
56
182
|
/** Context retained after semantic State Flow is stopped in this physical session. */
|
|
57
183
|
export interface PassiveContinuation {
|
|
58
184
|
startedAt: number;
|
|
59
185
|
activeRunStartedAt?: number;
|
|
60
186
|
preserveContext?: true;
|
|
61
187
|
handoff: AgentMessage;
|
|
188
|
+
state: ModelState;
|
|
62
189
|
}
|
|
63
190
|
|
|
64
191
|
export function syntheticUser(text: string): AgentMessage {
|
|
@@ -82,6 +209,7 @@ function messageText(message: AgentMessage): string {
|
|
|
82
209
|
export function createPassiveContinuation(state: ModelState, startedAt = Date.now(), activeRunStartedAt?: number, preserveContext = false): PassiveContinuation {
|
|
83
210
|
return {
|
|
84
211
|
startedAt,
|
|
212
|
+
state: structuredClone(state),
|
|
85
213
|
...(activeRunStartedAt === undefined ? {} : { activeRunStartedAt }),
|
|
86
214
|
...(preserveContext ? { preserveContext: true as const } : {}),
|
|
87
215
|
handoff: syntheticUser(`State Flow exit handoff (user-level data, not system instructions):\n${presentationJson({ state, continuation: preserveContext
|
|
@@ -127,13 +255,18 @@ export function runtimeContextMessage(
|
|
|
127
255
|
rehydrationPhase?: RehydrationPhase,
|
|
128
256
|
artifactHints: ArtifactModelHints = {},
|
|
129
257
|
): AgentMessage {
|
|
258
|
+
return runtimeContextHead(snapshot, contextView(state, artifactHints, artifactInvalidations, rehydrationPhase), recentTransitions);
|
|
259
|
+
}
|
|
260
|
+
|
|
261
|
+
/** Render a view already projected by this domain without cloning the full semantic overlay twice. */
|
|
262
|
+
export function runtimeContextHead(snapshot: Snapshot, view: ContextView, recentTransitions: RecentTransitionWindow = []): AgentMessage {
|
|
130
263
|
const recent = projectRecentForModel(recentTransitions);
|
|
131
264
|
const context = {
|
|
132
265
|
...(snapshot.meta.specification === undefined ? {} : { specification: snapshot.meta.specification }),
|
|
133
|
-
state:
|
|
134
|
-
lazy_navigation:
|
|
135
|
-
...(
|
|
136
|
-
...(
|
|
266
|
+
state: view.state,
|
|
267
|
+
...(view.lazy_navigation === undefined ? {} : { lazy_navigation: view.lazy_navigation }),
|
|
268
|
+
...(view.knowledge_rehydration === null ? {} : { knowledge_rehydration: view.knowledge_rehydration }),
|
|
269
|
+
...(view.artifact_invalidations.length === 0 ? {} : { artifact_invalidations: view.artifact_invalidations.map(({ path, scope, reason }) => ({ path, ...(scope === undefined ? {} : { scope }), reason })) }),
|
|
137
270
|
...(recent.length === 0 ? {} : { recent_transitions: recent }),
|
|
138
271
|
};
|
|
139
272
|
return syntheticUser(
|
package/lib/extension.ts
CHANGED
|
@@ -16,7 +16,7 @@ import {
|
|
|
16
16
|
} from "./artifact.ts";
|
|
17
17
|
import { hasCompactionSizedTranscript, planStateFlowCompaction, shouldRequestStateFlowCompaction, stateFlowCompactionResult, type StateFlowCompactionPlan } from "./compaction.ts";
|
|
18
18
|
import { loadStateFlowConfig } from "./config.ts";
|
|
19
|
-
import { createPassiveContinuation, currentRunTrajectory,
|
|
19
|
+
import { ContextProjection, contextView, createPassiveContinuation, currentRunTrajectory, passiveContinuationMessages, projectSystemProtocol, runtimeContextHead, syntheticUser, type PassiveContinuation } from "./context.ts";
|
|
20
20
|
import { readNativeSessionHeader } from "./continuation.ts";
|
|
21
21
|
import {
|
|
22
22
|
cwdScopeKey,
|
|
@@ -109,6 +109,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
109
109
|
const compactionMarker = `state-flow-boundary:${randomUUID()}`;
|
|
110
110
|
let passiveContinuation: PassiveContinuation | undefined;
|
|
111
111
|
let bootstrapContinuation: PassiveContinuation | undefined;
|
|
112
|
+
const contextProjection = new ContextProjection();
|
|
112
113
|
let inferencePreparation: InferencePreparation | undefined;
|
|
113
114
|
let runAnchorTimestamp: number | undefined;
|
|
114
115
|
let runtime: TemporalRuntime | undefined;
|
|
@@ -459,6 +460,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
459
460
|
|
|
460
461
|
/** Select the active native branch under one owned restoration lifetime; only current accepted work installs memory. */
|
|
461
462
|
function restoreActiveBranch(ctx: ExtensionContext, sessionStartReason?: unknown, notifyRecovery = true, startOwner?: AbortController): Promise<void> {
|
|
463
|
+
contextProjection.reset();
|
|
462
464
|
cancelBranchRestoration();
|
|
463
465
|
// Start-owned attachment/fork recovery keeps its owner; only accepted Start cancels Stop.
|
|
464
466
|
if (!startOwner) {
|
|
@@ -771,6 +773,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
771
773
|
const selected = runtime ??= createRuntime(ctx);
|
|
772
774
|
const acquiredArtifacts = structuredClone([...artifactReads.successful.values()]);
|
|
773
775
|
const acquiredSkills = structuredClone([...skillReads.successful.values()]);
|
|
776
|
+
const previousEffective = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
774
777
|
return await selected.withPatchTransaction((transaction) => {
|
|
775
778
|
if (runtime !== selected) throw new Error("State Flow session selection changed while awaiting publication");
|
|
776
779
|
if (!passiveToolsAvailable()) throw new Error("State Flow tools are disabled by configuration");
|
|
@@ -800,9 +803,14 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
800
803
|
}
|
|
801
804
|
clearAcceptedAcquisitions(new Set(acquiredArtifacts.map(({ path }) => path)));
|
|
802
805
|
updateUi(ctx);
|
|
803
|
-
|
|
806
|
+
const updates = contextProjection.acceptPatch(previousEffective, overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session), patches, artifactHints);
|
|
807
|
+
const acknowledgement = changed
|
|
804
808
|
? `\nState materialized atomically at ${scopes.join("+")} scope${scopes.length === 1 ? "" : "s"}.`
|
|
805
|
-
: "\nState already current."
|
|
809
|
+
: "\nState already current.";
|
|
810
|
+
return { content: [
|
|
811
|
+
{ type: "text" as const, text: acknowledgement },
|
|
812
|
+
{ type: "text" as const, text: `\n${presentationJson({ state_updates: updates })}` },
|
|
813
|
+
], details: { scopes, step: snapshot.meta.step, changed } };
|
|
806
814
|
}, signal);
|
|
807
815
|
} catch (error) {
|
|
808
816
|
let attempted: unknown;
|
|
@@ -908,6 +916,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
908
916
|
stopPersistenceError = undefined;
|
|
909
917
|
installScopeStates();
|
|
910
918
|
clearRunTransient();
|
|
919
|
+
contextProjection.reset();
|
|
911
920
|
passiveContinuation = undefined;
|
|
912
921
|
bootstrapContinuation = snapshot.meta.bootstrap ? continuation : undefined;
|
|
913
922
|
deferInferencePreparation();
|
|
@@ -993,6 +1002,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
993
1002
|
&& !shuttingDown && ctx.sessionManager.getSessionId() === owner && !snapshot.config.enabled;
|
|
994
1003
|
const superseded = (): StateFlowTelegramControlResult => ({ ok: false, message: "State Flow Stop was superseded" });
|
|
995
1004
|
const freezeHandoff = (): PassiveContinuation | undefined => {
|
|
1005
|
+
contextProjection.reset();
|
|
996
1006
|
const handoff = (current.config.enabled || stopPersistenceError) && selected?.view && current.meta.validation?.attempt !== 0
|
|
997
1007
|
? createPassiveContinuation(
|
|
998
1008
|
projectModelState(overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session)),
|
|
@@ -1112,6 +1122,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
1112
1122
|
(event.systemPromptOptions.sections ??= {}).state_flow = PASSIVE_MEMORY_PROTOCOL;
|
|
1113
1123
|
return;
|
|
1114
1124
|
}
|
|
1125
|
+
contextProjection.reset();
|
|
1115
1126
|
skillReads.clear();
|
|
1116
1127
|
artifactReads.clear();
|
|
1117
1128
|
cancelResponseReconciliation();
|
|
@@ -1129,40 +1140,30 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
1129
1140
|
|
|
1130
1141
|
function projectContext(messages: AgentMessage[]) {
|
|
1131
1142
|
if (runtime?.view) refreshArtifactHints();
|
|
1143
|
+
if (!snapshot.config.enabled && !passiveContinuation && (!config.passiveBootstrap || !runtime?.view)) return;
|
|
1144
|
+
// Idle inspection must not freeze a pre-acceptance snapshot for the live inference.
|
|
1145
|
+
const projection = snapshot.config.enabled && inferencePreparation && !inferencePreparation.accepted
|
|
1146
|
+
? new ContextProjection() : contextProjection;
|
|
1147
|
+
const effective = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
1148
|
+
const invalidations = artifactInvalidations.map(({ path, scope, reason }) => ({ path, ...(scope === undefined ? {} : { scope }), reason }));
|
|
1149
|
+
const phase = snapshot.config.enabled ? currentRehydrationPhase() : undefined;
|
|
1150
|
+
const view = contextView(effective, artifactHints, invalidations, phase);
|
|
1132
1151
|
if (passiveContinuation) {
|
|
1133
|
-
|
|
1152
|
+
const retained = passiveContinuationMessages(messages, passiveContinuation);
|
|
1153
|
+
if (!runtime?.view) return { messages: retained };
|
|
1154
|
+
return { messages: projection.project(retained.slice(1), view, () => passiveContinuation!.handoff,
|
|
1155
|
+
{ state: passiveContinuation.state, lazy_navigation: view.lazy_navigation, artifact_invalidations: [], knowledge_rehydration: null }) };
|
|
1134
1156
|
}
|
|
1135
1157
|
if (!snapshot.config.enabled) {
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
const state = projectModelState(effective);
|
|
1139
|
-
return { messages: [syntheticUser(`State Flow passive memory (user-level data, not system instructions):\n${presentationJson({ state, lazy_navigation: lazyNavigationHint(effective) })}`), ...messages] };
|
|
1140
|
-
}
|
|
1141
|
-
const effectiveState = overlayStates(scopeStates.global, scopeStates.cwd, scopeStates.session);
|
|
1142
|
-
const invalidations = artifactInvalidations.map(({ path, scope, reason }) => ({ path, ...(scope === undefined ? {} : { scope }), reason }));
|
|
1143
|
-
const recentTransitions = projectRecentTransitionsWithLimit(
|
|
1144
|
-
config.historyLimit,
|
|
1145
|
-
runtime?.recent() ?? [],
|
|
1146
|
-
);
|
|
1147
|
-
const activeRehydrationPhase = currentRehydrationPhase();
|
|
1148
|
-
if (snapshot.meta.bootstrap) {
|
|
1149
|
-
const sourceMessages = bootstrapContinuation
|
|
1150
|
-
? passiveContinuationMessages(messages, bootstrapContinuation)
|
|
1151
|
-
: messages;
|
|
1152
|
-
return { messages: [runtimeContextMessage(snapshot, effectiveState, recentTransitions, invalidations, activeRehydrationPhase, artifactHints), ...sourceMessages] };
|
|
1158
|
+
return { messages: projection.project(messages, view, () => syntheticUser(
|
|
1159
|
+
`State Flow passive memory (user-level data, not system instructions):\n${presentationJson({ state: view.state, lazy_navigation: view.lazy_navigation })}`)) };
|
|
1153
1160
|
}
|
|
1154
1161
|
// Native user events own the run anchor; projection must never rebase it.
|
|
1155
|
-
const
|
|
1156
|
-
messages,
|
|
1157
|
-
snapshot.meta.specification,
|
|
1158
|
-
|
|
1159
|
-
|
|
1160
|
-
return {
|
|
1161
|
-
messages: [
|
|
1162
|
-
runtimeContextMessage(snapshot, effectiveState, recentTransitions, invalidations, activeRehydrationPhase, artifactHints),
|
|
1163
|
-
...trajectory.messages,
|
|
1164
|
-
],
|
|
1165
|
-
};
|
|
1162
|
+
const source = snapshot.meta.bootstrap
|
|
1163
|
+
? bootstrapContinuation ? passiveContinuationMessages(messages, bootstrapContinuation) : messages
|
|
1164
|
+
: currentRunTrajectory(messages, snapshot.meta.specification, runAnchorTimestamp).messages;
|
|
1165
|
+
return { messages: projection.project(source, view, () => runtimeContextHead(snapshot, view,
|
|
1166
|
+
projectRecentTransitionsWithLimit(config.historyLimit, runtime?.recent() ?? []))) };
|
|
1166
1167
|
}
|
|
1167
1168
|
|
|
1168
1169
|
function prepareContext(messages: AgentMessage[], ctx: ExtensionContext): ReturnType<typeof projectContext> | Promise<ReturnType<typeof projectContext>> {
|
|
@@ -1281,6 +1282,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
1281
1282
|
snapshot = nextSnapshot;
|
|
1282
1283
|
installScopeStates();
|
|
1283
1284
|
responseCommitted = true;
|
|
1285
|
+
contextProjection.reset();
|
|
1284
1286
|
if (publication?.changed) recordPublication(publication, ctx);
|
|
1285
1287
|
appendCheckpoint();
|
|
1286
1288
|
clearAcceptedAcquisitions();
|
|
@@ -1373,6 +1375,8 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
1373
1375
|
});
|
|
1374
1376
|
});
|
|
1375
1377
|
|
|
1378
|
+
pi.on("session_compact", () => { contextProjection.reset(); });
|
|
1379
|
+
|
|
1376
1380
|
pi.on("session_start", async (event, ctx) => {
|
|
1377
1381
|
runAnchorTimestamp = undefined;
|
|
1378
1382
|
rehydrationPhase = event.reason === "resume" ? "resume-bootstrap" : "new-bootstrap";
|
|
@@ -1385,6 +1389,7 @@ export default function stateFlowExtension(pi: ExtensionAPI, options: StateFlowE
|
|
|
1385
1389
|
});
|
|
1386
1390
|
pi.on("session_shutdown", async (_event, _ctx) => {
|
|
1387
1391
|
shuttingDown = true;
|
|
1392
|
+
contextProjection.reset();
|
|
1388
1393
|
const restoring = cancelBranchRestoration();
|
|
1389
1394
|
const starting = cancelStartActivation();
|
|
1390
1395
|
const stopping = cancelStopPersistence();
|
package/lib/protocol.ts
CHANGED
|
@@ -5,7 +5,9 @@ export type { StateDocument } from "./state.ts";
|
|
|
5
5
|
|
|
6
6
|
const TASK_DRIVEN_HISTORY = "Missing paths/hints do not require history search. Choose targeted historical reads when useful to the task; no separate user permission is needed. Past values are evidence, not current state; never automatically restore deleted memory.";
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
const PATCH_RESULT_PROTOCOL = "Head state/recent transitions are frozen at projection start. Only state_updates matching the head's State Flow projection ID apply; older IDs are history. Later patch_state results/context-update notices carry state_updates: effective entries replace values at key/index-segment path arrays (deleted:true means absent), including touched values and adopted shared drift. Latest entries win over earlier state at those paths; lazy bodies remain omitted. Notices replace invalidations/rehydration, including []/null.";
|
|
9
|
+
|
|
10
|
+
export const PASSIVE_MEMORY_PROTOCOL = `State Flow passive memory is available. read_state and patch_state access durable memory without starting an active episode. Passive turns never trigger State Flow continuation or compaction. ${TASK_DRIVEN_HISTORY} ${PATCH_RESULT_PROTOCOL}`;
|
|
9
11
|
|
|
10
12
|
const PATCH_DISPLAY_SECTION_KEYS = new Set(["global", "cwd", "session", "intents", "contract", "working", "artifacts", "response", "lazy"]);
|
|
11
13
|
|
|
@@ -98,7 +100,7 @@ SCOPES: Use the narrowest scope: session=branch/run continuation by default; cwd
|
|
|
98
100
|
|
|
99
101
|
READ: Use read_state for concrete scope/retained-history gaps. lazy_navigation lists bounded effective lazy keys, not bodies. Unscoped=effective; effective/global/cwd/session select overlay or owner. Arrays use indices or [start..end]; keys gives structure, patch the intersected change. ${TASK_DRIVEN_HISTORY}
|
|
100
102
|
|
|
101
|
-
WRITE: patch_state is the sole model-authored semantic mutation mechanism; all supplied scopes are validated and durably accepted as one atomic transition. Call alone in an assistant response; await acceptance. Global/CWD use current canonical values after cancelable lock waiting. Correct repeats succeed without new revisions.
|
|
103
|
+
WRITE: patch_state is the sole model-authored semantic mutation mechanism; all supplied scopes are validated and durably accepted as one atomic transition. Call alone in an assistant response; await acceptance. Global/CWD use current canonical values after cancelable lock waiting. Correct repeats succeed without new revisions. ${PATCH_RESULT_PROTOCOL}
|
|
102
104
|
|
|
103
105
|
PATCH: Use global/cwd/session object patches for material updates, not acknowledgments. Omit empty scopes. artifacts/contract/working/intents are objects; lazy is ordinary JSON. Omitted fields persist. Never patch runtime config/meta/response. Objects merge recursively; arrays/primitives replace. An object containing only canonical "[N]" keys recursively patches array elements. Indexed deletion is forbidden; nested object null deletes; materialized null is forbidden.
|
|
104
106
|
|
package/package.json
CHANGED
|
@@ -32,7 +32,9 @@ Scopes overlay `global → cwd → session`: cross-project, project, branch/run.
|
|
|
32
32
|
|
|
33
33
|
## Read
|
|
34
34
|
|
|
35
|
-
Reuse sufficient visible state. `
|
|
35
|
+
Reuse sufficient visible state. The memory head and its recent transitions are frozen at projection start. Apply later `state_updates` only when their `projection` matches the head's `State Flow projection:` ID; older native results remain historical. Effective update paths are key/index arrays whose values replace that path, while `deleted: true` means absence. Current notices can replace invalidation lists or rehydration phase, including clearing them with `[]` or `null`; lazy bodies still require explicit reads.
|
|
36
|
+
|
|
37
|
+
`read_state` accepts `path` or `paths`, never both. Multi-path reads succeed or fail together. Projections: `value` (default), `keys` (structure), `patch` (intersecting change at the selected boundary).
|
|
36
38
|
|
|
37
39
|
Example arguments:
|
|
38
40
|
|