world-model-optimizer 0.2.0__py3-none-any.whl

wmo/harness/vendor/pi-agent/docs/durable-harness.md

@@ -0,0 +1,212 @@
+# Durable AgentHarness and session design
+
+<!-- Synced from jot zmnps2zu. Edit this file in-repo going forward. -->
+
+Durable AgentHarness / session design notes.
+
+## Framing
+
+A fully durable `AgentHarness` is not realistic by itself because important dependencies are runtime JS supplied by the host app:
+
+- tool implementations
+- model/auth providers
+- extensions and hook handlers
+- resource loaders
+- system-prompt callbacks/modifiers
+
+Tool registries are runtime dependencies. The harness should persist serializable tool configuration, such as active tool names, but not concrete tool implementations.
+
+The practical target is a semi-durable harness:
+
+- session is the durable append-only state tree
+- harness persists the state it owns into session entries
+- the host app is responsible for recreating compatible non-persistable dependencies on resume
+- recovery restarts from durable boundaries, not from an in-flight provider stream
+
+## Session owns durable state
+
+Treat session as all durable agent state, not just transcript history.
+
+Existing session state already includes harness state:
+
+- model changes
+- thinking-level changes
+- active-tool changes
+- leaf entries
+- labels
+- compactions and branch summaries
+- custom messages and custom entries
+
+That suggests continuing with one durable session log rather than adding harness sidecars. Sidecars may still be useful for large blobs, but the session entry should remain the source-of-truth reference.
+
+## What the app must provide on resume
+
+The app must recreate compatible runtime dependencies:
+
+- model registry / model objects
+- tool registry
+- extension set, versions, and ordering
+- resource loaders
+- system prompt providers/hooks
+- auth providers
+- app-specific hooks
+
+Harness can validate stable IDs/versions/hashes when available, but it cannot serialize these dependencies itself.
+
+## Runtime configuration and restore
+
+Constructor options remain explicit runtime configuration and do not read session state. Hidden async restore in a constructor would make failure handling ambiguous.
+
+A future async builder/factory should own durable restore:
+
+```ts
+const harness = await AgentHarness.builder()
+  .env(env)
+  .session(session)
+  .model(defaultModel)
+  .tools(runtimeTools)
+  .defaultActiveTools(["read", "edit"])
+  .restore({ missingActiveTools: "fail" });
+```
+
+`restore()` should read the active branch, reduce durable harness configuration, apply defaults for missing entries, validate against app-supplied runtime dependencies, construct the harness, and optionally emit `source: "restore"` update events after construction.
+
+For active tools:
+
+- `active_tools_change` entries are branch-scoped durable config.
+- If no `active_tools_change` exists on the branch, restore uses builder defaults, or all registered tools if no default active names were supplied.
+- Active tool names must be unique.
+- Tool registry names must be unique.
+- Missing restored active tool names should fail restore by default; permissive drop/disable policies can be added explicitly later.
+- Concrete tools are never restored from session; the host app must provide compatible tools.
+
+## What harness should persist
+
+Minimum useful durability entries:
+
+- branch-scoped active tool names
+- queued steer/followUp/nextTurn messages
+- queue consumption tied to a turn
+- pending session writes accepted during active operations
+- pending write application status
+- operation start/finish/interruption
+- turn start/finish
+- provider request start/finish, if needed for recovery diagnostics
+- tool call start/finish, if we want safe tool recovery
+
+Potential entries:
+
+```ts
+type DurableHarnessEntry =
+  | QueueEnqueuedEntry
+  | QueueConsumedEntry
+  | PendingWriteEnqueuedEntry
+  | PendingWriteAppliedEntry
+  | OperationStartedEntry
+  | OperationFinishedEntry
+  | OperationInterruptedEntry
+  | TurnStartedEntry
+  | TurnFinishedEntry
+  | ProviderRequestStartedEntry
+  | ProviderRequestFinishedEntry
+  | ToolCallStartedEntry
+  | ToolCallFinishedEntry;
+```
+
+Every accepted mutation must be durable before the public API resolves.
+
+## Recovery model
+
+On startup:
+
+1. Host app registers tools/models/extensions/resources/auth/hooks.
+2. Harness opens session.
+3. Harness reduces session entries into:
+   - current leaf
+   - conversation branch
+   - harness config, including active tool names
+   - queues
+   - pending writes
+   - active operation/turn/tool state
+4. Harness validates required runtime dependencies, including restored active tool names against the app-provided tool registry.
+5. Harness reconciles unfinished operation state.
+
+Provider streams are not resumable. Recovery can only retry from a durable boundary or mark the operation interrupted.
+
+## Recovery policies
+
+Default conservative policy:
+
+- unfinished agent turn: mark interrupted, preserve durable queues/pending writes, return idle
+- unfinished provider request: mark interrupted; do not retry automatically
+- unfinished tool call: append interrupted/error tool result; retry only if the tool declares retry-safe/idempotent
+- unfinished compaction: rerun if no compaction entry exists
+- unfinished branch summary/tree navigation: rerun/apply missing summary or leaf entries if safe
+
+Optional policy:
+
+```ts
+recovery: "mark_interrupted" | "retry_unfinished"
+```
+
+`retry_unfinished` must be guarded around non-idempotent tool calls.
+
+## Critical scenarios
+
+### Queues
+
+- Crash before `queue_enqueued`: message was not accepted.
+- Crash after `queue_enqueued`: message is restored.
+- Crash after queue drain but before durable turn record: risk of loss/duplication.
+- Required invariant: consumed queue IDs must be recorded in `turn_started` or equivalent before they are considered consumed.
+
+### Pending writes
+
+- Crash before `pending_write_enqueued`: write was not accepted.
+- Crash after enqueue before apply: recovery applies it.
+- Crash after apply before applied marker: deterministic target entry IDs let recovery detect the entry already exists and mark it applied.
+
+### Agent loop turn
+
+- Crash before provider request: retry or mark interrupted.
+- Crash during provider request: mark interrupted by default.
+- Crash after provider response before assistant message persisted: response is lost unless provider result was journaled.
+- Crash after assistant message persisted: recover from durable message.
+
+### Tool calls
+
+- Crash after tool call starts but before result: external side effects may already have happened.
+- Default recovery should not rerun non-idempotent tools.
+- Tool calls need stable IDs and retry-safety metadata for automatic recovery.
+
+### Compaction
+
+- Crash before summary generation: rerun preparation/summary.
+- Crash after generated summary but before compaction entry: rerun unless summary was journaled.
+- Crash after compaction entry: operation is complete; append finish marker if missing.
+
+### Branch summary / tree navigation
+
+- Crash before summary: rerun or mark interrupted.
+- Crash after summary entry before leaf entry: append missing leaf entry.
+- Crash after leaf entry: operation is complete; append finish marker if missing.
+
+## Minimum viable spike
+
+1. Add durable queue entries.
+2. Add durable pending write entries with deterministic target IDs.
+3. Add operation start/finish/interrupted entries.
+4. Add turn start with consumed queue IDs.
+5. Recover by reducing the session log.
+6. Mark unfinished agent turns interrupted by default.
+7. Rerun unfinished compaction/tree operations only when no final entry exists.
+8. Do not retry unfinished tool calls unless tool metadata says retry-safe.
+
+## Open questions
+
+- Which remaining harness config entries should move into session first: resources, stream options, system prompt refs?
+- Should resolved system prompt text be snapshotted per turn for audit/debug?
+- Do we require strict dependency ID/version matching on resume?
+- How much provider request data should be journaled?
+- Should recovery append user-visible assistant interruption messages or only internal operation entries?
+- Should storage support truncating a final partial JSONL line during recovery?

wmo/harness/vendor/pi-agent/docs/hooks.md

@@ -0,0 +1,445 @@
+# AgentHarness hooks design
+
+<!-- Synced from jot 3utlzkxy. Edit this file in-repo going forward. -->
+
+Final design.
+
+## Core model
+
+Events carry their result type as a type-only phantom:
+
+```ts
+declare const HookResult: unique symbol;
+
+interface HookEvent<TType extends string, TResult = void> {
+	type: TType;
+	readonly [HookResult]?: TResult;
+}
+
+type ResultOf<E> = E extends { readonly [HookResult]?: infer R } ? R : void;
+
+type HookHandler<E, Ctx> = (
+	event: E,
+	ctx: Ctx,
+	signal?: AbortSignal,
+) => ResultOf<E> | void | Promise<ResultOf<E> | void>;
+
+type HookObserver<E, Ctx> = (
+	event: E,
+	ctx: Ctx,
+	signal?: AbortSignal,
+) => void | Promise<void>;
+```
+
+Example:
+
+```ts
+interface ContextEvent extends HookEvent<"context", { messages?: AgentMessage[] }> {
+	type: "context";
+	messages: AgentMessage[];
+}
+
+interface ToolCallEvent extends HookEvent<"tool_call", { block?: boolean; reason?: string }> {
+	type: "tool_call";
+	toolName: string;
+	input: Record<string, unknown>;
+}
+
+interface MessageEndEvent extends HookEvent<"message_end"> {
+	type: "message_end";
+	message: AgentMessage;
+}
+```
+
+No result map. No spec table. The event type defines its own result.
+
+## Hooks interface
+
+```ts
+interface AgentHarnessHooks<E extends HookEvent<string, unknown>, Ctx> {
+	context: Ctx;
+
+	setContext(ctx: Ctx): void;
+
+	observe(handler: HookObserver<E, Ctx>): () => void;
+
+	on<TType extends E["type"]>(
+		type: TType,
+		handler: HookHandler<Extract<E, { type: TType }>, Ctx>,
+	): () => void;
+
+	emit<TEvent extends E>(
+		event: TEvent,
+		signal?: AbortSignal,
+	): Promise<ResultOf<TEvent> | undefined>;
+
+	addCleanup(cleanup: () => void | Promise<void>): () => void;
+
+	clear(): Promise<void>;
+	dispose(): Promise<void>;
+}
+```
+
+Important split:
+
+- `observe()` sees all events, read-only, return ignored.
+- `on(type, handler)` participates in that event’s semantics.
+- `emit(event)` is the only thing `AgentHarness` calls.
+- `clear()` removes observers/handlers and runs cleanups.
+
+## Default implementation internals
+
+```ts
+class DefaultAgentHarnessHooks<E extends HookEvent<string, unknown>, Ctx>
+	implements AgentHarnessHooks<E, Ctx> {
+	context: Ctx;
+
+	private observers = new Set<HookObserver<E, Ctx>>();
+	private handlers = new Map<string, Set<HookHandler<any, Ctx>>>();
+	private cleanups = new Set<() => void | Promise<void>>();
+
+	constructor(ctx: Ctx) {
+		this.context = ctx;
+	}
+
+	setContext(ctx: Ctx): void {
+		this.context = ctx;
+	}
+
+	observe(handler: HookObserver<E, Ctx>): () => void {
+		this.observers.add(handler);
+		return () => this.observers.delete(handler);
+	}
+
+	on(type, handler): () => void {
+		let handlers = this.handlers.get(type);
+		if (!handlers) {
+			handlers = new Set();
+			this.handlers.set(type, handlers);
+		}
+		handlers.add(handler);
+		return () => handlers.delete(handler);
+	}
+
+	async emit(event, signal?) {
+		for (const observer of this.observers) {
+			await observer(event, this.context, signal);
+		}
+
+		switch (event.type) {
+			case "context":
+				return this.emitContext(event, signal);
+			case "before_provider_request":
+				return this.emitBeforeProviderRequest(event, signal);
+			case "before_provider_payload":
+				return this.emitBeforeProviderPayload(event, signal);
+			case "before_agent_start":
+				return this.emitBeforeAgentStart(event, signal);
+			case "tool_call":
+				return this.emitToolCall(event, signal);
+			case "tool_result":
+				return this.emitToolResult(event, signal);
+			case "session_before_compact":
+			case "session_before_tree":
+				return this.emitFirstCancelOrLast(event, signal);
+			default:
+				await this.emitObservationHandlers(event, signal);
+				return undefined;
+		}
+	}
+}
+```
+
+Internal casts are acceptable inside the implementation because `Map<string, ...>` loses specificity. Public API remains typed.
+
+## Mutation semantics
+
+### Observation
+
+```ts
+await hooks.emit({ type: "message_end", message }, signal);
+```
+
+Observers run. `message_end` handlers run. Return ignored unless that event later gets a result type.
+
+### Context transform
+
+Handlers run in order. Each sees current messages.
+
+```ts
+let current = event;
+
+for (const handler of handlers("context")) {
+	const result = await handler(current, ctx, signal);
+	if (result?.messages) {
+		current = { ...current, messages: result.messages };
+	}
+}
+
+return current.messages === event.messages ? undefined : { messages: current.messages };
+```
+
+### Provider request / payload
+
+Sequential transform. Each handler sees previous output.
+
+```ts
+let current = event;
+
+for (const handler of handlers("before_provider_payload")) {
+	const result = await handler(current, ctx, signal);
+	if (result !== undefined) {
+		current = { ...current, payload: result.payload };
+	}
+}
+
+return changed ? { payload: current.payload } : undefined;
+```
+
+### Before agent start
+
+Collect injected messages, chain system prompt.
+
+```ts
+let systemPrompt = event.systemPrompt;
+const messages = [];
+
+for (const handler of handlers("before_agent_start")) {
+	const result = await handler({ ...event, systemPrompt }, ctx, signal);
+	if (result?.messages) messages.push(...result.messages);
+	if (result?.systemPrompt !== undefined) systemPrompt = result.systemPrompt;
+}
+
+return messages.length || systemPrompt !== event.systemPrompt
+	? { messages, systemPrompt }
+	: undefined;
+```
+
+### Tool call
+
+Sequential, early exit on block.
+
+```ts
+for (const handler of handlers("tool_call")) {
+	const result = await handler(event, ctx, signal);
+	if (result?.block) return result;
+}
+```
+
+### Tool result
+
+Sequential patch accumulation. Each handler sees current patched result.
+
+```ts
+let current = event;
+let modified = false;
+
+for (const handler of handlers("tool_result")) {
+	const result = await handler(current, ctx, signal);
+	if (!result) continue;
+
+	current = {
+		...current,
+		content: result.content ?? current.content,
+		details: result.details ?? current.details,
+		isError: result.isError ?? current.isError,
+	};
+
+	modified = true;
+}
+
+return modified
+	? { content: current.content, details: current.details, isError: current.isError }
+	: undefined;
+```
+
+### Session-before events
+
+Sequential, early exit on cancel.
+
+```ts
+let last;
+
+for (const handler of handlers(event.type)) {
+	const result = await handler(event, ctx, signal);
+	if (!result) continue;
+	last = result;
+	if (result.cancel) return result;
+}
+
+return last;
+```
+
+## Harness usage
+
+Harness only does this:
+
+```ts
+await this.hooks.emit(event, signal);
+```
+
+or:
+
+```ts
+const result = await this.hooks.emit({ type: "context", messages }, signal);
+return result?.messages ?? messages;
+```
+
+Harness does not store handlers, chain listeners, or know extension policy.
+
+## Context
+
+Context is a normal object, not rebuilt per emit.
+
+```ts
+const hooks = new CodingAgentHooks({
+	harness: harnessFacade,
+	session: sessionFacade,
+	ui: noUiFacade,
+});
+```
+
+Later:
+
+```ts
+hooks.setContext({
+	...hooks.context,
+	ui: tuiFacade,
+});
+```
+
+For dynamic state, prefer stable facades/methods over getter maze:
+
+```ts
+interface CodingAgentHookContext {
+	harness: HarnessFacade;
+	session: SessionFacade;
+	ui: UiFacade;
+	models: ModelFacade;
+}
+```
+
+Per-run `signal` is passed as the third handler arg.
+
+## Extension loading later
+
+Extension loading can live next to harness and construct hooks:
+
+```ts
+const hooks = await loadExtensions({
+	paths,
+	context,
+	hooks: new CodingAgentHooks(context),
+});
+const harness = new AgentHarness({ ..., hooks });
+```
+
+The loader registers into hooks:
+
+```ts
+hooks.on("context", handler);
+hooks.on("tool_call", handler);
+hooks.addCleanup(cleanup);
+```
+
+For reload:
+
+```ts
+await hooks.clear();
+const nextHooks = await loadExtensions(...);
+harness.setHooks(nextHooks); // idle-only if supported
+```
+
+## Poking holes
+
+### 1. Error policy must be explicit
+
+Existing coding-agent catches extension errors, reports them, and continues. New hooks need the same policy, likely:
+
+```ts
+errorMode: "continue" | "throw"
+onError(error)
+```
+
+For coding-agent, default should be `"continue"`.
+
+### 2. Source metadata matters
+
+Existing runner knows which extension produced an error/resource/tool. Plain `on()` loses that unless we add registration metadata or scopes.
+
+Probably needed:
+
+```ts
+const scope = hooks.createScope({ sourceInfo });
+scope.on("context", handler);
+scope.addCleanup(...);
+```
+
+Or `on(type, handler, { sourceInfo })`.
+
+### 3. Some extension capabilities are registries, not hooks
+
+These are not covered by `emit()` and should stay as registries on `CodingAgentHooks` or an extension host:
+
+- tools
+- commands
+- shortcuts
+- flags
+- message renderers
+- provider registrations
+- OAuth providers
+- custom model providers
+
+That is fine. They do not belong in `AgentHarness`.
+
+### 4. Existing coding-agent events can be represented
+
+No blocker for:
+
+- `context`
+- `before_provider_request`
+- `after_provider_response`
+- `before_agent_start`
+- `message_end`
+- `tool_call`
+- `tool_result`
+- `input`
+- `user_bash`
+- `resources_discover`
+- `session_before_*`
+- `session_*`
+- model/thinking selection events
+- agent/turn/message/tool lifecycle events
+
+They become additional event types handled by `CodingAgentHooks`.
+
+### 5. Need to preserve exact old semantics
+
+When porting coding-agent, special cases must be copied:
+
+- `input`: transform chain, `handled` short-circuits.
+- `user_bash`: first meaningful result wins.
+- `message_end`: replacement must keep same role.
+- `before_agent_start`: `ctx.getSystemPrompt()` must reflect current chained prompt.
+- `resources_discover`: aggregate paths and keep extension source.
+- `tool_call`: argument mutation remains visible to later handlers.
+- `tool_result`: later handlers see prior patches.
+
+The design allows all of that, but the default/coding hooks implementation must encode it.
+
+### 6. `emit()` switch can miss custom mutation events
+
+If a subclass adds a result-producing event but forgets to override `emit()`, it will behave observationally. Tests should catch this. Could add a protected strategy registry later if this becomes error-prone, but not initially.
+
+### 7. Observer semantics are intentionally limited
+
+Observers see the original emitted event once. They do not see every intermediate mutation. If something needs final transformed state, emit a separate final event or use an event-specific handler.
+
+## Verdict
+
+This design can implement a new coding-agent. It is simpler than the current runner, keeps harness clean, and preserves the important extension capabilities as long as `CodingAgentHooks` adds source-aware scopes, registries, cleanup, and the exact old event semantics.
+
+--- Comments ---
+
+Thread hn2xk0tzhj on "addCleanup(cleanup"
+  [tmluyaub9v] Owner (2026-05-14T12:55:45.500Z): cleanup should be passed along optionally to on/observe