pi-goal-x 0.6.0

package/docs/architecture.md

@@ -0,0 +1,239 @@
+# pi-goal Architecture
+
+This document describes the shipped `pi-goal` extension as it exists now. It focuses on implemented behavior.
+
+## Runtime shape
+
+`extensions/goal.ts` is the orchestration layer. It owns pi integration points:
+
+- slash commands;
+- tool registration;
+- session events;
+- auto-continue timers;
+- usage accounting;
+- coordination with extracted prompt, storage, policy, questionnaire, goal-pool, and widget modules.
+
+The runtime is a focused-goal view over a project goal pool:
+
+```ts
+let goalsById: Map<string, GoalRecord>;
+let focusedGoalId: string | null;
+```
+
+`goalsById` is reconstructed from `.pi/goals/active_goal_*.md` plus compatible legacy session entries. `focusedGoalId` is reconstructed from branch-local `pi-goal-focus` session entries. The focused id is not serialized into goal markdown.
+
+Reusable logic is split into smaller modules:
+
+| Module | Responsibility |
+|---|---|
+| `goal-record.ts` | Goal record types, creation, cloning, usage normalization, persisted-record migration |
+| `goal-pool.ts` | Open-goal pool helpers, focus resolution, list output, selector labels, unfocused summaries |
+| `goal-core.ts` | Compact display formatting, status labels, objective title cleanup |
+| `goal-draft.ts` | Lightweight confirmation prompt, plain-text draft confirmation report, proposal validation, drafting-stage tool gate |
+| `goal-policy.ts` | Lifecycle policy, abort/pause/resume/complete validation, compaction policy, full result reports |
+| `goal-auditor.ts` | Independent pi auditor agent prompt/config/decision parsing and completion audit execution |
+| `goal-questionnaire.ts` | Built-in questionnaire types, normalization, answer formatting, TUI question runner, proposal confirmation dialog, question-tool registration |
+| `goal-tool-names.ts` | Published tool-name constants, active-tool lists, post-stop allowlist, goal work-tool list, question-like tool detection |
+| `prompts/goal-prompts.ts` | Active-goal, continuation, tweak-drafting, and stale-checkpoint prompt builders |
+| `storage/goal-files.ts` | Goal path safety, serialization/parsing, active-file scanning, active-file writes, archive writes, prompt-body merge from disk |
+| `widgets/goal-widget.ts` | Above-editor Goal Beacon component, blocker/status rendering, `+N open` and unfocused rendering |
+| `widgets/goal-notifications.ts` | Widget-style notification text for goal lifecycle toasts |
+
+## Lifecycle
+
+```text
+/user command
+  ├─ /goals or /sisyphus
+  │    └─ confirmationIntent = {focus, originalTopic, startedAt}
+  │         ├─ agent clarifies, researches, or grills only when needed
+  │         ├─ targeted reconnaissance is prompt-guided, not hard-blocked
+  │         └─ propose_goal_draft validates focus/objective and asks user to confirm
+  │              ├─ Continue Chatting: keep clarifying without creating a goal
+  │              └─ Confirm: create active goal, write .pi/goals file, focus it, print full objective
+  │
+  ├─ /goals-set or /sisyphus-set
+  │    └─ direct user command creates active goal, writes .pi/goals file, focuses it, and starts execution
+  │
+  ├─ focused active goal
+  │    ├─ autoContinue queues checkpoint turns
+  │    ├─ pause_goal pauses on real blockers
+  │    ├─ abort_goal aborts/archives obsolete or impossible goals
+  │    └─ update_goal starts independent auditor; <approved/> archives and prints full completion report
+  │
+  ├─ paused goal
+  │    ├─ /goal-resume restarts autoContinue
+  │    ├─ update_goal can complete from existing evidence
+  │    └─ abort_goal can archive without resuming
+  │
+  ├─ multiple open goals
+  │    ├─ /goal-list shows the project goal pool
+  │    ├─ /goal-focus chooses the session focus
+  │    └─ unfocused sessions guide the user to choose instead of letting the agent decide
+  │
+  └─ /goal-clear or /goal-abort archives the focused goal or cancels drafting
+```
+
+## Goal pool and session focus
+
+The disk layout already supports multiple active files. The extension now treats those files as the durable project-level open goal pool:
+
+```text
+.pi/goals/active_goal_<timestamp>_<id>.md
+```
+
+`readActiveGoalPool(ctx)` scans that directory, ignores invalid files and symlinks, parses each safe active file, sanitizes metadata paths, drops completed records, and returns a deterministic `Map<goalId, GoalRecord>`.
+
+Session focus is separate. Focus changes append a custom session entry:
+
+```ts
+{
+  version: 1,
+  focusedGoalId: string | null,
+  reason: "created" | "selected" | "resumed" | "completed" | "cleared" | "aborted" | "migrated"
+}
+```
+
+Because this is stored with `pi.appendEntry("pi-goal-focus", ...)`, it is session/branch-local and is not sent to the LLM. On `session_start` and `session_tree`, `loadState(ctx)` scans `ctx.sessionManager.getBranch()` for the latest focus entry, scans active goal files, and resolves focus as follows:
+
+1. Use a valid focused id from the latest focus entry.
+2. If the latest focus entry explicitly has `focusedGoalId: null`, or points at a missing/stale goal, remain unfocused.
+3. If no focus entry exists, merge a compatible legacy `pi-goal-state { version: 3, goal }` goal and focus it. If disk already has the same id, the disk record wins and the legacy session record only supplies focus.
+4. If no focus entry exists and exactly one open goal exists, auto-focus it for compatibility.
+5. If multiple open goals exist and no valid focus exists, remain unfocused until `/goal-focus`, `/goal-resume`, `/goal-clear`, `/goal-abort`, `/goal-pause`, or `/goal-tweak` asks the user to choose.
+
+Focus is human-owned. No agent tool can switch focus. Lifecycle tools operate only on the focused goal.
+
+## Goal styles
+
+### Regular goal
+
+Regular goals are open-ended objectives. The agent decides the next concrete action each checkpoint turn, then completes only after the objective is actually satisfied.
+
+### Sisyphus goal
+
+Sisyphus is a light variant of the same goal lifecycle. It does not have a separate execution state machine or step counter. The only differences are prompt/criteria level:
+
+- drafting asks for a patient ordered-execution style when relevant;
+- continuations remind the agent not to rush, skip, or invent preflight steps;
+- completion still uses `update_goal(status="complete")`, with the stricter expectation that the whole ordered objective is actually satisfied.
+
+The legacy `step_complete` tool remains registered as a hidden compatibility no-op for old transcripts, but it is not exposed as an active work tool and is not required for completion.
+
+## Drafting and confirmation
+
+Drafting is now a lightweight user-intent confirmation conversation. For `/goals` and `/sisyphus`, the runtime stores only a thin session-local `confirmationIntent` with the requested focus, original topic, and start time. The agent may ask a focused question when the topic is vague, perform targeted read-only research when it improves the goal contract, grill assumptions or ordered steps, or proceed directly to `propose_goal_draft` when the request is already concrete.
+
+`propose_goal_draft` enforces:
+
+- a confirmation intent must be active;
+- objective must be non-empty;
+- `sisyphus` must match the command the user invoked.
+
+A deprecated optional `draftId` parameter is accepted for compatibility but ignored; normal goal confirmation no longer depends on hidden prompt identity. Confirming a draft creates a new active goal and focuses it, leaving other active files untouched. Confirmation UI errors fail closed: the goal is not created and confirmation remains active. After confirmation, normal work tools are available for execution immediately.
+
+## Command focus behavior
+
+- `/goals` and `/sisyphus` start discussion-based confirmation before creating a new focused goal.
+- `/goals-set` and `/sisyphus-set` directly create and focus a new open goal from the supplied objective.
+- `/goal-list` prints all open goals with id, status, mode, usage, objective title, path, and a focus marker.
+- `/goal-focus` uses `ctx.ui.select` when multiple goals are open and updates only session focus.
+- `/goal-status` and `/goal` show the focused goal plus an `other open goals` hint.
+- `/goal-resume` resumes the focused paused goal; when unfocused with multiple open goals, it asks the user to choose. Choosing an already active goal only focuses it.
+- `/goal-clear` and `/goal-abort` archive only the focused/selected goal and never clear the whole pool at once.
+- During goal confirmation, `/goal-clear` and `/goal-abort` only cancel the confirmation flow; they do not archive an unrelated focused goal unless the user invokes a lifecycle command after confirmation is cancelled.
+- `/goal-tweak` revises only the focused active or paused goal; when unfocused with open goals, it asks the user to choose one.
+- `/goal-pause` also asks the user to choose when the session is unfocused and open goals exist.
+- `/goal-settings` opens extension settings. The current settings screen contains `auditor`, where provider/model/thinking_level are edited via free-text inputs.
+
+When `propose_goal_draft` asks for confirmation, the UI shows a full plain-text draft report rather than a Markdown preview. On confirmation, the result prints the full finalized objective in the conversation. The same objective is also written to the active goal file.
+
+## Tool visibility
+
+Tool visibility is recomputed whenever state changes. Built-in work tools remain registered in the base prompt so they can be used after a confirmed draft; lifecycle-specific gates decide whether a call is allowed.
+
+- Goal confirmation keeps `propose_goal_draft` stable and exposes `goal_question`, `goal_questionnaire`, and `get_goal` when structured clarification helps; workhorse tools are prompt-guided rather than hidden by a hard whitelist.
+- Tweak drafting exposes question tools, `get_goal`, and `apply_goal_tweak`.
+- Active goals expose `get_goal`, `update_goal`, `pause_goal`, and `abort_goal`.
+- Paused goals expose `get_goal`, `update_goal`, and `abort_goal`, so the agent can complete or abandon a paused goal without resuming substantive work.
+- Unfocused sessions with open goals expose no lifecycle mutation tools; prompts and status guide the user to `/goal-focus`.
+- `step_complete` is hidden legacy compatibility.
+- `create_goal` remains hidden and direct calls are rejected; normal creation goes through `propose_goal_draft`.
+
+The `tool_call` interceptor blocks:
+
+- non-`get_goal` tools after a stop tool has fired in the same turn.
+
+## Disk format
+
+Active and archived goal files live under `.pi/goals/`. Multiple active files may exist simultaneously.
+
+```text
+.pi/goals/active_goal_<timestamp>_<id>.md
+.pi/goals/archived/goal_<timestamp>_<id>.md
+```
+
+Each file has extension-owned metadata and a user-editable `# Goal Prompt` section. Before focused commands, tools, and lifecycle hooks act, the runtime re-reads the focused active file and reconciles lifecycle state from disk. External pause/archive/delete/status changes therefore win over stale memory and deleted active files are not resurrected. Prompt-body edits are still picked up from `# Goal Prompt`; session focus is never written to these files.
+
+Path safety checks reject absolute paths, traversal, NUL bytes, symlinks, and paths outside the goal directories.
+
+## Auto-continue and stop conditions
+
+When `autoContinue` is on, the extension queues continuation prompts after agent turns for the focused goal only. The loop stops or pauses when:
+
+- the agent calls `update_goal(status="complete")`;
+- the agent calls `pause_goal`;
+- the agent calls `abort_goal`;
+- the user invokes `/goal-pause`, `/goal-clear`, or `/goal-abort`;
+- the user aborts the turn;
+- a turn ends without meaningful goal-work tool activity.
+
+Continuation prompts include a goal id so stale prompts can be detected and neutralized. If focus changes or the goal is archived before a queued checkpoint runs, the checkpoint becomes stale and cannot drive task work.
+
+`get_goal`, question tools, and draft proposal tools are not meaningful progress for the empty-turn gate. Only lifecycle mutations and actual workhorse tools mark a turn as goal work for continuation purposes.
+
+## Completion output
+
+Completion is intentionally verbose in the tool result and guarded by an independent auditor agent. `update_goal(status="complete")` is valid for active and paused goals; paused goals do not need to be resumed just to record completion when existing evidence is sufficient.
+
+Before archiving, the tool starts a separate in-memory pi session with a focused auditor prompt. The auditor receives the objective, executor completion summary, and goal metadata, can inspect the workspace with `read`, `grep`, `find`, `ls`, and `bash`, and must end with exactly one marker:
+
+- `<approved/>` allows archiving;
+- `<disapproved/>`, no marker, an error, or abort rejects completion and leaves the goal open.
+
+The auditor uses the current/default model unless `.pi/goal-auditor.json` or `PI_GOAL_AUDITOR_PROVIDER`, `PI_GOAL_AUDITOR_MODEL`, and `PI_GOAL_AUDITOR_THINKING_LEVEL` override provider/model/thinking. `/goal-settings` opens a small UI menu with an `auditor` item; inside it, `provider`, `model`, and `thinking_level` each open a free-text input and save back to `.pi/goal-auditor.json`.
+
+The user sees:
+
+- a `Goal complete.` header;
+- the executor's optional completion summary/evidence;
+- the auditor's approval report;
+- the full current goal details.
+
+This mirrors creation: the finalized goal is visible when created, and the final report is visible when completed. The gate is intentionally semantic rather than paperwork-based: scaffold-only, alpha, generated-template, proxy-metric, build-only, or weakly verified completions should be disapproved by the auditor.
+
+## Tests
+
+Fast local tests live in `tests/` and run with:
+
+```bash
+npm test
+npm run check
+```
+
+They cover:
+
+- parsing and display helpers;
+- lightweight confirmation prompt and proposal gates;
+- questionnaire normalization and answer formatting;
+- tool-name constants and question-like detection;
+- lifecycle policy, including abort and paused-goal completion;
+- auditor config/prompt/marker parsing, including disapproval winning over approval;
+- goal-pool and focus resolution helpers;
+- active goal file scanning;
+- unfocused prompt guidance;
+- focused/unfocused widget rendering;
+- Sisyphus prompt-style behavior;
+- auto-continue empty-turn guard behavior;
+- full creation/completion report formatting.
+
+The `experiments/` harness provides end-to-end coverage with real pi sessions and model calls.

package/docs/goal-ts-refactor-test-strategy.md

@@ -0,0 +1,82 @@
+# Refactor and Test Baseline
+
+This document records the current safety net for the `pi-goal` componentization work.
+
+## Commands
+
+```bash
+npm test
+npm run check
+npm pack --dry-run
+```
+
+## Unit test coverage
+
+The fast local suite uses Node's built-in `node:test` runner and currently covers the core modules across:
+
+- `tests/goal-core.test.ts`
+- `tests/goal-draft.test.ts`
+- `tests/goal-policy.test.ts`
+- `tests/goal-questionnaire.test.ts`
+- `tests/goal-tool-names.test.ts`
+- `tests/goal-record.test.ts`
+- `tests/goal-files.test.ts`
+- `tests/goal-pool.test.ts`
+- `tests/goal-prompts.test.ts`
+- `tests/goal-notifications.test.ts`
+- `tests/goal-widget.test.ts`
+
+## Extracted modules
+
+`extensions/goal.ts` remains the orchestration layer. The following logic has been extracted and covered by tests:
+
+| Module | Covered behavior |
+|---|---|
+| `extensions/goal-record.ts` | Goal creation, normalization/migration, usage cloning, persisted record shape |
+| `extensions/goal-pool.ts` | Open-goal pool creation, deterministic ordering, explicit-null/stale focus resolution, disk-wins legacy migration, disk lifecycle reconciliation helpers, list and selector labels, unfocused summaries |
+| `extensions/goal-core.ts` | Compact duration/token/status display, objective-title cleanup |
+| `extensions/goal-draft.ts` | Lightweight confirmation prompt, draft summary, safe objective escaping, focus/mode gate, Sisyphus prompt-style guidance, drafting tool gate, multi-open draft creation allowance |
+| `extensions/goal-policy.ts` | Creation/completion-from-active-or-paused, abort/pause/resume/clear policy, multi-open creation slot allowance, compaction reminder, full creation/completion reports |
+| `extensions/goal-auditor.ts` | Independent pi auditor agent config parsing, prompt construction, approval marker parsing, and completion audit execution |
+| `extensions/goal-questionnaire.ts` | Question normalization, duplicate id handling, option filtering, recommended-index validation, answer formatting, confirm/cancel mapping, `goal_question` and `goal_questionnaire` registration |
+| `extensions/goal-tool-names.ts` | Published tool constants, active/paused/drafting tool lists, goal work-tool list, progress-tool list for empty-turn gating, post-stop allowlist, question-like tool detection |
+| `extensions/prompts/goal-prompts.ts` | Active-goal, continuation, tweak-drafting, stale-checkpoint, and unfocused multi-open prompt text |
+| `extensions/storage/goal-files.ts` | Safe goal paths, serialize/parse round trip, prompt-body disk edits, active-goal scans, active/archive writes |
+| `extensions/widgets/goal-widget.ts` | Goal Beacon rendering, Sisyphus style label, status/path lines, blocker/suggested-action display, `+N open` and unfocused guidance |
+| `extensions/widgets/goal-notifications.ts` | Widget-style notification text for goal lifecycle toasts |
+
+## Refactor rule
+
+1. Add or update tests before moving behavior.
+2. Extract pure helpers or narrow adapter boundaries first.
+3. Keep published tool names, slash commands, file formats, and UI semantics stable unless the user explicitly asks for a new public affordance such as `/goal-abort` or `abort_goal`.
+4. Run `npm test` and `npm run check` after each slice.
+5. Run `npm pack --dry-run` before release or packaging changes.
+
+## Remaining runtime-sensitive areas
+
+The following remain intentionally in `goal.ts` until a stronger mock `ExtensionAPI` / `ExtensionContext` harness exists:
+
+- pi command registration;
+- tool registration beyond the questionnaire pair;
+- session event hooks;
+- timers and auto-continue scheduling;
+- live TUI widget rendering.
+
+These areas are protected by TypeScript, focused unit helpers, and the end-to-end experiment harness rather than isolated component tests.
+
+## Multi-goal focus test notes
+
+The current suite specifically covers the multi-open goal architecture through pure helpers and storage seams:
+
+- focus entry normalization in `tests/goal-record.test.ts`;
+- active goal file scanning and invalid/symlink filtering in `tests/goal-files.test.ts`;
+- goal pool sorting, focus resolution, explicit no-focus/stale focus behavior, disk-wins legacy fallback, disk lifecycle merge, list output, and selector labels in `tests/goal-pool.test.ts`;
+- multi-open draft creation allowance, missing confirmation intent rejection, deprecated `draftId` compatibility, concrete-topic direct proposal guidance, and lightweight confirmation prompt text in `tests/goal-draft.test.ts`;
+- no-focus prompt guidance in `tests/goal-prompts.test.ts`;
+- continuation and compaction policy in `tests/goal-policy.test.ts`;
+- auditor marker/config/prompt behavior in `tests/goal-auditor.test.ts`;
+- drafting-phase lifecycle-tool suspension and progress-tool exclusion of `get_goal`, question tools, and draft proposal tools in `tests/goal-tool-names.test.ts`;
+- focused widget `+N open` and unfocused `/goal-focus` guidance in `tests/goal-widget.test.ts`.
+
+Release is intentionally separate from implementation validation. The local validation gate is `npm test`, `npm run check`, `npm pack --dry-run`, and `git diff --check`; `npm version`, `npm publish`, `git push`, and `pi update` only happen on explicit release request.

package/docs/pi-autoresearch-survey.md

@@ -0,0 +1,45 @@
+# Implemented Borrowed Patterns
+
+This document records external patterns that are actually implemented in `pi-goal` today.
+
+## From pi-codex-goal
+
+`pi-goal` uses several goal-loop stability patterns inspired by `pi-codex-goal`:
+
+- **Goal-id continuation markers**: continuation prompts include a goal id so stale prompts can be detected.
+- **Context interceptor**: stale continuation context is neutralized instead of letting an old goal keep driving the agent.
+- **Abort pause**: user abort / Ctrl-C pauses the active goal rather than leaving it in a misleading active state.
+- **Disk-backed active goal file**: the current objective is materialized on disk and can be audited outside the chat.
+
+## From pi-autoresearch
+
+`pi-goal` uses several autonomous-loop safety patterns inspired by `pi-autoresearch`:
+
+- **Empty-turn gate**: auto-continue does not advance when the agent did no meaningful goal-work tool activity.
+- **Post-compaction reminder**: after compaction, the next agent turn is reminded to re-read the objective and continue from actual artifacts/state.
+
+## pi-goal-specific work
+
+The current extension also adds behavior specific to goal drafting and lifecycle safety:
+
+- **Intent-before-run**: `/goals` and `/sisyphus` start a drafting discussion instead of immediate execution.
+- **Direct set shortcut**: `/goals-set` and `/sisyphus-set` immediately create a goal when the user already knows the final objective.
+- **Confirm-before-commit for discussions**: `propose_goal_draft` is the normal discussion-based creation path; `create_goal` stays hidden.
+- **Sisyphus as style**: `/sisyphus` and `/sisyphus-set` use the same lifecycle and tools as regular goals; they only change drafting/continuation wording and completion expectations.
+- **Full draft confirmation and creation output**: draft confirmation uses a plain-text report, and after confirmation the finalized objective is printed directly into the conversation.
+- **Full completion output**: completion prints a report directly into the conversation, including optional evidence and full goal details.
+- **Built-in question tools**: `goal_question` and `goal_questionnaire` provide package-local user-dialogue tools with `goal_` prefixes.
+- **Centralized tool names**: published tool names and allowlists live in `goal-tool-names.ts`.
+- **Questionnaire componentization**: normalization, answer formatting, proposal confirmation, and question-tool registration live in `goal-questionnaire.ts`.
+- **Widget module split**: the above-editor Goal Beacon and widget-style notification text live under `extensions/widgets/`.
+- **Record/prompt/storage split**: goal record normalization, prompt construction, and disk serialization now live in separate tested modules instead of the orchestration file.
+
+## Current validation
+
+Run locally:
+
+```bash
+npm test
+npm run check
+npm pack --dry-run
+```