@cad0p/pi-tree-navigator 0.1.0-20260731.0

package/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [0.1.0] - 2026-05-25
+
+<!-- USER-EDITABLE SECTION START -->
+
+Initial release.
+
+`navigate_tree` is an agent-callable pi tool with three actions:
+
+- `anchor` — label the current point in the conversation as a milestone.
+- `rewind` — collapse work between an anchor and the current leaf into a model-generated `branch_summary`, freeing context.
+- `list` — show all anchors on the active branch with cumulative context %.
+
+Designed for long autonomous sessions where the agent itself decides when to summarize. Survives mid-loop rewinds (the next assistant turn within the same `prompt()` call sees the reduced context) and produces structurally valid Anthropic chains by injecting a synthetic `tool_use` to pair with the rewind's `tool_result`.
+
+User-visible specifics worth knowing on day one:
+
+- Anchor names are kebab-case (lowercase alphanumeric segments separated by single hyphens; max 40 chars). Re-anchoring with a name already on the active branch moves the prior label to the new leaf rather than duplicating it; the same move-on-collision applies to `rewind`'s `labelEnd`.
+- `rewind` requires a `summaryFocus` of ≥20 chars after trim; the rejection message lists what the focus should preserve so the agent can self-correct without user intervention.
+- The `branch_summary` boilerplate strip in `list` hints is sentinel-anchored — a user-authored doc whose first H2 happens to be `## Goal` is preserved untouched.
+- If the `AgentSession.prototype` patch isn't installed (typically only after a pi internals shape change), `list` and `rewind` surface a `⚠ reflection bootstrap missing` warning. The hint suggests `/reload` first (lighter — re-runs the prototype patch on the current process) and `Restart pi` as the heavier-handed alternative.
+
+<!-- USER-EDITABLE SECTION END -->

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Pier Carlo Cadoppi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,123 @@
+# pi-tree-navigator
+
+🌳 Agent-callable session tree navigation for [pi](https://github.com/badlogic/pi-mono).
+
+Lets a pi agent anchor named milestones in its own conversation, then collapse work between them into a model-generated `branch_summary` to free up context — without tripping Anthropic's `tool_use` ↔ `tool_result` validation, and with the freed context immediately available to the next assistant turn (even within the same `prompt()` call).
+
+## Install
+
+Stable npm release:
+
+```bash
+pi install npm:@cad0p/pi-tree-navigator
+```
+
+Pre-release npm snapshots from `main` are published with the `next` dist-tag:
+
+```bash
+pi install npm:@cad0p/pi-tree-navigator@next
+```
+
+You can also install directly from the git source when testing unreleased branches:
+
+```bash
+pi install git:github.com/cad0p/pi-tree-navigator
+```
+
+## Publishing
+
+This repo uses [`cad0p/semver-calver-release`](https://github.com/cad0p/semver-calver-release)'s npm-package workflow:
+
+- Pushes to `main` compute the next hybrid SemVer + CalVer version, tag a GitHub prerelease, and publish to npm with the `next` dist-tag.
+- Curated release PRs from `release/from-v*` branches bump the base `package.json` version and publish stable npm releases.
+- npm publishing uses GitHub OIDC / npm trusted publishing via `.github/workflows/release.yml` (`id-token: write`) and `publishConfig.access: public`.
+
+### Requirements
+
+- **pi 0.74+** with at least one model provider configured.
+- Peer dependencies (the source of truth is `package.json` `peerDependencies`):
+  - `@earendil-works/pi-coding-agent >=0.74.0`
+  - `@earendil-works/pi-agent-core >=0.74.0`
+  - `typebox ^1.0.0` (used to declare the tool's parameter schema; bundled with pi but listed explicitly so a standalone install resolves correctly).
+- The reflection bootstrap depends on five plain (not `#`-private) internal pi/agent fields: `AgentSession.prototype.prompt`, `agent.state.messages`, `agent.state.systemPrompt`, `agent.state.tools`, and `agent.prepareNextTurn`. Verified against pi 0.75.x.
+
+## What you get
+
+A single agent-callable tool, `navigate_tree`, with three actions:
+
+| action | params | effect |
+|---|---|---|
+| `anchor` | `name` | Label the current point in the conversation as a milestone. |
+| `rewind` | `labelStart`, `labelEnd`, `summaryFocus` | Collapse work between `labelStart` and the current leaf into a `branch_summary` entry. The summary is itself labeled with `labelEnd`, so you can chain rewinds. `summaryFocus` is required (non-trivial focus required; floor enforced at runtime by `MIN_SUMMARY_FOCUS_LENGTH`). Despite the verb, `rewind` does not restore prior state — it forks a sibling branch from `labelStart` and continues forward from a model-generated summary; the original subtree is preserved on disk but no longer on the active path. |
+| `list` | — | Show all anchors on the active branch with cumulative context %. |
+
+`name` (written by `anchor`) and `labelEnd` (written by `rewind`) both share the reserved `anchor:` label prefix; `labelStart` resolves against that same namespace. Every label written by `anchor` and every `labelEnd` written by `rewind` is referenceable by any subsequent `rewind`'s `labelStart`, and `list` shows all of them.
+
+## How it works
+
+A typical autonomous-loop pattern:
+
+```
+agent: navigate_tree(action="anchor", name="impl-start")
+  → [anchor 'impl-start'] set at 1.9% of 1.0M (after: “implement the parser”)
+
+agent: ...does work, runs tools, accumulates context to 30%...
+
+agent: navigate_tree(action="rewind", labelStart="impl-start", labelEnd="impl-end",
+                     summaryFocus="record only the public API of the parser
+                                   and the open issue with edge case X")
+  → [rewind 'impl-start' → 'impl-end'] · context 30.4% → 4.1% of 1.0M
+  → A branch_summary recording the work just collapsed has been appended
+    to your context. Items under '### Done' are complete. ...
+
+agent: ...continues with the freed context, the next API call is back at ~4%...
+```
+
+The freed context is available to the **next assistant turn within the same `prompt()` call**, not just on the next user prompt. This is the key feature — autonomous agents don't have to wait for a user round-trip to benefit from a rewind.
+
+## Implementation notes
+
+Why this is more involved than just calling pi's `branchWithSummary`:
+
+1. **Anthropic's tool_use ↔ tool_result pairing.** When a tool call rewinds the session tree, the tool's own `tool_use` lives in the assistant message that issued it — which `branchWithSummary` puts on the abandoned branch. Pi unconditionally writes the tool's `tool_result` to the new branch, leaving the result orphaned. Anthropic 400s the next API call with `Improperly formed request`. The fix is to inject a synthetic assistant message whose single `tool_call` has the same id as the in-flight call, *after* `branchWithSummary` but *before* the tool returns. Pi then writes the real `tool_result` as a child of that synthetic assistant — and the chain stays structurally valid.
+
+2. **In-loop context refresh.** Pi's `Agent` class snapshots `state.messages` once at the start of `prompt()` and pushes new messages onto its own array. A rewind issued mid-loop wouldn't reduce the next API call's size until the user sent a fresh prompt. We wire `agent.prepareNextTurn` from a prototype patch on `AgentSession.prototype.prompt`, returning a fresh context built from `sessionManager.buildSessionContext()` between every turn boundary. After a rewind, the very next assistant turn within the same `prompt()` sees the rewound chain.
+
+3. **Reflection bootstrap.** Pi's slash-command `navigateTree` has access to `commandCtx.navigateTree`, which mutates `agent.state.messages`. Tool executes don't get that ctx, so we capture every `AgentSession` instance via the prompt patch and replicate the mutation manually. Without it, the on-disk leaf moves but `agent.state.messages` stays stale.
+
+4. **`summaryFocus` is mandatory.** The summary is the only thing the agent will see of the collapsed work. The first time the agent uses `rewind`, blanket prompts produce vague summaries; subsequent rewinds are weaker. Forcing the agent to articulate `summaryFocus` (passed to pi's `generateBranchSummary` as `customInstructions`) measurably improves what survives.
+
+### Synthetic assistant token bias
+
+The synthetic assistant we inject after each rewind carries the **post-rewind chain estimate** in `usage.totalTokens` (so `estimateContextTokens` reads a sensible baseline immediately after the move). The synthetic itself adds a ~50-token toolCall block re-emitted on every subsequent turn until the next rewind — that overhead is **not** reflected in any `usage.*` field, so future `estimateContextTokens` calls understate the chain by ~50 tokens until the next assistant turn writes a fresh usage block. Negligible at typical anchor cadence; mention if you're benchmarking exact token deltas, ignore otherwise.
+
+## Limitations
+
+- **Brittle to pi version bumps.** The fix uses five independent reflection points on internals that aren't part of pi's public API: `AgentSession.prototype.prompt`, `agent.state.messages`, `agent.state.systemPrompt`, `agent.state.tools`, and `agent.prepareNextTurn`. If a future pi release renames any of these, switches them to private (`#`) fields, or restructures the class hierarchy, this breaks. The extension fails loudly: `anchor` still works, `rewind` reports `⚠ reflection bootstrap missing — the rewind landed on disk but the next assistant turn may still see the pre-rewind context. Run \`/reload\` (or restart pi) to recover.`, and you'd see context corruption return on the next prompt.
+
+- **Anchor early in the turn.** Whatever's in `agent.state.messages` *before* the `anchor` tool call stays in the kept chain. Everything after gets summarized. Anchor at the *start* of a stage for maximum context savings.
+
+- **Abandoned branches grow the JSONL forever.** Each rewind preserves the abandoned subtree on disk. Session files get bigger over time even as live context shrinks. For very long autonomous runs (days), session files can hit hundreds of MB.
+
+- **Tested against Anthropic and Kiro providers.** The synthetic-tool_use trick is specifically for Anthropic's strict tool_use/tool_result pairing; the synthetic's `stopReason: "toolUse"` survives Kiro's `normalizeMessages` filter. Other providers may have different validation rules — untested.
+
+- **Loading the extension monkey-patches `AgentSession.prototype.prompt` globally.** Every session in the host pi process picks up the patch on import, including sessions that never call `navigate_tree`. The patch is install-on-import and not reversible within a running pi process; restart pi to fully unload it.
+
+- **`anchor:` is a reserved label prefix.** Any label written via pi's `/label` command or by another extension that begins with `anchor:` will be picked up by `list` and addressable by `rewind`'s `labelStart` / `labelEnd`. Avoid the prefix in manually-set labels.
+
+- **Disk-fault during `rewind` (rare).** Pi's `branchWithSummary` advances the in-memory leaf before persisting the new entry to disk. If pi's session-write fails mid-call (full disk, FS error on a persisted session), the in-memory leaf has already moved past the original assistant turn but the synthetic-assistant injection in this extension never runs — pi's tool-result then lands without a matching tool_use, surfacing as the same `context_length_exceeded` 400 the synthetic exists to prevent. Production risk: low (in-memory tests don't reach this case; pi's session-write is robust on POSIX disk). Tracked for an additional salvage layer wrapping `branchWithSummary` itself in v0.2.0.
+
+## Development
+
+```bash
+bun install
+bun test          # helpers + dispatch / reflection bootstrap / salvage path
+bunx biome check extensions/
+bunx tsc --noEmit
+```
+
+Tests cover `extensions/navigate-tree/helpers.ts` (pure helpers in `helpers.test.ts`) and `extensions/navigate-tree/index.ts` (action dispatch, schema shape, synthetic-assistant injection, reflection bootstrap, salvage path — in `index.test.ts`). The `summarize` factory option injects a stub for `generateBranchSummary` so no real LLM call fires during rewind tests. Additional manual e2e validation against pi 0.75.x is recommended for any pi version bump (the reflection bootstrap depends on internal field shapes).
+
+## License
+
+MIT — see [LICENSE](LICENSE).

package/extensions/navigate-tree/helpers.ts

@@ -0,0 +1,135 @@
+/**
+ * Pure helpers for the navigate-tree extension.
+ *
+ * Imported by `./index.ts` and `./helpers.test.ts`. Pi's extension loader
+ * loads `./index.ts` and ignores everything else in this directory unless
+ * referenced from there — so this file isn't loaded as a separate extension.
+ *
+ * No pi runtime imports — these are pure functions over plain JS values.
+ */
+
+// ---------------------------------------------------------------------------
+// Exported boundary constants below (MAX_NAME_LENGTH, MAX_BOILERPLATE_LEAD_IN).
+//
+// Stability: these are internal tunables. Exported only so the test suite
+// can pin boundary cases by constant rather than literal. Re-tuning is
+// NOT a semver-breaking change for this package — production callers
+// should rely on the registered `navigate_tree` tool surface, not import
+// these constants directly.
+// ---------------------------------------------------------------------------
+
+// Hard cap on label-name length. 40 chars accommodates descriptive names
+// (e.g. 'parser-edge-case-investigation', 31 chars) while keeping list
+// output column-friendly under common terminal widths and preventing a
+// runaway label string from poisoning the JSONL on disk.
+export const MAX_NAME_LENGTH = 40;
+/**
+ * Kebab-case anchor name: lowercase alphanumeric segments separated by
+ * single hyphens. No leading / trailing / double hyphens (the `isValidName`
+ * test suite enforces these rejections). Mirrors the pattern documented in
+ * the tool description and README — if this regex relaxes, both surfaces
+ * need the same update.
+ */
+const NAME_RE = /^[a-z0-9]+(-[a-z0-9]+)*$/;
+
+// Maximum lead-in distance for pi's branch-summary boilerplate marker.
+// Pi's standard prelude ("The user explored a different conversation
+// branch...") fits in the first ~150 chars; 200 is a generous upper bound.
+// A "## Goal" found later than this is treated as in-content prose, not
+// the boilerplate marker, and the strip is a no-op.
+export const MAX_BOILERPLATE_LEAD_IN = 200;
+
+// Sentinel that pi's branch-summary prelude always starts with. Gating
+// the strip on this prefix makes the helper a no-op for any non-
+// boilerplate text that happens to contain `## Goal` early (e.g. a
+// user-authored markdown doc whose first H2 is "Goal"). Verified
+// against pi 0.75.5's `dist/core/compaction/branch-summarization.js`;
+// if pi reworks the prelude wording, the strip falls back to a no-op
+// (`findLabelHint` shows the unmodified prelude). Verify when bumping
+// the peer-dep floor.
+const BRANCH_SUMMARY_SENTINEL =
+  "The user explored a different conversation branch";
+
+// Pi 0.75.5's branch-summary boilerplate places `## Goal` after the prelude
+// (verified against `dist/core/compaction/branch-summarization.js`). Used to
+// anchor the strip cut-point in `stripBranchSummaryBoilerplate` — a single
+// constant so the indexOf probe and the slice-length advance can't drift.
+const GOAL_HEADER = "## Goal";
+
+/** Validate a kebab-case name suitable for use as a navigate-tree label. */
+export function isValidName(s: unknown): s is string {
+  return (
+    typeof s === "string" &&
+    s.length > 0 &&
+    s.length <= MAX_NAME_LENGTH &&
+    NAME_RE.test(s)
+  );
+}
+
+/** Truncate text to a one-line preview, collapsing whitespace. */
+export function toOneLine(text: string, maxLen: number): string | null {
+  const t = text.replace(/\s+/g, " ").trim();
+  if (t.length === 0) return null;
+  if (maxLen <= 1) return t.length > 0 ? "…" : null;
+  return t.length > maxLen ? `${t.slice(0, maxLen - 1)}…` : t;
+}
+
+/**
+ * Format token count as a percentage with one decimal, or as `Nk` if no window.
+ */
+export function formatPct1(tokens: number, contextWindow: number): string {
+  if (contextWindow <= 0) return `${(tokens / 1000).toFixed(1)}k`;
+  return `${((tokens / contextWindow) * 100).toFixed(1)}%`;
+}
+
+/** Format a context window size as `1.0M` or `200k`. Empty string when unknown. */
+export function formatWindow(contextWindow: number): string {
+  if (contextWindow <= 0) return "";
+  if (contextWindow >= 1_000_000)
+    return `${(contextWindow / 1_000_000).toFixed(1)}M`;
+  return `${(contextWindow / 1000).toFixed(0)}k`;
+}
+
+export function formatContextDelta(
+  beforeTokens: number,
+  afterTokens: number,
+  contextWindow: number,
+): string {
+  if (contextWindow > 0) {
+    return `context ${formatPct1(beforeTokens, contextWindow)} → ${formatPct1(afterTokens, contextWindow)} of ${formatWindow(contextWindow)}`;
+  }
+  return `tokens ${beforeTokens} → ${afterTokens}`;
+}
+
+/**
+ * Strip pi's standard branch-summary boilerplate ("The user explored a
+ * different conversation branch...") so the hint shows the actual content.
+ */
+export function stripBranchSummaryBoilerplate(text: string): string {
+  if (!text.startsWith(BRANCH_SUMMARY_SENTINEL)) return text;
+  const goalIdx = text.indexOf(GOAL_HEADER);
+  if (goalIdx > 0 && goalIdx < MAX_BOILERPLATE_LEAD_IN) {
+    return text.slice(goalIdx + GOAL_HEADER.length);
+  }
+  return text;
+}
+
+/**
+ * Extract a string from a message-like content field. Handles both the
+ * legacy string shape and the modern array-of-blocks shape, joining all
+ * text blocks with spaces.
+ */
+export function extractTextContent(content: unknown): string {
+  if (typeof content === "string") return content;
+  if (!Array.isArray(content)) return "";
+  return content
+    .filter(
+      (c): c is { type: "text"; text: string } =>
+        typeof c === "object" &&
+        c !== null &&
+        (c as { type?: string }).type === "text" &&
+        typeof (c as { text?: unknown }).text === "string",
+    )
+    .map((c) => c.text)
+    .join(" ");
+}