@thurstonsand/pi-librarian 0.1.0 → 0.1.1

package/CONTEXT.md

@@ -9,9 +9,6 @@ The research agent spawned by the `librarian` tool; runs in a nested, in-memory
 One invocation of the librarian: query in, findings out, with a recorded tool-call trace.
 _Avoid_: session (reserved for pi sessions)
 
-**Run id**:
-The pi session uuid of a librarian run, so that it may be continued for follow up questions.
-
 **Findings**:
 The structured output of a librarian run, produced by `provide_results`: summary, locations, optional description.
 
@@ -29,4 +26,3 @@ _Avoid_: load, enable
 
 - A **Librarian run** ends with exactly one **Findings**, containing zero or more **Locations**.
 - **Attached tools** are the same **Repo tools** a **Librarian** uses, minus `provide_results`.
-- a **Librarian run** is persisted in its own session directory, identified by a **Run id**.

package/RELEASE.md

@@ -2,6 +2,16 @@
 
 # Release notes
 
+## 0.1.1
+
+Adds automated release and dependency maintenance infrastructure.
+
+### Added
+
+- Added GitHub Actions workflows for CI and tag-driven npm publishing.
+- Added Renovate configuration for dependency update PRs.
+- Added a helper script for annotated release tags.
+
 ## 0.1.0
 
 Initial release of `@thurstonsand/pi-librarian`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@thurstonsand/pi-librarian",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "type": "module",
   "description": "GitHub research subagent for pi: deep-dive specific repos, discover across the ecosystem",
   "license": "MIT",

package/docs/designs/02-continuable-runs.md

@@ -1,101 +0,0 @@
-# Continuable librarian runs
-
-## Status
-
-Draft
-
-## Decision Summary
-
-Persist librarian run transcripts as real pi session files in a librarian-owned directory, and add an optional `continue_from: <run uuid>` parameter to the `librarian` tool that reopens a prior run with its full research context. Follow-up questions ("just one more thing about that flow you traced") reuse everything the run already read instead of starting cold. One machinery for both cheap follow-ups and full second research legs.
-
-## Problem Statement / Background
-
-A librarian run often answers 90% of a question, and the main session discovers the missing 10% only after reading the findings — "you traced the D1 prepared-statement flow; does the same apply to batch()?" Today every run is ephemeral (`SessionManager.inMemory`, disposed on completion), so the follow-up pays full price: re-clone context, re-grep, re-read, re-derive. The run's accumulated context — files read, structure understood, dead ends eliminated — is exactly the asset a follow-up needs.
-
-pi already has the machinery: `SessionManager.create(cwd, sessionDir)` persists transcripts to any directory, `SessionManager.open(path)` reopens a specific file, and session files are named `<timestamp>_<sessionId>.jsonl` so a uuid resolves to a file by glob.
-
-## Goals
-
-- A follow-up to a prior run answers from that run's context when sufficient — potentially zero new tool calls — and researches further when not, with no special-casing between the two.
-- The run uuid is discoverable by both the main agent (in the tool result content) and the human (in the rendered output), so continuation survives compaction boundaries: the user can supply the uuid if the agent lost it.
-- Continuations work across pi restarts and from different main sessions.
-
-## Non-Goals
-
-- No integration with pi-sessions indexing/search in v1 (see Alternatives).
-- No transcript eviction policy in v1 (mirrors the checkout-cache decision; `rm -rf /tmp/pi-librarian/sessions` is the manual escape for default settings).
-- No lighter-weight result shape for cheap follow-ups — `provide_results` ceremony applies to every run.
-
-## Exposed Shape
-
-- `librarian` tool gains `continue_from?: string` — the uuid of a prior run. When present, the run resumes that transcript; `query` carries the follow-up question. `repos`/`owners` scope params remain usable.
-- Every run's findings content ends with a `run: <uuid>` line (for the agent), and the done-state render shows the same uuid as a muted line above the footer (for the human).
-- `LibrarianRunDetails` carries `runId`.
-- Tool description documents: pass `continue_from` for follow-ups to earlier research; omit it for new topics.
-
-## Design Decisions
-
-### 1. Transcripts are pi session files in a librarian-owned directory
-
-New runs use `SessionManager.create(cacheDir, <cacheDir>/sessions)` instead of `inMemory`. Continuations glob `<cacheDir>/sessions/*_<uuid>.jsonl` and reopen via `SessionManager.open(path)`. The directory lives outside pi's default session store, so runs never appear in resume pickers and are invisible to pi-sessions indexing.
-
-**Tradeoff**: research memory stays siloed from the pi-sessions "sessions are memory" thesis. Deliberate for v1 — see Alternatives.
-
-### 2. Continuation rebuilds the runtime, the transcript provides the memory
-
-Tools, system prompt, and model are never read from the transcript; each continuation re-registers the current repo tools, rebuilds the system prompt, and resolves the model fresh (configured → current session model — possibly a different model than the original run; pi handles resumed transcripts under a new model). Only the message history is resumed. This keeps continuation semantics identical to fresh runs — including `provide_results` enforcement and the reminder loop — and means prompt/tool improvements apply retroactively to old runs.
-
-### 3. One machinery for cheap follow-ups and big second legs
-
-No fast path. A follow-up that the prior context already answers naturally produces a zero-tool-call run ending in `provide_results`; a big second leg just runs longer. The system prompt gains a continuation expectation: runs may be resumed later; on resume, answer from prior context when it suffices, research further when it does not, and always finish with `provide_results`.
-
-### 4. uuid surfaced in two places, by design redundancy
-
-The agent reads `run: <uuid>` from the result content; the human sees it in the render. If the agent loses the uuid across a compaction boundary, the user can paste it back. Inelegant, deliberately so — one fallback beats zero.
-
-## Edge Cases & Failure Modes
-
-- **Unknown/missing `continue_from` uuid**: structured error result telling the agent the run was not found and to start a fresh run (the sessions dir may have been cleaned).
-- **Concurrent continuations of the same run**: not locked; last write wins. Documented, not defended — a single-user tool.
-- **Original run errored or was aborted**: the transcript still exists and is continuable; the follow-up prompt simply lands after the failure. Often the right move ("try again, but look at X").
-- **Model changed between runs**: allowed; transcript resumes under the newly resolved model.
-- **Aborted continuation**: same abort path as fresh runs; the transcript keeps whatever was persisted, and remains continuable.
-
-## Alternatives
-
-### Persist into the normal pi session store (pi-sessions integration)
-
-- **Status:** Deferred
-- **Decision or open issue:** Landing runs in the regular store would let pi-sessions index them — past research searchable, askable, and linked into lineage. It also pollutes resume pickers with untitled subagent sessions and couples the packages.
-- **Retained discussion:** The cleaner future shape is probably pi-sessions optionally indexing `<cacheDir>/sessions` as an additional source, keeping picker separation while gaining research memory.
-- **Next step:** Revisit once continuations prove their worth in practice.
-
-### In-memory session map for the pi process lifetime
-
-- **Status:** Rejected
-- **Decision or open issue:** Dies on restart and cannot serve a different main session; disk persistence costs nearly nothing given pi's existing machinery.
-
-### Lighter result shape for one-line follow-ups
-
-- **Status:** Rejected
-- **Decision or open issue:** Two result shapes for the main agent to handle; `provide_results` on a zero-tool-call run is cheap enough.
-
-## Implementation Plan
-
-- [ ] Phase 1: Persistent run sessions + `continue_from`
-  - Goal: Runs persist to `<cacheDir>/sessions/`; `continue_from` resumes them end to end.
-  - Files: `extensions/librarian/run.ts`, `extensions/librarian/sessions.ts` (new: dir resolution, uuid→file glob), `extensions/librarian.ts`, `extensions/librarian/prompt.ts`.
-  - Work: swap `SessionManager.inMemory` for `create(cacheDir, sessionsDir)`; capture `session.sessionManager.getSessionId()` — verify the accessor path on the created session — into `LibrarianRunDetails.runId` and append `run: <uuid>` to findings content; add `continue_from` param, resolve uuid→file, `SessionManager.open`, structured not-found error; system prompt continuation expectation; tool description guidance.
-  - Validation: unit tests for uuid→file resolution and not-found handling; `npm run check` green.
-
-- [ ] Phase 2: Render the run id
-  - Goal: Human-visible uuid in the done state.
-  - Files: `extensions/librarian/view.ts`.
-  - Work: muted `run <uuid>` line above the footer in the done render (fresh and continued runs alike).
-  - Validation: view unit test; visual check in TUI.
-
-- [ ] Phase 3: Live smoke
-  - Goal: Continuation proven in a real pi session.
-  - Files: `SMOKE.md`.
-  - Work: run a deep dive; ask a follow-up via `continue_from` in the same session (expect few/zero tool calls); restart pi and continue the same run from a fresh session using the rendered uuid; record evidence.
-  - Validation: SMOKE.md entries with captured output.

package/docs/designs/03-librarian-research-tool-hardening.md

@@ -1,141 +0,0 @@
-# Librarian research-tool hardening
-
-## Status
-
-Deferred
-
-## Decision Summary
-
-Capture three future improvements for librarian runs: stricter reusable checkout cache mechanics, AST-aware local-code tools, and richer result metadata that distinguishes candidates from proof. These are not required for the current implementation, but they are the ideas from other librarian/code-research systems that fit this project without weakening the clone-first, read-only model.
-
-## Problem Statement / Background
-
-The current librarian design already makes the main reliability move: within-repo research happens from local blob-less clones, while search APIs are candidate finders. A survey of adjacent implementations suggested several refinements worth preserving before the details are lost:
-
-- Mitsuhiko's librarian skill treats checkouts as stable cached infrastructure: repo references normalize to a predictable cache path, clones are reused, and refreshes are throttled.
-- Code-research tools such as Octocode and codebase-research-agent emphasize exact regions, line anchors, structured tool contracts, and smaller useful code chunks rather than raw search dumps.
-- This project should borrow those patterns only where they reinforce its own shape. It should not become an API-first search tool, a heavy MCP service, or a semantic index whose snippets are mistaken for evidence.
-
-The useful scenario is a repeated question about the same repo: the librarian should reuse the existing clone, make it match upstream exactly, locate the relevant function/class instead of handing back many grep lines, and report whether each tool result is evidence or only a lead.
-
-## Goals
-
-- Reuse cached clones safely while preserving the librarian's read-only research contract.
-- Improve code citations by returning meaningful syntactic regions when local searches find relevant lines.
-- Make tool results more self-describing, especially around completeness, truncation, errors, and whether output is citable proof.
-
-## Non-Goals
-
-- No dependency on Octocode or any external MCP service.
-- No shift away from clone-first within-repo research.
-- No semantic/vector index as part of this design.
-- No repo-profile cache for now; the value is less clear than the three improvements captured here.
-
-## Exposed Shape
-
-These additions would preserve the current user-facing `librarian` tool and `/librarian` attach command. The exposed changes would be inside the repo-tool surface:
-
-- `checkout_repo` becomes stricter about cache reuse and reports exactly what it did.
-- New AST-aware tools, or AST-aware modes on existing tools, expose code regions with file and line ranges.
-- Existing tools return richer structured details so renderers and prompts can distinguish complete results, partial results, errors, candidates, and proof.
-
-## Design Decisions
-
-### 1. Cached checkouts are disposable infrastructure, not worktrees
-
-A reused checkout should be forced back to the requested upstream state. Unlike a human working clone, librarian checkouts are cache infrastructure. If a previous run or aborted process left local changes, generated files, or an old branch behind, `checkout_repo` should not preserve them.
-
-For default-branch checkouts, the desired shape is:
-
-1. verify the cache path is a git repo for the requested normalized remote;
-2. fetch from origin, with a TTL/debounce to avoid needless network work;
-3. checkout the default branch;
-4. hard reset to `origin/<defaultBranch>`;
-5. remove untracked files;
-6. return the pinned HEAD sha and cache status.
-
-For explicit refs, the tool should fetch the requested ref when needed, force checkout/detach to the resolved commit, and clean the tree. If the cached remote does not match the requested repo, the safe behavior is to discard the cache path and clone again.
-
-**Tradeoff:** this destroys any local modifications inside the checkout cache. That is acceptable because librarian runs are read-only by contract; cache contents are not user-owned work.
-
-### 2. AST-aware tools should return useful code units, not just matches
-
-Plain grep is still the right baseline because it is universal, fast, and transparent. The missing primitive is a way to turn a found line into the smallest useful enclosing code unit: function, method, class, interface, test case, or other language-relevant node.
-
-A future AST-aware addition could be either:
-
-- a new tool such as `read_enclosing_node(path, line, language?)`; or
-- AST-aware options on search tools, such as `context: "enclosing_node"`.
-
-The first version should target high-value languages in this project’s research patterns, likely TypeScript/JavaScript first. `ast-grep` is a plausible implementation substrate because it is language-aware without requiring a bespoke parser per language.
-
-Fallback behavior matters: when parsing fails or the language is unsupported, the tool should return a normal line window with explicit metadata saying it is a fallback. The output must still cite file and line ranges.
-
-**Tradeoff:** AST tools add dependency and language-support complexity. They earn their place only if they reduce token waste and improve citation quality over grep/read chains.
-
-### 3. Tool results should distinguish candidates from proof
-
-The current prompt already says `search_code` results are candidates to verify, not evidence. That discipline should be reflected in tool-result metadata, not only in prose.
-
-Useful metadata fields include:
-
-- `evidenceKind`: `candidate` or `proof`;
-- `complete`: whether the result set or file range is complete;
-- `truncated`: whether output was clipped;
-- `continuation`: how to read more when output is partial;
-- `recoveryHint`: what to try after an empty or failed result;
-- `resolvedRef`/`headSha`: the commit or ref the result is tied to when available.
-
-This would make the renderer clearer and give the prompt a structural hook for citation honesty. Search tools produce candidates. Checked-out file reads and AST-enclosing-node reads produce proof when tied to a concrete path and line range.
-
-**Tradeoff:** richer metadata makes tool details more verbose and requires tests across more branches. The payoff is fewer ambiguous outputs and less prompt-only enforcement.
-
-## Edge Cases & Failure Modes
-
-- **Cached repo remote mismatch:** discard and reclone rather than trying to repair an ambiguous cache entry.
-- **Dirty cached checkout:** hard reset and clean; do not preserve changes.
-- **Fetch fails but stale clone exists:** report a structured error instead of silently researching stale code, unless a future explicit offline mode exists.
-- **Explicit commit not reachable after fetch:** return a ref-resolution error with a recovery hint to verify the ref or repo.
-- **AST parse failure:** fall back to a line window and mark the result as a fallback, not an AST-derived region.
-- **Search result clipped by limits:** mark it as partial and provide the next action instead of implying completeness.
-
-## Alternatives
-
-### Keep checkout cache reuse as-is
-
-- **Status:** Open
-- **Open Issue:** The current implementation already does a force checkout/reset for the default branch and force checkout for explicit refs, but it does not yet fully encode the stricter cache-infrastructure policy described here.
-- **Discussion:** If real-world use never leaves dirty/untracked cache state or remote mismatches, the current behavior may be enough. The stricter policy is still easier to reason about.
-- **Next step:** Audit the current checkout implementation against this design before changing it.
-
-### Add Octocode as a dependency
-
-- **Status:** Rejected
-- **Decision**: Octocode is an external code-research/MCP-style system, not a pi-native tool. Its useful contribution here is contract design: exact regions, line anchors, pagination, bulk/error semantics, and candidate-vs-proof discipline.
-- **Discussion:** Borrow the result-shape ideas, not the infrastructure.
-
-### Add repo profiles
-
-- **Status:** Rejected
-- **Decision**: Repo profiles are less compelling for now than cache correctness, AST-aware reads, and result metadata.
-- **Discussion:** A profile cache could store package manager, source roots, manifests, and test commands, but it risks becoming stale orientation data that agents over-trust. Evidence should continue to come from actual file reads.
-
-## Implementation Plan
-
-- [ ] Phase 1: Checkout cache hardening
-  - Goal: Make reused clones deterministic infrastructure that always match the requested upstream ref.
-  - Files: `extensions/librarian/checkout.ts`, `extensions/librarian/tools/checkout-repo.ts`, checkout tests.
-  - Work: Verify cached remote identity; discard/reclone on mismatch; fetch with existing debounce; hard reset and clean for default branch and explicit refs; include cache action metadata in tool details.
-  - Validation: Unit tests for dirty checkout cleanup, untracked file cleanup, remote mismatch reclone, explicit ref checkout, and fetch failure behavior; `npm run check`.
-
-- [ ] Phase 2: Result metadata contract
-  - Goal: Encode candidate/proof and completeness semantics in tool details.
-  - Files: repo tool modules under `extensions/librarian/tools/`, `extensions/librarian/view.ts`, prompt/tests.
-  - Work: Add shared metadata fields where useful; mark `search_code` as candidate; mark checked-out/read file outputs as proof when they include concrete refs and line ranges; expose truncation and continuation hints.
-  - Validation: Tool unit tests and view tests showing partial/candidate/error states; `npm run check`.
-
-- [ ] Phase 3: AST-aware local-code primitive
-  - Goal: Provide a first syntactic-region helper for cloned repos.
-  - Files: new tool module under `extensions/librarian/tools/`, tool registration, prompt/tests.
-  - Work: Choose implementation substrate, likely `ast-grep`; support TypeScript/JavaScript first; return enclosing node line ranges; fall back to a line window with explicit fallback metadata.
-  - Validation: Fixture tests for functions/classes/methods/tests, unsupported language fallback, parse failure fallback, and live smoke on a real TypeScript repo.