@aeondave/pi-skill-optimizer 3.0.0 → 3.1.0

package/BENCHMARK.md

@@ -62,7 +62,7 @@ The benchmark asks:
 2. Is the stable base byte-identical across unrelated human requests?
 3. Does bounded prefetch improve first-turn recall without changing that base?
 4. Can `skill_search` recover relevant tail skills through bounded pagination?
-5. Can `tool_search` defer schemas without making required tools unavailable?
+5. Can `optimizer_tool_search` defer schemas without making required tools unavailable?
 6. Do output and history reductions preserve protected evidence and exact recovery?
 7. Is every accepted serialized transformation smaller?
 8. What do providers report for tokens, cache reads, cache writes, and cost?
@@ -174,7 +174,7 @@ Blocking invariants:
 
 - core, configured, and already used tools remain active;
 - inactive schemas are absent from the initial provider request;
-- `tool_search` remains active while searchable definitions remain;
+- `optimizer_tool_search` remains active while searchable definitions remain;
 - activation makes selected tools callable in the same session;
 - activation is additive and cannot strand an active provider tool loop;
 - schema indexing work and response pagination remain bounded;
@@ -188,7 +188,7 @@ Reported metrics:
 tool_recall
 initial_tool_schema_bytes
 activated_tool_schema_bytes
-tool_search_calls
+optimizer_tool_search_calls
 activation_failures
 ```
 

package/CHANGELOG.md

@@ -2,6 +2,31 @@
 
 All notable changes are documented here.
 
+## 3.1.0 - 2026-10-03
+
+### Breaking
+
+- Rename the optimizer's model-callable `tool_search` to `optimizer_tool_search`. Explicit tool calls must use the new name. Pi's built-in `tool_search` now loads alongside the optimizer without a replacement warning.
+
+### Fixed
+
+- Preserve an active built-in `tool_search` during optimizer replanning.
+- Retain tools activated later by Pi or another extension, and honor `alwaysTools` for initially inactive `deferred` or `codemode` tools while respecting hidden and inactive direct tools.
+- Capture an explicitly requested benchmark catalog from Pi 1.0's `before_agent_start` skill options.
+
+### Changed
+
+- Require Pi 1.0.0 or newer for this release.
+- Run typechecking and host integration tests against Pi 1.0.0 development packages.
+- Add a reproducible Pi CLI black-box smoke test for the packed artifact.
+
+## 3.0.1 - 2026-09-30
+
+### Fixed
+
+- Keep Pi 0.99's initially inactive `deferred` and `codemode` tools discoverable through the optimizer's `tool_search`. Activate only matching search results, retain them across later replanning, and leave initially unavailable direct or hidden tools outside the catalog.
+- Reject blank tool searches without activating fallback results. Document how to disable Pi's replaceable built-in `tool-search` at the same scope as the optimizer, avoiding its startup conflict warning.
+
 ## 3.0.0 - 2026-09-28
 
 ### Scoped distribution - 2026-09-29

package/README.md

@@ -14,7 +14,7 @@ No embedding service or extra model call for ordinary routing. No changes to you
 
 ## Install
 
-Requires **Pi 0.87.1+** and **Node.js 26.3+**. Upgrade both before installing v3.
+Requires **Pi 1.0.0+** and **Node.js 26.3+**. Upgrade both before installing v3.1.
 
 ```bash
 pi install npm:@aeondave/pi-skill-optimizer
@@ -26,12 +26,18 @@ Restart Pi, then check that it loaded:
 /skill-optimizer
 ```
 
+On Pi 1.0.0, the optimizer's search tool is `optimizer_tool_search`. Pi's
+built-in `tool_search` can load alongside it without a name conflict. If you
+previously disabled `builtin:tool-search` to avoid the warning, you can remove
+that setting after updating the optimizer. See the
+[tool-search configuration](docs/CONFIGURATION.md#tool-search-and-pi-100).
+
 That's enough to start. Optionally run `/skill-optimizer init` to generate routing hints for your library. **Init uses your current Pi model and may incur cost**; normal discovery does not.
 
 | Need | Command |
 | --- | --- |
 | Install only in this project | `pi install -l npm:@aeondave/pi-skill-optimizer` |
-| Pin the current release | `pi install npm:@aeondave/pi-skill-optimizer@3.0.0` |
+| Pin the current release | `pi install npm:@aeondave/pi-skill-optimizer@3.1.0` |
 | Update an unpinned install | `pi update npm:@aeondave/pi-skill-optimizer` |
 
 Restart after updating. Pinned installs stay pinned until you explicitly install another version.
@@ -58,7 +64,7 @@ AUTO keeps a compact, stable index of **all eligible names**, adds relevant desc
 | Area | What AUTO does | What stays available |
 | --- | --- | --- |
 | Skills | Shortens catalog descriptions; prefetches relevant ones | Every eligible name and exact full-instruction loading |
-| Tools | Defers less relevant schemas behind `tool_search` | Protected, configured, already-used and activated tools |
+| Tools | Defers less relevant schemas behind `optimizer_tool_search` | Protected, configured, already-used and activated tools |
 | Large results | Applies deterministic reduction; lossless columnar JSON where suitable | Complete originals through `retrieve_output` during archive retention |
 | Repeated results | Replaces later exact duplicates with recovery references | The first result, errors, protected evidence and mixed media |
 
@@ -76,7 +82,7 @@ You can leave AUTO running with its defaults.
 
 An optional setup hint appears at the first eligible prompt when initialization is missing or needs attention. It never starts init for you or adds instructions to the model's context.
 
-Pi gets three tools: `skill_search`, `tool_search` and `retrieve_output`. Skill search is paginated and exact loads accept only registered names. Output recovery uses opaque identifiers, not exposed archive paths.
+Pi gets three optimizer tools: `skill_search`, `optimizer_tool_search` and `retrieve_output`. Pi may also provide its built-in `tool_search`. Skill search is paginated and exact loads accept only registered names. Output recovery uses opaque identifiers, not exposed archive paths.
 
 Skills can be useful even when you never mention them. Init can tag architecture,
 testing and behavioral guidance by activity, so a matched specialist can bring

package/docs/ARCHITECTURE.md

@@ -30,9 +30,9 @@ expand dependencies or treat a behavioral tag as universal. See
 
 ## Dynamic tools
 
-AUTO keeps protected and predicted tools active and leaves the rest behind `tool_search`. Search is bounded and paginated. A returned result is activated through Pi's runtime API without replacing already active tools. Core, configured, and previously used tools remain callable, so an in-progress tool loop cannot be stranded. Weak or absent routing signal fails open to every permitted tool; safety wins, but that request may retain the full schema cost.
+AUTO keeps protected and predicted tools active and leaves the rest behind `optimizer_tool_search`. Search is bounded and paginated. A returned result is activated through Pi's runtime API without replacing already active tools. Core, configured, and previously used tools remain callable, so an in-progress tool loop cannot be stranded. Pi's built-in `tool_search` remains active when the host enables it. Weak or absent routing signal fails open to every permitted tool; safety wins, but that request may retain the full schema cost.
 
-Selection is recalculated before each agent run, not just the first request. Used tools and explicit search results stay active for the session. The discoverable set starts from Pi's initially active tools plus the optimizer's own resolvers; tools that were initially unavailable are not silently added. This is schema routing, not authorization: another extension's call-time permission checks remain authoritative.
+Selection is recalculated before each agent run, not just the first request. Used tools and explicit search results stay active for the session. The discoverable set starts from Pi's initially active tools plus the optimizer's own resolvers. Newly activated host tools are retained; inactive `deferred` and `codemode` tools can be selected through search or `alwaysTools`. Inactive direct and hidden tools are not silently added. This is schema routing, not authorization: another extension's call-time permission checks remain authoritative.
 
 See Pi's [extension and dynamic-tool API](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/extensions.md).
 

package/docs/CONFIGURATION.md

@@ -7,13 +7,19 @@
 | Tool | Purpose |
 | --- | --- |
 | `skill_search` | Paginated skill search plus exact `load`, contained `resource`, and explicit `location` actions |
-| `tool_search` | Search deferred tool definitions and activate bounded matches additively |
+| `optimizer_tool_search` | Search deferred tool definitions and activate bounded matches additively |
 | `retrieve_output` | Recover archived output or a deduplicated history result by opaque identifier |
 
 Exact skill loading accepts only registered names. Resource and location resolution use canonical paths and containment checks to reject traversal and symlink escape. Search cursors bind to the query, catalog and ranked name order; a changed profile or usage prior that reorders results requires a new search, rather than duplicating or skipping a page. This is an exact local resolver guarantee, not a guarantee that the model will select the right skill; see the separate [ranking diagnostics](RANKING.md) and [historical model benchmark](../BENCHMARK.md).
 
 `excludeSkills` removes names from discovery and search, not from explicit exact loading. It is a routing preference, not an access-control boundary. The three resolver tools return their content without another output-reduction pass, so full skill instructions and recovered originals remain readable.
 
+### Tool search and Pi 1.0.0
+
+The extension registers `optimizer_tool_search`, leaving Pi's built-in `tool_search` available under its own name. The optimizer indexes eligible registered tools, including ones it deferred from Pi's active set and Pi-registered `deferred` or `codemode` tools. Matching inactive results activate additively; unrelated fallback results remain inactive. An already active built-in `tool_search` stays active during optimizer replanning. Pi's built-in MCP extension can activate its own `tool_search` when a deferred server connects.
+
+If an older optimizer release led you to add `-builtin:tool-search` in `~/.pi/agent/settings.json` or `.pi/settings.json`, remove that entry after updating if you want Pi's built-in search available again. The built-in is inactive by default unless enabled in Pi's `defaultTools` or activated by Pi for deferred MCP tools; the optimizer's search stays available independently. Existing explicit calls to the optimizer's old `tool_search` name must use `optimizer_tool_search` after this update.
+
 ## Commands
 
 | Command | Purpose |
@@ -49,7 +55,7 @@ Global configuration lives at `~/.pi/agent/skill-optimizer/config.json`. A proje
 | `providers` | `[]` | Provider allowlist; empty means all |
 | `toolPrefetchMax` | `5` | Maximum proactive tool activation |
 | `toolSearchPageSize` | `8` | Default tool-search page size |
-| `alwaysTools` | `[]` | Tools that remain active |
+| `alwaysTools` | `[]` | Keep initially active tools and named `deferred` or `codemode` tools active; hidden tools remain unavailable |
 | `historyDedupMinBytes` | `4096` | Minimum result size for history deduplication |
 | `usageHalfLifeDays` | `30` | Usage-evidence decay half-life |
 | `usageMaxEntries` | `2048` | Persistent usage-evidence bound |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@aeondave/pi-skill-optimizer",
-  "version": "3.0.0",
+  "version": "3.1.0",
   "private": false,
   "type": "module",
   "description": "Less repeated context for Pi: local skill routing, on-demand tools, and recoverable output reduction.",
@@ -43,12 +43,13 @@
   ],
   "scripts": {
     "typecheck": "tsc --noEmit",
-    "test": "node --import tsx --test test/request.test.ts test/history.test.ts test/skills.test.ts test/skill-loader.test.ts test/tools.test.ts test/optimize.test.ts test/profile.test.ts test/activation.test.ts test/generate.test.ts test/init.test.ts test/init-readiness.test.ts test/config.test.ts test/fuzz.test.ts test/output.test.ts test/output-recovery.test.ts test/runtime.test.ts test/host-runtime.test.ts test/stats.test.ts test/usage.test.ts test/persistence.test.ts test/corpus.test.ts test/evaluation.test.ts test/luna.test.ts test/release-metadata.test.ts",
+    "test": "node --import tsx --test test/request.test.ts test/history.test.ts test/skills.test.ts test/skill-loader.test.ts test/tools.test.ts test/optimize.test.ts test/profile.test.ts test/activation.test.ts test/generate.test.ts test/init.test.ts test/init-readiness.test.ts test/config.test.ts test/fuzz.test.ts test/output.test.ts test/output-recovery.test.ts test/runtime.test.ts test/host-runtime.test.ts test/stats.test.ts test/usage.test.ts test/persistence.test.ts test/corpus.test.ts test/capture-catalog.test.ts test/evaluation.test.ts test/luna.test.ts test/release-metadata.test.ts",
     "measure": "node --import tsx scripts/measure.ts",
     "bench": "node --import tsx scripts/bench.ts",
     "bench:perf": "node --import tsx scripts/perf-bench.ts",
     "bench:ranking": "node --import tsx scripts/ranking-bench.ts",
     "bench:output": "node --import tsx scripts/output-bench.ts",
+    "smoke:blackbox": "node --import tsx scripts/black-box-smoke.ts",
     "corpus:build": "node --import tsx scripts/build-real-corpus.ts",
     "bench:real": "node --import tsx scripts/evaluate-real-corpus.ts"
   },
@@ -69,8 +70,8 @@
     }
   },
   "devDependencies": {
-    "@earendil-works/pi-ai": "^0.87.1",
-    "@earendil-works/pi-coding-agent": "^0.87.1",
+    "@earendil-works/pi-ai": "^1.0.0",
+    "@earendil-works/pi-coding-agent": "^1.0.0",
     "@types/node": "^26.6.3",
     "tsx": "^4.23.13",
     "typebox": "^1.3.34",
@@ -80,6 +81,6 @@
     "extensions": [
       "./src/index.ts"
     ],
-    "image": "https://raw.githubusercontent.com/AeonDave/pi-skill-optimizer/v3.0.0/assets/banner.png"
+    "image": "https://raw.githubusercontent.com/AeonDave/pi-skill-optimizer/v3.1.0/assets/banner.png"
   }
 }

package/src/index.ts

@@ -110,7 +110,16 @@ const FLUSH_INTERVAL_MS = 15_000;
 const LOAD_RETRY_MS = 1_000;
 const MAX_SEEN_EVIDENCE = 4_096;
 const MAX_HISTORY_ARTIFACTS = 128;
-const PROTECTED_RUNTIME_TOOLS = ["skill_search", "tool_search", "retrieve_output"] as const;
+const PROTECTED_RUNTIME_TOOLS = ["skill_search", "optimizer_tool_search", "retrieve_output"] as const;
+const RESOLVER_RESULT_TOOLS = new Set<string>([...PROTECTED_RUNTIME_TOOLS, "tool_search"]);
+
+function isHostSearchableTool(tool: object): boolean {
+	return "exposure" in tool && (tool.exposure === "deferred" || tool.exposure === "codemode");
+}
+
+function isHostHiddenTool(tool: object): boolean {
+	return "exposure" in tool && tool.exposure === "hidden";
+}
 
 type ConfigRecord = ReturnType<typeof getConfig>;
 
@@ -565,12 +574,34 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 	};
 
 	const rebuildToolCatalog = (state: RuntimeSnapshot): ToolDiscoveryCatalog => {
+		const allTools = pi.getAllTools();
+		const activeNames = new Set(pi.getActiveTools().map((name) => name.toLowerCase()));
+		const alwaysToolPrefixes = state.config.alwaysTools
+			.map((pattern) => pattern.trim().toLowerCase())
+			.filter((pattern) => pattern.length > 0);
 		permittedToolNames ??= new Set([
-			...pi.getActiveTools().map((name) => name.toLowerCase()),
+			...activeNames,
 			...PROTECTED_RUNTIME_TOOLS,
 		]);
-		const discoverable = pi.getAllTools().filter((tool) =>
-			permittedToolNames?.has(tool.name.toLowerCase()),
+		for (const tool of allTools) {
+			const name = tool.name.toLowerCase();
+			if (isHostHiddenTool(tool)) {
+				permittedToolNames.delete(name);
+				retainedToolNames.delete(name);
+				deactivatedToolNames.delete(name);
+				continue;
+			}
+			if (activeNames.has(name)) {
+				const newlyActive = !permittedToolNames.has(name);
+				const reactivated = deactivatedToolNames.delete(name);
+				if (newlyActive || reactivated) retainedToolNames.add(name);
+				permittedToolNames.add(name);
+			} else if (isHostSearchableTool(tool) && alwaysToolPrefixes.some((pattern) => name.startsWith(pattern))) {
+				permittedToolNames.add(name);
+			}
+		}
+		const discoverable = allTools.filter((tool) =>
+			!isHostHiddenTool(tool) && permittedToolNames?.has(tool.name.toLowerCase()),
 		);
 		toolCatalog = buildToolDiscoveryCatalog(discoverable, {
 			protect: [...PROTECTED_RUNTIME_TOOLS, ...state.config.alwaysTools],
@@ -578,13 +609,25 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 		return toolCatalog;
 	};
 
+	const buildToolSearchCatalog = (state: RuntimeSnapshot): ToolDiscoveryCatalog => {
+		rebuildToolCatalog(state);
+		return buildToolDiscoveryCatalog(pi.getAllTools().filter((tool) =>
+			!isHostHiddenTool(tool)
+				&& (permittedToolNames?.has(tool.name.toLowerCase()) || isHostSearchableTool(tool)),
+		), {
+			protect: [...PROTECTED_RUNTIME_TOOLS, ...state.config.alwaysTools],
+		});
+	};
+
 	const activateAdditively = (names: readonly string[]): string[] => {
 		const requested = new Set(names.map((name) => name.toLowerCase()));
 		const current = new Set(pi.getActiveTools().map((name) => name.toLowerCase()));
 		const added: string[] = [];
 		for (const tool of pi.getAllTools()) {
-			if (!permittedToolNames?.has(tool.name.toLowerCase())) continue;
+			if (isHostHiddenTool(tool)) continue;
+			if (!permittedToolNames?.has(tool.name.toLowerCase()) && !isHostSearchableTool(tool)) continue;
 			if (!requested.has(tool.name.toLowerCase())) continue;
+			permittedToolNames?.add(tool.name.toLowerCase());
 			retainedToolNames.add(tool.name.toLowerCase());
 			if (current.has(tool.name.toLowerCase())) continue;
 			current.add(tool.name.toLowerCase());
@@ -592,8 +635,8 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 			added.push(tool.name);
 		}
 		const ordered = pi.getAllTools()
-			.map((tool) => tool.name)
-			.filter((name) => permittedToolNames?.has(name.toLowerCase()) && current.has(name.toLowerCase()));
+			.filter((tool) => !isHostHiddenTool(tool) && current.has(tool.name.toLowerCase()))
+			.map((tool) => tool.name);
 		pi.setActiveTools(ordered);
 		return added;
 	};
@@ -607,8 +650,8 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 		for (const name of deactivatedToolNames) active.add(name);
 		pi.setActiveTools(
 			pi.getAllTools()
-				.map((tool) => tool.name)
-				.filter((name) => active.has(name.toLowerCase())),
+				.filter((tool) => !isHostHiddenTool(tool) && active.has(tool.name.toLowerCase()))
+				.map((tool) => tool.name),
 		);
 		deactivatedToolNames.clear();
 		toolCatalog = undefined;
@@ -656,14 +699,15 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 	});
 
 	pi.registerTool({
-		name: "tool_search",
-		label: "Tool Search",
-		description: "Search inactive tools locally. Returned tools are activated additively without disabling current tools.",
+		name: "optimizer_tool_search",
+		label: "Optimizer Tool Search",
+		description: "Search local tool definitions and activate matching inactive tools without disabling current tools.",
 		promptSnippet: "Search and activate additional tools on demand",
 		parameters: TOOL_SEARCH_PARAMS,
 		async execute(_id, params, _signal, _update, ctx) {
+			if (!params.query.trim()) throw new TypeError("query must not be empty");
 			const state = ensureSnapshot(ctx);
-			const catalog = toolCatalog ?? rebuildToolCatalog(state);
+			const catalog = buildToolSearchCatalog(state);
 			const page = searchToolDiscoveryCatalog(catalog, params.query, {
 				cursor: params.cursor,
 				pageSize: Math.min(
@@ -671,7 +715,9 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 					state.config.toolSearchPageSize,
 				),
 			});
-			const activated = activateAdditively(page.results.map((result) => result.name));
+			const activated = activateAdditively(page.results
+				.filter((result) => result.matchedTerms.length > 0)
+				.map((result) => result.name));
 			return {
 				content: [{ type: "text", text: JSON.stringify({ ...page, activated }) }],
 				details: { activated, confidence: page.confidence, failOpen: page.failOpen },
@@ -1146,7 +1192,7 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 
 	pi.on("tool_result", async (event, ctx) => {
 		usedTools.add(event.toolName.toLowerCase());
-		if (PROTECTED_RUNTIME_TOOLS.includes(event.toolName as typeof PROTECTED_RUNTIME_TOOLS[number])) return;
+		if (RESOLVER_RESULT_TOOLS.has(event.toolName)) return;
 		if (isDisabled(ctx.cwd) || event.isError) return;
 		const config = getRuntimeConfig(ctx);
 		const providers = config.providers.length > 0 ? config.providers : undefined;
@@ -1342,7 +1388,7 @@ export default function skillOptimizer(pi: ExtensionAPI) {
 				`  enabled:        ${isDisabled(ctx.cwd) ? "no" : "yes"}`,
 				"  skills mode:    AUTO (stable TSV + skill_search resolver)",
 				`  catalog budget:${lastBudget.usedChars}/${lastBudget.requestedChars} chars, ${lastBudget.nameOnlyCount} name-only`,
-				`  tools:          ${pi.getActiveTools().length}/${pi.getAllTools().length} active; tool_search additive`,
+				`  tools:          ${pi.getActiveTools().length}/${pi.getAllTools().length} active; optimizer_tool_search additive`,
 				`  output:         ${getOutputConfig(state.config).mode}; RTK ${rtkExtensionPresent() ? "detected per result" : "not detected"}`,
 				`  scope:          ${state.config.providers.join(", ") || "all providers"}`,
 				`  init readiness: ${retryAfter !== 0 ? "unavailable (profile could not be read)" : readiness.skillCount === 0 ? "not available (no model-invocable skills)" : initReadinessSummary(readiness)}`,

package/src/tools.ts

@@ -23,7 +23,7 @@ export const CORE_TOOLS = new Set(
 		"read", "bash", "edit", "write", "multiedit", "str_replace_editor", "notebookedit",
 		"glob", "grep", "ls", "find", "todo", "todowrite", "todoread",
 		"subagent", "fork", "task", "mcp", "skill", "exitplanmode", "enterplanmode",
-		"webfetch", "websearch", "askuserquestion", "powershell",
+		"webfetch", "websearch", "askuserquestion", "powershell", "tool_search",
 	].map((name) => name.toLowerCase()),
 );