agent-regression-lab 0.3.0 → 0.5.0

package/docs/superpowers/plans/2026-04-13-phase-one-npm-tools-plan.md

@@ -0,0 +1,502 @@
+# Phase One Npm Tools Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Finish the Phase 1 friction-removal work by adding npm-installable tool support, shipping package-style example tools, and making the `init` and docs flow work cleanly for `npx` users.
+
+**Architecture:** Keep existing repo-local `modulePath` tool loading intact and add an explicit package-based loading path via `package`. Validate that each tool registration uses exactly one source, resolve file-backed tools from the project root, and resolve package-backed tools through normal Node package resolution from the current project. Ship minimal example packages in-repo to demonstrate the supported authoring shape and update `init` plus docs to make the new flow the default recommendation for installed usage.
+
+**Tech Stack:** TypeScript, Node.js ESM dynamic import, YAML config, node:test, tsx
+
+---
+
+## File Map
+
+**Core types and config**
+- Modify: `src/types.ts`
+- Modify: `src/config.ts`
+
+**Tool loading**
+- Modify: `src/tools.ts`
+
+**Init flow**
+- Modify: `src/init.ts`
+
+**Docs**
+- Modify: `README.md`
+- Modify: `docs/tools.md`
+- Modify: `docs/troubleshooting.md`
+- Modify: `.claude/project.md`
+- Modify: `.claude/active-tasks.md`
+
+**Examples**
+- Create: `examples/support-tools/package.json`
+- Create: `examples/support-tools/index.js`
+- Create: `examples/support-tools/README.md`
+- Create: `examples/coding-tools/package.json`
+- Create: `examples/coding-tools/index.js`
+- Create: `examples/coding-tools/README.md`
+
+**Tests**
+- Modify: `tests/config.tier-one.test.ts`
+- Create: `tests/packageTools.test.ts`
+- Modify: `tests/cliPackaging.test.ts`
+- Modify: `tests/init.test.ts`
+
+---
+
+### Task 1: Add Explicit Package-Based Tool Registration
+
+**Files:**
+- Modify: `src/types.ts`
+- Modify: `src/config.ts`
+- Modify: `tests/config.tier-one.test.ts`
+
+- [ ] **Step 1: Write failing config tests for package-backed tools**
+
+Add tests covering:
+
+```ts
+test("config accepts a package-backed tool registration", () => {
+  // tool has package + exportName and no modulePath
+});
+
+test("config rejects a tool with both modulePath and package", () => {
+  assert.throws(() => loadAgentLabConfig(), /exactly one of 'modulePath' or 'package'/i);
+});
+
+test("config rejects a tool with neither modulePath nor package", () => {
+  assert.throws(() => loadAgentLabConfig(), /exactly one of 'modulePath' or 'package'/i);
+});
+```
+
+- [ ] **Step 2: Run the config tests to verify failure**
+
+Run: `npx tsx --test tests/config.tier-one.test.ts`
+
+Expected: FAIL because `ToolRegistration` and `validateToolRegistration()` currently require `modulePath`.
+
+- [ ] **Step 3: Extend the tool registration type**
+
+Update `src/types.ts`:
+
+```ts
+export type ToolRegistration = ToolSpec & {
+  modulePath?: string;
+  package?: string;
+  exportName?: string;
+};
+```
+
+- [ ] **Step 4: Update config validation for explicit source selection**
+
+Update `validateToolRegistration()` in `src/config.ts` so it enforces:
+
+```ts
+const hasModulePath = typeof value.modulePath === "string" && value.modulePath.length > 0;
+const hasPackage = typeof value.package === "string" && value.package.length > 0;
+
+if ((hasModulePath ? 1 : 0) + (hasPackage ? 1 : 0) !== 1) {
+  throw new Error(`Tool '${value.name}' must define exactly one of 'modulePath' or 'package'.`);
+}
+```
+
+Retain:
+
+- required `name`
+- required `exportName`
+- required `description`
+- required `inputSchema`
+
+And only enforce repo-boundary + existence checks when `modulePath` is used.
+
+- [ ] **Step 5: Re-run the config tests**
+
+Run: `npx tsx --test tests/config.tier-one.test.ts`
+
+Expected: PASS
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add src/types.ts src/config.ts tests/config.tier-one.test.ts
+git commit -m "feat: support package-backed tool registrations"
+```
+
+---
+
+### Task 2: Load Tools From Installed Packages
+
+**Files:**
+- Modify: `src/tools.ts`
+- Create: `tests/packageTools.test.ts`
+
+- [ ] **Step 1: Write failing loader tests for package-backed tools**
+
+Create `tests/packageTools.test.ts` with coverage for:
+
+```ts
+test("loadToolRegistry loads a package-backed tool from node_modules", async () => {
+  // temp workspace with package tool registration and a stub package in node_modules
+});
+
+test("loadToolRegistry surfaces a clear error when package import fails", async () => {
+  // missing package should mention tool name and package name
+});
+
+test("loadToolRegistry still loads repo-local modulePath tools", async () => {
+  // regression guard for existing behavior
+});
+```
+
+- [ ] **Step 2: Run the new loader tests to verify failure**
+
+Run: `npx tsx --test tests/packageTools.test.ts`
+
+Expected: FAIL because `loadConfiguredTool()` only imports `modulePath`.
+
+- [ ] **Step 3: Refactor tool loading into source-specific helpers**
+
+In `src/tools.ts`, split the loading path:
+
+```ts
+async function loadConfiguredTool(tool: ToolRegistration): Promise<LoadedTool> {
+  const module = tool.package
+    ? await importConfiguredPackageTool(tool)
+    : await importConfiguredFileTool(tool);
+
+  const candidate = module[tool.exportName!];
+  if (typeof candidate !== "function") {
+    throw new Error(`Tool '${tool.name}' export '${tool.exportName}' is not a function.`);
+  }
+
+  return {
+    spec: {
+      name: tool.name,
+      description: tool.description,
+      inputSchema: tool.inputSchema,
+    },
+    handler: candidate as ToolHandler,
+  };
+}
+```
+
+Add:
+
+```ts
+async function importConfiguredFileTool(tool: ToolRegistration) {
+  const moduleUrl = pathToFileURL(resolve(tool.modulePath!)).href;
+  return await import(moduleUrl);
+}
+
+async function importConfiguredPackageTool(tool: ToolRegistration) {
+  try {
+    return await import(tool.package!);
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    throw new Error(`Tool '${tool.name}' failed to load package '${tool.package}': ${message}`);
+  }
+}
+```
+
+- [ ] **Step 4: Re-run the loader tests**
+
+Run: `npx tsx --test tests/packageTools.test.ts`
+
+Expected: PASS
+
+- [ ] **Step 5: Run existing tool-related regression tests**
+
+Run: `npx tsx --test tests/benchmarkExpansion.test.ts tests/cliPackaging.test.ts`
+
+Expected: PASS
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add src/tools.ts tests/packageTools.test.ts tests/cliPackaging.test.ts
+git commit -m "feat: load custom tools from installed packages"
+```
+
+---
+
+### Task 3: Update Init To Teach The Package-Tool Path
+
+**Files:**
+- Modify: `src/init.ts`
+- Modify: `tests/init.test.ts`
+
+- [ ] **Step 1: Write failing init tests**
+
+Add or extend tests to assert the generated config includes comments for both styles:
+
+```ts
+assert.match(config, /modulePath: \.\/tools\/customTool\.ts/);
+assert.match(config, /package: "@agentlab\/example-support-tools"/);
+```
+
+Also verify the next-step guidance mentions package installation:
+
+```ts
+assert.match(output, /npm install @agentlab\/example-support-tools/);
+```
+
+- [ ] **Step 2: Run init tests to verify failure**
+
+Run: `npx tsx --test tests/init.test.ts`
+
+Expected: FAIL because the current init template only documents repo-local paths.
+
+- [ ] **Step 3: Update the generated config and next-step guidance**
+
+Modify `src/init.ts` so `SAMPLE_CONFIG` shows:
+
+```yaml
+# Tools can come from:
+# 1. repo-local files
+# 2. installed npm packages
+
+# tools:
+#   - name: my.local_tool
+#     modulePath: ./tools/customTool.ts
+#     exportName: customTool
+#
+#   - name: support.find_duplicate_charge
+#     package: "@agentlab/example-support-tools"
+#     exportName: findDuplicateCharge
+```
+
+And update console output to include:
+
+```ts
+console.log("  npm install @agentlab/example-support-tools");
+console.log("  # then register package-backed tools in agentlab.config.yaml");
+```
+
+- [ ] **Step 4: Re-run init tests**
+
+Run: `npx tsx --test tests/init.test.ts`
+
+Expected: PASS
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add src/init.ts tests/init.test.ts
+git commit -m "docs: teach init flow about package-backed tools"
+```
+
+---
+
+### Task 4: Add Minimal Example Tool Packages
+
+**Files:**
+- Create: `examples/support-tools/package.json`
+- Create: `examples/support-tools/index.js`
+- Create: `examples/support-tools/README.md`
+- Create: `examples/coding-tools/package.json`
+- Create: `examples/coding-tools/index.js`
+- Create: `examples/coding-tools/README.md`
+- Create: `tests/packageTools.test.ts` (extend)
+
+- [ ] **Step 1: Create the support tools example package**
+
+Create `examples/support-tools/package.json`:
+
+```json
+{
+  "name": "@agentlab/example-support-tools",
+  "private": true,
+  "type": "module",
+  "exports": {
+    ".": "./index.js"
+  }
+}
+```
+
+Create `examples/support-tools/index.js` with a minimal exported function:
+
+```js
+export async function findDuplicateCharge(input) {
+  const customerId = String(input?.customer_id ?? "");
+  if (!customerId) {
+    throw new Error("customer_id is required");
+  }
+  return { order_id: `dup_${customerId}` };
+}
+```
+
+- [ ] **Step 2: Create the coding tools example package**
+
+Create `examples/coding-tools/package.json`:
+
+```json
+{
+  "name": "@agentlab/example-coding-tools",
+  "private": true,
+  "type": "module",
+  "exports": {
+    ".": "./index.js"
+  }
+}
+```
+
+Create `examples/coding-tools/index.js`:
+
+```js
+export async function readRepoHint(input) {
+  const path = String(input?.path ?? "");
+  if (!path) {
+    throw new Error("path is required");
+  }
+  return { path, hint: "Check the target file before editing." };
+}
+```
+
+- [ ] **Step 3: Add short READMEs showing intended registration**
+
+Each README should include an `agentlab.config.yaml` snippet using `package` + `exportName`.
+
+- [ ] **Step 4: Extend package tool tests to verify the example package shape**
+
+Add a test that imports the example package directly:
+
+```ts
+const mod = await import(resolve("examples/support-tools/index.js"));
+assert.equal(typeof mod.findDuplicateCharge, "function");
+```
+
+- [ ] **Step 5: Run example-package tests**
+
+Run: `npx tsx --test tests/packageTools.test.ts`
+
+Expected: PASS
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add examples/support-tools examples/coding-tools tests/packageTools.test.ts
+git commit -m "feat: add package-style example tool packages"
+```
+
+---
+
+### Task 5: Update Docs For Npx And Package-Based Tools
+
+**Files:**
+- Modify: `README.md`
+- Modify: `docs/tools.md`
+- Modify: `docs/troubleshooting.md`
+- Modify: `.claude/project.md`
+- Modify: `.claude/active-tasks.md`
+
+- [ ] **Step 1: Update public docs to describe both tool sources**
+
+In `docs/tools.md`, change the core framing from:
+
+```md
+Custom tools are registered in `agentlab.config.yaml` and loaded from repo-local JS or TS modules.
+```
+
+to:
+
+```md
+Custom tools are registered in `agentlab.config.yaml` and can be loaded from either repo-local modules or installed npm packages.
+```
+
+Add example blocks for both `modulePath` and `package`.
+
+- [ ] **Step 2: Update README npx flow**
+
+Add a package-tool example under the `npx` / install guidance:
+
+```bash
+npm install @agentlab/example-support-tools
+```
+
+And explain that package-backed tools are the preferred extension path for installed and `npx`-based projects.
+
+- [ ] **Step 3: Update troubleshooting**
+
+Add failure cases for:
+
+- missing installed package
+- wrong `exportName` from a package
+- specifying both `modulePath` and `package`
+
+Use direct messages matching the implementation.
+
+- [ ] **Step 4: Update internal phase tracking**
+
+In `.claude/project.md` and `.claude/active-tasks.md`:
+
+- mark npm-installable tools complete
+- keep Phase 1 open only if example packages / starter-repo / README rewrite still remain
+
+- [ ] **Step 5: Run a docs-adjacent regression smoke**
+
+Run: `npm run smoke:cli`
+
+Expected: PASS
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add README.md docs/tools.md docs/troubleshooting.md .claude/project.md .claude/active-tasks.md
+git commit -m "docs: document npm-installable tools for npx users"
+```
+
+---
+
+### Task 6: Full Verification For Phase 1 Tooling
+
+**Files:**
+- Modify as needed from previous tasks only
+
+- [ ] **Step 1: Run focused tests**
+
+Run:
+
+```bash
+npx tsx --test tests/config.tier-one.test.ts tests/packageTools.test.ts tests/init.test.ts tests/cliPackaging.test.ts
+```
+
+Expected: PASS
+
+- [ ] **Step 2: Run full suite**
+
+Run:
+
+```bash
+npm test
+```
+
+Expected: PASS
+
+- [ ] **Step 3: Run full release checks**
+
+Run:
+
+```bash
+npm run check
+npm run build
+npm run smoke:cli
+npm_config_cache=/tmp/agentlab-npm-cache npm pack --dry-run
+```
+
+Expected: PASS
+
+- [ ] **Step 4: Reconcile internal status**
+
+If verification is green:
+
+- keep `.claude/project.md` Phase 1 marked in progress only if remaining tasks are truly outstanding
+- otherwise mark the package-tool portion complete explicitly
+
+- [ ] **Step 5: Final commit**
+
+```bash
+git add -A
+git commit -m "feat: finish phase-one npm tool support"
+```
+