agent-regression-lab 0.3.0 → 0.4.0

package/docs/tools.md

@@ -1,6 +1,6 @@
 # Custom Tools
 
-Custom tools are registered in `agentlab.config.yaml` and loaded from repo-local JS or TS modules.
+Custom tools are registered in `agentlab.config.yaml` and can be loaded from repo-local JS/TS modules or installed npm packages.
 
 This is the main extension point when built-in tools are not enough.
 
@@ -9,12 +9,14 @@ This is the main extension point when built-in tools are not enough.
 Each tool entry must define:
 
 - `name`
-- `modulePath`
+- exactly one source:
+  - `modulePath`, or
+  - `package`
 - `exportName`
 - `description`
 - `inputSchema`
 
-Example:
+Repo-local example:
 
 ```yaml
 tools:
@@ -33,6 +35,25 @@ tools:
         - customer_id
 ```
 
+Installed package example:
+
+```yaml
+tools:
+  - name: support.find_duplicate_charge
+    package: "@agentlab/example-support-tools"
+    exportName: findDuplicateCharge
+    description: Find the duplicated charge order id for a given customer.
+    inputSchema:
+      type: object
+      additionalProperties: false
+      properties:
+        customer_id:
+          type: string
+          description: Customer id to inspect for duplicated charges.
+      required:
+        - customer_id
+```
+
 ## Tool Module Shape
 
 The exported function should be async and should return JSON-serializable output.
@@ -48,11 +69,15 @@ export async function myTool(input: unknown): Promise<{ ok: boolean }> {
 The existing working example is:
 
 - `user_tools/findDuplicateCharge.ts`
+- `examples/support-tools`
+- `examples/coding-tools`
 
 ## Important Constraints
 
+- each tool must define exactly one of `modulePath` or `package`
 - `modulePath` must stay within the repo
 - the module must exist at load time
+- installed packages must be resolvable from the current project
 - the named export must exist
 - tool input should be validated defensively inside the tool
 - tool output should be deterministic and JSON-serializable
@@ -100,3 +125,9 @@ Typical config failures:
 - invalid `inputSchema` shape
 
 See [troubleshooting.md](troubleshooting.md) for failure examples and fixes.
+
+For installed-package workflows, a good local path is:
+
+```bash
+npm install @agentlab/example-support-tools
+```

package/docs/troubleshooting.md

@@ -138,14 +138,69 @@ Typical reasons:
 - `agentlab.config.yaml` is missing
 - the configured `name` does not match the CLI `--agent` value
 - `modulePath` points outside the repo
+- both `modulePath` and `package` were provided for the same tool
+- the configured npm package is not installed
 - the configured export or command does not exist
 
 Working references in this repo:
 
 - tool config: `agentlab.config.yaml`
 - custom tool: `user_tools/findDuplicateCharge.ts`
+- package-style tools: `examples/support-tools`, `examples/coding-tools`
 - external agents: `custom_agents/node_agent.mjs`, `custom_agents/python_agent.py`
 
+### `Tool '<name>' must define exactly one of 'modulePath' or 'package'`
+
+Your tool registration is ambiguous or incomplete.
+
+Valid:
+
+```yaml
+tools:
+  - name: support.find_duplicate_charge
+    modulePath: ./user_tools/findDuplicateCharge.ts
+    exportName: findDuplicateCharge
+```
+
+Also valid:
+
+```yaml
+tools:
+  - name: support.find_duplicate_charge
+    package: "@agentlab/example-support-tools"
+    exportName: findDuplicateCharge
+```
+
+Invalid:
+
+- setting both `modulePath` and `package`
+- setting neither of them
+
+### `Tool '<name>' failed to load package '<pkg>'`
+
+The package-backed tool could not be resolved from the current project.
+
+Check:
+
+- the package is installed in the current project
+- the package name is correct
+- the package exports the named function you configured
+
+Typical fix:
+
+```bash
+npm install @agentlab/example-support-tools
+```
+
+### `Tool '<name>' export '<export>' is not a function`
+
+The module loaded successfully, but the named export does not exist or is not callable.
+
+Check:
+
+- `exportName` matches the actual exported function name
+- the package or local module uses ESM exports as expected
+
 ---
 
 ## HTTP agent errors

package/examples/coding-tools/README.md

@@ -0,0 +1,21 @@
+# Example Coding Tools
+
+Minimal package-style coding-tool example for Agent Regression Lab.
+
+Register it in `agentlab.config.yaml` like this:
+
+```yaml
+tools:
+  - name: coding.read_repo_hint
+    package: "@agentlab/example-coding-tools"
+    exportName: readRepoHint
+    description: Return a small repo hint for the target path.
+    inputSchema:
+      type: object
+      additionalProperties: false
+      properties:
+        path:
+          type: string
+      required:
+        - path
+```

package/examples/coding-tools/index.js

@@ -0,0 +1,11 @@
+export async function readRepoHint(input) {
+  const path = String(input?.path ?? "");
+  if (!path) {
+    throw new Error("path is required");
+  }
+
+  return {
+    path,
+    hint: "Check the target file before editing.",
+  };
+}

package/examples/coding-tools/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "@agentlab/example-coding-tools",
+  "private": true,
+  "type": "module",
+  "exports": {
+    ".": "./index.js"
+  }
+}

package/examples/support-tools/README.md

@@ -0,0 +1,21 @@
+# Example Support Tools
+
+Minimal package-style tool example for Agent Regression Lab.
+
+Register it in `agentlab.config.yaml` like this:
+
+```yaml
+tools:
+  - name: support.find_duplicate_charge
+    package: "@agentlab/example-support-tools"
+    exportName: findDuplicateCharge
+    description: Find the duplicated charge order id for a given customer.
+    inputSchema:
+      type: object
+      additionalProperties: false
+      properties:
+        customer_id:
+          type: string
+      required:
+        - customer_id
+```

package/examples/support-tools/index.js

@@ -0,0 +1,8 @@
+export async function findDuplicateCharge(input) {
+  const customerId = String(input?.customer_id ?? "");
+  if (!customerId) {
+    throw new Error("customer_id is required");
+  }
+
+  return { order_id: `dup_${customerId}` };
+}

package/examples/support-tools/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "@agentlab/example-support-tools",
+  "private": true,
+  "type": "module",
+  "exports": {
+    ".": "./index.js"
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-regression-lab",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "private": false,
   "description": "Local-first scenario-based evaluation harness for AI agents.",
   "license": "MIT",
@@ -21,16 +21,18 @@
   ],
   "type": "module",
   "bin": {
-    "agentlab": "./dist/index.js"
+    "agentlab": "bin/agentlab.js"
   },
   "files": [
+    "bin",
     "dist",
     "dist/ui-assets",
     "README.md",
-    "docs"
+    "docs",
+    "examples"
   ],
   "engines": {
-    "node": ">=22"
+    "node": ">=18"
   },
   "scripts": {
     "build": "tsc -p tsconfig.json && npm run build:ui",