create-expert 0.0.38 → 0.0.39

package/dist/perstack.toml

@@ -68,23 +68,62 @@ pick = ["readTextFile", "writeTextFile", "listDirectory", "think", "attemptCompl
 3. Based on the user's request, draft the expert definition
 4. Create the expert in memory using `createExpert` to validate the definition
 5. Add it as a delegate using `addDelegate` so you can test it
-6. Test the expert by calling the delegate tool with a simple, realistic query
-7. Review the result: check that the expert behaves as expected
-8. If the test shows errors or unexpected behavior:
+6. **Practical test**: Call the delegate with a realistic query that matches what the user would actually ask (see Testing Guide below)
+7. **Verify outputs**: After the delegate returns, verify the actual artifacts and process (see Testing Guide below)
+8. If the test shows errors, missing artifacts, or quality issues:
    - Use `removeDelegate` to remove the current delegate
    - Modify the definition and call `createExpert` again with the same key
    - Add it as a delegate again with `addDelegate` and re-test
-9. Once the expert works correctly, write the final `perstack.toml` using `writeTextFile`
+9. Once the expert produces correct, complete outputs, write the final `perstack.toml` using `writeTextFile`
 10. Use `attemptCompletion` when the expert is created and verified
 
-## Testing with createExpert + addDelegate
+## Testing Guide
 
-After drafting an expert definition, always test it in memory before writing perstack.toml:
-- Use `createExpert` with the expert key, instruction, description, skills, and other fields
-- Use `addDelegate` with the expert key to make it callable
-- Call the delegate tool with a simple query that exercises the expert's core functionality
-- Review the result to verify correctness
-- If issues arise, iterate: `removeDelegate` -> fix -> `createExpert` -> `addDelegate` -> re-test
+You MUST perform practical, end-to-end testing before writing perstack.toml. The test must simulate the user's actual use case, not just check that the expert "runs without errors".
+
+### Step 1: Design a realistic test query
+
+Before calling the delegate, think about what the user will actually ask this expert to do. The test query should be a concrete, representative task — not a trivial or abstract one.
+
+- If the expert generates code: ask it to generate a small but complete, realistic piece (e.g., "Create a responsive landing page with a hero section, feature cards, and a contact form")
+- If the expert writes documentation: ask it to document a specific scenario (e.g., "Write API documentation for a user authentication endpoint with examples")
+- If the expert performs analysis: give it real-looking data to analyze
+- If the expert manages a workflow with sub-experts: give it a task that exercises delegation to at least one sub-expert
+
+### Step 2: Verify the artifacts after delegation
+
+After the delegate returns its text result, you must verify what was actually produced. Do NOT just read the delegate's response text and assume success.
+
+**For experts that create files:**
+1. Use `listDirectory` to confirm all expected files were created
+2. Use `readTextFile` to read each generated file
+3. Check that file contents are correct, complete, and well-structured
+4. Verify no placeholder content (e.g., "TODO", "Lorem ipsum" where real content is expected)
+
+**For experts that modify existing files:**
+1. Use `readTextFile` to read the modified files
+2. Verify the changes are correct and the file is still valid
+3. Check that unrelated parts of the file were not damaged
+
+**For experts that perform tasks (build, test, deploy, etc.):**
+1. Use `exec` to run `perstack logs --last` to inspect the execution process
+2. Verify the task steps were performed in the correct order
+3. Check that the final state matches expectations (files created, commands run, etc.)
+
+**For experts with delegates (coordinator/lead experts):**
+1. Use `exec` to run `perstack logs --last` to verify delegation occurred
+2. Confirm that each sub-expert was called with appropriate queries
+3. Verify the coordinator properly synthesized results from sub-experts
+
+### Step 3: Evaluate quality, not just correctness
+
+Ask yourself: "If I were the user, would I be satisfied with this output?"
+- Is the output complete, or are parts missing?
+- Is the quality appropriate for the task?
+- Does the expert follow its instruction faithfully?
+- Would the user need to manually fix or redo anything?
+
+If the answer to any of these is unsatisfactory, iterate: fix the instruction, recreate, and re-test.
 
 ## Important Rules
 
@@ -108,6 +147,7 @@ pick = [
   "writeTextFile",
   "listDirectory",
   "getFileInfo",
+  "exec",
   "think",
   "attemptCompletion",
   "createExpert",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "create-expert",
-  "version": "0.0.38",
+  "version": "0.0.39",
   "description": "Create and modify Perstack expert definitions",
   "author": "Wintermute Technologies, Inc.",
   "license": "Apache-2.0",
@@ -22,10 +22,10 @@
     "@types/node": "^25.2.3",
     "tsdown": "^0.20.3",
     "typescript": "^5.9.3",
-    "@perstack/core": "0.0.50",
     "@perstack/perstack-toml": "0.0.6",
-    "@perstack/runtime": "0.0.108",
-    "@perstack/tui": "0.0.9"
+    "@perstack/core": "0.0.50",
+    "@perstack/runtime": "0.0.110",
+    "@perstack/tui": "0.0.10"
   },
   "engines": {
     "node": ">=22.0.0"