agentv 2.2.0 → 2.5.2

package/dist/templates/.claude/skills/agentv-prompt-optimizer/SKILL.md

@@ -1,77 +1,78 @@
----
-description: Iteratively optimize prompt files against AgentV evaluation datasets by analyzing failures and refining instructions.
----
-
-# AgentV Prompt Optimizer
-
-## Input Variables
-- `eval-path`: Path or glob pattern to the AgentV evaluation file(s) to optimize against
-- `optimization-log-path` (optional): Path where optimization progress should be logged
-
-## Workflow
-
-1.  **Initialize**
-    - Verify `<eval-path>` (file or glob) targets the correct system.
-    - **Identify Prompt Files**:
-        - Infer prompt files from the eval file content (look for `file:` references in `input_messages` that match these patterns).
-        - Recursively check referenced prompt files for *other* prompt references (dependencies).
-        - If multiple prompts are found, consider ALL of them as candidates for optimization.
-    - **Identify Optimization Log**:
-        - If `<optimization-log-path>` is provided, use it.
-        - If not, create a new one in the parent directory of the eval files: `optimization-[timestamp].md`.
-    - Read content of the identified prompt file.
-
-2.  **Optimization Loop** (Max 10 iterations)
-    - **Execute (The Generator)**: Run `agentv eval <eval-path>`.
-        - *Targeted Run*: If iterating on specific stubborn failures, use `--eval-id <case_id>` to run only the relevant eval cases.
-    - **Analyze (The Reflector)**:
-        - Locate the results file path from the console output (e.g., `.agentv/results/eval_...jsonl`).
-        - **Orchestrate Subagent**: Use `runSubagent` to analyze the results.
-            - **Task**: Read the results file, calculate pass rate, and perform root cause analysis.
-            - **Output**: Return a structured analysis including:
-                - **Score**: Current pass rate.
-                - **Root Cause**: Why failures occurred (e.g., "Ambiguous definition", "Hallucination").
-                - **Insight**: Key learning or pattern identified from the failures.
-                - **Strategy**: High-level plan to fix the prompt (e.g., "Clarify section X", "Add negative constraint").
-    - **Decide**:
-        - If **100% pass**: STOP and report success.
-        - If **Score decreased**: Revert last change, try different approach.
-        - If **No improvement** (2x): STOP and report stagnation.
-    - **Refine (The Curator)**:
-        - **Orchestrate Subagent**: Use `runSubagent` to apply the fix.
-            - **Task**: Read the relevant prompt file(s), apply the **Strategy** from the Reflector, and generate the log entry.
-            - **Output**: The **Log Entry** describing the specific operation performed.
-                  ```markdown
-                  ### Iteration [N]
-                  - **Operation**: [ADD / UPDATE / DELETE]
-                  - **Target**: [Section Name]
-                  - **Change**: [Specific text added/modified]
-                  - **Trigger**: [Specific failing test case or error pattern]
-                  - **Rationale**: [From Reflector: Root Cause]
-                  - **Score**: [From Reflector: Current Pass Rate]
-                  - **Insight**: [From Reflector: Key Learning]
-                  ```
-        - **Strategy**: Treat the prompt as a structured set of rules. Execute atomic operations:
-            - **ADD**: Insert a new rule if a constraint was missed.
-            - **UPDATE**: Refine an existing rule to be clearer or more general.
-                - *Clarify*: Make ambiguous instructions specific.
-                - *Generalize*: Refactor specific fixes into high-level principles (First Principles).
-            - **DELETE**: Remove obsolete, redundant, or harmful rules.
-                - *Prune*: If a general rule covers specific cases, delete the specific ones.
-            - **Negative Constraint**: If hallucinating, explicitly state what NOT to do. Prefer generalized prohibitions over specific forbidden tokens where possible.
-            - **Safety Check**: Ensure new rules don't contradict existing ones (unless intended).
-        - **Constraint**: Avoid rewriting large sections. Make surgical, additive changes to preserve existing behavior.
-    - **Log Result**:
-        - Append the **Log Entry** returned by the Curator to the optimization log file.
-
-3.  **Completion**
-    - Report final score.
-    - Summarize key changes made to the prompt.
-    - **Finalize Optimization Log**: Add a summary header to the optimization log file indicating the session completion and final score.
-
-## Guidelines
-- **Generalization First**: Prefer broad, principle-based guidelines over specific examples or "hotfixes". Only use specific rules if generalized instructions fail to achieve the desired score.
-- **Simplicity ("Less is More")**: Avoid overfitting to the test set. If a specific rule doesn't significantly improve the score compared to a general one, choose the general one.
-- **Structure**: Maintain existing Markdown headers/sections.
-- **Progressive Disclosure**: If the prompt grows too large (>200 lines), consider moving specialized logic into a separate file or skill.
-- **Quality Criteria**: Ensure the prompt defines a clear persona, specific task, and measurable success criteria.
+---
+name: agentv-prompt-optimizer
+description: Iteratively optimize prompt files against AgentV evaluation datasets by analyzing failures and refining instructions.
+---
+
+# AgentV Prompt Optimizer
+
+## Input Variables
+- `eval-path`: Path or glob pattern to the AgentV evaluation file(s) to optimize against
+- `optimization-log-path` (optional): Path where optimization progress should be logged
+
+## Workflow
+
+1.  **Initialize**
+    - Verify `<eval-path>` (file or glob) targets the correct system.
+    - **Identify Prompt Files**:
+        - Infer prompt files from the eval file content (look for `file:` references in `input_messages` that match these patterns).
+        - Recursively check referenced prompt files for *other* prompt references (dependencies).
+        - If multiple prompts are found, consider ALL of them as candidates for optimization.
+    - **Identify Optimization Log**:
+        - If `<optimization-log-path>` is provided, use it.
+        - If not, create a new one in the parent directory of the eval files: `optimization-[timestamp].md`.
+    - Read content of the identified prompt file.
+
+2.  **Optimization Loop** (Max 10 iterations)
+    - **Execute (The Generator)**: Run `agentv eval <eval-path>`.
+        - *Targeted Run*: If iterating on specific stubborn failures, use `--eval-id <case_id>` to run only the relevant eval cases.
+    - **Analyze (The Reflector)**:
+        - Locate the results file path from the console output (e.g., `.agentv/results/eval_...jsonl`).
+        - **Orchestrate Subagent**: Use `runSubagent` to analyze the results.
+            - **Task**: Read the results file, calculate pass rate, and perform root cause analysis.
+            - **Output**: Return a structured analysis including:
+                - **Score**: Current pass rate.
+                - **Root Cause**: Why failures occurred (e.g., "Ambiguous definition", "Hallucination").
+                - **Insight**: Key learning or pattern identified from the failures.
+                - **Strategy**: High-level plan to fix the prompt (e.g., "Clarify section X", "Add negative constraint").
+    - **Decide**:
+        - If **100% pass**: STOP and report success.
+        - If **Score decreased**: Revert last change, try different approach.
+        - If **No improvement** (2x): STOP and report stagnation.
+    - **Refine (The Curator)**:
+        - **Orchestrate Subagent**: Use `runSubagent` to apply the fix.
+            - **Task**: Read the relevant prompt file(s), apply the **Strategy** from the Reflector, and generate the log entry.
+            - **Output**: The **Log Entry** describing the specific operation performed.
+                  ```markdown
+                  ### Iteration [N]
+                  - **Operation**: [ADD / UPDATE / DELETE]
+                  - **Target**: [Section Name]
+                  - **Change**: [Specific text added/modified]
+                  - **Trigger**: [Specific failing test case or error pattern]
+                  - **Rationale**: [From Reflector: Root Cause]
+                  - **Score**: [From Reflector: Current Pass Rate]
+                  - **Insight**: [From Reflector: Key Learning]
+                  ```
+        - **Strategy**: Treat the prompt as a structured set of rules. Execute atomic operations:
+            - **ADD**: Insert a new rule if a constraint was missed.
+            - **UPDATE**: Refine an existing rule to be clearer or more general.
+                - *Clarify*: Make ambiguous instructions specific.
+                - *Generalize*: Refactor specific fixes into high-level principles (First Principles).
+            - **DELETE**: Remove obsolete, redundant, or harmful rules.
+                - *Prune*: If a general rule covers specific cases, delete the specific ones.
+            - **Negative Constraint**: If hallucinating, explicitly state what NOT to do. Prefer generalized prohibitions over specific forbidden tokens where possible.
+            - **Safety Check**: Ensure new rules don't contradict existing ones (unless intended).
+        - **Constraint**: Avoid rewriting large sections. Make surgical, additive changes to preserve existing behavior.
+    - **Log Result**:
+        - Append the **Log Entry** returned by the Curator to the optimization log file.
+
+3.  **Completion**
+    - Report final score.
+    - Summarize key changes made to the prompt.
+    - **Finalize Optimization Log**: Add a summary header to the optimization log file indicating the session completion and final score.
+
+## Guidelines
+- **Generalization First**: Prefer broad, principle-based guidelines over specific examples or "hotfixes". Only use specific rules if generalized instructions fail to achieve the desired score.
+- **Simplicity ("Less is More")**: Avoid overfitting to the test set. If a specific rule doesn't significantly improve the score compared to a general one, choose the general one.
+- **Structure**: Maintain existing Markdown headers/sections.
+- **Progressive Disclosure**: If the prompt grows too large (>200 lines), consider moving specialized logic into a separate file or skill.
+- **Quality Criteria**: Ensure the prompt defines a clear persona, specific task, and measurable success criteria.

package/dist/templates/.github/prompts/agentv-eval-build.prompt.md

@@ -1,5 +1,5 @@
----
-description: 'Create and maintain AgentV YAML evaluation files'
----
-
+---
+description: 'Create and maintain AgentV YAML evaluation files'
+---
+
 #file:../../.claude/skills/agentv-eval-builder/SKILL.md

package/dist/templates/.github/prompts/agentv-optimize.prompt.md

@@ -1,4 +1,4 @@
----
-description: Iteratively optimize prompt files against an AgentV evaluation suite
----
+---
+description: Iteratively optimize prompt files against an AgentV evaluation suite
+---
 #file:../../.claude/skills/agentv-prompt-optimizer/SKILL.md

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agentv",
-  "version": "2.2.0",
+  "version": "2.5.2",
   "description": "CLI entry point for AgentV",
   "type": "module",
   "repository": {
@@ -14,10 +14,7 @@
   "bin": {
     "agentv": "./dist/cli.js"
   },
-  "files": [
-    "dist",
-    "README.md"
-  ],
+  "files": ["dist", "README.md"],
   "scripts": {
     "dev": "bun --watch src/index.ts",
     "build": "tsup && bun run copy-readme",
@@ -31,7 +28,7 @@
     "test:watch": "bun test --watch"
   },
   "dependencies": {
-    "@agentv/core": "2.0.2",
+    "@agentv/core": "2.5.1",
     "@mariozechner/pi-agent": "^0.9.0",
     "@mariozechner/pi-ai": "^0.37.2",
     "cmd-ts": "^0.14.3",