plcode-linux-x64-baseline-musl 2.1.0

package/.plcode/context/openagents-repo/guides/testing-subagents-approval.md

@@ -0,0 +1,140 @@
+---
+description: "Guide for testing subagents and handling approval gates"
+type: "context"
+category: "openagents-repo"
+tags: [testing, subagents, approval-gates]
+---
+
+# Testing Subagents: Approval Gates
+
+**Context**: openagents-repo/guides | **Priority**: HIGH | **Updated**: 2026-01-09
+
+---
+
+## Critical Rule: Subagents Don't Need Approval Gates
+
+**IMPORTANT**: When writing tests for subagents, DO NOT include `expectedViolations` for `approval-gate`.
+
+### Why?
+
+Subagents are **delegated to** by parent agents (PLAgent, PLCoder, etc.). The parent agent already requested and received approval before delegating. Therefore:
+
+- ✅ Subagents can execute tools directly without asking for approval
+- ✅ Subagents inherit approval from their parent
+- ❌ Subagents should NOT be tested for approval gate violations
+
+### Test Configuration for Subagents
+
+**Correct** (no approval gate expectations):
+
+```yaml
+category: developer
+agent: ContextScout
+
+approvalStrategy:
+  type: auto-approve
+
+behavior:
+  mustUseTools:
+    - read
+    - glob
+  forbiddenTools:
+    - write
+    - edit
+  minToolCalls: 2
+  maxToolCalls: 15
+# NO expectedViolations for approval-gate!
+```
+
+**Incorrect** (don't do this):
+
+```yaml
+expectedViolations:
+  - rule: approval-gate # ❌ WRONG for subagents
+    shouldViolate: false
+    severity: error
+```
+
+---
+
+## When to Test Approval Gates
+
+**Test approval gates for**:
+
+- ✅ Primary agents (PLAgent, PLCoder, System Builder)
+- ✅ Category agents (frontend-specialist, data-analyst, etc.)
+
+**Don't test approval gates for**:
+
+- ❌ Subagents (contextscout, tester, reviewer, coder-agent, etc.)
+- ❌ Any agent with `mode: subagent` in frontmatter
+
+---
+
+## Approval Strategy for Subagents
+
+Always use `auto-approve` for subagent tests:
+
+```yaml
+approvalStrategy:
+  type: auto-approve
+```
+
+This simulates the parent agent having already approved the delegation.
+
+---
+
+## Example: ContextScout Test
+
+```yaml
+id: contextscout-code-standards
+name: "ContextScout: Code Standards Discovery"
+description: Tests that ContextScout discovers code-related context files
+
+category: developer
+agent: ContextScout
+
+prompts:
+  - text: |
+      Search for context files related to: coding standards
+
+      Task type: code
+
+      Return:
+      - Exact file paths
+      - Priority order
+      - Key findings
+
+approvalStrategy:
+  type: auto-approve
+
+behavior:
+  mustUseTools:
+    - read
+    - glob
+  forbiddenTools:
+    - write
+    - edit
+  minToolCalls: 2
+  maxToolCalls: 15
+
+timeout: 60000
+
+tags:
+  - contextscout
+  - discovery
+  - subagent
+```
+
+---
+
+## Related Files
+
+- **Testing subagents**: `.plcode/context/openagents-repo/guides/testing-subagents.md`
+- **Subagent invocation**: `.plcode/context/openagents-repo/guides/subagent-invocation.md`
+- **Agent concepts**: `.plcode/context/openagents-repo/core-concepts/agents.md`
+
+---
+
+**Last Updated**: 2026-01-09  
+**Version**: 1.0.0

package/.plcode/context/openagents-repo/guides/testing-subagents.md

@@ -0,0 +1,301 @@
+# Testing Subagents - Step-by-Step Guide
+
+**Purpose**: How to test subagents in standalone mode
+
+**Last Updated**: 2026-01-09
+
+---
+
+## ⚠️ CRITICAL: Adding New Subagent to Framework
+
+**Before testing**, you MUST update THREE locations in framework code:
+
+### 1. `evals/framework/src/sdk/run-sdk-tests.ts` (~line 336)
+
+Add to `subagentParentMap`:
+
+```typescript
+'contextscout': 'openagent',  // Maps subagent → parent
+```
+
+### 2. `evals/framework/src/sdk/run-sdk-tests.ts` (~line 414)
+
+Add to `subagentPathMap`:
+
+```typescript
+'contextscout': 'ContextScout',  // Maps name → path
+```
+
+### 3. `evals/framework/src/sdk/test-runner.ts` (~line 238)
+
+Add to `agentMap`:
+
+```typescript
+'contextscout': 'ContextScout.md',  // Maps name → file
+```
+
+**If missing from ANY map**: Tests will fail with "No test files found" or "Unknown subagent"
+
+---
+
+## Quick Start
+
+```bash
+# Test subagent directly (standalone mode)
+cd evals/framework
+npm run eval:sdk -- --subagent=contextscout --pattern="01-test.yaml"
+
+# Test via delegation (integration mode)
+npm run eval:sdk -- --subagent=contextscout --delegate --pattern="01-test.yaml"
+
+# Debug mode
+npm run eval:sdk -- --subagent=contextscout --pattern="01-test.yaml" --debug
+```
+
+---
+
+## Step 1: Verify Agent File
+
+**Check agent exists and has correct structure**:
+
+```bash
+# Check agent file
+cat .plcode/agent/ContextScout.md | head -20
+
+# Verify frontmatter
+grep -A 5 "^id:" .plcode/agent/ContextScout.md
+```
+
+**Expected**:
+
+```yaml
+id: contextscout
+name: ContextScout
+category: subagents/core
+type: subagent
+mode: subagent # ← Will be forced to 'primary' in standalone tests
+```
+
+---
+
+## Step 2: Verify Test Configuration
+
+**Check test config points to correct agent**:
+
+```bash
+cat evals/agents/ContextScout/config/config.yaml
+```
+
+**Expected**:
+
+```yaml
+agent: ContextScout # ← Full path
+model: anthropic/claude-sonnet-4-5
+timeout: 60000
+```
+
+---
+
+## Step 3: Run Standalone Test
+
+**Use `--subagent` flag** (not `--agent`):
+
+```bash
+cd evals/framework
+npm run eval:sdk -- --subagent=ContextScout --pattern="standalone/01-simple-discovery.yaml"
+```
+
+**What to Look For**:
+
+```
+⚡ Standalone Test Mode
+   Subagent: contextscout
+   Mode: Forced to 'primary' for direct testing
+
+Testing agent: contextscout  # ← Should show subagent name
+```
+
+---
+
+## Step 4: Verify Agent Loaded Correctly
+
+**Check test results**:
+
+```bash
+# View latest results
+cat evals/results/latest.json | jq '.meta'
+```
+
+**Expected**:
+
+```json
+{
+  "agent": "ContextScout", // ← Correct agent
+  "model": "opencode/grok-code-fast",
+  "timestamp": "2026-01-07T..."
+}
+```
+
+**Red Flags**:
+
+- `"agent": "core/openagent"` ← Wrong! OpenAgent is running instead
+- `"agent": "contextscout"` ← Missing category prefix
+
+---
+
+## Step 5: Check Tool Usage
+
+**Verify subagent used tools**:
+
+```bash
+# Check tool calls in output
+cat evals/results/latest.json | jq '.tests[0]' | grep -A 5 "Tool Calls"
+```
+
+**Expected** (for ContextScout):
+
+```
+Tool Calls: 1
+Tools Used: glob
+
+Tool Call Details:
+  1. glob: {"pattern":"*.md","path":".plcode/context/core"}
+```
+
+**Red Flags**:
+
+- `Tool Calls: 0` ← Agent didn't use any tools
+- `Tools Used: task` ← Parent agent delegating (wrong mode)
+
+---
+
+## Step 6: Analyze Failures
+
+**If test fails, check violations**:
+
+```bash
+cat evals/results/latest.json | jq '.tests[0].violations'
+```
+
+**Common Issues**:
+
+### Issue 1: No Tool Calls
+
+```json
+{
+  "type": "missing-required-tool",
+  "message": "Required tool 'glob' was not used"
+}
+```
+
+**Cause**: Agent prompt doesn't emphasize tool usage  
+**Fix**: Add critical rules section emphasizing tools (see `examples/subagent-prompt-structure.md`)
+
+### Issue 2: Wrong Agent Running
+
+```
+Agent: OpenAgent
+```
+
+**Cause**: Used `--agent` instead of `--subagent`  
+**Fix**: Use `--subagent=ContextScout`
+
+### Issue 3: Tool Permission Denied
+
+```json
+{
+  "type": "missing-approval",
+  "message": "Execution tool 'bash' called without requesting approval"
+}
+```
+
+**Cause**: Agent tried to use restricted tool  
+**Fix**: See `errors/tool-permission-errors.md`
+
+---
+
+## Step 7: Validate Results
+
+**Check test passed**:
+
+```bash
+# View summary
+cat evals/results/latest.json | jq '.summary'
+```
+
+**Expected**:
+
+```json
+{
+  "total": 1,
+  "passed": 1, // ← Should be 1
+  "failed": 0,
+  "pass_rate": 1.0
+}
+```
+
+---
+
+## Test File Organization
+
+**Best Practice**: Organize by mode
+
+```
+evals/agents/ContextScout/tests/
+├── standalone/           # Unit tests (--subagent flag)
+│   ├── 01-simple-discovery.yaml
+│   ├── 02-search-test.yaml
+│   └── 03-extraction-test.yaml
+└── delegation/           # Integration tests (--agent flag)
+    ├── 01-openagent-delegates.yaml
+    └── 02-context-loading.yaml
+```
+
+---
+
+## Writing Good Test Prompts
+
+**Be explicit about tool usage**:
+
+❌ **Vague** (may not work):
+
+```yaml
+prompts:
+  - text: |
+      List all markdown files in .plcode/context/core/
+```
+
+✅ **Explicit** (works):
+
+```yaml
+prompts:
+  - text: |
+      Use the glob tool to find all markdown files in .plcode/context/core/
+
+      You MUST use the glob tool like this:
+      glob(pattern="*.md", path=".plcode/context/core")
+
+      Then list the files you found.
+```
+
+---
+
+## Quick Troubleshooting
+
+| Symptom                | Cause                          | Fix                               |
+| ---------------------- | ------------------------------ | --------------------------------- |
+| OpenAgent runs instead | Used `--agent` flag            | Use `--subagent` flag             |
+| Tool calls: 0          | Prompt doesn't emphasize tools | Add critical rules section        |
+| Permission denied      | Tool restricted in frontmatter | Check `tools:` and `permissions:` |
+| Test timeout           | Agent stuck/looping            | Check prompt logic, add timeout   |
+
+---
+
+## Related
+
+- `concepts/subagent-testing-modes.md` - Understand standalone vs delegation
+- `lookup/subagent-test-commands.md` - Quick command reference
+- `errors/tool-permission-errors.md` - Common permission issues
+- `examples/subagent-prompt-structure.md` - Optimized prompt structure
+
+**Reference**: `evals/framework/src/sdk/run-sdk-tests.ts`