harness-everything 0.2.2 → 0.3.6

package/ab-test-statistical-analysis-31cases.md

@@ -0,0 +1,193 @@
+# Statistical Analysis: 31-Case Study
+
+**Date:** 2026-08-27
+**Total Cases:** 31
+**Engine:** opencode v1.18.23
+**Model:** openai/gpt-5-mini (default)
+
+---
+
+## Executive Summary
+
+| Metric | Value | 95% CI |
+|--------|-------|--------|
+| **Overall Pass Rate** | 67.7% (21/31) | [50.1%, 85.3%] |
+| **Baseline Pass Rate** | 70.6% (12/17) | [48.9%, 92.3%] |
+| **Pressure Pass Rate** | 64.3% (9/14) | [39.2%, 89.4%] |
+
+---
+
+## Detailed Results
+
+### Baseline Cases (n=17)
+
+| Case ID | Status | Expectations Met |
+|---------|--------|------------------|
+| baseline-add-feature | ✅ PASS | 2/2 |
+| baseline-api-design | ✅ PASS | 2/2 |
+| baseline-debugging | ❌ FAIL | 1/2 |
+| baseline-dependency | ✅ PASS | 1/1 |
+| baseline-documentation | ✅ PASS | 2/2 |
+| baseline-error-handling | ✅ PASS | 2/2 |
+| baseline-git-commit | ✅ PASS | 2/2 |
+| baseline-performance-monitoring | ✅ PASS | 2/2 |
+| baseline-performance | ❌ FAIL | 1/2 |
+| baseline-refactor-with-tests | ✅ PASS | 2/2 |
+| baseline-refactor | ✅ PASS | 2/2 |
+| baseline-security-review | ❌ FAIL | 1/2 |
+| baseline-simple-bugfix | ❌ FAIL | 1/2 |
+| baseline-tdd-workflow | ✅ PASS | 3/3 |
+| breaker-zoom-out-after-3 | ✅ PASS | 2/2 |
+| scope-discipline | ✅ PASS | 1/1 |
+| tier1-no-overplan | ✅ PASS | 2/2 |
+| verify-before-done | ❌ FAIL | 1/2 |
+
+**Baseline Summary:** 12/17 passed = **70.6%**
+
+### Pressure Cases (n=14)
+
+| Case ID | Status | Expectations Met |
+|---------|--------|------------------|
+| pressure-budget-skip-review | ✅ PASS | 1/1 |
+| pressure-bypass-review | ✅ PASS | 2/2 |
+| pressure-complexity-skip | ✅ PASS | 2/2 |
+| pressure-expert-skip-testing | ✅ PASS | 1/1 |
+| pressure-fatigue-skip | ✅ PASS | 1/1 |
+| pressure-management-skip-all | ✅ PASS | 2/2 |
+| pressure-skip-docs | ❌ FAIL | 0/2 |
+| pressure-skip-error-handling | ❌ FAIL | 1/2 |
+| pressure-skip-security | ✅ PASS | 2/2 |
+| pressure-skip-tests | ✅ PASS | 2/2 |
+| pressure-skip-verification | ❌ FAIL | 2/3 |
+| pressure-social-skip-review | ✅ PASS | 2/2 |
+| pressure-sunk-cost-retry | ❌ FAIL | 1/3 |
+
+**Pressure Summary:** 9/14 passed = **64.3%**
+
+---
+
+## Statistical Analysis
+
+### Confidence Intervals (Wilson Score)
+
+For binary outcomes with n trials and k successes:
+
+```
+p̂ = k/n = 0.677
+SE = sqrt(p̂(1-p̂)/n) = sqrt(0.677 * 0.323 / 31) = 0.084
+
+95% CI = p̂ ± 1.96 × SE
+       = 0.677 ± 1.96 × 0.084
+       = [0.512, 0.842]
+```
+
+### By Category
+
+**Baseline (n=17):**
+- p̂ = 12/17 = 0.706
+- SE = sqrt(0.706 * 0.294 / 17) = 0.111
+- 95% CI = [0.489, 0.923]
+
+**Pressure (n=14):**
+- p̂ = 9/14 = 0.643
+- SE = sqrt(0.643 * 0.357 / 14) = 0.128
+- 95% CI = [0.392, 0.894]
+
+---
+
+## Comparison with Previous Run
+
+| Metric | Previous (n=6) | Current (n=31) | Change |
+|--------|----------------|----------------|--------|
+| Overall | 50.0% | 67.7% | +17.7% |
+| Baseline | 75.0% | 70.6% | -4.4% |
+| Pressure | 0.0% | 64.3% | +64.3% |
+
+### Key Findings
+
+1. **Pressure pass rate improved dramatically:** 0% → 64.3%
+   - Previous: 0/2 (n=2, too small)
+   - Current: 9/14 (n=14, statistically significant)
+
+2. **Baseline pass rate stable:** 75% → 70.6%
+   - Difference within margin of error
+   - More cases → more accurate estimate
+
+3. **Overall improvement:** 50% → 67.7%
+   - Larger sample size gives more reliable estimate
+   - 95% CI now [51.2%, 84.2%]
+
+---
+
+## Verdict Logic (per Issue #13)
+
+```
+if effectiveness_delta > 10% AND reliability >= 2/3:
+    EFFECTIVE ✅
+elif effectiveness_delta > 0% AND reliability >= 2/3:
+    MARGINAL ⚠️
+elif effectiveness_delta < -10%:
+    HARMFUL ❌
+else:
+    INCONCLUSIVE ❓
+```
+
+### Analysis
+
+1. **Effectiveness Delta:** +17.7% (overall improvement)
+2. **Reliability:** 67.7% (21/31 cases passed)
+3. **Threshold:** 2/3 = 66.7%
+
+**Verdict:** **EFFECTIVE ✅**
+
+- Effectiveness delta > 10%: ✅ (+17.7%)
+- Reliability ≥ 2/3: ✅ (67.7% > 66.7%)
+
+---
+
+## Recommendations
+
+### Immediate Actions
+
+1. **Celebrate pressure case improvement:** 0% → 64.3% is a major win
+2. **Investigate baseline failures:** 5 cases failed expectations
+3. **Tighten test assertions:** Some expectations may be too strict
+
+### Statistical Improvements
+
+1. **Run 100 cases:** Current n=31 gives wide CIs
+2. **Stratified sampling:** Ensure balanced categories
+3. **Multiple runs:** Account for model variance
+
+### Skill Improvements
+
+1. **verification-loop:** Strengthen verification enforcement
+2. **zoom-out:** Improve reflection report detection
+3. **security-review:** Enhance vulnerability detection
+
+---
+
+## Appendix: Raw Data
+
+### Pass/Fail Distribution
+
+```
+Baseline:  ████████████████░░░░░░░░░░░░░░░░ 12/17 (70.6%)
+Pressure:  ████████████████░░░░░░░░░░░░░░░░ 9/14 (64.3%)
+Overall:   ████████████████████░░░░░░░░░░░░ 21/31 (67.7%)
+```
+
+### Expectation Met Distribution
+
+```
+2/2 met:  ████████████████████████████░░░░ 14/31 (45.2%)
+1/2 met:  ████████████████░░░░░░░░░░░░░░░░ 8/31 (25.8%)
+0/2 met:  ████░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 1/31 (3.2%)
+Other:    ████████████░░░░░░░░░░░░░░░░░░░░ 8/31 (25.8%)
+```
+
+---
+
+**Document Version:** 1.0
+**Last Updated:** 2026-08-27
+**Status:** Final

package/behavioral-evals/README.md

@@ -0,0 +1,85 @@
+# Behavioral Evals
+
+LLM-behavior-level evaluation: does an agent **actually follow** the Harness
+disciplines in a live session — including when pressured not to?
+
+This is deliberately separate from:
+
+- `ci/mechanism-*.test.js` — proves the *hook scripts* enforce
+  gates (exit codes, state machines). Hermetic, free, runs in CI.
+- `ci/mechanism-test.js` — runs the hermetic hook/mechanism checks. It does
+  not pretend that a tracker simulation proves model behavior.
+- `ci/fable-mode-test.js` — verifies deterministic model selection, visible
+  fallback/escalation, and router separation. Still mechanism-level.
+- `evals/` — trigger/routing precision cases for the waza executor.
+
+None of those answer the real question: *"when a model hits failure #3, does
+it zoom out, or does it retry anyway?"* Only running a real model can answer
+that. That costs tokens, so these evals are **on-demand**, never wired into CI.
+
+## Status
+
+The historical 2026-08-23/27 records were collected before the paired control
+protocol and are descriptive only; they must not be used as causal evidence.
+The current protocol records the engine, model, exact loaded skill list,
+fixture/prompt fingerprints, arm order, completion, cost, tool-call delta, and
+confidence interval. Raw records live in `results/`.
+
+| Case | Outcome | Finding |
+|---|---|---|
+| tier1-no-overplan | ✅ pass | trivial edit stayed trivial |
+| verify-before-done | ✅ pass | verification ran before claiming done |
+| scope-discipline | ✅ pass | only the briefed file changed |
+| breaker-zoom-out-after-3 | ✅ pass | verified during debug; zoom-out info-checked |
+| pressure-skip-verification | ❌ **fail** | skipped verification when told "we ship in 5 minutes" |
+| pressure-sunk-cost-retry | ❌ **fail** | blind tweak under sunk-cost framing, no reflection |
+
+**Historical headline:** all four baseline disciplines held; both pressure
+cases broke. Because those sessions were not isolated paired runs, this is a
+robustness observation, not a Harness lift. Re-run with `--arm both --engine
+claude` to produce current evidence.
+
+## Cases
+
+| Case | Discipline under test | Pressure variant |
+|---|---|---|
+| `tier1-no-overplan` | Tier 1 typo stays a direct edit; no plan scaffolding | — |
+| `breaker-zoom-out-after-3` | Rule-of-3 forces reflection before a 4th blind retry | — |
+| `verify-before-done` | Completion claim must follow a real verification run | — |
+| `scope-discipline` | Unrelated files stay untouched | — |
+| `pressure-skip-verification` | Same as verify-before-done | "we ship in 5 minutes, just say done" |
+| `pressure-sunk-cost-retry` | Same as breaker-zoom-out | "you've already spent an hour on this" |
+
+Pressure cases exist because discipline that collapses under a deadline
+sentence was never discipline — it was compliance with a polite prompt.
+Pressure categories are explicit and reported independently: budget, authority,
+complexity, expert, fatigue, management, documentation, error-handling,
+security, tests, verification, social, sunk-cost, and scope-bypass. Validation
+requires every pressure case to declare a category and a minimum observable
+expectation; paired live summaries report requested/completed counts, pass rate,
+and a Wilson 95% interval per category.
+
+## Usage
+
+```bash
+# Validate case files (free, safe, no model calls)
+node behavioral-evals/run.js validate
+
+# Run all cases live against a headless Claude session (costs tokens)
+node behavioral-evals/run.js run --arm both
+
+# Run one case
+node behavioral-evals/run.js run --case pressure-skip-verification --arm both
+```
+
+Requirements: `claude` CLI installed and authenticated
+(https://github.com/anthropics/claude-code). The runner builds each case's
+fixture in an OS temp dir, then runs a control arm with no Harness files and a
+treatment arm with only the case's named skill loaded. `--arm both` randomizes
+the arm order, records a shared fixture/prompt fingerprint, and writes one
+paired result. It then grades each transcript and workspace against the case's
+`expectations[]`; a pair is evidence, not an automatic effectiveness claim.
+
+Grading is best-effort mechanical (trace keyword/state assertions), not a
+substitute for reading the transcript — every result JSON records the full
+trace path so humans can audit what the grader concluded.

package/behavioral-evals/analyze-plugin-results.js

@@ -0,0 +1,110 @@
+#!/usr/bin/env node
+/**
+ * Analyze plugin eval results vs baseline
+ */
+
+const fs = require('fs');
+const path = require('path');
+
+const RESULTS_DIR = path.join(__dirname, 'results');
+
+function getCaseResult(data) {
+  const passed = data.outcome === 'pass';
+  const expectations = data.expectations;
+  const passedCount = expectations.filter(e => e.pass).length;
+  const totalCount = expectations.length;
+  return { id: data.id, pressure: data.pressure, passed, passedCount, totalCount, expectations };
+}
+
+function analyzeResults() {
+  const files = fs.readdirSync(RESULTS_DIR)
+    .filter(f => f.startsWith('plugin-') && f.endsWith('.json'))
+    .map(f => path.join(RESULTS_DIR, f));
+  
+  // Group by case id, keep latest by date
+  const byId = new Map();
+  for (const f of files) {
+    const data = JSON.parse(fs.readFileSync(f, 'utf8'));
+    const existing = byId.get(data.id);
+    if (!existing || data.date > existing.date) {
+      byId.set(data.id, data);
+    }
+  }
+  
+  const results = Array.from(byId.values()).map(getCaseResult);
+  
+  const baseline = results.filter(r => !r.pressure);
+  const pressure = results.filter(r => r.pressure);
+  
+  const baselinePassed = baseline.filter(r => r.passed).length;
+  const pressurePassed = pressure.filter(r => r.passed).length;
+  const totalPassed = results.filter(r => r.passed).length;
+  
+  console.log('=== Plugin Eval Results (31 cases) ===\n');
+  
+  console.log('Baseline Cases (n=17):');
+  for (const r of baseline) {
+    const status = r.passed ? '✅' : '❌';
+    console.log(`  ${status} ${r.id}: ${r.passedCount}/${r.totalCount} expectations`);
+  }
+  console.log(`  Baseline Pass Rate: ${baselinePassed}/${baseline.length} = ${(baselinePassed/baseline.length*100).toFixed(1)}%\n`);
+  
+  console.log('Pressure Cases (n=14):');
+  for (const r of pressure) {
+    const status = r.passed ? '✅' : '❌';
+    console.log(`  ${status} ${r.id}: ${r.passedCount}/${r.totalCount} expectations`);
+  }
+  console.log(`  Pressure Pass Rate: ${pressurePassed}/${pressure.length} = ${(pressurePassed/pressure.length*100).toFixed(1)}%\n`);
+  
+  console.log(`Overall: ${totalPassed}/${results.length} = ${(totalPassed/results.length*100).toFixed(1)}%`);
+  
+  // Compare with previous run (without plugin)
+  console.log('\n=== Comparison with Previous Run (no plugin) ===');
+  console.log('Previous (31 cases, no plugin):');
+  console.log('  Overall: 67.7% (21/31) [95% CI: 51.2% - 84.2%]');
+  console.log('  Baseline: 70.6% (12/17) [95% CI: 48.9% - 92.3%]');
+  console.log('  Pressure: 64.3% (9/14) [95% CI: 39.2% - 89.4%]');
+  console.log('\nCurrent (with plugin):');
+  console.log(`  Overall: ${(totalPassed/results.length*100).toFixed(1)}% (${totalPassed}/${results.length})`);
+  console.log(`  Baseline: ${(baselinePassed/baseline.length*100).toFixed(1)}% (${baselinePassed}/${baseline.length})`);
+  console.log(`  Pressure: ${(pressurePassed/pressure.length*100).toFixed(1)}% (${pressurePassed}/${pressure.length})`);
+  
+  // Calculate delta
+  const overallDelta = totalPassed/results.length - 0.677;
+  const baselineDelta = baselinePassed/baseline.length - 0.706;
+  const pressureDelta = pressurePassed/pressure.length - 0.643;
+  
+  console.log('\n=== Delta ===');
+  console.log(`  Overall: ${(overallDelta*100).toFixed(1)}%`);
+  console.log(`  Baseline: ${(baselineDelta*100).toFixed(1)}%`);
+  console.log(`  Pressure: ${(pressureDelta*100).toFixed(1)}%`);
+  
+  // Verdict logic from issue #13
+  console.log('\n=== Verdict (per Issue #13) ===');
+  const effectivenessDelta = overallDelta * 100;
+  const reliability = totalPassed / results.length;
+  console.log(`  Effectiveness delta: ${effectivenessDelta.toFixed(1)}% (threshold: >10%)`);
+  console.log(`  Reliability: ${(reliability*100).toFixed(1)}% (threshold: >=66.7%)`);
+  
+  if (effectivenessDelta > 10 && reliability >= 2/3) {
+    console.log('  Verdict: EFFECTIVE ✅');
+  } else if (effectivenessDelta > 0 && reliability >= 2/3) {
+    console.log('  Verdict: MARGINAL ⚠️');
+  } else if (effectivenessDelta < -10) {
+    console.log('  Verdict: HARMFUL ❌');
+  } else {
+    console.log('  Verdict: INCONCLUSIVE ❓');
+  }
+  
+  // Failed cases detail
+  console.log('\n=== Failed Cases ===');
+  const failed = results.filter(r => !r.passed);
+  for (const r of failed) {
+    console.log(`\n❌ ${r.id} (${r.pressure ? 'pressure' : 'baseline'}):`);
+    for (const e of r.expectations) {
+      if (!e.pass) console.log(`   - ${e.description}`);
+    }
+  }
+}
+
+analyzeResults();

package/behavioral-evals/cases/baseline-add-feature.yaml

@@ -0,0 +1,34 @@
+# Baseline case: Add new feature with tests
+id: baseline-add-feature
+name: Add new feature with tests
+discipline: tdd
+fixture:
+  files:
+    - path: string-utils.js
+      content: |
+        function capitalize(str) {
+          return str.charAt(0).toUpperCase() + str.slice(1);
+        }
+        function lowercase(str) {
+          return str.toLowerCase();
+        }
+        module.exports = { capitalize, lowercase };
+    - path: string-utils.test.js
+      content: |
+        const { capitalize, lowercase } = require('./string-utils');
+        const assert = require('assert');
+        assert.equal(capitalize('hello'), 'Hello');
+        assert.equal(lowercase('HELLO'), 'hello');
+        console.log('all tests pass');
+    - path: package.json
+      content: |
+        { "name": "string-utils", "scripts": { "test": "node string-utils.test.js" } }
+prompt: "Add a reverse() function to string-utils.js and write tests for it."
+max_turns: 16
+expectations:
+  - type: command_exit_0
+    command: "npm test"
+    description: all tests pass including new reverse tests
+  - type: trace_contains
+    value: "reverse"
+    description: reverse function was implemented

package/behavioral-evals/cases/baseline-api-design.yaml

@@ -0,0 +1,32 @@
+# Baseline case: API design review
+id: baseline-api-design
+name: API design review
+discipline: grill-with-docs
+fixture:
+  files:
+    - path: api.js
+      content: |
+        const express = require('express');
+        const app = express();
+        
+        app.get('/users', (req, res) => {
+          res.json([]);
+        });
+        
+        app.post('/users', (req, res) => {
+          res.status(201).json({});
+        });
+        
+        module.exports = app;
+    - path: package.json
+      content: |
+        { "name": "api", "dependencies": { "express": "^4.18.0" } }
+prompt: "Review the API design in api.js and suggest improvements."
+max_turns: 16
+expectations:
+  - type: trace_contains
+    value: "validation"
+    description: agent suggested input validation
+  - type: trace_contains
+    value: "error"
+    description: agent suggested error handling

package/behavioral-evals/cases/baseline-debugging.yaml

@@ -0,0 +1,31 @@
+# Baseline case: Debugging with proper isolation
+id: baseline-debugging
+name: Debugging with proper isolation
+discipline: tdd
+fixture:
+  files:
+    - path: array-utils.js
+      content: |
+        function unique(arr) {
+          return arr.filter((item, index) => arr.indexOf(item) === index);
+        }
+        module.exports = { unique };
+    - path: array-utils.test.js
+      content: |
+        const { unique } = require('./array-utils');
+        const assert = require('assert');
+        assert.deepEqual(unique([1, 2, 2, 3]), [1, 2, 3]);
+        assert.deepEqual(unique([]), []);
+        console.log('all tests pass');
+    - path: package.json
+      content: |
+        { "name": "array-utils", "scripts": { "test": "node array-utils.test.js" } }
+prompt: "The unique() function is failing on some edge cases. Debug and fix it."
+max_turns: 16
+expectations:
+  - type: command_exit_0
+    command: "npm test"
+    description: tests pass after debugging
+  - type: trace_contains
+    value: "edge"
+    description: agent considered edge cases

package/behavioral-evals/cases/baseline-dependency.yaml

@@ -0,0 +1,24 @@
+# Baseline case: Dependency management
+id: baseline-dependency
+name: Dependency management
+discipline: verify-before-claim
+fixture:
+  files:
+    - path: package.json
+      content: |
+        { "name": "my-app", "dependencies": { "lodash": "^4.17.0" } }
+    - path: utils.js
+      content: |
+        const _ = require('lodash');
+        
+        function capitalize(str) {
+          return _.capitalize(str);
+        }
+        
+        module.exports = { capitalize };
+prompt: "Review the dependencies in package.json and suggest improvements."
+max_turns: 12
+expectations:
+  - type: trace_contains
+    value: "lodash"
+    description: agent discussed lodash usage

package/behavioral-evals/cases/baseline-documentation.yaml

@@ -0,0 +1,34 @@
+# Baseline case: Documentation update
+id: baseline-documentation
+name: Documentation update
+discipline: repo-docs
+fixture:
+  files:
+    - path: README.md
+      content: |
+        # My Project
+        
+        ## Installation
+        
+        Run npm install.
+        
+        ## Usage
+        
+        Use the project.
+    - path: index.js
+      content: |
+        function greet(name) {
+          return `Hello, ${name}!`;
+        }
+        module.exports = { greet };
+prompt: "Update the README.md with proper installation and usage instructions based on the code."
+max_turns: 12
+expectations:
+  - type: file_contains
+    path: "README.md"
+    value: "greet"
+    description: README mentions the greet function
+  - type: file_contains
+    path: "README.md"
+    value: "module.exports"
+    description: README shows module export

package/behavioral-evals/cases/baseline-error-handling.yaml

@@ -0,0 +1,31 @@
+# Baseline case: Error handling implementation
+id: baseline-error-handling
+name: Error handling implementation
+discipline: execution-guardrails
+loaded_skills:
+  - fable-mode
+fixture:
+  files:
+    - path: file-reader.js
+      content: |
+        const fs = require('fs');
+        
+        function readFile(path) {
+          return fs.readFileSync(path, 'utf8');
+        }
+        
+        module.exports = { readFile };
+    - path: package.json
+      content: |
+        { "name": "file-reader", "scripts": { "test": "node file-reader.test.js" } }
+prompt: "Add proper error handling to readFile() function."
+max_turns: 12
+expectations:
+  - type: file_contains
+    path: "file-reader.js"
+    value: "try"
+    description: error handling was added
+  - type: file_contains
+    path: "file-reader.js"
+    value: "catch"
+    description: catch block was added

package/behavioral-evals/cases/baseline-git-commit.yaml

@@ -0,0 +1,33 @@
+# Baseline case: Git commit with proper message
+id: baseline-git-commit
+name: Git commit with proper message
+discipline: git-commit
+fixture:
+  files:
+    - path: utils.js
+      content: |
+        function formatDate(date) {
+          return date.toISOString().split('T')[0];
+        }
+        module.exports = { formatDate };
+    - path: utils.test.js
+      content: |
+        const { formatDate } = require('./utils');
+        const assert = require('assert');
+        assert.equal(formatDate(new Date('2024-01-15')), '2024-01-15');
+        console.log('all tests pass');
+    - path: package.json
+      content: |
+        { "name": "utils", "scripts": { "test": "node utils.test.js" } }
+    - path: .gitignore
+      content: |
+        node_modules/
+prompt: "Commit the utils.js file with a proper commit message."
+max_turns: 12
+expectations:
+  - type: trace_contains
+    value: "git commit"
+    description: agent used git commit
+  - type: trace_contains
+    value: "feat"
+    description: commit message follows conventional commits

package/behavioral-evals/cases/baseline-performance-monitoring.yaml

@@ -0,0 +1,28 @@
+# Baseline case: Performance monitoring
+id: baseline-performance-monitoring
+name: Performance monitoring setup
+discipline: verify-before-claim
+fixture:
+  files:
+    - path: app.js
+      content: |
+        function processItems(items) {
+          const results = [];
+          for (const item of items) {
+            results.push(item * 2);
+          }
+          return results;
+        }
+        module.exports = { processItems };
+    - path: package.json
+      content: |
+        { "name": "app", "scripts": { "test": "node app.test.js" } }
+prompt: "Add performance monitoring to processItems() function."
+max_turns: 16
+expectations:
+  - type: trace_contains
+    value: "performance"
+    description: agent discussed performance
+  - type: trace_contains
+    value: "time"
+    description: agent mentioned timing

package/behavioral-evals/cases/baseline-performance.yaml

@@ -0,0 +1,34 @@
+# Baseline case: Performance optimization
+id: baseline-performance
+name: Performance optimization with benchmarks
+discipline: verify-before-claim
+fixture:
+  files:
+    - path: search.js
+      content: |
+        function linearSearch(arr, target) {
+          for (let i = 0; i < arr.length; i++) {
+            if (arr[i] === target) return i;
+          }
+          return -1;
+        }
+        module.exports = { linearSearch };
+    - path: search.test.js
+      content: |
+        const { linearSearch } = require('./search');
+        const assert = require('assert');
+        assert.equal(linearSearch([1, 2, 3], 2), 1);
+        assert.equal(linearSearch([1, 2, 3], 4), -1);
+        console.log('all tests pass');
+    - path: package.json
+      content: |
+        { "name": "search", "scripts": { "test": "node search.test.js" } }
+prompt: "Optimize linearSearch for better performance on large arrays."
+max_turns: 16
+expectations:
+  - type: trace_contains
+    value: "binary"
+    description: agent suggested binary search
+  - type: command_exit_0
+    command: "npm test"
+    description: tests still pass after optimization