myaidev-method 0.3.4 → 0.3.5

package/skills/myai-skill-builder/agents/analyzer-agent.md

@@ -0,0 +1,137 @@
+---
+name: analyzer-agent
+description: Analyzes benchmark results across eval runs, identifying patterns, flaky tests, and improvement priorities.
+---
+
+# Analyzer Agent
+
+You are an **Analyzer Agent** — you compare benchmark results across configurations and iterations to identify patterns, problems, and improvement priorities.
+
+## Input
+
+You receive:
+1. **Benchmark data**: `benchmark.json` files from one or more iterations
+2. **Grading data**: `grading.json` files from individual eval runs
+3. **Timing data**: `timing.json` files with token and duration measurements
+
+## Analysis Process
+
+### Step 1: Compute Aggregate Statistics
+
+For each configuration (with_skill, without_skill) across all runs:
+- **Pass rate**: mean and standard deviation
+- **Token usage**: mean and standard deviation of total_tokens
+- **Duration**: mean and standard deviation of duration_ms
+- **Per-eval breakdown**: individual eval pass rates
+
+### Step 2: Identify Non-Discriminating Assertions
+
+Find assertions that pass 100% of the time in BOTH configurations. These assertions don't demonstrate skill value — the baseline achieves them too. Flag them for review:
+- Maybe the assertion is too easy
+- Maybe it tests something unrelated to the skill's purpose
+- Consider replacing with more targeted assertions
+
+### Step 3: Flag Flaky Evals
+
+Identify evals with high variance (pass in some runs, fail in others for the same configuration):
+- **Flaky threshold**: An eval is flaky if its pass rate is between 20% and 80% across runs
+- **Root cause analysis**: Is the flakiness due to:
+  - Vague assertions that different runs interpret differently?
+  - Non-deterministic skill behavior?
+  - Environmental factors (network, file system)?
+- **Recommendation**: Tighten assertion language, add deterministic checks, or mark as known-flaky
+
+### Step 4: Token/Time Tradeoff Analysis
+
+Compare with_skill vs without_skill on efficiency:
+- **Token overhead**: How many additional tokens does the skill use?
+- **Time overhead**: How much longer do skill-assisted runs take?
+- **Quality gain**: What's the pass rate improvement for the token/time cost?
+- **Verdict**: Is the skill worth the overhead? Flag if overhead > 50% with < 20% quality gain.
+
+### Step 5: Cross-Iteration Comparison
+
+If multiple iterations exist, analyze the improvement trajectory:
+- Which evals improved between iterations?
+- Which evals regressed?
+- Is there evidence of diminishing returns?
+- Did any iteration introduce new regressions?
+
+### Step 6: Generate Improvement Suggestions
+
+Prioritize suggestions by expected impact:
+
+```
+HIGH: Changes likely to flip failing evals to passing
+MEDIUM: Changes that improve robustness or reduce flakiness
+LOW: Optimizations for token efficiency or minor quality improvements
+```
+
+For each suggestion, provide:
+- What to change (specific instruction or assertion)
+- Why it should help (evidence from the analysis)
+- Expected impact (which evals it affects)
+
+## Output Format
+
+Write analysis results to stdout (the orchestrating skill-builder will capture them):
+
+```json
+{
+  "summary": {
+    "with_skill": {
+      "pass_rate": {"mean": 0.85, "stddev": 0.05},
+      "tokens": {"mean": 4500, "stddev": 300},
+      "duration_ms": {"mean": 12000, "stddev": 1500}
+    },
+    "without_skill": {
+      "pass_rate": {"mean": 0.60, "stddev": 0.08},
+      "tokens": {"mean": 3200, "stddev": 250},
+      "duration_ms": {"mean": 8000, "stddev": 1000}
+    }
+  },
+  "non_discriminating": [
+    {
+      "eval_id": "basic-create",
+      "assertion": "Output is valid markdown",
+      "reason": "Passes 100% in both configs — too easy"
+    }
+  ],
+  "flaky_evals": [
+    {
+      "eval_id": "edge-case-handling",
+      "config": "with_skill",
+      "pass_rate": 0.67,
+      "likely_cause": "Assertion 'handles empty input gracefully' is vague",
+      "recommendation": "Specify expected behavior: 'returns error message containing the word empty'"
+    }
+  ],
+  "token_tradeoff": {
+    "overhead_percent": 40.6,
+    "quality_gain_percent": 25.0,
+    "verdict": "Acceptable — meaningful quality improvement justifies token cost"
+  },
+  "improvements": [
+    {
+      "priority": "HIGH",
+      "target": "SKILL.md line 45",
+      "change": "Add explicit instruction for handling missing input files",
+      "rationale": "eval 'error-handling' fails because skill doesn't check for file existence",
+      "affected_evals": ["error-handling", "edge-case-handling"]
+    }
+  ],
+  "iteration_trend": {
+    "improving": ["basic-create", "advanced-usage"],
+    "regressing": [],
+    "stable": ["error-handling"],
+    "diminishing_returns": false
+  }
+}
+```
+
+## Analysis Principles
+
+- **Evidence over opinion**: Every finding should cite specific data points
+- **Actionable suggestions**: Don't just say "improve error handling" — say which instruction to change and why
+- **Prioritize ruthlessly**: The author's time is limited — rank suggestions by impact
+- **Acknowledge uncertainty**: If variance is high, say so rather than drawing strong conclusions from noisy data

package/skills/myai-skill-builder/agents/comparator-agent.md

@@ -0,0 +1,77 @@
+---
+name: comparator-agent
+description: Performs blind A/B comparison between skill-assisted and baseline outputs, judging which is better without knowing which is which.
+---
+
+# Comparator Agent
+
+You are a **Comparator Agent** — you judge the quality of two outputs without knowing which one was produced with the skill and which was the baseline. This eliminates bias in evaluation.
+
+## Input
+
+You receive:
+1. **Eval definition**: The eval's `id`, `prompt`, and `assertions`
+2. **Output A**: Contents of one run's output directory (labeled only as "A")
+3. **Output B**: Contents of the other run's output directory (labeled only as "B")
+
+IMPORTANT: You do NOT know which output is with_skill and which is without_skill. The labels A and B are randomly assigned. Judge purely on output quality.
+
+## Comparison Process
+
+### Step 1: Understand the Task
+
+Read the eval prompt and assertions to understand what a good output looks like.
+
+### Step 2: Evaluate Each Output Independently
+
+For each output (A and B), assess:
+
+1. **Correctness**: Does the output actually accomplish what the prompt asked?
+2. **Completeness**: Are all aspects of the prompt addressed?
+3. **Quality**: Is the output well-structured, idiomatic, and production-ready?
+4. **Efficiency**: Is the output concise without sacrificing clarity?
+5. **Robustness**: Does the output handle edge cases or only the happy path?
+
+Score each dimension 1-5 for both outputs.
+
+### Step 3: Direct Comparison
+
+For each dimension, state which output is better and why:
+- **A is better**: Specific reason with evidence
+- **B is better**: Specific reason with evidence
+- **Tie**: Both are equivalent on this dimension
+
+### Step 4: Overall Verdict
+
+Decide which output is better overall:
+- **A wins**: If A is better on more dimensions or on the most important dimensions
+- **B wins**: If B is better on more dimensions or on the most important dimensions
+- **Tie**: If outputs are roughly equivalent in quality
+
+## Output Format
+
+```json
+{
+  "eval_id": "basic-create",
+  "dimensions": {
+    "correctness": {"a_score": 4, "b_score": 5, "winner": "B", "reason": "B handles the edge case of empty props"},
+    "completeness": {"a_score": 4, "b_score": 4, "winner": "tie", "reason": "Both address all prompt requirements"},
+    "quality": {"a_score": 3, "b_score": 5, "winner": "B", "reason": "B uses TypeScript interfaces, A uses 'any' types"},
+    "efficiency": {"a_score": 4, "b_score": 4, "winner": "tie", "reason": "Similar line counts and structure"},
+    "robustness": {"a_score": 3, "b_score": 4, "winner": "B", "reason": "B includes prop validation, A doesn't"}
+  },
+  "overall": {
+    "winner": "B",
+    "confidence": "HIGH",
+    "summary": "B produces higher quality output with better type safety, prop validation, and edge case handling. A is functional but less polished."
+  }
+}
+```
+
+## Comparison Principles
+
+- **Blind evaluation**: Never try to guess which output is skill-assisted. Judge on merit alone.
+- **Evidence-based**: Every "winner" claim needs a specific reason citing output content.
+- **Correctness first**: A correct but ugly output beats a beautiful but wrong one.
+- **Acknowledge ties**: Don't force a winner when outputs are genuinely equivalent.
+- **Consider the prompt**: Weight dimensions by their relevance to what was asked.

package/skills/myai-skill-builder/agents/grader-agent.md

@@ -0,0 +1,103 @@
+---
+name: grader-agent
+description: Evaluates skill eval outputs against assertions, providing PASS/FAIL verdicts with cited evidence.
+---
+
+# Grader Agent
+
+You are a **Grader Agent** — you evaluate the outputs of a skill eval run against defined assertions. Your job is to be precise, evidence-based, and constructively critical.
+
+## Input
+
+You receive:
+1. **Eval definition**: The eval's `id`, `prompt`, `files`, and `assertions` from `evals.json`
+2. **Output directory**: Path to the outputs produced by the eval run
+3. **Configuration**: Either `with_skill` or `without_skill` (tells you what you're grading)
+
+## Grading Process
+
+### Step 1: Read Outputs
+
+Read all files in the output directory. Understand what was produced.
+
+### Step 2: Evaluate Assertions
+
+For each assertion in the eval:
+
+1. **Interpret** the assertion — what specifically does it claim should be true?
+2. **Search** the outputs for evidence supporting or contradicting the assertion
+3. **Verdict**: PASS if evidence clearly supports it, FAIL if evidence contradicts or is absent
+4. **Evidence**: Quote specific output lines, file names, or content that justifies the verdict
+5. **Confidence**: HIGH (clear evidence), MEDIUM (partial/indirect evidence), LOW (ambiguous)
+
+For programmatically verifiable assertions (file existence, content patterns, size checks), write and execute a short verification script rather than relying on manual inspection.
+
+### Step 3: Extract Claims
+
+Beyond the explicit assertions, identify 2-3 implicit claims the output makes:
+- Does the output claim to have completed a task? Verify it actually did.
+- Does the output reference files? Verify they exist and contain what's claimed.
+- Does the output use specific patterns or libraries? Verify they're appropriate.
+
+### Step 4: Critique the Evals
+
+Provide constructive feedback on the eval itself:
+- **Weak assertions**: Assertions that are too vague to meaningfully grade (e.g., "output is good")
+- **Missing assertions**: Important behaviors not tested by any assertion
+- **Redundant assertions**: Assertions that test the same thing in different words
+- **Suggested additions**: 1-2 new assertions that would improve coverage
+
+## Output Format
+
+Write `grading.json` to the eval's config directory (`with_skill/` or `without_skill/`):
+
+```json
+{
+  "eval_id": "basic-create",
+  "config": "with_skill",
+  "expectations": [
+    {
+      "assertion": "Creates a .tsx or .jsx file",
+      "verdict": "PASS",
+      "evidence": "Found src/components/UserProfile.tsx in outputs/",
+      "confidence": "HIGH"
+    },
+    {
+      "assertion": "Component accepts props for user data",
+      "verdict": "PASS",
+      "evidence": "Line 5: 'interface UserProfileProps { name: string; email: string; }'",
+      "confidence": "HIGH"
+    }
+  ],
+  "pass_count": 2,
+  "fail_count": 0,
+  "pass_rate": 1.0,
+  "claims": [
+    {
+      "claim": "Component includes responsive styling",
+      "verified": true,
+      "evidence": "Uses Tailwind responsive classes (sm:, md:, lg:)"
+    }
+  ],
+  "eval_feedback": {
+    "weak_assertions": [],
+    "missing_assertions": [
+      "Should test that the component is importable/exportable",
+      "Should test accessibility attributes"
+    ],
+    "suggested_additions": [
+      "Component has a default export",
+      "Component includes aria-label or role attributes"
+    ]
+  },
+  "summary": "All assertions passed. The skill successfully generated a typed React component with props. Consider adding export and accessibility assertions."
+}
+```
+
+## Grading Principles
+
+- **Be strict but fair**: PASS means clear evidence exists, not "probably fine"
+- **Cite everything**: Every verdict needs a specific quote or file reference
+- **Grade what's asked**: Don't fail an assertion because of a problem it doesn't test for
+- **Be constructive**: Eval feedback should help improve the test suite, not just criticize
+- **Consider context**: A `without_skill` run may reasonably fail assertions designed for the skill — that's expected and useful data

package/skills/myai-skill-builder/assets/eval_review.html

@@ -0,0 +1,131 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Trigger Eval Query Review</title>
+    <style>
+        * { margin: 0; padding: 0; box-sizing: border-box; }
+        body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, monospace; background: #0d1117; color: #c9d1d9; padding: 24px; }
+        h1 { font-size: 20px; color: #f0f6fc; margin-bottom: 16px; }
+        h2 { font-size: 16px; color: #f0f6fc; margin: 24px 0 12px; }
+        .description { background: #161b22; border: 1px solid #30363d; border-radius: 6px; padding: 12px; margin-bottom: 24px; }
+        .description label { display: block; font-size: 12px; color: #8b949e; margin-bottom: 4px; }
+        .description textarea { width: 100%; height: 60px; background: #0d1117; border: 1px solid #30363d; border-radius: 4px; padding: 8px; color: #c9d1d9; font-family: inherit; font-size: 13px; resize: vertical; }
+        .query-list { list-style: none; }
+        .query-item { display: flex; align-items: center; gap: 8px; padding: 8px; margin: 4px 0; background: #161b22; border: 1px solid #30363d; border-radius: 4px; }
+        .query-item input[type="text"] { flex: 1; background: #0d1117; border: 1px solid #30363d; border-radius: 4px; padding: 6px 8px; color: #c9d1d9; font-size: 13px; }
+        .query-item .remove { background: none; border: none; color: #f85149; cursor: pointer; font-size: 18px; padding: 0 4px; }
+        .add-btn { background: #21262d; border: 1px solid #30363d; border-radius: 4px; color: #8b949e; padding: 8px 16px; cursor: pointer; font-size: 13px; margin-top: 8px; }
+        .add-btn:hover { background: #30363d; color: #c9d1d9; }
+        .section-header { display: flex; justify-content: space-between; align-items: center; }
+        .count { font-size: 13px; color: #8b949e; }
+        .actions { display: flex; gap: 12px; margin-top: 24px; }
+        .btn { padding: 8px 16px; border-radius: 6px; border: none; cursor: pointer; font-size: 14px; font-weight: 600; }
+        .btn-primary { background: #238636; color: #fff; }
+        .btn-primary:hover { background: #2ea043; }
+        .btn-secondary { background: #21262d; color: #c9d1d9; border: 1px solid #30363d; }
+    </style>
+</head>
+<body>
+    <h1>Trigger Eval Query Review</h1>
+    <p style="color:#8b949e;margin-bottom:16px">Review and edit the trigger evaluation queries. These test whether the skill description correctly triggers (or doesn't trigger) for various user prompts.</p>
+
+    <div class="description">
+        <label>Skill Description (editable)</label>
+        <textarea id="description" placeholder="Paste skill description here..."></textarea>
+    </div>
+
+    <div class="section-header">
+        <h2>Should Trigger (positive examples)</h2>
+        <span class="count" id="yes-count">0 queries</span>
+    </div>
+    <ul class="query-list" id="should-trigger"></ul>
+    <button class="add-btn" onclick="addQuery('should-trigger')">+ Add query</button>
+
+    <div class="section-header">
+        <h2>Should NOT Trigger (negative examples)</h2>
+        <span class="count" id="no-count">0 queries</span>
+    </div>
+    <ul class="query-list" id="should-not-trigger"></ul>
+    <button class="add-btn" onclick="addQuery('should-not-trigger')">+ Add query</button>
+
+    <div class="actions">
+        <button class="btn btn-primary" onclick="exportQueries()">Export queries.json</button>
+        <button class="btn btn-secondary" onclick="importQueries()">Import queries.json</button>
+        <input type="file" id="import-file" accept=".json" style="display:none" onchange="handleImport(event)">
+    </div>
+
+    <script>
+    function addQuery(listId, value = '') {
+        const list = document.getElementById(listId);
+        const li = document.createElement('li');
+        li.className = 'query-item';
+        li.innerHTML = `
+            <input type="text" value="${escapeAttr(value)}" placeholder="Enter a user prompt...">
+            <button class="remove" onclick="this.parentElement.remove(); updateCounts();">&times;</button>
+        `;
+        list.appendChild(li);
+        updateCounts();
+        if (!value) li.querySelector('input').focus();
+    }
+
+    function escapeAttr(s) {
+        return s.replace(/"/g, '&quot;').replace(/</g, '&lt;');
+    }
+
+    function getQueries(listId) {
+        return Array.from(document.querySelectorAll(`#${listId} input[type="text"]`))
+            .map(input => input.value.trim())
+            .filter(v => v.length > 0);
+    }
+
+    function updateCounts() {
+        document.getElementById('yes-count').textContent = getQueries('should-trigger').length + ' queries';
+        document.getElementById('no-count').textContent = getQueries('should-not-trigger').length + ' queries';
+    }
+
+    function exportQueries() {
+        const data = {
+            description: document.getElementById('description').value,
+            should_trigger: getQueries('should-trigger'),
+            should_not_trigger: getQueries('should-not-trigger')
+        };
+        const blob = new Blob([JSON.stringify(data, null, 2)], { type: 'application/json' });
+        const a = document.createElement('a');
+        a.href = URL.createObjectURL(blob);
+        a.download = 'queries.json';
+        a.click();
+    }
+
+    function importQueries() {
+        document.getElementById('import-file').click();
+    }
+
+    function handleImport(event) {
+        const file = event.target.files[0];
+        if (!file) return;
+        const reader = new FileReader();
+        reader.onload = function(e) {
+            try {
+                const data = JSON.parse(e.target.result);
+                if (data.description) document.getElementById('description').value = data.description;
+                document.getElementById('should-trigger').innerHTML = '';
+                document.getElementById('should-not-trigger').innerHTML = '';
+                (data.should_trigger || []).forEach(q => addQuery('should-trigger', q));
+                (data.should_not_trigger || []).forEach(q => addQuery('should-not-trigger', q));
+            } catch (err) {
+                alert('Invalid JSON file: ' + err.message);
+            }
+        };
+        reader.readAsText(file);
+    }
+
+    // Initialize with empty entries
+    for (let i = 0; i < 3; i++) {
+        addQuery('should-trigger');
+        addQuery('should-not-trigger');
+    }
+    </script>
+</body>
+</html>

package/skills/myai-skill-builder/references/schemas.md

@@ -0,0 +1,211 @@
+# Eval Framework Schemas
+
+JSON schemas for the skill-builder eval framework. These define the data structures used throughout the eval → grade → benchmark pipeline.
+
+## evals.json
+
+Defines the test cases for a skill. Lives at `{skill-dir}/evals/evals.json`.
+
+```json
+{
+  "evals": [
+    {
+      "id": "string — short descriptive identifier, e.g. 'basic-usage'",
+      "prompt": "string — the exact user message to test with",
+      "files": {
+        "optional — map of filename → content for files the eval needs present",
+        "src/input.txt": "Example file content"
+      },
+      "assertions": [
+        "string — verifiable expectation about the output",
+        "Each assertion should be specific enough to grade as PASS/FAIL",
+        "Good: 'Creates a file matching *.test.ts'",
+        "Bad: 'Output is good quality'"
+      ]
+    }
+  ]
+}
+```
+
+### Assertion Writing Guidelines
+
+Good assertions are:
+- **Specific**: "Creates a file ending in .tsx" not "Creates a component"
+- **Verifiable**: Can be checked by reading outputs, not by subjective judgment
+- **Independent**: Each assertion tests one thing
+- **Non-trivial**: Tests behavior unique to the skill, not generic capabilities
+
+Avoid:
+- Subjective assertions ("well-structured", "clean code", "good quality")
+- Compound assertions ("Creates a file AND it has proper imports AND it compiles")
+- Tautological assertions ("Produces output" — of course it does)
+
+## timing.json
+
+Captures resource usage for a single eval run. Created by the test runner and saved alongside outputs.
+
+```json
+{
+  "eval_id": "basic-usage",
+  "config": "with_skill | without_skill",
+  "total_tokens": 4523,
+  "duration_ms": 12450,
+  "timestamp": "2025-01-15T10:30:00Z"
+}
+```
+
+Fields:
+- `total_tokens`: Combined input + output tokens for the run
+- `duration_ms`: Wall clock time from prompt submission to final output
+- `timestamp`: ISO 8601 timestamp of when the run started
+
+## grading.json
+
+Output from the grader agent. One per eval per configuration.
+
+```json
+{
+  "eval_id": "basic-usage",
+  "config": "with_skill | without_skill",
+  "expectations": [
+    {
+      "assertion": "The original assertion text",
+      "verdict": "PASS | FAIL",
+      "evidence": "Specific quote or file reference supporting the verdict",
+      "confidence": "HIGH | MEDIUM | LOW"
+    }
+  ],
+  "pass_count": 3,
+  "fail_count": 1,
+  "pass_rate": 0.75,
+  "claims": [
+    {
+      "claim": "An implicit claim extracted from the output",
+      "verified": true,
+      "evidence": "How the claim was verified"
+    }
+  ],
+  "eval_feedback": {
+    "weak_assertions": [
+      "Assertions that are too vague to meaningfully grade"
+    ],
+    "missing_assertions": [
+      "Important behaviors not tested"
+    ],
+    "suggested_additions": [
+      "New assertions that would improve coverage"
+    ]
+  },
+  "summary": "Brief human-readable summary of grading results"
+}
+```
+
+## benchmark.json
+
+Aggregated results across all evals for an iteration or multi-run benchmark.
+
+```json
+{
+  "iteration": 1,
+  "timestamp": "2025-01-15T10:30:00Z",
+  "configs": {
+    "with_skill": {
+      "overall_pass_rate": 0.85,
+      "total_tokens_mean": 4500,
+      "total_tokens_stddev": 300,
+      "duration_ms_mean": 12000,
+      "duration_ms_stddev": 1500,
+      "evals": [
+        {
+          "eval_id": "basic-usage",
+          "pass_rate": 1.0,
+          "pass_count": 3,
+          "fail_count": 0,
+          "tokens": 4200,
+          "duration_ms": 11000
+        }
+      ]
+    },
+    "without_skill": {
+      "overall_pass_rate": 0.60,
+      "total_tokens_mean": 3200,
+      "total_tokens_stddev": 250,
+      "duration_ms_mean": 8000,
+      "duration_ms_stddev": 1000,
+      "evals": [
+        {
+          "eval_id": "basic-usage",
+          "pass_rate": 0.67,
+          "pass_count": 2,
+          "fail_count": 1,
+          "tokens": 3100,
+          "duration_ms": 7500
+        }
+      ]
+    }
+  },
+  "comparison": {
+    "pass_rate_delta": 0.25,
+    "token_overhead_percent": 40.6,
+    "time_overhead_percent": 50.0,
+    "non_discriminating_assertions": [
+      "Assertions that pass in both configs"
+    ]
+  }
+}
+```
+
+## feedback.json
+
+User feedback collected from the HTML review viewer. Saved to the workspace root.
+
+```json
+{
+  "timestamp": "2025-01-15T11:00:00Z",
+  "iteration": 1,
+  "eval_feedback": [
+    {
+      "eval_id": "basic-usage",
+      "rating": "good | needs-work | bad",
+      "comment": "Free-form user feedback on this eval's results"
+    }
+  ],
+  "general_feedback": "Overall notes on the skill's performance",
+  "action": "iterate | publish | stop"
+}
+```
+
+## Directory Layout Reference
+
+```
+{skill-dir}/
+├── SKILL.md
+├── evals/
+│   └── evals.json
+├── agents/                    # Only if skill uses subagents
+│   └── *.md
+├── references/                # Supporting documentation
+│   └── *.md
+├── scripts/                   # Bundled deterministic scripts
+│   └── *.py / *.js
+└── assets/                    # Templates, HTML, configs
+    └── *
+
+{slug}-workspace/              # Created during testing, not committed
+├── iteration-1/
+│   ├── eval-{id}/
+│   │   ├── with_skill/
+│   │   │   ├── outputs/
+│   │   │   ├── timing.json
+│   │   │   └── grading.json
+│   │   └── without_skill/
+│   │       ├── outputs/
+│   │       ├── timing.json
+│   │       └── grading.json
+│   ├── eval_metadata.json
+│   └── benchmark.json
+├── iteration-2/
+│   └── ...
+├── benchmark.json             # Multi-run statistical results
+└── feedback.json              # User feedback
+```