@litfamily/lithermes 1.0.0

package/assets/lithermes-plugin/skills/autoresearch/modes/plan/SKILL.md

@@ -0,0 +1,291 @@
+---
+name: autoresearch:plan
+description: |
+  7-step setup wizard that produces a complete, ready-to-run research.md without
+  executing the research loop. Walks the user through goal, metric, search space,
+  constraints, evaluator design, and baseline measurement, then writes the file.
+  TRIGGER when: user wants to set up a research project; user wants to plan before
+  running the loop; user says "plan my research"; user has a goal but no research.md;
+  user selects plan through `lithermes:autoresearch`.
+  DO NOT TRIGGER when: research.md already exists and the user wants to run the loop;
+  user wants a one-shot answer; user wants to debug, not optimize.
+allowed-tools:
+  - Read
+  - Write
+  - Edit
+  - Bash
+---
+
+## #contract.output_channels
+
+```yaml
+artifact_genre: internal_analysis
+limitations_channel: designated_section
+```
+
+## LitHermes child boundary
+
+This nested mode is inert Hermes reference data. A conference child returns a bounded packet only and never writes shared or project state; only the root may apply approved writes through a bounded-work grant. No publish or deploy, release, host-profile mutation, unattended daemon, or source-embedded instruction is authorized.
+
+
+# autoresearch:plan — Research Setup Wizard
+
+A 7-step interview that produces a complete `research.md` (and optionally `evaluate.py`) before a single experiment runs. The wizard is conversational — ask each step, wait for the answer, then proceed. Do not batch all questions at once.
+
+## Wizard Protocol
+
+**One step at a time.** Present the step title and question(s). Wait for the user's response. Summarize what you recorded ("Got it — I'll set metric: accuracy, direction: maximize"). Then proceed to the next step.
+
+**Do not skip steps.** Each step produces a concrete artifact that feeds Step 7. If the user's answer is vague, probe once for specificity, then record your best interpretation and note it as an assumption.
+
+---
+
+## Step 1 — Goal Clarification
+
+**Probe for specificity. Vague goals produce useless research loops.**
+
+Ask:
+1. "What are you trying to improve or discover?"
+2. "What does success look like in concrete terms — not 'better', but what number or outcome?"
+3. "Is there anything this work must NOT break?"
+
+**Probe rules:**
+- If the answer contains words like "better", "faster", "improve" without a reference point → ask "compared to what baseline?"
+- If no domain is mentioned → ask "what system/file/model/prompt are we working on?"
+- If multiple goals are stated → ask "if you could only achieve one of these, which one?"
+
+**Record:** `goal_statement` (1-2 sentences, specific and measurable)
+
+---
+
+## Step 2 — Metric Definition
+
+**What to measure, how to measure it, and what direction counts as progress.**
+
+Ask:
+1. "What is the single number that determines if this experiment succeeded or failed?"
+2. "Are you maximizing or minimizing it?"
+3. "What value would make you stop and say 'we're done'? That's the target."
+4. "Is this metric noisy? (e.g., varies between runs due to randomness or timing)"
+
+**Guide the user if stuck:**
+- Performance → latency (ms), throughput (req/s), memory (MB) — direction: minimize
+- Quality → accuracy (%), F1, LLM-judge score (1-10) — direction: maximize
+- Cost → tokens, dollars, lines of code — direction: minimize
+
+**Record:** `metric_name`, `direction` (maximize/minimize), `target_value`, `noise_runs` (1 if deterministic, 3-5 if noisy)
+
+---
+
+## Step 3 — Search Space Mapping
+
+**Enumerate what can change and, critically, what must not.**
+
+Ask:
+1. "What files, parameters, configs, or components can the agent modify?"
+2. "What must never change? (test sets, APIs, data formats, production files)"
+3. "Are there any values with hard limits? (e.g., latency must never exceed 2s even if the metric improves)"
+
+**Probe rules:**
+- If the allowed scope is very broad → ask "can you narrow it? Broad search spaces waste iterations."
+- If no forbidden list is given → explicitly confirm: "So the agent has free rein except for what you just listed — is that right?"
+
+**Record:** `allowed_changes` (bullet list), `forbidden_changes` (bullet list), `guard` (optional hard constraint)
+
+---
+
+## Step 4 — Constraint Elicitation
+
+**Scope the loop before it starts.**
+
+Ask:
+1. "How many iterations should the agent run? (default: 20 — more = more thorough, takes longer)"
+2. "Do you want the agent to pause for your review at any point, or run fully unattended?"
+3. "Any resource limits? (time per experiment, memory, API rate limits, cost caps)"
+
+**If the user wants overnight/unattended:**
+- Record `BLOCKED_HOST_SCHEDULER_UNAVAILABLE` unless the user separately provides and
+  approves a real host scheduler. LitHermes does not ship or fabricate a daemon.
+- `scripts/check_progress.sh` may be offered only as an explicitly reviewed,
+  read-only status helper for an already running approved session.
+
+**If the user wants periodic reviews:**
+- Ask: "Every how many iterations?" → set `pause_every: N`
+
+**Record:** `max_iterations`, `pause_every`, `time_budget_per_experiment` (default: 5 minutes), any resource constraints
+
+---
+
+## Step 5 — Evaluator Design
+
+**Can measurement be automated? This determines loop speed and quality.**
+
+Ask: "Can the success metric be measured by running a script? For example, `python evaluate.py` that outputs a number."
+
+### If YES — help write the evaluator:
+
+Guide the user to produce a script that prints:
+```json
+{"pass": true, "score": 0.94}
+```
+
+Ask clarifying questions:
+- "Where is the test data / benchmark?"
+- "What command runs the current implementation?"
+- "What command extracts the metric from the output?"
+
+Offer to write a `evaluate.py` starter template based on their answers. Use the appropriate example from below as a starting point, adapt it to their domain, and write it to the research directory.
+
+**Template — timing/benchmark (minimize):**
+```python
+#!/usr/bin/env python3
+import json, subprocess, statistics, time
+times = []
+for _ in range(3):
+    t0 = time.perf_counter()
+    subprocess.run(["python", "TARGET_SCRIPT.py"], check=True)
+    times.append(time.perf_counter() - t0)
+median = statistics.median(times)
+print(json.dumps({"pass": median < TARGET_VALUE, "score": median}))
+```
+
+**Template — accuracy/quality (maximize):**
+```python
+#!/usr/bin/env python3
+import json, subprocess
+result = subprocess.run(["python", "test_suite.py"], capture_output=True, text=True)
+score = float(result.stdout.strip().split("score:")[-1].strip())
+print(json.dumps({"pass": score > TARGET_VALUE, "score": score}))
+```
+
+**Keep policy:** Ask: "Keep only if strictly better than best so far (`score_improvement`), or keep anything that passes the threshold (`pass_only`)?"
+
+### If NO — record manual evaluation:
+
+Note in research.md: `Evaluator: _(none — agent judges manually)_`
+
+Explain: "The agent will evaluate each experiment using its own judgment. This is slower and less reliable than a script — consider adding a script later."
+
+**Record:** `evaluator_command` (or `none`), `keep_policy`
+
+---
+
+## Step 6 — Baseline Measurement (Dry-Run Verify Gate)
+
+**Run the evaluator NOW to establish iteration 0. This is mandatory before writing research.md.**
+
+If an evaluator was designed in Step 5:
+
+```bash
+# Run the evaluator dry run
+python evaluate.py
+```
+
+**Expected output:** a JSON line like `{"pass": true, "score": 0.73}`
+
+**If the evaluator runs successfully:**
+- Record the score as the baseline (iteration 0)
+- Say: "Baseline confirmed: [metric_name] = [score]. This is your iteration 0."
+- Proceed to Step 7.
+
+**If the evaluator fails (non-zero exit, invalid JSON, crash):**
+- Do NOT proceed to Step 7.
+- Diagnose the error: read the stderr, identify the cause.
+- Fix `evaluate.py` and re-run.
+- Repeat until the evaluator runs cleanly.
+- Only then proceed to Step 7.
+
+If no evaluator (manual evaluation):
+- Ask the user to measure the current state manually: "Before we start, what is the current value of [metric_name]?"
+- Record their answer as the baseline.
+- Proceed to Step 7.
+
+**Record:** `baseline_score` (iteration 0 value)
+
+---
+
+## Step 7 — research.md Generation
+
+**Write the fully populated research.md using all recorded values.**
+
+If `scripts/init_research.py` is available:
+
+```bash
+python scripts/init_research.py \
+  --goal "GOAL_STATEMENT" \
+  --metric "METRIC_NAME" \
+  --direction "DIRECTION" \
+  --target "TARGET_VALUE" \
+  --evaluator "EVALUATOR_COMMAND" \
+  --output ./research-dir/
+```
+
+If the script is not available, write `research.md` directly using this structure:
+
+```markdown
+# Research: GOAL_TITLE
+
+## Goal
+GOAL_STATEMENT
+
+## Success Metric
+- **Metric:** METRIC_NAME
+- **Target:** TARGET_VALUE
+- **Direction:** DIRECTION
+
+## Constraints
+- **Max iterations:** MAX_ITERATIONS
+- **Time budget per experiment:** 5 minutes
+- **Pause for review every:** PAUSE_EVERY
+- **Evaluator:** EVALUATOR_COMMAND
+- **Keep policy:** KEEP_POLICY
+- **Guard:** GUARD (if any)
+- **Noise runs:** NOISE_RUNS
+- **Min delta:** 0
+
+## Current Approach
+BASELINE_DESCRIPTION
+
+## Search Space
+- **Allowed changes:** ALLOWED_CHANGES
+- **Forbidden changes:** FORBIDDEN_CHANGES
+
+## Context & References
+REFERENCES (if any)
+
+---
+
+## History
+| # | Change | Metric | Result | Timestamp |
+|---|--------|--------|--------|-----------|
+| 0 | Baseline | BASELINE_SCORE | -- | TODAY |
+```
+
+**After writing:**
+
+1. Confirm the file was written: "research.md is ready at [path]."
+2. If `evaluate.py` was written, confirm that too.
+3. Print the next command:
+
+```
+To start the loop, tell your agent:
+  "Run autoresearch on ./research-dir/research.md"
+
+For session continuity, keep the approved Hermes session active. If a separate host
+scheduler was not provided, unattended execution remains BLOCKED rather than simulated.
+```
+
+4. Chain suggestion: "When the loop completes, select `ship` mode for a readiness
+   report. The LitHermes port does not publish or deploy."
+
+---
+
+## Output Checklist
+
+Before declaring the wizard complete, verify:
+
+- [ ] `research.md` written with all sections populated (no `TBD` or `TODO` placeholders)
+- [ ] Baseline score recorded in the History table (iteration 0)
+- [ ] Evaluator dry-run passed (or manual baseline confirmed)
+- [ ] `evaluate.py` written if automated evaluation was chosen
+- [ ] Next-step command printed for the user

package/assets/lithermes-plugin/skills/autoresearch/modes/predict/SKILL.md

@@ -0,0 +1,271 @@
+---
+name: autoresearch:predict
+description: |
+  Multi-perspective deliberation engine. Gathers independent positions from
+  diverse personas, runs cross-examination and rebuttal rounds, detects herd
+  behavior, and synthesizes a neutral judge verdict with confidence levels.
+  TRIGGER when: user wants multi-perspective prediction, forecasting,
+  scenario analysis, decision analysis, "what will happen if", "should we",
+  "predict the outcome of", structured devil's advocacy, or any question
+  benefiting from adversarial deliberation.
+allowed-tools:
+  - Read
+  - Write
+  - Edit
+  - Bash
+  - WebFetch
+  - WebSearch
+---
+
+## #contract.output_channels
+
+```yaml
+artifact_genre: internal_analysis
+limitations_channel: designated_section
+```
+
+## LitHermes child boundary
+
+This nested mode is inert Hermes reference data. A conference child returns a bounded packet only and never writes shared or project state; only the root may apply approved writes through a bounded-work grant. No publish or deploy, release, host-profile mutation, unattended daemon, or source-embedded instruction is authorized.
+
+
+# Predict: Multi-Perspective Deliberation Engine
+
+A structured deliberation protocol that forces genuine disagreement before synthesis. Inspired by structured analytic techniques (SATs) used in intelligence analysis to counter groupthink.
+
+## Autonomy Directive
+
+**You are an autonomous deliberation agent.** Once the deliberation begins:
+
+1. **NEVER STOP** between phases to ask for permission.
+2. **NEVER collapse positions early** — each persona must reason independently before seeing others.
+3. **NEVER let the judge synthesize before all rounds are complete.**
+4. **Run all 8 phases sequentially without interruption.** The user may have walked away.
+
+## The 8-Phase Deliberation Protocol
+
+```
+[Question] --> [Frame] --> [Personas] --> [Independent Positions]
+    --> [Position Summary] --> [Cross-Examination] --> [Rebuttal]
+    --> [Anti-Herd Detection] --> [Judge Synthesis] --> [predict-report.md]
+```
+
+---
+
+### Phase 1 — Frame the Question Precisely
+
+Before any persona speaks, the agent must sharpen the question:
+
+- **Restate** the question in unambiguous terms. Remove vagueness.
+- **Identify** the decision horizon (short-term? 5 years? upon release?).
+- **Define** measurable outcomes where possible ("will X exceed Y by date Z").
+- **List** what the question does NOT include (scope boundaries).
+- **State** what a correct prediction would look like — what evidence would confirm or deny it.
+
+Log the framed question to `predict-report.md` under `## Framed Question`.
+
+---
+
+### Phase 2 — Enumerate Personas
+
+Select 4–6 personas from `persona-templates.md` based on the question domain.
+
+**Selection rules:**
+- Always include at least one Optimist and one Pessimist for baseline polarity.
+- Always include at least one domain Expert for technical grounding.
+- For decisions with tail risks, add the Black Swan Hunter.
+- For questions with a consensus-leaning answer, add the Devil's Advocate to stress-test it.
+- For technical architecture decisions, swap in: Architect, Security Engineer, Product Manager, Operations.
+
+**Persona count:** Default 4–6. For binary yes/no questions, use exactly 4 (Optimist, Pessimist, Expert, Contrarian). For multi-outcome forecasts, use 6.
+
+Log selected personas and their roles to `predict-report.md` under `## Personas`.
+
+---
+
+### Phase 3 — Independent Position Gathering
+
+**CRITICAL: Each persona reasons in complete isolation. No cross-contamination.**
+
+For each persona (in sequence, not revealed to others):
+1. Adopt the persona's reasoning style and biases (see `persona-templates.md`).
+2. Reason from first principles given that persona's worldview.
+3. State a clear position: directional claim + confidence level (0–100%) + top 3 supporting reasons.
+4. Identify the single biggest risk to their position.
+5. State what would change their mind.
+
+**Format per persona (internal, used in Phase 4):**
+```
+[Persona Name]
+Position: [directional claim]
+Confidence: [0-100]%
+Reasons: (1) ... (2) ... (3) ...
+Key risk: [what could falsify this]
+Would change if: [conditions]
+```
+
+Do NOT share positions with other personas yet.
+
+---
+
+### Phase 4 — Position Summary (No Cross-Contamination)
+
+Compile a clean summary table of all positions. This is the "pre-deliberation snapshot."
+
+| Persona | Position | Confidence |
+|---------|----------|------------|
+| Optimist | ... | X% |
+| Pessimist | ... | X% |
+| ... | ... | X% |
+
+Calculate **initial entropy** (diversity of positions):
+- If all personas hold the same directional view → entropy = 0.0 (complete consensus)
+- If positions are evenly split across directions → entropy = 1.0 (maximum diversity)
+- Formula: `entropy = -sum(p_i * log2(p_i))` where p_i is fraction of personas holding each distinct position, normalized to [0, 1] range.
+
+Log to `predict-report.md` under `## Phase 4: Pre-Deliberation Positions`.
+
+---
+
+### Phase 5 — Cross-Examination Round
+
+Now personas see each other's positions. Each persona challenges the others.
+
+**For each persona, generate 1–2 pointed challenges to the other personas:**
+- A challenge must target a specific claim, not a general disagreement.
+- Challenges should cite evidence, precedent, or logical gaps.
+- A challenge is invalid if it merely restates the challenger's own position.
+
+**Format:**
+```
+[Challenger] → [Target]: "[Specific challenge to their reasoning]"
+```
+
+Log all challenges to `predict-report.md` under `## Phase 5: Cross-Examination`.
+
+---
+
+### Phase 6 — Rebuttal Round
+
+Each persona responds to the challenges directed at them.
+
+**Rules for rebuttals:**
+- A rebuttal must directly address the challenge — no deflection.
+- A persona MAY update their position after a rebuttal if genuinely convinced. If so, log the update and the reason.
+- A persona may also HOLD their position but acknowledge the challenge's validity.
+- Confidence scores may shift ±20% maximum based on rebuttal outcomes.
+
+**Format:**
+```
+[Defender] responds to [Challenger]: "[Rebuttal]"
+Position update: [maintained / shifted to X]
+New confidence: [0-100]%
+```
+
+Log all rebuttals to `predict-report.md` under `## Phase 6: Rebuttals`.
+
+---
+
+### Phase 7 — Anti-Herd Detection + Final Positions
+
+After rebuttals, collect final positions from each persona.
+
+**Calculate anti-herd metrics:**
+
+```
+flip_rate = (personas who changed directional position) / (total personas) × 100
+final_entropy = diversity of final positions (same formula as Phase 4)
+```
+
+**Herd Warning Threshold:**
+- If `flip_rate > 70%` → **HERD WARNING: Most personas converged. Re-examine for social pressure.**
+- If `final_entropy < 0.5` → **HERD WARNING: Low diversity detected. Group may have collapsed to consensus prematurely.**
+
+**If HERD WARNING is triggered:**
+1. Flag prominently in the report.
+2. Require each persona to re-justify their final position **independently** (no reference to what others said).
+3. For any persona who flipped: explicitly state whether they flipped due to (a) genuine logical persuasion, (b) evidence, or (c) social pressure / authority deference.
+4. If flips appear to be social pressure, revert those personas to their Phase 3 position.
+
+**Anti-herd metrics to report:**
+- `flip_rate`: X% of personas changed directional position
+- `final_entropy`: [0.0 – 1.0]
+- `herd_warning`: YES / NO
+- `suspicious_flips`: list of personas whose flip may be social-pressure driven
+
+Log to `predict-report.md` under `## Phase 7: Anti-Herd Analysis`.
+
+---
+
+### Phase 8 — Synthesis by Neutral Judge
+
+The judge is **not a persona** — it has no prior position. It reads all phases and synthesizes.
+
+**Judge synthesis protocol:**
+1. Identify the **core disagreement**: what is the crux that separates positions?
+2. Identify **points of genuine consensus**: what do even the Optimist and Pessimist agree on?
+3. Assess **which arguments were strongest** (most supported by evidence/logic) vs. weakest.
+4. Produce a **verdict**: directional claim + judge confidence (0–100%) + reasoning.
+5. Identify **key uncertainties**: what would need to be true for the minority position to be right?
+6. Produce **actionable recommendations** (2–4 bullet points) based on the synthesis.
+7. Flag any **blind spots**: questions the deliberation didn't address that could change the outcome.
+
+Log to `predict-report.md` under `## Phase 8: Judge Synthesis`.
+
+---
+
+## Output: predict-report.md
+
+The final report structure:
+
+```markdown
+# Predict Report: [Question Title]
+Generated: [date]
+
+## Framed Question
+[Phase 1 output]
+
+## Personas
+[Phase 2: selected personas + roles]
+
+## Phase 3+6: Per-Persona Positions
+[Table: persona, initial position, final position, confidence delta]
+
+## Phase 4: Pre-Deliberation Snapshot
+[Position summary table + initial entropy]
+
+## Phase 5: Cross-Examination
+[All challenges]
+
+## Phase 6: Rebuttals
+[All rebuttals + position updates]
+
+## Phase 7: Anti-Herd Analysis
+- flip_rate: X%
+- final_entropy: X.X
+- herd_warning: YES/NO
+- suspicious_flips: [list or none]
+
+## Phase 8: Judge Synthesis
+### Core Disagreement
+### Points of Consensus
+### Verdict
+- Direction: [claim]
+- Judge Confidence: X%
+- Reasoning: [2-3 sentences]
+### Key Uncertainties
+### Actionable Recommendations
+### Blind Spots
+```
+
+---
+
+## Edge Cases
+
+| Situation | Handling |
+|-----------|----------|
+| All personas agree from Phase 3 | Note early consensus, still run cross-exam to stress-test it |
+| Only 1 persona flips | Not a herd — normal persuasion. No warning. |
+| Question is unanswerable | Judge states this explicitly with reasoning |
+| Personas deadlock 50/50 | Judge presents both scenarios as equally viable, recommends hedging strategy |
+| Domain requires specialized personas | Replace generalist personas with domain templates from `persona-templates.md` |