hyperresearch 0.6.0__tar.gz → 0.7.0__tar.gz

{hyperresearch-0.6.0 → hyperresearch-0.7.0}/CHANGELOG.md

@@ -1,5 +1,50 @@
 # Changelog
 
+## [0.7.0] - 2026-04-17
+
+### Architecture — `/research-ensemble` retired, `/research-layercake` introduced
+
+This release replaces the three-parallel-drafts-plus-merger ensemble design with a seven-phase layered pipeline. Width is discovered first, depth loci are derived from the width corpus (not pre-assigned framings), one draft is written from the combined evidence, three adversarial critics run in parallel against it, and the draft is then modified ONLY by surgical Edit hunks — never regenerated.
+
+### New
+
+- **7-phase layercake pipeline** — (1) width sweep via parallel fetchers, (2) two parallel loci-analysts identify 1–8 depth loci from the corpus, (3) one depth-investigator per locus writes an `interim-<locus>.md` note, (4) orchestrator writes ONE draft, (5) dialectic / depth / width critics return structured findings JSONs, (6) the patcher applies findings as Edit hunks, (7) the polish auditor cuts filler and strips hygiene leaks via more Edit hunks. Protocol lives at `.claude/skills/research-layercake/SKILL.md`.
+- **Tool-locked patcher + polish auditor** — both agents register with tools `[Read, Edit]` ONLY. They physically cannot Write. Every hunk is capped at 500 chars of net expansion — any critic that proposes a larger patch escalates to the orchestrator instead of triggering a rewrite. This is the load-bearing invariant that enforces PATCH-NOT-REGEN at the tool level, not the prompt level.
+- **`NoteType.INTERIM`** — new first-class note type for depth-investigator outputs. Persisted in the vault with `type: interim` and tagged `locus-<name>` for indexability. Added to the SQLite CHECK constraint via migration v7.
+- **`locus-coverage` lint rule** — reads `research/loci.json` (Layer 2 output) and verifies every identified locus has a corresponding interim-report note. Missing interims flag as errors.
+- **`patch-surgery` lint rule** — reads `research/patch-log.json` (Layer 6 output) and surfaces any critical finding the patcher skipped. The 500-char "patch too large" regeneration guard is also surfaced at warning severity.
+- **`instruction-coverage` lint rule** — reads `research/prompt-decomposition.json` and verifies every atomic item (entity, required format) appears in the final report. Catches drafts that drifted from the user's explicit ask.
+- **Layer 0.5 — prompt decomposition** — new orchestrator step before Layer 1 produces `research/prompt-decomposition.json`, a structured breakdown of the atomic items the user's prompt named (sub-questions, entities, required formats, required sections, time horizons, scope conditions). This becomes a first-class contract that flows through Layer 4 drafting and Layer 5 instruction-critique.
+- **`hyperresearch-instruction-critic`** — fourth adversarial critic (Opus, `[Bash, Read]` only). Reads the Layer 4 draft against the prompt-decomposition and emits findings for missing / under-covered / mis-ordered / mis-formatted atomic items. Spawned in parallel with dialectic / depth / width critics in Layer 5.
+- **Pipeline-awareness contract** — every subagent now receives the verbatim research_query AND an explicit pipeline-position statement in its Task prompt. The skill file documents the three-piece spawn contract (research_query / pipeline position / inputs) and provides a copy-paste template so the orchestrator applies it consistently to every Task call.
+- **Schema v7 migration** — safely rebuilds the `notes` table with `'interim'` added to the type CHECK constraint on existing vaults.
+
+### Removed
+
+- **`/research-ensemble` skill** — the three-parallel-sub-run ensemble protocol is gone. The slash command no longer registers.
+- **Retired subagents** — `hyperresearch-analyst`, `hyperresearch-auditor`, `hyperresearch-rewriter`, `hyperresearch-subrun`, `hyperresearch-merger` are no longer installed. On reinstall, any vault that had them gets them pruned automatically by `_prune_retired_agents()`.
+- **`analyst-coverage` lint rule** — superseded by `locus-coverage` (extracts were the ensemble era's per-source deep-read artifact; interim notes are the layercake equivalent scoped per locus).
+
+### New subagent roster (9 agents)
+
+| Agent | Model | Tools | Role |
+|---|---|---|---|
+| `hyperresearch-fetcher` | Haiku | Bash, Read | URL → vault note (unchanged) |
+| `hyperresearch-loci-analyst` | Sonnet | Bash, Read, Write | Returns 1–8 depth loci from width corpus |
+| `hyperresearch-depth-investigator` | Sonnet | Bash, Read, Write, Task | Investigates one locus, writes one interim note |
+| `hyperresearch-dialectic-critic` | Opus | Bash, Read | Finds counter-evidence gaps |
+| `hyperresearch-depth-critic` | Opus | Bash, Read | Finds shallow spots |
+| `hyperresearch-width-critic` | Opus | Bash, Read | Finds topical coverage gaps |
+| `hyperresearch-instruction-critic` | Opus | Bash, Read | Finds atomic items the draft missed from prompt-decomposition |
+| `hyperresearch-patcher` | Sonnet | **Read, Edit** | Applies critic findings as Edit hunks |
+| `hyperresearch-polish-auditor` | Sonnet | **Read, Edit** | Cuts filler + strips hygiene leaks |
+
+### Breaking changes
+
+- Scripts calling `hyperresearch install` on a pre-v0.7 vault will get the old agent files pruned. Pre-existing `research/audit_findings.json` and extract notes stay in the vault (no user data is deleted) but the protocol no longer references them.
+- `analyst-coverage` in `hyperresearch lint --rule ...` is gone — use `locus-coverage` and `patch-surgery`.
+- The `benchmark-report` lint rule is renamed to `wrapper-report`. The rule's logic is unchanged — it fires whenever `research/prompt.txt` or `research/wrapper_contract.json` is present and enforces the wrapper's contract on the final report. The rename reflects what the rule actually does (wrapper-contract enforcement) rather than the specific harness context where it was first used.
+
 ## [0.4.0] - 2026-04-13
 
 ### New

{hyperresearch-0.6.0 → hyperresearch-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hyperresearch
-Version: 0.6.0
+Version: 0.7.0
 Summary: Claude Code harness for disciplined, adversarially-audited deep research with full provenance.
 Project-URL: Homepage, https://github.com/jordan-gibbs/hyperresearch
 Project-URL: Repository, https://github.com/jordan-gibbs/hyperresearch
@@ -53,13 +53,7 @@ Description-Content-Type: text/markdown
 
 ---
 
-**HyoerResearch is a deep research harness for Claude Code, currently leading the deep research bench internally. Creates fully persistent knowledge bases for continued research, with state of the art query adherence and comprehensiveness.**
-
-<p align="center">
-  <img src="assets/benchmark.png" alt="DeepResearch-Bench top-5 — hyperresearch at 57.5, ahead of Tongyi DeepResearch (55.8), Kimi Researcher (53.2), Gemini Deep Research (42.1), OpenAI Deep Research (37.4)" width="780">
-</p>
-
-<p align="center"><sub>Preliminary — one-query pilot lift (+2.08 over v1 baseline) projected against current DeepResearch-Bench leaders. Full 100-query sweep pending.</sub></p>
+**HyperResearch is a deep research harness for Claude Code. It produces adversarially-audited reports with full source provenance and creates a persistent, searchable knowledge base that compounds across sessions.**
 
 ## Installation
 
@@ -68,9 +62,9 @@ pip install hyperresearch
 hyperresearch install
 ```
 
-In Claude Code, type `/research <anything>` and the full protocol fires.
+In Claude Code, type `/research-layercake <anything>` and the full flagship protocol fires — **a 7-phase pipeline that discovers width first, derives depth loci from the width corpus, runs parallel depth investigations, drafts once, and patches the draft via adversarial critics (no regeneration).** This is the default way to use hyperresearch. If you want a faster single-pass run instead, type `/research <anything>`.
 
-That single sequence creates a v6 SQLite vault, auto-installs Chromium for headless browsing, generates `CLAUDE.md`, installs the `/research` skill (dispatcher + 4 modality files), registers three subagents (fetcher / analyst / auditor) into `.claude/agents/`, and wires PreToolUse hooks.
+That single install command creates a v7 SQLite vault, auto-installs Chromium for headless browsing, generates `CLAUDE.md`, installs both skills (`/research-layercake` and `/research`), registers eight subagents (fetcher / loci-analyst / depth-investigator / dialectic-critic / depth-critic / width-critic / patcher / polish-auditor) into `.claude/agents/`, and wires PreToolUse hooks.
 
 ---
 
@@ -131,6 +125,64 @@ The save gate blocks until every CRITICAL is **genuinely** resolved. Bookkeeping
 
 ---
 
+## `/research-layercake` — 7 phases, one draft, patched not regenerated (the default)
+
+The flagship research surface and the default way to use hyperresearch. Type `/research-layercake <query>` in Claude Code.
+
+**Width before depth, then one draft patched by adversarial critics.** An Opus orchestrator walks a seven-phase pipeline: wide corpus sweep → rabbithole identification → parallel depth investigations → single draft → three adversarial readings → surgical patch pass → polish audit.
+
+### The 7 phases
+
+1. **Width sweep.** The orchestrator spawns parallel `hyperresearch-fetcher` Haiku agents to build a 30–80 source corpus covering the topic's corners. Academic APIs (Semantic Scholar, arXiv, OpenAlex, PubMed) run BEFORE web search — citation-ranked canonical papers before derivative commentary.
+2. **Loci analysis.** Two `hyperresearch-loci-analyst` Sonnet agents read the width corpus in parallel and each returns 1–8 specific "depth loci" — questions where deeper investigation would meaningfully improve the final report. The orchestrator dedupes their outputs and clamps to 6.
+3. **Depth investigation.** One `hyperresearch-depth-investigator` Sonnet agent per locus, in parallel. Each investigator can spawn its own fetchers (capped at 10 new sources), reads the existing vault, and writes ONE `interim-<locus>.md` note with dense synthesis, direct quotes, and citations. The orchestrator consumes these interim notes during Layer 4.
+4. **Draft.** The orchestrator writes `research/notes/final_report.md` ONCE, weaving the width corpus with the interim notes under the modality file's substance rules (collect / synthesize / compare / forecast).
+5. **Adversarial critique.** Three Opus critics fire in parallel: `dialectic-critic` (counter-evidence the draft missed), `depth-critic` (shallow spots the interim notes could fill), `width-critic` (topical corners the draft ignores despite the corpus supporting them). Each returns a findings JSON with structured `suggested_patch` entries — never a rewrite.
+6. **Patch pass.** `hyperresearch-patcher` (Sonnet, **tool-locked to `[Read, Edit]`**) reads all three findings files and applies them as surgical Edit hunks. Per-hunk cap: `new_string` may be at most 500 chars longer than `old_string`. The patcher physically cannot Write — it has no path to regenerate the draft.
+7. **Polish audit.** `hyperresearch-polish-auditor` (Sonnet, also `[Read, Edit]` only) strips hygiene leaks (YAML frontmatter, scaffold sections, prompt echoes), cuts filler phrases ("Importantly", "It is worth noting"), removes redundancy, and breaks long sentences — all via surgical Edits. Structural mismatches (wrong format for the prompt, missing sections) escalate to the orchestrator rather than getting force-patched.
+
+### Why the tool lock matters
+
+The load-bearing invariant of layercake is: **after Layer 4 the draft is only ever modified by small Edit hunks.** This is enforced at the Claude Code tool-allowlist level — the patcher and polish auditor do not have `Write` or `Bash`. If a critic's finding cannot fit into a ≤500-char hunk, it escalates to the orchestrator as a structural issue instead of triggering a regeneration. This prevents the "just rewrite it" failure mode that plagues post-hoc review in long-running agent pipelines.
+
+### Opting out — faster single-pass mode
+
+Layercake is the default because width-plus-depth-plus-adversarial-review beats a single investigator on almost every real research query. If you want a faster single-pass run — e.g., for a quick sanity-check or a prompt where one pass is plenty — type `/research <query>` instead. Single-pass fires the same modality rules on one Sonnet agent, no loci analysis, no critics, no patcher.
+
+A rule of thumb: use `/research-layercake` for anything you would want a human researcher to take seriously. Use `/research` when you just want the vault populated and a first-pass draft in front of you quickly.
+
+---
+
+## The vault: persistent, searchable, compounding
+
+hyperresearch is not a one-shot report generator. Every fetched source lands in a durable SQLite-indexed vault that every future research session can reuse. Knowledge compounds across sessions; the tool's value grows with the corpus.
+
+### What persists
+
+- **Raw source material, not rewritten summaries.** Fetched content keeps its original formatting — headings, tables, code blocks, technical details. If the source is a PDF, pymupdf extracts the full text and the original PDF lives at `research/raw/<note-id>.pdf`. Every note carries its source URL, fetch timestamp, tier classification, and `--suggested-by` breadcrumb pointing at whichever note surfaced it.
+- **Extract notes** — per-source analyst reads with direct quotes, context markers, one-line summaries, and inherited tier + content-type tags. These are the second derivative of raw content and stay in the vault as first-class, queryable notes.
+- **Scaffolds, comparisons, audit findings** — every research session's planning artifacts and adversarial review trail persist alongside the sources they drew on. Past synthesis notes are discoverable next to the raw sources that justified them.
+- **The knowledge graph** — `[[wiki-links]]` between notes form a queryable graph. `hyperresearch graph hubs` surfaces the most-connected notes in a domain; `hyperresearch graph backlinks <id>` shows every note that references a given source.
+
+### Searchability
+
+Before any web search the vault gets queried first — the hook reminds Claude Code, and the protocol checks explicitly:
+
+```bash
+hyperresearch search "ion-trap gate fidelity" -j           # Full-text across every note
+hyperresearch search "quantum" --tier ground_truth -j      # Filter by epistemic tier
+hyperresearch search "attention" --include-body -j         # Full-body search, not just titles
+hyperresearch note show <id> <id> <id> -j                  # Batch-read notes in one call
+```
+
+When the vault already holds ≥10 relevant notes on a topic, `/research` tells the user so and asks whether to dig deeper, take a new angle, or synthesize from the existing corpus — instead of re-fetching what's already there. The sunk-cost of past research pays forward.
+
+### Markdown is truth, SQLite is cache
+
+Notes live as plain markdown with YAML frontmatter in `research/notes/`. The SQLite index is fully rebuildable — delete it and `hyperresearch sync` reconstructs it from the markdown. The vault is inspectable in any editor, version-controllable in git, and readable without the tool installed. hyperresearch stores and indexes; it does not hide data behind the tool.
+
+---
+
 ## Why it works on hard research
 
 Your AI agent searches the web, skims sources, writes an answer that sounds good, and ships it. There's no scaffold, no audit, no source-vs-source comparison, no provenance, no way to know if it actually engaged with the strongest counter-position. Hyperresearch breaks that pattern through structural enforcement:
@@ -183,13 +235,18 @@ Your AI agent searches the web, skims sources, writes an answer that sounds good
 └─────────────────────────────────────────────────────────────────────┘
 ```
 
-**Subagent triad**, each picked for its job:
+**Subagent roster**, each picked for its job:
 
 | Agent | Model | Role |
 |---|---|---|
 | `hyperresearch-fetcher` | **Haiku** | Mechanical URL fetching via crawl4ai. Cheap, fast, parallel. |
-| `hyperresearch-analyst` | **Sonnet** | Reads one source with research goal in hand. Writes a focused extract, proposes 2-5 next URLs for the loop. |
-| `hyperresearch-auditor` | **Opus** | Adversarial review in two modes against the verbatim prompt. Persists structured findings the save gate verifies. |
+| `hyperresearch-loci-analyst` | **Sonnet** | Reads the width corpus, returns 1–8 depth loci with rationale. Spawn 2 in parallel, dedupe. |
+| `hyperresearch-depth-investigator` | **Sonnet** | Investigates one locus: fetches new sources, writes one interim-report note per locus. |
+| `hyperresearch-dialectic-critic` | **Opus** | Finds counter-evidence the draft missed, emits structured findings with surgical patches. |
+| `hyperresearch-depth-critic` | **Opus** | Finds shallow spots where interim notes could fill specifics. |
+| `hyperresearch-width-critic` | **Opus** | Finds topical corners the corpus supports but the draft ignores. |
+| `hyperresearch-patcher` | **Sonnet** | Tool-locked `[Read, Edit]`. Applies critic findings as surgical Edit hunks. Cannot regenerate. |
+| `hyperresearch-polish-auditor` | **Sonnet** | Tool-locked `[Read, Edit]`. Cuts filler, strips hygiene leaks, breaks long sentences. |
 
 **Routing** — the dispatcher classifies every request by **what the output needs to do**, not what the subject is:
 
@@ -206,16 +263,17 @@ A query about a fictional franchise can be `collect` (per-character enumeration)
 
 ## What's enforced
 
-Eight invariants the protocol structurally prevents from breaking:
+Nine invariants the protocol structurally prevents from breaking:
 
-1. **Verbatim prompt as gospel** — `scaffold-prompt` lint blocks at Checkpoint 3 if the scaffold doesn't open with the user's exact prompt
+1. **Verbatim prompt as gospel** — `scaffold-prompt` lint blocks if the scaffold doesn't open with the user's exact prompt
 2. **Rooted-tree provenance** — `--suggested-by` chain must form a real tree from at least one seed; isolated breadcrumbs flagged
-3. **Analyst coverage** — at least 1 extract per 3 sources, no silent skipping
-4. **Adversarial dissent** — at least one source explicitly contradicts the dominant view, named in writing
-5. **Audit-gate self-cert detection** — CRITICAL findings marked fixed get their underlying lint rules re-run
-6. **Save blocked** until every CRITICAL is genuinely resolved
-7. **Schema integrity** — `tier` and `content_type` are SQLite CHECK-constrained vocabularies; corrupted frontmatter cannot poison the index
-8. **PDF + raw artifact persistence** — fetched PDFs land in `research/raw/` and the `raw_file` frontmatter field survives every re-serialization
+3. **Locus coverage** — every Layer 2 locus must have a Layer 3 `interim-<name>.md` note; missing interims flag as errors
+4. **Patch-only draft modification** — after Layer 4, the patcher and polish auditor are tool-locked to `[Read, Edit]`. They cannot regenerate; the ≤500-char hunk-expansion cap is a mechanical tripwire
+5. **Critical findings never silently skip** — `patch-surgery` lint surfaces any critical finding the patcher couldn't apply
+6. **Schema integrity** — `tier`, `content_type`, and `type` are SQLite CHECK-constrained vocabularies; corrupted frontmatter cannot poison the index
+7. **PDF + raw artifact persistence** — fetched PDFs land in `research/raw/` and the `raw_file` frontmatter field survives every re-serialization
+8. **Interim notes are persisted, not ephemeral** — depth-investigator outputs stay in the vault forever; future sessions can query them
+9. **Hygiene leaks caught on the way out** — scaffold sections, YAML frontmatter, and prompt echoes are stripped by the polish auditor before ship
 
 ---
 
@@ -236,11 +294,8 @@ Eight invariants the protocol structurally prevents from breaking:
 - **Forecasting + strategy** — "Will US inflation stay above 3% through 2026?" → ground-truth statistics + institutional analysis + named contrarians, probability language not hedge
 - **Interpretive analysis** — "What does *Blood Meridian*'s violence mean?" → primary text + critical tradition + dissenting reading, every paragraph fuses fact with interpretive claim
 - **Enumerative coverage** — "For each Napoleonic marshal, cover key campaigns and fate" → every named entity gets every requested field, no silent downgrades
-- **Persistent knowledge base** — every source ever fetched stays searchable. Future sessions check the vault before searching the web. Knowledge compounds.
 
-### `/research-ensemble` — 3 parallel sub-runs + Opus merger (what was benchmarked)
-
-For queries where depth-of-corpus and argument stability matter more than wall-clock time, use `/research-ensemble <query>`. Opus orchestrates; three `hyperresearch-subrun` Sonnet agents run the full protocol in parallel against one shared vault, each with a subtly different framing (evidentiary breadth / citation-chain depth / dialectical tension). A fourth agent (`hyperresearch-merger`, Opus) scores each draft on comprehensiveness / readability / argument strength / citation quality, picks the strongest as the base, and splices in unique evidence from the other two. Cost is ~5× a normal run; the payoff is detailed in [Benchmarks](#benchmarks) below.
+All of the above run under the default [`/research-layercake`](#research-layercake--7-phases-one-draft-patched-not-regenerated-the-default) mode: width sweep → loci analysis → depth investigations → one draft → three adversarial critics → surgical patch pass → polish audit. For faster turnaround when the query doesn't warrant the full pipeline, type `/research <query>` for a single-pass run.
 
 ---
 
@@ -250,8 +305,8 @@ One command sets up the full integration:
 
 - **`.claude/settings.json`** — PreToolUse hook that nudges Claude Code to check the vault before any raw web search
 - **`.claude/skills/hyperresearch/`** — `/research` skill (dispatcher + 4 modality files: collect / synthesize / compare / forecast)
-- **`.claude/skills/research-ensemble/`** — `/research-ensemble` skill (3× parallel sub-runs + Opus merger; see [Benchmarks](#benchmarks))
-- **`.claude/agents/`** — six registered subagents: `hyperresearch-fetcher` (Haiku), `hyperresearch-analyst` (Sonnet), `hyperresearch-auditor` (Opus), `hyperresearch-rewriter` (Sonnet), `hyperresearch-subrun` (Sonnet, ensemble), `hyperresearch-merger` (Opus, ensemble)
+- **`.claude/skills/research-layercake/`** — `/research-layercake` skill (the 7-phase pipeline)
+- **`.claude/agents/`** — eight registered subagents: `hyperresearch-fetcher` (Haiku), `hyperresearch-loci-analyst` (Sonnet), `hyperresearch-depth-investigator` (Sonnet), `hyperresearch-dialectic-critic` (Opus), `hyperresearch-depth-critic` (Opus), `hyperresearch-width-critic` (Opus), `hyperresearch-patcher` (Sonnet, `[Read, Edit]`-locked), `hyperresearch-polish-auditor` (Sonnet, `[Read, Edit]`-locked)
 - **`CLAUDE.md`** at the vault root — the full research workflow, automatically loaded by Claude Code on every session
 
 hyperresearch is Claude Code-only for now. Codex, Cursor, and Gemini support was trimmed from v0.6 to focus the surface area — may return as real integrations later.
@@ -311,26 +366,6 @@ LinkedIn, Twitter, Facebook, Instagram, and TikTok automatically use a visible b
 
 ---
 
-## Benchmarks
-
-Hyperresearch is being evaluated on the [DeepResearch Bench](https://github.com/Ayanami0730/deep_research_bench) RACE eval (Gemini-2.5-Pro judges reports on comprehensiveness, insight, instruction-following, readability). Full per-modality scores and head-to-head comparisons vs other research harnesses will land here as the 100-query sweep completes. **Status:** in progress.
-
-### Ensemble lift — preliminary data point
-
-Query 91 from DeepResearch-Bench ("detailed analysis of the Saint Seiya franchise, structured around armor classes, per-character power/techniques/arcs/fate") is a deep-collect query with ~100 named entities. One data point so far:
-
-| Variant | Overall | Comp. | Insight | Inst. | Read. | Cost | Time |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| `/research` single-run (sonnet, v1 baseline) | **52.74** | 52.95 | 58.74 | 51.28 | 51.90 | ~$1.50 | ~8 min |
-| `/research-ensemble` (3× sonnet + opus merger) | **54.82** | 54.52 | 58.57 | 53.30 | 56.10 | $21.98 | 57 min |
-| **Delta** | **+2.08** | +1.57 | −0.17 | +2.02 | +4.20 | ~14× | ~7× |
-
-Readability is the biggest single lift (+4.20) — the merger smooths splice boundaries and harmonizes voice across the three sub-runs. Instruction-following (+2.02) gains from three independent passes cross-checking the prompt's entity list; prompt-named entities the base run collapsed get restored from sibling runs. Comprehensiveness (+1.57) comes from the 3-way evidence union (167 sources fetched into the shared corpus vs ~30 for a single run). Insight is flat as expected — the merger is a compiler, not a re-thinker; it can't deepen an argument beyond what the strongest sub-run already had.
-
-One query is not a benchmark. Full 100-query sweep coming next. The cost ceiling is honest: ~5× per query at the tool-call level, ~14× here because `/research` ran on Sonnet and `/research-ensemble` requires Opus orchestration + Opus merger. Use ensemble where the query earns it; use `/research` for everything else.
-
----
-
 ## Philosophy
 
 - **Process is load-bearing.** A draft without a scaffold, comparisons, audit findings, and a clean provenance graph is unfinished — regardless of how good the prose reads.

{hyperresearch-0.6.0 → hyperresearch-0.7.0}/README.md

@@ -13,13 +13,7 @@
 
 ---
 
-**HyoerResearch is a deep research harness for Claude Code, currently leading the deep research bench internally. Creates fully persistent knowledge bases for continued research, with state of the art query adherence and comprehensiveness.**
-
-<p align="center">
-  <img src="assets/benchmark.png" alt="DeepResearch-Bench top-5 — hyperresearch at 57.5, ahead of Tongyi DeepResearch (55.8), Kimi Researcher (53.2), Gemini Deep Research (42.1), OpenAI Deep Research (37.4)" width="780">
-</p>
-
-<p align="center"><sub>Preliminary — one-query pilot lift (+2.08 over v1 baseline) projected against current DeepResearch-Bench leaders. Full 100-query sweep pending.</sub></p>
+**HyperResearch is a deep research harness for Claude Code. It produces adversarially-audited reports with full source provenance and creates a persistent, searchable knowledge base that compounds across sessions.**
 
 ## Installation
 
@@ -28,9 +22,9 @@ pip install hyperresearch
 hyperresearch install
 ```
 
-In Claude Code, type `/research <anything>` and the full protocol fires.
+In Claude Code, type `/research-layercake <anything>` and the full flagship protocol fires — **a 7-phase pipeline that discovers width first, derives depth loci from the width corpus, runs parallel depth investigations, drafts once, and patches the draft via adversarial critics (no regeneration).** This is the default way to use hyperresearch. If you want a faster single-pass run instead, type `/research <anything>`.
 
-That single sequence creates a v6 SQLite vault, auto-installs Chromium for headless browsing, generates `CLAUDE.md`, installs the `/research` skill (dispatcher + 4 modality files), registers three subagents (fetcher / analyst / auditor) into `.claude/agents/`, and wires PreToolUse hooks.
+That single install command creates a v7 SQLite vault, auto-installs Chromium for headless browsing, generates `CLAUDE.md`, installs both skills (`/research-layercake` and `/research`), registers eight subagents (fetcher / loci-analyst / depth-investigator / dialectic-critic / depth-critic / width-critic / patcher / polish-auditor) into `.claude/agents/`, and wires PreToolUse hooks.
 
 ---
 
@@ -91,6 +85,64 @@ The save gate blocks until every CRITICAL is **genuinely** resolved. Bookkeeping
 
 ---
 
+## `/research-layercake` — 7 phases, one draft, patched not regenerated (the default)
+
+The flagship research surface and the default way to use hyperresearch. Type `/research-layercake <query>` in Claude Code.
+
+**Width before depth, then one draft patched by adversarial critics.** An Opus orchestrator walks a seven-phase pipeline: wide corpus sweep → rabbithole identification → parallel depth investigations → single draft → three adversarial readings → surgical patch pass → polish audit.
+
+### The 7 phases
+
+1. **Width sweep.** The orchestrator spawns parallel `hyperresearch-fetcher` Haiku agents to build a 30–80 source corpus covering the topic's corners. Academic APIs (Semantic Scholar, arXiv, OpenAlex, PubMed) run BEFORE web search — citation-ranked canonical papers before derivative commentary.
+2. **Loci analysis.** Two `hyperresearch-loci-analyst` Sonnet agents read the width corpus in parallel and each returns 1–8 specific "depth loci" — questions where deeper investigation would meaningfully improve the final report. The orchestrator dedupes their outputs and clamps to 6.
+3. **Depth investigation.** One `hyperresearch-depth-investigator` Sonnet agent per locus, in parallel. Each investigator can spawn its own fetchers (capped at 10 new sources), reads the existing vault, and writes ONE `interim-<locus>.md` note with dense synthesis, direct quotes, and citations. The orchestrator consumes these interim notes during Layer 4.
+4. **Draft.** The orchestrator writes `research/notes/final_report.md` ONCE, weaving the width corpus with the interim notes under the modality file's substance rules (collect / synthesize / compare / forecast).
+5. **Adversarial critique.** Three Opus critics fire in parallel: `dialectic-critic` (counter-evidence the draft missed), `depth-critic` (shallow spots the interim notes could fill), `width-critic` (topical corners the draft ignores despite the corpus supporting them). Each returns a findings JSON with structured `suggested_patch` entries — never a rewrite.
+6. **Patch pass.** `hyperresearch-patcher` (Sonnet, **tool-locked to `[Read, Edit]`**) reads all three findings files and applies them as surgical Edit hunks. Per-hunk cap: `new_string` may be at most 500 chars longer than `old_string`. The patcher physically cannot Write — it has no path to regenerate the draft.
+7. **Polish audit.** `hyperresearch-polish-auditor` (Sonnet, also `[Read, Edit]` only) strips hygiene leaks (YAML frontmatter, scaffold sections, prompt echoes), cuts filler phrases ("Importantly", "It is worth noting"), removes redundancy, and breaks long sentences — all via surgical Edits. Structural mismatches (wrong format for the prompt, missing sections) escalate to the orchestrator rather than getting force-patched.
+
+### Why the tool lock matters
+
+The load-bearing invariant of layercake is: **after Layer 4 the draft is only ever modified by small Edit hunks.** This is enforced at the Claude Code tool-allowlist level — the patcher and polish auditor do not have `Write` or `Bash`. If a critic's finding cannot fit into a ≤500-char hunk, it escalates to the orchestrator as a structural issue instead of triggering a regeneration. This prevents the "just rewrite it" failure mode that plagues post-hoc review in long-running agent pipelines.
+
+### Opting out — faster single-pass mode
+
+Layercake is the default because width-plus-depth-plus-adversarial-review beats a single investigator on almost every real research query. If you want a faster single-pass run — e.g., for a quick sanity-check or a prompt where one pass is plenty — type `/research <query>` instead. Single-pass fires the same modality rules on one Sonnet agent, no loci analysis, no critics, no patcher.
+
+A rule of thumb: use `/research-layercake` for anything you would want a human researcher to take seriously. Use `/research` when you just want the vault populated and a first-pass draft in front of you quickly.
+
+---
+
+## The vault: persistent, searchable, compounding
+
+hyperresearch is not a one-shot report generator. Every fetched source lands in a durable SQLite-indexed vault that every future research session can reuse. Knowledge compounds across sessions; the tool's value grows with the corpus.
+
+### What persists
+
+- **Raw source material, not rewritten summaries.** Fetched content keeps its original formatting — headings, tables, code blocks, technical details. If the source is a PDF, pymupdf extracts the full text and the original PDF lives at `research/raw/<note-id>.pdf`. Every note carries its source URL, fetch timestamp, tier classification, and `--suggested-by` breadcrumb pointing at whichever note surfaced it.
+- **Extract notes** — per-source analyst reads with direct quotes, context markers, one-line summaries, and inherited tier + content-type tags. These are the second derivative of raw content and stay in the vault as first-class, queryable notes.
+- **Scaffolds, comparisons, audit findings** — every research session's planning artifacts and adversarial review trail persist alongside the sources they drew on. Past synthesis notes are discoverable next to the raw sources that justified them.
+- **The knowledge graph** — `[[wiki-links]]` between notes form a queryable graph. `hyperresearch graph hubs` surfaces the most-connected notes in a domain; `hyperresearch graph backlinks <id>` shows every note that references a given source.
+
+### Searchability
+
+Before any web search the vault gets queried first — the hook reminds Claude Code, and the protocol checks explicitly:
+
+```bash
+hyperresearch search "ion-trap gate fidelity" -j           # Full-text across every note
+hyperresearch search "quantum" --tier ground_truth -j      # Filter by epistemic tier
+hyperresearch search "attention" --include-body -j         # Full-body search, not just titles
+hyperresearch note show <id> <id> <id> -j                  # Batch-read notes in one call
+```
+
+When the vault already holds ≥10 relevant notes on a topic, `/research` tells the user so and asks whether to dig deeper, take a new angle, or synthesize from the existing corpus — instead of re-fetching what's already there. The sunk-cost of past research pays forward.
+
+### Markdown is truth, SQLite is cache
+
+Notes live as plain markdown with YAML frontmatter in `research/notes/`. The SQLite index is fully rebuildable — delete it and `hyperresearch sync` reconstructs it from the markdown. The vault is inspectable in any editor, version-controllable in git, and readable without the tool installed. hyperresearch stores and indexes; it does not hide data behind the tool.
+
+---
+
 ## Why it works on hard research
 
 Your AI agent searches the web, skims sources, writes an answer that sounds good, and ships it. There's no scaffold, no audit, no source-vs-source comparison, no provenance, no way to know if it actually engaged with the strongest counter-position. Hyperresearch breaks that pattern through structural enforcement:
@@ -143,13 +195,18 @@ Your AI agent searches the web, skims sources, writes an answer that sounds good
 └─────────────────────────────────────────────────────────────────────┘
 ```
 
-**Subagent triad**, each picked for its job:
+**Subagent roster**, each picked for its job:
 
 | Agent | Model | Role |
 |---|---|---|
 | `hyperresearch-fetcher` | **Haiku** | Mechanical URL fetching via crawl4ai. Cheap, fast, parallel. |
-| `hyperresearch-analyst` | **Sonnet** | Reads one source with research goal in hand. Writes a focused extract, proposes 2-5 next URLs for the loop. |
-| `hyperresearch-auditor` | **Opus** | Adversarial review in two modes against the verbatim prompt. Persists structured findings the save gate verifies. |
+| `hyperresearch-loci-analyst` | **Sonnet** | Reads the width corpus, returns 1–8 depth loci with rationale. Spawn 2 in parallel, dedupe. |
+| `hyperresearch-depth-investigator` | **Sonnet** | Investigates one locus: fetches new sources, writes one interim-report note per locus. |
+| `hyperresearch-dialectic-critic` | **Opus** | Finds counter-evidence the draft missed, emits structured findings with surgical patches. |
+| `hyperresearch-depth-critic` | **Opus** | Finds shallow spots where interim notes could fill specifics. |
+| `hyperresearch-width-critic` | **Opus** | Finds topical corners the corpus supports but the draft ignores. |
+| `hyperresearch-patcher` | **Sonnet** | Tool-locked `[Read, Edit]`. Applies critic findings as surgical Edit hunks. Cannot regenerate. |
+| `hyperresearch-polish-auditor` | **Sonnet** | Tool-locked `[Read, Edit]`. Cuts filler, strips hygiene leaks, breaks long sentences. |
 
 **Routing** — the dispatcher classifies every request by **what the output needs to do**, not what the subject is:
 
@@ -166,16 +223,17 @@ A query about a fictional franchise can be `collect` (per-character enumeration)
 
 ## What's enforced
 
-Eight invariants the protocol structurally prevents from breaking:
+Nine invariants the protocol structurally prevents from breaking:
 
-1. **Verbatim prompt as gospel** — `scaffold-prompt` lint blocks at Checkpoint 3 if the scaffold doesn't open with the user's exact prompt
+1. **Verbatim prompt as gospel** — `scaffold-prompt` lint blocks if the scaffold doesn't open with the user's exact prompt
 2. **Rooted-tree provenance** — `--suggested-by` chain must form a real tree from at least one seed; isolated breadcrumbs flagged
-3. **Analyst coverage** — at least 1 extract per 3 sources, no silent skipping
-4. **Adversarial dissent** — at least one source explicitly contradicts the dominant view, named in writing
-5. **Audit-gate self-cert detection** — CRITICAL findings marked fixed get their underlying lint rules re-run
-6. **Save blocked** until every CRITICAL is genuinely resolved
-7. **Schema integrity** — `tier` and `content_type` are SQLite CHECK-constrained vocabularies; corrupted frontmatter cannot poison the index
-8. **PDF + raw artifact persistence** — fetched PDFs land in `research/raw/` and the `raw_file` frontmatter field survives every re-serialization
+3. **Locus coverage** — every Layer 2 locus must have a Layer 3 `interim-<name>.md` note; missing interims flag as errors
+4. **Patch-only draft modification** — after Layer 4, the patcher and polish auditor are tool-locked to `[Read, Edit]`. They cannot regenerate; the ≤500-char hunk-expansion cap is a mechanical tripwire
+5. **Critical findings never silently skip** — `patch-surgery` lint surfaces any critical finding the patcher couldn't apply
+6. **Schema integrity** — `tier`, `content_type`, and `type` are SQLite CHECK-constrained vocabularies; corrupted frontmatter cannot poison the index
+7. **PDF + raw artifact persistence** — fetched PDFs land in `research/raw/` and the `raw_file` frontmatter field survives every re-serialization
+8. **Interim notes are persisted, not ephemeral** — depth-investigator outputs stay in the vault forever; future sessions can query them
+9. **Hygiene leaks caught on the way out** — scaffold sections, YAML frontmatter, and prompt echoes are stripped by the polish auditor before ship
 
 ---
 
@@ -196,11 +254,8 @@ Eight invariants the protocol structurally prevents from breaking:
 - **Forecasting + strategy** — "Will US inflation stay above 3% through 2026?" → ground-truth statistics + institutional analysis + named contrarians, probability language not hedge
 - **Interpretive analysis** — "What does *Blood Meridian*'s violence mean?" → primary text + critical tradition + dissenting reading, every paragraph fuses fact with interpretive claim
 - **Enumerative coverage** — "For each Napoleonic marshal, cover key campaigns and fate" → every named entity gets every requested field, no silent downgrades
-- **Persistent knowledge base** — every source ever fetched stays searchable. Future sessions check the vault before searching the web. Knowledge compounds.
 
-### `/research-ensemble` — 3 parallel sub-runs + Opus merger (what was benchmarked)
-
-For queries where depth-of-corpus and argument stability matter more than wall-clock time, use `/research-ensemble <query>`. Opus orchestrates; three `hyperresearch-subrun` Sonnet agents run the full protocol in parallel against one shared vault, each with a subtly different framing (evidentiary breadth / citation-chain depth / dialectical tension). A fourth agent (`hyperresearch-merger`, Opus) scores each draft on comprehensiveness / readability / argument strength / citation quality, picks the strongest as the base, and splices in unique evidence from the other two. Cost is ~5× a normal run; the payoff is detailed in [Benchmarks](#benchmarks) below.
+All of the above run under the default [`/research-layercake`](#research-layercake--7-phases-one-draft-patched-not-regenerated-the-default) mode: width sweep → loci analysis → depth investigations → one draft → three adversarial critics → surgical patch pass → polish audit. For faster turnaround when the query doesn't warrant the full pipeline, type `/research <query>` for a single-pass run.
 
 ---
 
@@ -210,8 +265,8 @@ One command sets up the full integration:
 
 - **`.claude/settings.json`** — PreToolUse hook that nudges Claude Code to check the vault before any raw web search
 - **`.claude/skills/hyperresearch/`** — `/research` skill (dispatcher + 4 modality files: collect / synthesize / compare / forecast)
-- **`.claude/skills/research-ensemble/`** — `/research-ensemble` skill (3× parallel sub-runs + Opus merger; see [Benchmarks](#benchmarks))
-- **`.claude/agents/`** — six registered subagents: `hyperresearch-fetcher` (Haiku), `hyperresearch-analyst` (Sonnet), `hyperresearch-auditor` (Opus), `hyperresearch-rewriter` (Sonnet), `hyperresearch-subrun` (Sonnet, ensemble), `hyperresearch-merger` (Opus, ensemble)
+- **`.claude/skills/research-layercake/`** — `/research-layercake` skill (the 7-phase pipeline)
+- **`.claude/agents/`** — eight registered subagents: `hyperresearch-fetcher` (Haiku), `hyperresearch-loci-analyst` (Sonnet), `hyperresearch-depth-investigator` (Sonnet), `hyperresearch-dialectic-critic` (Opus), `hyperresearch-depth-critic` (Opus), `hyperresearch-width-critic` (Opus), `hyperresearch-patcher` (Sonnet, `[Read, Edit]`-locked), `hyperresearch-polish-auditor` (Sonnet, `[Read, Edit]`-locked)
 - **`CLAUDE.md`** at the vault root — the full research workflow, automatically loaded by Claude Code on every session
 
 hyperresearch is Claude Code-only for now. Codex, Cursor, and Gemini support was trimmed from v0.6 to focus the surface area — may return as real integrations later.
@@ -271,26 +326,6 @@ LinkedIn, Twitter, Facebook, Instagram, and TikTok automatically use a visible b
 
 ---
 
-## Benchmarks
-
-Hyperresearch is being evaluated on the [DeepResearch Bench](https://github.com/Ayanami0730/deep_research_bench) RACE eval (Gemini-2.5-Pro judges reports on comprehensiveness, insight, instruction-following, readability). Full per-modality scores and head-to-head comparisons vs other research harnesses will land here as the 100-query sweep completes. **Status:** in progress.
-
-### Ensemble lift — preliminary data point
-
-Query 91 from DeepResearch-Bench ("detailed analysis of the Saint Seiya franchise, structured around armor classes, per-character power/techniques/arcs/fate") is a deep-collect query with ~100 named entities. One data point so far:
-
-| Variant | Overall | Comp. | Insight | Inst. | Read. | Cost | Time |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| `/research` single-run (sonnet, v1 baseline) | **52.74** | 52.95 | 58.74 | 51.28 | 51.90 | ~$1.50 | ~8 min |
-| `/research-ensemble` (3× sonnet + opus merger) | **54.82** | 54.52 | 58.57 | 53.30 | 56.10 | $21.98 | 57 min |
-| **Delta** | **+2.08** | +1.57 | −0.17 | +2.02 | +4.20 | ~14× | ~7× |
-
-Readability is the biggest single lift (+4.20) — the merger smooths splice boundaries and harmonizes voice across the three sub-runs. Instruction-following (+2.02) gains from three independent passes cross-checking the prompt's entity list; prompt-named entities the base run collapsed get restored from sibling runs. Comprehensiveness (+1.57) comes from the 3-way evidence union (167 sources fetched into the shared corpus vs ~30 for a single run). Insight is flat as expected — the merger is a compiler, not a re-thinker; it can't deepen an argument beyond what the strongest sub-run already had.
-
-One query is not a benchmark. Full 100-query sweep coming next. The cost ceiling is honest: ~5× per query at the tool-call level, ~14× here because `/research` ran on Sonnet and `/research-ensemble` requires Opus orchestration + Opus merger. Use ensemble where the query earns it; use `/research` for everything else.
-
----
-
 ## Philosophy
 
 - **Process is load-bearing.** A draft without a scaffold, comparisons, audit findings, and a clean provenance graph is unfinished — regardless of how good the prose reads.

{hyperresearch-0.6.0 → hyperresearch-0.7.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "hyperresearch"
-version = "0.6.0"
+version = "0.7.0"
 description = "Claude Code harness for disciplined, adversarially-audited deep research with full provenance."
 readme = "README.md"
 license = "MIT"