@ara-commons/ara-skills 0.1.0

package/skills/research-manager/references/event-taxonomy.md

@@ -0,0 +1,160 @@
+# Event Taxonomy & Routing Rules
+
+Canonical reference for **Stage 2 (Event Router)** of the Live PM pipeline. Loaded on
+demand at epilogue time. SKILL.md owns the pipeline orchestration, closure signals,
+crystallization procedure, contradiction trigger, and schemas — this file does not
+duplicate those.
+
+This document covers two axes:
+
+| Axis | Question | Outcome |
+|------|----------|---------|
+| **Kind** | What kind of event is this? | Picks the schema and target layer. |
+| **Routing** | Is this a journey fact or interpretation? | Picks **direct** vs **staged**. |
+
+A journey fact records *what occurred* (a choice, a run, an abandonment). It is immutable
+and goes direct. An interpretive claim records *what something means or what is generally
+true*. It is revisable, goes staged, and only crystallizes on a closure signal.
+
+## Direct-Routed Events (Journey Layer)
+
+Write to `trace/exploration_tree.yaml` immediately at end of turn.
+
+| Type | Signals | Required payload |
+|------|---------|------------------|
+| `question` | "What if...", "Should we...", "How does...", a research direction opened | `description` |
+| `decision` | User chose between alternatives, committed to a direction | `choice`, `alternatives`, `evidence` |
+| `experiment` | Code ran a test/benchmark, results produced | `result`, `evidence` |
+| `dead_end` | Approach abandoned, hypothesis falsified, "doesn't work", reverted | `hypothesis`, `failure_mode`, `lesson` |
+| `pivot` | Major direction change triggered by evidence | `from`, `to`, `trigger` |
+
+A `decision` node MAY reference a staged observation as evidence — this counts as
+**artifact-commitment** for that observation (closure signal; see SKILL.md Stage 3).
+
+`ai-action` events (AI wrote code, ran a command) go to the session record's `ai_actions`
+list, **not** to the exploration tree.
+
+## Staged-Routed Events (Interpretive — Buffered for Maturity)
+
+Write to `staging/observations.yaml` first, with `potential_type` indicating where they
+would crystallize. They do **not** enter `logic/` until a closure signal fires (see
+SKILL.md Stage 3).
+
+| Candidate Event | Signals | Crystallizes To | `potential_type` |
+|-----------------|---------|-----------------|------------------|
+| `claim` | "I believe...", "The system achieves...", falsifiable assertion about capability/property | `logic/claims.md` | `claim` |
+| `heuristic` | "The trick is...", "You need to...", implementation rule with rationale | `logic/solution/heuristics.md` | `heuristic` |
+| `concept` | New term defined, disambiguation needed | `logic/concepts.md` | `concept` |
+| `constraint` | "This only works when...", boundary condition | `logic/solution/constraints.md` | `constraint` |
+| `architecture` | System design statement, component relationship | `logic/solution/architecture.md` | `architecture` |
+| (unclassified) | Interesting but not yet typed | (stays staged) | `unknown` |
+
+Evidence artifacts (tables, figures, metrics) referenced by a direct-routed `experiment`
+get written to `evidence/` immediately — they are raw data, not interpretation.
+
+## Routing Decision Tree
+
+```
+What KIND of event is this?
+
+  Journey fact (something that happened)?
+    Was a choice made between alternatives?
+      → decision  [DIRECT to trace/]
+    Did code/test produce a result?
+      → experiment  [DIRECT to trace/, plus evidence/ for artifacts]
+    Was an approach abandoned with a reason?
+      → dead_end  [DIRECT to trace/]
+    Was there a major direction change?
+      → pivot  [DIRECT to trace/]
+    Was a research question opened?
+      → question  [DIRECT to trace/]
+    Did the AI perform an action (write code, run command)?
+      → ai-action  [session record only]
+
+  Interpretation (something asserted to be true / general)?
+    Falsifiable assertion about the system?
+      → STAGE as potential_type: claim
+    Implementation rule with rationale?
+      → STAGE as potential_type: heuristic
+    Term definition?
+      → STAGE as potential_type: concept
+    Boundary condition?
+      → STAGE as potential_type: constraint
+    System-design statement?
+      → STAGE as potential_type: architecture
+    Doesn't fit?
+      → STAGE as potential_type: unknown
+```
+
+## Skip Filter (no record)
+
+Do not write any record for these:
+- Routine file reads with no downstream decision
+- Typo fixes, formatting changes, lint passes
+- Git status checks, dependency installs, environment setup
+- Greetings, acknowledgments, "thanks"
+- Clarifying questions whose answer added no new content
+- Pure restatement of the user's request
+
+If a turn contains only skip-filter activity, print
+`[PM] Turn skipped: no research events.` (or stay silent) and exit the epilogue.
+
+## Provenance Assignment
+
+```
+Who generated this information?
+
+User said it directly (typed it, stated it, confirmed it)
+  → provenance: user
+
+AI inferred it from code, output, or conversation context
+  → provenance: ai-suggested
+
+AI performed an action (wrote code, ran test, made edit)
+  → provenance: ai-executed
+
+User modified an AI suggestion ("no, actually..." / "more like...")
+  → provenance: user-revised
+
+Uncertain?
+  → provenance: ai-suggested  (conservative default)
+```
+
+`ai-suggested` never auto-upgrades. A subsequent **verbal-affirmation** closure signal
+upgrades it to `user-revised` (or `user` if the affirmation reproduces the assertion
+verbatim). The other three closure signals license crystallization but do **not** change
+provenance.
+
+### Trust calibration
+
+The provenance distribution of an artifact is itself a quality signal: a project full of
+`ai-suggested` claims is less trustworthy than one full of `user` / `user-revised` claims.
+Reviewers and downstream tools (e.g., rigor-reviewer L2) inspect this distribution.
+
+## ID Conventions
+
+| Type | Prefix | Example | Scope |
+|------|--------|---------|-------|
+| Exploration node | N | N01, N02 | Global (across all turns and sessions) |
+| Claim | C | C01, C02 | Global; assigned at crystallization, not at staging |
+| Heuristic | H | H01, H02 | Global; assigned at crystallization |
+| Experiment plan | E | E01, E02 | Global |
+| Observation | O | O01, O02 | Global; assigned at staging |
+| Session | date_seq | 2026-04-27_001 | Unique per calendar day |
+
+Always read the target file to find the highest existing ID before assigning a new one.
+
+## Forensic Binding Checklist
+
+Establish at write time. If a binding is not yet possible, write `[pending]` and leave a
+TODO comment so a future epilogue can complete it.
+
+- **Claim → Proof**: at crystallization, what evidence supports/refutes it?
+- **Experiment → Claim**: which staged or crystallized claim does this experiment test?
+  This binding is what enables the **empirical-resolution** closure signal.
+- **Heuristic → Code**: where in the codebase is this implemented?
+- **Decision → Evidence**: which exploration nodes or evidence artifacts motivated it?
+- **Dead End → Lesson**: what was learned that prevents repeating the mistake?
+- **Observation → Bound nodes**: at staging time, list `bound_to: [N{XX}, ...]` for any
+  exploration nodes the observation depends on. Without this list, empirical-resolution
+  cannot be detected automatically.

package/skills/rigor-reviewer/SKILL.md

@@ -0,0 +1,332 @@
+---
+name: rigor-reviewer
+description: |
+  ARA Seal Level 2: Semantic Epistemic Review. Acts as an objective research
+  reviewer for Agent-Native Research Artifacts. Assumes Level 1 structural
+  validation has already passed. Evaluates six dimensions of epistemic quality
+  through semantic reasoning over the ARA's content. Produces a scored review
+  with per-dimension strengths/weaknesses/suggestions, severity-ranked findings,
+  and an overall recommendation (Strong Accept to Reject).
+
+  TRIGGERS: level2, seal level 2, verify level 2, epistemic audit, review ara, audit claims
+argument-hint: "<artifact_dir>"
+allowed-tools: Read, Write, Glob, Grep
+metadata:
+  category: research-tooling
+  version: "3.0.0"
+  last_updated: "2026-04-16"
+user-invocable: true
+---
+
+# ARA Seal Level 2: Semantic Epistemic Review
+
+You are an objective research reviewer for Agent-Native Research Artifacts. You receive an
+ARA directory path and produce a comprehensive review as `level2_report.json` at the
+artifact root. You operate entirely through your native tools (Read, Write, Glob, Grep).
+You do NOT execute code, fetch URLs, or consult external sources.
+
+**Prerequisite**: Level 1 (structural validation) has already passed. All references
+resolve, required fields exist, the exploration tree parses correctly, and cross-layer
+links are bidirectionally consistent. Level 2 does NOT re-check any of this. Instead, it
+evaluates whether the *content* of the ARA is epistemically sound: whether evidence
+actually supports claims, whether the argument is coherent, and whether the research
+process is honestly documented.
+
+Your review is **constructive**: identify both strengths and weaknesses, provide actionable
+suggestions, and give a calibrated overall assessment. You are not a bug detector; you are
+a reviewer who helps authors improve their work.
+
+---
+
+## Six Review Dimensions
+
+Each dimension is scored 1-5 and includes strengths, weaknesses, and suggestions.
+All checks are semantic: they require reading comprehension and reasoning, not structural validation.
+
+| Dimension | What it evaluates |
+|-----------|-------------------|
+| **D1. Evidence Relevance** | Does the cited evidence actually support each claim in substance, not just by reference? |
+| **D2. Falsifiability Quality** | Are falsification criteria meaningful, actionable, and well-scoped? |
+| **D3. Scope Calibration** | Do claims assert exactly what their evidence supports, no more, no less? |
+| **D4. Argument Coherence** | Does the narrative follow a logical arc from problem to solution to evidence? |
+| **D5. Exploration Integrity** | Does the exploration tree document genuine research process, including failures? |
+| **D6. Methodological Rigor** | Are experiments well-designed with adequate baselines, ablations, and reporting? |
+
+---
+
+## Procedure
+
+### Step 1: Read the ARA
+
+Read files in this fixed order. Record the list as `read_order` in the report.
+
+1. `PAPER.md`
+2. `logic/claims.md`
+3. `logic/experiments.md`
+4. `logic/problem.md`
+5. `logic/concepts.md`
+6. `logic/solution/architecture.md`, `algorithm.md`, `constraints.md`, `heuristics.md`
+7. `logic/related_work.md`
+8. `trace/exploration_tree.yaml`
+9. `evidence/README.md` (if exists)
+10. Spot-check 2-3 evidence files from `evidence/tables/` or `evidence/figures/`
+
+### Step 2: Parse Entities
+
+**Claims** (from `logic/claims.md`): each `## C{NN}: {title}` section. Extract:
+- `Statement`, `Status`, `Falsification criteria`, `Proof` (experiment IDs), `Dependencies` (claim IDs), `Tags`
+
+**Experiments** (from `logic/experiments.md`): each `## E{NN}: {title}` section. Extract:
+- `Verifies` (claim IDs), `Setup`, `Procedure`, `Metrics`, `Expected outcome`, `Baselines`, `Dependencies`
+
+**Heuristics** (from `logic/solution/heuristics.md`): each `## H{NN}` section. Extract:
+- `Rationale`, `Sensitivity`, `Bounds`, `Code ref`
+
+**Observations and Gaps** (from `logic/problem.md`): each `O{N}` and `G{N}`.
+
+**Exploration tree** (from `trace/exploration_tree.yaml`): all nodes with `id`, `type`, `title`, and type-specific fields (`failure_mode`, `lesson`, `choice`, `alternatives`, `result`).
+
+### Step 3: Build Working Maps
+
+Construct these maps as inputs for semantic analysis. Do NOT validate structural integrity
+(Level 1 guarantees it).
+
+- **claim_proof_map**: for each claim, the set of experiment IDs in its Proof
+- **experiment_verifies_map**: for each experiment, the set of claim IDs in its Verifies
+- **claim_dependency_edges**: directed edges from each claim to its Dependencies
+- **gap_set**: all G{N} from problem.md
+- **rejected_nodes**: exploration tree nodes with type = `dead_end` or `pivot`
+- **decision_nodes**: exploration tree nodes with type = `decision`
+
+### Step 4: Evaluate Each Dimension
+
+For each dimension, perform semantic reasoning over the parsed content. Record strengths, weaknesses, and suggestions as you go.
+
+---
+
+#### D1. Evidence Relevance
+
+For each claim-experiment pair linked through Proof/Verifies:
+
+- **Relevance**: Does the experiment's Setup/Procedure/Metrics actually address what the claim asserts? (Not just "link exists" but "link is substantively relevant.")
+- **Type-aware entailment**: Infer claim type from Statement cues, check experiment design matches:
+  - Causal ("causes", "leads to", "enables") → needs isolating ablation
+  - Generalization ("generalizes", "robust", "across") → needs heterogeneous test conditions
+  - Improvement ("outperforms", "better", "improves") → needs baseline comparison
+  - Descriptive ("accounts for", "distribution", "pattern") → needs representative sampling
+  - Scoping ("when", "under conditions", "limited to") → needs declared bounds
+- **Evidence sufficiency**: Is a single experiment enough to support this claim, or does the claim's scope demand multiple independent experiments?
+
+**Scoring anchors:**
+- **5**: Type-appropriate, relevant evidence for every claim; multi-experiment support where needed
+- **4**: Evidence relevant for all claims, minor type mismatches (e.g., causal claim with correlation-only evidence)
+- **3**: Most claim-experiment pairs are relevant, 1-2 weak matches where evidence doesn't quite address the claim
+- **2**: Multiple claims where cited experiments don't substantively address what the claim asserts
+- **1**: Majority of claims cite experiments that are irrelevant to their statements
+
+---
+
+#### D2. Falsifiability Quality
+
+For each claim's Falsification criteria field:
+
+- **Actionability**: Could an independent researcher execute this criterion? Does it specify what to measure, what threshold constitutes failure, and under what conditions?
+- **Non-triviality**: Is the criterion non-tautological? ("If the method doesn't work" is trivial. "Re-evaluation on the same 77-paper set where GPT-5 is not the top model" is actionable.)
+- **Scope match**: Does the falsification criterion address the same scope as the Statement? (A claim about "all datasets" with falsification mentioning only one dataset is mismatched.)
+- **Independence**: Could the criterion be tested without access to the authors' proprietary data or systems?
+
+**Scoring anchors:**
+- **5**: Every claim has specific, actionable, independently testable falsification criteria matching the claim's scope
+- **4**: Most criteria are strong, 1-2 are vague or hard to operationalize
+- **3**: Mixed quality; some actionable, some trivial or scope-mismatched
+- **2**: Most criteria are trivial, tautological, or scope-mismatched
+- **1**: Falsification criteria meaningless across claims
+
+---
+
+#### D3. Scope Calibration
+
+- **Over-claiming**: Does any Statement use universal scope markers ("all models", "any dataset", "state-of-the-art across all") while cited experiments cover only specific, narrow conditions? The gap must be substantial.
+- **Under-claiming**: Are there important experimental results present in evidence/ that are not captured by any claim? (Evidence without a corresponding claim.)
+- **Assumption explicitness**: Are key assumptions stated in problem.md (Assumptions section) or constraints.md? Are there unstated assumptions implied by the experimental design?
+- **Generalization boundaries**: Does the artifact clearly state what the claims do NOT apply to? Check constraints.md and limitations in the exploration tree.
+- **Qualifier consistency**: When claims use hedging ("tends to", "in most cases"), is this consistent with the evidence strength?
+
+**Scoring anchors:**
+- **5**: All claims precisely match evidence scope, assumptions explicit, limits clearly stated
+- **4**: Claims well-scoped with minor gaps in assumption documentation
+- **3**: Some claims slightly over/under-reach, assumptions partially stated
+- **2**: Multiple over-claims or significant undocumented assumptions
+- **1**: Pervasive scope mismatch between claims and evidence
+
+---
+
+#### D4. Argument Coherence
+
+- **Observation → Gap derivation**: Do the stated gaps follow logically from the observations? Or are they asserted without connection?
+- **Gap → Insight connection**: Does the key insight in problem.md address the identified gaps?
+- **Insight → Solution alignment**: Does the solution architecture implement the key insight?
+- **Solution → Claims coverage**: Do the claims cover the solution's main contributions?
+- **Cross-layer consistency**: Do claims, exploration tree, and evidence tell the same story? Flag contradictions.
+- **Narrative completeness**: Are there motivating questions from problem.md that are neither answered nor explicitly deferred?
+- **Gap coverage**: For each gap in problem.md, is there at least one claim that substantively addresses it? Flag gaps that are motivated but never resolved.
+
+**Scoring anchors:**
+- **5**: Clear logical arc (observations → gaps → insight → solution → claims → evidence), all gaps addressed, no contradictions
+- **4**: Strong flow with minor logical gaps or one unaddressed gap
+- **3**: General flow present but some disconnects between layers
+- **2**: Significant misalignment between problem statement and claims, or unresolved contradictions
+- **1**: No coherent logical flow; layers tell different stories
+
+---
+
+#### D5. Exploration Integrity
+
+- **Dead-end quality**: Is the `failure_mode` specific enough to be actionable? ("Didn't work" is bad. "Divergence after 1000 steps due to gradient explosion" is good.) Is the `lesson` a genuine transferable insight?
+- **Decision rationale quality**: Do rationales explain WHY the chosen path was preferred over alternatives? Are alternatives real alternatives or strawmen?
+- **Rebutted-branch consistency**: Does any claim advocate an approach marked as dead_end or pivot in the tree? (This is a logical contradiction.)
+- **Exploration breadth**: For the paper's main design choices, were at least 2 alternatives considered and documented?
+- **Honesty signal**: Does the tree document genuine negative results, or does it read like a post-hoc justification? A tree with zero dead-ends or only trivial failures is suspicious.
+
+**Scoring anchors:**
+- **5**: Rich tree with well-documented dead-ends (specific failure modes, actionable lessons), thorough decision rationale, genuine negative results
+- **4**: Good tree with minor gaps in dead-end documentation or decision rationale
+- **3**: Tree present but dead-ends lack specificity or decisions lack alternatives
+- **2**: Boilerplate documentation; dead-ends and decisions read as formulaic rather than authentic
+- **1**: Tree contradicts claims or reads entirely as post-hoc justification
+
+---
+
+#### D6. Methodological Rigor
+
+- **Baseline adequacy**: Are the right things being compared? Are baselines recent and relevant? Flag experiments with "no baseline" for comparative claims.
+- **Ablation coverage**: For claims involving multiple components, does at least one experiment isolate individual contributions?
+- **Statistical reporting**: Do experiments mention variance, confidence intervals, number of runs, or statistical tests? Flag single-run results for quantitative claims.
+- **Metric-claim alignment**: Does the metric actually measure what the claim asserts? (A claim about "generalization" measured only by accuracy on one test set is misaligned.)
+- **Reproducibility signals**: Are experiment setups specific enough for independent replication? (Model name, dataset, hardware, hyperparameters.)
+
+**Scoring anchors:**
+- **5**: Comprehensive baselines, proper ablations, statistical rigor, metrics precisely match claims, fully reproducible setup
+- **4**: Strong methodology with minor gaps (e.g., missing variance on one experiment)
+- **3**: Adequate but missing some baselines or statistical details
+- **2**: Significant gaps; missing baselines for comparative claims or no ablations
+- **1**: No baselines, no ablations, metrics don't match claims
+
+---
+
+### Step 5: Compile Findings
+
+Collect all issues found across the six dimensions into a single findings list. Assign each finding:
+
+- **finding_id**: F01, F02, ... (sequential)
+- **dimension**: which of D1-D6
+- **severity**: one of:
+  - `critical` — fundamental epistemic flaw; the claim or argument cannot stand as written
+  - `major` — significant weakness that undermines a claim or dimension score
+  - `minor` — noticeable issue that doesn't invalidate the work
+  - `suggestion` — constructive improvement opportunity, not a flaw
+- **target_file**: which ARA file
+- **target_entity**: C{NN}, E{NN}, H{NN}, G{N}, or node ID (if applicable)
+- **evidence_span**: verbatim substring from the ARA that triggered the finding (MUST be exact quote; omit if the finding is about an absence)
+- **observation**: what you found (factual)
+- **reasoning**: why it matters (analytical)
+- **suggestion**: how to fix or improve it (constructive)
+
+Sort findings by severity: critical first, then major, minor, suggestion.
+
+### Step 6: Compute Overall Grade
+
+Calculate the mean of the six dimension scores. Apply the grade mapping:
+
+| Grade | Condition |
+|-------|-----------|
+| **Strong Accept** | mean ≥ 4.5 AND no dimension < 3 |
+| **Accept** | mean ≥ 3.8 AND no dimension < 2 |
+| **Weak Accept** | mean ≥ 3.0 AND no dimension < 2 |
+| **Weak Reject** | mean ≥ 2.0 AND (mean < 3.0 OR any dimension < 2) |
+| **Reject** | mean < 2.0 OR any dimension = 1 |
+
+### Step 7: Write Report
+
+Write `level2_report.json` to the artifact root:
+
+```json
+{
+  "artifact": "<name>",
+  "artifact_dir": "<path>",
+  "review_version": "3.0.0",
+  "prerequisite": "Level 1 passed",
+
+  "overall": {
+    "grade": "Accept",
+    "mean_score": 4.1,
+    "one_line_summary": "<1 sentence: what makes this ARA strong or weak>",
+    "strengths_summary": ["<top 2-3 strengths across all dimensions>"],
+    "weaknesses_summary": ["<top 2-3 weaknesses across all dimensions>"]
+  },
+
+  "dimensions": {
+    "D1_evidence_relevance": {
+      "score": 4,
+      "strengths": ["Evidence is substantively relevant for all 6 claims"],
+      "weaknesses": ["C02 cites a correlation study but makes a causal claim"],
+      "suggestions": ["Add an ablation experiment to isolate the causal mechanism for C02"]
+    },
+    "D2_falsifiability": {
+      "score": 4,
+      "strengths": ["..."],
+      "weaknesses": ["C02 falsification criteria is hard to operationalize independently"],
+      "suggestions": ["Specify a concrete re-annotation protocol for C02"]
+    },
+    "D3_scope_calibration": { "score": 4, "..." : "..." },
+    "D4_argument_coherence": { "score": 4, "..." : "..." },
+    "D5_exploration_integrity": { "score": 3, "..." : "..." },
+    "D6_methodological_rigor": { "score": 4, "..." : "..." }
+  },
+
+  "findings": [
+    {
+      "finding_id": "F01",
+      "dimension": "D6_methodological_rigor",
+      "severity": "major",
+      "target_file": "logic/experiments.md",
+      "target_entity": "E03",
+      "evidence_span": "**Baselines**: No random or retrieval-only baseline reported",
+      "observation": "E03 evaluates four LLMs on research ideation but includes no non-LLM baseline.",
+      "reasoning": "Without a random or retrieval-only baseline, it is impossible to assess whether LLM performance is meaningfully above chance.",
+      "suggestion": "Add a retrieval-only baseline (e.g., BM25 nearest-neighbor from predecessor abstracts) to contextualize Hit@10 scores."
+    }
+  ],
+
+  "questions_for_authors": [
+    "What is the inter-annotator agreement on thinking-pattern classification? A single LLM pass without human validation on the full corpus leaves taxonomy reliability uncertain.",
+    "..."
+  ],
+
+  "read_order": ["PAPER.md", "logic/claims.md", "..."]
+}
+```
+
+---
+
+## Critical Rules
+
+1. **Verbatim evidence_span**: Findings about content present in the ARA MUST quote an exact substring. Findings about absences (missing baseline, scope mismatch) may omit evidence_span.
+
+2. **Constructive tone**: Every weakness must come with a suggestion. You are helping authors improve, not punishing them.
+
+3. **Calibrated scoring**: Most competent ARAs should land in the 3-4 range. A score of 5 means genuinely excellent, not just "no problems found." A score of 1 means fundamental problems, not just "could be better."
+
+4. **No false grounding**: Support must flow through Proof → experiments.md → evidence/. Agreement in prose (problem.md, architecture.md) does not substitute for experimental evidence.
+
+5. **Artifact-only**: Do not fetch external URLs, execute code, or consult external sources. Take the ARA's reported evidence at face value.
+
+6. **Balanced review**: Actively look for strengths, not just weaknesses. A review that only lists problems is not useful.
+
+7. **No structural re-checks**: Do NOT verify reference resolution, field presence, YAML parsing, or cross-link consistency. Level 1 has already validated all of this. Focus entirely on whether the *content* is epistemically sound.
+
+---
+
+## Reference
+
+See `references/review-dimensions.md` for scoring anchor details and check inventories per dimension.

package/skills/rigor-reviewer/references/review-dimensions.md

@@ -0,0 +1,181 @@
+# Level 2 Review Dimensions — Scoring Anchors and Check Inventory
+
+Six dimensions of epistemic quality. All checks are semantic: they require reading
+comprehension and reasoning over the ARA's content. Structural validation (reference
+resolution, field presence, YAML parsing) is handled entirely by Level 1.
+
+---
+
+## D1. Evidence Relevance
+
+**Question**: Does the cited evidence actually support each claim in substance, not just by reference?
+
+### Checks
+
+| Check | What to verify | Finding severity |
+|-------|---------------|-----------------|
+| Relevance | Experiment's Setup/Procedure addresses what the claim actually asserts | major |
+| Type-aware entailment | Experiment design matches claim type (causal→ablation, generalization→heterogeneous, improvement→baseline, descriptive→sampling, scoping→bounds) | major |
+| Evidence sufficiency | Is a single experiment enough to support this claim, or are multiple needed? | suggestion |
+
+### Scoring Anchors
+
+| Score | Description |
+|-------|-------------|
+| 5 | Type-appropriate, relevant evidence for every claim; multi-experiment support where needed |
+| 4 | Evidence relevant for all claims, minor type mismatches |
+| 3 | Most claim-experiment pairs relevant, 1-2 weak matches |
+| 2 | Multiple claims where cited experiments don't substantively address the claim |
+| 1 | Majority of claims cite experiments irrelevant to their statements |
+
+---
+
+## D2. Falsifiability Quality
+
+**Question**: Are claims genuinely falsifiable with meaningful, actionable criteria?
+
+### Checks
+
+| Check | What to verify | Finding severity |
+|-------|---------------|-----------------|
+| Actionability | Could an independent researcher execute this? Specifies what to measure, failure threshold, and conditions? | major |
+| Non-triviality | Is the criterion more than a tautology? ("If the method doesn't work" = trivial) | major |
+| Scope match | Does the criterion address the same scope as the Statement? | major |
+| Independence | Could it be tested without proprietary data or systems? | minor |
+
+### Scoring Anchors
+
+| Score | Description |
+|-------|-------------|
+| 5 | Every claim has specific, actionable, independently testable criteria matching claim scope |
+| 4 | Most criteria are strong, 1-2 vague or hard to operationalize |
+| 3 | Mixed; some actionable, some trivial or scope-mismatched |
+| 2 | Most criteria trivial, tautological, or scope-mismatched |
+| 1 | Criteria meaningless across claims |
+
+---
+
+## D3. Scope Calibration
+
+**Question**: Do claims assert exactly what their evidence supports — no more, no less?
+
+### Checks
+
+| Check | What to verify | Finding severity |
+|-------|---------------|-----------------|
+| Over-claiming | Statement uses universal scope while evidence covers narrow conditions | critical if extreme, major if moderate |
+| Under-claiming | Evidence files or experiment results not captured by any claim | minor |
+| Assumption explicitness | Key assumptions stated in problem.md or constraints.md | major if unstated assumptions affect validity |
+| Generalization boundaries | Artifact states what claims do NOT apply to | minor |
+| Qualifier consistency | Hedging language matches evidence strength | minor |
+
+### Scoring Anchors
+
+| Score | Description |
+|-------|-------------|
+| 5 | All claims precisely match evidence scope, assumptions explicit, limits stated |
+| 4 | Well-scoped with minor gaps in assumption documentation |
+| 3 | Some claims slightly over/under-reach, assumptions partially stated |
+| 2 | Multiple over-claims or significant undocumented assumptions |
+| 1 | Pervasive scope mismatch between claims and evidence |
+
+---
+
+## D4. Argument Coherence
+
+**Question**: Does the argument follow a coherent path from problem to solution to evidence?
+
+### Checks
+
+| Check | What to verify | Finding severity |
+|-------|---------------|-----------------|
+| Observation → Gap derivation | Gaps follow logically from observations | major |
+| Gap → Insight connection | Key insight addresses the identified gaps | major |
+| Insight → Solution alignment | Solution architecture implements the key insight | major |
+| Solution → Claims coverage | Claims cover the solution's main contributions | minor |
+| Cross-layer consistency | Claims, tree, and evidence tell the same story | major |
+| Narrative completeness | Motivating questions are answered or explicitly deferred | minor |
+| Gap coverage | Every gap is substantively addressed by at least one claim | major |
+
+### Scoring Anchors
+
+| Score | Description |
+|-------|-------------|
+| 5 | Clear arc from observations → gaps → insight → solution → claims → evidence, all gaps addressed |
+| 4 | Strong flow with minor gaps or one unaddressed gap |
+| 3 | General flow present but disconnects between layers |
+| 2 | Significant misalignment between problem and claims, or contradictions |
+| 1 | No coherent logical flow; layers tell different stories |
+
+---
+
+## D5. Exploration Integrity
+
+**Question**: Does the exploration tree faithfully document the research journey?
+
+### Checks
+
+| Check | What to verify | Finding severity |
+|-------|---------------|-----------------|
+| Dead-end specificity | failure_mode is concrete, lesson is transferable | major |
+| Decision rationale quality | Rationale explains why chosen path preferred over real alternatives | major |
+| Rebutted-branch consistency | No claim advocates a dead_end or pivot approach | critical |
+| Exploration breadth | Main design choices have ≥2 documented alternatives | minor |
+| Honesty signal | Tree documents genuine negatives, not post-hoc justification | suggestion |
+
+### Scoring Anchors
+
+| Score | Description |
+|-------|-------------|
+| 5 | Rich tree, specific failure modes, actionable lessons, thorough rationale, genuine negatives |
+| 4 | Good tree with minor gaps in dead-end or decision documentation |
+| 3 | Tree present but dead-ends lack specificity or decisions lack alternatives |
+| 2 | Boilerplate documentation; dead-ends and decisions read as formulaic |
+| 1 | Tree contradicts claims or reads entirely as post-hoc justification |
+
+---
+
+## D6. Methodological Rigor
+
+**Question**: Are experiments well-designed with adequate baselines and reporting?
+
+### Checks
+
+| Check | What to verify | Finding severity |
+|-------|---------------|-----------------|
+| Baseline adequacy | Right things compared? Baselines recent and relevant? | major |
+| Ablation coverage | Multi-component claims have experiments isolating individual contributions | major |
+| Statistical reporting | Variance, CI, number of runs, or tests mentioned | major for quantitative claims |
+| Metric-claim alignment | Metric measures what claim asserts | major |
+| Reproducibility signals | Setup specific enough for replication (model, dataset, hardware, hyperparameters) | minor |
+
+### Scoring Anchors
+
+| Score | Description |
+|-------|-------------|
+| 5 | Comprehensive baselines, proper ablations, statistical rigor, precise metric-claim alignment |
+| 4 | Strong methodology with minor gaps |
+| 3 | Adequate but missing some baselines or statistical details |
+| 2 | Significant gaps; missing baselines for comparative claims or no ablations |
+| 1 | No baselines, no ablations, metrics don't match claims |
+
+---
+
+## Overall Grade Mapping
+
+| Grade | Condition |
+|-------|-----------|
+| **Strong Accept** | mean ≥ 4.5 AND no dimension < 3 |
+| **Accept** | mean ≥ 3.8 AND no dimension < 2 |
+| **Weak Accept** | mean ≥ 3.0 AND no dimension < 2 |
+| **Weak Reject** | mean ≥ 2.0 AND (mean < 3.0 OR any dimension < 2) |
+| **Reject** | mean < 2.0 OR any dimension = 1 |
+
+## Finding Severity Definitions
+
+| Severity | Meaning | Example |
+|----------|---------|---------|
+| `critical` | Fundamental epistemic flaw; the claim or argument cannot stand as written | Causal claim supported only by correlation; claim advocates a dead-end approach |
+| `major` | Significant weakness that undermines a claim or dimension | Comparative claim with no baseline; trivial falsification criteria; metric doesn't match claim |
+| `minor` | Noticeable issue that doesn't invalidate the work | Missing generalization boundaries; hedging inconsistent with evidence |
+| `suggestion` | Constructive improvement, not a flaw | Adding a retrieval baseline for context; documenting exploration breadth |