@ara-commons/ara-skills 0.1.0

package/skills/compiler/references/ara-schema.md

@@ -0,0 +1,438 @@
+# ARA Directory Schema — Complete Field-Level Reference
+
+## Directory Structure
+
+```
+PAPER.md                            # Level 1: Root manifest + layer index
+logic/
+  problem.md                        # Why: observations → gaps → key insight
+  claims.md                         # Falsifiable assertions
+  concepts.md                       # All key technical terms (one ## per term)
+  experiments.md                    # Declarative experiment plans (NOT scripts)
+  solution/
+    architecture.md                 # System design + component graph
+    algorithm.md                    # Math formulation + pseudocode
+    constraints.md                  # Boundary conditions + limitations
+    heuristics.md                   # Convergence tricks + rationale
+  related_work.md                   # Typed dependency graph (RDO)
+src/
+  configs/
+    training.md                     # Training hyperparameters with rationale
+    model.md                        # Architecture/model configs
+  execution/
+    {module}.py                     # Minimal code stubs (core algorithm only)
+  environment.md                    # Dependencies, hardware, seeds
+trace/
+  exploration_tree.yaml             # Research DAG: nested YAML tree with typed nodes
+evidence/
+  README.md                         # Index mapping every evidence file to claims
+  tables/                           # Raw result tables (exact cell values)
+  figures/                          # Raw figure data (extracted data points)
+rubric/                             # (Only if rubric provided)
+  requirements.md                   # Leaf-level rubric requirements mapped to ARA files
+```
+
+Additional files or subdirectories may be created on demand when the source contains
+content that does not fit the standard layers (for example, appendix-sourced worked
+examples, prompt templates, or enumerated taxonomies). Place such content in the ARA
+layer where it best belongs.
+
+## Progressive Disclosure (3 Levels)
+
+- **Level 1 — PAPER.md** (~200 tokens): Frontmatter + layer index. Agent reads ONLY this to decide relevance.
+- **Level 2 — Layer files** (problem.md, claims.md, experiments.md, evidence/README.md): Loaded on demand.
+- **Level 3 — Detail files** (algorithm.md, code stubs, individual evidence tables): Loaded when drilling in.
+
+---
+
+## PAPER.md
+
+YAML frontmatter MUST include:
+```yaml
+---
+title: "{full paper title}"
+authors: [{author list}]
+year: {year}
+venue: "{venue}"
+doi: "{DOI or arXiv ID}"
+ara_version: "1.0"
+domain: "{research domain}"
+keywords: [{5-10 keywords}]
+claims_summary:
+  - "{one-line summary of main claim 1}"
+  - "{one-line summary of main claim 2}"
+  - "{one-line summary of main claim 3}"
+abstract: "{paper abstract}"
+---
+```
+
+Body MUST include a Layer Index — a table for each layer listing every file:
+
+```markdown
+# {Paper Title}
+
+## Overview
+{1-2 paragraph summary of the contribution}
+
+## Layer Index
+
+### Cognitive Layer (`/logic`)
+| File | Description |
+|------|-------------|
+| [problem.md](logic/problem.md) | Observations → gaps → key insight |
+| [claims.md](logic/claims.md) | {N} falsifiable claims (C01–C{NN}) |
+| ...
+
+### Physical Layer (`/src`)
+| File | Description | Claims |
+|------|-------------|--------|
+| [execution/{module}.py](src/execution/{module}.py) | {what} | C{NN} |
+| ...
+
+### Exploration Graph (`/trace`)
+| File | Description |
+|------|-------------|
+| [exploration_tree.yaml](trace/exploration_tree.yaml) | {N}-node research DAG |
+
+### Evidence (`/evidence`)
+| File | Description |
+|------|-------------|
+| [README.md](evidence/README.md) | Full index of {N} tables + {N} figures |
+```
+
+---
+
+## Evidence Naming and Fidelity
+
+The evidence layer has two different object types:
+
+1. **Raw source evidence**
+   - Faithful transcription of one source table or figure
+   - Must preserve the original source identifier and caption
+   - Example: `evidence/tables/table3_imagenet_validation.md`
+
+2. **Derived subset evidence**
+   - Filtered or recomposed view created for a specific claim
+   - Must NOT masquerade as the original source object
+   - Filename should include `derived_`, `subset_`, or equivalent
+   - Must declare which raw source object it came from
+   - Example: `evidence/tables/derived_from_table3_residual_depth_slice.md`
+
+Rule: if a filename includes a source label such as `table3` or `figure4`, it should faithfully represent that exact source object rather than a curated subset.
+
+---
+
+## logic/problem.md
+
+```markdown
+# Problem Specification
+
+## Observations
+
+### O{N}: {title}
+- **Statement**: {precise empirical fact with numbers}
+- **Evidence**: {source — figure, table, measurement, citation}
+- **Implication**: {what this means for the problem}
+
+## Gaps
+
+### G{N}: {title}
+- **Statement**: {what's missing or broken}
+- **Caused by**: {which observations, e.g., O1, O2}
+- **Existing attempts**: {what's been tried}
+- **Why they fail**: {specific failure mode}
+
+## Key Insight
+- **Insight**: {the creative leap, stated precisely}
+- **Derived from**: {which observations}
+- **Enables**: {what solution approach this unlocks}
+
+## Assumptions
+- A1: {assumption}
+- A2: {assumption}
+```
+
+---
+
+## logic/claims.md
+
+Each claim MUST have ALL fields:
+```markdown
+## C{NN}: {Short title}
+- **Statement**: {Precise, falsifiable assertion}
+- **Status**: {hypothesis|supported|refuted}
+- **Falsification criteria**: {What would disprove this}
+- **Proof**: [{experiment IDs: E01, E02}]
+- **Evidence basis**: {What the cited evidence directly shows}
+- **Interpretation**: {Optional broader reading that should not be confused with the raw evidence}
+- **Dependencies**: {other claim IDs, if any}
+- **Tags**: {comma-separated keywords}
+```
+
+Proof MUST reference experiment IDs from experiments.md.
+Each proofed experiment should in turn be backed by evidence files whose rows or measurements actually match the claim being asserted.
+`Statement` should stay at the strongest level directly supported by the cited evidence. Use `Interpretation` for broader synthesis.
+
+---
+
+## logic/concepts.md
+
+≥5 concepts. One section per concept:
+```markdown
+## {Term Name}
+- **Notation**: {LaTeX or symbolic notation}
+- **Definition**: {Formal definition}
+- **Boundary conditions**: {When does this concept apply/not apply}
+- **Related concepts**: {other concept names}
+```
+
+---
+
+## logic/experiments.md
+
+≥3 experiments. Declarative plans, NOT scripts. NO exact numerical results.
+
+```markdown
+## E{NN}: {Short title}
+- **Verifies**: {claim IDs, e.g., C01, C02}
+- **Setup**:
+  - Model: {model name and size}
+  - Hardware: {GPU type, count, memory}
+  - Dataset: {dataset name, size, source}
+  - System: {system configuration}
+- **Procedure**:
+  1. {Step 1}
+  2. {Step 2}
+- **Metrics**: {what to measure, with units}
+- **Expected outcome**:
+  - {directional/relative ONLY, e.g., "A outperforms B on metric X"}
+  - NEVER exact numbers (those go in evidence/)
+- **Baselines**: {methods to compare against}
+- **Dependencies**: {other experiment IDs, or "none"}
+```
+
+---
+
+## logic/solution/architecture.md
+
+Component graph. For each component: name, purpose, inputs, outputs, interactions, key design choices.
+
+## logic/solution/algorithm.md
+
+- Mathematical formulation (LaTeX)
+- Pseudocode
+- Step-by-step explanation
+- Complexity analysis
+
+## logic/solution/constraints.md
+
+- Boundary conditions
+- Assumptions
+- Known limitations
+
+## logic/solution/heuristics.md
+
+Each heuristic MUST have ALL fields:
+```markdown
+## H{NN}: {Short description}
+- **Rationale**: {Why this trick is needed}
+- **Sensitivity**: {low|medium|high}
+- **Bounds**: {acceptable range or limits}
+- **Code ref**: [{path to src/execution/ file}]
+- **Source**: {Section/table in the paper}
+```
+
+---
+
+## logic/related_work.md
+
+```markdown
+## RW{NN}: {Author et al., Year}
+- **DOI**: {DOI or arXiv ID}
+- **Type**: {imports|bounds|baseline|extends|refutes}
+- **Delta**:
+  - What changed: {specific technical delta}
+  - Why: {motivation}
+- **Claims affected**: {claim IDs}
+- **Adopted elements**: {what was kept}
+```
+
+Works with a specific technical delta get full `RW` blocks as above. Additional citations
+from the paper that do not have a technical delta (background, historical, infrastructure,
+or inline-comparison references) should still be captured more briefly so the ARA preserves
+the paper's full citation footprint.
+
+---
+
+## src/configs/training.md
+
+```markdown
+## {Parameter name}
+- **Value**: {exact value}
+- **Rationale**: {why this value}
+- **Search range**: {if mentioned}
+- **Sensitivity**: {low|medium|high}
+- **Source**: {section/table}
+```
+
+## src/configs/model.md
+
+Same format as training.md for model/architecture configs.
+
+## src/execution/{module}.py
+
+- Typed function signatures (input/output types, tensor shapes)
+- Docstrings explaining what each function does
+- Implementation logic for the NOVEL contribution
+- NO scaffolding (no argparse, logging, distributed wrappers)
+- Import only standard libraries + torch/numpy
+
+## src/environment.md
+
+```markdown
+# Environment
+- **Python**: {version}
+- **Framework**: {PyTorch version, etc.}
+- **Hardware**: {GPU type, count, memory}
+- **Key dependencies**: {list with versions}
+- **Random seeds**: {if specified}
+```
+
+---
+
+## evidence/tables/{file}.md
+
+Raw source-table transcription:
+
+```markdown
+# Table {N} - {Caption or short description}
+
+**Source**: Table {N} in {paper/report title}
+**Caption**: {verbatim or near-verbatim caption}
+**Extraction type**: raw_table
+
+| ... | ... |
+| --- | --- |
+| ... | ... |
+```
+
+Derived subset:
+
+```markdown
+# Derived subset - {Short description}
+
+**Source**: Derived from Table {N} in {paper/report title}
+**Caption**: {what part of the source table this subset preserves}
+**Extraction type**: derived_subset
+**Derived from**: `table{N}_{raw_file_name}.md`
+
+| ... | ... |
+| --- | --- |
+| ... | ... |
+```
+
+Rules:
+- Raw source-table files should reproduce the original row set relevant to that table, not a claim-specific slice
+- If you drop rows, rename the file as a derived subset and declare the parent source
+- Do not combine rows from multiple source tables while retaining a single original table number in the filename
+
+---
+
+## trace/exploration_tree.yaml
+
+Each node should distinguish direct source support from reconstruction:
+
+```yaml
+tree:
+  - id: N01
+    type: question
+    support_level: explicit | inferred
+    source_refs: ["Table 2", "§4.1"]   # recommended for explicit nodes
+    title: "{...}"
+    description: "{...}"
+```
+
+Rules:
+- `support_level: explicit` means the node is directly grounded in the provided source material
+- `support_level: inferred` means the node is a reconstruction of the paper's logic, not a literal session record
+- Explicit nodes should include `source_refs`
+- Inferred nodes must not be presented as if they were directly observed historical events
+
+---
+
+## evidence/README.md
+
+```markdown
+# Evidence Index
+
+## Tables
+| File | Source | Claims | Description |
+|------|--------|--------|-------------|
+| [tables/{name}.md](tables/{name}.md) | Table N, §X.Y | C01, C02 | {one sentence} |
+
+## Figures
+| File | Source | Claims | Description |
+|------|--------|--------|-------------|
+| [figures/{name}.md](figures/{name}.md) | Figure N, §X.Y | C03 | {one sentence} |
+```
+
+## evidence/tables/{name}.md
+
+ALL result tables, exact cell values:
+```markdown
+# Table N: {Title}
+- **Source**: Table N, Section X.Y
+- **Caption**: "{caption}"
+
+| Column1 | Column2 | ... |
+|---------|---------|-----|
+| exact   | values  | ... |
+```
+
+## evidence/figures/{name}.md
+
+ALL quantitative figures (not diagrams). Extract data points:
+```markdown
+# Figure N: {Title}
+- **Source**: Figure N, Section X.Y
+- **Caption**: "{caption}"
+- **Axes**: X = {label, units}, Y = {label, units}
+
+| X | Y (Series A) | Y (Series B) | ... |
+|---|-------------|-------------|-----|
+| v | v           | v           | ... |
+```
+
+Mark approximate readings with "≈".
+
+---
+
+## Appendix-sourced content
+
+Appendix sections commonly carry worked examples, prompt templates, enumerated taxonomies,
+annotation schemas, extended analyses, and prescriptive content. Route each into the ARA
+layer where it best fits, preserving the granularity the source uses (for example, keep
+per-entry descriptive fields for taxonomies rather than collapsing to names + frequencies).
+The existing layer conventions above apply; create additional files only when no existing
+file is a natural home.
+
+---
+
+## rubric/requirements.md (Only if rubric provided)
+
+```markdown
+# Rubric Requirements — {paper_id}
+
+**Source**: PaperBench expert-authored reproduction rubric
+**Total leaf requirements**: {N}
+
+## {Category Group}
+
+### R{NN}: {Short title}
+- **Rubric ID**: {uuid}
+- **Category**: {task_category} / {finegrained_task_category}
+- **Weight**: {weight}
+- **Requirement**: {verbatim from rubric}
+- **ARA coverage**: {path to most specific ARA file, or "Not covered"}
+- **Key detail**: {exact value from paper, or "Not specified in paper"}
+```

package/skills/compiler/references/exploration-tree-spec.md

@@ -0,0 +1,124 @@
+# Exploration Tree YAML Specification
+
+The exploration tree is the "git log" for research — a structured, traversable record of every
+successful branch, failed attempt, and design decision that shaped the final result.
+
+## Format
+
+```yaml
+# Exploration Tree — {paper_id}
+# Research DAG: nested tree with cross-edges (also_depends_on) forming a DAG.
+# Node types: question | experiment | dead_end | decision | pivot
+
+tree:
+  - id: N01
+    type: question
+    support_level: explicit
+    source_refs: ["§1", "Table 2"]
+    title: "{Central research question}"
+    description: "{What question is being investigated}"
+    children:
+
+      - id: N02
+        type: experiment
+        support_level: explicit
+        source_refs: ["Figure 4", "Table 2"]
+        title: "{What was tried}"
+        result: "{What was observed}"
+        evidence: [C01, "Figure 3", "§2.2"]
+        children:
+
+          - id: N04
+            type: decision
+            support_level: inferred
+            title: "{What was decided}"
+            choice: "{The chosen approach}"
+            alternatives:
+              - "{Alternative 1}"
+              - "{Alternative 2}"
+            evidence: "{What informed this decision}"
+            children:
+              # ... deeper nesting
+
+      - id: N03
+        type: dead_end
+        support_level: inferred
+        title: "{What was tried and failed}"
+        hypothesis: "{What was expected}"
+        failure_mode: "{Why it failed}"
+        lesson: "{What was learned; what it led to}"
+        # dead_end nodes have NO children — they are leaf nodes
+
+  # For DAG edges (node with multiple parents):
+  - id: N10
+    type: experiment
+    support_level: explicit
+    source_refs: ["Table 5"]
+    title: "{Convergent experiment}"
+    also_depends_on: [N07, N08]  # additional parents beyond nesting
+    result: "{What was observed}"
+    evidence: [C05]
+```
+
+## Node Types
+
+### question
+The root driver. What is being investigated?
+- **Required fields**: `description`
+- **Children**: experiments, decisions, other questions
+
+### experiment
+An attempt to answer a question or validate a decision.
+- **Required fields**: `result`
+- **Optional fields**: `evidence` (list of claim IDs, figure/table refs, section refs)
+- **Children**: decisions, dead_ends, more experiments
+
+### dead_end
+A failed approach. THE MOST VALUABLE NODE TYPE for downstream agents.
+- **Required fields**: `hypothesis`, `failure_mode`, `lesson`
+- **NO children** — always a leaf node
+- Dead ends save agents from rediscovering known failures
+
+### decision
+A design choice with documented alternatives.
+- **Required fields**: `choice`, `alternatives`
+- **Optional fields**: `evidence`
+- **Children**: experiments that test the decision, further decisions
+
+### pivot
+A change in research direction.
+- **Required fields**: `from`, `to`, `trigger`
+- **Children**: the new research direction
+
+## Rules
+
+1. **Nested YAML**: Children appear inline under parent node's `children` list
+2. **Valid DAG**: No cycles. All `also_depends_on` IDs must exist in the tree
+3. **Minimum 8 nodes**: Cover the paper's key research trajectory
+4. **Must include dead_end nodes**: At least 1 from ablations or rejected alternatives
+5. **Must include decision nodes**: At least 1 documenting a design choice
+6. **Every node has**: `id` (N01, N02...), `type`, `title`
+7. **Every node has `support_level`**: `explicit` or `inferred`
+8. **Explicit nodes should have `source_refs`**: table/figure/section references from the input material
+9. **`also_depends_on`**: Only for DAG convergence (node has multiple parents beyond nesting)
+
+## Extraction Strategy
+
+When building from a PDF:
+- **Central questions** → root nodes
+- **"We tried X" / "We evaluated Y"** → experiment nodes
+- **"We considered X but chose Y because..."** → decision nodes with alternatives
+- **Ablation results showing X hurts** → dead_end nodes
+- **"We initially pursued X but found..."** → pivot nodes
+- **"This approach fails because..."** → dead_end nodes
+
+Support-level guidance:
+- Mark a node `explicit` only if the paper directly reports it
+- Mark a node `inferred` if you are reconstructing a plausible research decision from the narrative structure
+- Prefer omission over fabricating a highly specific inferred node
+
+When building from experiment logs:
+- Each experiment run → experiment node
+- Failed runs → dead_end nodes with actual error messages as failure_mode
+- Parameter sweeps → decision nodes with sweep results informing the choice
+- Direction changes → pivot nodes with the triggering observation

package/skills/compiler/references/validation-checklist.md

@@ -0,0 +1,148 @@
+# ARA Seal Level 1 — Validation Checklist
+
+These are all checks the Seal validator runs. Fix ALL failures before reporting success.
+
+## 1. Directory Existence
+
+All must exist as directories:
+- `logic/`
+- `logic/solution/`
+- `src/`
+- `src/configs/`
+- `trace/`
+- `evidence/`
+
+## 2. Mandatory File Existence (non-empty)
+
+All must exist with >10 bytes:
+- `PAPER.md`
+- `logic/problem.md`
+- `logic/claims.md`
+- `logic/concepts.md`
+- `logic/experiments.md`
+- `logic/solution/architecture.md`
+- `logic/solution/algorithm.md`
+- `logic/solution/constraints.md`
+- `logic/solution/heuristics.md`
+- `logic/related_work.md`
+- `src/configs/training.md`
+- `src/configs/model.md`
+- `src/environment.md`
+- `trace/exploration_tree.yaml`
+- `evidence/README.md`
+
+## 3. PAPER.md Checks
+
+- Starts with `---` (YAML frontmatter)
+- Frontmatter is valid YAML mapping
+- Contains keys: `title`, `authors`, `year`
+- Body contains "Layer Index" section
+
+## 4. Field-Level Checks (regex patterns)
+
+### logic/claims.md
+- Has `## C\d+` blocks (at least one claim)
+- Contains `**Statement**`
+- Contains `**Status**`
+- Contains `**Falsification criteria**`
+- Contains `**Proof**`
+- Contains `**Evidence basis**`
+- Contains `**Interpretation**`
+
+### logic/problem.md
+- Has `### O\d+` blocks (observations)
+- Has `### G\d+` blocks (gaps)
+- Has Key Insight section (`## Key Insight` or `**Insight**`)
+
+### logic/experiments.md
+- Has `## E\d+` blocks (at least 3)
+- Contains `**Verifies**`
+- Contains `**Setup**`
+- Contains `**Procedure**`
+- Contains `**Expected outcome**` or `**Expected results**`
+
+### logic/solution/heuristics.md
+- Has `## H\d+` blocks
+- Contains `**Rationale**`
+- Contains `**Sensitivity**`
+- Contains `**Bounds**`
+
+### logic/related_work.md
+- Has `## RW\d+` blocks
+- Contains `**Type**`
+- Contains `**Delta**`
+- Coverage should extend beyond the closest predecessors to reflect the paper's full
+  citation footprint
+
+### logic/concepts.md
+- Has `## ` sections (at least 5)
+- Contains `**Definition**`
+
+## 5. Count Checks
+
+- `logic/concepts.md`: ≥5 concept sections (`## ` headers)
+- `logic/experiments.md`: ≥3 experiment blocks (`## E\d+`)
+- `src/execution/`: ≥1 `.py` file
+- `evidence/tables/` or `evidence/figures/`: ≥1 `.md` file
+
+## 5b. Appendix Coverage
+
+When the source has appendices, every appendix section should be traceable to at least
+one ARA file, with the granularity of the source preserved.
+
+## 6. Evidence Quality
+
+For each file in `evidence/tables/*.md` and `evidence/figures/*.md`:
+- Must contain a Markdown table (`|...|...|` pattern)
+- Must contain `**Source**` field
+- If the filename includes `table{N}` or `figure{N}`, the `**Source**` field must reference the same identifier
+- If the file is a derived subset, it must say so explicitly via `**Extraction type**: derived_subset` or equivalent
+- Raw source-table files should not silently omit rows while still presenting themselves as the original table
+
+## 7. evidence/README.md
+
+- Must contain a Markdown table (file index)
+- Numbered tables and figures from the source (main text and appendices) should be
+  reflected in the index
+
+## 8. Exploration Tree (YAML)
+
+- Parses as valid YAML
+- Has top-level `tree` key
+- ≥8 nodes total (counted recursively through children)
+- All node types in {question, decision, experiment, dead_end, pivot}
+- At least 1 `dead_end` node exists
+- At least 1 `decision` node exists
+- Every node has `id` and `type` fields
+- Every node has `support_level` in {explicit, inferred}
+- Type-specific required fields:
+  - question: `description`
+  - experiment: `result`
+  - dead_end: `hypothesis`, `failure_mode`, `lesson`
+  - decision: `choice`, `alternatives`
+  - pivot: `from`, `to`, `trigger`
+- All `also_depends_on` references resolve to existing node IDs
+- Nodes with `support_level: explicit` should include `source_refs`
+
+## 9. Cross-Layer Binding
+
+### Claim Proof → Experiment Resolution
+- Every `E\d+` in a claim's `**Proof**: [...]` must exist in experiments.md
+- Proof-linked experiments should have evidence files whose labels and row contents actually match the compared systems or measurements
+- Claim wording should be auditable against `Evidence basis`; broader language should be isolated to `Interpretation`
+
+### Experiment Verifies → Claim Resolution
+- Every `C\d+` in an experiment's `**Verifies**` must exist in claims.md
+
+### Heuristic Code Ref → File Resolution
+- Every `src/...` path in `**Code ref**: [...]` must be an existing file
+
+### Architecture Components → Code Stubs (fuzzy)
+- Significant words from `## ` headings in architecture.md should appear somewhere in src/execution/ code
+
+### Tree Evidence → Claims (YAML)
+- Any `C\d+` in a tree node's `evidence` field must exist in claims.md
+
+### Trace Hygiene
+- Do not add dead_end, decision, or experiment nodes that are unsupported by the provided source material
+- If a node is reconstructed from partial evidence rather than stated explicitly, it should be marked as inferred or excluded from Seal Level 1 outputs