@interf/compiler 0.1.12 → 0.2.1

package/skills/knowledge-base/summarize/SKILL.md

@@ -27,12 +27,11 @@ Harness role:
 1. Read `interf.json` and confirm `type: knowledge-base`.
 2. Read `.interf/stage-contract.json` and treat it as authoritative.
 3. Read `.interf/summarize-targets.json` when present and treat `targets[]` as the source of truth for which files to summarize.
+   If a target includes `output`, write the summary to that exact relative path.
 4. Read any local docs listed by the stage contract.
 5. Create one summary file in `summaries/` for each target source file.
 6. Write `.interf/inventory.json`.
-7. Update `.interf/state.json`.
-8. Refresh `.interf/health.json`.
-9. Emit `DONE:` and stop.
+7. Emit `DONE:` and stop.
 
 If the CLI provided a summarize plan, do not re-audit the whole knowledge base before writing summaries.
 
@@ -40,14 +39,39 @@ If the CLI provided a summarize plan, do not re-audit the whole knowledge base b
 
 When launched by the Interf CLI:
 - emit exactly one startup line: `STATUS: loaded summarize plan N files`
-- emit `STATUS: batch committed` only after summary files, inventory, and state were actually written
-- emit `DONE: summarized N/N` when the required writes are complete
+- emit `STATUS: batch committed` only after summary files and inventory were actually written
+- emit `DONE: summarize complete N/N` when the required writes are complete
 - do not emit per-file progress
 
+## Execution bias
+
+This stage is a light per-file evidence pass, not a mini research project.
+
+- process only the files listed in `.interf/summarize-targets.json`
+- for a single large PDF, deck, or report, do one lightweight evidence pass and then stop
+- for a single large PDF, deck, or report, do at most one routing pass across the document plus one focused follow-up read that covers no more than 3 routed pages or sections
+- capture the file's scope, headline facts that are directly exposed, and whether important evidence also lives in charts or tables
+- if the document repeats the same template across many pages or entities, sample only enough pages to describe that repeated structure and preserve routing for later interfaces
+- do not chase every page, every market, every appendix, or every historical chart in this stage
+- leave narrow task-specific extraction to interface workflows
+- once you can state the document scope, the exposed headline metrics, and where deeper chart/table evidence lives, write the summary immediately
+- once the summary file and inventory are written, stop immediately
+- keep bulk PDF/OCR extraction output out of the run log. Save scratch output to temp files and inspect only the routed slices you need for the summary.
+- keep scratch extraction commands single-purpose and non-destructive. Do not use `rm`, wildcard cleanup, `;`, or `&&` chains during summarize.
+- if a shell pattern is blocked, do not retry the same chained command with minor edits. Switch to a simpler local read path and keep moving.
+- if you have already checked whether the summary file exists and it does not, the next step is to write it rather than doing another exploratory read
+- do not precreate `.interf/inventory.json` with `touch`; write the final JSON document directly
+- when updating `.interf/inventory.json`, rewrite the full JSON document in one whole-file write instead of appending or patching partial lines
+- do not use `apply_patch` or line-matched diffs on `.interf/inventory.json`; replace the file in one complete write
+- when you are ready to write, prefer a direct whole-file shell write such as `cat > path <<'EOF' ... EOF` for summary markdown and the inventory artifact
+- whole-file shell writes for stage artifacts are allowed and preferred here; do not stall deciding between patch-based editing and direct writes
+
 ## Summary output format
 
 Create one markdown file per source file under `summaries/`.
 Every summary filename must end in `.md`, even when the source file is `.txt`, `.pdf`, or another format.
+If the source file already ends in `.md`, keep a single `.md` suffix instead of appending another one.
+When `.interf/summarize-targets.json` provides `targets[].output`, use that exact path instead of inferring a filename.
 
 Each summary must include JSON frontmatter with:
 - `source`
@@ -85,38 +109,44 @@ Keep summaries source-grounded:
 - keep titles descriptive and conservative
 - preserve evidence tiers and truth modes
 - keep links sparse and obvious
+- for PDFs, decks, and reports, capture material table/chart/figure evidence when it affects the source's meaning
+- preserve the main headline metrics for the top-level sections or market groups when they are present in extractable text
+- preserve easy headline values from prose or clearly exposed tables when they are central to the source
+- for market reports and dashboards, preserve the main peer groups that are trivial to recover from prose or headline tables, but do not turn summarize into a full market-by-market extraction pass
+- if important evidence appears to live in charts or other visuals, say that clearly in the summary and preserve enough routing detail for a later interface or manual query step
+- do not turn the knowledge-base summarize stage into an exhaustive visual extraction pass across the whole report
+- if exact chart numbers are not directly recoverable from the same page with light local extraction, say so explicitly in the summary instead of flattening the evidence into vague prose
+- when useful, note whether important evidence came from prose, tables, or charts so later interface stages know when raw visual inspection may still be needed
+- distinguish `chart present and relevant` from `exact chart values not yet extracted`
 
 This is a per-file evidence stage only. Cross-file synthesis belongs to `knowledge-base/compile`.
 
-## Inventory and state
+## Inventory output
 
 Write `.interf/inventory.json` so every source file is accounted for.
 
 Minimum shape:
+- prefer the current runtime shape: top-level `entries` plus `total`
+- `total` equal to the actual current source-file count
+- one `entries[]` object per source file
+- each entry should preserve at least `source`, `summary`, and a conservative summarize `status` or `state`
+- per-file routing must let the runtime and validators see which raw file maps to which summary
 
-```json
-{
-  "total": 3,
-  "files": [
-    {
-      "raw": "company-overview.md",
-      "summary": "summaries/company overview.md",
-      "status": "summarized"
-    }
-  ]
-}
-```
-
-Update `.interf/state.json` with:
-- `pending`
-- `summarized`
-- `last_summarize`
+Do not copy canned totals or filenames into the runtime inventory.
+Do not invent new top-level inventory keys when `entries` already fits the run.
 
-Refresh `.interf/health.json` after state is updated.
+The CLI reconciles `.interf/state.json` and refreshes `.interf/health.json` after stage validation.
+For a one-file summarize run, the finish sequence is strict: write the summary file, rewrite `.interf/inventory.json`, emit `STATUS: batch committed`, emit the standard summarize `DONE:` line using the current run counts, and stop.
 
 ## Guardrails
 
 - Do not modify source files.
 - Do not emit `STATUS: batch committed` before files exist on disk.
+- Do not create placeholder JSON files with `touch` before writing the final documents.
+- Do not append a second JSON object to `.interf/inventory.json`. Rewrite the whole file once.
+- Do not use `apply_patch` or line-matched diffs to edit `.interf/inventory.json`.
+- Use a single whole-file write for each required artifact as soon as the summary text is ready.
+- If you check whether a required summary or inventory artifact exists and it is missing, create it next instead of doing another exploratory read.
+- Do not keep mining the source after a usable conservative summary already exists.
 - Do not keep browsing after the required writes are complete.
 - If a required path is missing or unreadable, emit one `BLOCKED:` or `ERROR:` line with the concrete reason and stop.

package/templates/interface/README.md

@@ -44,13 +44,12 @@ State bridges between steps. If step 2 fails, the next run resumes from step 2.
 
 ## Workflows
 
-Shipped interface workflows:
-- `briefing`
-- `research`
-- `audit`
+Shipped built-in interface workflow:
+- `interf`
 
 The selected workflow is written to `interf.json` and can seed starter local `workflow/` docs.
-Create new reusable workflows from the wizard when you want a named local variation.
+Use `compile-plan.md` for interface-specific direction such as target entities, years, metrics, chart families, and output focus.
+Create a reusable local workflow only when you want a named method variation you expect to reuse across multiple interfaces.
 
 ## Bundled stage skills
 
@@ -69,17 +68,18 @@ Minimal. Source of truth for type and knowledge-base reference.
 {
   "type": "interface",
   "name": "weekly-briefing",
-  "workflow": "briefing",
+  "workflow": "interf",
   "knowledge_base": {
     "path": "../.."
   }
 }
 ```
 
-`knowledge_base.path` points to the parent knowledge base. All filter criteria, extraction goals, and output specs live in `compile-plan.md`.
-`workflow` records the selected methodology. Interf generates per-run stage contracts from it for this interface.
+`knowledge_base.path` points to the parent knowledge base. All interface-specific filter criteria, extraction goals, and output specs live in `compile-plan.md`.
+`workflow` records the selected reusable method. Interf generates per-run stage contracts from it for this interface.
 
-If you want different retrieval, ontology, or output bias, use a workflow and local `workflow/`.
+If you want to specialize this specific interface, refine `compile-plan.md`.
+If you want a reusable retrieval, ontology, or output method, use a workflow and local `workflow/`.
 If you want a different stage graph, proof model, or required artifacts, that is effectively a new workflow.
 
 `AGENTS.md` is the source of truth. `CLAUDE.md` is a generated mirror so Claude Code picks up the same interface instructions automatically.
@@ -109,7 +109,7 @@ If you need a different stage chain, artifact contract, or verifier flow, create
 If your coding agent is sandboxed to the current directory tree, launch it from the source folder so the source root, knowledge base, and this interface all stay reachable.
 Use `../../.interf/source-access.json` to verify that raw files are actually reachable before you depend on raw fallback in a manual agent session. `suggested_checks` are canonical absolute file paths so they remain valid from inside the interface folder.
 
-If Obsidian viewer defaults are enabled during `interf init`, newly created interfaces get minimal `.obsidian/graph.json` defaults. They are not registered as standalone vaults by default; browse them from the parent knowledge-base vault so knowledge-base summaries and source links remain navigable.
+If Obsidian viewer defaults are enabled during `interf init`, newly created interfaces get minimal `.obsidian/graph.json` defaults. They are not registered as standalone workspaces by default; browse them from the parent knowledge-base workspace in Obsidian so knowledge-base summaries and source links remain navigable.
 
 ## Runtime files
 
@@ -136,7 +136,8 @@ Generated by `interface/create`. Defines:
 - Stage 2: extraction goals and allowed runtime refinements
 - Stage 3: output specs for the local context shell
 
-This is the initial interface definition. Compile can refine it when the evidence clearly demands better local structure.
+This is the interface-specific customization layer for the built-in `interf` workflow unless you choose a reusable local workflow instead.
+Compile can refine it when the evidence clearly demands better local structure.
 
 ## Key principles
 
@@ -153,6 +154,6 @@ This is the initial interface definition. Compile can refine it when the evidenc
 ## Reset
 
 ```text
-interf reset compile remove knowledge/ + briefs/ + summaries/, keep config
+interf reset compile remove knowledge/ + briefs/, keep summaries/ + config
 interf reset all     remove all generated content, preserve config files
 ```

package/templates/interface/interfaces.md

@@ -1,14 +1,17 @@
-# Interface Workflows
+# Interface Patterns
 
-Pre-designed interface workflows. Each defines a use case, guiding questions, filter criteria, output inventory, and benchmark queries for validation.
+Interf currently ships one built-in interface workflow: `interf`.
+That workflow always runs retrieve -> analyze -> compile.
 
-When `interf create interface` builds an interface, it starts from one of these workflows and adapts it to the selected knowledge-base data.
+What changes from interface to interface is the use case, the `compile-plan.md` customization, and any optional local workflow extension docs.
 
-Benchmark queries exist today as validation prompts and expected behaviors. The automated benchmark runner and score logging loop are planned, not yet implemented in the engine.
+Use the patterns below as planning lenses you can encode in `compile-plan.md` or a reusable local interface workflow. They are not built-in workflow ids.
+
+Benchmark queries exist today as validation prompts and expected behaviors. The automated benchmark runner and score logging loop are implemented separately from this shipped template reference.
 
 ---
 
-## briefing
+## Current-State Briefing Lens
 
 Current-state briefing. Best when you want the latest truth, recent changes, and immediate next steps.
 
@@ -34,7 +37,7 @@ Current-state briefing. Best when you want the latest truth, recent changes, and
 
 ---
 
-## research
+## Research Synthesis Lens
 
 Research synthesis. Best when you want themes, contradictions, evidence strength, and open questions from a dense knowledge base.
 
@@ -60,7 +63,7 @@ Research synthesis. Best when you want themes, contradictions, evidence strength
 
 ---
 
-## audit
+## Gap Review Lens
 
 Audit and gap-finding. Best when you want to improve knowledge-base quality, spot stale areas, and surface missing or contradictory evidence.
 
@@ -84,12 +87,12 @@ Audit and gap-finding. Best when you want to improve knowledge-base quality, spo
 
 ---
 
-## Self-improving loop (planned runner)
+## Self-improving Loop
 
-Every interface workflow includes benchmark queries. The intended validation cycle is:
+Every interface can carry benchmark queries. The intended validation cycle is:
 
-1. Build the interface from its config
-2. Run benchmark queries against the vault
+1. Build the interface from its config and current compile plan
+2. Run benchmark queries against the compiled interface workspace
 3. Score: did each query return expected behavior?
 4. If score is below threshold:
    - Analyze failures

package/templates/knowledge-base/README.md

@@ -133,6 +133,5 @@ See `docs/runtime-contract.md` for the human-readable contract and `src/lib/sche
 
 ```text
 interf reset compile   remove knowledge/, keep summaries/
-interf reset summarize remove summaries/
 interf reset all       remove all generated content, preserve source files
 ```

package/templates/knowledge-base/registry.md

@@ -9,8 +9,8 @@ The registry at `~/.interf/registry.json` tracks all knowledge bases and interfa
   "version": 5,
   "knowledgeBases": [
     {
-      "name": "research",
-      "path": "/path/to/research",
+      "name": "market-intel",
+      "path": "/path/to/market-intel",
       "workflow": "interf",
       "created": "2026-03-30T14:00:00Z"
     },
@@ -23,19 +23,19 @@ The registry at `~/.interf/registry.json` tracks all knowledge bases and interfa
   ],
   "interfaces": [
     {
-      "name": "weekly-briefing",
-      "path": "/path/to/weekly-briefing",
+      "name": "q2-launch-brief",
+      "path": "/path/to/q2-launch-brief",
       "knowledgeBase": "ops",
       "knowledgeBasePath": "/path/to/ops",
-      "workflow": "briefing",
+      "workflow": "interf",
       "created": "2026-03-30T16:00:00Z"
     },
     {
-      "name": "gap-audit",
-      "path": "/path/to/gap-audit",
-      "knowledgeBase": "research",
-      "knowledgeBasePath": "/path/to/research",
-      "workflow": "audit",
+      "name": "pricing-gap-review",
+      "path": "/path/to/pricing-gap-review",
+      "knowledgeBase": "market-intel",
+      "knowledgeBasePath": "/path/to/market-intel",
+      "workflow": "interf",
       "created": "2026-03-30T16:30:00Z"
     }
   ]
@@ -55,7 +55,7 @@ The registry at `~/.interf/registry.json` tracks all knowledge bases and interfa
 - `path` — absolute path to interface root
 - `knowledgeBase` — name of the connected knowledge base
 - `knowledgeBasePath` — absolute path to the connected knowledge base
-- `workflow` — selected methodology
+- `workflow` — selected methodology (`interf` for the shipped default, or a local workflow id)
 - `created` — ISO timestamp
 
 ## When it's updated
@@ -96,13 +96,13 @@ $ interf list
 
   Knowledge Bases
   ─────────────────────────────────
-  research    1,240 files    compiled
-  ops           892 files    3 pending
+  market-intel  1,240 files    compiled
+  ops             892 files    3 pending
 
   Interfaces
   ─────────────────────────────────
-  weekly-briefing   → ops         compiled
-  gap-audit         → research    stale (12 new)
+  q2-launch-brief      → ops            compiled
+  pricing-gap-review   → market-intel   stale (12 new)
 ```
 
 ## Creation

package/templates/workflow-package/README.md

@@ -0,0 +1,16 @@
+# {{workflow_label}}
+
+This is the local {{target_type}} workflow package. Interf uses it when you run `interf compile`.
+
+- `workflow.json` defines the stage graph and contract kinds for this workspace.
+- `create/` explains what the scaffold/setup step produces.
+- `compile/stages/` contains one stage folder per compile stage.
+- `use/query/` explains how to use the compiled knowledge base or interface in manual agent sessions.
+
+Workflow id: `{{workflow_id}}`
+Method: {{workflow_hint}}
+
+Compile stages:
+{{stage_lines}}
+
+Interf CLI remains the referee for contracts, proofs, validators, and runtime state under `.interf/`.

package/templates/workflow-package/create/SKILL.md

@@ -0,0 +1,8 @@
+# Create
+
+This doc explains what Interf scaffolded for the selected {{target_type}} workflow.
+
+- selected workflow: `{{workflow_id}}`
+- method: {{workflow_hint}}
+- the CLI owns create-time validation, folder creation, config writing, and bootstrap docs
+- edit this workflow package after scaffold if you want to refine how later compile/query behavior works

package/templates/workflow-package/interface-query/SKILL.md

@@ -0,0 +1,29 @@
+# Interface Query
+
+{{top_preflight}}
+Use this skill when answering questions from inside this interface in a manual agent session.
+AGENTS.md should route manual questions here before an agent starts browsing notes on its own.
+Treat this as the interface-local query layer, not a replacement for the parent knowledge-base query loop.
+This is a workspace-local query doc, not a required globally installed slash-command skill.
+
+Default loop:
+- start with `AGENTS.md`, then `home.md`
+- browse local `knowledge/`, `briefs/`, and local `summaries/` first
+- answer direct target questions from the hero brief before widening scope
+- follow wikilinks and backlinks outward before treating one note as complete
+- if local interface artifacts are insufficient, load and follow the parent knowledge-base query guidance at `../../workflow/use/query/SKILL.md` before going raw
+- use parent knowledge-base artifacts in this order: `../../home.md`, then `../../knowledge/`, then `../../summaries/`
+- when a caller asks for a trace, audit log, or `artifacts_consulted` list, record local `workflow/use/query/SKILL.md` after you inspect it
+- when you climb to the parent knowledge-base query loop, record `../../workflow/use/query/SKILL.md` after you inspect it
+
+First-reply rule:
+{{first_reply_rule}}
+
+Task bias:
+- reuse the exact metric and period words from the question
+- if the target ledger marks a value approximate, say the literal word `approximate`
+- if the target ledger marks a value chart-derived or table-derived, say that literally
+- prefer the target ledger's `bin_choice` when the question is bin-shaped, and the `value_display` when the question asks for the approximate reading itself
+- do not add raw-access commentary unless the question asks about raw access
+
+Keep the interface-specific routing here. Keep raw-source, PDF/chart, and general fallback policy in `../../workflow/use/query/SKILL.md`.

package/templates/workflow-package/interface-stage/SKILL.md

@@ -0,0 +1,13 @@
+---
+{
+  "mode": "extend"
+}
+---
+# {{stage_label}}
+
+This workflow stage customizes the built-in {{contract_type}} contract for the `{{workflow_id}}` interface workflow.
+
+- purpose: {{stage_description}}
+{{stage_notes}}
+
+Interf still enforces the same contract reads, writes, coverage proof rules, validators, and runtime reconciliation for this stage.

package/templates/workflow-package/knowledge-base-query/SKILL.md

@@ -0,0 +1,36 @@
+# Knowledge Base Query
+
+{{top_preflight}}
+Use this skill when answering questions from inside this knowledge base in a manual agent session.
+AGENTS.md should route manual questions here before an agent starts browsing notes on its own.
+This is the canonical parent-knowledge-base query loop that attached interfaces can inherit before raw fallback.
+This is a workspace-local query doc, not a required globally installed slash-command skill.
+
+Default loop:
+- start with `AGENTS.md`, then `home.md`
+- browse `knowledge/` indexes, entities, and claims first
+- follow wikilinks and backlinks outward before treating one note as complete
+- use `summaries/` for deeper evidence
+- use raw source files only when needed for verification, direct quotes, ambiguity, or missing context
+- when a caller asks for a trace, audit log, or `artifacts_consulted` list, record `workflow/use/query/SKILL.md` after you inspect it
+
+Raw-source gate:
+- inspect `.interf/source-access.json` before depending on raw files
+- actually open and inspect at least one `suggested_checks` path before claiming raw-file verification; a plain existence check is not enough
+- if raw files are not reachable, say the answer is based on compiled knowledge-base artifacts only
+- when the question depends on exact figures, direct quotes, chart values, table values, or image-derived evidence, raw inspection is required before answering confidently
+
+PDF / chart rule:
+- if the source is a PDF, deck, or report and the needed evidence may live in charts, tables, or figures, do not assume the default text layer or existing notes captured everything
+- if summaries do not preserve the needed numeric detail, inspect the raw source and say whether the value was text-derived, table-derived, or chart-derived
+- opening the PDF with `Read` is not enough for an exact-value chart question; take a machine-readable extraction path before answering
+- choose that extraction path from the local environment you actually have instead of assuming specific tools
+- verify the chosen local extraction path is actually available before depending on it; if it is missing, switch once to another available local path
+- if the first local extraction method misses the number, attempt another local recovery path before settling for a visual estimate
+- if the chart does not expose exact numbers as text, say that clearly instead of pretending the compiled notes were sufficient
+- if you can only estimate from the chart, label the answer `approximate` and keep the precision gap explicit
+
+First-reply rule:
+{{first_reply_rule}}
+
+You can edit this file to bias manual question-answering behavior without changing workflow stage logic.

package/templates/workflow-package/knowledge-base-stage/SKILL.md

@@ -0,0 +1,13 @@
+---
+{
+  "mode": "extend"
+}
+---
+# {{stage_label}}
+
+This workflow stage customizes the built-in {{contract_type}} contract for the `{{workflow_id}}` knowledge-base workflow.
+
+- purpose: {{stage_description}}
+{{stage_notes}}
+
+Interf still enforces the same contract reads, writes, proof rules, validators, and runtime reconciliation for this stage.

package/templates/workflow-starters/interface/interf/README.md

@@ -0,0 +1,13 @@
+---
+{
+  "mode": "extend"
+}
+---
+# Interf Interface Workflow
+
+Bias this interface toward the concrete user task, durable evidence routing, and answer-ready outputs.
+
+- turn the requested task into a bounded target set before broadening scope
+- preserve named entities, periods, metrics, units, and comparators instead of collapsing them into generic prose
+- when exact values matter, keep text-derived, table-derived, and chart-derived evidence distinct
+- prefer one concise hero brief and the smallest supporting note set that lets weaker agents answer reliably

package/templates/workflow-starters/interface/interf/create/SKILL.md

@@ -0,0 +1,15 @@
+---
+{
+  "mode": "extend"
+}
+---
+# Interf Create Bias
+
+Turn the user's requested task into a concrete compile plan the later stages can actually execute.
+
+- treat `compile-plan.md` as the primary durable output of create; leave scaffolded `AGENTS.md` alone unless it is materially wrong
+- preserve the exact requested use-case line in `compile-plan.md` instead of leaving the generic workflow placeholder
+- carry named markets, years, metrics, units, comparators, and source families verbatim into the target ledger
+- if the task depends on exact values from reports, PDFs, charts, or tables, require a concrete local extraction path instead of vague raw-reading language
+- keep the interface bounded; do not widen into a general report summary unless a comparator or calibration anchor is genuinely needed
+- for automated runs, emit these canonical status markers verbatim and on their own lines: `STATUS: loaded interface creation contract.`, `STATUS: analyzed knowledge-base.`, `STATUS: wrote compile plan.`

package/templates/workflow-starters/knowledge-base/interf/README.md

@@ -0,0 +1,13 @@
+---
+{
+  "mode": "extend"
+}
+---
+# Interf Knowledge Base Workflow
+
+Bias this knowledge base toward source-grounded summaries, explicit evidence tiers, and retrieval-ready compile outputs.
+
+- keep summary titles and abstracts conservative and source-grounded
+- preserve evidence tiers, truth modes, and draft-vs-final distinctions
+- compile durable entities, claims, indexes, and navigation that make later interfaces easier to retrieve from
+- prefer structures that help agents prove what they scanned, selected, and excluded before synthesis

package/templates/workflow-starters/knowledge-base/karpathy/README.md

@@ -0,0 +1,13 @@
+---
+{
+  "mode": "extend"
+}
+---
+# Karpathy Knowledge Base Workflow
+
+Bias this knowledge base toward the source folder -> summaries -> compiled markdown knowledge-base loop.
+
+- keep per-source summaries legible and source-grounded
+- prefer markdown-native knowledge artifacts an agent can browse and file back into the base
+- preserve backlinks and local browseability across the knowledge layer
+- treat the compiled base as a maintained working surface for later questions, not just a one-shot export