@interf/compiler 0.6.1 → 0.6.3

package/README.md

@@ -1,48 +1,86 @@
 # Interf
 
-Interf helps make data ready for agent work.
+Interf prepares task context for your agents.
 
-Measure how well your agent can answer the questions your task depends on. If raw files are not enough, Interf can prepare a compiled dataset and test it again on the same saved checks.
+Start with the files behind a task and a few checks. Interf tests the files as-is first, shows what already works, and builds a local context folder when the current context is not good enough.
 
-Start with checks. Give Interf the files behind a task and a few saved checks. It shows how well local agents do on the raw files first, so you can see what already works and which agent performs best.
+The result is a real folder your agent can open and work from. Interf retests it on the same checks, and self-improving loops can revise the workflow and try again until it passes or reaches the loop limit.
 
-The compiled dataset is a real folder your agent can open and work from. If the first workflow still fails, Interf can retry the same workflow variation or edit the workflow across self-improving loops within the configured budgets and test each new variation on the same saved checks.
+Raw files stay the source of truth. The context folder adds task-specific structure on top of them.
 
-Interf runs local workflows that prepare and structure the files for the task before your agent uses them.
+```bash
+cd ~/my-project
+interf
+interf test
+interf compile
+interf test
+```
+
+The context folder Interf builds is rooted at:
+
+```text
+interf/<dataset>/
+  AGENTS.md
+  CLAUDE.md
+  raw/
+  <workflow-declared compiled outputs>
+  .interf/
+    interf.json
+    workflow/
+    runtime/
+    tests/
+```
 
-A recent run in this repo on the CBRE chart sanity dataset produced:
+A recent run in this repo on the CBRE chart sanity task produced:
 
 <!-- PUBLIC_BENCHMARK_TABLE:START -->
-| Agent | Files as-is | Compiled dataset |
+| Agent | Files as-is | Context folder |
 | --- | --- | --- |
 | Codex (GPT-5.4, xhigh) | `2/2` | `2/2` |
 | Claude Code (Claude Opus 4.6, max) | `0/2` | `2/2` |
 <!-- PUBLIC_BENCHMARK_TABLE:END -->
 
-Same task. Same saved checks. Same local setup.
+Those scores came from checks like:
+
+- `What were Bristol's annual take-up values in 2018 and 2016?`
+- `What were Bristol's availability values in 2018 and 2016?`
+
+Same task. Same checks. Same local setup.
+
+## How It Works
+
+1. Start with a task and the files relevant to it.
+2. Save a few checks that tell Interf what "ready" means for that task.
+3. Run a workflow that prepares a source-backed context folder for the task.
+4. Test the result on the same checks and keep it only if it helped.
+
+The raw baseline is still important. It shows what already works on the files as-is and which local agent performs best before you spend time improving anything.
 
 ## Why It Exists
 
-Local agents do well when the working surface is already shaped for the job. Raw task data usually is not. Reports, decks, transcripts, exports, PDFs, notes, and mixed folders are all technically available, but they are not prepared in a way that makes local agent work reliable.
+Local agents can open raw files and still reason badly over them. Reports, decks, transcripts, exports, PDFs, notes, and mixed folders may all be present, but they are often missing the structure a real task needs.
+
+Interf takes the files behind a task, prepares them into a local context folder your agents can use, and then proves whether the result helped.
 
 Interf keeps the loop honest:
 
-- `Measure first`: test files as-is on saved truth checks before claiming improvement.
-- `One output`: keep one compiled dataset your agent can use directly.
-- `File-native`: the compiled dataset is a real folder you can inspect, diff, review, and version.
+- `Value first`: prepare the data for a specific task, not just dump files into another app.
+- `Proof-backed`: test raw files and the prepared result on the same checks.
+- `One folder`: the context folder is a real folder you can inspect, diff, review, and version.
 - `Source-backed`: raw files remain the source of truth.
-- `Bring your own agent`: use Claude Code, Codex, or another local agent on the same compiled dataset.
-- `Self-improving`: when truth checks fail, Interf can inspect the failed run, edit the workflow, and retest new workflow variations on the same checks. Retries keep the same workflow variation; loops change the workflow itself.
+- `Bring your own agent`: use the context folder manually from other local agents too. Automated Interf runs currently support Claude Code and Codex.
+- `Self-improving`: when checks fail, Interf can inspect the failed run, edit the workflow, and retest new workflow variations on the same checks. Retries keep the same workflow variation; loops change the workflow itself.
 
 ## What You Get
 
-A project using one dataset stays simple:
+A project with one saved setup stays simple:
 
 ```text
-your-task-folder/
-  report.pdf
-  notes.md
-  exports/
+your-project/
+  task-files/
+    report.pdf
+    notes.md
+    exports/
   interf.json
   interf/
     <dataset>/
@@ -50,22 +88,7 @@ your-task-folder/
       <dataset>/
 ```
 
-A compiled dataset looks like:
-
-```text
-interf/<dataset>/
-  AGENTS.md
-  CLAUDE.md
-  raw/
-  <workflow-declared compiled outputs>
-  .interf/
-    interf.json
-    workflow/
-    runtime/
-    tests/
-```
-
-The compiled dataset is the folder your agent should work from.
+The context folder is the folder your agents should work from after Interf prepares the data for the task.
 
 ## Quick Start
 
@@ -80,23 +103,17 @@ Install:
 npm install -g @interf/compiler
 ```
 
-Start from a folder that already contains the files for the task:
-
-```bash
-cd ~/my-task-folder
-interf
-interf test
-interf compile
-interf test
-```
+Start from a project folder that contains one or more source folders for the task.
 
 The first run can:
 
 - draft `interf.json`
-- help you create truth checks for the task, or let you add them manually
-- run a files-as-is baseline on those checks
-- build the compiled dataset
-- let you test the compiled dataset on the same checks
+- let you choose the source folder for the task
+- default the dataset id from that folder name, then let you change it
+- help you create checks for the task, or let you add them manually
+- run a files-as-is baseline on those checks so you can see what already works
+- build the context folder that stores the compiled context
+- let you test the context folder on the same checks
 
 If Interf cannot find your local executor setup, run:
 
@@ -104,11 +121,13 @@ If Interf cannot find your local executor setup, run:
 interf doctor --live
 ```
 
-## Truth Checks
+## Checks
+
+Checks are the questions and expected answers Interf uses to prove whether the prepared result is good enough for this task.
 
-Truth checks are just question-and-answer pairs you already know how to verify from the files behind the task.
+They are just question-and-answer pairs you already know how to verify from the files behind the task.
 
-Good first truth checks are small and practical:
+Good first checks are small and practical:
 
 - one exact number from a chart, table, or filing
 - one short statement that should be true or false
@@ -122,6 +141,7 @@ A maintained public test run in this repo uses checks like this:
   "datasets": [
     {
       "name": "cbre-chart-sanity",
+      "path": "./task-files",
       "about": "Bristol historical take-up and availability chart lookup.",
       "checks": [
         {
@@ -141,33 +161,33 @@ A maintained public test run in this repo uses checks like this:
 
 ## What `interf test` Proves
 
-`interf test` compares files as-is and the compiled dataset on the same saved truth checks.
+`interf test` compares files as-is and the context folder on the same checks.
 
 That comparison is the product record:
 
 - same task
-- same saved checks
+- same checks
 - same local agent setup
 - raw files on one side
-- compiled dataset on the other
+- compiled context on the other
 
 Use `interf test` on your own files instead of trusting a frozen benchmark snapshot in the docs.
 
-Interf saves the latest comparison plus detailed raw and compiled runs under `interf/tests/` in the same folder.
+Interf saves the latest comparison plus detailed raw-side and context-folder runs under `interf/tests/` in the same folder.
 
 ## The Core Loop
 
-1. Save a few truth checks for the task in `interf.json`.
-2. Optionally run `interf test` to measure files as-is first.
-3. Run `interf compile` to build the compiled dataset.
-4. Run `interf test` again on the same saved checks.
+1. Save a few checks for the task in `interf.json`.
+2. Optionally run `interf test` to see how the raw files do first.
+3. Run `interf compile` to prepare the data into a context folder.
+4. Run `interf test` again on the same checks.
 5. If loops are enabled, let Interf retry the same workflow variation or edit the workflow and test the new variation.
 
-The project root owns dataset setup and saved truth checks. The compiled dataset carries the local runtime copy needed for repeat runs.
+The project root stores the source-folder setup and checks. The context folder carries the local compiled context needed for repeat runs.
 
 ## Workflows
 
-A workflow tells Interf how to prepare the files for this task.
+A workflow tells Interf how to prepare and structure the files for a task before your agent uses them.
 
 Interf ships with a built-in `interf` workflow for the common case. If you need a different method, create one locally:
 
@@ -180,7 +200,7 @@ Workflow creation supports two paths:
 - draft a workflow from the current project with a local agent
 - copy an existing workflow and edit stage guidance directly
 
-After assignment, compile the dataset and run `interf test` on the same truth checks.
+After assignment, build the context folder with `interf compile` and run `interf test` on the same checks.
 
 ## Compile Loops
 
@@ -188,9 +208,9 @@ After assignment, compile the dataset and run `interf test` on the same truth ch
 
 Retries keep the target fixed:
 
-- same dataset
+- same configured dataset
 - same workflow variation
-- same truth checks
+- same checks
 - same measurement
 
 `max_loops` enables self-improving workflow edits in the normal `interf compile` path.
@@ -199,13 +219,13 @@ In that loop, the thing that changes is the workflow itself.
 
 Each loop can:
 
-- run the current workflow variation on the dataset
-- test it on the same truth checks
+- run the current workflow variation on the same configured dataset
+- test it on the same checks
 - inspect failed traces, preserved stage shells, and test artifacts
 - edit the workflow
 - build and test the next workflow variation
 
-Interf preserves workflow-improvement shells, workflow-before / workflow-after snapshots, failed stage shells, and saved test runs from each loop so you can inspect exactly what changed.
+Interf preserves workflow-improvement shells, workflow-before / workflow-after snapshots, failed stage shells, and test runs from each loop so you can inspect exactly what changed.
 
 ## Use It With Your Agent
 
@@ -216,19 +236,19 @@ Paste something like this into your agent:
 ```text
 Install `@interf/compiler`, run `interf` in this folder, and use the local agent executor.
 
-If `interf.json` is missing, draft one dataset entry for this task with a few truth checks this agent should be able to answer from the selected files and add the expected answers for me to confirm.
+If `interf.json` is missing, draft one dataset entry for this task with a few checks this agent should be able to answer from the selected files and add the expected answers for me to confirm.
 
-Then run a files-as-is baseline if helpful, compile the dataset, and run `interf test`.
+Then run a files-as-is baseline if helpful, compile the context folder to prepare the data for that task, and run `interf test`.
 
-Tell me whether the compiled dataset passes the truth checks, and only recommend it if it does.
+Tell me whether the context folder passes the checks, and only recommend it if it does.
 ```
 
 ## More Docs
 
-- [docs/architecture.md](./docs/architecture.md) for the short system map
-- [docs/interf-primitives.md](./docs/interf-primitives.md) for runtime and workflow concepts
-- [docs/workflow-spec.md](./docs/workflow-spec.md) for the workflow model
-- [docs/runtime-contract.md](./docs/runtime-contract.md) for the runtime execution ABI
+- [src/packages/README.md](./src/packages/README.md) for the short system map
+- [src/packages/compiler/PACKAGE.md](./src/packages/compiler/PACKAGE.md) for compiler primitives and the runtime ABI
+- [src/packages/workflow-package/PACKAGE.md](./src/packages/workflow-package/PACKAGE.md) for the workflow package model
+- [src/packages/workflow-authoring/PACKAGE.md](./src/packages/workflow-authoring/PACKAGE.md) for workflow authoring and self-improving loops
 
 Maintainers should use [CONTRIBUTING.md](./CONTRIBUTING.md) for test and release gates.
 

package/builtin-workflows/interf/README.md

@@ -1,11 +1,11 @@
 # Interf (Built-in Workflow)
 
-Interf's built-in workflow: summarize source-grounded evidence, structure the cross-file layer, then shape the final compiled dataset around its task focus and saved truth checks.
+Interf's built-in workflow: summarize source-grounded evidence, structure the cross-file layer, then shape the final context folder around its task focus and checks.
 
 ## Purpose
 
-- General agent-ready dataset preparation
-- Prepare mixed raw files into evidence-backed summaries, cross-file structure, and a usable entrypoint for agents answering the questions this task depends on.
+- General compiled context workflow for agent use
+- Compile mixed raw files into evidence-backed summaries, cross-file structure, and a usable entrypoint for agents answering the questions this task depends on.
 
 ## Zones
 
@@ -21,11 +21,11 @@ Interf's built-in workflow: summarize source-grounded evidence, structure the cr
 
 - `summarize` — Turn source files into per-file summaries. (compiled-file-evidence; reads: raw, runtime; writes: summaries)
 - `structure` — Build the cross-file knowledge structure from the summaries. (compiled-knowledge-structure; reads: summaries, runtime; writes: knowledge-entities, knowledge-claims, knowledge-indexes)
-- `shape` — Shape the final compiled dataset around the saved task focus and truth checks. (compiled-query-shape; reads: raw, summaries, knowledge-entities, knowledge-claims, knowledge-indexes, runtime; writes: knowledge-indexes, home)
+- `shape` — Shape the final context folder around the saved task focus and checks. (compiled-query-shape; reads: raw, summaries, knowledge-entities, knowledge-claims, knowledge-indexes, runtime; writes: knowledge-indexes, home)
 
 ## Why `home.md` exists here
 
-This built-in workflow creates `home.md` as the main agent entrypoint for the compiled dataset.
+This built-in workflow creates `home.md` as the main agent entrypoint for the context folder.
 That is behavior of the `interf` workflow, not a compiler-wide rule.
 
 This package is the built-in seed for `interf`.

package/builtin-workflows/interf/compile/stages/shape/SKILL.md

@@ -1,6 +1,6 @@
 # Shape
 
-Shape the final compiled dataset around the saved task focus and truth checks.
+Shape the final context folder around the saved task focus and checks.
 
 Contract type: `compiled-query-shape`
 
@@ -18,12 +18,12 @@ Contract type: `compiled-query-shape`
 - If a mark touches or nearly touches a labeled gridline, anchor the answer at that gridline or the immediately adjacent half-band.
 - Do not widen a chart-derived range across multiple visible bands unless the chart genuinely supports that uncertainty.
 - When you settle on a bounded chart read, keep that same band consistent across `home.md`, focused indexes, and claim/entity notes for that metric and year.
-- Do not copy expected answers into the compiled dataset.
+- Do not copy expected answers into the context folder.
 
 ## Notes
 
-- Use the saved task focus and truth checks to bias the final compiled dataset toward the job it should be especially good at.
-- Do not copy expected answers into the final compiled dataset just because the checks imply them.
+- Use the saved task focus and checks to bias the final context folder toward the job it should be especially good at.
+- Do not copy expected answers into the final context folder just because the checks imply them.
 - Prefer the saved summary evidence and structured notes when they already preserve the bounded chart/table reads plus provenance you need.
 - Reopen `raw/` during shaping only when the compiled layer is missing the needed value, the metric family is ambiguous, or the earlier bounded read is clearly inconsistent.
 - If a saved truth check depends on chart-derived or table-derived values, carry the final bounded reads forward into focused notes with provenance instead of repeatedly recomputing them from raw.

package/builtin-workflows/interf/improve/SKILL.md

@@ -8,11 +8,11 @@ The improver edits this local package directly.
 Default loop:
 1. Read the loop context first.
 2. Review preserved stage shells, runtime logs, and saved test runs from failed attempts.
-3. Edit only the local workflow package for this compiled dataset to create a better workflow variation for this dataset.
+3. Edit only the local workflow package for this context folder to create a better workflow variation for this task.
 4. Keep `workflow.json`, `workflow.schema.json`, and any changed stage docs aligned.
 
 Guardrails:
-- do not edit truth checks, test specs, or raw dataset files
+- do not edit checks, test specs, or raw source files
 - do not hardcode expected answers into workflow docs
 - keep this package standalone; do not add or rely on runtime `extends` or fallback behavior
 - prefer small, defensible workflow changes over random churn

package/builtin-workflows/interf/use/query/SKILL.md

@@ -25,4 +25,4 @@ Answering rule:
 - if multiple compiled notes mention the same chart read, keep the answer consistent with the most focused compiled note rather than synthesizing a new midpoint or shifted band
 - when the compiled layer is insufficient, verify in `raw/` and then answer
 
-You can edit this file to bias manual question-answering behavior for this compiled dataset.
+You can edit this file to bias manual question-answering behavior for this context folder.

package/builtin-workflows/interf/workflow.json

@@ -6,11 +6,11 @@
     "version": 1
   },
   "purpose": {
-    "label": "General agent-ready dataset preparation",
-    "task_hint": "Prepare mixed raw files into evidence-backed summaries, cross-file structure, and a usable entrypoint for agents answering the questions this task depends on."
+    "label": "General compiled context workflow for agent use",
+    "task_hint": "Compile mixed raw files into evidence-backed summaries, cross-file structure, and a usable entrypoint for agents answering the questions this task depends on."
   },
   "label": "Interf (Built-in)",
-  "hint": "Interf's built-in workflow: summarize source-grounded evidence, structure the cross-file layer, then shape the final compiled dataset around its task focus and saved truth checks.",
+  "hint": "Interf's built-in workflow: summarize source-grounded evidence, structure the cross-file layer, then shape the final context folder around its task focus and checks.",
   "stages": [
     {
       "id": "summarize",
@@ -82,7 +82,7 @@
     {
       "id": "shape",
       "label": "Shape",
-      "description": "Shape the final compiled dataset around the saved task focus and truth checks.",
+      "description": "Shape the final context folder around the saved task focus and checks.",
       "contract_type": "compiled-query-shape",
       "skill_dir": "shape",
       "reads": [
@@ -135,8 +135,8 @@
       "For summary references, prefer explicit links like `[[summaries/<file-stem>]]` or plain code paths. For knowledge notes, prefer the final filename stem under `knowledge/`."
     ],
     "shape": [
-      "Use the saved task focus and truth checks to bias the final compiled dataset toward the job it should be especially good at.",
-      "Do not copy expected answers into the final compiled dataset just because the checks imply them.",
+      "Use the saved task focus and checks to bias the final context folder toward the job it should be especially good at.",
+      "Do not copy expected answers into the final context folder just because the checks imply them.",
       "If a saved truth check depends on chart-derived or table-derived values, verify the needed evidence in `raw/` while shaping and write focused notes that preserve bounded values plus provenance.",
       "Prefer better routing, prioritization, and focused navigation over speculative synthesis.",
       "Any wikilinks you add to `home.md` or indexes must resolve to real compiled note basenames or explicit relative paths."

package/builtin-workflows/interf/workflow.schema.json

@@ -67,7 +67,7 @@
       "owned_by": [
         "shape"
       ],
-      "description": "Primary entrypoint note for agents using the compiled dataset."
+      "description": "Primary entrypoint note for agents using the context folder."
     },
     {
       "id": "runtime",

package/dist/bin.js

@@ -1,29 +1,3 @@
 #!/usr/bin/env node
-import yargs from "yargs";
-import { hideBin } from "yargs/helpers";
-import { initCommand } from "./commands/init.js";
-import { createCommand } from "./commands/create.js";
-import { compileCommand } from "./commands/compile.js";
-import { testCommand } from "./commands/test.js";
-import { doctorCommand } from "./commands/doctor.js";
-import { listCommand } from "./commands/list.js";
-import { statusCommand } from "./commands/status.js";
-import { resetCommand } from "./commands/reset.js";
-import { defaultCommand } from "./commands/default.js";
-import { verifyCommand } from "./commands/verify.js";
-yargs(hideBin(process.argv))
-    .scriptName("interf")
-    .command(defaultCommand)
-    .command(initCommand)
-    .command(compileCommand)
-    .command(testCommand)
-    .command(createCommand)
-    .command(doctorCommand)
-    .command(listCommand)
-    .command(statusCommand)
-    .command(verifyCommand)
-    .command(resetCommand)
-    .strict()
-    .help()
-    .version()
-    .parse();
+import { runCli } from "./cli/index.js";
+await runCli();

package/dist/{commands → cli/commands}/check-draft.d.ts

@@ -1,5 +1,5 @@
-import type { WorkflowExecutor } from "../lib/executors.js";
-import type { SourceTruthCheck } from "../lib/schema.js";
+import type { WorkflowExecutor } from "../../packages/agents/lib/executors.js";
+import type { SourceTruthCheck } from "../../packages/project-model/lib/schema.js";
 export declare function buildTruthCheckDraftPrompt(options: {
     datasetName: string;
     datasetPath: string;

package/dist/{commands → cli/commands}/check-draft.js

@@ -17,22 +17,22 @@ export function buildTruthCheckDraftPrompt(options) {
         ]
         : [
             "No primary task was provided.",
-            "Prefer broad, representative checks that reflect the most useful questions a human would verify from this dataset.",
+            "Prefer broad, representative checks that reflect the most useful questions a human would verify from this source folder.",
         ];
     return [
-        "You are drafting Interf truth checks for a local dataset.",
+        "You are drafting Interf checks for a local source folder.",
         "Read the files in this folder and draft a small set of checks that a human can verify from the source material.",
         "Do not ask follow-up questions.",
-        "Keep the checks concrete, explicit, and useful for measuring whether an agent can reason correctly over the data in this dataset.",
+        "Keep the checks concrete, explicit, and useful for measuring whether an agent can reason correctly over the files in this source folder.",
         "",
-        `Dataset id: ${options.datasetName}`,
-        `Dataset folder: ${options.datasetPath}`,
+        `Setup id: ${options.datasetName}`,
+        `Source folder: ${options.datasetPath}`,
         ...taskLines,
         "",
         `Write exactly one JSON array to ${JSON.stringify(options.outputPath)}.`,
-        `Draft ${options.targetCount} checks unless the dataset is too small; in that case write fewer.`,
+        `Draft ${options.targetCount} checks unless the source folder is too small; in that case write fewer.`,
         "Each array item must be an object with exactly these keys:",
-        '- "question": a plain-English question answerable from this dataset',
+        '- "question": a plain-English question answerable from this source folder',
         '- "answer": the expected correct answer in plain English',
         "",
         "Quality bar:",
@@ -72,8 +72,8 @@ export async function draftTruthChecks(options) {
             return {
                 checks: null,
                 error: code === 0
-                    ? "The local agent finished without writing draft truth checks."
-                    : "The local agent did not produce draft truth checks.",
+                    ? "The local agent finished without writing draft checks."
+                    : "The local agent did not produce draft checks.",
             };
         }
         let parsed;
@@ -83,15 +83,15 @@ export async function draftTruthChecks(options) {
         catch (error) {
             return {
                 checks: null,
-                error: `Draft truth checks were not valid JSON: ${error instanceof Error ? error.message : String(error)}`,
+                error: `Draft checks were not valid JSON: ${error instanceof Error ? error.message : String(error)}`,
             };
         }
         const validated = DraftTruthChecksSchema.safeParse(parsed);
         if (!validated.success) {
-            const detail = validated.error.issues[0]?.message ?? "invalid draft truth checks";
+            const detail = validated.error.issues[0]?.message ?? "invalid draft checks";
             return {
                 checks: null,
-                error: `Draft truth checks did not match the required shape: ${detail}`,
+                error: `Draft checks did not match the required shape: ${detail}`,
             };
         }
         return {

package/dist/{commands → cli/commands}/compile-controller.d.ts

@@ -1,6 +1,6 @@
-import type { WorkflowExecutionProfile, WorkflowExecutor } from "../lib/executors.js";
-import type { SourceDatasetConfig } from "../lib/schema.js";
-import type { StageShellRetentionMode } from "../lib/workflows.js";
+import type { WorkflowExecutionProfile, WorkflowExecutor } from "../../packages/agents/lib/executors.js";
+import type { SourceDatasetConfig } from "../../packages/project-model/lib/schema.js";
+import type { StageShellRetentionMode } from "../../packages/compiler/workflows.js";
 export declare function runConfiguredCompiledCompile(options: {
     executor: WorkflowExecutor;
     testExecutor?: WorkflowExecutor | null;