@interf/compiler 0.6.6 → 0.6.8

package/README.md

@@ -1,32 +1,34 @@
 # Interf
 
-Interf prepares context for your agents.
-Use it to check whether your files are ready for the work you want your agents to do.
+Interf prepares portable context for your agents.
+Check the source files first. Build portable context when the agent misses evidence, loses links across files, or sounds confident without seeing the full picture.
 
-Tell Interf what you need from those files. It can compare how your local agents do on the source files. If the files alone are not enough, it builds portable context and keeps improving that context until more questions pass.
+**The files can be right. The agent can still be wrong.**
 
-Interf Compiler builds that portable context as a local folder next to your files.
-Your source files stay the source of truth. Interf builds on them; it does not replace them.
+Agents now run real work from folders of PDFs, exports, notes, and transcripts. The failure often shows up late: missed evidence, shallow analysis, bad comparisons, or answers that sound confident but are wrong.
 
-For the built-in `interf` workflow, portable context for your agents includes:
+A chart inside one PDF can be obvious to a human analyst and easy for an agent to miss. Across the folder, the real problem is rarely one missing fact. The agent never assembles the full picture from the files on its own.
+
+`Interf Compiler` is the local, open-source runtime that runs context-preparation workflows on your files. A workflow tells Interf how to process, organize, and structure those files for agent use. Your local agent runs each stage. Interf Compiler runs the pipeline and enforces the output shape. `interf test` scores the result on your own questions. The portable context is a proof, not a promise.
 
 ```text
-interf/market-report/
-  AGENTS.md      # how your agent should use this folder
-  CLAUDE.md      # same guidance for Claude Code
-  home.md        # top-level map for the work
-  summaries/     # file-by-file summaries
-  knowledge/     # linked notes built from those files
+project-root/
+  bristol-office-analysis/
+    report.pdf
+    notes.md
+    exports/
+  interf.json                   # main config: works, source paths, questions, workflow choices, and defaults
+  interf/
+    bristol-office-analysis/    # portable context your agent starts from
+    tests/
+      bristol-office-analysis/  # saved source-files vs portable-context comparisons
 ```
 
-That is one example. Different workflows can build different folders. In every case, your agent starts from one local folder.
+The project root is the control plane. Your agent starts from `interf/<work>/` when the source files are not enough.
 
-One public example from a PDF market report uses checks like:
+## What a Run Looks Like
 
-- `What were Bristol's annual take-up values in 2018 and 2016?`
-- `What were Bristol's availability values in 2018 and 2016?`
-
-On those same checks, one recent public comparison produced:
+A recent public run — one PDF market report, two questions, two agents on the same setup — produced:
 
 <!-- PUBLIC_BENCHMARK_TABLE:START -->
 | Agent | Source files | Portable context |
@@ -35,119 +37,164 @@ On those same checks, one recent public comparison produced:
 | Claude Code (Claude Opus 4.6, max) | `0/2` | `2/2` |
 <!-- PUBLIC_BENCHMARK_TABLE:END -->
 
-That means:
-
-- Codex already passed on the source files.
-- Claude Code needed the portable context to pass.
+Codex passed on the raw files; Claude Code only passed after Interf prepared the portable context. Both numbers come from `interf test`, which scores the same questions against the source files and the portable context.
 
 That gives you a readiness signal for the work and a fair comparison on the same files.
 
-Same work. Same checks. Same local setup.
-
-## How It Works
+The table is a sample, not a leaderboard. Run it on your own files. If you run more than one local agent, the same pass gives you a fair comparison between them on the same files.
 
-1. Pick the files for the work.
-2. Tell Interf what you need from them.
-3. Review or edit the questions.
-4. Run a source-files baseline if you want to check readiness first.
-5. Build the portable context for your agents.
-6. Run `interf test` on the same questions.
+## Design Choices
 
-Check the source files first if you want the baseline. If the files already pass, keep them. If the portable context does better, use it.
+- `Check before build`: source files are sometimes already enough. Interf will say so.
+- `Prove the full picture`: every build is scored on your own questions. If the portable context does not help, `interf test` tells you.
+- `File over app`: the portable context is a local folder next to the source files. Inspect it, diff it, version it.
+- `Bring your own AI`: use Claude Code, Codex, or another local agent.
+- `Explicit`: no hidden store, no hidden index. The portable context is plain files on disk.
 
-## Why It Exists
+## Install
 
-Raw files are often not enough.
+```bash
+npm install -g @interf/compiler
+interf            # opens the setup wizard in the current folder
+```
 
-Local agents can open reports, decks, transcripts, exports, PDFs, notes, and mixed folders. They can still miss the structure the work needs.
+Requires Node.js 20+ and a local coding agent such as Claude Code or Codex. Run `interf doctor --live` if the executor is not detected.
 
-Interf checks the source files first. If they are already enough, keep them. If they are not, Interf builds a local context layer next to those files and checks whether it helped.
+## Quick Start
 
-Interf is built around a few simple rules:
+1. Run `interf` in the project root.
+2. Pick the source folder for one work.
+3. Describe what the agent should do with those files.
+4. Review the suggested questions.
+5. Run `interf test --target raw` to check the source files.
+6. Run `interf compile` to build portable context.
+7. Run `interf test` again to compare source files and portable context on the same questions.
+8. If the portable context does better, point your agent at `interf/<work>/`.
 
-- Explicit: the preparation stays on disk in a local folder you can inspect, diff, review, and version.
-- File over app: the result lives next to the source files.
-- Bring your own AI: use Claude Code, Codex, or another local agent. Automated Interf runs currently support Claude Code and Codex.
-- Proof over vibes: check source files and portable context on the same work.
-- Source files stay the source of truth.
+If the source files already pass, stop there. Interf is allowed to tell you that you do not need portable context for that work.
 
-## Folder Layout
+## Portable Context
 
-If your folder already contains the files for the work, Interf adds a small local structure around them:
+`interf/<work>/` is the local folder your agent starts from when the source files are not enough. For the built-in `interf` workflow, it includes:
 
 ```text
-your-folder/
-  market-report/
-    report.pdf
-    notes.md
-    exports/
-  interf.json
-  interf/
-    market-report/
-    tests/
-      market-report/
+interf/bristol-office-analysis/
+  AGENTS.md       # how to use this folder
+  CLAUDE.md       # same guidance for Claude Code
+  raw/            # snapshot of the source files for this work
+  home.md         # top-level map for the work
+  summaries/      # per-file summaries
+  knowledge/      # linked notes built from the files
 ```
 
-`market-report` is just the work name. Interf reuses it under `interf/` and `interf/tests/`.
+The source files stay the source of truth. Interf builds next to them; it does not replace them.
 
-When Interf finishes, point your agents at `interf/market-report/` first. Go back to the source files whenever you need to verify something.
+The portable context is a sibling folder you can inspect, diff, version, or hand to a different agent. The portable context carries its own `raw/` snapshot, so the evidence the agent works from stays attached to the result.
 
-## Quick Start
+## Workflow Packages
 
-Requirements:
+A workflow is Interf's preparation method for a kind of work. It tells the compiler how to process the source files and what folder shape to build for the agent. The workflow declares the compilation stages. Your local agent runs each one. The compiler writes the result as one folder on disk.
 
-- Node.js 20+
-- a local coding agent such as Claude Code or Codex
+A **context interface** is the shape of that output — the folder tree, the named files, and the rule that every one of them must exist on disk before the build is called done. The workflow defines the interface. The compiler enforces it.
 
-Install:
+The built-in `interf` workflow is the default. Run `interf create workflow` to draft a reusable workflow package for a different kind of work — a different method, a different output shape, or both.
 
-```bash
-npm install -g @interf/compiler
+```text
+interf/workflows/office-analysis/
+  workflow.json         # method: how to process and organize the files
+  workflow.schema.json  # output contract: what folder shape the compiler must build
+  README.md             # what this workflow is for, for humans and agents
+  compile/
+    stages/
+      summarize/        # define your own stages here
+      structure/
+      shape/
+  use/
+    query/              # how your agent reads the portable context
+  improve/              # how Interf revises this workflow if questions still fail
 ```
 
-Then run Interf in a project folder that already contains the files for the work:
+Think of it as method -> output shape.
 
-```bash
-interf
-```
+- `workflow.json` describes the method: expected inputs, stages, and how the files should be processed.
+- `workflow.schema.json` describes the output contract: the folder shape and outputs the compiler must build.
+- `compile/stages/<stage>/` is where you author one folder per stage — a small, specific job like `summarize` or `structure`. You write the instructions; the compiler runs them with your local agent.
+- `use/query/` holds the instructions your agent follows when it reads the portable context for live work.
+- `improve/` holds the instructions Interf follows when the first build does not pass and the workflow itself needs editing.
 
-The first run opens the setup wizard for that folder. It can:
+A workflow package is the reusable preparation method. Interf Compiler runs it on your files. Your local agent runs each stage. The result is a local folder your agents can read.
 
-- draft `interf.json`
-- let you choose the source folder for the work
-- ask what work you need help with
-- recommend questions you can verify from the files, or let you add them manually
-- run a source-files baseline so you can see whether the files are already enough
-- build the portable context your agents will use for that work
-- check that portable context on the same questions
+## Compiler Primitives
 
-If Interf cannot find your local executor setup, run:
+Every workflow runs on the same three primitives. Interf Compiler enforces the contract. The workflow chooses the method.
 
-```bash
-interf doctor --live
+### Input snapshot
+
+The workflow declares what source files it expects — one PDF, a folder of mixed exports, a tree of notes. Interf Compiler copies those inputs into `raw/` before any stage runs. The evidence stays attached to the result.
+
+### Context interface
+
+The workflow declares the folder shape the agent must read. Interf Compiler checks on disk that the shape holds before the build is called done. If a stage writes nothing, the build fails loudly.
+
+### Stage execution
+
+The workflow declares ordered compilation stages — small, specific jobs like `summarize`, `structure`, `shape`. You write the instructions per stage. Interf Compiler runs each one through your local agent and checks the output before moving to the next.
+
+The context interface is the key guarantee. The shape holds on disk, or `interf test` reports the gap on your own questions.
+
+## Self-Improving Workflows
+
+When the first build still misses questions, Interf edits the workflow package itself and compiles again. Same files, same questions, different preparation. The workflow is the unit of improvement — not just the output folder.
+
+Interf saves every scored run under `interf/tests/<work>/`. You can inspect what changed and why a later build did better.
+
+## How the Pieces Fit
+
+- `interf.json` is the project control file. It names the work, points at the source files, saves the questions, and selects the workflow.
+- A workflow package is the reusable preparation method.
+- `workflow.json` defines that method.
+- `workflow.schema.json` defines the output contract, or context interface, the compiler must build.
+- `Interf Compiler` is the local runtime. It runs the workflow, your agent runs each stage, and the portable context lands next to the source files.
+- The portable context is the local folder built for your agents. Technically, that folder is the compiled context.
+
+## interf.json
+
+`interf.json` is the main config file in the project root. Each entry describes one work: the source path, the saved questions, the workflow that prepares the context, and optional defaults.
+
+```text
+{
+  "datasets": [
+    {
+      "name": "bristol-office-analysis",
+      "path": "./bristol-office-analysis",
+      "about": "Read the report and answer the saved questions.",
+      "workflow": "interf",
+      "checks": [
+        { "question": "...", "answer": "..." }
+      ]
+    }
+  ]
+}
 ```
 
-## Checks
+The `workflow` field is the key choice per work. Set it to `interf` for the built-in workflow, or to a workflow you have authored. A different workflow is a different method and a different output shape. Same files, different portable context.
 
-Interf uses question-and-answer checks.
-They are small questions that tell you whether the work is actually covered.
+## Questions
 
-Interf runs the same checks on the source files and on the portable context. That gives you a source-files baseline and a portable-context comparison on the same task.
+In the flow, they look like plain questions. In `interf.json`, they live under `checks[]`.
 
-Good first checks are small and easy to verify from the files:
+They should be small, verifiable facts you already know from the files:
 
 - one exact number from a chart, table, or filing
 - one short statement that should be true or false
-- one simple comparison across years, files, or sections
-
-Interf can recommend a starting set from the files and the work description. You can confirm, edit, or replace them.
+- one comparison across years, files, or sections
 
-In the CLI, you mostly see them as questions. In `interf.json`, they live under the technical field `checks[]`.
+Interf proposes an initial set from the files and the work description. You confirm, edit, or replace them.
 
 A maintained public test example in this repo stores them like this:
 
 <!-- PUBLIC_TEST_CHECKS:START -->
-```jsonc
+```text
 {
   "datasets": [
     {
@@ -170,106 +217,29 @@ A maintained public test example in this repo stores them like this:
 ```
 <!-- PUBLIC_TEST_CHECKS:END -->
 
-## What `interf test` Measures
-
-`interf test` compares the source files and the portable context your agents would use on the same questions.
-
-That makes it a small benchmark for the work at hand:
-
-- same work
-- same checks
-- same local agent setup
-- source files on one side
-- portable context your agents use on the other
-
-If you run more than one local agent, it also gives you a fair comparison between them on the same files.
-
-If the portable context does not help, keep the source files.
-
-Use `interf test` on your own files instead of trusting a frozen benchmark snapshot in the docs.
-
-Interf saves the latest comparison plus the underlying source-files and portable-context runs under `interf/tests/` in the same project.
-
-## The Compile And Check Loop
-
-1. Save a few checks for the work in `interf.json`.
-2. Run `interf test --target raw` if you want the source-files baseline.
-3. Run `interf compile`.
-4. Run `interf test`.
-5. If self-improving loops are enabled, let Interf revise the workflow and try again on the same work and checks.
-
-`interf.json` stores the saved setup and checks. Technically, that portable context is the compiled context Interf reuses for repeat runs.
-
-## Workflows
-
-A workflow is the preparation method Interf runs on your files.
-
-In plain language, it is the method that organizes and structures those files for agent use.
-
-It can summarize files, extract structure, and build the local context layer your agent starts from.
-
-Interf runs that method with your local coding agent and builds the result on disk as portable context.
-
-The folder shape it produces for the agent is the context interface. The reusable method itself is the workflow package.
+## What Interf Is Not
 
-Publicly, you mostly care about the portable context it produces. Technically, that portable context is the compiled context.
+- Not a second brain or memory product. One local folder per work; nothing global.
+- Not a vector store or RAG server. The portable context is plain files.
+- Not a hosted context layer. Interf runs locally and `interf/` is yours.
+- Not an agent harness. Interf prepares the folder; your existing agent reads it.
+- Not a public leaderboard. Every score comes from your files, your questions, and your agent.
 
-Interf ships with a built-in `interf` workflow for the common case. If you need a different method, create one locally:
-
-```bash
-interf create workflow
-```
-
-Workflow creation supports two paths:
-
-- draft a workflow from the current project with a local agent
-- copy an existing workflow and edit stage guidance directly
-
-After assignment, build the portable context for your agents with `interf compile` and run `interf test` on the same questions.
-
-## Self-Improving Loops
-
-Interf can do more than one compile pass.
-
-If the first portable context is still weak, Interf can revise the workflow and compile again.
-
-Each loop keeps the work and the checks fixed. The thing that changes is the preparation method.
-
-If self-improving loops are enabled, Interf can:
-
-- build the current portable context
-- run the same checks
-- inspect failures and traces
-- revise the workflow
-- build and test the next version
-
-Lower-level retry budgets live in `interf.json` and the technical package docs.
-
-## Use It With Your Agent
-
-If you already work through a local coding agent, hand it the whole flow:
-
-Paste something like this into your agent:
-
-```text
-Install `@interf/compiler`, run `interf` in this folder, and use the local agent executor.
-
-If `interf.json` is missing, draft one dataset entry for this work. Ask what I need from the selected files. Recommend a few questions I can verify from those files, and add the expected answers for me to confirm under `checks[]`.
-
-Run a source-files baseline if useful. Build the portable context for my agents for that work. Then run `interf test` on the same questions.
-
-Only recommend the portable context if it passes those questions. If you also ran the source-files baseline, tell me whether it did better than the source files.
-```
+## Useful Commands
 
-## More Docs
+- `interf` or `interf init` — open the wizard and save the first setup
+- `interf compile` — build portable context for your agents
+- `interf test` — compare source files and portable context on the same questions
+- `interf create workflow` — draft a reusable workflow package
+- `interf doctor --live` — verify the local executor
 
-- [src/packages/README.md](./src/packages/README.md) for the short system map
-- [src/packages/compiler/PACKAGE.md](./src/packages/compiler/PACKAGE.md) for compiler primitives and the runtime contract
-- [src/packages/workflow-package/PACKAGE.md](./src/packages/workflow-package/PACKAGE.md) for the workflow package model
-- [src/packages/workflow-authoring/PACKAGE.md](./src/packages/workflow-authoring/PACKAGE.md) for workflow authoring and self-improving loops
+## Docs
 
-Maintainers should use [CONTRIBUTING.md](./CONTRIBUTING.md) for test and release gates.
+- [src/packages/README.md](./src/packages/README.md) — system map
+- [src/packages/compiler/PACKAGE.md](./src/packages/compiler/PACKAGE.md) — compiler primitives and runtime contract
+- [src/packages/workflow-package/PACKAGE.md](./src/packages/workflow-package/PACKAGE.md) — workflow package model
+- [src/packages/workflow-authoring/PACKAGE.md](./src/packages/workflow-authoring/PACKAGE.md) — workflow authoring and workflow improvement
 
-## License
+Contributors: see [CONTRIBUTING.md](./CONTRIBUTING.md).
 
-Code is licensed under Apache 2.0. The `Interf` name and branding are reserved; see [TRADEMARKS.md](./TRADEMARKS.md).
+Code: Apache 2.0. Name and branding: see [TRADEMARKS.md](./TRADEMARKS.md).

package/dist/cli/commands/init.js

@@ -456,7 +456,7 @@ export const initCommand = {
 };
 export async function runInitCommand() {
     p.intro(chalk.bold("Interf"));
-    p.log.info("Interf prepares context for your agents from the source folder you choose. Start with the work, review the suggested questions, check the source files first, and build a local folder when the files need more structure.");
+    p.log.info("Interf prepares portable context for your agents from the source folder you choose. Start with the work, review the suggested questions, check the source files first, and build a local folder when the files need more structure.");
     const cwd = process.cwd();
     const detected = detectInterf(cwd);
     const sourcePath = detected ? resolveSourceControlPath(detected.path) : cwd;

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@interf/compiler",
-  "version": "0.6.6",
-  "description": "Interf prepares context for your agents by checking your files first, building a local folder when needed, and using self-improving loops until more questions pass.",
+  "version": "0.6.8",
+  "description": "Interf prepares portable context for your agents by checking your files first, building a local folder when needed, and using self-improving loops until more questions pass.",
   "type": "module",
   "bin": {
     "interf": "dist/bin.js"