@interf/compiler 0.2.5 → 0.3.0

package/README.md

@@ -1,39 +1,19 @@
 # Interf
 
-Interf is an eval-first knowledge compiler for agents such as Claude Code and Codex.
+Open-source knowledge compiler for your files.
 
-If you use OpenClaw, Hermes, or your own local retrieval workflow, the real problem is not opening a folder. It is getting the agent to work correctly on raw filesystem data without missing evidence, doing shallow analysis, or hallucinating once the task spans several files.
+Interf measures and improves how accurately local agents answer questions from your files.
 
-- your files stay on your machine
-- you choose the local agent
-- you decide what must be true
-
-Agents start missing things when a task spans PDFs, charts, and several files in one folder. That usually shows up when the job depends on:
-
-- reading reports and filings
-- extracting a number from a chart
-- understanding what is inside a folder before doing work
-- pulling context together across several files
-- checking the raw source when the answer has to be exact
-
-Raw filesystem data often looks fine until it is too late. The failure shows up as a missed number, a bad comparison, or an answer that sounds confident but is wrong.
-
-That is why compilation matters. Before the agent does the real job, the folder needs preparation.
-
-Interf Knowledge Compiler runs a local data pipeline with your agent as the executor. It produces a compiled workspace beside the raw files, with distilled notes and cross-file structure so the agent can understand what is in the folder, navigate it faster, and retrieve the right content without rediscovering everything from scratch.
+If you use Claude Code, Codex, OpenClaw, Hermes, or your own local agent setup on folders full of PDFs, docs, spreadsheets, and notes, the failure often shows up late: missed evidence, shallow analysis, bad comparisons, or answers that sound confident but are wrong.
 
-Interf gives you a simple loop:
+Interf lets you write a few questions and expected answers about your files, build a compiled workspace on top of those files, and see whether the result actually passes.
 
-- point Interf at a folder
-- say what must be true in `interf.config.json`
-- run `interf benchmark` to see how your agent does on the raw folder
-- compile a workspace beside the raw files
-- run the same evals again
-- keep the compiled workspace only if it performs better on your evals
-
-The first output is a shared compiled workspace for the whole folder.
+- your files stay on your machine
+- you choose the local agent
+- your raw files stay the source of truth
+- Interf adds a file-based layer on top
 
-Create an interface only when you want a second compiled workspace for one recurring job, with narrower retrieval, job-specific outputs, and extra evals for that job.
+It runs local data-processing pipelines with your agents as executors and produces a compiled workspace: a file-based layer on top of your raw files that agents can navigate, inspect, and work from.
 
 ## Quick Start
 
@@ -48,108 +28,98 @@ Install:
 npm install -g @interf/compiler
 ```
 
-Then run Interf in any folder:
+Start by writing a few questions and expected answers in `interf.config.json`:
+
+```json
+{
+  "workspaces": [
+    {
+      "name": "default",
+      "about": "General compiled workspace for the quarterly results folder.",
+      "checks": [
+        {
+          "question": "What full-year revenue range did the company maintain?",
+          "answer": "$4.8B to $5.0B in revenue."
+        },
+        {
+          "question": "Did gross margin improve or decline year over year?",
+          "answer": "Gross margin declined year over year."
+        }
+      ]
+    }
+  ]
+}
+```
+
+Then run Interf in that folder:
 
 ```bash
 cd ~/my-folder
 interf init
-interf benchmark
+interf create workspace
 interf compile
 interf benchmark
 ```
 
-That is the whole first loop:
-
-- point Interf at a folder you already have
-- let `interf init` write the first evals in `interf.config.json`
-- run `interf benchmark` on the raw folder first
-- compile the workspace
-- run `interf benchmark` again to see pass/fail on raw vs compiled
-
-`interf init` chooses your local agent, can draft `interf.config.json` if it is missing, and can attach the current folder right away. It does not move or replace your files.
+That first run gives you three concrete things:
 
-The compiled workspace is just a normal folder. Open it in your editor, in your agent, or in Obsidian if you want the graph view.
+- `interf/workspaces/default/` with the compiled workspace for your files
+- `interf/benchmarks/runs/...` with the saved benchmark result
+- a pass/fail score on the same questions and expected answers you wrote
 
-If Interf cannot find your local agent or compile setup, run:
+If `interf.config.json` is missing, `interf init` can draft it with you before the first compile. If Interf cannot find your local agent or compile setup, run:
 
 ```bash
 interf doctor
 ```
 
-Fastest sample loop:
+The first flow is:
+
+- write down a few questions your agent should be able to answer from your files
+- let `interf init` save those checks in `interf.config.json`
+- run `interf create workspace` and `interf compile` to build the compiled workspace
+- run `interf benchmark` to see whether that compiled workspace passes the checks
+- add another workspace only when one recurring context needs a narrower setup
+- if needed, rerun compile or use the advanced retry path until it is good enough
+
+## Why This Approach
+
+Interf is built around a few simple design principles:
+
+- `Explicit`: the output is visible and inspectable, not hidden memory
+- `Local`: your files stay on your machine
+- `File over app`: the output is just files, so you can use your editor, Unix tools, Obsidian, or your own software on top
+- `BYOAI`: use Claude Code, Codex, OpenClaw, Hermes, or your own model
+
+Interf does not replace your data with an opaque store. It keeps the raw files in place and adds a file-based layer on top for agents.
+
+Sample flow:
 
 ```bash
 cp -r examples/benchmark-demo /tmp/interf-demo
 cd /tmp/interf-demo
 interf init
-interf benchmark
-interf compile
-interf benchmark
-```
-
-If you want a second compiled workspace shaped for one recurring job, add an interface:
-
-```bash
-interf create interface
+interf create workspace
 interf compile
 interf benchmark
 ```
 
-An interface is useful when the shared workspace is still too broad and you want:
+## Start With A Few Questions
 
-- narrower retrieval for one job
-- job-specific outputs
-- extra evals on top of the shared baseline
+`interf.config.json` is where you write the questions and expected answers for a folder.
 
-## Start With One Small Eval
+That file uses one `workspaces` array:
 
-`interf.config.json` is where you write what must be true.
+- the first workspace is the default compiled workspace for the folder
+- later workspaces are optional and only exist when one recurring context needs a narrower setup
+- each workspace carries its own `checks`
 
 If the file is missing, `interf init` can draft it with you before the first compile. You can edit it any time.
 
-Use it for:
-
-- top-level `evals` for shared baseline checks
-- `interfaces[].evals` for task-specific additional checks
-
-Both live in the same root `interf.config.json`.
-
-Example shape:
-
-Top-level `evals` are shared baseline checks for the workspace and every interface. Each entry in `interfaces` adds extra checks for one dedicated job.
-
-```json
-{
-  "evals": [
-    {
-      "question": "What changed in full-year guidance?",
-      "answer": "Full-year guidance was maintained at $4.8B to $5.0B in revenue."
-    },
-    {
-      "question": "Did gross margin improve or decline year over year?",
-      "answer": "Gross margin declined year over year."
-    }
-  ],
-  "interfaces": [
-    {
-      "name": "operator-briefing",
-      "about": "Prepare tomorrow's operator briefing from the quarterly results folder.",
-      "evals": [
-        {
-          "question": "What revenue range did the company maintain for full-year guidance?",
-          "answer": "$4.8B to $5.0B in revenue."
-        },
-        {
-          "question": "What should the operator pay attention to next quarter?",
-          "answer": "Watch guidance, gross margin, and any demand changes mentioned in the report."
-        }
-      ]
-    }
-  ]
-}
-```
+That example is just `interf.config.json`.
+Advanced retry settings do not live there.
 
-Good first evals are small and practical:
+Good first checks are small and practical:
 
 - one exact number from a chart, table, or filing
 - one short statement that should be true or false
@@ -158,158 +128,187 @@ Good first evals are small and practical:
 Then run:
 
 ```bash
-interf benchmark
+interf create workspace
 interf compile
 interf benchmark
 ```
 
-If the benchmark does not show an improvement over raw files, keep iterating on evals or workflow choice first. Use the experiment loop below only when you want the advanced automated path.
+## What `interf benchmark` Compares
 
-## Compare Three Things
+`interf benchmark` compares compiled workspaces, not raw chat sessions.
 
-Compare:
+It lets you answer a simple question:
 
-1. the raw folder
-2. the workspace
-3. an interface for one specific job
+- does this compiled workspace pass the checks?
+- which compiled workspace or workflow performs better on the same folder?
+- does another workspace help more on a narrower recurring context?
 
-`interf benchmark` runs the same evals against each one and saves a pass/fail report.
+By default it loads checks from `interf.config.json`, discovers eligible compiled workspaces under `interf/workspaces/`, and saves the run under `interf/benchmarks/runs/`.
 
-That gives you one clear question:
+If you run `interf benchmark` from inside a workspace, it uses that workspace's checks. If you run it from the source folder, it uses the default workspace checks.
 
-- is the raw folder enough?
-- does the workspace retrieve better?
-- does a dedicated interface do better than both?
+If you need raw-file probes too, use the advanced eval-pack runner in [docs/eval-loop.md](./docs/eval-loop.md).
 
-## What `interf compile` Actually Does
+## What `interf compile` Does
 
-`interf compile` runs a workflow over your folder.
+`interf compile` runs the Interf data-processing pipeline over your files.
 
-That workflow is the compilation pipeline:
+By default, that means:
 
 - read the files
-- write processed notes and navigation files
-- build the workspace your agent can use
-- optionally build an interface for one specific job
+- write summaries and navigation docs
+- organize cross-file knowledge
+- build the compiled workspace for the folder
+- optionally build additional workspaces for recurring contexts you care about
+
+In public docs, `pipeline` is the thing Interf runs. `workflow` is the saved method that defines or customizes that pipeline.
 
 The default workflow is built in. If you want a different method, you can define your own workflow package and benchmark it on the same folder.
 
-## Experiment Loop
+## What Gets Created
 
-Interf Knowledge Compiler also supports an advanced experiment loop above compile + benchmark.
+After compile, Interf writes into `./interf/` beside your source files.
 
-This is the eval-first part of the product. You give Interf the folder and the evals that must pass. Interf keeps running controlled compile + benchmark attempts against that same truth surface until it either gets a working result or runs out of attempts.
+- `interf/workspaces/<name>/` is a compiled workspace over the folder
+- `interf/benchmarks/runs/...` stores saved benchmark runs
 
-Each attempt reruns the compilation workflow, reruns the benchmark, and records what changed. It stops when:
+Inside those workspaces you will see things like:
 
-- the evals pass
-- or the experiment budget is exhausted
+- summaries of source files
+- navigation notes and entrypoints for agents
+- cross-file knowledge notes
+- workspace-specific outputs for one recurring context
+- benchmark artifacts you can inspect later
 
-In practice, that means:
+The compiled workspace is just a normal folder. Open it in your editor, in your agent, or in Obsidian if you want the graph view.
 
-- `retry_policy.max_attempts_per_profile` controls how many experiment attempts each compile profile gets
-- stronger diagnostic profiles can be used only after the default ones fail
-- the loop is still judged on the same eval truth from your folder
-- failure summaries can be captured between attempts for diagnosis
+If you use Obsidian, open `interf/workspaces/<name>/` as the vault for the compiled workspace.
 
-Today that advanced path is configured through eval packs and explained in the deeper docs. The workflow is the part that changes. The experiment loop is the controller that keeps trying workflows and profiles against the same evals with a fixed attempt budget.
+## Terminology
 
-Use the simple loop first. Use the experiment loop when you want Interf to keep improving the local compilation workflow until the workspace is ready for your task or the attempt budget runs out.
+Public terms:
 
-## Use It With Your Agent
+- `your files` = the source folder Interf reads from
+- `questions and expected answers` = the checks you want your agent to pass
+- `checks` = the pass/fail questions each workspace should satisfy
+- `benchmark` = score compiled workspaces on the same checks
+- `compiled workspace` = the output Interf produces on top of a folder
+- `workspace` = an optional additional compiled workspace for one recurring context
 
-If you already work through Claude Code, Codex, OpenClaw, or Hermes, the agent can run this loop for you.
+Technical terms:
 
-Paste something like this into Claude Code, Codex, OpenClaw, or Hermes:
+- `source folder` = the raw files Interf reads from
+- `workflow` = the saved method that defines or customizes the pipeline
+- `.interf/` = runtime state, proofs, and health artifacts
 
-```text
-Install @interf/compiler, run `interf init` in this folder, and use the local agent executor.
+## Advanced: Additional Workspaces
 
-If `interf.config.json` is missing, draft evals for what must be true for this task and ask me to confirm them.
+Start with the default compiled workspace first.
 
-Then run `interf benchmark`, `interf compile`, and `interf benchmark` again.
+Most folders only need one compiled workspace.
 
-Tell me whether the processed workspace beat raw files, and only recommend it if it did.
-```
+Create another workspace only when one recurring context needs a narrower setup.
 
-That is the basic loop:
+Use the default compiled workspace for broad questions such as:
 
-- the user or agent defines what must be true
-- benchmark the raw folder first
-- Interf prepares the compiled workspace
-- benchmark again and keep it only if it helped
+- what is in this folder?
+- what changed?
+- where is the source evidence?
 
-## What Gets Created
+Create another workspace when that broad layer is no longer enough and the work becomes a repeatable context, for example:
 
-After compile, Interf writes into `./interf/` beside your source files.
+- operator briefings
+- board prep
+- finance reporting
+- diligence review
+- chart extraction for a report set
+- one recurring research set
 
-- `interf/<name>/` is the shared workspace over the folder
-- `interf/<name>/interfaces/<name>/` is a task-specific workspace for one job
-- `interf/benchmarks/runs/...` stores saved benchmark runs
+Why create one:
 
-Inside those workspaces you will see things like:
+- it narrows what the agent should retrieve for that context
+- it prepares guidance and structure on top of the raw folder for that recurring context
+- it lets you benchmark that context separately
 
-- summaries of source files
-- navigation notes and entrypoints for agents
-- task-specific outputs for one interface
-- benchmark artifacts you can inspect later
+If the default compiled workspace is already enough, do not create another workspace yet.
+
+## Advanced: Keep Improving Until It Passes
 
-In the CLI, the main Interf workspace is called a **knowledge base**. A task-specific workspace inside it is called an **interface**.
+Interf also supports an advanced experiment path above the normal build + benchmark flow.
 
-Conceptually, the compiled workspace is a knowledge representation of the folder for agents. In the product and CLI, the concrete objects are the compiled workspace, the knowledge base, and the interface.
+Give it the same folder and the same checks. Interf can keep rerunning compile + benchmark attempts until the benchmark passes or the attempt budget runs out.
 
-If you use Obsidian, open `interf/<name>/` as the vault for the main compiled workspace. If you are working with interfaces, open the parent knowledge-base folder so links across summaries, knowledge notes, and interfaces keep resolving.
+In practice:
 
-## What An Interface Is
+- `retry_policy.max_attempts_per_profile` controls how many attempts each compile profile gets
+- stronger diagnostic profiles can be used only after the default ones fail
+- the checks stay the same across every attempt
+- each attempt records what changed
+
+Example eval-pack shape:
+
+```jsonc
+{
+  "workspaces": [
+    {
+      "name": "default",
+      "checks": [
+        {
+          "question": "What full-year revenue range did the company maintain?",
+          "answer": "$4.8B to $5.0B in revenue."
+        }
+      ]
+    }
+  ],
+  // Advanced only: retry settings live in eval packs, not in interf.config.json.
+  "retry_policy": {
+    "max_attempts_per_profile": 3
+  }
+}
+```
 
-Start with one workspace.
+Today this lives in the advanced eval-pack runner, not in `interf.config.json` and not in a top-level `interf compile --max-retries` flag.
 
-An interface is a second compiled workspace for one recurring job.
+Use the normal benchmark flow first. Use this advanced path when you want Interf to keep improving the local preparation workflow until the workspace is good enough for your task or the attempt budget runs out. It spends more tokens, so use it when that extra spend is worth the accuracy target.
 
-The main compiled workspace is the shared layer for the whole folder. Use it for broad questions such as:
+## Use It With Your Agent
 
-- what is in this folder?
-- what changed?
-- where is the source evidence?
+If you already work through Claude Code, Codex, OpenClaw, or Hermes, the agent can run this process for you.
 
-Create an interface when that broad layer is no longer enough and the work becomes a repeatable job, for example:
+Paste something like this into your agent:
 
-- prepare tomorrow's operator briefing
-- run diligence on this deal room
-- extract chart values from this report set
-- answer one recurring research question set
+```text
+Install @interf/compiler, run `interf init` in this folder, and use the local agent executor.
 
-Why create one:
+If `interf.config.json` is missing, draft a default workspace with a few checks this agent should be able to answer from these files and add the expected answers for me to confirm.
 
-- it narrows what the agent should retrieve for that job
-- it writes job-specific outputs on top of the shared workspace
-- it lets you add extra task-specific evals on top of the shared evals
+Then run `interf create workspace`, `interf compile`, and `interf benchmark`.
 
-If the shared workspace is already enough for the job, do not create an interface yet.
+Tell me whether the compiled workspace passes the checks, and only recommend it if it does.
+```
 
 ## Custom Workflows
 
 Interf ships with a default workflow.
 
-If you want to change how compilation happens on your data, this is the part you customize:
+If you want to change how the data-processing pipeline runs on your files, this is the part you customize:
 
 ```bash
 interf create workflow
 interf verify workflow --path <path>
 ```
 
-Then benchmark that workflow on the same folder and the same evals.
+Then benchmark that workflow on the same folder and the same checks.
 
 Workflow package docs live in [docs/workflow-spec.md](./docs/workflow-spec.md).
 
 ## Core Commands
 
-- `interf init` = choose your local executor and optionally attach the current folder
-- `interf create knowledge-base` = create the shared processed workspace for this folder
-- `interf create interface` = create a task-specific workspace on top
+- `interf init` = choose your local executor and draft checks
+- `interf create workspace` = create a compiled workspace for this folder
 - `interf create workflow` = create a reusable local workflow package
-- `interf compile` = build the current workspace
-- `interf benchmark` = compare raw files vs processed workspaces on your evals
+- `interf compile` = build the current compiled workspace
+- `interf benchmark` = score compiled workspaces on your checks
 - `interf doctor` = check local executor setup
 - `interf verify <check>` = run deterministic checks on major workflow steps
 - `interf reset <scope>` = remove generated state while keeping source files
@@ -319,7 +318,7 @@ Workflow package docs live in [docs/workflow-spec.md](./docs/workflow-spec.md).
 - [docs/workflow-spec.md](./docs/workflow-spec.md) for custom workflow packages
 - [docs/runtime-contract.md](./docs/runtime-contract.md) for the exact on-disk contract
 - [docs/architecture.md](./docs/architecture.md) for the deeper system model
-- [docs/eval-loop.md](./docs/eval-loop.md) for the advanced benchmark and experiment loop
+- [docs/eval-loop.md](./docs/eval-loop.md) for advanced eval-pack experiments across workflows and models
 
 Maintainers should use [CONTRIBUTING.md](./CONTRIBUTING.md) for test and release gates.
 

package/dist/commands/benchmark.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"benchmark.d.ts","sourceRoot":"","sources":["../../src/commands/benchmark.ts"],"names":[],"mappings":"AA0BA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AA2W3C,eAAO,MAAM,gBAAgB,EAAE,aA8D9B,CAAC"}
+{"version":3,"file":"benchmark.d.ts","sourceRoot":"","sources":["../../src/commands/benchmark.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AA6B3C,eAAO,MAAM,gBAAgB,EAAE,aA2E9B,CAAC"}