@interf/compiler 0.2.0 → 0.2.2

package/README.md

@@ -1,168 +1,120 @@
 # Interf
 
-The open-source eval-first knowledge compiler.
+Interf Knowledge Compiler uses local agents such as Claude Code and Codex to run a data-processing workflow over your files.
 
-Interf compiles a workspace beside your files for agents: a knowledge representation they can navigate, cross-check against raw source, and prove on your evals.
+It creates a workspace with notes and navigation so the agent can see what is in the folder and what to retrieve.
 
-Your files stay the truth. Interf adds a compiled workspace and benchmark proof.
+Then you test that workspace on your evals.
 
-- point it at a folder you already have
-- define or confirm what must be true for the task in `interf.config.json`
-- compile a shared knowledge base plus task-specific interfaces
-- benchmark raw files vs compiled workspaces and keep the best result
+- your files stay on your machine
+- you choose the local agent
+- you decide what must be true
 
-Most "AI knowledge base" tools optimize for a demo. Interf optimizes for proof. It keeps the raw files on disk, compiles a visible workspace your agent can use, and makes workflows compete on your evals instead of on marketing claims.
+Agents start missing things when a task spans PDFs, charts, and several files in one folder. That usually shows up when the job depends on:
 
-Interf is not a chat shell, a hosted notes app, or a generic agent OS. It is the compile + benchmark loop for turning real folders into better agent workspaces and proving they help on a real task.
+- reading reports and filings
+- extracting a number from a chart
+- understanding what is inside a folder before doing work
+- pulling context together across several files
+- checking the raw source when the answer has to be exact
 
-## What Happens
+The workspace exists so the agent does not have to rediscover the folder from scratch on every run.
 
-```text
-raw folder
-  -> compiled workspace beside the raw files
-  -> benchmark proof on your evals
-```
-
-The compiled workspace is for agents. It gives them:
-
-- a clearer map of the data
-- task-specific outputs when broad summarization is not enough
-- better evidence paths back to the raw source
-- proof of whether the compiled workspace actually helped
-
-Interf does not replace your agent. It gives your agent a better workspace to use.
-
-## Trust Boundary
-
-Interf keeps one trust boundary:
-
-- raw files in the source folder are the content truth
-- `interf.config.json` is the user-approved task truth
-- the compiled workspace is the generated working surface
-- the benchmark result is the proof of whether that generated surface is good enough
-
-That means:
+The point is proof on your data, not generic AI claims.
 
-- agents may draft evals
-- users approve accepted task truth
-- raw files remain the final source of evidence
-- compiled workspaces earn trust only if they pass the benchmark
+The simplest way to use Interf is to compare the same task before and after compilation:
 
-## Who It’s For
+- run the task on the raw folder
+- compile the folder with Interf
+- run the same task again from the workspace
+- if you want a recorded pass/fail result, add evals and run `interf benchmark`
 
-Interf is for people already trying to get real work done with agents on real folders:
-
-- Claude Code and Codex users
-- OpenClaw and Hermes-style local-agent users
-- technical founders, researchers, and operators with messy source folders
-- teams who want to test whether compiled workspaces beat raw files on their own tasks
-
-If you want a generic chat UI, this is not that product.
-
-## Mental Model
-
-- **Source folder**: your real files stay where they are
-- **Compiled workspace**: the generated workspace beside those files for agents
-- **Knowledge base**: the shared compiled workspace over the folder
-- **Interface**: the task-specific compiled workspace for one job
-- **Workflow**: the reusable compile method
-- **Eval**: what must be true for the task
-- **Benchmark**: the proof loop that compares raw and compiled results
-
-One source folder can host multiple knowledge bases under `interf/` if you want to compare workflows on the same data.
-
-## Install
+## Quick Start
 
 Requirements:
 
 - Node.js 20+
-- one local coding agent executor: Claude Code or Codex
+- a local coding agent: Claude Code or Codex
 
-Install the published package:
+Install and check setup:
 
 ```bash
 npm install -g @interf/compiler
-```
-
-Sanity check the local setup:
-
-```bash
 interf doctor
 ```
 
-If you already use Claude Code or Codex locally, that is the intended path. Interf uses your local agent as the executor for compile and benchmark runs.
-
-## Quick Start
-
-Initialize Interf in any folder:
+Then run Interf in any folder:
 
 ```bash
 cd ~/my-folder
 interf init
+interf compile
+interf benchmark
 ```
 
-That flow can:
+That is the whole first loop:
 
-1. choose an executor like Claude Code or Codex
-2. optionally install helper skills
-3. attach the current folder as a knowledge base
-4. optionally compile the knowledge base immediately
+- point Interf at a folder you already have
+- let `interf init` write the first evals in `interf.config.json`
+- compile the workspace
+- ask the agent to use it
+- run `interf benchmark` to compare raw vs compiled
 
-Then:
+`interf init` chooses your local agent, can draft `interf.config.json` if it is missing, and can attach the current folder right away. It does not move or replace your files.
+
+Fastest sample loop:
 
 ```bash
-interf create interface
+cp -r examples/benchmark-demo /tmp/interf-demo
+cd /tmp/interf-demo
+interf init
 interf compile
 interf benchmark
 ```
 
-Fastest way to see the full loop:
+If you want a task-specific workspace for one job, add an interface:
 
 ```bash
-cp -r examples/benchmark-demo /tmp/benchmark-demo
-cd /tmp/benchmark-demo
-interf init
+interf create interface
 interf compile
 interf benchmark
 ```
 
-What success looks like on disk:
+## Start With One Small Eval
 
-- `interf/<kb>/` = shared compiled workspace over the folder
-- `interf/<kb>/interfaces/<name>/` = task-specific compiled workspace
-- `interf/benchmarks/runs/...` = saved benchmark evidence for that folder
+`interf.config.json` is where you write what must be true.
 
-## 5-Minute Example
+If the file is missing, `interf init` can draft it with you before the first compile.
 
-Try the full loop on the shipped sample folder:
+Use it for:
 
-```bash
-cp -r examples/benchmark-demo /tmp/interf-demo
-cd /tmp/interf-demo
-interf init
-interf compile
-interf benchmark
-```
-
-This sample already includes an `interf.config.json`, so you can see the compile and benchmark loop without writing your own evals first.
-
-## Simple Eval Example
+- top-level `evals` for the main folder
+- `interfaces[].evals` for task-specific checks
 
-The default public eval file is `interf.config.json` at the source-folder root.
+Example shape:
 
-Minimal example:
+Top-level `evals` are for the main folder. Each entry in `interfaces` adds evals for one dedicated job.
 
 ```json
 {
+  "evals": [
+    {
+      "question": "What was Bristol annual take-up in 2018, in millions of square feet?",
+      "answer": "About 0.5 million square feet. Accept answers between 0.3 and 0.6 if they clearly refer to Bristol annual take-up in 2018."
+    }
+  ],
   "interfaces": [
     {
-      "name": "weekly-briefing",
-      "about": "Summarize what changed, why it matters, and what to do next.",
+      "name": "market-briefing",
+      "about": "Prepare a short briefing from the office market report.",
       "evals": [
         {
-          "question": "From the compiled interface only, what changed and what should the operator do next?",
-          "answer": "A good answer says what changed, why it matters, and the next action.",
-          "strictness": "approximate"
+          "question": "What was Bristol availability in 2018, in millions of square feet?",
+          "answer": "About 0.6 million square feet. Accept answers between 0.5 and 0.7 if they clearly refer to Bristol availability in 2018."
+        },
+        {
+          "question": "Did Bristol annual take-up rise or fall between 2016 and 2018?",
+          "answer": "It fell. The chart shows roughly 0.7 to 0.8 million square feet in 2016 and about 0.5 million square feet in 2018."
         }
       ]
     }
@@ -170,221 +122,158 @@ Minimal example:
 }
 ```
 
-That is enough to start. You do not need a large benchmark harness to use Interf:
+Good first evals are small and practical:
 
-1. write one or two questions that matter
-2. say what a good answer must preserve
-3. compile the workspace
-4. run `interf benchmark`
+- one exact number from a chart, table, or filing
+- one short statement that should be true or false
+- one simple comparison across years, files, or sections
 
-If the compiled workspace does not beat raw files on those evals, do not trust it yet.
+Then run:
 
-## Use It With Your Agent
+```bash
+interf compile
+interf benchmark
+```
 
-For many users, the agent is the operator.
+If the benchmark does not show an improvement over raw files, keep iterating or move to the experiment loop described below.
 
-A practical agent-native loop looks like this:
+## Compare Three Things
 
-1. the agent gets a real task against a real folder
-2. it inspects raw files or prior benchmark evidence
-3. it drafts or updates evals in `interf.config.json`
-4. it asks the user to confirm the task truth when needed
-5. it runs compile + benchmark
-6. it only promotes the compiled workspace for real use once the benchmark says it helped
+Compare:
 
-Paste something like this into Claude Code, Codex, OpenClaw, or Hermes:
+1. the raw folder
+2. the workspace
+3. an interface for one specific job
 
-```text
-Install @interf/compiler, run `interf init` in this folder, choose the local agent executor, and compile the workspace.
+`interf benchmark` is how you compare those on the same evals.
 
-If `interf.config.json` is missing or incomplete, draft evals for what must be true for this task and ask me to confirm them before benchmarking.
+That gives you one clear question:
 
-Then run `interf benchmark` and tell me whether raw files or the compiled workspace performed better.
-```
+- is the raw folder enough?
+- does the workspace retrieve better?
+- does a dedicated interface do better than both?
 
-## What The Agent Sees
+## What `interf compile` Actually Does
 
-The compiled folder is the agent-facing product surface.
+`interf compile` runs a workflow over your folder.
 
-Important files in a knowledge base or interface:
+That workflow is the compilation pipeline:
 
-- `interf.json` = what this workspace is
-- `AGENTS.md` = canonical bootstrap and navigation
-- `CLAUDE.md` = generated compatibility mirror of `AGENTS.md`
-- `workflow/` = the editable local method package
-- `home.md` = entry document
-- `summaries/`, `knowledge/`, and `briefs/` = compiled outputs
+- read the files
+- write processed notes and navigation files
+- build the workspace your agent can use
+- optionally build an interface for one specific job
 
-Interf supports two agent modes:
+The default workflow is built in. If you want a different method, you can define your own workflow package and benchmark it on the same folder.
 
-- **executor mode**: the CLI launches a local agent to satisfy one stage contract during create, compile, or benchmark flows
-- **use mode**: a human opens the compiled knowledge base or interface and asks an agent to navigate the finished workspace
+## Experiment Loop
 
-Manual use looks like this:
+Interf Knowledge Compiler also supports an experiment loop above compile + benchmark.
 
-1. open the knowledge base or interface folder
-2. read `AGENTS.md`
-3. follow `workflow/use/query/SKILL.md`
-4. for interfaces, use local interface artifacts first, then the parent knowledge-base loop, then raw files if needed
+It runs controlled experiments against the same folder and the same evals. Each attempt reruns the compilation workflow, reruns the benchmark, and records what changed. It stops when:
 
-Interf does not require globally installed slash skills for workspace behavior. Local `workflow/.../SKILL.md` files are workspace instruction docs routed by `AGENTS.md` and stage contracts.
+- the evals pass
+- or the experiment budget is exhausted
 
-## Benchmark Proof
+In practice, that means:
 
-Interf is benchmark-first.
+- `retry_policy.max_attempts_per_profile` controls how many experiment attempts each compile profile gets
+- stronger diagnostic profiles can be used only after the default ones fail
+- the loop is still judged on the same eval truth from your folder
+- failure summaries can be captured between attempts for diagnosis
 
-The default eval file lives at the source-folder root:
+Today that advanced path is configured through eval packs and explained in the deeper docs. The workflow is the part you change. The experiment loop is the controller that runs those experiments against the same evals with a fixed attempt budget.
 
-```text
-source-folder/
-  interf.config.json
-```
+Use the simple loop first. Use the experiment loop when you want to test workflow or profile changes against the same evals until one passes or the attempt budget runs out.
 
-Saved benchmark runs live under:
+## Use It With Your Agent
 
-```text
-source-folder/
-  interf/
-    benchmarks/
-      runs/
-```
+If you already work through Claude Code, Codex, OpenClaw, or Hermes, the agent can run this loop for you.
 
-Use benchmarks to answer questions like:
+Paste something like this into Claude Code, Codex, OpenClaw, or Hermes:
 
-- does the compiled workspace beat raw files on this task?
-- which workflow wins on this folder?
-- which interface is best for this job?
-- which model performs best on the same compiled target?
+```text
+Install @interf/compiler, run `interf init` in this folder, and use the local agent executor.
 
-`interf benchmark` uses your evals, opens the compiled target like a real user session, asks the questions, and grades the answers. The point is not a hidden score. The point is a benchmark artifact you can inspect, diff, and rerun locally.
+If `interf.config.json` is missing, draft evals for what must be true for this task and ask me to confirm them.
 
-## Power Mode
+Then run `interf compile` and `interf benchmark`.
 
-Most users do not need to think about improvement loops.
+Tell me whether the processed workspace beat raw files, and only recommend it if it did.
+```
 
-The basic story is:
+That is the basic loop:
 
-1. compile
-2. benchmark
-3. trust the result only if it passes
+- the user or agent defines what must be true
+- Interf prepares processed data for retrieval
+- the benchmark shows whether that helped
 
-Power users and agent-native setups can go further:
+## What Gets Created
 
-- compare workflows on the same folder
-- compare models on the same compiled target
-- draft custom local workflows
-- rerun compile + benchmark until a task-specific interface passes
+After compile, Interf writes into `./interf/` beside your source files.
 
-That improvement loop is a real capability, but it is not the main thing users need to understand first.
+- `interf/<name>/` is the shared workspace over the folder
+- `interf/<name>/interfaces/<name>/` is a task-specific workspace for one job
+- `interf/benchmarks/runs/...` stores saved benchmark runs
 
-## Layout On Disk
+Inside those workspaces you will see things like:
 
-```text
-source-folder/
-  ...your files...
-  interf.config.json
-  interf/
-    workflows/
-    benchmarks/
-    {knowledge-base-name}/
-      interf.json
-      AGENTS.md
-      CLAUDE.md
-      home.md
-      workflow/
-      summaries/
-      knowledge/
-      interfaces/
-        {interface-name}/
-          interf.json
-          compile-plan.md
-          AGENTS.md
-          CLAUDE.md
-          home.md
-          workflow/
-          knowledge/
-          briefs/
-          summaries/
-```
+- summaries of source files
+- navigation notes and entrypoints for agents
+- task-specific outputs for one interface
+- benchmark artifacts you can inspect later
 
-## Core Commands
+In the CLI, the main Interf workspace is called a **knowledge base**. A task-specific workspace inside it is called an **interface**.
 
-- `interf init` = global setup first; if run inside a normal folder, it can also attach and compile a knowledge base there
-- `interf create interface` = create an interface for the current folder's knowledge base
-- `interf compile` = compile the current knowledge base or interface
-- `interf benchmark` = compare compiled knowledge bases or interfaces with evals from `interf.config.json` or an explicit spec file
+## When To Create An Interface
 
-Advanced commands still exist for workflow authoring and diagnostics:
+Start with one workspace.
 
-- `interf create workflow`
-- `interf doctor`
-- `interf status`
-- `interf verify <check>`
-- `interf reset <scope>`
+Create an interface when your agent needs outputs shaped for one specific job, for example:
 
-Useful run flags:
+- weekly briefing
+- diligence on a deal room
+- extracting chart values from research PDFs
+- a focused research assistant for one question set
 
-- `--model <name>` = pin the agent model for this run
-- `--profile <name>` = pass an agent-specific profile when supported
-- `--effort <level>` = override model reasoning effort
-- `--timeout-ms <ms>` = interrupt the local executor after this much inactivity
+If that workspace is enough for the job, you do not need an interface yet.
 
-## Workflows
+## Custom Workflows
 
-A workflow is a package, not just a prompt.
+Interf ships with a default workflow.
 
-It has two layers:
+If you want to change how compilation happens on your data, this is the part you customize:
 
-- machine layer: `workflow.json`
-- human/agent layer: `workflow/` docs
-
-Typical reusable workflow package:
-
-```text
-interf/workflows/knowledge-base/<workflow-id>/
-  workflow.json
-  README.md
-  create/
-    SKILL.md
-  compile/
-    stages/
-      <stage-id>/
-        SKILL.md
-  use/
-    query/
-      SKILL.md
+```bash
+interf create workflow
+interf verify workflow --path <path>
 ```
 
-Interf keeps the public command surface stable while letting workflows vary the internal stage pipeline. The engine still owns contract kinds, required artifacts, and state flow.
+Then benchmark that workflow on the same folder and the same evals.
 
-Current shipped policy:
+Workflow package docs live in [docs/workflow-spec.md](./docs/workflow-spec.md).
 
-- built-in knowledge-base workflows: `interf`, `karpathy`
-- built-in interface workflow: `interf`
-- if you need a custom interface method, create a local workflow package and benchmark it before treating it as better than the default
-
-## Builder Docs
-
-If you want to create your own workflows, start here:
-
-1. [`docs/workflow-spec.md`](./docs/workflow-spec.md)
-2. [`docs/runtime-contract.md`](./docs/runtime-contract.md)
-3. [`docs/architecture.md`](./docs/architecture.md)
-4. [`docs/eval-loop.md`](./docs/eval-loop.md)
+## Core Commands
 
-Contributor and release-testing commands live in [`CONTRIBUTING.md`](./CONTRIBUTING.md).
+- `interf init` = choose your local executor and optionally attach the current folder
+- `interf create knowledge-base` = create the shared processed workspace for this folder
+- `interf create interface` = create a task-specific workspace on top
+- `interf create workflow` = create a reusable local workflow package
+- `interf compile` = build the current workspace
+- `interf benchmark` = compare raw files vs processed workspaces on your evals
+- `interf doctor` = check local executor setup
+- `interf verify <check>` = run deterministic checks on major workflow steps
+- `interf reset <scope>` = remove generated state while keeping source files
 
-## Design Choices
+## More Docs
 
-- filesystem-first, not service-first
-- raw files remain the truth
-- compiled workspaces remain visible on disk
-- workflow packages over hidden orchestration
-- contract-checked stages instead of prompt-only trust
-- benchmarkability as a core product feature
+- [docs/workflow-spec.md](./docs/workflow-spec.md) for custom workflow packages
+- [docs/runtime-contract.md](./docs/runtime-contract.md) for the exact on-disk contract
+- [docs/architecture.md](./docs/architecture.md) for the deeper system model
+- [docs/eval-loop.md](./docs/eval-loop.md) for the advanced benchmark and experiment loop
 
-Interf is not trying to win by hiding complexity. It is trying to make the method visible, enforceable, and comparable.
+Maintainers should use [CONTRIBUTING.md](./CONTRIBUTING.md) for test and release gates.
 
 ## License
 
-Code is licensed under Apache 2.0. The `Interf` name and branding are reserved; see [`TRADEMARKS.md`](./TRADEMARKS.md).
+Code is licensed under Apache 2.0. The `Interf` name and branding are reserved; see [TRADEMARKS.md](./TRADEMARKS.md).

package/dist/commands/create.d.ts

@@ -1,5 +1,6 @@
 import type { CommandModule } from "yargs";
 import type { WorkflowExecutionProfile } from "../lib/executors.js";
+import type { SourceInterfaceConfig } from "../lib/schema.js";
 export { type WorkflowWizardPrompts, buildKnowledgeBaseWorkflowOptions, buildInterfaceWorkflowOptions, buildStandaloneInterfaceWorkflowOptions, selectWorkflowTargetType, formatWorkflowLabel, CREATE_NEW_WORKFLOW_VALUE, chooseKnowledgeBaseWorkflow, chooseInterfaceWorkflow, createWorkflowWizard, createKnowledgeBaseWorkflowWizard, createInterfaceWorkflowWizard, } from "./create-workflow-wizard.js";
 export declare const createCommand: CommandModule;
 export declare function createKnowledgeBaseWizard(options?: {
@@ -12,5 +13,6 @@ export declare function createInterfaceWizard(options?: {
     knowledgeBasePathOverride?: string;
     knowledgeBaseNameOverride?: string;
     executionProfile?: WorkflowExecutionProfile;
+    suggestedInterface?: Pick<SourceInterfaceConfig, "name" | "about">;
 }): Promise<void>;
 //# sourceMappingURL=create.d.ts.map

package/dist/commands/create.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"create.d.ts","sourceRoot":"","sources":["../../src/commands/create.ts"],"names":[],"mappings":"AAiCA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AAO3C,OAAO,KAAK,EAAE,wBAAwB,EAAE,MAAM,qBAAqB,CAAC;AAOpE,OAAO,EACL,KAAK,qBAAqB,EAC1B,iCAAiC,EACjC,6BAA6B,EAC7B,uCAAuC,EACvC,wBAAwB,EACxB,mBAAmB,EACnB,yBAAyB,EACzB,2BAA2B,EAC3B,uBAAuB,EACvB,oBAAoB,EACpB,iCAAiC,EACjC,6BAA6B,GAC9B,MAAM,6BAA6B,CAAC;AAwErC,eAAO,MAAM,aAAa,EAAE,aA8D3B,CAAC;AAEF,wBAAsB,yBAAyB,CAC7C,OAAO,GAAE;IACP,KAAK,CAAC,EAAE,OAAO,CAAC;IAChB,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,gBAAgB,CAAC,EAAE,wBAAwB,CAAC;CACxC,iBAwJP;AAED,wBAAsB,qBAAqB,CACzC,OAAO,GAAE;IACP,KAAK,CAAC,EAAE,OAAO,CAAC;IAChB,yBAAyB,CAAC,EAAE,MAAM,CAAC;IACnC,yBAAyB,CAAC,EAAE,MAAM,CAAC;IACnC,gBAAgB,CAAC,EAAE,wBAAwB,CAAC;CACxC,iBAqQP"}
+{"version":3,"file":"create.d.ts","sourceRoot":"","sources":["../../src/commands/create.ts"],"names":[],"mappings":"AAiCA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AAO3C,OAAO,KAAK,EAAE,wBAAwB,EAAE,MAAM,qBAAqB,CAAC;AAEpE,OAAO,KAAK,EAAE,qBAAqB,EAAE,MAAM,kBAAkB,CAAC;AAO9D,OAAO,EACL,KAAK,qBAAqB,EAC1B,iCAAiC,EACjC,6BAA6B,EAC7B,uCAAuC,EACvC,wBAAwB,EACxB,mBAAmB,EACnB,yBAAyB,EACzB,2BAA2B,EAC3B,uBAAuB,EACvB,oBAAoB,EACpB,iCAAiC,EACjC,6BAA6B,GAC9B,MAAM,6BAA6B,CAAC;AAwErC,eAAO,MAAM,aAAa,EAAE,aA8D3B,CAAC;AAEF,wBAAsB,yBAAyB,CAC7C,OAAO,GAAE;IACP,KAAK,CAAC,EAAE,OAAO,CAAC;IAChB,gBAAgB,CAAC,EAAE,OAAO,CAAC;IAC3B,gBAAgB,CAAC,EAAE,wBAAwB,CAAC;CACxC,iBA8JP;AAED,wBAAsB,qBAAqB,CACzC,OAAO,GAAE;IACP,KAAK,CAAC,EAAE,OAAO,CAAC;IAChB,yBAAyB,CAAC,EAAE,MAAM,CAAC;IACnC,yBAAyB,CAAC,EAAE,MAAM,CAAC;IACnC,gBAAgB,CAAC,EAAE,wBAAwB,CAAC;IAC5C,kBAAkB,CAAC,EAAE,IAAI,CAAC,qBAAqB,EAAE,MAAM,GAAG,OAAO,CAAC,CAAC;CAC/D,iBAuQP"}