@interf/compiler 0.1.12 → 0.2.1

package/README.md

@@ -1,272 +1,277 @@
 # Interf
 
-The open-source knowledge compiler.
+Interf Knowledge Compiler uses local agents such as Claude Code and Codex to run a data-processing workflow over your files.
 
-Interf compiles folders into knowledge bases and task-specific interfaces: agent-ready workspaces with proof, structure, and benchmarks.
+It creates a workspace with notes and navigation so the agent can see what is in the folder and what to retrieve.
 
-- compile any folder into a knowledge base
-- create focused interfaces for specific tasks
-- run evals and benchmarks on your own files
+Then you test that workspace on your evals.
 
-Most LLM knowledge-base repos optimize for a demo. Interf optimizes for proof. It keeps your files on disk, compiles a visible folder an agent can actually use, and makes workflows compete on your evals instead of on marketing claims.
+- your files stay on your machine
+- you choose the local agent
+- you decide what must be true
 
-## Why Interf
+Agents start missing things when a task spans PDFs, charts, and several files in one folder. That usually shows up when the job depends on:
 
-Interf is built around three ideas:
+- reading reports and filings
+- extracting a number from a chart
+- understanding what is inside a folder before doing work
+- pulling context together across several files
+- checking the raw source when the answer has to be exact
 
-- the product surface is the compiled folder, not a hidden service
-- the workflow should leave proof of work on disk
-- the method should be benchmarkable on your task
+The workspace exists so the agent does not have to rediscover the folder from scratch on every run.
 
-That gives you a simple loop:
+The point is proof on your data, not generic AI claims.
 
-1. point Interf at a folder
-2. compile a knowledge base
-3. create an interface for a job
-4. run evals and benchmarks to see what actually works
+The simplest way to use Interf is to compare the same task before and after compilation:
 
-## Core concepts
+- run the task on the raw folder
+- compile the folder with Interf
+- run the same task again from the workspace
+- if you want a recorded pass/fail result, add evals and run `interf benchmark`
 
-- **Source folder**: your real files stay where they are
-- **Knowledge base**: `interf/{name}/`, the shared compiled layer over that folder
-- **Interface**: `interf/{name}/interfaces/{interface-name}/`, a task-specific surface on top of one knowledge base
-- **Workflow**: the reusable method package that defines the compile pipeline
-- **Benchmark**: running evals across compiled knowledge bases or interfaces on the same folder
+## Quick Start
 
-One source folder can host multiple knowledge bases under `interf/` when you want to compare workflows like `interf` vs `karpathy` on the same data.
+Requirements:
 
-## Interf primitives
+- Node.js 20+
+- a local coding agent: Claude Code or Codex
 
-Interf gives you a few strong primitives instead of a giant abstraction layer:
+Install and check setup:
 
-- **workflow package**: `workflow.json` plus local `workflow/` docs define the method
-- **stage contract**: every compile stage gets a deterministic acceptance boundary
-- **declarative acceptance**: workflows can declare extra acceptance rules in `workflow.json`
-- **CLI enforcement**: the CLI checks whether a stage actually complied instead of trusting the agent's summary
-- **benchmark specs**: file-based evals let you compare workflows and interfaces on the same folder
-
-That is the core product promise:
-
-- define what the agent should do in plain English
-- give the agent local workspace docs and stage contracts
-- validate the result deterministically
-
-For workflow authors, the important surface is:
-
-- `workflow.json`
-- `workflow/create/`
-- `workflow/compile/stages/<stage>/`
-- `workflow/use/query/`
-- [`docs/workflow-spec.md`](./docs/workflow-spec.md)
-
-## What the agent sees
-
-The compiled folder is the agent-facing product surface: an agent-ready workspace.
-
-Important files in a KB or interface:
+```bash
+npm install -g @interf/compiler
+interf doctor
+```
 
-- `interf.json` = what this workspace is
-- `AGENTS.md` = where to start and how to navigate
-- `workflow/` = the editable local method package
-- `home.md` = entry document
-- `summaries/`, `knowledge/`, and `briefs/` = compiled outputs
+Then run Interf in any folder:
 
-Manual query/use works like this:
+```bash
+cd ~/my-folder
+interf init
+interf compile
+interf benchmark
+```
 
-- open the KB or interface folder
-- read `AGENTS.md`
-- follow `workflow/use/query/SKILL.md`
-- for interfaces, use local interface artifacts first, then the parent KB loop, then raw files if needed
+That is the whole first loop:
 
-Interf does not require globally installed slash skills for workspace behavior. Local `workflow/.../SKILL.md` files are workspace instruction docs routed by `AGENTS.md` and stage contracts.
+- point Interf at a folder you already have
+- compile the workspace
+- ask the agent to use it
+- add evals later when you want to check the result more formally
 
-## Quick start
+`interf init` chooses your local agent and can attach the current folder right away. It does not move or replace your files.
 
-Install the published package:
+Fastest sample loop:
 
 ```bash
-npm install -g @interf/compiler
+cp -r examples/benchmark-demo /tmp/interf-demo
+cd /tmp/interf-demo
+interf init
+interf compile
+interf benchmark
 ```
 
-Or install from source while contributing:
+If you want a task-specific workspace for one job, add an interface:
 
 ```bash
-npm install
-npm run build
-npm install -g .
+interf create interface
+interf compile
+interf benchmark
 ```
 
-Initialize Interf in any folder:
-
-```bash
-cd ~/my-notes
-interf init
+## Start With One Small Eval
+
+`interf.config.json` is where you write what must be true.
+
+Use it for:
+
+- broad checks on the folder as a whole
+- task-specific checks for one interface
+
+Example shape:
+
+```json
+{
+  "knowledge_base": {
+    "name": "uk-office-market-report",
+    "evals": [
+      {
+        "question": "What was Bristol annual take-up in 2018, in millions of square feet?",
+        "answer": "About 0.5 million square feet. Accept answers between 0.3 and 0.6 if they clearly refer to Bristol annual take-up in 2018."
+      }
+    ]
+  },
+  "interfaces": [
+    {
+      "name": "market-briefing",
+      "about": "Prepare a short briefing from the office market report.",
+      "evals": [
+        {
+          "question": "What was Bristol availability in 2018, in millions of square feet?",
+          "answer": "About 0.6 million square feet. Accept answers between 0.5 and 0.7 if they clearly refer to Bristol availability in 2018."
+        },
+        {
+          "question": "Did Bristol annual take-up rise or fall between 2016 and 2018?",
+          "answer": "It fell. The chart shows roughly 0.7 to 0.8 million square feet in 2016 and about 0.5 million square feet in 2018."
+        }
+      ]
+    }
+  ]
+}
 ```
 
-That flow can:
+Good first evals are small and practical:
 
-- choose an executor like Claude Code or Codex
-- optionally install global helper skills
-- attach the current folder as a knowledge base
-- compile the knowledge base immediately
+- one exact number from a chart, table, or filing
+- one short statement that should be true or false
+- one simple comparison across years, files, or sections
 
-Then you can:
+Then run:
 
 ```bash
-interf create interface
 interf compile
 interf benchmark
 ```
 
-## Example layout
+If the benchmark does not show an improvement over raw files, keep iterating or move to the experiment loop described below.
 
-```text
-source-folder/
-  ...your files...
-  interf/
-    workflows/
-    benchmarks/
-    {knowledge-base-name}/
-      interf.json
-      AGENTS.md
-      CLAUDE.md
-      home.md
-      workflow/
-      summaries/
-      knowledge/
-      interfaces/
-        {interface-name}/
-          interf.json
-          compile-plan.md
-          AGENTS.md
-          CLAUDE.md
-          home.md
-          workflow/
-          knowledge/
-          briefs/
-          summaries/
-```
+## Compare Three Things
 
-## Commands
+Compare:
 
-- `interf init` = global setup first; if run inside a normal folder, it can also attach and compile a knowledge base there
-- `interf create` = chooser when type is omitted
-- `interf create knowledge-base` = attach current folder
-- `interf create interface` = create an interface for the current folder's knowledge base
-- `interf create workflow` = create a reusable workflow package
-- `interf compile` = compile the current knowledge base or interface
-- `interf benchmark` = compare compiled knowledge bases or interfaces with file-based evals
-- `interf doctor` = preflight local executor setup before a real compile
-- `interf status` = show deterministic health
-- `interf verify <check>` = internal deterministic referee for major workflow steps
-- `interf reset <scope>` = reset generated state while keeping source files
+1. the raw folder
+2. the workspace
+3. an interface for one specific job
 
-## Workflows
+`interf benchmark` is how you compare those on the same evals.
 
-A workflow is a package, not just a prompt.
+That gives you one clear question:
 
-It has two layers:
+- is the raw folder enough?
+- does the workspace retrieve better?
+- does a dedicated interface do better than both?
 
-- machine layer: `workflow.json`
-- human/agent layer: `workflow/` docs
+## What `interf compile` Actually Does
 
-Typical reusable workflow package:
+`interf compile` runs a workflow over your folder.
 
-```text
-interf/workflows/knowledge-base/<workflow-id>/
-  workflow.json
-  README.md
-  create/
-    SKILL.md
-  compile/
-    stages/
-      <stage-id>/
-        SKILL.md
-  use/
-    query/
-      SKILL.md
-```
+That workflow is the compilation pipeline:
+
+- read the files
+- write processed notes and navigation files
+- build the workspace your agent can use
+- optionally build an interface for one specific job
+
+The default workflow is built in. If you want a different method, you can define your own workflow package and benchmark it on the same folder.
+
+## Experiment Loop
+
+Interf Knowledge Compiler also supports an experiment loop above compile + benchmark.
 
-Interf keeps the public command surface stable while letting workflows vary the internal stage pipeline. The engine still owns contract kinds, required artifacts, and state flow.
+It runs controlled experiments against the same folder and the same evals. Each attempt reruns the compilation workflow, reruns the benchmark, and records what changed. It stops when:
 
-## Benchmarks and evals
+- the evals pass
+- or the experiment budget is exhausted
 
-Interf is benchmark-first.
+In practice, that means:
 
-You can:
+- `retry_policy.max_attempts_per_profile` controls how many experiment attempts each compile profile gets
+- stronger diagnostic profiles can be used only after the default ones fail
+- the loop is still judged on the same eval truth from your folder
+- failure summaries can be captured between attempts for diagnosis
 
-- build multiple knowledge bases over the same folder
-- compare workflows on the same source set
-- compare interfaces for the same business task
-- inspect proofs, outputs, and costs locally
+Today that advanced path is configured through eval packs and explained in the deeper docs. The workflow is the part you change. The experiment loop is the controller that runs those experiments against the same evals with a fixed attempt budget.
 
-Reusable benchmark specs and saved runs live under:
+Use the simple loop first. Use the experiment loop when you want to test workflow or profile changes against the same evals until one passes or the attempt budget runs out.
+
+## Use It With Your Agent
+
+If you already work through Claude Code, Codex, OpenClaw, or Hermes, the agent can run this loop for you.
+
+Paste something like this into Claude Code, Codex, OpenClaw, or Hermes:
 
 ```text
-source-folder/
-  interf/
-    benchmarks/
-      knowledge-base/*.json
-      interface/*.json
-      runs/
+Install @interf/compiler, run `interf init` in this folder, and use the local agent executor.
+
+If `interf.config.json` is missing, draft evals for what must be true for this task and ask me to confirm them.
+
+Then run `interf compile` and `interf benchmark`.
+
+Tell me whether the processed workspace beat raw files, and only recommend it if it did.
 ```
 
-This is the trust loop: don't trust a repo because it says its knowledge base is better. Run the benchmark on your folder.
+That is the basic loop:
 
-## Builder docs
+- the user or agent defines what must be true
+- Interf prepares processed data for retrieval
+- the benchmark shows whether that helped
 
-If you want to create your own workflows, start here:
+## What Gets Created
 
-1. [`docs/workflow-spec.md`](./docs/workflow-spec.md)
-2. [`docs/runtime-contract.md`](./docs/runtime-contract.md)
-3. [`docs/architecture.md`](./docs/architecture.md)
+After compile, Interf writes into `./interf/` beside your source files.
 
-## Maintainer test loop
+- `interf/<name>/` is the shared workspace over the folder
+- `interf/<name>/interfaces/<name>/` is a task-specific workspace for one job
+- `interf/benchmarks/runs/...` stores saved benchmark runs
 
-Smoke suite:
+Inside those workspaces you will see things like:
 
-```bash
-npm test
-```
+- summaries of source files
+- navigation notes and entrypoints for agents
+- task-specific outputs for one interface
+- benchmark artifacts you can inspect later
 
-Real executor end-to-end:
+In the CLI, the main Interf workspace is called a **knowledge base**. A task-specific workspace inside it is called an **interface**.
 
-```bash
-npm run test:e2e
-npm run test:e2e:compare
-```
+## When To Create An Interface
 
-Cached quick real-executor loop:
+Start with one workspace.
 
-```bash
-npm run test:e2e:quick
-npm run test:full
-```
+Create an interface when your agent needs outputs shaped for one specific job, for example:
+
+- weekly briefing
+- diligence on a deal room
+- extracting chart values from research PDFs
+- a focused research assistant for one question set
 
-Underlying acceptance commands:
+If that workspace is enough for the job, you do not need an interface yet.
+
+## Custom Workflows
+
+Interf ships with a default workflow.
+
+If you want to change how compilation happens on your data, this is the part you customize:
 
 ```bash
-npm run test:acceptance-live
-npm run test:acceptance-compare
-npm run test:acceptance-cache:refresh
-npm run test:acceptance-quick:create-interface
-npm run test:acceptance-quick:query-interface
+interf create workflow
+interf verify workflow --path <path>
 ```
 
-The cached quick fixture lives under `.interf-test-cache/latest-quick/`.
-`npm test` is the fast smoke/integration suite. When you want a real agent/executor end-to-end run, use `npm run test:e2e` or `npm run test:e2e:quick`.
-`npm run test:full` is the convenient day-to-day command: smoke suite plus cached quick real-executor checks.
+Then benchmark that workflow on the same folder and the same evals.
+
+Workflow package docs live in [docs/workflow-spec.md](./docs/workflow-spec.md).
+
+## Core Commands
+
+- `interf init` = choose your local executor and optionally attach the current folder
+- `interf create knowledge-base` = create the shared processed workspace for this folder
+- `interf create interface` = create a task-specific workspace on top
+- `interf create workflow` = create a reusable local workflow package
+- `interf compile` = build the current workspace
+- `interf benchmark` = compare raw files vs processed workspaces on your evals
+- `interf doctor` = check local executor setup
+- `interf verify <check>` = run deterministic checks on major workflow steps
+- `interf reset <scope>` = remove generated state while keeping source files
 
-## Design choices
+## More Docs
 
-- filesystem-first, not service-first
-- workflow packages over hidden orchestration
-- contract-checked stages instead of prompt-only trust
-- benchmarkability as a core product feature
-- local control: your files stay on disk and run in your environment
+- [docs/workflow-spec.md](./docs/workflow-spec.md) for custom workflow packages
+- [docs/runtime-contract.md](./docs/runtime-contract.md) for the exact on-disk contract
+- [docs/architecture.md](./docs/architecture.md) for the deeper system model
+- [docs/eval-loop.md](./docs/eval-loop.md) for the advanced benchmark and experiment loop
 
-Interf is not trying to win by hiding complexity. It is trying to make the method visible, enforceable, and comparable.
+Maintainers should use [CONTRIBUTING.md](./CONTRIBUTING.md) for test and release gates.
 
 ## License
 
-Code is licensed under Apache 2.0. The `Interf` name and branding are reserved; see [`TRADEMARKS.md`](./TRADEMARKS.md).
+Code is licensed under Apache 2.0. The `Interf` name and branding are reserved; see [TRADEMARKS.md](./TRADEMARKS.md).

package/dist/commands/benchmark.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"benchmark.d.ts","sourceRoot":"","sources":["../../src/commands/benchmark.ts"],"names":[],"mappings":"AAuBA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AAkY3C,eAAO,MAAM,gBAAgB,EAAE,aAmD9B,CAAC"}
+{"version":3,"file":"benchmark.d.ts","sourceRoot":"","sources":["../../src/commands/benchmark.ts"],"names":[],"mappings":"AA0BA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AA2W3C,eAAO,MAAM,gBAAgB,EAAE,aA8D9B,CAAC"}