@interf/compiler 0.3.1 → 0.3.3

package/README.md

@@ -1,19 +1,29 @@
 # Interf
 
-Open-source knowledge compiler for your files.
+Open-source toolkit for preparing local files for agents.
 
-Interf measures and improves how accurately local agents answer questions from your files.
+Turn PDFs, docs, spreadsheets, and notes into a local workspace your agent can navigate, verify, and answer from.
 
 If you use Claude Code, Codex, OpenClaw, Hermes, or your own local agent setup on folders full of PDFs, docs, spreadsheets, and notes, the failure often shows up late: missed evidence, shallow analysis, bad comparisons, or answers that sound confident but are wrong.
 
-Interf lets you write a few questions and expected answers about your files, test the raw files first if you want a baseline, build a compiled workspace on top of those files, and see whether the result actually passes.
+Interf lets you set a few truth checks on a dataset, measure the raw baseline first if you want it, compile a workspace on top of that dataset, and see whether the result actually passes.
 
-- your files stay on your machine
-- you choose the local agent
+- your dataset stays on your machine
+- BYOAI: use Claude Code, Codex, OpenClaw, Hermes, or your own local setup
 - your raw files stay the source of truth
-- Interf adds a file-based layer on top
+- the workflow package is the reusable method
+- the compiler runtime executes that workflow package and builds a file-based layer on top
 
-It runs local data-processing pipelines with your agents as executors and produces a compiled workspace: a file-based layer on top of your raw files that agents can navigate, inspect, and work from.
+`interf compile` runs a workflow package with your agents as executors and produces a compiled workspace: a file-based layer on top of your raw files that agents can navigate, inspect, and work from.
+
+Each compiled workspace carries its own `raw/` snapshot, so agents can work from one self-contained folder instead of reaching back into the source-folder control plane.
+
+Interf also projects native agent shells from that workspace:
+
+- the compiled workspace itself is the folder your agent works from
+- each compile stage runs inside its own ephemeral execution shell with stage-specific instructions and mounted zone aliases under `inputs/` and `outputs/`
+
+The main reusable artifact is the workflow package. In the advanced looped mode, Interf can keep rerunning that workflow package against the same dataset and truth checks until it either passes or exhausts the attempt budget.
 
 ## Quick Start
 
@@ -37,12 +47,13 @@ interf
 
 If you want to see the config shape first, this is what Interf writes:
 
-```json
+```jsonc
 {
   "workspaces": [
     {
-      "name": "default",
+      "name": "my-workspace",
       "about": "General compiled workspace for the quarterly results folder.",
+      "max_attempts": 3, // rerun compile + test until this workspace passes the saved truth checks or hits this limit
       "checks": [
         {
           "question": "What full-year revenue range did the company maintain?",
@@ -68,16 +79,25 @@ interf test
 
 The first guided run can:
 
-- save a few questions and expected answers for this folder
+- save a few truth checks for the dataset in this folder
 - run a baseline test on the raw files
 - compile the workspace
+- optionally keep compiling and retesting until it passes or reaches the attempt limit
 - run the same test against the compiled workspace
 
 That gives you three concrete things:
 
-- `interf/workspaces/default/` with the compiled workspace for your files
+- `interf/workspaces/my-workspace/` with the compiled workspace for your dataset
 - `interf/benchmarks/runs/...` with the saved test result
-- a pass/fail score on the same questions and expected answers you wrote
+- a pass/fail score on the same truth checks you wrote
+
+Saved test runs keep the details you need later:
+
+- whether the run tested `raw`, `workspace`, or both
+- per-question pass/fail results
+- the saved run path under `interf/benchmarks/runs/...`
+- the preserved sandbox path when a failed run is kept for review or you use `interf test --keep-sandboxes`
+- executor metadata such as agent, command, model, effort, and profile when available
 
 If `interf.config.json` is missing, `interf` or `interf init` can draft it with you before the first compile. If Interf cannot find your local agent or compile setup, run:
 
@@ -87,7 +107,7 @@ interf doctor
 
 The first flow is:
 
-- write down a few questions your agent should be able to answer from your files
+- write down a few truth checks your agent should be able to pass on the dataset
 - let `interf` or `interf init` save those checks in `interf.config.json`
 - optionally run a baseline test on the raw files
 - run `interf compile` to build the compiled workspace
@@ -100,7 +120,7 @@ The first flow is:
 Interf is built around a few simple design principles:
 
 - `Explicit`: the output is visible and inspectable, not hidden memory
-- `Local`: your files stay on your machine
+- `Local`: your dataset stays on your machine
 - `File over app`: the output is just files, so you can use your editor, Unix tools, Obsidian, or your own software on top
 - `BYOAI`: use Claude Code, Codex, OpenClaw, Hermes, or your own model
 
@@ -116,22 +136,22 @@ interf compile
 interf test
 ```
 
-## Start With A Few Questions
+## Start With Your Own Truth Checks
+
+Start with your own truth checks: questions where you already know the correct answer from the dataset.
 
-`interf.config.json` is where you write the questions and expected answers for a folder.
+`interf.config.json` is where you save those truth checks for a dataset folder.
 
 That file uses one `workspaces` array:
 
 - most folders only need one workspace
-- add another workspace only if you want a separate compiled setup with different checks
+- add another workspace only if you want a separate compiled setup with different truth checks
 - each workspace carries its own `checks`
+- each workspace can optionally carry `max_attempts` for the self-improving compile loop
 
 If the file is missing, `interf init` can draft it with you before the first compile. You can edit it any time.
 
-That example is just `interf.config.json`.
-Advanced retry settings do not live there.
-
-Good first checks are small and practical:
+Good first truth checks are small and practical:
 
 - one exact number from a chart, table, or filing
 - one short statement that should be true or false
@@ -144,34 +164,49 @@ interf compile
 interf test
 ```
 
-## What `interf test` Compares
+## What `interf test` Does
 
-`interf test` scores either the raw files, a compiled workspace, or both on the same saved questions and expected answers.
+`interf test` scores either the raw files, a compiled workspace, or both on the same saved truth checks.
 
 It lets you answer a simple question:
 
 - what is the current baseline on the raw files?
 - does this compiled workspace improve on that baseline?
-- which compiled workspace or workflow performs better on the same folder?
-- does a separate workspace with different checks work better for that job?
+- which compiled workspace or workflow performs better on the same dataset?
+- does a separate workspace with different truth checks work better for that job?
+
+By default it loads truth checks from `interf.config.json`, can run a raw baseline in an isolated raw-files sandbox, can test eligible compiled workspaces under `interf/workspaces/`, and saves the run under `interf/benchmarks/runs/`.
+
+For live runs:
+
+- raw tests execute from a sanitized raw-only sandbox
+- compiled-workspace tests execute from a copied workspace sandbox with embedded sanitized `raw/`
+- neither sandbox includes `interf.config.json` or the source-folder `interf/` control plane
+- failed test sandboxes are kept automatically for review
+- `interf test --keep-sandboxes` keeps every sandbox, even successful ones
 
-By default it loads checks from `interf.config.json`, can run a raw baseline in an isolated raw-files sandbox, can test eligible compiled workspaces under `interf/workspaces/`, and saves the run under `interf/benchmarks/runs/`.
+Each saved run includes:
 
-If you run `interf test` from inside a workspace, it uses that workspace's checks and tests that workspace. If you run it from the source folder, it lets you choose a saved workspace and then choose raw files, the compiled workspace, or both.
+- the benchmark target and mode
+- per-question results and traces
+- the preserved sandbox path when one was kept
+- the executor metadata for that run
 
-Live test runs use an isolated sandbox. For raw baselines, Interf gives the agent sanitized raw files only. For compiled-workspace tests, it gives the agent a copied workspace plus sanitized raw files. The source-folder control plane, `interf.config.json`, and saved test runs are not part of those sandboxes.
+If you run `interf test` from inside a workspace, it uses that workspace's truth checks and tests that workspace. If you run it from the source folder, it lets you choose a saved workspace and then choose raw files, the compiled workspace, or both.
+
+Live test runs use an isolated sandbox. For raw baselines, Interf gives the agent sanitized raw files only. For compiled-workspace tests, it gives the agent a copied workspace with its own embedded sanitized `raw/` fallback via `source.path`. The source-folder control plane, `interf.config.json`, and saved test runs are not part of those sandboxes.
 
 If you need repeated isolated experiments across workflows or models, use the advanced eval-pack runner in [docs/eval-loop.md](./docs/eval-loop.md).
 
 ## What `interf compile` Does
 
-`interf compile` runs the Interf data-processing pipeline over your files.
+`interf compile` runs the selected workflow package over a dataset.
 
 By default, that means:
 
 - summarize the source files into per-file evidence notes
 - structure the cross-file knowledge layer into entities, claims, and indexes
-- shape the final workspace around its saved focus and questions
+- shape the final workspace around its saved focus and truth checks
 
 In other words, the built-in workflow is:
 
@@ -179,9 +214,30 @@ In other words, the built-in workflow is:
 2. `structure`
 3. `shape`
 
-In public docs, `pipeline` is the thing Interf runs. `workflow` is the saved method that defines or customizes that pipeline.
+The default workflow is built in. If you want a different method, you can define your own workflow package and test it on the same dataset.
+
+Each workflow package combines:
+
+- `workflow.json` for stage order, compiler API target, and deterministic contract mapping
+- `workspace.schema.json` for the deterministic output shape of the compiled workspace
+- stage `reads` / `writes` declarations that reference schema-defined zone ids
+- local `SKILL.md` files as the authoring source for query and stage-execution behavior
+
+Interf then projects that package into the native agent surfaces it actually runs:
+
+- the compiled workspace gets a generated native query shell
+- each compile stage gets a generated native execution shell
+- that shell keeps its own `AGENTS.md`, `CLAUDE.md`, and native local skills
+- schema-declared workspace zones are mounted both at their workflow-relative paths and as shell-local `inputs/<zone-id>` / `outputs/<zone-id>` aliases
+- the workspace root itself is not linked into the shell
+
+If a workspace has `max_attempts`, or if you run `interf compile --max-attempts <n>`, Interf can keep compiling, testing, and retrying until that workspace passes or reaches the attempt limit. If several attempts fail, Interf keeps the best-performing compiled workspace from that run.
 
-The default workflow is built in. If you want a different method, you can define your own workflow package and benchmark it on the same folder.
+For stage-level review:
+
+- successful stage shells are pruned by default
+- failed stage shells stay under `.interf/execution-shells/`
+- `interf compile --keep-stage-shells` keeps every stage shell so you can inspect the exact native instruction surface, mounted inputs, and mounted outputs for each stage
 
 ## What Gets Created
 
@@ -192,6 +248,9 @@ After compile, Interf writes into `./interf/` beside your source files.
 
 Inside those workspaces you will see things like:
 
+- a workspace-local `raw/` snapshot for direct evidence and verification
+- `workflow/workspace.schema.json` describing the deterministic output shape
+- `AGENTS.md`, `CLAUDE.md`, and generated local query skills for manual agent use
 - summaries of source files
 - navigation notes and entrypoints for agents
 - cross-file knowledge notes
@@ -199,24 +258,26 @@ Inside those workspaces you will see things like:
 
 The compiled workspace is just a normal folder. Open it in your editor, in your agent, or in Obsidian if you want the graph view.
 
+For manual use, the workspace is the native agent shell. The editable authoring source for that shell lives under `workflow/use/query/`, and Interf generates native local query skills from it inside the workspace.
+
 If you use Obsidian, open `interf/workspaces/<name>/` as the vault for the compiled workspace.
 
 ## Terminology
 
 Public terms:
 
-- `your files` = the source folder Interf reads from
-- `questions and expected answers` = the checks you want your agent to pass
-- `checks` = the pass/fail questions each workspace should satisfy
-- `test` = run the saved questions and get a score
+- `dataset` = the collection of files Interf prepares
+- `truth check` = one question plus the expected correct answer
+- `checks` = the config field that stores those truth checks
+- `test` = run the saved truth checks and get a score
 - `compiled workspace` = the output Interf produces on top of a folder
 - `workspace` = one compiled setup with its own checks
 
 Technical terms:
 
-- `source folder` = the raw files Interf reads from
+- `source folder` = the dataset root Interf reads from
 - `benchmark` = the technical alias and saved-run layer behind `interf test`
-- `workflow` = the saved method that defines or customizes the pipeline
+- `workflow package` = the saved method that defines or customizes how compile runs
 - `.interf/` = runtime state, proofs, and health artifacts
 
 ## Advanced: Separate Workspaces
@@ -232,22 +293,33 @@ Create another only when you want a different compiled setup with different chec
 
 Why create another one:
 
-- it keeps a separate set of questions and expected answers
+- it keeps a separate set of truth checks
 - it gives that job its own compiled output under `interf/workspaces/<name>/`
 - it lets you test that job separately
 
 ## Advanced: Keep Improving Until It Passes
 
-Interf also supports an advanced experiment path above the normal build + test flow.
+Interf also supports a deeper loop above the normal compile + test flow.
+
+The normal workspace flow already supports `max_attempts` inside `interf.config.json` or `interf compile --max-attempts <n>`.
 
-Give it the same folder and the same checks. Interf can keep rerunning compile + test attempts until the test passes or the attempt budget runs out.
+Give it the same dataset and the same truth checks. Interf can keep rerunning compile + test attempts until the test passes or the attempt budget runs out.
+
+That loop is the self-improving part of the product:
+
+- it reruns the same workflow package over the same dataset
+- it keeps the truth checks fixed, so the target does not move
+- it keeps the measurement fixed, so attempts stay comparable
+- it can vary the compile profile and follow-up diagnostics
+- it records which attempt performed best on the same saved test
 
 In practice:
 
-- `retry_policy.max_attempts_per_profile` controls how many attempts each compile profile gets
+- `max_attempts` controls how many total attempts a normal workspace compile gets
+- `retry_policy.max_attempts_per_profile` controls how many attempts each compile profile gets in eval packs
 - stronger diagnostic profiles can be used only after the default ones fail
-- the checks stay the same across every attempt
-- each attempt records what changed
+- the truth checks stay the same across every attempt
+- each attempt records what changed and which attempt performed best
 
 Example eval-pack shape:
 
@@ -255,7 +327,8 @@ Example eval-pack shape:
 {
   "workspaces": [
     {
-      "name": "default",
+      "name": "my-workspace",
+      "max_attempts": 3, // rerun compile + test until this workspace passes the saved truth checks or hits this limit
       "checks": [
         {
           "question": "What full-year revenue range did the company maintain?",
@@ -264,16 +337,13 @@ Example eval-pack shape:
       ]
     }
   ],
-  // Advanced only: retry settings live in eval packs, not in interf.config.json.
   "retry_policy": {
     "max_attempts_per_profile": 3
   }
 }
 ```
 
-Today this lives in the advanced eval-pack runner, not in `interf.config.json` and not in a top-level `interf compile --max-retries` flag.
-
-Use the normal test flow first. Use this advanced path when you want Interf to keep improving the local preparation workflow until the workspace is good enough for your task or the attempt budget runs out. It spends more tokens, so use it when that extra spend is worth the accuracy target.
+Use the normal workspace retry loop first. Use the eval-pack path when you want Interf to compare multiple compile profiles, add diagnostics, or keep iterating in a more controlled experiment loop. It spends more tokens, so use it when that extra spend is worth the accuracy target.
 
 ## Use It With Your Agent
 
@@ -284,25 +354,25 @@ Paste something like this into your agent:
 ```text
 Install @interf/compiler, run `interf` in this folder, and use the local agent executor.
 
-If `interf.config.json` is missing, draft one workspace with a few checks this agent should be able to answer from these files and add the expected answers for me to confirm.
+If `interf.config.json` is missing, draft one workspace with a few truth checks this agent should be able to answer from this dataset and add the expected answers for me to confirm.
 
 Then run a raw baseline if helpful, compile the workspace, and run `interf test`.
 
-Tell me whether the compiled workspace passes the checks, and only recommend it if it does.
+Tell me whether the compiled workspace passes the truth checks, and only recommend it if it does.
 ```
 
 ## Custom Workflows
 
 Interf ships with a default workflow.
 
-If you want to change how the data-processing pipeline runs on your files, this is the part you customize:
+If you want to change how the workflow package runs on your dataset, this is the part you customize:
 
 ```bash
 interf create workflow
 interf verify workflow --path <path>
 ```
 
-Then benchmark that workflow on the same folder and the same checks.
+Then test that workflow on the same dataset and the same truth checks.
 
 Workflow package docs live in [docs/workflow-spec.md](./docs/workflow-spec.md).
 
@@ -313,7 +383,7 @@ Workflow package docs live in [docs/workflow-spec.md](./docs/workflow-spec.md).
 - `interf create workspace` = create another compiled workspace when you need one
 - `interf create workflow` = create a reusable local workflow package
 - `interf compile` = build a selected workspace for the current folder
-- `interf test` = test the raw files, a compiled workspace, or both on saved checks
+- `interf test` = test the raw files, a compiled workspace, or both on saved truth checks
 - `interf benchmark` = alias for `interf test`
 - `interf doctor` = check local executor setup
 - `interf verify <check>` = run deterministic checks on major workflow steps

package/dist/commands/compile.d.ts

@@ -1,4 +1,16 @@
+import type { WorkflowExecutionProfile, WorkflowExecutor } from "../lib/executors.js";
+import type { SourceWorkspaceConfig } from "../lib/schema.js";
 import type { CommandModule } from "yargs";
+import type { StageShellRetentionMode } from "../lib/workflows.js";
 export declare const compileCommand: CommandModule;
 export declare function runCompileCommand(argv?: Record<string, unknown>): Promise<void>;
+export declare function runConfiguredWorkspaceCompile(options: {
+    executor: WorkflowExecutor;
+    workspacePath: string;
+    sourcePath: string;
+    workspaceConfig: SourceWorkspaceConfig | null;
+    executionProfile?: WorkflowExecutionProfile;
+    maxAttemptsOverride: number | null;
+    preserveStageShells?: StageShellRetentionMode;
+}): Promise<boolean>;
 //# sourceMappingURL=compile.d.ts.map

package/dist/commands/compile.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"compile.d.ts","sourceRoot":"","sources":["../../src/commands/compile.ts"],"names":[],"mappings":"AAWA,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AAY3C,eAAO,MAAM,cAAc,EAAE,aAO5B,CAAC;AAEF,wBAAsB,iBAAiB,CAAC,IAAI,GAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAwBzF"}
+{"version":3,"file":"compile.d.ts","sourceRoot":"","sources":["../../src/commands/compile.ts"],"names":[],"mappings":"AAUA,OAAO,KAAK,EAAE,wBAAwB,EAAE,gBAAgB,EAAE,MAAM,qBAAqB,CAAC;AAMtF,OAAO,KAAK,EAAE,qBAAqB,EAAE,MAAM,kBAAkB,CAAC;AAM9D,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,OAAO,CAAC;AAY3C,OAAO,KAAK,EAAE,uBAAuB,EAAE,MAAM,qBAAqB,CAAC;AAEnE,eAAO,MAAM,cAAc,EAAE,aAkB5B,CAAC;AAEF,wBAAsB,iBAAiB,CAAC,IAAI,GAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAgFzF;AAuDD,wBAAsB,6BAA6B,CACjD,OAAO,EAAE;IACP,QAAQ,EAAE,gBAAgB,CAAC;IAC3B,aAAa,EAAE,MAAM,CAAC;IACtB,UAAU,EAAE,MAAM,CAAC;IACnB,eAAe,EAAE,qBAAqB,GAAG,IAAI,CAAC;IAC9C,gBAAgB,CAAC,EAAE,wBAAwB,CAAC;IAC5C,mBAAmB,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC,mBAAmB,CAAC,EAAE,uBAAuB,CAAC;CAC/C,GACA,OAAO,CAAC,OAAO,CAAC,CA4HlB"}