@interf/compiler 0.6.4 → 0.6.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +104 -92
- package/dist/cli/commands/compile-controller.js +3 -3
- package/dist/cli/commands/compile.js +1 -1
- package/dist/cli/commands/create.js +1 -1
- package/dist/cli/commands/init.js +20 -20
- package/dist/cli/commands/source-config-wizard.js +2 -2
- package/dist/cli/commands/status.js +3 -3
- package/dist/cli/commands/test.js +8 -8
- package/package.json +3 -1
package/README.md
CHANGED
|
@@ -1,34 +1,32 @@
|
|
|
1
1
|
# Interf
|
|
2
2
|
|
|
3
3
|
Interf prepares context for your agents.
|
|
4
|
+
Use it to check whether your files are ready for the work you want your agents to do.
|
|
4
5
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
It writes the result as a local folder inside your project. Your agents can use that folder directly, and Interf can check it on the same questions to show whether it actually helped.
|
|
6
|
+
Tell Interf what you need from those files. It can compare how your local agents do on the source files. If the files alone are not enough, it builds portable context and keeps improving that context until more questions pass.
|
|
8
7
|
|
|
8
|
+
Interf Compiler builds that portable context as a local folder next to your files.
|
|
9
9
|
Your source files stay the source of truth. Interf builds on them; it does not replace them.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
For the built-in `interf` workflow, portable context for your agents includes:
|
|
12
12
|
|
|
13
13
|
```text
|
|
14
|
-
interf
|
|
15
|
-
AGENTS.md
|
|
16
|
-
CLAUDE.md
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
interf.json
|
|
21
|
-
workflow/
|
|
22
|
-
runtime/
|
|
23
|
-
tests/
|
|
14
|
+
interf/market-report/
|
|
15
|
+
AGENTS.md # how your agent should use this folder
|
|
16
|
+
CLAUDE.md # same guidance for Claude Code
|
|
17
|
+
home.md # top-level map for the work
|
|
18
|
+
summaries/ # file-by-file summaries
|
|
19
|
+
knowledge/ # linked notes built from those files
|
|
24
20
|
```
|
|
25
21
|
|
|
26
|
-
|
|
22
|
+
That is one example. Different workflows can build different folders. In every case, your agent starts from one local folder.
|
|
23
|
+
|
|
24
|
+
One public example from a PDF market report uses checks like:
|
|
27
25
|
|
|
28
26
|
- `What were Bristol's annual take-up values in 2018 and 2016?`
|
|
29
27
|
- `What were Bristol's availability values in 2018 and 2016?`
|
|
30
28
|
|
|
31
|
-
On those same
|
|
29
|
+
On those same checks, one recent public comparison produced:
|
|
32
30
|
|
|
33
31
|
<!-- PUBLIC_BENCHMARK_TABLE:START -->
|
|
34
32
|
| Agent | Source files | Portable context |
|
|
@@ -40,54 +38,59 @@ On those same questions, a recent run in this repo produced:
|
|
|
40
38
|
That means:
|
|
41
39
|
|
|
42
40
|
- Codex already passed on the source files.
|
|
43
|
-
- Claude Code
|
|
44
|
-
- Both passed after Interf built the portable context.
|
|
41
|
+
- Claude Code needed the portable context to pass.
|
|
45
42
|
|
|
46
|
-
|
|
43
|
+
That gives you a readiness signal for the work and a fair comparison on the same files.
|
|
44
|
+
|
|
45
|
+
Same work. Same checks. Same local setup.
|
|
47
46
|
|
|
48
47
|
## How It Works
|
|
49
48
|
|
|
50
|
-
1.
|
|
49
|
+
1. Pick the files for the work.
|
|
51
50
|
2. Tell Interf what you need from them.
|
|
52
|
-
3.
|
|
53
|
-
4.
|
|
54
|
-
5.
|
|
51
|
+
3. Review or edit the questions.
|
|
52
|
+
4. Run a source-files baseline if you want to check readiness first.
|
|
53
|
+
5. Build the portable context for your agents.
|
|
54
|
+
6. Run `interf test` on the same questions.
|
|
55
55
|
|
|
56
|
-
|
|
56
|
+
Check the source files first if you want the baseline. If the files already pass, keep them. If the portable context does better, use it.
|
|
57
57
|
|
|
58
58
|
## Why It Exists
|
|
59
59
|
|
|
60
|
-
|
|
60
|
+
Raw files are often not enough.
|
|
61
|
+
|
|
62
|
+
Local agents can open reports, decks, transcripts, exports, PDFs, notes, and mixed folders. They can still miss the structure the work needs.
|
|
61
63
|
|
|
62
|
-
Interf
|
|
64
|
+
Interf checks the source files first. If they are already enough, keep them. If they are not, Interf builds a local context layer next to those files and checks whether it helped.
|
|
63
65
|
|
|
64
66
|
Interf is built around a few simple rules:
|
|
65
67
|
|
|
66
|
-
-
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
70
|
-
-
|
|
71
|
-
- If the first pass is not good enough, let Interf revise the workflow and try again. Retries keep the same workflow variation; loops change the workflow itself.
|
|
68
|
+
- Explicit: the preparation stays on disk in a local folder you can inspect, diff, review, and version.
|
|
69
|
+
- File over app: the result lives next to the source files.
|
|
70
|
+
- Bring your own AI: use Claude Code, Codex, or another local agent. Automated Interf runs currently support Claude Code and Codex.
|
|
71
|
+
- Proof over vibes: check source files and portable context on the same work.
|
|
72
|
+
- Source files stay the source of truth.
|
|
72
73
|
|
|
73
|
-
##
|
|
74
|
+
## Folder Layout
|
|
74
75
|
|
|
75
|
-
|
|
76
|
+
If your folder already contains the files for the work, Interf adds a small local structure around them:
|
|
76
77
|
|
|
77
78
|
```text
|
|
78
|
-
your-
|
|
79
|
-
|
|
79
|
+
your-folder/
|
|
80
|
+
market-report/
|
|
80
81
|
report.pdf
|
|
81
82
|
notes.md
|
|
82
83
|
exports/
|
|
83
84
|
interf.json
|
|
84
85
|
interf/
|
|
85
|
-
|
|
86
|
+
market-report/
|
|
86
87
|
tests/
|
|
87
|
-
|
|
88
|
+
market-report/
|
|
88
89
|
```
|
|
89
90
|
|
|
90
|
-
|
|
91
|
+
`market-report` is just the work name. Interf reuses it under `interf/` and `interf/tests/`.
|
|
92
|
+
|
|
93
|
+
When Interf finishes, point your agents at `interf/market-report/` first. Go back to the source files whenever you need to verify something.
|
|
91
94
|
|
|
92
95
|
## Quick Start
|
|
93
96
|
|
|
@@ -102,18 +105,21 @@ Install:
|
|
|
102
105
|
npm install -g @interf/compiler
|
|
103
106
|
```
|
|
104
107
|
|
|
105
|
-
|
|
108
|
+
Then run Interf in a project folder that already contains the files for the work:
|
|
106
109
|
|
|
107
|
-
|
|
110
|
+
```bash
|
|
111
|
+
interf
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
The first run opens the setup wizard for that folder. It can:
|
|
108
115
|
|
|
109
116
|
- draft `interf.json`
|
|
110
117
|
- let you choose the source folder for the work
|
|
111
|
-
- default the dataset id from that folder name, then let you change it
|
|
112
118
|
- ask what work you need help with
|
|
113
|
-
- recommend questions
|
|
114
|
-
- run a source-files baseline
|
|
115
|
-
- build the portable context for that work
|
|
116
|
-
-
|
|
119
|
+
- recommend questions you can verify from the files, or let you add them manually
|
|
120
|
+
- run a source-files baseline so you can see whether the files are already enough
|
|
121
|
+
- build the portable context your agents will use for that work
|
|
122
|
+
- check that portable context on the same questions
|
|
117
123
|
|
|
118
124
|
If Interf cannot find your local executor setup, run:
|
|
119
125
|
|
|
@@ -121,22 +127,23 @@ If Interf cannot find your local executor setup, run:
|
|
|
121
127
|
interf doctor --live
|
|
122
128
|
```
|
|
123
129
|
|
|
124
|
-
##
|
|
125
|
-
|
|
126
|
-
Interf uses simple question-and-answer checks to see whether the portable context is good enough for this work.
|
|
127
|
-
|
|
128
|
-
Each saved check stores a question and the expected answer. In `interf.json`, those definitions live under the technical field `checks[]`.
|
|
130
|
+
## Checks
|
|
129
131
|
|
|
130
|
-
|
|
132
|
+
Interf uses question-and-answer checks.
|
|
133
|
+
They are small questions that tell you whether the work is actually covered.
|
|
131
134
|
|
|
132
|
-
|
|
135
|
+
Interf runs the same checks on the source files and on the portable context. That gives you a source-files baseline and a portable-context comparison on the same task.
|
|
133
136
|
|
|
134
|
-
Good first
|
|
137
|
+
Good first checks are small and easy to verify from the files:
|
|
135
138
|
|
|
136
139
|
- one exact number from a chart, table, or filing
|
|
137
140
|
- one short statement that should be true or false
|
|
138
141
|
- one simple comparison across years, files, or sections
|
|
139
142
|
|
|
143
|
+
Interf can recommend a starting set from the files and the work description. You can confirm, edit, or replace them.
|
|
144
|
+
|
|
145
|
+
In the CLI, you mostly see them as questions. In `interf.json`, they live under the technical field `checks[]`.
|
|
146
|
+
|
|
140
147
|
A maintained public test example in this repo stores them like this:
|
|
141
148
|
|
|
142
149
|
<!-- PUBLIC_TEST_CHECKS:START -->
|
|
@@ -163,37 +170,49 @@ A maintained public test example in this repo stores them like this:
|
|
|
163
170
|
```
|
|
164
171
|
<!-- PUBLIC_TEST_CHECKS:END -->
|
|
165
172
|
|
|
166
|
-
## What `interf test`
|
|
173
|
+
## What `interf test` Measures
|
|
167
174
|
|
|
168
|
-
`interf test` compares the source files and the portable context on the same questions.
|
|
175
|
+
`interf test` compares the source files and the portable context your agents would use on the same questions.
|
|
169
176
|
|
|
170
|
-
That
|
|
177
|
+
That makes it a small benchmark for the work at hand:
|
|
171
178
|
|
|
172
179
|
- same work
|
|
173
|
-
- same
|
|
180
|
+
- same checks
|
|
174
181
|
- same local agent setup
|
|
175
182
|
- source files on one side
|
|
176
|
-
- portable context on the other
|
|
183
|
+
- portable context your agents use on the other
|
|
184
|
+
|
|
185
|
+
If you run more than one local agent, it also gives you a fair comparison between them on the same files.
|
|
186
|
+
|
|
187
|
+
If the portable context does not help, keep the source files.
|
|
177
188
|
|
|
178
189
|
Use `interf test` on your own files instead of trusting a frozen benchmark snapshot in the docs.
|
|
179
190
|
|
|
180
191
|
Interf saves the latest comparison plus the underlying source-files and portable-context runs under `interf/tests/` in the same project.
|
|
181
192
|
|
|
182
|
-
## The
|
|
193
|
+
## The Compile And Check Loop
|
|
183
194
|
|
|
184
|
-
1. Save a few
|
|
185
|
-
2.
|
|
186
|
-
3. Run `interf compile
|
|
187
|
-
4. Run `interf test
|
|
188
|
-
5. If loops are enabled, let Interf
|
|
195
|
+
1. Save a few checks for the work in `interf.json`.
|
|
196
|
+
2. Run `interf test --target raw` if you want the source-files baseline.
|
|
197
|
+
3. Run `interf compile`.
|
|
198
|
+
4. Run `interf test`.
|
|
199
|
+
5. If self-improving loops are enabled, let Interf revise the workflow and try again on the same work and checks.
|
|
189
200
|
|
|
190
|
-
|
|
201
|
+
`interf.json` stores the saved setup and checks. Technically, that portable context is the compiled context Interf reuses for repeat runs.
|
|
191
202
|
|
|
192
203
|
## Workflows
|
|
193
204
|
|
|
194
|
-
A workflow
|
|
205
|
+
A workflow is the preparation method Interf runs on your files.
|
|
206
|
+
|
|
207
|
+
In plain language, it is the method that organizes and structures those files for agent use.
|
|
195
208
|
|
|
196
|
-
|
|
209
|
+
It can summarize files, extract structure, and build the local context layer your agent starts from.
|
|
210
|
+
|
|
211
|
+
Interf runs that method with your local coding agent and builds the result on disk as portable context.
|
|
212
|
+
|
|
213
|
+
The folder shape it produces for the agent is the context interface. The reusable method itself is the workflow package.
|
|
214
|
+
|
|
215
|
+
Publicly, you mostly care about the portable context it produces. Technically, that portable context is the compiled context.
|
|
197
216
|
|
|
198
217
|
Interf ships with a built-in `interf` workflow for the common case. If you need a different method, create one locally:
|
|
199
218
|
|
|
@@ -206,53 +225,46 @@ Workflow creation supports two paths:
|
|
|
206
225
|
- draft a workflow from the current project with a local agent
|
|
207
226
|
- copy an existing workflow and edit stage guidance directly
|
|
208
227
|
|
|
209
|
-
After assignment, build the portable context with `interf compile` and run `interf test` on the same questions.
|
|
210
|
-
|
|
211
|
-
## Compile Loops
|
|
212
|
-
|
|
213
|
-
`max_attempts` is a retry budget for the same workflow variation.
|
|
228
|
+
After assignment, build the portable context for your agents with `interf compile` and run `interf test` on the same questions.
|
|
214
229
|
|
|
215
|
-
|
|
230
|
+
## Self-Improving Loops
|
|
216
231
|
|
|
217
|
-
|
|
218
|
-
- same workflow variation
|
|
219
|
-
- same questions
|
|
220
|
-
- same measurement
|
|
232
|
+
Interf can do more than one compile pass.
|
|
221
233
|
|
|
222
|
-
|
|
234
|
+
If the first portable context is still weak, Interf can revise the workflow and compile again.
|
|
223
235
|
|
|
224
|
-
|
|
236
|
+
Each loop keeps the work and the checks fixed. The thing that changes is the preparation method.
|
|
225
237
|
|
|
226
|
-
|
|
238
|
+
If self-improving loops are enabled, Interf can:
|
|
227
239
|
|
|
228
|
-
-
|
|
229
|
-
-
|
|
230
|
-
- inspect
|
|
231
|
-
-
|
|
232
|
-
- build and test the next
|
|
240
|
+
- build the current portable context
|
|
241
|
+
- run the same checks
|
|
242
|
+
- inspect failures and traces
|
|
243
|
+
- revise the workflow
|
|
244
|
+
- build and test the next version
|
|
233
245
|
|
|
234
|
-
|
|
246
|
+
Lower-level retry budgets live in `interf.json` and the technical package docs.
|
|
235
247
|
|
|
236
248
|
## Use It With Your Agent
|
|
237
249
|
|
|
238
|
-
If you already work through a local coding agent, it
|
|
250
|
+
If you already work through a local coding agent, hand it the whole flow:
|
|
239
251
|
|
|
240
252
|
Paste something like this into your agent:
|
|
241
253
|
|
|
242
254
|
```text
|
|
243
255
|
Install `@interf/compiler`, run `interf` in this folder, and use the local agent executor.
|
|
244
256
|
|
|
245
|
-
If `interf.json` is missing, draft one dataset entry for this work. Ask what I need from the selected files
|
|
257
|
+
If `interf.json` is missing, draft one dataset entry for this work. Ask what I need from the selected files. Recommend a few questions I can verify from those files, and add the expected answers for me to confirm under `checks[]`.
|
|
246
258
|
|
|
247
|
-
|
|
259
|
+
Run a source-files baseline if useful. Build the portable context for my agents for that work. Then run `interf test` on the same questions.
|
|
248
260
|
|
|
249
|
-
|
|
261
|
+
Only recommend the portable context if it passes those questions. If you also ran the source-files baseline, tell me whether it did better than the source files.
|
|
250
262
|
```
|
|
251
263
|
|
|
252
264
|
## More Docs
|
|
253
265
|
|
|
254
266
|
- [src/packages/README.md](./src/packages/README.md) for the short system map
|
|
255
|
-
- [src/packages/compiler/PACKAGE.md](./src/packages/compiler/PACKAGE.md) for compiler primitives and the runtime
|
|
267
|
+
- [src/packages/compiler/PACKAGE.md](./src/packages/compiler/PACKAGE.md) for compiler primitives and the runtime contract
|
|
256
268
|
- [src/packages/workflow-package/PACKAGE.md](./src/packages/workflow-package/PACKAGE.md) for the workflow package model
|
|
257
269
|
- [src/packages/workflow-authoring/PACKAGE.md](./src/packages/workflow-authoring/PACKAGE.md) for workflow authoring and self-improving loops
|
|
258
270
|
|
|
@@ -219,10 +219,10 @@ function printPostCompileNextStep(options) {
|
|
|
219
219
|
return;
|
|
220
220
|
}
|
|
221
221
|
if (options.testedDuringCompile) {
|
|
222
|
-
console.log(chalk.dim(" Next: inspect the portable context, or run `interf test` if you also want a source-files-versus-portable-context comparison."));
|
|
222
|
+
console.log(chalk.dim(" Next: inspect the portable context your agents will use, or run `interf test` if you also want a source-files-versus-portable-context comparison."));
|
|
223
223
|
return;
|
|
224
224
|
}
|
|
225
|
-
console.log(chalk.dim(" Next: run `interf test` to compare the source files and the portable context."));
|
|
225
|
+
console.log(chalk.dim(" Next: run `interf test` to compare the source files and the portable context your agents would use."));
|
|
226
226
|
}
|
|
227
227
|
function formatVariationQuestionSummary(summary) {
|
|
228
228
|
return `${summary.passed_questions}/${summary.total_questions}`;
|
|
@@ -260,7 +260,7 @@ export async function runConfiguredCompiledCompile(options) {
|
|
|
260
260
|
else if (loopEnabled) {
|
|
261
261
|
if (maxAttempts > 1) {
|
|
262
262
|
console.log(chalk.dim(` Retry mode: up to ${maxAttempts} compile attempts.`));
|
|
263
|
-
console.log(chalk.dim(" Interf will rerun the same workflow variation, check the portable context, and stop early if it passes."));
|
|
263
|
+
console.log(chalk.dim(" Interf will rerun the same workflow variation, check the portable context for your agents, and stop early if it passes."));
|
|
264
264
|
}
|
|
265
265
|
else {
|
|
266
266
|
console.log(chalk.dim(" Compile mode: 1 attempt per workflow variation."));
|
|
@@ -11,7 +11,7 @@ import { runConfiguredCompiledCompile } from "./compile-controller.js";
|
|
|
11
11
|
export { runConfiguredCompiledCompile } from "./compile-controller.js";
|
|
12
12
|
export const compileCommand = {
|
|
13
13
|
command: "compile",
|
|
14
|
-
describe: "Build portable context and optionally run retry or self-improving compile loops",
|
|
14
|
+
describe: "Build portable context for your agents and optionally run retry or self-improving compile loops",
|
|
15
15
|
builder: (yargs) => addExecutionProfileOptions(yargs).option("max-attempts", {
|
|
16
16
|
type: "number",
|
|
17
17
|
describe: "Retry compile and test the same workflow until the dataset passes or reaches this total attempt limit",
|
|
@@ -139,7 +139,7 @@ export async function createCompiledWizard(options = {}) {
|
|
|
139
139
|
if (savedDatasets.length > 0) {
|
|
140
140
|
p.log.info(`This project already has ${savedDatasets.length} saved setup${savedDatasets.length === 1 ? "" : "s"} in interf.json. Add another only when you need a separate folder, focus, or question set.`);
|
|
141
141
|
}
|
|
142
|
-
p.log.info("Interf works one source-folder setup at a time. Start with the work, the relevant files, and a few questions, then build portable context
|
|
142
|
+
p.log.info("Interf works one source-folder setup at a time. Start with the work, the relevant files, and a few questions, then build portable context for your agents when the files alone are not enough.");
|
|
143
143
|
const existingConfig = loadSourceFolderConfig(cwd);
|
|
144
144
|
const draft = await promptSingleCompiledConfig({
|
|
145
145
|
projectPath: cwd,
|
|
@@ -46,7 +46,7 @@ function printDatasetSummary(options) {
|
|
|
46
46
|
p.log.info(describeSavedQuestions(options.dataset));
|
|
47
47
|
p.log.info(options.built ? "Portable context is available." : "Portable context has not been built yet.");
|
|
48
48
|
if (options.latestComparison?.raw && options.latestComparison?.compiled) {
|
|
49
|
-
p.log.info(`Latest saved comparison: source files ${options.latestComparison.raw.passed_cases}/${options.latestComparison.raw.total_cases}, portable context ${options.latestComparison.compiled.passed_cases}/${options.latestComparison.compiled.total_cases}.`);
|
|
49
|
+
p.log.info(`Latest saved comparison: source files ${options.latestComparison.raw.passed_cases}/${options.latestComparison.raw.total_cases}, portable context for your agents ${options.latestComparison.compiled.passed_cases}/${options.latestComparison.compiled.total_cases}.`);
|
|
50
50
|
}
|
|
51
51
|
else if (options.latestComparison?.raw) {
|
|
52
52
|
p.log.info(`Latest saved source-files run: ${options.latestComparison.raw.passed_cases}/${options.latestComparison.raw.total_cases}.`);
|
|
@@ -70,12 +70,12 @@ async function promptDatasetAction(dataset, built, latestComparison) {
|
|
|
70
70
|
options.push({
|
|
71
71
|
value: "test",
|
|
72
72
|
label: built
|
|
73
|
-
? "Compare source files and portable context (Recommended)"
|
|
73
|
+
? "Compare source files and the portable context your agents would use (Recommended)"
|
|
74
74
|
: hasSavedRawBaseline
|
|
75
75
|
? "Rerun the source-files baseline"
|
|
76
76
|
: "Measure the source-files baseline (Recommended)",
|
|
77
77
|
hint: built
|
|
78
|
-
? "See whether the portable context helps on the saved questions"
|
|
78
|
+
? "See whether the portable context helps your agents on the saved questions"
|
|
79
79
|
: hasSavedRawBaseline
|
|
80
80
|
? "Refresh the saved source-files baseline on the current questions"
|
|
81
81
|
: "See whether the source files are already good enough before compiling",
|
|
@@ -83,16 +83,16 @@ async function promptDatasetAction(dataset, built, latestComparison) {
|
|
|
83
83
|
}
|
|
84
84
|
options.push({
|
|
85
85
|
value: "compile",
|
|
86
|
-
label: built ? "Rebuild portable context" : "Build portable context",
|
|
86
|
+
label: built ? "Rebuild portable context for your agents" : "Build portable context for your agents",
|
|
87
87
|
hint: built
|
|
88
88
|
? "Refresh it from the current source files"
|
|
89
89
|
: dataset.checks.length > 0
|
|
90
90
|
? latestRawPasses
|
|
91
|
-
? "Optional: build portable context for a side-by-side comparison with the latest saved source-files run"
|
|
91
|
+
? "Optional: build portable context for your agents for a side-by-side comparison with the latest saved source-files run"
|
|
92
92
|
: hasSavedRawBaseline
|
|
93
|
-
? "Some questions still fail on the source files. Build portable context and compare it on the same questions"
|
|
94
|
-
: "Build portable context and compare it on the same questions"
|
|
95
|
-
: "Build it first, then measure it
|
|
93
|
+
? "Some questions still fail on the source files. Build portable context for your agents and compare it on the same questions"
|
|
94
|
+
: "Build portable context for your agents and compare it on the same questions"
|
|
95
|
+
: "Build it for your agents first, then measure it",
|
|
96
96
|
}, {
|
|
97
97
|
value: "dataset",
|
|
98
98
|
label: "Add another setup",
|
|
@@ -126,14 +126,14 @@ function printDatasetRecommendation(dataset, built, latestComparison) {
|
|
|
126
126
|
if (!built) {
|
|
127
127
|
if (latestComparison?.raw) {
|
|
128
128
|
p.log.info(latestComparison.raw.passed_cases === latestComparison.raw.total_cases
|
|
129
|
-
? `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Rerun the baseline if you want current numbers for a different agent set, or build portable context if you want a side-by-side comparison.`
|
|
130
|
-
: `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Some questions still fail on the source files. Build portable context if you want Interf to prepare context for this work and compare it on the same questions.`);
|
|
129
|
+
? `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Rerun the baseline if you want current numbers for a different agent set, or build portable context for your agents if you want a side-by-side comparison.`
|
|
130
|
+
: `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Some questions still fail on the source files. Build portable context for your agents if you want Interf to prepare context for this work and compare it on the same questions.`);
|
|
131
131
|
return;
|
|
132
132
|
}
|
|
133
133
|
p.log.info("Recommended first step: measure the source-files baseline before compiling.");
|
|
134
134
|
return;
|
|
135
135
|
}
|
|
136
|
-
p.log.info("Recommended first step: compare the source-files baseline and the portable context.");
|
|
136
|
+
p.log.info("Recommended first step: compare the source-files baseline and the portable context your agents would use.");
|
|
137
137
|
}
|
|
138
138
|
async function promptPostBaselineAction(rows) {
|
|
139
139
|
const status = currentRunTargetStatus(rows, "raw");
|
|
@@ -154,13 +154,13 @@ async function promptPostBaselineAction(rows) {
|
|
|
154
154
|
{
|
|
155
155
|
value: "compile",
|
|
156
156
|
label: compileRecommended
|
|
157
|
-
? "Build portable context (Recommended)"
|
|
158
|
-
: "Build portable context",
|
|
157
|
+
? "Build portable context for your agents (Recommended)"
|
|
158
|
+
: "Build portable context for your agents",
|
|
159
159
|
hint: compileRecommended
|
|
160
160
|
? status === "mixed"
|
|
161
|
-
? "Some selected agents still failed on the source files. Build portable context and compare it on the same questions"
|
|
162
|
-
: "Build portable context and compare it on the same questions"
|
|
163
|
-
: "Build portable context and compare it on the same questions",
|
|
161
|
+
? "Some selected agents still failed on the source files. Build portable context for your agents and compare it on the same questions"
|
|
162
|
+
: "Build portable context for your agents and compare it on the same questions"
|
|
163
|
+
: "Build portable context for your agents and compare it on the same questions",
|
|
164
164
|
},
|
|
165
165
|
{
|
|
166
166
|
value: "edit",
|
|
@@ -407,7 +407,7 @@ async function runCompiledActionMenu(sourcePath, compiledConfig, options = {}) {
|
|
|
407
407
|
return;
|
|
408
408
|
}
|
|
409
409
|
if (compileResult.testedDuringCompile) {
|
|
410
|
-
p.log.info("This compile run already checked the portable context on the saved questions.");
|
|
410
|
+
p.log.info("This compile run already checked the portable context for your agents on the saved questions.");
|
|
411
411
|
p.log.info("Run `interf test` later if you want a fresh side-by-side comparison summary.");
|
|
412
412
|
return;
|
|
413
413
|
}
|
|
@@ -426,14 +426,14 @@ async function runCompiledActionMenu(sourcePath, compiledConfig, options = {}) {
|
|
|
426
426
|
if (compiledConfig.checks.length === 0)
|
|
427
427
|
return;
|
|
428
428
|
if (compileResult.testedDuringCompile) {
|
|
429
|
-
p.log.info("This compile run already checked the portable context on the compile agent.");
|
|
429
|
+
p.log.info("This compile run already checked the portable context for your agents on the compile agent.");
|
|
430
430
|
}
|
|
431
431
|
const runCompiledTest = await p.confirm({
|
|
432
432
|
message: compileResult.testedDuringCompile
|
|
433
433
|
? "Run a fresh source-files versus portable-context comparison now?"
|
|
434
434
|
: builtCompiledPath
|
|
435
|
-
? "Run source files and portable context on the saved questions now?"
|
|
436
|
-
: "Compare source files and portable context on the saved questions now?",
|
|
435
|
+
? "Run source files and the portable context your agents would use on the saved questions now?"
|
|
436
|
+
: "Compare source files and the portable context your agents would use on the saved questions now?",
|
|
437
437
|
initialValue: true,
|
|
438
438
|
});
|
|
439
439
|
if (p.isCancel(runCompiledTest) || !runCompiledTest)
|
|
@@ -370,7 +370,7 @@ async function promptCompiledChecks(options) {
|
|
|
370
370
|
if (options.initialAbout) {
|
|
371
371
|
p.log.info(`About: ${options.initialAbout}`);
|
|
372
372
|
}
|
|
373
|
-
p.log.info("Questions are simple question-and-answer pairs Interf uses to check accuracy
|
|
373
|
+
p.log.info("Questions are simple question-and-answer pairs Interf uses to check accuracy, so you can decide whether to trust the portable context.");
|
|
374
374
|
const mode = await p.select({
|
|
375
375
|
message: "How should Interf create them?",
|
|
376
376
|
options: [
|
|
@@ -544,7 +544,7 @@ export async function promptCompileLoopSelection(options) {
|
|
|
544
544
|
once: {
|
|
545
545
|
value: "once",
|
|
546
546
|
label: "Compile once",
|
|
547
|
-
hint: "Build portable context once with the selected workflow",
|
|
547
|
+
hint: "Build portable context for your agents once with the selected workflow",
|
|
548
548
|
},
|
|
549
549
|
"self-improving": {
|
|
550
550
|
value: "self-improving",
|
|
@@ -18,7 +18,7 @@ function statusColor(status) {
|
|
|
18
18
|
}
|
|
19
19
|
export const statusCommand = {
|
|
20
20
|
command: "status",
|
|
21
|
-
describe: "Show deterministic health for portable context",
|
|
21
|
+
describe: "Show deterministic health for the portable context your agents use",
|
|
22
22
|
handler: async () => {
|
|
23
23
|
let compiledPath = null;
|
|
24
24
|
const detected = detectInterf(process.cwd());
|
|
@@ -34,7 +34,7 @@ export const statusCommand = {
|
|
|
34
34
|
if (local.length === 0) {
|
|
35
35
|
process.exitCode = 1;
|
|
36
36
|
console.log(chalk.red(" No portable contexts found."));
|
|
37
|
-
console.log(chalk.dim(" Run `interf`, save questions, and compile portable context first."));
|
|
37
|
+
console.log(chalk.dim(" Run `interf`, save questions, and compile portable context for your agents first."));
|
|
38
38
|
return;
|
|
39
39
|
}
|
|
40
40
|
if (local.length === 1) {
|
|
@@ -75,7 +75,7 @@ export const statusCommand = {
|
|
|
75
75
|
console.log();
|
|
76
76
|
console.log(chalk.dim(latestComparisonState.stale
|
|
77
77
|
? " Saved test results are stale for the current questions. Run `interf test` again."
|
|
78
|
-
: " No saved comparison yet. Run `interf test` to measure source files and the portable context."));
|
|
78
|
+
: " No saved comparison yet. Run `interf test` to measure source files and the portable context your agents would use."));
|
|
79
79
|
}
|
|
80
80
|
console.log();
|
|
81
81
|
const metricOrder = [
|
|
@@ -9,7 +9,7 @@ import { printAgentTestFailures, printAgentTestMatrix, printSavedTestComparisonS
|
|
|
9
9
|
import { listRunAgentOptions, promptForTestAgents, resolveNamedLocalExecutor, resolveOrConfigureLocalExecutor, } from "./executor-flow.js";
|
|
10
10
|
export const testCommand = {
|
|
11
11
|
command: "test",
|
|
12
|
-
describe: "Compare source files and portable context on saved questions",
|
|
12
|
+
describe: "Compare source files and the portable context your agents would use on saved questions",
|
|
13
13
|
builder: (yargs) => addExecutionProfileOptions(yargs)
|
|
14
14
|
.option("dataset", {
|
|
15
15
|
type: "string",
|
|
@@ -17,7 +17,7 @@ export const testCommand = {
|
|
|
17
17
|
})
|
|
18
18
|
.option("target", {
|
|
19
19
|
choices: ["both", "raw", "compiled"],
|
|
20
|
-
describe: "Test source files, the portable context, or both. Default: both when portable context exists, otherwise source files.",
|
|
20
|
+
describe: "Test source files, the portable context your agents would use, or both. Default: both when portable context exists, otherwise source files.",
|
|
21
21
|
})
|
|
22
22
|
.option("keep-sandboxes", {
|
|
23
23
|
type: "boolean",
|
|
@@ -49,8 +49,8 @@ async function promptTestMode(hasBuiltCompiled) {
|
|
|
49
49
|
options: [
|
|
50
50
|
{
|
|
51
51
|
value: "both",
|
|
52
|
-
label: "Compare source files and portable context (Recommended)",
|
|
53
|
-
hint: "Measure whether the portable context helps on the saved questions",
|
|
52
|
+
label: "Compare source files and the portable context your agents would use (Recommended)",
|
|
53
|
+
hint: "Measure whether the portable context helps your agents on the saved questions",
|
|
54
54
|
},
|
|
55
55
|
{
|
|
56
56
|
value: "raw",
|
|
@@ -60,7 +60,7 @@ async function promptTestMode(hasBuiltCompiled) {
|
|
|
60
60
|
{
|
|
61
61
|
value: "compiled",
|
|
62
62
|
label: "Portable context only",
|
|
63
|
-
hint: "Measure the current portable context on the saved questions",
|
|
63
|
+
hint: "Measure the current portable context for your agents on the saved questions",
|
|
64
64
|
},
|
|
65
65
|
],
|
|
66
66
|
});
|
|
@@ -240,13 +240,13 @@ export async function runTestCommand(argv = {}) {
|
|
|
240
240
|
return false;
|
|
241
241
|
if (selectedMode === "compiled" && !hasBuiltCompiled) {
|
|
242
242
|
process.exitCode = 1;
|
|
243
|
-
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context yet.`));
|
|
243
|
+
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context for your agents yet.`));
|
|
244
244
|
console.log(chalk.dim(" Run `interf compile` first."));
|
|
245
245
|
return false;
|
|
246
246
|
}
|
|
247
247
|
if (selectedMode === "both" && !hasBuiltCompiled) {
|
|
248
248
|
process.exitCode = 1;
|
|
249
|
-
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context yet, so Interf cannot compare source files and the portable context.`));
|
|
249
|
+
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context for your agents yet, so Interf cannot compare source files and the portable context your agents would use.`));
|
|
250
250
|
console.log(chalk.dim(" Run `interf compile` first, or rerun with `--target raw`."));
|
|
251
251
|
return false;
|
|
252
252
|
}
|
|
@@ -268,7 +268,7 @@ export async function runTestCommand(argv = {}) {
|
|
|
268
268
|
})));
|
|
269
269
|
if ((mode === "compiled" || mode === "both") && rows.some((row) => !row.compiledOutcome)) {
|
|
270
270
|
process.exitCode = 1;
|
|
271
|
-
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context yet.`));
|
|
271
|
+
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context for your agents yet.`));
|
|
272
272
|
console.log(chalk.dim(" Run `interf compile` first."));
|
|
273
273
|
return false;
|
|
274
274
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@interf/compiler",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.6",
|
|
4
4
|
"description": "Interf prepares context for your agents by checking your files first, building a local folder when needed, and using self-improving loops until more questions pass.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -43,7 +43,9 @@
|
|
|
43
43
|
"scripts": {
|
|
44
44
|
"build": "node -e \"require('fs').rmSync('dist',{recursive:true,force:true})\" && tsc && chmod 755 dist/bin.js",
|
|
45
45
|
"dev": "tsc --watch",
|
|
46
|
+
"setup:local-skills": "bash scripts/setup-local-skills.sh",
|
|
46
47
|
"refresh:bootstrap-mirror": "node scripts/docs/sync-bootstrap-mirror.mjs",
|
|
48
|
+
"docs:sync-public-test-example": "npm run build && node scripts/docs/sync-public-test-example.mjs",
|
|
47
49
|
"test:matrix": "npm run build && node scripts/matrix/run.mjs",
|
|
48
50
|
"test:smoke": "npm run build && node --test test/**/*.test.mjs",
|
|
49
51
|
"test": "npm run test:smoke",
|