@interf/compiler 0.6.4 → 0.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +66 -74
- package/dist/cli/commands/compile-controller.js +3 -3
- package/dist/cli/commands/compile.js +1 -1
- package/dist/cli/commands/create.js +1 -1
- package/dist/cli/commands/init.js +20 -20
- package/dist/cli/commands/source-config-wizard.js +2 -2
- package/dist/cli/commands/status.js +3 -3
- package/dist/cli/commands/test.js +8 -8
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -2,28 +2,27 @@
|
|
|
2
2
|
|
|
3
3
|
Interf prepares context for your agents.
|
|
4
4
|
|
|
5
|
-
Start with the files for the work you want to do. Tell Interf what you need, let it suggest a few questions to check
|
|
5
|
+
Start with the files for the work you want to do. Tell Interf what you need, let it suggest a few questions to check accuracy, and build portable context for your agents when the files alone are not enough.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Interf Compiler builds portable context in a local folder inside your project.
|
|
8
|
+
Your agents can use it directly. Interf can check it on those same questions.
|
|
8
9
|
|
|
9
10
|
Your source files stay the source of truth. Interf builds on them; it does not replace them.
|
|
10
11
|
|
|
11
|
-
|
|
12
|
+
For the built-in `interf` workflow, a portable context can include:
|
|
12
13
|
|
|
13
14
|
```text
|
|
14
15
|
interf/<dataset>/
|
|
15
|
-
AGENTS.md
|
|
16
|
-
CLAUDE.md
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
interf.json
|
|
21
|
-
workflow/
|
|
22
|
-
runtime/
|
|
23
|
-
tests/
|
|
16
|
+
AGENTS.md # how your agent should use this folder
|
|
17
|
+
CLAUDE.md # same guidance for Claude Code
|
|
18
|
+
home.md # top-level map for the work
|
|
19
|
+
summaries/ # file-by-file summaries
|
|
20
|
+
knowledge/ # linked notes built from those files
|
|
24
21
|
```
|
|
25
22
|
|
|
26
|
-
|
|
23
|
+
That is an example, not a fixed ABI. Different workflows can build different folders. The simple part is that your agent gets one local folder to start from.
|
|
24
|
+
|
|
25
|
+
This repo includes a small PDF example with questions like:
|
|
27
26
|
|
|
28
27
|
- `What were Bristol's annual take-up values in 2018 and 2016?`
|
|
29
28
|
- `What were Bristol's availability values in 2018 and 2016?`
|
|
@@ -40,39 +39,40 @@ On those same questions, a recent run in this repo produced:
|
|
|
40
39
|
That means:
|
|
41
40
|
|
|
42
41
|
- Codex already passed on the source files.
|
|
43
|
-
- Claude Code
|
|
44
|
-
- Both passed after Interf built the portable context.
|
|
42
|
+
- Claude Code needed the portable context to pass.
|
|
45
43
|
|
|
46
44
|
Same work. Same questions. Same local setup.
|
|
47
45
|
|
|
48
46
|
## How It Works
|
|
49
47
|
|
|
50
|
-
1.
|
|
48
|
+
1. Pick the files for the work.
|
|
51
49
|
2. Tell Interf what you need from them.
|
|
52
|
-
3.
|
|
53
|
-
4. Build the portable context for
|
|
54
|
-
5. Run `interf test` on the same questions
|
|
50
|
+
3. Review or edit the questions.
|
|
51
|
+
4. Build the portable context for your agents.
|
|
52
|
+
5. Run `interf test` on the same questions.
|
|
55
53
|
|
|
56
|
-
|
|
54
|
+
Check the source files first if you want the baseline. If the files already pass, keep them. If the portable context does better, use it.
|
|
57
55
|
|
|
58
56
|
## Why It Exists
|
|
59
57
|
|
|
60
|
-
|
|
58
|
+
Raw files are often not enough.
|
|
59
|
+
|
|
60
|
+
Local agents can open reports, decks, transcripts, exports, PDFs, notes, and mixed folders. They can still miss the structure the work needs.
|
|
61
61
|
|
|
62
|
-
Interf
|
|
62
|
+
Interf builds a local folder for that work and checks whether it helped.
|
|
63
63
|
|
|
64
64
|
Interf is built around a few simple rules:
|
|
65
65
|
|
|
66
66
|
- Start from the files you already have.
|
|
67
|
-
- Check
|
|
67
|
+
- Check source files and portable context on the same questions.
|
|
68
68
|
- Keep the result in a local folder you can inspect, diff, review, and version.
|
|
69
|
-
- Keep
|
|
69
|
+
- Keep source files as the source of truth.
|
|
70
70
|
- Use the portable context manually from Claude Code, Codex, or another local agent. Automated Interf runs currently support Claude Code and Codex.
|
|
71
|
-
- If the first pass is not
|
|
71
|
+
- If the first pass is not enough, let Interf retry the same workflow or improve the workflow itself.
|
|
72
72
|
|
|
73
73
|
## What You Get
|
|
74
74
|
|
|
75
|
-
A project
|
|
75
|
+
A small project can stay simple:
|
|
76
76
|
|
|
77
77
|
```text
|
|
78
78
|
your-project/
|
|
@@ -87,7 +87,7 @@ your-project/
|
|
|
87
87
|
<dataset>/
|
|
88
88
|
```
|
|
89
89
|
|
|
90
|
-
After Interf runs, point your agents at
|
|
90
|
+
After Interf runs, point your agents at `interf/<dataset>/` first. Go back to the source files whenever you need to verify something.
|
|
91
91
|
|
|
92
92
|
## Quick Start
|
|
93
93
|
|
|
@@ -102,18 +102,21 @@ Install:
|
|
|
102
102
|
npm install -g @interf/compiler
|
|
103
103
|
```
|
|
104
104
|
|
|
105
|
-
|
|
105
|
+
Then run Interf in a project folder that already contains the files for the work:
|
|
106
106
|
|
|
107
|
-
|
|
107
|
+
```bash
|
|
108
|
+
interf
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
The first run opens the dataset wizard for that folder. It can:
|
|
108
112
|
|
|
109
113
|
- draft `interf.json`
|
|
110
114
|
- let you choose the source folder for the work
|
|
111
|
-
- default the dataset id from that folder name, then let you change it
|
|
112
115
|
- ask what work you need help with
|
|
113
|
-
- recommend questions to check accuracy
|
|
114
|
-
- run a source-files baseline
|
|
115
|
-
- build the portable context for that work
|
|
116
|
-
-
|
|
116
|
+
- recommend questions to check accuracy, or let you add them manually
|
|
117
|
+
- run a source-files baseline so you can see what already works
|
|
118
|
+
- build the portable context your agents will use for that work
|
|
119
|
+
- check that portable context on the same questions
|
|
117
120
|
|
|
118
121
|
If Interf cannot find your local executor setup, run:
|
|
119
122
|
|
|
@@ -123,20 +126,18 @@ interf doctor --live
|
|
|
123
126
|
|
|
124
127
|
## Questions
|
|
125
128
|
|
|
126
|
-
Interf uses simple
|
|
127
|
-
|
|
128
|
-
Each saved check stores a question and the expected answer. In `interf.json`, those definitions live under the technical field `checks[]`.
|
|
129
|
-
|
|
130
|
-
They are just question-and-answer pairs you can verify from the files you selected for the work.
|
|
131
|
-
|
|
132
|
-
You do not need to write them from scratch before you know the data well. Interf can recommend a starting set from the files and the work description, then you can confirm, edit, or replace them. Their job is simple: check accuracy before you trust the portable context.
|
|
129
|
+
Interf uses simple questions to check whether the portable context is good enough for your agents on this work.
|
|
133
130
|
|
|
134
|
-
Good first questions are small and
|
|
131
|
+
Good first questions are small and easy to verify from the files:
|
|
135
132
|
|
|
136
133
|
- one exact number from a chart, table, or filing
|
|
137
134
|
- one short statement that should be true or false
|
|
138
135
|
- one simple comparison across years, files, or sections
|
|
139
136
|
|
|
137
|
+
Interf can recommend a starting set from the files and the work description. You can confirm, edit, or replace them.
|
|
138
|
+
|
|
139
|
+
Each saved question includes the expected answer. In `interf.json`, those definitions live under the technical field `checks[]`.
|
|
140
|
+
|
|
140
141
|
A maintained public test example in this repo stores them like this:
|
|
141
142
|
|
|
142
143
|
<!-- PUBLIC_TEST_CHECKS:START -->
|
|
@@ -165,15 +166,17 @@ A maintained public test example in this repo stores them like this:
|
|
|
165
166
|
|
|
166
167
|
## What `interf test` Proves
|
|
167
168
|
|
|
168
|
-
`interf test` compares the source files and the portable context on the same questions.
|
|
169
|
+
`interf test` compares the source files and the portable context your agents would use on the same questions.
|
|
169
170
|
|
|
170
|
-
That comparison is the proof:
|
|
171
|
+
That comparison is the proof step:
|
|
171
172
|
|
|
172
173
|
- same work
|
|
173
174
|
- same questions
|
|
174
175
|
- same local agent setup
|
|
175
176
|
- source files on one side
|
|
176
|
-
- portable context on the other
|
|
177
|
+
- portable context your agents use on the other
|
|
178
|
+
|
|
179
|
+
If the portable context does not help, keep the source files.
|
|
177
180
|
|
|
178
181
|
Use `interf test` on your own files instead of trusting a frozen benchmark snapshot in the docs.
|
|
179
182
|
|
|
@@ -182,18 +185,20 @@ Interf saves the latest comparison plus the underlying source-files and portable
|
|
|
182
185
|
## The Core Loop
|
|
183
186
|
|
|
184
187
|
1. Save a few questions for the work in `interf.json`.
|
|
185
|
-
2.
|
|
186
|
-
3. Run `interf compile` to build the portable context your
|
|
187
|
-
4. Run `interf test`
|
|
188
|
-
5. If loops are enabled, let Interf retry the same workflow variation or
|
|
188
|
+
2. Check the source files first if you want the baseline.
|
|
189
|
+
3. Run `interf compile` to build the portable context your agents can use.
|
|
190
|
+
4. Run `interf test` on the same questions.
|
|
191
|
+
5. If loops are enabled, let Interf retry the same workflow variation or improve the workflow and test again.
|
|
189
192
|
|
|
190
|
-
The project root stores the source-folder setup and saved questions. Technically, that portable context is the compiled context
|
|
193
|
+
The project root stores the source-folder setup and saved questions. Technically, that portable context is the compiled context Interf reuses for repeat runs.
|
|
191
194
|
|
|
192
195
|
## Workflows
|
|
193
196
|
|
|
194
|
-
A workflow
|
|
197
|
+
A workflow is the method Interf uses to prepare files for a kind of work.
|
|
198
|
+
|
|
199
|
+
A context interface is the folder shape that workflow builds for your agent.
|
|
195
200
|
|
|
196
|
-
Technically, a workflow package defines
|
|
201
|
+
Technically, a workflow package defines that context interface: the zones, paths, and stage ownership of the compiled context Interf will build for that work.
|
|
197
202
|
|
|
198
203
|
Interf ships with a built-in `interf` workflow for the common case. If you need a different method, create one locally:
|
|
199
204
|
|
|
@@ -206,47 +211,34 @@ Workflow creation supports two paths:
|
|
|
206
211
|
- draft a workflow from the current project with a local agent
|
|
207
212
|
- copy an existing workflow and edit stage guidance directly
|
|
208
213
|
|
|
209
|
-
After assignment, build the portable context with `interf compile` and run `interf test` on the same questions.
|
|
214
|
+
After assignment, build the portable context for your agents with `interf compile` and run `interf test` on the same questions.
|
|
210
215
|
|
|
211
216
|
## Compile Loops
|
|
212
217
|
|
|
213
218
|
`max_attempts` is a retry budget for the same workflow variation.
|
|
214
219
|
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
- same configured dataset
|
|
218
|
-
- same workflow variation
|
|
219
|
-
- same questions
|
|
220
|
-
- same measurement
|
|
221
|
-
|
|
222
|
-
`max_loops` enables self-improving workflow edits in the normal `interf compile` path.
|
|
223
|
-
|
|
224
|
-
In that loop, the thing that changes is the workflow itself.
|
|
220
|
+
`max_loops` lets Interf improve the workflow itself between retry batches.
|
|
225
221
|
|
|
226
|
-
|
|
222
|
+
With `max_attempts`, the dataset, questions, and workflow stay fixed.
|
|
227
223
|
|
|
228
|
-
|
|
229
|
-
- test it on the same questions
|
|
230
|
-
- inspect failed traces, preserved stage shells, and test artifacts
|
|
231
|
-
- edit the workflow
|
|
232
|
-
- build and test the next workflow variation
|
|
224
|
+
With `max_loops`, Interf can edit the workflow, build the next variation, and test it again on the same dataset and questions.
|
|
233
225
|
|
|
234
|
-
Interf preserves workflow
|
|
226
|
+
Interf preserves workflow snapshots, failed stage shells, and test runs from each loop so you can inspect what changed.
|
|
235
227
|
|
|
236
228
|
## Use It With Your Agent
|
|
237
229
|
|
|
238
|
-
If you already work through a local coding agent, it
|
|
230
|
+
If you already work through a local coding agent, hand it the whole flow:
|
|
239
231
|
|
|
240
232
|
Paste something like this into your agent:
|
|
241
233
|
|
|
242
234
|
```text
|
|
243
235
|
Install `@interf/compiler`, run `interf` in this folder, and use the local agent executor.
|
|
244
236
|
|
|
245
|
-
If `interf.json` is missing, draft one dataset entry for this work. Ask what I need from the selected files
|
|
237
|
+
If `interf.json` is missing, draft one dataset entry for this work. Ask what I need from the selected files. Recommend a few questions I can verify from those files, and add the expected answers for me to confirm under `checks[]`.
|
|
246
238
|
|
|
247
|
-
|
|
239
|
+
Run a source-files baseline if useful. Build the portable context for my agents for that work. Then run `interf test` on the same questions.
|
|
248
240
|
|
|
249
|
-
|
|
241
|
+
Only recommend the portable context if it passes those questions. If you also ran the source-files baseline, tell me whether it did better than the source files.
|
|
250
242
|
```
|
|
251
243
|
|
|
252
244
|
## More Docs
|
|
@@ -219,10 +219,10 @@ function printPostCompileNextStep(options) {
|
|
|
219
219
|
return;
|
|
220
220
|
}
|
|
221
221
|
if (options.testedDuringCompile) {
|
|
222
|
-
console.log(chalk.dim(" Next: inspect the portable context, or run `interf test` if you also want a source-files-versus-portable-context comparison."));
|
|
222
|
+
console.log(chalk.dim(" Next: inspect the portable context your agents will use, or run `interf test` if you also want a source-files-versus-portable-context comparison."));
|
|
223
223
|
return;
|
|
224
224
|
}
|
|
225
|
-
console.log(chalk.dim(" Next: run `interf test` to compare the source files and the portable context."));
|
|
225
|
+
console.log(chalk.dim(" Next: run `interf test` to compare the source files and the portable context your agents would use."));
|
|
226
226
|
}
|
|
227
227
|
function formatVariationQuestionSummary(summary) {
|
|
228
228
|
return `${summary.passed_questions}/${summary.total_questions}`;
|
|
@@ -260,7 +260,7 @@ export async function runConfiguredCompiledCompile(options) {
|
|
|
260
260
|
else if (loopEnabled) {
|
|
261
261
|
if (maxAttempts > 1) {
|
|
262
262
|
console.log(chalk.dim(` Retry mode: up to ${maxAttempts} compile attempts.`));
|
|
263
|
-
console.log(chalk.dim(" Interf will rerun the same workflow variation, check the portable context, and stop early if it passes."));
|
|
263
|
+
console.log(chalk.dim(" Interf will rerun the same workflow variation, check the portable context for your agents, and stop early if it passes."));
|
|
264
264
|
}
|
|
265
265
|
else {
|
|
266
266
|
console.log(chalk.dim(" Compile mode: 1 attempt per workflow variation."));
|
|
@@ -11,7 +11,7 @@ import { runConfiguredCompiledCompile } from "./compile-controller.js";
|
|
|
11
11
|
export { runConfiguredCompiledCompile } from "./compile-controller.js";
|
|
12
12
|
export const compileCommand = {
|
|
13
13
|
command: "compile",
|
|
14
|
-
describe: "Build portable context and optionally run retry or self-improving compile loops",
|
|
14
|
+
describe: "Build portable context for your agents and optionally run retry or self-improving compile loops",
|
|
15
15
|
builder: (yargs) => addExecutionProfileOptions(yargs).option("max-attempts", {
|
|
16
16
|
type: "number",
|
|
17
17
|
describe: "Retry compile and test the same workflow until the dataset passes or reaches this total attempt limit",
|
|
@@ -139,7 +139,7 @@ export async function createCompiledWizard(options = {}) {
|
|
|
139
139
|
if (savedDatasets.length > 0) {
|
|
140
140
|
p.log.info(`This project already has ${savedDatasets.length} saved setup${savedDatasets.length === 1 ? "" : "s"} in interf.json. Add another only when you need a separate folder, focus, or question set.`);
|
|
141
141
|
}
|
|
142
|
-
p.log.info("Interf works one source-folder setup at a time. Start with the work, the relevant files, and a few questions, then build portable context
|
|
142
|
+
p.log.info("Interf works one source-folder setup at a time. Start with the work, the relevant files, and a few questions, then build portable context for your agents when the files alone are not enough.");
|
|
143
143
|
const existingConfig = loadSourceFolderConfig(cwd);
|
|
144
144
|
const draft = await promptSingleCompiledConfig({
|
|
145
145
|
projectPath: cwd,
|
|
@@ -46,7 +46,7 @@ function printDatasetSummary(options) {
|
|
|
46
46
|
p.log.info(describeSavedQuestions(options.dataset));
|
|
47
47
|
p.log.info(options.built ? "Portable context is available." : "Portable context has not been built yet.");
|
|
48
48
|
if (options.latestComparison?.raw && options.latestComparison?.compiled) {
|
|
49
|
-
p.log.info(`Latest saved comparison: source files ${options.latestComparison.raw.passed_cases}/${options.latestComparison.raw.total_cases}, portable context ${options.latestComparison.compiled.passed_cases}/${options.latestComparison.compiled.total_cases}.`);
|
|
49
|
+
p.log.info(`Latest saved comparison: source files ${options.latestComparison.raw.passed_cases}/${options.latestComparison.raw.total_cases}, portable context for your agents ${options.latestComparison.compiled.passed_cases}/${options.latestComparison.compiled.total_cases}.`);
|
|
50
50
|
}
|
|
51
51
|
else if (options.latestComparison?.raw) {
|
|
52
52
|
p.log.info(`Latest saved source-files run: ${options.latestComparison.raw.passed_cases}/${options.latestComparison.raw.total_cases}.`);
|
|
@@ -70,12 +70,12 @@ async function promptDatasetAction(dataset, built, latestComparison) {
|
|
|
70
70
|
options.push({
|
|
71
71
|
value: "test",
|
|
72
72
|
label: built
|
|
73
|
-
? "Compare source files and portable context (Recommended)"
|
|
73
|
+
? "Compare source files and the portable context your agents would use (Recommended)"
|
|
74
74
|
: hasSavedRawBaseline
|
|
75
75
|
? "Rerun the source-files baseline"
|
|
76
76
|
: "Measure the source-files baseline (Recommended)",
|
|
77
77
|
hint: built
|
|
78
|
-
? "See whether the portable context helps on the saved questions"
|
|
78
|
+
? "See whether the portable context helps your agents on the saved questions"
|
|
79
79
|
: hasSavedRawBaseline
|
|
80
80
|
? "Refresh the saved source-files baseline on the current questions"
|
|
81
81
|
: "See whether the source files are already good enough before compiling",
|
|
@@ -83,16 +83,16 @@ async function promptDatasetAction(dataset, built, latestComparison) {
|
|
|
83
83
|
}
|
|
84
84
|
options.push({
|
|
85
85
|
value: "compile",
|
|
86
|
-
label: built ? "Rebuild portable context" : "Build portable context",
|
|
86
|
+
label: built ? "Rebuild portable context for your agents" : "Build portable context for your agents",
|
|
87
87
|
hint: built
|
|
88
88
|
? "Refresh it from the current source files"
|
|
89
89
|
: dataset.checks.length > 0
|
|
90
90
|
? latestRawPasses
|
|
91
|
-
? "Optional: build portable context for a side-by-side comparison with the latest saved source-files run"
|
|
91
|
+
? "Optional: build portable context for your agents for a side-by-side comparison with the latest saved source-files run"
|
|
92
92
|
: hasSavedRawBaseline
|
|
93
|
-
? "Some questions still fail on the source files. Build portable context and compare it on the same questions"
|
|
94
|
-
: "Build portable context and compare it on the same questions"
|
|
95
|
-
: "Build it first, then measure it
|
|
93
|
+
? "Some questions still fail on the source files. Build portable context for your agents and compare it on the same questions"
|
|
94
|
+
: "Build portable context for your agents and compare it on the same questions"
|
|
95
|
+
: "Build it for your agents first, then measure it",
|
|
96
96
|
}, {
|
|
97
97
|
value: "dataset",
|
|
98
98
|
label: "Add another setup",
|
|
@@ -126,14 +126,14 @@ function printDatasetRecommendation(dataset, built, latestComparison) {
|
|
|
126
126
|
if (!built) {
|
|
127
127
|
if (latestComparison?.raw) {
|
|
128
128
|
p.log.info(latestComparison.raw.passed_cases === latestComparison.raw.total_cases
|
|
129
|
-
? `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Rerun the baseline if you want current numbers for a different agent set, or build portable context if you want a side-by-side comparison.`
|
|
130
|
-
: `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Some questions still fail on the source files. Build portable context if you want Interf to prepare context for this work and compare it on the same questions.`);
|
|
129
|
+
? `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Rerun the baseline if you want current numbers for a different agent set, or build portable context for your agents if you want a side-by-side comparison.`
|
|
130
|
+
: `Latest source-files run: ${latestComparison.raw.passed_cases}/${latestComparison.raw.total_cases}. Some questions still fail on the source files. Build portable context for your agents if you want Interf to prepare context for this work and compare it on the same questions.`);
|
|
131
131
|
return;
|
|
132
132
|
}
|
|
133
133
|
p.log.info("Recommended first step: measure the source-files baseline before compiling.");
|
|
134
134
|
return;
|
|
135
135
|
}
|
|
136
|
-
p.log.info("Recommended first step: compare the source-files baseline and the portable context.");
|
|
136
|
+
p.log.info("Recommended first step: compare the source-files baseline and the portable context your agents would use.");
|
|
137
137
|
}
|
|
138
138
|
async function promptPostBaselineAction(rows) {
|
|
139
139
|
const status = currentRunTargetStatus(rows, "raw");
|
|
@@ -154,13 +154,13 @@ async function promptPostBaselineAction(rows) {
|
|
|
154
154
|
{
|
|
155
155
|
value: "compile",
|
|
156
156
|
label: compileRecommended
|
|
157
|
-
? "Build portable context (Recommended)"
|
|
158
|
-
: "Build portable context",
|
|
157
|
+
? "Build portable context for your agents (Recommended)"
|
|
158
|
+
: "Build portable context for your agents",
|
|
159
159
|
hint: compileRecommended
|
|
160
160
|
? status === "mixed"
|
|
161
|
-
? "Some selected agents still failed on the source files. Build portable context and compare it on the same questions"
|
|
162
|
-
: "Build portable context and compare it on the same questions"
|
|
163
|
-
: "Build portable context and compare it on the same questions",
|
|
161
|
+
? "Some selected agents still failed on the source files. Build portable context for your agents and compare it on the same questions"
|
|
162
|
+
: "Build portable context for your agents and compare it on the same questions"
|
|
163
|
+
: "Build portable context for your agents and compare it on the same questions",
|
|
164
164
|
},
|
|
165
165
|
{
|
|
166
166
|
value: "edit",
|
|
@@ -407,7 +407,7 @@ async function runCompiledActionMenu(sourcePath, compiledConfig, options = {}) {
|
|
|
407
407
|
return;
|
|
408
408
|
}
|
|
409
409
|
if (compileResult.testedDuringCompile) {
|
|
410
|
-
p.log.info("This compile run already checked the portable context on the saved questions.");
|
|
410
|
+
p.log.info("This compile run already checked the portable context for your agents on the saved questions.");
|
|
411
411
|
p.log.info("Run `interf test` later if you want a fresh side-by-side comparison summary.");
|
|
412
412
|
return;
|
|
413
413
|
}
|
|
@@ -426,14 +426,14 @@ async function runCompiledActionMenu(sourcePath, compiledConfig, options = {}) {
|
|
|
426
426
|
if (compiledConfig.checks.length === 0)
|
|
427
427
|
return;
|
|
428
428
|
if (compileResult.testedDuringCompile) {
|
|
429
|
-
p.log.info("This compile run already checked the portable context on the compile agent.");
|
|
429
|
+
p.log.info("This compile run already checked the portable context for your agents on the compile agent.");
|
|
430
430
|
}
|
|
431
431
|
const runCompiledTest = await p.confirm({
|
|
432
432
|
message: compileResult.testedDuringCompile
|
|
433
433
|
? "Run a fresh source-files versus portable-context comparison now?"
|
|
434
434
|
: builtCompiledPath
|
|
435
|
-
? "Run source files and portable context on the saved questions now?"
|
|
436
|
-
: "Compare source files and portable context on the saved questions now?",
|
|
435
|
+
? "Run source files and the portable context your agents would use on the saved questions now?"
|
|
436
|
+
: "Compare source files and the portable context your agents would use on the saved questions now?",
|
|
437
437
|
initialValue: true,
|
|
438
438
|
});
|
|
439
439
|
if (p.isCancel(runCompiledTest) || !runCompiledTest)
|
|
@@ -370,7 +370,7 @@ async function promptCompiledChecks(options) {
|
|
|
370
370
|
if (options.initialAbout) {
|
|
371
371
|
p.log.info(`About: ${options.initialAbout}`);
|
|
372
372
|
}
|
|
373
|
-
p.log.info("Questions are simple question-and-answer pairs Interf uses to check accuracy
|
|
373
|
+
p.log.info("Questions are simple question-and-answer pairs Interf uses to check accuracy, so you can decide whether to trust the portable context.");
|
|
374
374
|
const mode = await p.select({
|
|
375
375
|
message: "How should Interf create them?",
|
|
376
376
|
options: [
|
|
@@ -544,7 +544,7 @@ export async function promptCompileLoopSelection(options) {
|
|
|
544
544
|
once: {
|
|
545
545
|
value: "once",
|
|
546
546
|
label: "Compile once",
|
|
547
|
-
hint: "Build portable context once with the selected workflow",
|
|
547
|
+
hint: "Build portable context for your agents once with the selected workflow",
|
|
548
548
|
},
|
|
549
549
|
"self-improving": {
|
|
550
550
|
value: "self-improving",
|
|
@@ -18,7 +18,7 @@ function statusColor(status) {
|
|
|
18
18
|
}
|
|
19
19
|
export const statusCommand = {
|
|
20
20
|
command: "status",
|
|
21
|
-
describe: "Show deterministic health for portable context",
|
|
21
|
+
describe: "Show deterministic health for the portable context your agents use",
|
|
22
22
|
handler: async () => {
|
|
23
23
|
let compiledPath = null;
|
|
24
24
|
const detected = detectInterf(process.cwd());
|
|
@@ -34,7 +34,7 @@ export const statusCommand = {
|
|
|
34
34
|
if (local.length === 0) {
|
|
35
35
|
process.exitCode = 1;
|
|
36
36
|
console.log(chalk.red(" No portable contexts found."));
|
|
37
|
-
console.log(chalk.dim(" Run `interf`, save questions, and compile portable context first."));
|
|
37
|
+
console.log(chalk.dim(" Run `interf`, save questions, and compile portable context for your agents first."));
|
|
38
38
|
return;
|
|
39
39
|
}
|
|
40
40
|
if (local.length === 1) {
|
|
@@ -75,7 +75,7 @@ export const statusCommand = {
|
|
|
75
75
|
console.log();
|
|
76
76
|
console.log(chalk.dim(latestComparisonState.stale
|
|
77
77
|
? " Saved test results are stale for the current questions. Run `interf test` again."
|
|
78
|
-
: " No saved comparison yet. Run `interf test` to measure source files and the portable context."));
|
|
78
|
+
: " No saved comparison yet. Run `interf test` to measure source files and the portable context your agents would use."));
|
|
79
79
|
}
|
|
80
80
|
console.log();
|
|
81
81
|
const metricOrder = [
|
|
@@ -9,7 +9,7 @@ import { printAgentTestFailures, printAgentTestMatrix, printSavedTestComparisonS
|
|
|
9
9
|
import { listRunAgentOptions, promptForTestAgents, resolveNamedLocalExecutor, resolveOrConfigureLocalExecutor, } from "./executor-flow.js";
|
|
10
10
|
export const testCommand = {
|
|
11
11
|
command: "test",
|
|
12
|
-
describe: "Compare source files and portable context on saved questions",
|
|
12
|
+
describe: "Compare source files and the portable context your agents would use on saved questions",
|
|
13
13
|
builder: (yargs) => addExecutionProfileOptions(yargs)
|
|
14
14
|
.option("dataset", {
|
|
15
15
|
type: "string",
|
|
@@ -17,7 +17,7 @@ export const testCommand = {
|
|
|
17
17
|
})
|
|
18
18
|
.option("target", {
|
|
19
19
|
choices: ["both", "raw", "compiled"],
|
|
20
|
-
describe: "Test source files, the portable context, or both. Default: both when portable context exists, otherwise source files.",
|
|
20
|
+
describe: "Test source files, the portable context your agents would use, or both. Default: both when portable context exists, otherwise source files.",
|
|
21
21
|
})
|
|
22
22
|
.option("keep-sandboxes", {
|
|
23
23
|
type: "boolean",
|
|
@@ -49,8 +49,8 @@ async function promptTestMode(hasBuiltCompiled) {
|
|
|
49
49
|
options: [
|
|
50
50
|
{
|
|
51
51
|
value: "both",
|
|
52
|
-
label: "Compare source files and portable context (Recommended)",
|
|
53
|
-
hint: "Measure whether the portable context helps on the saved questions",
|
|
52
|
+
label: "Compare source files and the portable context your agents would use (Recommended)",
|
|
53
|
+
hint: "Measure whether the portable context helps your agents on the saved questions",
|
|
54
54
|
},
|
|
55
55
|
{
|
|
56
56
|
value: "raw",
|
|
@@ -60,7 +60,7 @@ async function promptTestMode(hasBuiltCompiled) {
|
|
|
60
60
|
{
|
|
61
61
|
value: "compiled",
|
|
62
62
|
label: "Portable context only",
|
|
63
|
-
hint: "Measure the current portable context on the saved questions",
|
|
63
|
+
hint: "Measure the current portable context for your agents on the saved questions",
|
|
64
64
|
},
|
|
65
65
|
],
|
|
66
66
|
});
|
|
@@ -240,13 +240,13 @@ export async function runTestCommand(argv = {}) {
|
|
|
240
240
|
return false;
|
|
241
241
|
if (selectedMode === "compiled" && !hasBuiltCompiled) {
|
|
242
242
|
process.exitCode = 1;
|
|
243
|
-
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context yet.`));
|
|
243
|
+
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context for your agents yet.`));
|
|
244
244
|
console.log(chalk.dim(" Run `interf compile` first."));
|
|
245
245
|
return false;
|
|
246
246
|
}
|
|
247
247
|
if (selectedMode === "both" && !hasBuiltCompiled) {
|
|
248
248
|
process.exitCode = 1;
|
|
249
|
-
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context yet, so Interf cannot compare source files and the portable context.`));
|
|
249
|
+
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context for your agents yet, so Interf cannot compare source files and the portable context your agents would use.`));
|
|
250
250
|
console.log(chalk.dim(" Run `interf compile` first, or rerun with `--target raw`."));
|
|
251
251
|
return false;
|
|
252
252
|
}
|
|
@@ -268,7 +268,7 @@ export async function runTestCommand(argv = {}) {
|
|
|
268
268
|
})));
|
|
269
269
|
if ((mode === "compiled" || mode === "both") && rows.some((row) => !row.compiledOutcome)) {
|
|
270
270
|
process.exitCode = 1;
|
|
271
|
-
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context yet.`));
|
|
271
|
+
console.log(chalk.red(` Setup "${selectedCompiled.name}" does not have portable context for your agents yet.`));
|
|
272
272
|
console.log(chalk.dim(" Run `interf compile` first."));
|
|
273
273
|
return false;
|
|
274
274
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@interf/compiler",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.5",
|
|
4
4
|
"description": "Interf prepares context for your agents by checking your files first, building a local folder when needed, and using self-improving loops until more questions pass.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -44,6 +44,7 @@
|
|
|
44
44
|
"build": "node -e \"require('fs').rmSync('dist',{recursive:true,force:true})\" && tsc && chmod 755 dist/bin.js",
|
|
45
45
|
"dev": "tsc --watch",
|
|
46
46
|
"refresh:bootstrap-mirror": "node scripts/docs/sync-bootstrap-mirror.mjs",
|
|
47
|
+
"docs:sync-public-test-example": "npm run build && node scripts/docs/sync-public-test-example.mjs",
|
|
47
48
|
"test:matrix": "npm run build && node scripts/matrix/run.mjs",
|
|
48
49
|
"test:smoke": "npm run build && node --test test/**/*.test.mjs",
|
|
49
50
|
"test": "npm run test:smoke",
|