@f-o-h/cli 0.1.12 → 0.1.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -57
- package/dist/foh.js +10 -7
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@ AI-operator provisioning CLI for Front Of House.
|
|
|
4
4
|
|
|
5
5
|
Public mirror: https://github.com/iiko38/front-of-house-cli
|
|
6
6
|
|
|
7
|
-
Current published baseline: `@f-o-h/cli@0.1.
|
|
7
|
+
Current published baseline: `@f-o-h/cli@0.1.13`
|
|
8
8
|
|
|
9
9
|
This mirror is a generated release artifact. The private product monorepo is not
|
|
10
10
|
published here, and no open-source license is granted unless stated separately.
|
|
@@ -85,67 +85,67 @@ The CLI defaults to the production API at `https://api.frontofhouse.okii.uk`.
|
|
|
85
85
|
|
|
86
86
|
## External-Agent Eval Capture
|
|
87
87
|
|
|
88
|
-
Use this when testing whether a clean coding agent can start from public docs
|
|
89
|
-
and the public npm package without private repo context:
|
|
90
|
-
|
|
91
|
-
```bash
|
|
92
|
-
foh eval external-agent batch \
|
|
93
|
-
--models openai/codex,anthropic/claude,cursor/agent \
|
|
94
|
-
--prompt-version blank-setup.v1 \
|
|
95
|
-
--json
|
|
96
|
-
```
|
|
97
|
-
|
|
98
|
-
Run each returned launch command in a clean agent terminal:
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
foh eval external-agent run \
|
|
88
|
+
Use this when testing whether a clean coding agent can start from public docs
|
|
89
|
+
and the public npm package without private repo context:
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
foh eval external-agent batch \
|
|
93
|
+
--models openai/codex,anthropic/claude,cursor/agent \
|
|
94
|
+
--prompt-version blank-setup.v1 \
|
|
95
|
+
--json
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Run each returned launch command in a clean agent terminal:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
foh eval external-agent run \
|
|
102
102
|
--model-provider openai \
|
|
103
103
|
--model-name codex \
|
|
104
104
|
--prompt-version blank-setup.v1
|
|
105
105
|
```
|
|
106
106
|
|
|
107
|
-
The command writes a versioned prompt, launches an instrumented shell, captures
|
|
108
|
-
FOH CLI commands into `commands.ndjson`, and finalizes `run.json` as an
|
|
109
|
-
`external_agent_run.v1` artifact when the shell exits.
|
|
110
|
-
|
|
111
|
-
For guarded programmable-runner planning:
|
|
112
|
-
|
|
113
|
-
```bash
|
|
114
|
-
foh eval external-agent execute \
|
|
115
|
-
--runner codex \
|
|
116
|
-
--batch test-results/external-agent-runs/<batch>/batch.json \
|
|
117
|
-
--dry-run \
|
|
118
|
-
--json
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
This writes `executor-plan.json`, creates intentionally empty clean workspaces
|
|
122
|
-
outside the private repo, validates the local Codex binary/help flags, and
|
|
123
|
-
prints exact `codex exec` commands without executing them.
|
|
124
|
-
|
|
125
|
-
Before promoting run artifacts, scan and redact them:
|
|
126
|
-
|
|
127
|
-
```bash
|
|
128
|
-
foh eval external-agent scan-artifacts \
|
|
129
|
-
--run-dir test-results/external-agent-runs/<batch>/<run-id> \
|
|
130
|
-
--private-repo-root <private-repo-root> \
|
|
131
|
-
--write-redacted \
|
|
132
|
-
--json
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
After dry-run review, one controlled Codex run can be launched explicitly with
|
|
136
|
-
`--live`. Live mode is intentionally limited to one run per batch and finalizes
|
|
137
|
-
`run.json` even on timeout or non-zero exit:
|
|
138
|
-
|
|
139
|
-
```bash
|
|
140
|
-
foh eval external-agent execute \
|
|
141
|
-
--runner codex \
|
|
142
|
-
--batch test-results/external-agent-runs/<one-model-batch>/batch.json \
|
|
143
|
-
--timeout-minutes 30 \
|
|
144
|
-
--live \
|
|
145
|
-
--json
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
## Local Scenario Suites
|
|
107
|
+
The command writes a versioned prompt, launches an instrumented shell, captures
|
|
108
|
+
FOH CLI commands into `commands.ndjson`, and finalizes `run.json` as an
|
|
109
|
+
`external_agent_run.v1` artifact when the shell exits.
|
|
110
|
+
|
|
111
|
+
For guarded programmable-runner planning:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
foh eval external-agent execute \
|
|
115
|
+
--runner codex \
|
|
116
|
+
--batch test-results/external-agent-runs/<batch>/batch.json \
|
|
117
|
+
--dry-run \
|
|
118
|
+
--json
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
This writes `executor-plan.json`, creates intentionally empty clean workspaces
|
|
122
|
+
outside the private repo, validates the local Codex binary/help flags, and
|
|
123
|
+
prints exact `codex exec` commands without executing them.
|
|
124
|
+
|
|
125
|
+
Before promoting run artifacts, scan and redact them:
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
foh eval external-agent scan-artifacts \
|
|
129
|
+
--run-dir test-results/external-agent-runs/<batch>/<run-id> \
|
|
130
|
+
--private-repo-root <private-repo-root> \
|
|
131
|
+
--write-redacted \
|
|
132
|
+
--json
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
After dry-run review, one controlled Codex run can be launched explicitly with
|
|
136
|
+
`--live`. Live mode is intentionally limited to one run per batch and finalizes
|
|
137
|
+
`run.json` even on timeout or non-zero exit:
|
|
138
|
+
|
|
139
|
+
```bash
|
|
140
|
+
foh eval external-agent execute \
|
|
141
|
+
--runner codex \
|
|
142
|
+
--batch test-results/external-agent-runs/<one-model-batch>/batch.json \
|
|
143
|
+
--timeout-minutes 30 \
|
|
144
|
+
--live \
|
|
145
|
+
--json
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
## Local Scenario Suites
|
|
149
149
|
|
|
150
150
|
`foh test run --suite <file>` runs deterministic widget-runtime checks for a
|
|
151
151
|
specific agent. The suite format supports reply text checks plus structured
|
package/dist/foh.js
CHANGED
|
@@ -32640,7 +32640,7 @@ var StdioServerTransport = class {
|
|
|
32640
32640
|
};
|
|
32641
32641
|
|
|
32642
32642
|
// src/lib/cli-version.ts
|
|
32643
|
-
var CLI_VERSION = "0.1.
|
|
32643
|
+
var CLI_VERSION = "0.1.13";
|
|
32644
32644
|
|
|
32645
32645
|
// src/commands/mcp-serve.ts
|
|
32646
32646
|
var DEFAULT_TIMEOUT_MS = 12e4;
|
|
@@ -38739,6 +38739,9 @@ function proofArtifactPasses(runDir) {
|
|
|
38739
38739
|
function readIfExists(path2) {
|
|
38740
38740
|
return (0, import_fs14.existsSync)(path2) ? (0, import_fs14.readFileSync)(path2, "utf8") : "";
|
|
38741
38741
|
}
|
|
38742
|
+
function relativeArtifactName(path2) {
|
|
38743
|
+
return (0, import_path12.basename)(path2);
|
|
38744
|
+
}
|
|
38742
38745
|
function classifyRun(input) {
|
|
38743
38746
|
if (input.timedOut) return { status: "hold", reasonCode: "codex_runner_timeout" };
|
|
38744
38747
|
if (!input.artifactSafetyOk) return { status: "fail", reasonCode: "external_agent_artifact_safety_blocked" };
|
|
@@ -38791,21 +38794,21 @@ function buildExecutedRunArtifact(input) {
|
|
|
38791
38794
|
commands_run: commands.map((command) => command.command),
|
|
38792
38795
|
docs_pages_used: [],
|
|
38793
38796
|
artifacts: {
|
|
38794
|
-
terminal_transcript: input.run.outputs.jsonl,
|
|
38797
|
+
terminal_transcript: relativeArtifactName(input.run.outputs.jsonl),
|
|
38795
38798
|
command_log: (0, import_fs14.existsSync)((0, import_path12.join)(input.run.run_dir, "commands.ndjson")) ? "commands.ndjson" : null,
|
|
38796
38799
|
proof_bundle: (0, import_fs14.existsSync)((0, import_path12.join)(input.run.run_dir, "proof.json")) ? "proof.json" : null,
|
|
38797
38800
|
replay_packet: (0, import_fs14.existsSync)((0, import_path12.join)(input.run.run_dir, "replay.json")) ? "replay.json" : null,
|
|
38798
38801
|
knowledge_packet: (0, import_fs14.existsSync)((0, import_path12.join)(input.run.run_dir, "knowledge.json")) ? "knowledge.json" : null,
|
|
38799
38802
|
improvement_packet: input.status === "pass" ? null : "improvement-packet.json",
|
|
38800
38803
|
notes: (0, import_fs14.existsSync)((0, import_path12.join)(input.run.run_dir, "notes.md")) ? "notes.md" : null,
|
|
38801
|
-
codex_last_message: input.run.outputs.last_message,
|
|
38802
|
-
codex_stderr: input.run.outputs.stderr,
|
|
38803
|
-
artifact_safety: input.run.outputs.artifact_safety
|
|
38804
|
+
codex_last_message: relativeArtifactName(input.run.outputs.last_message),
|
|
38805
|
+
codex_stderr: relativeArtifactName(input.run.outputs.stderr),
|
|
38806
|
+
artifact_safety: relativeArtifactName(input.run.outputs.artifact_safety)
|
|
38804
38807
|
},
|
|
38805
38808
|
summary: input.status === "pass" ? "Controlled Codex external-agent run produced passing proof evidence." : `Controlled Codex external-agent run ended as ${input.status} with reason ${input.reasonCode}.`,
|
|
38806
38809
|
next_commands: input.status === "pass" ? ["corepack pnpm eval:external-agent:runs:summary"] : [
|
|
38807
|
-
|
|
38808
|
-
|
|
38810
|
+
"foh eval external-agent scan-artifacts --run-dir <run_dir> --private-repo-root <private_repo_root> --write-redacted --json",
|
|
38811
|
+
"foh bug improve --from external-agent-run --file <run_dir>/run.json --out <run_dir>/improvement-packet.json --json",
|
|
38809
38812
|
"corepack pnpm eval:external-agent:runs:summary"
|
|
38810
38813
|
]
|
|
38811
38814
|
};
|