@ckanner/flywheel 0.31.0 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +122 -5
- package/CONTRIBUTING.md +19 -0
- package/LICENSE +21 -0
- package/README.en.md +4 -171
- package/README.md +152 -269
- package/README.zh-CN.md +116 -0
- package/bin/flywheel-claude-code-pro-worker.js +1 -1
- package/bin/flywheel-codex-pro-worker.js +1 -1
- package/bin/flywheel-dsh-worker.js +1 -1
- package/bin/flywheel.js +1 -1
- package/bin/runtime.js +1 -1
- package/docs/README.md +86 -0
- package/docs/README.zh-CN.md +64 -0
- package/docs/archive/README.md +13 -0
- package/docs/archive/codex-pro-adapter-handoff.md +197 -0
- package/docs/archive/design-v0.24.md +540 -0
- package/docs/design.md +812 -0
- package/docs/en/claude-code-pro.md +25 -16
- package/docs/en/configuration.md +405 -145
- package/docs/en/contributing.md +303 -0
- package/docs/en/flow-console.md +142 -24
- package/docs/en/getting-started.md +232 -126
- package/docs/en/investigation.md +263 -0
- package/docs/en/jev-shadow.md +5 -0
- package/docs/en/jev.md +240 -0
- package/docs/en/model-policy.md +154 -62
- package/docs/en/releasing.md +207 -0
- package/docs/en/session-evaluation.md +4 -2
- package/docs/en/session-recovery.md +88 -6
- package/docs/en/workflow.md +241 -114
- package/docs/evaluation/README.md +12 -0
- package/docs/evaluation/jev-historical-provisional.json +138 -0
- package/docs/evaluation/jev-historical-result-2026-11-10.json +174 -0
- package/docs/plans/2026-11-01-dsh-integration.md +67 -0
- package/docs/plans/2026-11-01-model-profiles-preflight.md +29 -0
- package/docs/plans/2026-11-01-npm-installation.md +19 -0
- package/docs/plans/2026-11-03-codex-pro-default.md +28 -0
- package/docs/plans/2026-11-06-adaptive-workflow.md +365 -0
- package/docs/plans/2026-11-07-session-lifecycle.en.md +424 -0
- package/docs/plans/2026-11-07-session-lifecycle.md +337 -0
- package/docs/plans/2026-11-08-flow-console.md +271 -0
- package/docs/plans/2026-11-10-jev-shadow.md +32 -0
- package/docs/plans/2026-11-10-task-discovery.md +17 -0
- package/docs/plans/2026-11-10-workbench-redesign.md +19 -0
- package/docs/plans/2026-11-12-feedback-driven-workflows.md +739 -0
- package/docs/plans/2026-11-12-unified-jev-decisions.md +89 -0
- package/docs/plans/2026-11-13-jev-execution-decisions.md +110 -0
- package/docs/plans/2026-11-13-p5-ready-queue-prewarm.md +178 -0
- package/docs/plans/README.md +25 -0
- package/docs/reviews/2026-08-24-v0.18-review-fixplan.md +265 -0
- package/docs/reviews/2026-09-12-pr4-pr5-review.md +70 -0
- package/docs/reviews/README.md +12 -0
- package/docs/validation/2026-11-01-npm-installation.md +105 -0
- package/docs/validation/2026-11-01-profiles-preflight.md +67 -0
- package/docs/validation/2026-11-01-results.md +114 -0
- package/docs/validation/2026-11-03-codex-pro-default.md +90 -0
- package/docs/validation/2026-11-04-dsh-native-view.md +85 -0
- package/docs/validation/2026-11-04-release-0.28.1.md +33 -0
- package/docs/validation/2026-11-04-release-0.28.2.md +35 -0
- package/docs/validation/2026-11-05-dsh-0.2.2-compatibility.md +26 -0
- package/docs/validation/2026-11-05-registry-smoke-recovery.md +26 -0
- package/docs/validation/2026-11-05-release-0.28.3.md +32 -0
- package/docs/validation/2026-11-05-subscription-model-access.md +60 -0
- package/docs/validation/2026-11-05-view-order.md +14 -0
- package/docs/validation/2026-11-07-adaptive-workflow.md +79 -0
- package/docs/validation/2026-11-07-claude-code-pro.md +144 -0
- package/docs/validation/2026-11-07-session-lifecycle.md +136 -0
- package/docs/validation/2026-11-08-session-p3-p4.md +99 -0
- package/docs/validation/2026-11-09-flow-console.md +81 -0
- package/docs/validation/2026-11-10-jev-shadow.md +50 -0
- package/docs/validation/2026-11-10-task-discovery.md +45 -0
- package/docs/validation/2026-11-10-workbench-redesign.md +54 -0
- package/docs/validation/2026-11-11-release-0.30.2.md +65 -0
- package/docs/validation/2026-11-11-release-0.31.0.md +46 -0
- package/docs/validation/2026-11-12-dsh-0.2.2-rc.2.md +58 -0
- package/docs/validation/2026-11-12-feedback-driven-workflows.md +187 -0
- package/docs/validation/2026-11-12-release-0.31.1.md +58 -0
- package/docs/validation/2026-11-12-session-console-ux.md +46 -0
- package/docs/validation/2026-11-12-unified-jev-decisions.md +93 -0
- package/docs/validation/2026-11-13-dsh-settings-controls.md +104 -0
- package/docs/validation/2026-11-13-jev-execution-decisions.md +140 -0
- package/docs/validation/2026-11-13-p5-ready-queue-prewarm.md +138 -0
- package/docs/validation/2026-11-13-p5-review-closure.md +114 -0
- package/docs/validation/2026-11-13-p5-tracked-runtime-fixes.md +108 -0
- package/docs/validation/2026-11-13-prewarm-receipts-and-expiry-tests.md +92 -0
- package/docs/validation/2026-11-13-workspace-redesign.md +66 -0
- package/docs/validation/README.md +51 -0
- package/docs/validation/evidence/2026-11-08-session-p3-p4.json +5430 -0
- package/docs/validation/evidence/2026-11-13-p5-npm-candidate.json +54 -0
- package/docs/validation/evidence/claude-node-bridges.txt +118 -0
- package/docs/validation/evidence/claude-npm-build.txt +15 -0
- package/docs/validation/evidence/claude-npm-candidate.json +54 -0
- package/docs/validation/evidence/claude-npm-install.txt +21 -0
- package/docs/validation/evidence/claude-npm-tests.txt +145 -0
- package/docs/validation/evidence/claude-plugin-tests.txt +205 -0
- package/docs/validation/evidence/claude-quality-gate.txt +29 -0
- package/docs/validation/evidence/claude-race.txt +7 -0
- package/docs/validation/evidence/codex-pro-070-probes.txt +4 -0
- package/docs/validation/evidence/dsh-bridge.txt +4 -0
- package/docs/validation/evidence/dsh-native-view-bridge-tests.txt +1 -0
- package/docs/validation/evidence/dsh-native-view-build.txt +15 -0
- package/docs/validation/evidence/dsh-native-view-candidate.json +54 -0
- package/docs/validation/evidence/dsh-native-view-inspect.txt +27 -0
- package/docs/validation/evidence/dsh-native-view-install-tests.txt +19 -0
- package/docs/validation/evidence/dsh-native-view-npm-tests.txt +143 -0
- package/docs/validation/evidence/dsh-native-view-plugin-tests.txt +99 -0
- package/docs/validation/evidence/dsh-native-view-quality-gate.txt +29 -0
- package/docs/validation/evidence/dsh-plugin-tests.txt +48 -0
- package/docs/validation/evidence/function-coverage.txt +378 -0
- package/docs/validation/evidence/npm-build.txt +13 -0
- package/docs/validation/evidence/npm-contract-tests.txt +65 -0
- package/docs/validation/evidence/npm-dsh-bridge.txt +18 -0
- package/docs/validation/evidence/npm-dsh-plugin-tests.txt +62 -0
- package/docs/validation/evidence/npm-function-coverage.txt +420 -0
- package/docs/validation/evidence/npm-install-tests.txt +20 -0
- package/docs/validation/evidence/npm-quality-gate.txt +29 -0
- package/docs/validation/evidence/npm-race.txt +7 -0
- package/docs/validation/evidence/pr-dsh-bridge.txt +4 -0
- package/docs/validation/evidence/pr-dsh-plugin-tests.txt +48 -0
- package/docs/validation/evidence/pr-function-coverage.txt +379 -0
- package/docs/validation/evidence/pr-quality-gate.txt +28 -0
- package/docs/validation/evidence/pr-race.txt +6 -0
- package/docs/validation/evidence/profiles-control-smoke.json +1 -0
- package/docs/validation/evidence/profiles-dsh-bridge.txt +18 -0
- package/docs/validation/evidence/profiles-dsh-plugin-tests.txt +59 -0
- package/docs/validation/evidence/profiles-function-coverage.txt +411 -0
- package/docs/validation/evidence/profiles-quality-gate.txt +29 -0
- package/docs/validation/evidence/profiles-race.txt +7 -0
- package/docs/validation/evidence/quality-gate.txt +28 -0
- package/docs/validation/evidence/race.txt +6 -0
- package/docs/validation/evidence/watchdog-startup-after.txt +2 -0
- package/docs/validation/evidence/watchdog-startup-before.txt +6 -0
- package/docs/validation/evidence/watchdog-startup-quality-gate.txt +29 -0
- package/docs/validation/evidence/watchdog-startup-race.txt +7 -0
- package/docs/{claude-code-pro.md → zh-CN/claude-code-pro.md} +23 -17
- package/docs/{configuration.md → zh-CN/configuration.md} +397 -142
- package/docs/zh-CN/contributing.md +240 -0
- package/docs/zh-CN/flow-console.md +216 -0
- package/docs/zh-CN/getting-started.md +352 -0
- package/docs/zh-CN/investigation.md +213 -0
- package/docs/zh-CN/jev-shadow.md +5 -0
- package/docs/zh-CN/jev.md +192 -0
- package/docs/zh-CN/model-policy.md +128 -0
- package/docs/zh-CN/releasing.md +120 -0
- package/docs/{session-evaluation.md → zh-CN/session-evaluation.md} +4 -2
- package/docs/{session-recovery.md → zh-CN/session-recovery.md} +75 -12
- package/docs/zh-CN/workflow.md +284 -0
- package/npm/install.js +1 -1
- package/package.json +12 -25
- package/docs/flow-console.md +0 -122
- package/docs/getting-started.md +0 -418
- package/docs/model-policy.md +0 -65
- package/docs/workflow.md +0 -314
|
@@ -0,0 +1,263 @@
|
|
|
1
|
+
# Bounded investigation and feedback-driven workflows
|
|
2
|
+
|
|
3
|
+
[Documentation](../README.md) · English · [简体中文](../zh-CN/investigation.md) · [Workflow](workflow.md)
|
|
4
|
+
|
|
5
|
+
Task is the recommended goal entry. It identifies the requested deliverable and
|
|
6
|
+
prepares the appropriate contract. Auto chooses Quick or Full only after a valid
|
|
7
|
+
implementation contract exists. An investigation can deliver a report on its own;
|
|
8
|
+
it is not a third implementation completion mode.
|
|
9
|
+
|
|
10
|
+
| Daily request | Entry | Engine behavior and endpoint |
|
|
11
|
+
| --- | --- | --- |
|
|
12
|
+
| Diagnose and explain | `/flywheel-task` | Managed evidence collection, approved diagnostic experiments and a report; no implementation |
|
|
13
|
+
| Diagnose and repair within scope | `/flywheel-task` | Supported report, then Spec and Auto; retain provenance, usage and scope |
|
|
14
|
+
| Repair an identified issue | Task Auto or explicit Quick | Qualification, mandatory checks, independent final Review and Git delivery |
|
|
15
|
+
| Propose a complex change for review | `/flywheel-new` | Stop after preparation; Run Full after implementation authorization |
|
|
16
|
+
| Continue an existing task | `/flywheel-run` | Resume its original preparation or implementation ledger |
|
|
17
|
+
| Run a particular stage | Plan / Exec / Verify / Review | Direct controls remain; stage success is not whole-task completion |
|
|
18
|
+
|
|
19
|
+
Kimi uses `/skill:flywheel-*`. Status and Stop work for investigations and
|
|
20
|
+
implementation. Existing repair authorization is retained; new scope or an
|
|
21
|
+
unresolved product decision needs a separate decision.
|
|
22
|
+
|
|
23
|
+
## Preparation contract
|
|
24
|
+
|
|
25
|
+
The host prepares an input from the user's goal and repository facts. Go validates
|
|
26
|
+
it and owns `preparation.json`. Users do not need to fill another large form, and
|
|
27
|
+
the host never edits preparation state or `state.json` directly. The installed CLI
|
|
28
|
+
exposes `preparation.schema.json`, `preparation-state.schema.json` and
|
|
29
|
+
`preparation.example.json` through `flywheel template`.
|
|
30
|
+
|
|
31
|
+
Version 1 records the goal, expected result, environment, time window, known
|
|
32
|
+
evidence, falsifiable hypotheses, exact read paths, concrete actions and budgets.
|
|
33
|
+
`intent` is `investigate` or `investigate-and-fix`. Repair intent requires a nonempty
|
|
34
|
+
`implementation-scope`; report-only intent grants no implementation scope. A Spec
|
|
35
|
+
is not required while only preparation exists.
|
|
36
|
+
|
|
37
|
+
Go executes the evidence actions. Models choose only among candidates already
|
|
38
|
+
approved in the contract:
|
|
39
|
+
|
|
40
|
+
| kind | Capability | Boundaries |
|
|
41
|
+
| --- | --- | --- |
|
|
42
|
+
| `read_file` | Read one approved UTF-8 file | Reject symlinks, special files and excessive input |
|
|
43
|
+
| `search_text` | Literal search in one approved file, with line numbers | No regular expressions or command interpolation |
|
|
44
|
+
| `list_files` | List names in one approved directory | No recursive content grant; at most 4096 names |
|
|
45
|
+
| `run_command` | Run exact argv against a temporary copy of approved inputs for reproduction or diagnosis | Explicit `host-process` authority, action timeout, controlled environment, process-group cancellation and cleanup |
|
|
46
|
+
| `http_read` | GET or HEAD against an exact approved URL | HTTPS (loopback HTTP for tests), no redirects/environment proxy, bounded response and output |
|
|
47
|
+
|
|
48
|
+
File paths must be clean repository-relative paths, and file actions reference an
|
|
49
|
+
approved path. Every action references existing hypotheses. The opened inode must
|
|
50
|
+
match the inspected inode; `os.Root` confines traversal. A directory grant permits
|
|
51
|
+
names only, not its descendants' contents. Command and HTTP arguments belong to the
|
|
52
|
+
frozen contract. JEV cannot generate another command, URL or credential variable.
|
|
53
|
+
|
|
54
|
+
`limits` freezes action count, overall deadline, per-file bytes and total output
|
|
55
|
+
bytes. Resume does not reset them. The deadline starts at preparation, including
|
|
56
|
+
paused time. Bounded UTF-8 excerpts retain a content digest; `truncated` means the
|
|
57
|
+
display is incomplete, not that an entire log or test inventory was observed.
|
|
58
|
+
|
|
59
|
+
### Reproduction tests and diagnostic commands
|
|
60
|
+
|
|
61
|
+
`run_command` uses `command.argv` without an implicit shell. `command.inputs`
|
|
62
|
+
lists exact approved regular files from the contract's `read-paths`. Each invocation
|
|
63
|
+
copies these inputs with their relative directory structure and runs in that copy;
|
|
64
|
+
ordinary writes to the copy are not transferred back to the source workspace. This
|
|
65
|
+
is a bounded reproduction input set: at most 32 UTF-8 files, subject to the contract's
|
|
66
|
+
file and output limits, with executable bits preserved. A directory entry does not
|
|
67
|
+
copy its descendants automatically. The following
|
|
68
|
+
action fragment illustrates the fields. Replace paths and the hypothesis ID with
|
|
69
|
+
those actually approved in the current contract:
|
|
70
|
+
|
|
71
|
+
```json
|
|
72
|
+
{
|
|
73
|
+
"id": "reproduce-timeout",
|
|
74
|
+
"kind": "run_command",
|
|
75
|
+
"purpose": "Reproduce the observed timeout with isolated inputs",
|
|
76
|
+
"hypotheses": ["timeout"],
|
|
77
|
+
"command": {
|
|
78
|
+
"argv": ["/usr/bin/python3", "tests/reproduce.py"],
|
|
79
|
+
"inputs": ["tests/reproduce.py", "src/settings.py"],
|
|
80
|
+
"env": [],
|
|
81
|
+
"timeout-ms": 30000,
|
|
82
|
+
"expect-exit": 1,
|
|
83
|
+
"execution-boundary": "host-process"
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
`host-process` explicitly means execution with the current user's real host
|
|
89
|
+
permissions. The copy keeps ordinary relative writes outside the original workspace;
|
|
90
|
+
**it is not an operating-system sandbox or a network boundary**. Approve the concrete
|
|
91
|
+
program and arguments within the task's existing authority. Full-access settings
|
|
92
|
+
and the JEV switch do not grant that action authority on their own. `env` contains
|
|
93
|
+
environment-variable names, never secret values. An expected nonzero exit can
|
|
94
|
+
establish a reproduced failure. Timeout, cancellation, signals, output truncation
|
|
95
|
+
and cleanup errors cannot stand in for a normal expected exit.
|
|
96
|
+
An expected nonzero exit can be valid reproduction evidence, but does not by itself
|
|
97
|
+
qualify the remaining collection for deferral or early finish.
|
|
98
|
+
|
|
99
|
+
An observation's `execution` records physical exit/signal, timeout, cancellation,
|
|
100
|
+
input digest and copy cleanup. `removed` means cleanup completed, `retained` keeps
|
|
101
|
+
the experiment for reconciliation, and `not-created` means no copy was created.
|
|
102
|
+
Unknown process or cleanup outcomes leave the original invocation pending and block
|
|
103
|
+
further actions. Recovery reads its original receipt rather than executing again
|
|
104
|
+
to guess the result.
|
|
105
|
+
|
|
106
|
+
### Approved external queries
|
|
107
|
+
|
|
108
|
+
The `http_read` action's `http` object contains `url`, `method` (GET/HEAD),
|
|
109
|
+
`headers-env`, `timeout-ms`, `max-bytes` and `expect-status`. An approved health
|
|
110
|
+
query might use `headers-env: {"Authorization": "INCIDENT_STATUS_AUTH"}`. The
|
|
111
|
+
variable supplies the complete header value; the contract and receipt do not store
|
|
112
|
+
that secret value. The URL is an exact authorization: no redirects, environment
|
|
113
|
+
proxy or embedded credentials. Go bounds request time and bytes and records the
|
|
114
|
+
actual HTTP status and output digest. A GET/HEAD grant does not authorize arbitrary
|
|
115
|
+
services, production restarts, cache flushes or data repair.
|
|
116
|
+
|
|
117
|
+
Structured command/HTTP outcomes, such as exit code, status and completeness, use
|
|
118
|
+
`diagnostic-facts` and may participate under the default data policy. Raw output is
|
|
119
|
+
included as a bounded excerpt only when `logs` is explicitly allowed. Otherwise
|
|
120
|
+
the snapshot records that missing content, and JEV can still choose from structured
|
|
121
|
+
facts. File evidence uses `source`. Action approval does not add raw-log sharing
|
|
122
|
+
permission, and a permitted category does not remove the need to protect secrets.
|
|
123
|
+
|
|
124
|
+
## CLI and desktop controls
|
|
125
|
+
|
|
126
|
+
Create the actual task directory, then submit a complete preparation request:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
flywheel control investigate --task-dir "$task_dir" --cwd "$repo_dir" < start-request.json
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
The outer request is `{ "action": "start", "contract": <complete-contract> }`;
|
|
133
|
+
read the installed example and replace its paths, goals and permissions. `start`
|
|
134
|
+
saves ready preparation without executing actions. To run, submit
|
|
135
|
+
`{ "action": "run", "revision": <observed-revision> }` through the same command.
|
|
136
|
+
|
|
137
|
+
Go starts a background owner under the existing task lifetime lease and token-bound
|
|
138
|
+
stop protocol. In DSH, `flywheel_control` accepts `action: investigate` and the nested
|
|
139
|
+
`investigation` request, associating the exact task with the conversation. The panel
|
|
140
|
+
shows budgets, observations, excerpts, process exits/HTTP status, cancellation and
|
|
141
|
+
copy cleanup, decisions and reports, with continue, reconcile, stop and refresh
|
|
142
|
+
controls.
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
flywheel control status --task-dir "$task_dir" --cwd "$repo_dir"
|
|
146
|
+
flywheel status "$task_dir"
|
|
147
|
+
flywheel stop "$task_dir"
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
`ok:true` acknowledges a control request. `running` comes from the lifetime lease.
|
|
151
|
+
An investigation-only response contains `preparation`, not a fabricated code-task
|
|
152
|
+
`state`. Reconcile an uncertain result in the original directory before mutation.
|
|
153
|
+
|
|
154
|
+
## Report and repair handoff
|
|
155
|
+
|
|
156
|
+
`awaiting-report` means observations are available. The current host model submits
|
|
157
|
+
one evidence-based report; Go checks fields, hypothesis coverage and references.
|
|
158
|
+
Separate facts from inference, classify each hypothesis as supported, contradicted
|
|
159
|
+
or unknown, and include counter-evidence, limitations, recommendation, regression
|
|
160
|
+
checks and rollback conditions. References are real observation `invocation.id`
|
|
161
|
+
values; missing or failed observations cannot be cited as successful evidence.
|
|
162
|
+
|
|
163
|
+
Submit `action: report`, the observed `revision` and `report` through
|
|
164
|
+
`control investigate`. Terminal report states are `supported`, `inconclusive` and
|
|
165
|
+
`access-blocked`; cancellation produces `cancelled`. Supported means the analyst's
|
|
166
|
+
conclusion has structurally valid evidence references. Go does not independently
|
|
167
|
+
prove semantic causality. Uncertainty produces a report with limitations, not an
|
|
168
|
+
automatic expansion of investigation or implementation authority.
|
|
169
|
+
|
|
170
|
+
When the original request authorized repair and the investigation is supported
|
|
171
|
+
with all receipts resolved, prepare Spec/packets in the same workspace and task
|
|
172
|
+
directory, then launch normally. `preparation-source` records provenance, the report
|
|
173
|
+
digest, authorized implementation scope, actions and elapsed time. Known JEV usage
|
|
174
|
+
is imported once on its original dispatch day; missing usage and cost remain
|
|
175
|
+
unknown. Resume never double-counts. Actual implementation changes are checked
|
|
176
|
+
against the original scope, including after a Quick-to-Full or model upgrade.
|
|
177
|
+
|
|
178
|
+
Preparation state, raw observations and the decision journal are private local
|
|
179
|
+
runtime records excluded from automatic Git delivery. Do not publish credentials
|
|
180
|
+
or source excerpts from logs automatically. Write a separately reviewed report when
|
|
181
|
+
publication is part of the user's goal. Once admitted, deleting or changing the
|
|
182
|
+
preparation source cannot bypass scope or accounting.
|
|
183
|
+
|
|
184
|
+
## Feedback, progress and Plan
|
|
185
|
+
|
|
186
|
+
Executor prompts recommend targeted checks first. Go still runs the complete frozen
|
|
187
|
+
acceptance against the current candidate before independent Review. This removes a
|
|
188
|
+
fixed requirement to run the full suite twice; it does not cache or remove the gate.
|
|
189
|
+
Packets can declare up to eight approved short `feedback-checks` with the same
|
|
190
|
+
`cmd`/`expect` shape as acceptance commands. They support targeted reproduction
|
|
191
|
+
without changing final acceptance. With JEV enabled, the engine can choose one of
|
|
192
|
+
these checks or recheck a failed L1 command after its prior result is confirmed.
|
|
193
|
+
Short checks offered to JEV also require a nonblank `purpose` of 1–160 characters;
|
|
194
|
+
entries without it remain available to Executor. JEV candidates send the description
|
|
195
|
+
without appending `cmd`/`script`; the description itself must omit secrets and raw logs.
|
|
196
|
+
|
|
197
|
+
Feedback carries stable check IDs, expected and actual physical outcomes,
|
|
198
|
+
candidate/check-set/environment identities, a full failed-output digest and bounded
|
|
199
|
+
head/tail excerpts. `observationComplete` describes command-outcome completeness,
|
|
200
|
+
not a complete test inventory. A repeated identical failure is stalled only for
|
|
201
|
+
comparable candidate, checks and environment. A changed candidate without a trusted
|
|
202
|
+
oracle is unknown and retains the total attempt cap. File names alone do not prove
|
|
203
|
+
stagnation, and free text cannot establish a trustworthy “10 failures to 1” metric.
|
|
204
|
+
|
|
205
|
+
A Full Plan is reusable only after a confirmed terminal result, valid reloaded
|
|
206
|
+
contracts, Git persistence and worktree synchronization. Normal repairs can retain
|
|
207
|
+
that proof. Contract, verification or scheduling changes, an unknown external HEAD,
|
|
208
|
+
legacy missing proof, explicit Plan or a verification gap require planning again.
|
|
209
|
+
Speed changes the economics of feedback; it does not erase planning or quality gates.
|
|
210
|
+
|
|
211
|
+
## JEV decisions
|
|
212
|
+
|
|
213
|
+
Use the single `jev.enabled` setting described in the [JEV guide](jev.md). Start
|
|
214
|
+
resolves configuration and freezes the policy automatically; Task, New and Run do
|
|
215
|
+
not maintain separate decision modes. The default disabled path retains the
|
|
216
|
+
original action order and collection limits. Enabled decisions may choose the
|
|
217
|
+
next action, defer redundant evidence collection, or stop collection for a report
|
|
218
|
+
when Go's evidence prerequisites are met.
|
|
219
|
+
|
|
220
|
+
Deferred actions and early completion of collection are persisted with decision
|
|
221
|
+
and observation references. They do not classify a hypothesis as proven or
|
|
222
|
+
contradicted. The analyst still writes the evidence-grounded report, and repair
|
|
223
|
+
still uses the original scope plus the ordinary verification and review flow.
|
|
224
|
+
`awaiting-report` does not restart collection when Run is repeated.
|
|
225
|
+
|
|
226
|
+
After handoff to implementation, the same switch also controls the next step after
|
|
227
|
+
a failure: continue repair, collect an approved feedback check, request replanning,
|
|
228
|
+
or select a model/effort from allowed execution bindings. These choices share the
|
|
229
|
+
task decision allowance, including calls already used during investigation. Changing
|
|
230
|
+
packet, replanning or resuming does not reset that allowance. Feedback checks collect
|
|
231
|
+
facts and never replace complete final acceptance. See the [JEV guide](jev.md) for
|
|
232
|
+
candidate and state semantics.
|
|
233
|
+
|
|
234
|
+
The effective target, model, data categories, thresholds and budget are frozen at
|
|
235
|
+
preparation time. Current OFF, incompatible target changes or tightened policy
|
|
236
|
+
stop further optional decisions. In-flight results are rechecked before adoption;
|
|
237
|
+
known/unknown receipts are reconciled without resending. A preparation initially
|
|
238
|
+
created with JEV off keeps its baseline policy; a new preparation receives a newly
|
|
239
|
+
enabled policy. The desktop labels its stored status as last observed.
|
|
240
|
+
|
|
241
|
+
The [JEV guide](jev.md#replay-and-explicit-evaluation) documents local replay and
|
|
242
|
+
explicit online evaluation. Replay stays available without JEV. Online evaluation
|
|
243
|
+
requires the same enabled master and configured target; its data/call limits cannot
|
|
244
|
+
loosen configuration. Neither tool executes actions or changes task state.
|
|
245
|
+
|
|
246
|
+
## Measurement and remaining boundaries
|
|
247
|
+
|
|
248
|
+
Alongside invocation records, `metrics.jsonl` now contains `phase-start` and
|
|
249
|
+
`phase-end`. `control metrics` reports cumulative span duration separately from the
|
|
250
|
+
union of completed intervals. Parallel spans cannot be summed as wall time. Missing
|
|
251
|
+
ends, clock regressions and unmatched legacy calls remain explicit. Provider queue,
|
|
252
|
+
model and internal tool breakdown remains unknown. Investigation actions/runs use
|
|
253
|
+
the same phase API; unobserved host-preparation time is never filled with zero.
|
|
254
|
+
|
|
255
|
+
Offline checks do not establish fewer tokens, lower cost or equal final quality.
|
|
256
|
+
JEV remains off by default; enabling the master switch activates the supported
|
|
257
|
+
decisions. Default models and concurrency are unchanged. The
|
|
258
|
+
[ready queue and bounded DSH runtime preparation](configuration.md#ready-queues-and-runtime-preparation)
|
|
259
|
+
are now available as explicit scheduler settings, with batch dispatch and no
|
|
260
|
+
prewarming remaining the defaults. Real performance and any default change still
|
|
261
|
+
need measurement. Long-lived workers and general external-patch import remain
|
|
262
|
+
separate work; the existing managed Session claim/return path keeps its original
|
|
263
|
+
recovery and acceptance requirements.
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
# JEV guide moved
|
|
2
|
+
|
|
3
|
+
[English](../en/jev-shadow.md) · [简体中文](../zh-CN/jev-shadow.md)
|
|
4
|
+
|
|
5
|
+
The current [JEV decisions guide](jev.md) describes the single `jev.enabled` switch, actual diagnostic adoption, evidence-based collection controls and evaluation tools. The old automatic shadow observer and separate network client have been retired. Historical results remain in the [original validation record](../validation/2026-11-10-jev-shadow.md).
|
package/docs/en/jev.md
ADDED
|
@@ -0,0 +1,240 @@
|
|
|
1
|
+
# JEV decisions
|
|
2
|
+
|
|
3
|
+
[Documentation](../README.md) · English · [简体中文](../zh-CN/jev.md)
|
|
4
|
+
|
|
5
|
+
Flywheel has one JEV switch: **Enable JEV** (`jev.enabled`). It is off by default.
|
|
6
|
+
Enabling it activates every supported JEV decision point automatically. There is
|
|
7
|
+
no separate shadow, advice, adoption, or per-feature switch in the task workflow.
|
|
8
|
+
|
|
9
|
+
## Enable once, use the usual task entries
|
|
10
|
+
|
|
11
|
+
In the desktop, open **Flywheel → Settings → JEV** and choose the intended global
|
|
12
|
+
or project scope. Turn on **Enable JEV** to reveal the connection fields, fill or
|
|
13
|
+
confirm the System One endpoint, fixed model version and credential
|
|
14
|
+
environment-variable name, then select **Save changes**. The switch edits a draft;
|
|
15
|
+
activation takes effect when you save. Editing and saving do not test service
|
|
16
|
+
connectivity. Tasks do not ask for the same approval again.
|
|
17
|
+
|
|
18
|
+
The process running Flywheel must receive that environment variable. Configuration
|
|
19
|
+
stores its name, never its value. Advanced fields bound timeout, call count,
|
|
20
|
+
thresholds and shared data.
|
|
21
|
+
|
|
22
|
+
`key-env` accepts 1–120 ASCII letters, digits and underscores, beginning with a
|
|
23
|
+
letter or underscore. Uppercase, lowercase and mixed-case names retain their exact
|
|
24
|
+
spelling when an investigation freezes its configuration and reloads its state.
|
|
25
|
+
|
|
26
|
+
The equivalent configuration section is:
|
|
27
|
+
|
|
28
|
+
```json
|
|
29
|
+
{
|
|
30
|
+
"jev": {
|
|
31
|
+
"enabled": true,
|
|
32
|
+
"endpoint": "https://your-service.example/v1/systemone",
|
|
33
|
+
"model": "jev-1.14.0",
|
|
34
|
+
"key-env": "MEROUTER_API_KEY",
|
|
35
|
+
"timeout-ms": 5000,
|
|
36
|
+
"max-calls": 8,
|
|
37
|
+
"min-applicability": 0.8,
|
|
38
|
+
"min-choice": 0.6,
|
|
39
|
+
"allowed-data": ["task-summary", "diagnostic-facts", "source"]
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Replace the example endpoint with your service. The model must be a fixed
|
|
45
|
+
`jev-x.y.z` version matching the response. The distributed template leaves the
|
|
46
|
+
endpoint empty and JEV disabled. Enabled configurations with an invalid target
|
|
47
|
+
fail validation. Missing credentials at execution keep the deterministic baseline.
|
|
48
|
+
Existing decision/request records preserve their actual skip or failure reasons;
|
|
49
|
+
an implementation task can fail a precondition before creating its first runtime
|
|
50
|
+
decision ledger, so not every bypass has a receipt.
|
|
51
|
+
|
|
52
|
+
Configuration selects **one complete file** in task → project → global → embedded
|
|
53
|
+
order. A project configuration that leaves JEV off overrides a globally enabled
|
|
54
|
+
configuration; fields are not merged across files. Use `config inspect --cwd
|
|
55
|
+
<project> --task-dir <task> --json` to inspect the actual source. See
|
|
56
|
+
[configuration](configuration.md) for discovery and editing.
|
|
57
|
+
|
|
58
|
+
Use the normal entries after configuration:
|
|
59
|
+
|
|
60
|
+
- **Task** prepares and runs authorized investigation or implementation from your goal. Go resolves
|
|
61
|
+
the JEV policy automatically; the host does not handwrite a decision mode.
|
|
62
|
+
- **New** prepares an investigation contract or an implementation proposal and stops before execution.
|
|
63
|
+
- **Run** starts or resumes the prepared investigation or implementation task in its original directory.
|
|
64
|
+
- Auto, Quick and Full remain implementation workflow choices. Their eligibility,
|
|
65
|
+
acceptance and review requirements continue to be enforced by Go.
|
|
66
|
+
|
|
67
|
+
## What the switch controls
|
|
68
|
+
|
|
69
|
+
| Decision point | JEV enabled | JEV disabled |
|
|
70
|
+
| --- | --- | --- |
|
|
71
|
+
| Next evidence action | Choose among approved reads, searches, directory lists, diagnostic commands and HTTP queries when the choice is uncertain | Original deterministic action order |
|
|
72
|
+
| Redundant evidence collection | May defer remaining actions for an already observed hypothesis through an evidence-bound control choice | Original action set remains |
|
|
73
|
+
| Enough evidence for a report | May end collection early once the engine's evidence prerequisites are met | Collection ends at the original action, time or output limit |
|
|
74
|
+
| Next step after a confirmed failure | Choose among legal local repair, feedback-check and replanning candidates | Existing bounded repair and escalation rules |
|
|
75
|
+
| Model / effort selection | Choose a complete approved execution binding and record the actual switch | Existing fixed binding and deterministic quality/channel fallback |
|
|
76
|
+
| Feedback-check priority | Select an approved check to collect facts before continuing bounded execution | Existing feedback and acceptance order |
|
|
77
|
+
| Explicit online evaluation | Available under the same enabled setting, target and configured limits | Refused before a request or evaluation journal is created |
|
|
78
|
+
| Recorded offline replay | Local parser/evaluation tool; no model call | Still readable; no model call |
|
|
79
|
+
|
|
80
|
+
“All supported decision points” does not mean every task makes a JEV request.
|
|
81
|
+
Single legal actions and deterministic choices execute directly. Missing data
|
|
82
|
+
permission, hard budgets without an enforceable request ceiling, exhausted calls,
|
|
83
|
+
low applicability, errors and invalid responses retain the baseline. Any decision
|
|
84
|
+
or request records already created preserve their actual reasons; preconditions
|
|
85
|
+
that fail before recording do not imply a new skip receipt.
|
|
86
|
+
Auto still uses Go contract eligibility to choose Quick or Full. Entry routing does
|
|
87
|
+
not gain a mandatory JEV classification request.
|
|
88
|
+
|
|
89
|
+
Early finish and deferred actions are collection decisions, not proof of a root
|
|
90
|
+
cause. Go records the exact decision, observation and action bindings. The analyst
|
|
91
|
+
must still submit a report covering all hypotheses with actual observation
|
|
92
|
+
references and any uncertainty. An inconclusive report remains inconclusive. A
|
|
93
|
+
repair still needs its originally authorized scope and the ordinary implementation,
|
|
94
|
+
review and verification flow. See [investigation](investigation.md).
|
|
95
|
+
|
|
96
|
+
## Next steps after failure and model selection
|
|
97
|
+
|
|
98
|
+
Before generating candidates, the engine confirms that the previous invocation has
|
|
99
|
+
ended, its result has been consumed, candidate/failure evidence can be reconciled,
|
|
100
|
+
and attempts and budget remain. JEV returns a concrete action ID. Go rechecks the
|
|
101
|
+
identity and current switch and persists selection/effect atomically. Existing
|
|
102
|
+
failure caps, one-way Quick → Full upgrades, full acceptance and independent Review
|
|
103
|
+
remain effective. JEV cannot treat an unknown executor outcome as permission to retry.
|
|
104
|
+
|
|
105
|
+
| Action | Effect | Prerequisite |
|
|
106
|
+
| --- | --- | --- |
|
|
107
|
+
| `repair` | Continue local repair within the same packet | Another execution attempt and budget remain legal |
|
|
108
|
+
| `feedback-N` | Run one short check from the contract's `feedback-checks` and supply its physical result to the next attempt | The check is explicitly approved and has a `purpose`; models cannot create or change its command |
|
|
109
|
+
| `recheck-N` | Rerun a failed L1 check with a complete prior observation to collect fresh feedback | The check belongs to frozen acceptance and its failure is confirmed |
|
|
110
|
+
| `replan` | Use the existing bounded planning / workflow-upgrade path | Planning is legal, with original scope, revision limits and history retained |
|
|
111
|
+
| `model-N` | Select a specific allowed execution configuration | `execution-policy` is configured and the complete binding and switch limits are valid |
|
|
112
|
+
|
|
113
|
+
`N` indexes the frozen candidate set; it is not a model name or arbitrary command.
|
|
114
|
+
A model candidate resolves to its actual binding ID, harness, Provider, account channel,
|
|
115
|
+
model, effort, permission and session policy. Missing approved candidates do not
|
|
116
|
+
authorize guessing a model name or adding a paid channel. Model/effort changes
|
|
117
|
+
retain the existing cold-start rules and never pass Executor history to Reviewer.
|
|
118
|
+
AUTH/QUOTA and other channel failures keep their corresponding fallback rules;
|
|
119
|
+
they are not evidence of insufficient reasoning ability. See
|
|
120
|
+
[configuration](configuration.md) for the allowed binding set.
|
|
121
|
+
|
|
122
|
+
A short check's `purpose` is a nonblank description of 1–160 characters. The JEV
|
|
123
|
+
candidate sends this description without appending the `cmd`/`script` text. Keep
|
|
124
|
+
secrets, raw logs and commands out of the description itself.
|
|
125
|
+
Entries without `purpose` remain available to Executor but do not become targeted
|
|
126
|
+
JEV candidates. This field belongs only to `feedback-checks`; it does not extend
|
|
127
|
+
acceptance fields or task authority.
|
|
128
|
+
|
|
129
|
+
A feedback check supplies evidence for the next action. Even if it passes, the
|
|
130
|
+
engine still runs every required acceptance check against the final candidate;
|
|
131
|
+
the choice cannot remove checks. Task and node views show the request, adoption or
|
|
132
|
+
baseline reason, feedback check and recorded effect. `applied` means the control
|
|
133
|
+
decision was applied; subsequent repair, planning and model sessions remain in
|
|
134
|
+
the node records. `completed` means this control action settled, not that the whole
|
|
135
|
+
task was delivered. `feedback-completed` likewise means only that feedback settled.
|
|
136
|
+
`pending` requires request reconciliation. `feedback-running` and `feedback-unknown`
|
|
137
|
+
require the original action to be reconciled and must not be replayed automatically.
|
|
138
|
+
`feedback-stale` blocks execution because feedback changed candidate or verification
|
|
139
|
+
inputs. `replan-running` means replanning has not settled; recovery must reconcile
|
|
140
|
+
the original Plan session rather than dispatching again to guess its outcome.
|
|
141
|
+
Applied model changes show the actual before/after bindings, attempt,
|
|
142
|
+
reason and time. The UI never guesses a model from `model-N` or treats a configuration
|
|
143
|
+
change as proof that a new session ran successfully.
|
|
144
|
+
|
|
145
|
+
## Data, budgets and recovery
|
|
146
|
+
|
|
147
|
+
For an investigation, Go freezes the effective JEV target, data categories,
|
|
148
|
+
thresholds and call budget when preparation is created. Task-provided historical `decision` modes cannot turn
|
|
149
|
+
JEV on while the effective master setting is off. The default enabled data set
|
|
150
|
+
allows the task summary, diagnostic hypotheses, structured command/HTTP outcomes,
|
|
151
|
+
and bounded excerpts of approved file evidence. `allowed-data` can narrow this set.
|
|
152
|
+
Raw command/HTTP output is included only with explicit `logs` permission; otherwise
|
|
153
|
+
the snapshot records that missing content and uses structured facts. Permission is
|
|
154
|
+
not automatic redaction: any remaining unauthorized category skips the complete
|
|
155
|
+
request, and summarization cannot bypass the original data policy.
|
|
156
|
+
|
|
157
|
+
Disabling the setting stops subsequent JEV dispatches and adoption in existing
|
|
158
|
+
runs, including a result returned after the switch was disabled. Already dispatched
|
|
159
|
+
requests are reconciled using their frozen identity, and their known or unknown
|
|
160
|
+
usage is retained. Unknown requests are not retried. Unknown **action** outcomes
|
|
161
|
+
still block execution until reconciled.
|
|
162
|
+
|
|
163
|
+
An investigation preparation created while JEV was off stays on its baseline;
|
|
164
|
+
enabling JEV does not rewrite that preparation. A newly created preparation can
|
|
165
|
+
freeze the enabled policy.
|
|
166
|
+
|
|
167
|
+
Implementation freezes its own runtime decision policy at the first eligible
|
|
168
|
+
failure in an enabled run. If the current process started with JEV off and has no
|
|
169
|
+
runtime decision ledger, saving an enabled setting does not activate that process
|
|
170
|
+
immediately: start or resume the task so it reads the enabled configuration. Once a
|
|
171
|
+
runtime decision ledger exists, turning JEV off and back on retains its frozen
|
|
172
|
+
target, records and remaining allowance, including imported investigation calls.
|
|
173
|
+
Changes that invalidate a frozen target or tighten its permissions/limits disable
|
|
174
|
+
further optional decisions rather than silently rewriting the policy. Check the
|
|
175
|
+
reported JEV status and reason; do not copy ledgers or reset budgets to bypass it.
|
|
176
|
+
|
|
177
|
+
`max-calls` is a task allowance that includes unknown dispatched requests. Calls
|
|
178
|
+
used during investigation are imported into implementation's allowance. Multiple
|
|
179
|
+
packets, model switches, replanning and resume cannot reset it. This is a call cap,
|
|
180
|
+
not a guaranteed token or currency ceiling. A configured hard budget skips JEV when
|
|
181
|
+
the service cannot enforce a provable per-request bound. Requests record actual
|
|
182
|
+
usage when available; missing usage remains unknown.
|
|
183
|
+
|
|
184
|
+
Implementation's `runtime-decisions` record lives in `state.json` and retains the
|
|
185
|
+
frozen policy, used/imported call counts, requests, selections and effects. It
|
|
186
|
+
follows the existing state commit and delivery rules. The request journal under
|
|
187
|
+
`.investigation/runtime-decisions` is private and excluded from automatic delivery;
|
|
188
|
+
that exclusion does not apply to the copy of decision information in `state.json`.
|
|
189
|
+
|
|
190
|
+
The desktop displays bounded summaries and omits raw snapshots and target
|
|
191
|
+
configuration from its projection. This display projection does not redact Git
|
|
192
|
+
delivery. As with existing failure feedback, approved goals, facts, bindings and
|
|
193
|
+
feedback output must follow the existing rule against recording credentials.
|
|
194
|
+
Pending requests/actions are reconciled in their original task, without a new
|
|
195
|
+
journal or model switch bypassing an unknown outcome.
|
|
196
|
+
|
|
197
|
+
## Replay and explicit evaluation
|
|
198
|
+
|
|
199
|
+
Normal tasks save frozen `*.input.json` documents and receipts in their private
|
|
200
|
+
decision journal. Offline replay also accepts historical `*.snapshot.json` captures:
|
|
201
|
+
|
|
202
|
+
```bash
|
|
203
|
+
flywheel decision replay --input frozen.input.json --response native-response.json \
|
|
204
|
+
--endpoint "$JEV_ENDPOINT" --model jev-1.14.0 --expected-model jev-1.14.0
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Replay reads no credential, makes no HTTP request and writes no journal or task
|
|
208
|
+
state. A bare Snapshot requires `--policy reviewed-policy.json`; an explicit policy
|
|
209
|
+
replaces the complete captured policy rather than inheriting missing fields.
|
|
210
|
+
|
|
211
|
+
For a deliberate online evaluation, enable JEV in the relevant scope first:
|
|
212
|
+
|
|
213
|
+
```bash
|
|
214
|
+
flywheel decision evaluate --cwd "$repo_dir" --input frozen.input.json \
|
|
215
|
+
--directory ./decision-evaluation --max-calls 5
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
The endpoint, fixed model and credential variable come from the same configuration.
|
|
219
|
+
Legacy explicit target flags must match it. Evaluation forces advisory behavior;
|
|
220
|
+
it never dispatches an action or changes task state. Its call cap, timeout,
|
|
221
|
+
thresholds and data categories cannot loosen the configured bounds. Reusing an
|
|
222
|
+
input and journal reads the original receipt; it does not resend an unknown call.
|
|
223
|
+
|
|
224
|
+
Both tools return `adopted: false`. Live failures and unknown outcomes return exit
|
|
225
|
+
code 3, input/configuration errors return 1, and cancellation returns 143. Offline
|
|
226
|
+
fixture success is not evidence of live model accuracy or lower task latency.
|
|
227
|
+
|
|
228
|
+
## Migration from shadow evaluation
|
|
229
|
+
|
|
230
|
+
The old automatic observer and separate shadow network client have been removed.
|
|
231
|
+
Old `jev-shadow` configuration remains readable for migration: connection hints are
|
|
232
|
+
retained, but the old enabled flag never activates online decisions. An explicit
|
|
233
|
+
new `jev` section takes precedence. Saving through the editor writes the unified
|
|
234
|
+
section and removes the legacy key while preserving the normal backup/revision
|
|
235
|
+
checks. The retired `shadow` command reports the migration path without running
|
|
236
|
+
an experiment. Historical records and their schema remain available as evidence.
|
|
237
|
+
|
|
238
|
+
To assess value, compare complete tasks with JEV off and on: time to useful evidence,
|
|
239
|
+
actions performed/deferred, total latency, actual/unknown usage, and final quality.
|
|
240
|
+
A different action order or a passing fixture alone does not establish a speedup.
|