@awak-app/simy-cli 0.7.15 → 0.7.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +225 -98
- package/package.json +5 -3
- package/src/agent.js +129 -7
- package/src/backend-executable.js +7 -6
- package/src/bounded-local-task-subtask-pool.js +32 -6
- package/src/bounded-local-task-subtasks-contract.js +1 -1
- package/src/cli-contract.js +1 -1
- package/src/codex-plugins/simy/.codex-plugin/plugin.json +2 -2
- package/src/codex-plugins/simy/hooks/README.md +8 -3
- package/src/codex-plugins/simy/hooks/delivery-loop.mjs +302 -78
- package/src/codex-plugins/simy/hooks/hooks.json +1 -1
- package/src/codex-plugins/simy/hooks/loop-kernel/conformance.json +136 -0
- package/src/codex-plugins/simy/hooks/loop-kernel/index.mjs +42 -0
- package/src/codex-plugins/simy/hooks/loop-kernel/outcome-review.mjs +31 -0
- package/src/codex-plugins/simy/hooks/loop-kernel/pin.json +6 -0
- package/src/codex-plugins/simy/hooks/loop-kernel/policy.json +141 -0
- package/src/codex-plugins/simy/hooks/loop-kernel/task-pool.mjs +34 -0
- package/src/codex-plugins/simy/hooks/loop-kernel/work-frontier.mjs +132 -0
- package/src/codex-plugins/simy/hooks/loop-kernel.mjs +6 -0
- package/src/codex-plugins/simy/hooks/outcome-progress.mjs +16 -5
- package/src/codex-plugins/simy/hooks/purpose-completion-contract.mjs +10 -0
- package/src/codex-plugins/simy/hooks/purpose-cycle.mjs +12 -11
- package/src/codex-plugins/simy/hooks/purpose-evidence.mjs +28 -38
- package/src/codex-plugins/simy/hooks/purpose-rule-policy.mjs +1 -2
- package/src/codex-plugins/simy/hooks/purpose-rule-review.mjs +5 -63
- package/src/codex-plugins/simy/hooks/purpose-search.mjs +21 -4
- package/src/codex-plugins/simy/hooks/result-cycle.mjs +166 -0
- package/src/codex-plugins/simy/hooks/scene-reasoning.mjs +98 -12
- package/src/codex-plugins/simy/hooks/user-experience.mjs +26 -0
- package/src/codex-plugins/simy/hooks/work-frontier.mjs +2 -0
- package/src/codex-plugins/simy/skills/diagnose/SKILL.md +10 -4
- package/src/coding-run-recovery.js +4 -2
- package/src/delivery-hooks/claude-workflow.md +10 -7
- package/src/delivery-hooks/command.js +76 -10
- package/src/delivery-hooks/consistency-monitor.js +20 -4
- package/src/delivery-hooks/consult.js +60 -0
- package/src/delivery-hooks/diagnosis.js +2 -2
- package/src/delivery-hooks/legacy-hook-routing.js +46 -8
- package/src/delivery-hooks/local-runtime.js +81 -1
- package/src/delivery-hooks/local-supervisor.js +13 -4
- package/src/delivery-hooks/repair-continuation.js +80 -0
- package/src/delivery-hooks/repair.js +164 -0
- package/src/delivery-hooks/supervisor-handover.js +84 -0
- package/src/delivery-hooks/supervisor-owner.js +3 -2
- package/src/delivery-hooks/supervisor-service.js +38 -6
- package/src/delivery-hooks/supervisor-watchdog.js +15 -4
- package/src/delivery-hooks/supervisor.js +151 -18
- package/src/desktop-access.js +2 -1
- package/src/desktop-host.js +35 -2
- package/src/durable-local-task-worker.js +372 -58
- package/src/execution-capability-contract.js +15 -1
- package/src/index.js +3 -3
- package/src/local-photos-read.swift +33 -0
- package/src/local-photos.js +47 -0
- package/src/local-task-attachment-store.js +31 -0
- package/src/local-task-destination.js +174 -0
- package/src/local-task.js +119 -7
- package/src/native-delegation-contract.js +68 -0
- package/src/native-provider-identity.js +60 -0
- package/src/orchestrator/audit.js +24 -323
- package/src/orchestrator/budget.js +22 -11
- package/src/orchestrator/charter-validation.js +3 -41
- package/src/orchestrator/checklist-continuation.js +2 -14
- package/src/orchestrator/completion-contract.js +4 -49
- package/src/orchestrator/contract.js +5 -9
- package/src/orchestrator/delivery/WORKFLOW.md +30 -710
- package/src/orchestrator/delivery/references/acceptance-evidence.md +4 -216
- package/src/orchestrator/delivery/references/change-boundary-matrix.md +4 -103
- package/src/orchestrator/delivery/references/chrome-user-verification.md +5 -133
- package/src/orchestrator/delivery/references/cross-repository-control.md +4 -74
- package/src/orchestrator/delivery/references/delivery-levels.md +3 -117
- package/src/orchestrator/delivery/references/local-sqm-evidence.md +1 -1
- package/src/orchestrator/delivery/references/minimum-completion.md +4 -144
- package/src/orchestrator/delivery/references/pr-closure.md +4 -200
- package/src/orchestrator/delivery/references/shared-system-specifications.md +4 -161
- package/src/orchestrator/delivery/references/state-and-receipts.md +4 -245
- package/src/orchestrator/delivery/references/state-model.md +4 -185
- package/src/orchestrator/delivery/references/ux-specification.md +4 -95
- package/src/orchestrator/delivery-level.js +2 -10
- package/src/orchestrator/evidence.js +2 -3
- package/src/orchestrator/independent-audit.js +15 -164
- package/src/orchestrator/instruction.js +19 -254
- package/src/orchestrator/loop.js +35 -95
- package/src/orchestrator/outcome-monitor.js +21 -0
- package/src/orchestrator/pr-closure.js +2 -16
- package/src/orchestrator/recovery.js +32 -2
- package/src/orchestrator/ux-specification.js +3 -12
- package/src/provider-delegation.js +336 -0
- package/src/runner.js +5 -2
- package/src/session-supervision/acceptance-reads.js +34 -0
- package/src/session-supervision/artifact-observer.js +138 -0
- package/src/session-supervision/browser-cdp.js +59 -0
- package/src/session-supervision/browser-recovery.js +97 -0
- package/src/session-supervision/codex-app-server.js +61 -22
- package/src/session-supervision/completion-artifacts.js +44 -0
- package/src/session-supervision/consistency-auditor.js +8 -11
- package/src/session-supervision/continuation-analysis.js +47 -28
- package/src/session-supervision/final-response-recovery.js +5 -0
- package/src/session-supervision/index.js +22 -5
- package/src/session-supervision/interaction-proxy.js +250 -0
- package/src/session-supervision/interaction-task.js +189 -0
- package/src/session-supervision/native-interaction.js +102 -0
- package/src/session-supervision/provider-failure.js +40 -0
- package/src/session-supervision/purpose-instruction.js +14 -14
- package/src/session-supervision/purpose-recovery.js +32 -12
- package/src/session-supervision/purpose-verification.js +50 -216
- package/src/session-supervision/recovery-frontier.js +55 -0
- package/src/session-supervision/stop-notifications.js +8 -8
- package/src/sqm/agent-skill.js +1 -1
- package/src/sqm/checkers.js +12 -6
- package/src/sqm/claude-plugin-update.js +22 -8
- package/src/sqm/claude-plugin.js +6 -0
- package/src/sqm/local-tests.js +2 -1
- package/src/sqm/plugin-cache.js +67 -0
- package/src/sqm/proof.js +25 -0
- package/src/sqm/report.js +1 -0
- package/src/sqm/workflow-onboarding.js +3 -39
- package/src/sqm/workflow-update.js +17 -6
- package/src/startup-preflight.js +2 -20
- package/src/team-status-fence-store.js +32 -0
- package/src/team-threads.js +457 -0
|
@@ -1,218 +1,6 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Checking the requested result
|
|
2
|
+
Use the original request and later corrections to determine what must work. Exercise the actual product or inspect the actual output. Verify relevant error and recovery paths and repair failures. A process finishing or a test passing does not establish an unrelated user outcome.
|
|
2
3
|
|
|
3
|
-
|
|
4
|
+
Reuse direct observations and ordinary test output. No acceptance table, source-ID ledger, screenshot package, citation syntax or review receipt is required. Preserve useful failure details for repair, without creating documents solely to authorize completion.
|
|
4
5
|
|
|
5
|
-
|
|
6
|
-
through every phase. Preserve the complete source-linked interaction contract in
|
|
7
|
-
the canonical specification; the compact checklist is its status view, not a
|
|
8
|
-
replacement for the required behavior, branches, postconditions and recovery detail.
|
|
9
|
-
|
|
10
|
-
Freeze the user experience before proposing the solution. Read-only discovery may
|
|
11
|
-
inform it, but the first authored success record states the beneficiary and
|
|
12
|
-
context, their goal, the complete usable outcome, what they should see,
|
|
13
|
-
understand, and be able to do next at each observable moment, applicable recovery
|
|
14
|
-
and forbidden experiences, and how those moments will be observed. Assign stable
|
|
15
|
-
UX/acceptance IDs. A screen, component, endpoint, task, or stored row is an
|
|
16
|
-
implementation detail unless it is tied to this experience.
|
|
17
|
-
|
|
18
|
-
The controller externalizes every acceptance condition with a stable ID, observable
|
|
19
|
-
outcome, source independent of the candidate, and required checks. For each
|
|
20
|
-
verification gate, freeze its applicable criteria in `charter.json` and record the
|
|
21
|
-
SHA-256 of the exact file bytes in the controller-owned snapshot. Reconcile the
|
|
22
|
-
gate charters against the overall delivery charter so no requirement disappears.
|
|
23
|
-
Do not choose a convenient subset after observing failures.
|
|
24
|
-
|
|
25
|
-
Use these criteria as the structured form of the experience contract rather than
|
|
26
|
-
creating a second UX checklist. Cover the full path from its real trigger to its
|
|
27
|
-
usable outcome and the applicable recovery path. When several parts must work
|
|
28
|
-
together for the beneficiary to succeed, include an end-to-end criterion in
|
|
29
|
-
addition to narrower diagnostic criteria. Independent coverage review compares
|
|
30
|
-
the experience contract and original request to the frozen rows before coding.
|
|
31
|
-
|
|
32
|
-
For production-relevant experience IDs, extend the same charter with outcome and
|
|
33
|
-
guardrail measures. Each measure names the signal, independent source,
|
|
34
|
-
observation window, threshold, resulting decision and owner. A rollout policy
|
|
35
|
-
references those measure IDs for promotion and halt decisions. These fields are
|
|
36
|
-
part of the same acceptance source and readable results view, not a second
|
|
37
|
-
success checklist.
|
|
38
|
-
|
|
39
|
-
Each criterion has two layers:
|
|
40
|
-
|
|
41
|
-
- `user`: a goal-driven user journey with visible interactions and evidence of the
|
|
42
|
-
actual outcome. For CLI/document work, use the actual command or consumer.
|
|
43
|
-
- `system`: independent deterministic checks of the affected contract, API, stored
|
|
44
|
-
values, identity/tenant/authorization, concurrency, or artifacts.
|
|
45
|
-
|
|
46
|
-
For save operations, both must observe the same run, fixture and input. The user
|
|
47
|
-
saves, reloads and returns in a new session; the technical check queries persisted
|
|
48
|
-
values and ownership, tests unauthorized access and duplicate/retry behavior where
|
|
49
|
-
relevant, and correlates request IDs/logs. An HTTP 500, other unexpected HTTP error,
|
|
50
|
-
network failure, or a persistence mismatch fails the affected check. A 200 response
|
|
51
|
-
or success toast alone is not a persistence oracle.
|
|
52
|
-
|
|
53
|
-
Each layer is required unless genuinely inapplicable. Put a reason and authoritative
|
|
54
|
-
source in the frozen charter for an exception. Tool unavailability, failed tests,
|
|
55
|
-
lack of time, or missing authority mean pending/blocked, not inapplicable. At least
|
|
56
|
-
one layer must remain required. User-facing state changes normally require both.
|
|
57
|
-
|
|
58
|
-
## Update throughout the loop
|
|
59
|
-
|
|
60
|
-
Keep `matrix.json` for current results and `checklist.md` for the readable table.
|
|
61
|
-
After every evaluated action (test, repair, review or changed inputs):
|
|
62
|
-
|
|
63
|
-
1. Append an immutable receipt and retain the previous matrix/table snapshot.
|
|
64
|
-
2. Update current results with action, observation, timestamp, evidence paths and
|
|
65
|
-
hashes, status, unmet reason, and next action. Use `pending`, `failed`, `passed`;
|
|
66
|
-
render `not_applicable` only from the frozen charter.
|
|
67
|
-
3. Bind results to the current context digest. When code, base/companion revision,
|
|
68
|
-
environment, fixture, inputs, or acceptance source changes, invalidate affected
|
|
69
|
-
receipts under the state contract; display stale checks as pending. Do not copy
|
|
70
|
-
a new digest onto an old observation to make it current.
|
|
71
|
-
4. Run the validator and replace the readable table with its `markdown` output
|
|
72
|
-
(or `--markdown` stdout). Include the snapshot's revision and current gate in
|
|
73
|
-
the accompanying receipt. The table is a view of JSON, not a second authority.
|
|
74
|
-
5. Inspect raw evidence and remaining rows before choosing the next bounded action.
|
|
75
|
-
6. Perform that action when safe and authorized; do not stop after displaying the
|
|
76
|
-
table. Repair failures or missing behavior, run untested paths, collect missing
|
|
77
|
-
evidence, rerun stale checks, then validate and regenerate the table. Repeat
|
|
78
|
-
until all applicable requested outcomes and authorized endpoint gates pass.
|
|
79
|
-
A proposed next step is not a substitute for feasible execution.
|
|
80
|
-
|
|
81
|
-
Yellow and red tables are progress, not successful delivery. For in-progress
|
|
82
|
-
dependencies use available bounded waits/rechecks or approved monitoring without
|
|
83
|
-
busy-polling or duplicate jobs. Preserve authorization and progress checks; local time/attempt/token counters are informational and keep the
|
|
84
|
-
requester informed. If progress genuinely requires new authority, access, a
|
|
85
|
-
decision, dependency availability or a triggered no-progress/failure circuit, retain affected rows and record
|
|
86
|
-
observed blocker, safe alternatives tried, exact missing input, owner and resume
|
|
87
|
-
trigger. Continue independent work; end only with an explicitly unfinished
|
|
88
|
-
disposition when no further safe in-scope action remains or a safety/user stop
|
|
89
|
-
applies. Do not require another “continue” for already authorized feasible work.
|
|
90
|
-
An issue or remediation PR does not satisfy the row it tracks. Do not weaken
|
|
91
|
-
criteria, hide rows, invent evidence or change the endpoint to
|
|
92
|
-
force green. Local execution counters do not impose a completion limit. Reconcile the final full table against the original request and all
|
|
93
|
-
current endpoint gates before reporting verified_complete.
|
|
94
|
-
|
|
95
|
-
All rows, including pending/failed ones, stay visible. A green matrix cannot advance
|
|
96
|
-
an unrelated gate or prove merge/deployment. Validate again for the final environment.
|
|
97
|
-
|
|
98
|
-
The readable Markdown begins with the default compact `依頼チェックリスト`:
|
|
99
|
-
|
|
100
|
-
| # | ご依頼 | 現状 | 判定 |
|
|
101
|
-
| ---: | --- | --- | :---: |
|
|
102
|
-
| 1 | Frozen criterion outcome | Latest combined observation, or unmet reason and next repair | ✅ / 🟡 / ❌ |
|
|
103
|
-
|
|
104
|
-
Render one row per frozen criterion in charter order. Aggregate its layer results
|
|
105
|
-
deterministically: all applicable layers passed is `✅`; any applicable failed
|
|
106
|
-
layer is `❌`; otherwise it is `🟡`. A non-applicable layer remains visible with
|
|
107
|
-
its authoritative reason in the detailed table; every valid criterion keeps at
|
|
108
|
-
least one applicable layer. A mixture of passed and pending is yellow, not green.
|
|
109
|
-
Show green/yellow/red counts, the highest proven state, and validator status
|
|
110
|
-
immediately below the compact table. In a terminal receipt,
|
|
111
|
-
the controller also prints its disposition there.
|
|
112
|
-
|
|
113
|
-
The compact table is maintained in the evidence record after the charter is
|
|
114
|
-
frozen, after every evaluated attempt, and in the final evidence receipt.
|
|
115
|
-
`checklist.md` also retains the detailed layer table so evidence, methods, and
|
|
116
|
-
repairs remain reviewable. User-facing reporting follows the common
|
|
117
|
-
[outcome reporting contract](../../../workflow-reporting.md). Show the table when
|
|
118
|
-
requested or when it best answers the request, not after every internal step.
|
|
119
|
-
An A/B-only or no-test-detail instruction takes precedence over the default table.
|
|
120
|
-
Keep every non-green row internally and disclose any material effect on the
|
|
121
|
-
requested answers or completion claim; brevity cannot hide incomplete delivery. Do not turn `pr_ci_green`, a healthy service,
|
|
122
|
-
or a passing narrow check into a green request row unless every applicable layer
|
|
123
|
-
for that criterion passed against the current charter and context.
|
|
124
|
-
|
|
125
|
-
During staged release, append the stage, exposure, observed window, measure
|
|
126
|
-
results and deterministic `promote`, `hold`, `abort` or `rollback` decision to
|
|
127
|
-
the current results and receipt. Planned thresholds are not observations. A
|
|
128
|
-
healthy deployment cannot replace a missed outcome or breached guardrail.
|
|
129
|
-
|
|
130
|
-
After the exact candidate is built in a production-like local environment, run
|
|
131
|
-
the complete experience and regenerate `checklist.md` for that same revision,
|
|
132
|
-
environment, fixture, and input. Read the user and system rows together as a
|
|
133
|
-
single experience table: expected outcome, actual user observation, independent
|
|
134
|
-
postcondition/integration result, evidence, status, and next repair. Missing,
|
|
135
|
-
pending, failed, stale, or unverifiable applicable rows prevent
|
|
136
|
-
`user_like_local_verified`. Individually working components cannot override a
|
|
137
|
-
failed or absent end-to-end row.
|
|
138
|
-
|
|
139
|
-
## File contract
|
|
140
|
-
|
|
141
|
-
Charter example (illustrative values must be replaced with actual requirements):
|
|
142
|
-
|
|
143
|
-
```json
|
|
144
|
-
{
|
|
145
|
-
"schema_version": 1,
|
|
146
|
-
"run_id": "save-run-42",
|
|
147
|
-
"criteria": [{
|
|
148
|
-
"id": "SAVE-1",
|
|
149
|
-
"outcome": "An owner returning to the task sees the saved note, understands it was retained, and can continue editing",
|
|
150
|
-
"acceptance_source": "Approved issue acceptance criteria, revision 3",
|
|
151
|
-
"layers": {
|
|
152
|
-
"user": {"applicability": "required", "method": "Save in Chrome, reload, sign in again and compare the note"},
|
|
153
|
-
"system": {"applicability": "required", "method": "Read stored note and owner; check unauthorized read and duplicate-save behavior"}
|
|
154
|
-
}
|
|
155
|
-
}]
|
|
156
|
-
}
|
|
157
|
-
```
|
|
158
|
-
|
|
159
|
-
The matrix has `schema_version: 1`, `charter_sha256`, `context`, and `results`.
|
|
160
|
-
Context requires `run_id`, `stage`, `environment`, `fixture_id`, `input_sha256`,
|
|
161
|
-
and `source_revisions` mapping repository identifiers to full 40-character SHAs.
|
|
162
|
-
Include base/companion revisions, image digests, deployed revision, profile and
|
|
163
|
-
other relevant identities as additional context fields. `contextDigest(context)`
|
|
164
|
-
from the packaged script hashes recursively key-sorted JSON. Every context field
|
|
165
|
-
participates. Compute and retain it from the independently observed controller
|
|
166
|
-
snapshot, not from an untrusted result's requested context.
|
|
167
|
-
|
|
168
|
-
Each result is `{ "id": "SAVE-1", "layers": { "user": cell, "system": cell } }`.
|
|
169
|
-
A passed cell requires:
|
|
170
|
-
|
|
171
|
-
```json
|
|
172
|
-
{
|
|
173
|
-
"status": "passed",
|
|
174
|
-
"charter_sha256": "64 lowercase hex characters of the frozen charter bytes",
|
|
175
|
-
"context_sha256": "64 lowercase hex characters from the controller snapshot",
|
|
176
|
-
"action": "Actual operation or command executed",
|
|
177
|
-
"observation": "Actual observed result, including values or a comparison verdict",
|
|
178
|
-
"observed_at": "2026-09-13T10:00:00Z",
|
|
179
|
-
"evidence": [{"path": "evidence/save-check.json", "sha256": "64 lowercase hex characters of the file bytes"}]
|
|
180
|
-
}
|
|
181
|
-
```
|
|
182
|
-
|
|
183
|
-
Failed/pending cells keep the same charter/context digests, action/observation where available,
|
|
184
|
-
`reason`, and `next_action`. Missing cells render pending. Results cannot waive a
|
|
185
|
-
required layer. To change applicability, update the authoritative charter through
|
|
186
|
-
the existing charter-patch process and invalidate dependent results.
|
|
187
|
-
Every passed cell is bound to both digests. An approved charter patch cannot reuse
|
|
188
|
-
old cells simply by updating the matrix header; record new verification first.
|
|
189
|
-
|
|
190
|
-
## Run and interpret
|
|
191
|
-
|
|
192
|
-
From the packaged Web/CLI delivery directory, invoke the bundled script with Node
|
|
193
|
-
20 or later. Paths are explicit; never assume another machine's home directory:
|
|
194
|
-
|
|
195
|
-
```text
|
|
196
|
-
node <delivery-directory>/scripts/check-acceptance.mjs charter.json matrix.json <trusted-charter-sha256> <current-context-sha256>
|
|
197
|
-
```
|
|
198
|
-
|
|
199
|
-
Exit 0 means `evidence_complete`, exit 1 means `evidence_incomplete`, and exit 2
|
|
200
|
-
means invalid inputs. JSON output includes all checklist rows, errors and a Markdown
|
|
201
|
-
table; append `--markdown` for the table alone. A pending run may exit 1 normally;
|
|
202
|
-
use its rows to decide what to verify next. Invalid input cannot establish a gate.
|
|
203
|
-
|
|
204
|
-
Evidence files must be explicit regular files under the matrix directory, not
|
|
205
|
-
URLs, symlinks or parent paths. The checker reads and hashes only those referenced
|
|
206
|
-
files, sends nothing, executes nothing, and prints no file contents. JSON inputs
|
|
207
|
-
are limited to 1 MiB, evidence files to 16 MiB each. Store larger recordings in the
|
|
208
|
-
normal evidence store and use a bounded verified receipt here; the verifier must
|
|
209
|
-
check that receipt's target. Keep the bundle private and stable during validation.
|
|
210
|
-
|
|
211
|
-
This checks completeness, current-context binding and file integrity, not whether
|
|
212
|
-
a collector told the truth, a criterion is adequate, or a system is deployed.
|
|
213
|
-
Hashes are not signatures. The controller must protect the frozen digest, verify
|
|
214
|
-
requirement coverage and genuine non-applicability, and inspect raw evidence.
|
|
215
|
-
Independent review checks the oracle and findings; the validator is an additional
|
|
216
|
-
gate within this workflow, not a globally installed hook. The CLI now additionally
|
|
217
|
-
uses [minimum completion](minimum-completion.md) to enforce current functional
|
|
218
|
-
evidence at its promotion boundary; this basic validator alone is insufficient.
|
|
6
|
+
If a check cannot run, keep the affected behavior unverified and investigate the obstacle. Do not label it passed or silently drop the requested outcome. Report material remaining limits in the user's requested format.
|
|
@@ -1,105 +1,6 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Choosing checks for a change
|
|
2
|
+
Inspect affected behavior and shared dependencies. Start with a direct regression, then run the relevant repository checks and real execution path. Use the actual failure to decide what to fix. Reuse unaffected results rather than repeating every test.
|
|
2
3
|
|
|
3
|
-
|
|
4
|
-
prove the user outcome.
|
|
4
|
+
For persistence, check reload and retry behavior. For permissions, check allowed and denied access. For external operations, check duplicate prevention and unknown-result recovery. For shared APIs, check both producer and consumer. For UI changes, check the visible flow, clarity, keyboard access and error recovery where relevant.
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
- Preserve a failing reproduction or explain why it cannot be captured.
|
|
9
|
-
- Bind acceptance to a human-confirmed requirement, external contract, prior
|
|
10
|
-
regression, or another source independent of the candidate implementation.
|
|
11
|
-
- Run focused regressions, formatting, lint, type checks, unit tests, and the
|
|
12
|
-
real affected build/runtime in that order when available.
|
|
13
|
-
- Inspect the final diff for scope, secrets, generated files, and unrelated
|
|
14
|
-
churn.
|
|
15
|
-
- Use current source and dependency revisions in every evidence receipt.
|
|
16
|
-
|
|
17
|
-
## Architecture, rules, and documentation
|
|
18
|
-
|
|
19
|
-
- Read instruction files as routing maps and follow only the relevant links to
|
|
20
|
-
architecture, product, security, reliability, and intentional-debt sources.
|
|
21
|
-
- Verify dependency direction and other deterministic critical invariants with
|
|
22
|
-
existing structural tests or static analysis. Add or change enforcement only
|
|
23
|
-
after the five-response comparison in the main workflow.
|
|
24
|
-
- Distinguish merge-blocking invariants from advisory debt. Keep blocking gates
|
|
25
|
-
minimal and record non-blocking findings with evidence and an owner or explicit
|
|
26
|
-
follow-up condition.
|
|
27
|
-
- Check that changed behavior and source-of-truth documentation agree, that
|
|
28
|
-
required links resolve, and that superseded guidance is removed or marked.
|
|
29
|
-
- Flag deleted tests, weakened assertions, expected-value changes, disabled
|
|
30
|
-
checks, and CI or security-policy relaxation for independent review. Re-run
|
|
31
|
-
the affected gate against the newly approved boundary.
|
|
32
|
-
- Prefer small consolidation, deletion, or repair changes over broad cleanup.
|
|
33
|
-
Bound maintenance work by scope, change size, and an observable stop condition.
|
|
34
|
-
|
|
35
|
-
## UI and user-state changes
|
|
36
|
-
|
|
37
|
-
- Render the actual UI in a browser.
|
|
38
|
-
- Exercise refresh during and after completion, back/forward navigation,
|
|
39
|
-
another session and return, delayed stream/poll responses, and stale-cache or
|
|
40
|
-
replica responses where relevant.
|
|
41
|
-
- Verify follow-up and parent/child lineage, optimistic state reconciliation,
|
|
42
|
-
authorization, and tenant separation.
|
|
43
|
-
- Capture and inspect screenshots; capture video for streaming, recovery,
|
|
44
|
-
animation, or timing-dependent behavior.
|
|
45
|
-
|
|
46
|
-
## Authentication and session recovery (when touched)
|
|
47
|
-
|
|
48
|
-
- Derive the error/state matrix from the current provider contract and application
|
|
49
|
-
policy. Distinguish a temporarily unavailable or unknown response from a
|
|
50
|
-
confirmed invalid, deleted or banned identity. Verify the expected exit/recovery
|
|
51
|
-
path as well as the message; a permanent failure must not trap the user in retry.
|
|
52
|
-
- Include current membership suspension/removal, account/session switching and
|
|
53
|
-
stale in-flight results where affected. Preserve unknown-error handling unless
|
|
54
|
-
an authoritative contract changes it; do not turn all unknown errors into logout.
|
|
55
|
-
- For refresh/cookie changes, inventory affected callers, propagation and concurrent
|
|
56
|
-
refresh behavior. Record untested callers and load concerns separately from
|
|
57
|
-
confirmed defects; a review follow-up is not automatically a release blocker.
|
|
58
|
-
|
|
59
|
-
## Generated artifacts
|
|
60
|
-
|
|
61
|
-
- Download through the user-facing path.
|
|
62
|
-
- Verify non-empty bytes, expected type, stable hash, and ownership/tenant.
|
|
63
|
-
- Open with the real consumer. Render every page/slide/sheet when layout
|
|
64
|
-
matters and inspect the rendered output.
|
|
65
|
-
- Confirm final-result metadata and artifact storage agree.
|
|
66
|
-
|
|
67
|
-
## Runtime and packaging
|
|
68
|
-
|
|
69
|
-
- Build the exact production Dockerfile/image, not only a development image.
|
|
70
|
-
- Start the real entrypoint and execute a representative app-server turn.
|
|
71
|
-
- Check required binaries, fonts, renderers, runtime dependencies, health,
|
|
72
|
-
signals, and restart behavior inside the image.
|
|
73
|
-
- Record the image digest and bind later evidence to it.
|
|
74
|
-
|
|
75
|
-
## Backend, database, and asynchronous work
|
|
76
|
-
|
|
77
|
-
- Apply migrations against an isolated representative database and verify
|
|
78
|
-
rollback/compatibility expectations where required.
|
|
79
|
-
- Exercise the real API, authentication, authorization/RLS, persistence,
|
|
80
|
-
replay/idempotency, retries, concurrency, queue/worker delivery, and late
|
|
81
|
-
updates.
|
|
82
|
-
- Confirm visible state against persisted rows and runtime logs using the same
|
|
83
|
-
Run ID.
|
|
84
|
-
|
|
85
|
-
## Cross-repository changes
|
|
86
|
-
|
|
87
|
-
- Pin every repository and contract revision.
|
|
88
|
-
- Identify required merge/deploy order and backward-compatibility window.
|
|
89
|
-
- Compose companion and parallel changes before Ready. For an older candidate,
|
|
90
|
-
compare current base behavior and successor changes, including permission guards,
|
|
91
|
-
canonical ownership and supported persisted formats. Reapply the requested delta
|
|
92
|
-
without restoring superseded behavior, even when Git reports no conflict.
|
|
93
|
-
- When a regression expectation conflicts with a later contract, identify that
|
|
94
|
-
contract and preserve its protection. Independently review the expectation change
|
|
95
|
-
and test the combined behavior; do not weaken assertions just to obtain green.
|
|
96
|
-
- Run the user journey against the combined revisions and record all SHAs and
|
|
97
|
-
image digests.
|
|
98
|
-
|
|
99
|
-
## External providers
|
|
100
|
-
|
|
101
|
-
- Use real APIs only when the charter permits cost and side effects.
|
|
102
|
-
- Prefer scoped sandbox accounts, dedicated users/tenants, unique Run IDs, and
|
|
103
|
-
explicit cleanup.
|
|
104
|
-
- Require human authorization for production messages, destructive writes, or
|
|
105
|
-
other consequential effects not already authorized.
|
|
6
|
+
A matrix, incident inventory, independent review or new rule proposal is not required. Do not invent changes to satisfy a template. Preserve branch protection and required CI; skipped required checks are not passing checks.
|
|
@@ -1,136 +1,8 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Browser verification
|
|
2
|
+
When the request affects a browser experience, use the available authorized browser to exercise the relevant visible flow. Check what the user can understand and accomplish, including persistence and recovery when affected. Direct API tests can diagnose failures but do not establish usability of an untested UI.
|
|
2
3
|
|
|
3
|
-
|
|
4
|
-
Codex controlling the user's Chrome through the browser extension.
|
|
4
|
+
Use the correct candidate and environment. A local working journey does not establish deployment. After an authorized deployment, verify the target revision and the affected visible behavior.
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
This is goal-driven user simulation, not a direct API test and not merely a
|
|
9
|
-
fixed selector script. Codex receives a user outcome, perceives the real page,
|
|
10
|
-
and operates Chrome through normal visible interactions. Deterministic checks
|
|
11
|
-
then judge the resulting evidence.
|
|
12
|
-
|
|
13
|
-
Use distinct claims:
|
|
14
|
-
|
|
15
|
-
- `user_like_local_verified`: the exact candidate revisions passed in a local
|
|
16
|
-
production-like stack through Chrome;
|
|
17
|
-
- `live_verified`: the exact deployed revision passed in the target shared or
|
|
18
|
-
production environment through Chrome.
|
|
19
|
-
|
|
20
|
-
## Prepare the local environment
|
|
21
|
-
|
|
22
|
-
Prefer a Compose-style environment bundle over one monolithic container. Use
|
|
23
|
-
the production Dockerfiles and exact candidate revisions for applicable
|
|
24
|
-
services such as:
|
|
25
|
-
|
|
26
|
-
```text
|
|
27
|
-
web
|
|
28
|
-
API / orchestration
|
|
29
|
-
backend or edge functions
|
|
30
|
-
agent runtime
|
|
31
|
-
database and authentication
|
|
32
|
-
queue and workers
|
|
33
|
-
object storage
|
|
34
|
-
supporting renderers or processors
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
Run Chrome on the host and route it to the environment through stable local
|
|
38
|
-
URLs. Account for container networking: `127.0.0.1` inside a container is not
|
|
39
|
-
the host. Use the platform's supported host gateway or service DNS and record
|
|
40
|
-
the resolved endpoints.
|
|
41
|
-
|
|
42
|
-
Start services, apply migrations and seeds, verify health, and record service
|
|
43
|
-
SHAs plus image digests before opening the user journey.
|
|
44
|
-
|
|
45
|
-
## Connect Codex to Chrome
|
|
46
|
-
|
|
47
|
-
Use the Chrome/browser-control skill available in the current environment. The
|
|
48
|
-
user's Chrome profile must have the ChatGPT browser extension installed and
|
|
49
|
-
connected through the desktop app's Computer Use settings. Use the explicitly
|
|
50
|
-
selected Chrome surface; do not silently substitute another browser.
|
|
51
|
-
|
|
52
|
-
Use a dedicated verification profile and allowlist only the required local,
|
|
53
|
-
dev, and external-provider domains. Do not inspect cookies, passwords, browser
|
|
54
|
-
history, or profile storage. If authentication blocks the task, ask the user to
|
|
55
|
-
sign in in the selected Chrome profile and resume from the same journey.
|
|
56
|
-
|
|
57
|
-
## Define the user scenario
|
|
58
|
-
|
|
59
|
-
Give the browser actor a natural-language goal and observable success criteria,
|
|
60
|
-
not internal API steps. Example shape:
|
|
61
|
-
|
|
62
|
-
```text
|
|
63
|
-
Open the application, complete <real user objective>, wait for completion,
|
|
64
|
-
refresh the page, leave and return through another session, follow up on the
|
|
65
|
-
result, download the final artifact, and open it.
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
The actor should use visible UI interactions. Direct API, database, and log
|
|
69
|
-
access are secondary evidence and diagnosis channels, not substitutes for the
|
|
70
|
-
browser journey.
|
|
71
|
-
|
|
72
|
-
## Use real APIs safely
|
|
73
|
-
|
|
74
|
-
When permitted, use the actual model/provider and real integration APIs to
|
|
75
|
-
exercise streaming, tool calls, latency, rate limits, and output variability.
|
|
76
|
-
Require:
|
|
77
|
-
|
|
78
|
-
- sandbox or dedicated verification accounts;
|
|
79
|
-
- scoped credentials and tenant/user fixtures;
|
|
80
|
-
- unique Run IDs and timestamps;
|
|
81
|
-
- attempt, time, and spend caps;
|
|
82
|
-
- dedicated Slack/Gmail/calendar or other external destinations;
|
|
83
|
-
- reconciliation before retrying an ambiguous external effect;
|
|
84
|
-
- recorded cleanup and residual count zero.
|
|
85
|
-
|
|
86
|
-
Do not broaden existing authority. External messages, production writes,
|
|
87
|
-
destructive operations, and new paid effects still require the applicable
|
|
88
|
-
authorization.
|
|
89
|
-
|
|
90
|
-
## Execute and collect evidence
|
|
91
|
-
|
|
92
|
-
Codex may orchestrate the full sequence in one task:
|
|
93
|
-
|
|
94
|
-
1. Build/start/restart services from the shell.
|
|
95
|
-
2. Confirm health and runtime identity.
|
|
96
|
-
3. Operate the application through Chrome.
|
|
97
|
-
4. Capture screenshots and video when supported and relevant.
|
|
98
|
-
5. Record visible messages, navigation, refresh/session behavior, download
|
|
99
|
-
paths, browser console/network failures, and timestamps.
|
|
100
|
-
6. Correlate the visible result with backend logs, persisted state, Run IDs,
|
|
101
|
-
queue/worker activity, and artifact hashes.
|
|
102
|
-
7. Evaluate the evidence against the frozen charter.
|
|
103
|
-
8. On failure, classify the boundary, add a pre-fix regression, patch the
|
|
104
|
-
smallest surface, rebuild changed images, and repeat within the authorized scope while verification improves.
|
|
105
|
-
|
|
106
|
-
The same Codex task may perform actor and verifier phases, but the verifier must
|
|
107
|
-
use recorded observable evidence rather than accepting the actor's completion
|
|
108
|
-
claim.
|
|
109
|
-
|
|
110
|
-
## Default acceptance predicate
|
|
111
|
-
|
|
112
|
-
Tailor this to the charter:
|
|
113
|
-
|
|
114
|
-
```text
|
|
115
|
-
chrome_user_journey_passed =
|
|
116
|
-
visible_result_confirmed
|
|
117
|
-
&& refresh_persistence_confirmed
|
|
118
|
-
&& session_return_confirmed
|
|
119
|
-
&& follow_up_lineage_confirmed_when_applicable
|
|
120
|
-
&& backend_run_id_matched
|
|
121
|
-
&& no_unexpected_console_or_network_failure
|
|
122
|
-
&& artifact_download_open_hash_confirmed_when_applicable
|
|
123
|
-
&& fixture_cleanup_confirmed
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
Save the browser profile label, target URL/environment, source revisions, image
|
|
127
|
-
digests, user/tenant fixture, Run ID, start/end time, screenshots/video,
|
|
128
|
-
download hashes, and skipped checks in the receipt.
|
|
129
|
-
|
|
130
|
-
## After deployment
|
|
131
|
-
|
|
132
|
-
First verify that the target environment serves the exact merge revision or
|
|
133
|
-
derived image digest. Then repeat the smallest user journey that still crosses
|
|
134
|
-
the changed boundary. A preview, local environment, or healthy endpoint cannot
|
|
135
|
-
establish `live_verified` for another deployed environment.
|
|
6
|
+
Preserve authentication boundaries and use isolated test accounts or resources where needed. Do not inspect unrelated private browser data. Do not send messages, incur new costs or make production changes beyond existing authorization. Reconcile uncertain external effects before retries and clean up test resources you created.
|
|
136
7
|
|
|
8
|
+
No fixed browser brand, container topology, recording, screenshot package, receipt or universal journey template is required unless the user specifically requests it. Capture information when useful to understand or repair a concrete problem.
|
|
@@ -1,76 +1,6 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Coordinating related changes
|
|
2
|
+
Identify related changes and choose non-conflicting ownership for writes. Prefer isolated worktrees and test resources. Use leases or equivalent exclusive ownership for shared mutable databases, queues, profiles and ports; do not reset another task's resources.
|
|
2
3
|
|
|
3
|
-
|
|
4
|
-
or shared test infrastructure can affect one user outcome.
|
|
4
|
+
Check shared contracts and the combined user journey using the intended versions. A passing component test does not establish the whole result. Recheck affected boundaries when a dependency changes, retaining unaffected results. No integration matrix or typed delivery receipt is required.
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
Persist one `controller_id` for the user outcome. Discover related active tasks,
|
|
9
|
-
branches, PRs, worktrees, environments, and delivery snapshots before changing
|
|
10
|
-
code. A related task may own a lane, but it must not independently promote the
|
|
11
|
-
overall delivery state.
|
|
12
|
-
|
|
13
|
-
Each lane returns revision-bound typed receipts to the controller. The controller
|
|
14
|
-
alone may freeze companion revisions, order integration gates, declare Ready,
|
|
15
|
-
merge, deploy, or issue the completion receipt. If two controllers are found,
|
|
16
|
-
stop overlapping writes, select one controller, record the superseded controller,
|
|
17
|
-
and reconcile both snapshots before continuing.
|
|
18
|
-
|
|
19
|
-
## Freeze an integration snapshot
|
|
20
|
-
|
|
21
|
-
Before expensive verification, atomically record:
|
|
22
|
-
|
|
23
|
-
- every repository, base SHA, candidate SHA, companion SHA, and generated artifact digest;
|
|
24
|
-
- one owner and one dedicated worktree for each writable lane;
|
|
25
|
-
- one owner and one unique identity for each mutable database, port range, tenant,
|
|
26
|
-
browser profile, queue, and fixture set;
|
|
27
|
-
- dependency and merge order;
|
|
28
|
-
- the cross-boundary contract smoke test and the combined user journey;
|
|
29
|
-
- retry budgets and known failure fingerprints.
|
|
30
|
-
|
|
31
|
-
Reject a receipt whose revision or environment identity does not match the frozen
|
|
32
|
-
snapshot. A lane must not reuse another lane's worktree or mutable environment.
|
|
33
|
-
|
|
34
|
-
## Verify cheap boundaries first
|
|
35
|
-
|
|
36
|
-
Use this order unless the charter records a justified exception:
|
|
37
|
-
|
|
38
|
-
1. schema, receipt, API, and generated-artifact contract smoke tests;
|
|
39
|
-
2. focused regression tests at the escaped boundary;
|
|
40
|
-
3. repository formatting, lint, type checks, and unit/integration suites;
|
|
41
|
-
4. independent review as required by risk/repository rules; SQM only when
|
|
42
|
-
explicitly opted in for this task;
|
|
43
|
-
5. composed multi-repository runtime and user journey;
|
|
44
|
-
6. CI, authorized merge/deploy, and exact-revision live verification.
|
|
45
|
-
|
|
46
|
-
The first failing gate stops later expensive gates for that candidate. Preserve
|
|
47
|
-
unaffected passing receipts rather than rerunning every suite.
|
|
48
|
-
|
|
49
|
-
## Lock shared resources
|
|
50
|
-
|
|
51
|
-
Prefer isolated resources. When isolation is impossible, acquire a lease containing
|
|
52
|
-
`controller_id`, `lane_id`, `run_id`, resource identity, acquisition time, expiry,
|
|
53
|
-
and heartbeat. Compare-and-set acquisition and release. A reset, migration, seed,
|
|
54
|
-
queue purge, browser-profile mutation, or port rebinding without the matching lease
|
|
55
|
-
is an integrity failure, not a retryable test failure.
|
|
56
|
-
|
|
57
|
-
Local Supabase runs must use a unique project identity and non-overlapping ports per
|
|
58
|
-
lane, or a single leased stack. Never let two lanes reset the same stack.
|
|
59
|
-
|
|
60
|
-
## Bound retries and HEAD changes
|
|
61
|
-
|
|
62
|
-
Normalize each failure into a fingerprint containing gate, revision set,
|
|
63
|
-
environment identity, error category, and stable error signature. Default to three
|
|
64
|
-
evaluated attempts. The same fingerprint three times, two non-improving candidates,
|
|
65
|
-
or an A-B-A pattern requires root-cause repair or replan; do not repeat the command.
|
|
66
|
-
|
|
67
|
-
When a repository HEAD changes, update the snapshot first. Invalidate only gates
|
|
68
|
-
whose inputs changed according to the state-and-receipt matrix. Run the contract
|
|
69
|
-
smoke test before returning to full suites.
|
|
70
|
-
|
|
71
|
-
## Prove the combined state early
|
|
72
|
-
|
|
73
|
-
For overlapping features, define execution ownership before implementation. One
|
|
74
|
-
component must own each side effect, retry, wait, wake, cancellation, and completion
|
|
75
|
-
decision. Projections may display state but must consume the same canonical receipt
|
|
76
|
-
chain. Test the combined ownership contract before calling either feature complete.
|
|
6
|
+
Preserve operation identity, cancellation and duplicate prevention across components. If a check repeatedly fails without progress, investigate the cause or choose another useful approach instead of repeating the same operation. Preserve user limits and action-specific authority.
|