projmux 0.14.2 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README-ko.md +58 -114
- package/README.md +49 -110
- package/docs/agent-workflow.md +1420 -37
- package/docs/architecture.md +95 -70
- package/docs/assets/projmux-ai-attention-ko.gif +0 -0
- package/docs/assets/projmux-ai-attention.gif +0 -0
- package/docs/assets/projmux-overview-ko.gif +0 -0
- package/docs/assets/projmux-overview.gif +0 -0
- package/docs/assets/projmux-three-pane-workflow-ko.gif +0 -0
- package/docs/assets/projmux-three-pane-workflow.gif +0 -0
- package/docs/claude-coordination-endpoints.md +254 -0
- package/docs/cli-guide.md +283 -39
- package/docs/cli.md +2059 -56
- package/docs/codex-generation-pool.md +618 -0
- package/docs/codex-installed-compatibility.md +69 -0
- package/docs/codex-native-required-migration.md +98 -9
- package/docs/column-profiles.md +83 -0
- package/docs/configuration.md +43 -4
- package/docs/heterogeneous-dialogue-canary.md +339 -0
- package/docs/hooks.md +48 -15
- package/docs/npm-distribution.md +31 -0
- package/docs/operational-diagnostics.md +125 -0
- package/docs/pr-guideline.md +3 -2
- package/docs/replacement-contract.md +555 -0
- package/docs/session-restore.md +15 -5
- package/docs/settings-ia.md +5 -4
- package/docs/testing.md +12 -7
- package/docs/troubleshooting.md +19 -0
- package/docs/upgrading.md +48 -3
- package/npm/projmux.js +65 -0
- package/package.json +5 -5
|
@@ -0,0 +1,555 @@
|
|
|
1
|
+
# Replacement completion and restorability
|
|
2
|
+
|
|
3
|
+
Three layers of this application hold an execution image, and every consumer of
|
|
4
|
+
`make install`, `npm install`, or a provider generation upgrade believes one
|
|
5
|
+
sentence about all three at once. That sentence is false in a different way on
|
|
6
|
+
each layer. This document fixes what is guaranteed, what is not, and how a
|
|
7
|
+
report can be recovered from the tokens the diagnosis prints.
|
|
8
|
+
|
|
9
|
+
The layers are named `L1`, `L2`, and `L3` throughout, and they are the rows of
|
|
10
|
+
the `projmux doctor` replacement table:
|
|
11
|
+
|
|
12
|
+
| Layer | Subject | What holds an image here |
|
|
13
|
+
| --- | --- | --- |
|
|
14
|
+
| `L1` | `installed-executable-image` | the file the installed path publishes |
|
|
15
|
+
| `L2` | `long-lived-projmux-processes` | every running child of that file |
|
|
16
|
+
| `L3` | `provider-sessions-and-generations` | provider sessions and the managed generation pool |
|
|
17
|
+
|
|
18
|
+
`projmux doctor` renders the table on an unfiltered run and under
|
|
19
|
+
`projmux doctor --section replacement`. It is read-only in the same sense as
|
|
20
|
+
every other section: the Registry read is the zero-write snapshot read, the
|
|
21
|
+
process table is two file reads per process, and nothing on this path starts,
|
|
22
|
+
signals, stops, or repairs anything it reports.
|
|
23
|
+
|
|
24
|
+
## C-1 Replacement completion and restorability
|
|
25
|
+
|
|
26
|
+
**Assumption.** A consumer — an operator, an owning session, or CI — believes
|
|
27
|
+
that *from the moment `install` or `update` returns success, the old execution
|
|
28
|
+
image accepts no new work on any of the three layers*.
|
|
29
|
+
|
|
30
|
+
**Guarantee.** Per layer, what is actually true:
|
|
31
|
+
|
|
32
|
+
- `L1` publishes the new binary with a rename, so a process that starts after
|
|
33
|
+
the rename gets the new image. The *install* around that rename is not
|
|
34
|
+
atomic: config convergence runs before the rename and again after it, and a
|
|
35
|
+
failure between the two leaves a published binary whose live config was never
|
|
36
|
+
converged. There is no atomic install; there is one atomic step inside a
|
|
37
|
+
non-atomic install.
|
|
38
|
+
- `L2` is replaced for one role and reported for the rest. Replacing a file does
|
|
39
|
+
not replace the image of a process already running it, so an install has to
|
|
40
|
+
ask. `broker-runtime` is asked, through the drain this application already
|
|
41
|
+
ships: a runtime whose own image has been unlinked drains at the next session
|
|
42
|
+
that reaches it, refuses new work with `drain-required`, and closes when its
|
|
43
|
+
last binding goes. Every other role is reported with the route that does
|
|
44
|
+
replace it and is left alone — see *The L2 replacement policy* below. A
|
|
45
|
+
replacement that has not finished within the drain cutoff is reported as
|
|
46
|
+
`replacement-cutoff-reached` and is still not ended.
|
|
47
|
+
- `L3` cannot enter `draining` without a qualified version pair. Both doors into
|
|
48
|
+
a generation switch require a measured receipt for exactly the pair they are
|
|
49
|
+
about: managed activation refuses before it writes the journal, and a handover
|
|
50
|
+
resume refuses before it drives any effect — the one exception being a
|
|
51
|
+
generation nothing is bound to, which the planner has always been allowed to
|
|
52
|
+
retire because a receipt proves a thread survives a cross-version resume and
|
|
53
|
+
such a generation has no thread to carry. Each refusal names the action that
|
|
54
|
+
clears it, and that action is now runnable: `scripts/test-generation-pool-qualification.sh`
|
|
55
|
+
measures a declared pair in isolation and
|
|
56
|
+
`projmux agent app-server upgrade qualify --receipt <absolute-json>` installs
|
|
57
|
+
the receipt it writes where both doors read it. **What is not guaranteed is
|
|
58
|
+
the measurement's honesty.** The gate checks that the receipt's coverage
|
|
59
|
+
counters can have come from a run — an acceptance claim with no observation
|
|
60
|
+
behind it is refused as forged — but a determined forger who writes a coverage
|
|
61
|
+
count of one is not contradicted by anything else in the receipt. The gate
|
|
62
|
+
makes the receipt state its coverage; it does not attest to it.
|
|
63
|
+
|
|
64
|
+
**Non-Guarantee.** Explicitly outside this contract:
|
|
65
|
+
|
|
66
|
+
- Darwin observation of `L1` and `L2`. There is no `/proc/<pid>/exe`, so neither
|
|
67
|
+
the image link nor the process census can be taken, and both rows report
|
|
68
|
+
`unsupported-platform` with the `unknown` verdict on both axes. This contract
|
|
69
|
+
does not build a substitute observation for that platform; it states the gap.
|
|
70
|
+
- The provider's own upstream seamless-upgrade contract. `L3` reasons about the
|
|
71
|
+
pool this application manages, never about what the provider promises.
|
|
72
|
+
- An externally owned (`unmanaged`) app-server. Its state is not this
|
|
73
|
+
application's to observe or restore.
|
|
74
|
+
- Automatic classification of which changes form one invariant. See C-2.
|
|
75
|
+
|
|
76
|
+
**Scope.** In space: one machine's projmux installation, the long-lived
|
|
77
|
+
processes it owns, and the generation pool it manages. In time: from the return
|
|
78
|
+
of one `install` or `update` to the next. **Expiry: a verdict expires when the
|
|
79
|
+
installed binary's SHA changes.** A table printed before an install describes an
|
|
80
|
+
installation that no longer exists.
|
|
81
|
+
|
|
82
|
+
**Failure and recovery.** Per layer:
|
|
83
|
+
|
|
84
|
+
| Layer | Detection signal | Recovery route |
|
|
85
|
+
| --- | --- | --- |
|
|
86
|
+
| `L1` | this reader's own executable link carries the kernel's `(deleted)` suffix | none for the binary. No copy of the replaced image is retained, and the recovery text the install prints on failure is config convergence, never a rollback. Restorability is `not-restorable` on every supported path. |
|
|
87
|
+
| `L2` | the whole-fleet vintage census, the newest `install-residue.jsonl` record whose own census observed anything, and the last `install-replacement.json` pass | end and relaunch the residual process through the routes this application already ships. The drain cutoff bounds the wait, and reaching it changes the row rather than the fleet, so a failed replacement leaves every process exactly where a successful one would have found it. |
|
|
88
|
+
| `L3` | generation-pool status, its qualification result, and the Running-versus-live-session census | produce a receipt for the pair with `scripts/test-generation-pool-qualification.sh <old> <new> <output-dir>` and install it with `projmux agent app-server upgrade qualify --receipt <absolute-json>`. **The producer is the recovery route**: a pool that can be qualified can be entered, and one that cannot is refused before it enters. The 2026-09-07 measurement — no route back once `draining` was entered without a verdict — held because nothing read a produced receipt; the entry paths now refuse rather than reach that state. |
|
|
89
|
+
|
|
90
|
+
**Enforcement.** `TestDoctorReplacementLayerVerdictsAreFixedByInputCombination`
|
|
91
|
+
fixes every input combination to its two verdicts and its token.
|
|
92
|
+
`TestDoctorReplacementUnknownVerdictsNameTheEvidenceGap` requires each `unknown`
|
|
93
|
+
to name the evidence that was missing.
|
|
94
|
+
`TestDoctorReplacementDarwinReportsUnsupportedPlatformForImageAndProcessLayers`
|
|
95
|
+
holds the platform branch.
|
|
96
|
+
`TestDoctorReplacementQualificationMissingMakesGenerationPoolNotRestorable`
|
|
97
|
+
holds the `L3` mapping and
|
|
98
|
+
`TestDoctorReplacementL3RestorationMovesWhenTheQualificationLaneOpens` holds the
|
|
99
|
+
transition across it. For the `L3` guarantee above,
|
|
100
|
+
`TestActivateManagedCurrentRefusesEveryUnqualifiedRequestBeforeDraining` and
|
|
101
|
+
`TestResumeRefusesAnUnqualifiedPairBeforeAnyHandoverEffect` hold the two doors,
|
|
102
|
+
`TestResumeStillRetiresAVacantGenerationWithoutAReceipt` holds the vacancy
|
|
103
|
+
exception, and `TestQualificationGateRefusesEvidenceCountersNoObservationBacks`
|
|
104
|
+
holds the forgery refusal. For the `L2` guarantee above,
|
|
105
|
+
`TestBrokerRuntimeDrainsWhenItsOwnImageWasReplaced` holds the vintage entry
|
|
106
|
+
condition and that a runtime on the installed image is not drained by it,
|
|
107
|
+
`TestReplacementRolePoliciesMatchTheContractDocument` and
|
|
108
|
+
`TestReplacementSessionClientIsNeverADrainTarget` hold the policy table, and
|
|
109
|
+
`TestInstallReplacementPassOutcomesAreFixedByFleetAndRequest` fixes every
|
|
110
|
+
outcome of the install pass.
|
|
111
|
+
|
|
112
|
+
## C-2 The unit of replacement is an invariant, not a file
|
|
113
|
+
|
|
114
|
+
**When one invariant is held by code in several files, installing part of that
|
|
115
|
+
set is an atomicity violation.** The file boundary is not the unit; the
|
|
116
|
+
invariant is. A build that carries half of one invariant is not a smaller
|
|
117
|
+
version of the change — it is a state neither the old code nor the new code was
|
|
118
|
+
written to produce.
|
|
119
|
+
|
|
120
|
+
**The 2026-09-08 split install.** Removing a gate in
|
|
121
|
+
`claude_explicit_reply.go` and routing `PutReply` through the target-derived
|
|
122
|
+
adapter in `store.go` were one invariant. Only the first was installed. The
|
|
123
|
+
intermediate build wrote one Agent-message record with `adapter=codex-inbox`
|
|
124
|
+
against `target.provider=claude`, and the strict record validator then refused
|
|
125
|
+
the whole store: **46 records became unreadable, and `agent message send` failed
|
|
126
|
+
in every session on the machine.** Expiry and prune both run after a load, so
|
|
127
|
+
the store could not clear the bad record by itself — the failure was not
|
|
128
|
+
self-healing. The recovery was not deleting the file; it was rewriting the
|
|
129
|
+
mismatched `adapter` to the value the record's own target implies.
|
|
130
|
+
|
|
131
|
+
**And `L2` widens that window.** `make install` does not replace the image of a
|
|
132
|
+
running helper, so a process started from the intermediate build keeps writing
|
|
133
|
+
bad records after the source file has been corrected. That is exactly what
|
|
134
|
+
happened: the record was corrected once and reappeared. The three layers do not
|
|
135
|
+
fail independently — an `L1` partial publication is multiplied by the `L2`
|
|
136
|
+
processes still running the partial image.
|
|
137
|
+
|
|
138
|
+
**Non-Guarantee.** This contract does **not** decide automatically which change
|
|
139
|
+
set forms one invariant. There is no analysis here that groups files by the
|
|
140
|
+
invariants they hold, and none is implied. The contract says only that a partial
|
|
141
|
+
install of such a set is an atomicity violation, and that `L2` extends how long
|
|
142
|
+
that violation stays observable.
|
|
143
|
+
|
|
144
|
+
**Enforcement.** The `L2` row is what makes the widened window visible at all:
|
|
145
|
+
`TestDoctorReplacementLayerVerdictsAreFixedByInputCombination` fixes
|
|
146
|
+
`residual-processes-present` and `install-residue-recorded` to `not-replaced`,
|
|
147
|
+
so a report taken after an install states how much of the fleet that install did
|
|
148
|
+
not reach.
|
|
149
|
+
|
|
150
|
+
## C-3 A failure path preserves its discriminant
|
|
151
|
+
|
|
152
|
+
**Assumption.** A consumer believes that *when a refusal or failure is reported,
|
|
153
|
+
the reason token arrives together with enough of the underlying evidence to
|
|
154
|
+
identify which case produced it*.
|
|
155
|
+
|
|
156
|
+
**Guarantee.** This is a **partial** guarantee today, and points where the
|
|
157
|
+
underlying evidence is discarded are known. On the surface this document owns,
|
|
158
|
+
the guarantee is total: **every row carries at least one discriminant, and a row
|
|
159
|
+
with a token and no discriminant is a contract violation**.
|
|
160
|
+
|
|
161
|
+
**Non-Guarantee.** Exposing the *whole* underlying text on a user-facing
|
|
162
|
+
surface. What is required is a **recoverable discriminant**, not a dump, and
|
|
163
|
+
this contract does not dictate a logging surface.
|
|
164
|
+
|
|
165
|
+
**The 2026-09-08 case.** The only symptom was the single line
|
|
166
|
+
`malformed Agent message store`. It named neither the offending record nor the
|
|
167
|
+
condition that rejected it. A peer session had to re-implement the record
|
|
168
|
+
validator and walk all 46 records to identify the one. The `status` route took
|
|
169
|
+
the same load, so the fallback diagnosis path was dead too. **Failing closed was
|
|
170
|
+
correct. A fail-closed with no discriminant, combined with a deadlock, is
|
|
171
|
+
indistinguishable from unrecoverable.**
|
|
172
|
+
|
|
173
|
+
**How this surface preserves the discriminant.** Each row carries a `signals`
|
|
174
|
+
list of `key=value` pairs beside its reason token. The keys come from a closed
|
|
175
|
+
inventory and the values are either counters this surface formats or tokens
|
|
176
|
+
drawn from another closed vocabulary. `residual-processes-present` and
|
|
177
|
+
`install-residue-recorded` both say an install left work behind; only
|
|
178
|
+
`processes.residual`, `residual.role.*`, `residual.oldest-seconds`, and
|
|
179
|
+
`ledger.latest.residual` say how much, of what, how old, and whether the
|
|
180
|
+
reading was live or from the ledger. A `residual.role.*` key is emitted only
|
|
181
|
+
for a role that has at least one residual process, so the row states what an
|
|
182
|
+
install did not replace and never carries a zero.
|
|
183
|
+
|
|
184
|
+
**Where the discriminant is published, and why.** The signals are serialized in
|
|
185
|
+
`--json`, unlike `doctorFinding.Details`, which is `json:"-"` in every format.
|
|
186
|
+
That difference is deliberate and rests on one fact: a stored Registry refusal
|
|
187
|
+
reason quotes absolute paths, so it cannot be serialized without the support
|
|
188
|
+
report depending on a redaction allowlist. **No signal value here is free-form.**
|
|
189
|
+
Every value is a counter or a closed token, a regular expression rejects
|
|
190
|
+
anything else at construction, and a value that fails it is replaced with
|
|
191
|
+
`unclassified` rather than serialized. No path, argv word, pid, Agent UID, Pane
|
|
192
|
+
UID, thread id, or prose sentence can reach the JSON on this path. Because the
|
|
193
|
+
discriminant is closed, it needs no terminal-only channel, and putting it in
|
|
194
|
+
JSON is what lets CI and a support report recover a verdict at all.
|
|
195
|
+
|
|
196
|
+
**Registry `Running` is not a provider session.** `Running` is a statement about
|
|
197
|
+
a Pane's managed activation. It is not a statement about the provider session
|
|
198
|
+
behind it: a Pane can be present in the runtime while the session it was bound
|
|
199
|
+
to is gone, and the Registry spells both `Running`. The discriminant differs by
|
|
200
|
+
provider and that difference is structural — a Claude Pane carries a real
|
|
201
|
+
process handle at `status.activation.claude.process`, so liveness is a direct
|
|
202
|
+
question about a pid; a Codex Pane carries no pid at all, only a thread and a
|
|
203
|
+
composite authority, so the only checkable fact is whether that authority still
|
|
204
|
+
belongs to the runtime currently serving the state domain. The census therefore
|
|
205
|
+
reports three counts — `sessions.live`, `sessions.dead`, `sessions.unobservable`
|
|
206
|
+
— and the third is not a rounding bucket: it is the count of Running Agents
|
|
207
|
+
resting on the Registry alone.
|
|
208
|
+
|
|
209
|
+
The census is **count-only**. It names no Agent and no Pane, for the reason the
|
|
210
|
+
managed-authority census states: a diagnostics surface that names Agents is one
|
|
211
|
+
that cannot be shared. It also **repairs nothing**. A mismatch is reported and
|
|
212
|
+
left exactly as it was found.
|
|
213
|
+
|
|
214
|
+
**Enforcement.**
|
|
215
|
+
`TestDoctorReplacementEveryReasonTokenCarriesARecoverableDiscriminant` requires
|
|
216
|
+
zero rows with a token and no discriminant across every reachable input
|
|
217
|
+
combination. `TestDoctorReplacementSignalValuesCarryNoPathProcessOrFreeFormText`
|
|
218
|
+
holds the serialization rule, and
|
|
219
|
+
`TestDoctorReplacementSignalKeysMatchTheContractDocument` holds the key
|
|
220
|
+
inventory equal to the one published below.
|
|
221
|
+
`TestDoctorReplacementCensusSeparatesRegistryRunningFromLiveProviderSession`
|
|
222
|
+
fixes the Running-versus-live-session split against a fixture.
|
|
223
|
+
`TestDoctorRendersReplacementTableInDefaultRunAndSectionFilter` holds the output
|
|
224
|
+
contract for the unfiltered run, the section filter, and the JSON surface, and
|
|
225
|
+
`TestDoctorReplacementSectionCreatesNothing` holds the read-only claim by
|
|
226
|
+
running the section against an empty home and requiring it to stay empty.
|
|
227
|
+
|
|
228
|
+
## Reason token vocabulary
|
|
229
|
+
|
|
230
|
+
Exactly one token governs each row: the first entry of its layer's list below
|
|
231
|
+
that the evidence selects. Everything else the layer observed stays in that
|
|
232
|
+
row's signals, so a fact that loses the precedence contest is never lost from
|
|
233
|
+
the report. The lists are in precedence order.
|
|
234
|
+
|
|
235
|
+
`TestDoctorReplacementReasonTokensMatchTheContractDocument` holds this table and
|
|
236
|
+
the token constants in the code equal, in both directions.
|
|
237
|
+
|
|
238
|
+
| Layer | Token | Replacement | Restoration | Selected when |
|
|
239
|
+
| --- | --- | --- | --- | --- |
|
|
240
|
+
| `L1` | `unsupported-platform` | `unknown` | `unknown` | the platform exposes no executable link |
|
|
241
|
+
| `L1` | `image-unresolved` | `unknown` | `unknown` | this reader's own image could not be resolved back to its executable path |
|
|
242
|
+
| `L1` | `image-unlinked` | `not-replaced` | `not-restorable` | this reader's image was unlinked out from under it, so the diagnosis is taken from a superseded build |
|
|
243
|
+
| `L1` | `image-current` | `replaced` | `not-restorable` | the installed path still publishes the image this diagnosis runs |
|
|
244
|
+
| `L2` | `unsupported-platform` | `unknown` | `unknown` | the platform exposes no process table |
|
|
245
|
+
| `L2` | `replacement-cutoff-reached` | `not-replaced` | `restorable` | a residual process has outlived the drain cutoff, so this replacement is not going to complete on its own |
|
|
246
|
+
| `L2` | `residual-processes-present` | `not-replaced` | `restorable` | a live child is still running the image from before the last install, and none of them is past the cutoff |
|
|
247
|
+
| `L2` | `no-residual-processes` | `replaced` | `restorable` | live children were observed and every one runs the installed image |
|
|
248
|
+
| `L2` | `install-residue-recorded` | `not-replaced` | `restorable` | no live child was observable, and the newest ledger record whose own census observed anything says that install left residue |
|
|
249
|
+
| `L2` | `install-residue-clean` | `replaced` | `restorable` | no live child was observable, and the newest ledger record whose own census observed anything says that install left none |
|
|
250
|
+
| `L2` | `no-observed-processes` | `unknown` | `unknown` | neither the live census nor any ledger record observed anything. Every reading is silent; none says the fleet is clean |
|
|
251
|
+
| `L3` | `generation-pool-unobserved` | `unknown` | `unknown` | no pool diagnosis was read |
|
|
252
|
+
| `L3` | `registry-running-provider-session-dead` | pool-derived | `not-restorable` | provider evidence contradicts a Registry `Running` Agent |
|
|
253
|
+
| `L3` | `qualification-missing` | pool-derived | `not-restorable` | the pool is installed and carries no qualification result. A pool in this state predates the entry gates or had its receipt removed; the row's `pool.action` names the producer that clears it |
|
|
254
|
+
| `L3` | `generation-pool-blocked` | pool-derived | `not-restorable` | the pool diagnosis is blocked |
|
|
255
|
+
| `L3` | `generation-handover-required` | pool-derived | `restorable` | the pool has a pending operation with a handover route |
|
|
256
|
+
| `L3` | `registry-running-session-unobserved` | pool-derived | `unknown` | Running Agents rest on the Registry alone, with no provider handle to check |
|
|
257
|
+
| `L3` | `generation-pool-not-installed` | `not-replaced` | `unknown` | no generation journal exists. Absence of a journal is absence of evidence about a restore route, not evidence of one |
|
|
258
|
+
| `L3` | `generation-pool-ready` | pool-derived | `restorable` | the pool is installed, qualified, and settled |
|
|
259
|
+
|
|
260
|
+
`pool-derived` means the replacement axis follows the pool rather than the
|
|
261
|
+
token: `replaced` once a generation is draining, handover-pending, or retired —
|
|
262
|
+
because such a generation accepts no new admission, which is exactly the
|
|
263
|
+
sentence C-1's Assumption makes — and `not-replaced` otherwise.
|
|
264
|
+
|
|
265
|
+
## Process role vocabulary
|
|
266
|
+
|
|
267
|
+
`L2` counts every running child of the installed executable, and it names them
|
|
268
|
+
by **route** — the internal command words of the argv, with `argv[0]` and
|
|
269
|
+
everything behind a bare `--` discarded. That is the whole classifier. No pid,
|
|
270
|
+
no path, and no argv word reaches a diagnostics surface from it; the role name
|
|
271
|
+
is a token from the closed list below, and the value beside it is a count.
|
|
272
|
+
|
|
273
|
+
The same vocabulary is used by `projmux doctor`'s `L2` row and by every
|
|
274
|
+
`install-residue.jsonl` record, so the two surfaces name the same processes with
|
|
275
|
+
the same words.
|
|
276
|
+
|
|
277
|
+
| Role | Route | Cardinality | What ending it costs |
|
|
278
|
+
| --- | --- | --- | --- |
|
|
279
|
+
| `broker-runtime` | `internal codex-broker serve` | one per machine | nothing directly: a new broker starts on the installed image when a binding needs one |
|
|
280
|
+
| `lifecycle-observer` | `internal agent-hook ingest codex-broker-watch` | one per Codex pane | that pane's lifecycle ingestion until it is relaunched |
|
|
281
|
+
| `supervisor` | `internal supervise` | one per pane | the pane it supervises |
|
|
282
|
+
| `session-client` | `shell` | one per attached client | **the operator's whole attached session.** This role is the reason a drain cannot be uniform: every other role is background work, and this one is the person at the terminal |
|
|
283
|
+
| `agent-endpoint` | `internal claude-endpoint-helper` | one per registered agent activation | that agent's messaging endpoint until its pane is recreated |
|
|
284
|
+
| `usage-watcher` | `internal status usage --watch-codex-rate-limits` | one per machine, lease-held | rate-limit sampling until the lease is taken again |
|
|
285
|
+
| `other` | every route not named above | unbounded | — |
|
|
286
|
+
|
|
287
|
+
**`other` is a total guard, not a count of replacement targets.** It exists so
|
|
288
|
+
that a route this census has no name for cannot make the fleet look smaller than
|
|
289
|
+
it is, and naming more roles shrinks it without ever removing it. What remains
|
|
290
|
+
in it is the **short-lived invocation**: a status or statusbar render, a preview,
|
|
291
|
+
a picker, a hook callback, a `config apply`, the residue census itself. Those
|
|
292
|
+
processes are observed by whichever census happens to overlap them and are gone
|
|
293
|
+
seconds later. They are counted because the total has to be whole, and they are
|
|
294
|
+
not drain candidates because nothing has to end them.
|
|
295
|
+
|
|
296
|
+
That distinction is what the named roles buy. Before them, `other` was the
|
|
297
|
+
second-largest bucket on this repository's own ledger — 271 of 866 residual
|
|
298
|
+
observations — and it mixed the two kinds, so no reader could tell a long-lived
|
|
299
|
+
process an install failed to replace from a two-second render that happened to
|
|
300
|
+
be running. A residual `other` count is now read as measurement noise unless it
|
|
301
|
+
is large or persistent, in which case it names a long-lived route this list is
|
|
302
|
+
missing.
|
|
303
|
+
|
|
304
|
+
**Adding a role is not a change to the reason-token vocabulary.** The reason
|
|
305
|
+
tokens above are the closed list of *verdicts*; roles are the subjects those
|
|
306
|
+
verdicts are counted over. A role added here reaches the signal key inventory
|
|
307
|
+
automatically, because that inventory expands the census role order, and
|
|
308
|
+
`TestReplacementProcessRolesMatchTheContractDocument` holds this table and the
|
|
309
|
+
code's role order to each other in both directions.
|
|
310
|
+
|
|
311
|
+
## The L2 replacement policy
|
|
312
|
+
|
|
313
|
+
An install may end a long-lived process only where this application already
|
|
314
|
+
ships a way to ask one to stand down, and only where ending it costs nothing an
|
|
315
|
+
operator did not choose. Both halves of that sentence are decisions taken from
|
|
316
|
+
measurement, and they are fixed here rather than at a call site.
|
|
317
|
+
|
|
318
|
+
### Per-role disposition
|
|
319
|
+
|
|
320
|
+
`drain` means the install asks through a shipped path and the process decides
|
|
321
|
+
when it actually goes. `report-only` means no install touches it; the route that
|
|
322
|
+
does replace it is named instead, so a reader of the row knows what would.
|
|
323
|
+
|
|
324
|
+
`TestReplacementRolePoliciesMatchTheContractDocument` holds this table and the
|
|
325
|
+
code's policy map equal, in both directions.
|
|
326
|
+
|
|
327
|
+
| Role | Disposition | Replacement route |
|
|
328
|
+
| --- | --- | --- |
|
|
329
|
+
| `broker-runtime` | `drain` | `broker-drain` |
|
|
330
|
+
| `lifecycle-observer` | `report-only` | `pane-relaunch` |
|
|
331
|
+
| `supervisor` | `report-only` | `pane-relaunch` |
|
|
332
|
+
| `session-client` | `report-only` | `operator-reattach` |
|
|
333
|
+
| `agent-endpoint` | `report-only` | `pane-relaunch` |
|
|
334
|
+
| `usage-watcher` | `report-only` | `lease-expiry` |
|
|
335
|
+
| `other` | `report-only` | `process-exit` |
|
|
336
|
+
|
|
337
|
+
**Exactly one role drains, and that is a fact about what exists rather than a
|
|
338
|
+
first instalment.** A drain is a protocol: the runtime has to be able to hear
|
|
339
|
+
the request, refuse new work without failing anonymously, and carry the work it
|
|
340
|
+
already accepted to its end. `broker-runtime` is the only role that ships one.
|
|
341
|
+
Every other role is replaced by an event that already happens — a pane
|
|
342
|
+
recreated, an activation restarted, a lease taken again — and an install that
|
|
343
|
+
signalled them would sever live work to save an operator an action they can take
|
|
344
|
+
themselves.
|
|
345
|
+
|
|
346
|
+
**`session-client` is the entry that is a decision.** It is the operator's own
|
|
347
|
+
attached tmux session, the longest-lived role on this repository's ledger at
|
|
348
|
+
169h29m, and a uniform cutoff applied to it ends the terminal the install was
|
|
349
|
+
typed into. It is report-only and no cutoff reaches it.
|
|
350
|
+
`TestReplacementSessionClientIsNeverADrainTarget` holds that as a property of
|
|
351
|
+
the table rather than of any call site.
|
|
352
|
+
|
|
353
|
+
**No disposition severs work.** The one hard kill this repository has measured —
|
|
354
|
+
2026-09-05, four panes — brought three of the four back automatically and left
|
|
355
|
+
the fourth wedged on `backlog-overflow`. A recovery asymmetry that large is what
|
|
356
|
+
a default has to be chosen against, so `drain` asks and waits, and the cutoff
|
|
357
|
+
below reports rather than escalates.
|
|
358
|
+
|
|
359
|
+
### The vintage trigger
|
|
360
|
+
|
|
361
|
+
The drain already had one entry condition: a client arrives speaking a protocol
|
|
362
|
+
version this runtime cannot negotiate. That condition misses the case an install
|
|
363
|
+
produces. A new binary usually speaks the same protocol, so a compatible
|
|
364
|
+
handshake proves nothing about which image is behind it.
|
|
365
|
+
|
|
366
|
+
The second entry condition is the runtime's own image. A runtime reads
|
|
367
|
+
`/proc/self/exe` once per arriving session until the answer is yes; the kernel's
|
|
368
|
+
`(deleted)` suffix says the file it is running was unlinked, which is exactly
|
|
369
|
+
what a publication leaves behind. The runtime then enters **the same drain by
|
|
370
|
+
the same door** — the same `drain-required` refusal, the same wire frames, the
|
|
371
|
+
same "live work is carried to its end". No new protocol, no new refusal token,
|
|
372
|
+
and nothing added to the wire.
|
|
373
|
+
|
|
374
|
+
Two consequences follow from putting the trigger there rather than in the
|
|
375
|
+
installer. Any client of the newly installed binary drains a superseded runtime,
|
|
376
|
+
so the npm install path is covered without a second implementation. And a
|
|
377
|
+
platform with no executable link to read declines to drain rather than draining
|
|
378
|
+
on a guess: `defaultProjmuxImageReplaced` answers false on darwin, and the
|
|
379
|
+
absence is stated by this table's `unsupported-platform` row.
|
|
380
|
+
|
|
381
|
+
### The install pass
|
|
382
|
+
|
|
383
|
+
`projmux internal install-replace` runs as a step of `make install`, immediately
|
|
384
|
+
before the residue census so the census measures the fleet the pass left. It
|
|
385
|
+
takes the census, splits the residual processes by disposition, dials the
|
|
386
|
+
published broker runtime for this state domain, waits a bounded moment, and
|
|
387
|
+
writes `install-replacement.json`.
|
|
388
|
+
|
|
389
|
+
It **starts nothing** — a replacement pass that launched what it was sent to
|
|
390
|
+
replace would leave more behind than it found, so it dials and never ensures.
|
|
391
|
+
It **signals nothing**: the only request it makes is a socket handshake. And it
|
|
392
|
+
never fails an install: it runs after the install has already succeeded, and
|
|
393
|
+
everything it could not do is on the record it writes.
|
|
394
|
+
|
|
395
|
+
Its outcome vocabulary is closed, and it reaches the `L2` row as
|
|
396
|
+
`replacement.outcome`:
|
|
397
|
+
|
|
398
|
+
| Outcome | Meaning |
|
|
399
|
+
| --- | --- |
|
|
400
|
+
| `replacement-unsupported-platform` | no process table to take a census from |
|
|
401
|
+
| `replacement-no-target` | no residual process sits in a drainable role |
|
|
402
|
+
| `replacement-complete` | the drain was asked for and finished inside the settle window |
|
|
403
|
+
| `replacement-drain-pending` | the drain was accepted and is still carrying work |
|
|
404
|
+
| `replacement-target-unreachable` | a residual target the shipped path could not reach; `replacement.refusal` says which door was closed |
|
|
405
|
+
| `replacement-not-attempted` | written by no pass — what the row says when no record exists at all |
|
|
406
|
+
|
|
407
|
+
### The drain cutoff
|
|
408
|
+
|
|
409
|
+
A drain that carries live work to its end has no bound of its own: the runtime
|
|
410
|
+
stays up while it still has bindings, and a binding that is never released keeps
|
|
411
|
+
it up forever. **The install has to decide when to stop calling such a
|
|
412
|
+
replacement "in progress" and start calling it unfinished.**
|
|
413
|
+
|
|
414
|
+
`projmux internal install-residue --survival` answers that from measurement.
|
|
415
|
+
On this repository's ledger at 2026-09-08 — 55 records, 884 residual
|
|
416
|
+
observations, **137 distinct processes** — the fraction of residual processes
|
|
417
|
+
still running at each cutoff is:
|
|
418
|
+
|
|
419
|
+
| Cutoff | Residual processes still running | Truncation rate |
|
|
420
|
+
| --- | --: | --: |
|
|
421
|
+
| 1h | 84 of 137 | 61.3% |
|
|
422
|
+
| 4h | 38 of 137 | 27.7% |
|
|
423
|
+
| 12h | 19 of 137 | 13.9% |
|
|
424
|
+
| **24h** | **13 of 137** | **9.5%** |
|
|
425
|
+
| 72h | 7 of 137 | 5.1% |
|
|
426
|
+
| 168h | 4 of 137 | 2.9% |
|
|
427
|
+
|
|
428
|
+
These are the **process-identity** numbers, not the observation numbers. The
|
|
429
|
+
section below says why the two bases disagree in both directions at once and why
|
|
430
|
+
a cutoff chosen from the observation basis would be wrong at both ends.
|
|
431
|
+
|
|
432
|
+
**Adopted: 24h. Truncation rate 9.5% — 13 of 137 by process identity.**
|
|
433
|
+
|
|
434
|
+
The reasoning is the shape of the curve against the cost of being wrong in
|
|
435
|
+
either direction. 1h and 4h report a majority and a large minority of ordinary
|
|
436
|
+
drains as failures, which turns the token into noise rather than a discriminant.
|
|
437
|
+
Past 24h the curve flattens: 24h to 72h buys 4.4 points and costs two more days
|
|
438
|
+
before an operator learns a replacement is never going to complete. 24h is also
|
|
439
|
+
the horizon the answer is read on — a residual process a day old is not a drain
|
|
440
|
+
still finishing its work, it is a drain that is not going to.
|
|
441
|
+
|
|
442
|
+
**The truncation rate is what gets reported, not what gets severed.** Reaching
|
|
443
|
+
the cutoff ends the install's claim about the replacement and never the process:
|
|
444
|
+
the `L2` row turns to `replacement-cutoff-reached`, `replacement.beyond-cutoff`
|
|
445
|
+
counts how many are past it, and every one of them keeps running. That is the
|
|
446
|
+
whole difference between this bound and a kill timer, and it is why 9.5% is a
|
|
447
|
+
reporting rate rather than a severed-session rate.
|
|
448
|
+
|
|
449
|
+
The cutoff is `PROJMUX_REPLACEMENT_CUTOFF`, a Go duration, and it exists so the
|
|
450
|
+
branch is reachable in a bounded test and in an isolated-fleet smoke. An unset,
|
|
451
|
+
unparsable, or non-positive value is the adopted 24h. **There is no value that
|
|
452
|
+
disables it**: an unbounded drain is the state this bound rules out, and offering
|
|
453
|
+
a spelling for it would put that state one environment variable away.
|
|
454
|
+
|
|
455
|
+
**Re-evaluation.** The identity basis rests on 137 processes of which 128 carry a
|
|
456
|
+
start reconstructed by subtraction, exact to one second. As records written with
|
|
457
|
+
a recorded start instant accumulate, that proportion falls and the tail of this
|
|
458
|
+
distribution sharpens. The cutoff is re-derived from the same command when it
|
|
459
|
+
does.
|
|
460
|
+
|
|
461
|
+
## Reading residual survival: observations are not processes
|
|
462
|
+
|
|
463
|
+
The ledger records one census per install, and installs on a development
|
|
464
|
+
machine land minutes apart. A long-lived process is therefore recorded again in
|
|
465
|
+
record after record, once per install it survived. **Counting ledger samples
|
|
466
|
+
weights each process by its own longevity, which is the very quantity being
|
|
467
|
+
measured.**
|
|
468
|
+
|
|
469
|
+
`projmux internal install-residue --survival` reads the accumulated ledger and
|
|
470
|
+
answers each cutoff `T` twice: over samples, and over distinct processes. On
|
|
471
|
+
this repository's ledger at 2026-09-08 (54 records with a non-empty census, 866
|
|
472
|
+
residual samples, 128 distinct processes) the two bases disagree by several
|
|
473
|
+
times:
|
|
474
|
+
|
|
475
|
+
| Cutoff | By observation | By process identity |
|
|
476
|
+
| --- | --: | --: |
|
|
477
|
+
| 1h | 72.2% | 60.9% |
|
|
478
|
+
| 4h | 53.3% | 25.8% |
|
|
479
|
+
| 12h | 49.0% | 13.3% |
|
|
480
|
+
| 24h | 40.9% | 8.6% |
|
|
481
|
+
| 72h | 24.6% | 3.9% |
|
|
482
|
+
| 168h | 0.3% | **2.3%** |
|
|
483
|
+
|
|
484
|
+
The disagreement runs in both directions and neither one is a correction of the
|
|
485
|
+
other. Below 72h the sample basis is inflated, because the processes that
|
|
486
|
+
survive many installs contribute many samples each. At 168h it is *deflated*,
|
|
487
|
+
because the handful of week-old processes are diluted across a sample count they
|
|
488
|
+
did not generate. **A drain cutoff chosen from the sample basis would be wrong
|
|
489
|
+
in both directions at once.**
|
|
490
|
+
|
|
491
|
+
Identity is the pair `(role, start instant)`. The ledger has no pid, path, or
|
|
492
|
+
argv to key on, and does not acquire one for this: a start instant is when
|
|
493
|
+
something began, not which process it was. Records written from this change on
|
|
494
|
+
carry the start instants directly. Older records carry only ages, from which a
|
|
495
|
+
start is reconstructed by subtraction; both the instant and the age are whole
|
|
496
|
+
truncated seconds, so a reconstructed start lands on the true second or the one
|
|
497
|
+
after it, and the report says how many of its identities rest on that
|
|
498
|
+
reconstruction.
|
|
499
|
+
|
|
500
|
+
## Signal key inventory
|
|
501
|
+
|
|
502
|
+
| Key | Layer | Value |
|
|
503
|
+
| --- | --- | --- |
|
|
504
|
+
| `platform.observable` | `L1`, `L2` | `true` / `false` |
|
|
505
|
+
| `image.link` | `L1` | `current` / `unlinked` / `unresolved` |
|
|
506
|
+
| `retained.previous-image` | `L1` | `none` |
|
|
507
|
+
| `processes.observed` | `L2` | counter |
|
|
508
|
+
| `processes.residual` | `L2` | counter |
|
|
509
|
+
| `residual.role.broker-runtime` | `L2` | counter |
|
|
510
|
+
| `residual.role.lifecycle-observer` | `L2` | counter |
|
|
511
|
+
| `residual.role.supervisor` | `L2` | counter |
|
|
512
|
+
| `residual.role.session-client` | `L2` | counter |
|
|
513
|
+
| `residual.role.agent-endpoint` | `L2` | counter |
|
|
514
|
+
| `residual.role.usage-watcher` | `L2` | counter |
|
|
515
|
+
| `residual.role.other` | `L2` | counter |
|
|
516
|
+
| `residual.oldest-seconds` | `L2` | counter |
|
|
517
|
+
| `ledger.records` | `L2` | counter |
|
|
518
|
+
| `ledger.latest.installer` | `L2` | install path token, or `unclassified` |
|
|
519
|
+
| `ledger.latest.observed` | `L2` | counter |
|
|
520
|
+
| `ledger.latest.residual` | `L2` | counter |
|
|
521
|
+
| `replacement.cutoff-seconds` | `L2` | counter |
|
|
522
|
+
| `replacement.beyond-cutoff` | `L2` | counter |
|
|
523
|
+
| `replacement.outcome` | `L2` | install replacement pass outcome token |
|
|
524
|
+
| `replacement.refusal` | `L2` | broker refusal token |
|
|
525
|
+
| `replacement.attempted` | `L2` | counter |
|
|
526
|
+
| `replacement.drained` | `L2` | counter |
|
|
527
|
+
| `replacement.reported` | `L2` | counter |
|
|
528
|
+
| `registry.observed` | `L3` | `true` / `false` |
|
|
529
|
+
| `pool.status` | `L3` | pool status token, or `unobserved` when no pool diagnosis was read |
|
|
530
|
+
| `pool.reason` | `L3` | pool reason token |
|
|
531
|
+
| `pool.action` | `L3` | pool action token |
|
|
532
|
+
| `pool.generations.live` | `L3` | counter |
|
|
533
|
+
| `pool.generations.draining` | `L3` | counter |
|
|
534
|
+
| `qualification.verdict` | `L3` | qualification verdict token |
|
|
535
|
+
| `qualification.reason` | `L3` | qualification reason token |
|
|
536
|
+
| `sessions.running` | `L3` | counter |
|
|
537
|
+
| `sessions.live` | `L3` | counter |
|
|
538
|
+
| `sessions.dead` | `L3` | counter |
|
|
539
|
+
| `sessions.unobservable` | `L3` | counter |
|
|
540
|
+
|
|
541
|
+
## What this surface does not do
|
|
542
|
+
|
|
543
|
+
It does not replace anything, restore anything, end a process, start a process,
|
|
544
|
+
or repair a Registry. It reports, and a reported mismatch is left exactly as it
|
|
545
|
+
was found. `TestReplacementMeasurementPathEndsNoProcess` holds that as a
|
|
546
|
+
property of the source: the census, the residue ledger, the survival report, and
|
|
547
|
+
this table are checked to carry no process termination, signalling, or restart.
|
|
548
|
+
|
|
549
|
+
The replacement policy above is the one thing on this page that acts, and it
|
|
550
|
+
acts through a strictly narrower door. `TestReplacementPathEndsNoProcess` holds
|
|
551
|
+
the same guard over the policy table and the install pass: they carry no
|
|
552
|
+
termination, no signalling, and no restart either. **The whole of the action is
|
|
553
|
+
a socket handshake, and the runtime decides.** `L1` atomicity and the `L3`
|
|
554
|
+
qualification gate are separate work on separate doors, and no change on this
|
|
555
|
+
path may alter their behavior.
|
package/docs/session-restore.md
CHANGED
|
@@ -33,6 +33,14 @@ direct shell as the optional default, and mint identities only for missing
|
|
|
33
33
|
items. An Agent-only Window is valid with an empty default. Repeating the same
|
|
34
34
|
projection is a Registry zero-diff.
|
|
35
35
|
|
|
36
|
+
Resource metadata records the Registry schema that produced it. A v3 snapshot
|
|
37
|
+
is projected through the same root-wide same-kind duplicate-group and
|
|
38
|
+
destination-closure rule as Registry migration: affected resources receive
|
|
39
|
+
their exact UID names, unique names outside the closure are preserved, and no
|
|
40
|
+
numeric suffix is minted. A current-v4 snapshot containing a root-wide
|
|
41
|
+
same-kind collision is rejected as damaged before trust authorization,
|
|
42
|
+
Registry/tmux/provider mutation, or any snapshot write.
|
|
43
|
+
|
|
36
44
|
The committed Registry is then converged by the ordinary Project materializer.
|
|
37
45
|
For a restored offline Agent-anchor Window, `Continue project` visibly plans a
|
|
38
46
|
lazy default shell, creates the Window from that shell, and stages the Agent on
|
|
@@ -53,17 +61,19 @@ A closed Project has exactly two actions:
|
|
|
53
61
|
materializer. A retained graph keeps its Project, Window, Pane, and Agent
|
|
54
62
|
UIDs. A zero-Window Project keeps its Project UID and atomically receives one
|
|
55
63
|
new canonical Window and shell UID before materialization.
|
|
56
|
-
- `
|
|
57
|
-
and one new canonical Window/shell UID chain
|
|
58
|
-
the old
|
|
64
|
+
- `Recreate Project` atomically replaces the same-root graph with a new Project
|
|
65
|
+
UID and one new canonical Window/shell UID chain, after a confirmation naming
|
|
66
|
+
the exact old Project UID and its Window/Pane/Agent counts. Declining returns
|
|
67
|
+
to the startup rows and writes nothing. It does not archive or retain the old
|
|
68
|
+
generation.
|
|
59
69
|
|
|
60
70
|
Esc/cancel returns to Projects; it is not an action row. Picker failure falls
|
|
61
71
|
back to the non-destructive `Continue project` action.
|
|
62
72
|
|
|
63
|
-
`
|
|
73
|
+
`Recreate Project` never deletes or overwrites autosave or named snapshot files. It
|
|
64
74
|
preserves the root, Git/worktrees, trust decision, and all unrelated Registry
|
|
65
75
|
graphs while changing the Project identity. A rejected commit retains the
|
|
66
|
-
exact old Registry preimage. Repeating `
|
|
76
|
+
exact old Registry preimage. Repeating `Recreate Project` replaces identity again;
|
|
67
77
|
each successful result has exactly one Project claiming the root.
|
|
68
78
|
|
|
69
79
|
## Snapshot contents and diagnostics
|
package/docs/settings-ia.md
CHANGED
|
@@ -69,15 +69,16 @@ step, never a silent no-op.
|
|
|
69
69
|
`get runtime` never read it. An unrecognized saved value applies the default
|
|
70
70
|
without writing and shows an invalid source.
|
|
71
71
|
`Closed Project startup` shows exactly two actions when no preference file
|
|
72
|
-
exists and reports `Continue project /
|
|
73
|
-
|
|
74
|
-
`Continue project /
|
|
72
|
+
exists and reports `Continue project / Recreate Project - default` (`이어서 열기 /
|
|
73
|
+
Project 다시 만들기 - 기본값` in ko-KR). Saved `on` keeps those choices and reports
|
|
74
|
+
`Continue project / Recreate Project - on - saved`. Saved `off` reports
|
|
75
75
|
`Continue project - off - saved`, skips the picker, and retains the
|
|
76
76
|
registered-Continue/unregistered-Fresh automatic
|
|
77
77
|
adjudication. Reading the row or opening/cancelling the picker never writes
|
|
78
78
|
the default or changes saved preference bytes/mtime.
|
|
79
79
|
`Continue project` materializes the Project's current Registry desired state
|
|
80
|
-
and then moves the client. `
|
|
80
|
+
and then moves the client. `Recreate Project` confirms the exact old Project
|
|
81
|
+
UID and per-kind counts before writing anything,
|
|
81
82
|
atomically replaces the old Project graph with a new Project UID and a new
|
|
82
83
|
canonical Window/shell UID pair, and leaves exactly one same-root claimant
|
|
83
84
|
before ordinary materialization. Esc returns to Projects. Neither action
|
package/docs/testing.md
CHANGED
|
@@ -29,10 +29,12 @@ and humans run the same entrypoints.
|
|
|
29
29
|
replays one scenario, `PROJMUX_E2E_LINUX_SHARD=<shard>` runs exactly one
|
|
30
30
|
Linux fixture with the terminal inventory its `linux-shards.tsv` row owns,
|
|
31
31
|
and `PROJMUX_E2E_SUITE=codex-lifecycle|npm-staging` runs one non-Linux suite.
|
|
32
|
-
Setting none keeps the
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
32
|
+
Setting none keeps the local acceptance profile: all four Linux shards run
|
|
33
|
+
serially in canonical `fixture-1` through `fixture-4` order on the one host,
|
|
34
|
+
followed by both suites. CI uses the shard/suite selectors to give every
|
|
35
|
+
suite its own runner, so each runner likewise has capacity for one shard.
|
|
36
|
+
`PROJMUX_E2E_LINUX_MODE=parallel make test-e2e` is the explicit same-host
|
|
37
|
+
stress profile; `serial` and `parallel` are the only accepted mode values.
|
|
36
38
|
- Every scenario wait states a budget in seconds rather than a loop count, and
|
|
37
39
|
`E2E_WAIT_SCALE=<factor>` multiplies all of them at once. Raise it when a
|
|
38
40
|
runner is slow or loaded; a wait that expires still fails with the description
|
|
@@ -42,9 +44,12 @@ and humans run the same entrypoints.
|
|
|
42
44
|
`make test-e2e-shards` validate typed attempt evidence, bounded semantic
|
|
43
45
|
waits/owned cleanup, and exhaustive four-shard isolation without rerunning
|
|
44
46
|
the full product matrix. The shard target also pins the CI job list to the
|
|
45
|
-
manifest
|
|
46
|
-
|
|
47
|
-
|
|
47
|
+
manifest. A deterministic fake-runner start/release barrier proves local
|
|
48
|
+
default overlap is one and explicit parallel overlap is four while both use
|
|
49
|
+
one immutable binary, the exact 21-scenario inventory, and the same result
|
|
50
|
+
hash. The target also pins one non-fail-fast CI job per shard and per suite,
|
|
51
|
+
each with its own runner, timeout, and uniquely named evidence artifacts, all
|
|
52
|
+
of them required children of the aggregate `Test` gate. It also pins the thin
|
|
48
53
|
`E2E Tests` job, which exists because the branch ruleset requires a status
|
|
49
54
|
check under that exact name; a required context that is never reported stays
|
|
50
55
|
pending rather than failing, so dropping that job would deadlock merges. The
|