@awebai/oats 0.61.0 → 0.63.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/README.md +1 -1
  2. package/bin/oats.mjs +27 -12
  3. package/docs/capabilities.md +1 -1
  4. package/docs/desktop-cli-api.md +30 -2
  5. package/docs/execution-targets.md +89 -4
  6. package/docs/implementation.md +2 -1
  7. package/docs/named-test-runs.md +382 -0
  8. package/docs/official-catalog.md +1 -1
  9. package/docs/packages.md +5 -5
  10. package/docs/release-lane.md +13 -0
  11. package/docs/release-notes/v0.62.0.md +223 -0
  12. package/docs/release-notes/v0.63.0.md +135 -0
  13. package/docs/souls-and-instances.md +6 -1
  14. package/docs/upgrading.md +135 -11
  15. package/docs/workspaces.md +1 -1
  16. package/lib/core.mjs +73 -23
  17. package/lib/doctor-capture.mjs +35 -0
  18. package/lib/session-viewer.mjs +86 -7
  19. package/lib/tree-copy.mjs +188 -7
  20. package/package-catalog.json +1 -1
  21. package/package.json +1 -1
  22. package/packages/client/cli-locator.mjs +2 -2
  23. package/packages/record/README.md +87 -13
  24. package/packages/record/bin/capture-hook.mjs +8 -0
  25. package/packages/record/bin/capture.mjs +200 -139
  26. package/packages/record/bin/recall.mjs +8 -3
  27. package/packages/record/bin/setup.mjs +116 -225
  28. package/packages/record/bin/turn-record.mjs +2 -2
  29. package/packages/record/docs/capture-service.md +1527 -28
  30. package/packages/record/docs/capture-status.schema.json +429 -12
  31. package/packages/record/lib/capture-admission.mjs +82 -0
  32. package/packages/record/lib/capture-aw-unit.mjs +137 -0
  33. package/packages/record/lib/capture-aw.mjs +31 -12
  34. package/packages/record/lib/capture-budget.mjs +31 -0
  35. package/packages/record/lib/capture-cc.mjs +39 -6
  36. package/packages/record/lib/capture-cleanup-queue.mjs +135 -0
  37. package/packages/record/lib/capture-cleanup-work.mjs +77 -0
  38. package/packages/record/lib/capture-command-operation.mjs +37 -0
  39. package/packages/record/lib/capture-command-output.mjs +62 -0
  40. package/packages/record/lib/capture-command-worker.mjs +6 -0
  41. package/packages/record/lib/capture-command.mjs +65 -0
  42. package/packages/record/lib/capture-completion.mjs +152 -0
  43. package/packages/record/lib/capture-context.mjs +50 -0
  44. package/packages/record/lib/capture-default-roots.mjs +30 -0
  45. package/packages/record/lib/capture-discovery.mjs +114 -0
  46. package/packages/record/lib/capture-failures.mjs +45 -0
  47. package/packages/record/lib/capture-health-doctor.mjs +13 -0
  48. package/packages/record/lib/capture-health-installation.mjs +117 -0
  49. package/packages/record/lib/capture-health.mjs +116 -0
  50. package/packages/record/lib/capture-hook-ownership.mjs +58 -0
  51. package/packages/record/lib/capture-hook-runner.mjs +17 -0
  52. package/packages/record/lib/capture-hook.mjs +26 -0
  53. package/packages/record/lib/capture-index-inventory.mjs +205 -0
  54. package/packages/record/lib/capture-index-outcomes.mjs +113 -0
  55. package/packages/record/lib/capture-index-reset.mjs +110 -0
  56. package/packages/record/lib/capture-index-work.mjs +77 -0
  57. package/packages/record/lib/capture-issue-preparation.mjs +73 -0
  58. package/packages/record/lib/capture-issues.mjs +15 -2
  59. package/packages/record/lib/capture-lock.mjs +137 -17
  60. package/packages/record/lib/capture-native-unit.mjs +322 -0
  61. package/packages/record/lib/capture-pass.mjs +116 -0
  62. package/packages/record/lib/capture-recovery-snapshot.mjs +182 -0
  63. package/packages/record/lib/capture-retained-scratch.mjs +213 -0
  64. package/packages/record/lib/capture-retry.mjs +102 -0
  65. package/packages/record/lib/capture-run.mjs +125 -0
  66. package/packages/record/lib/capture-scratch.mjs +12 -3
  67. package/packages/record/lib/capture-service-definition.mjs +104 -0
  68. package/packages/record/lib/capture-service-install.mjs +362 -0
  69. package/packages/record/lib/capture-service-ownership.mjs +137 -0
  70. package/packages/record/lib/capture-service-runner.mjs +18 -0
  71. package/packages/record/lib/capture-small-snapshot.mjs +85 -0
  72. package/packages/record/lib/capture-source-discovery.mjs +319 -0
  73. package/packages/record/lib/capture-source-queue.mjs +199 -0
  74. package/packages/record/lib/capture-source-work.mjs +85 -0
  75. package/packages/record/lib/capture-state.mjs +16 -71
  76. package/packages/record/lib/capture-status.mjs +171 -0
  77. package/packages/record/lib/capture-supervisor.mjs +136 -0
  78. package/packages/record/lib/capture-watch.mjs +32 -0
  79. package/packages/record/lib/capture-work-session.mjs +241 -0
  80. package/packages/record/lib/capture-worker.mjs +74 -0
  81. package/packages/record/lib/derived-state.mjs +58 -17
  82. package/packages/record/lib/file-lines.mjs +22 -6
  83. package/packages/record/lib/formats.mjs +4 -3
  84. package/packages/record/lib/index-capture.mjs +97 -0
  85. package/packages/record/lib/index-db.mjs +121 -66
  86. package/packages/record/lib/index-reset.mjs +19 -0
  87. package/packages/record/lib/store.mjs +46 -14
  88. package/packages/record/lib/transcript-prefix.mjs +125 -0
  89. package/skills/oats-getting-started/SKILL.md +1 -1
  90. package/docs/macos-test-runs.md +0 -246
@@ -0,0 +1,382 @@
1
+ # Named test runs on macOS and Linux
2
+
3
+ Pull-request CI runs the tests on Linux, in its own way: sharded, and with no
4
+ working `systemctl --user`. Some tests need something CI does not give them,
5
+ and two workflows run them by hand:
6
+
7
+ - **macOS.** Some tests rest on what only macOS does: a file system that
8
+ ignores case, `/var` and `/tmp` as links, long temporary paths, the BSD
9
+ tools. [`macos-tests.yml`](../.github/workflows/macos-tests.yml) runs named
10
+ test files of a named ref on a macOS runner.
11
+ - **Linux, with a user manager.** Some tests drive the runner user's own
12
+ systemd manager, a real `.timer` and `.service` under `systemctl --user`.
13
+ [`linux-tests.yml`](../.github/workflows/linux-tests.yml) runs named test
14
+ files of a named ref on a Linux runner. Before the run, the workflow enables
15
+ lingering for the runner user and waits for that user's manager to answer.
16
+
17
+ A person starts either one by hand; no push and no pull request ever does. The
18
+ link of a run goes on a pull request as that pull request's macOS or Linux
19
+ evidence.
20
+
21
+ Neither is a second CI. A run runs the files you name, each alone, and nothing
22
+ else: not `npm test`, not a shard.
23
+
24
+ Both workflows run one implementation,
25
+ [`scripts/named-test-run.mjs`](../scripts/named-test-run.mjs). Each runner's
26
+ own part is a small script beside it:
27
+ [`scripts/macos-tests.mjs`](../scripts/macos-tests.mjs) and
28
+ [`scripts/linux-tests.mjs`](../scripts/linux-tests.mjs). Everything this page
29
+ says of "a run" holds for both, unless a section names one runner.
30
+
31
+ ## Starting a run
32
+
33
+ You need write access to the repository. From a checkout:
34
+
35
+ ```bash
36
+ gh workflow run macos-tests.yml --ref main \
37
+ -f ref=my-branch \
38
+ -f files='test/tui-terminal.test.mjs test/fixture-isolation.test.mjs' \
39
+ -f tmux=true
40
+ gh run list --workflow=macos-tests.yml --limit 1
41
+
42
+ gh workflow run linux-tests.yml --ref main \
43
+ -f ref=my-branch \
44
+ -f files='test/some-real-manager.test.mjs'
45
+ gh run list --workflow=linux-tests.yml --limit 1
46
+ ```
47
+
48
+ Or use the repository's Actions page. Choose **macOS tests** or **Linux
49
+ tests**, then **Run workflow**. Leave the workflow's own branch at `main` (see
50
+ [What counts as evidence](#what-counts-as-evidence)).
51
+
52
+ | input | workflow | what it is |
53
+ |---|---|---|
54
+ | `ref` | both | The commit to test: a branch name, a tag name or a full commit id (40 characters) of this repository. |
55
+ | `files` | both | The test files, separated by spaces, commas or newlines. |
56
+ | `tmux` | macOS | `true` installs tmux first (`brew install tmux`). Off by default. Tests that need tmux skip without it. |
57
+
58
+ The Linux workflow has no `tmux` input. Whether tmux is on the runner's `PATH`
59
+ is printed in its header like any other fact of the runner.
60
+
61
+ ## What a run accepts
62
+
63
+ The runner's script checks the whole request before anything of the ref is
64
+ checked out, installed or run. A refusal names every entry it refuses and why,
65
+ and the run fails having run nothing.
66
+
67
+ **The ref** must be a commit of this repository: one reachable from a branch
68
+ or a tag of `origin`.
69
+
70
+ - A name is made of `A-Z a-z 0-9 . _ / -`, does not start with `-` and has no
71
+ `..`. A branch whose name has another character is given by its full commit
72
+ id. The text is taken as typed: a space before or after it is refused too.
73
+ - A shortened commit id is refused: give all 40 characters.
74
+ - **A pull request's number is not accepted.** A pull request from a fork has
75
+ a head that is not this repository's commit, and nothing is ever fetched by
76
+ what you typed. Name the branch, or give the commit id, of a pull request
77
+ whose branch is in this repository.
78
+ - A name that is both a branch and a tag is accepted when both are the same
79
+ commit. Otherwise give the commit id of the one you mean.
80
+
81
+ **Each file** must match `test/**/*.test.mjs` or
82
+ `packages/<name>/test/**/*.test.mjs`, with path segments made of
83
+ `A-Z a-z 0-9 . _ -`, none starting with a dot. It must be a regular file at
84
+ that commit, not a symbolic link. A file named twice runs once.
85
+
86
+ ## What a run does and prints
87
+
88
+ The job checks out two trees. `runner/` is the workflow's own commit, and the
89
+ script runs from it. `tested/` is the ref. Two trees are for two things: a
90
+ ref older than the workflow can be tested, and an honest branch cannot change
91
+ the runner by accident. They are not a defence against a commit that sets out
92
+ to forge its result (see [What a run does not protect](#what-a-run-does-not-protect)).
93
+
94
+ On Linux, two steps come before the validation, so that its header can report
95
+ their effect:
96
+
97
+ 1. `sudo loginctl enable-linger "$USER"` starts the runner user's systemd
98
+ manager without a login session. It is the only privileged step, and
99
+ nothing is installed system-wide.
100
+ 2. `XDG_RUNTIME_DIR=/run/user/<uid>` is set for the later steps, through
101
+ `$GITHUB_ENV`, so that `systemctl --user` finds the manager.
102
+
103
+ After the validation, the job installs as pull-request CI does: `npm ci` at
104
+ the root, then in `packages/desktop` without the Electron binary. Then it runs
105
+ each file alone, one after another, as a developer does: `node --test <file>`
106
+ from the root of the ref's tree.
107
+
108
+ The step **Validate the request and check out its commit** prints a header,
109
+ before any code of the ref has run. The header gives:
110
+
111
+ - the commit under test (full id) and the ref that was asked for;
112
+ - the workflow's own ref and commit, as the script reads them. Where a run
113
+ really ran from is GitHub's record of it, not this line: see
114
+ [What counts as evidence](#what-counts-as-evidence);
115
+ - on macOS, the macOS version; on Linux, the distribution and the kernel;
116
+ - the runner image and the architecture;
117
+ - on Linux:
118
+ - `XDG_RUNTIME_DIR`;
119
+ - whether lingering is on for the runner user (`loginctl show-user`);
120
+ - **whether the user manager answered**, with which state, how many
121
+ times it was asked and how long that took. Lingering only queues the
122
+ manager's start, so the step waits for it, at most 60 seconds in all.
123
+ Until the manager's bus is there, `systemctl --user` answers at once
124
+ (`offline` or `unknown`), so the step asks again every half second.
125
+ Once the bus is there, `systemctl --user is-system-running --wait` waits
126
+ for the manager to finish starting. `running` and `degraded` count as an
127
+ answer. `degraded` is a running manager with a failed unit, which the
128
+ image may have of its own;
129
+ - the Node version;
130
+ - whether the file system is case-sensitive, measured there (a file is made,
131
+ then looked for by another case);
132
+ - whether `tmux` is on `PATH`, and its version;
133
+ - whether a socket is at tmux's default place for the server named `oats`.
134
+
135
+ Its last line says so: `Everything above this line was printed before any code
136
+ of the ref ran.` On Linux, a user manager that did not answer ends the step
137
+ there instead. The header says what was answered, then `Refused: nothing was
138
+ installed or run.`, and the run fails before anything is installed.
139
+
140
+ The step **Run each file alone** prints the header again before the first
141
+ file. It is read after the installs, and the step says that it is the second
142
+ reading. On Linux the user manager is asked again, without waiting. If it does
143
+ not answer, no file runs and the run fails.
144
+
145
+ For each file the log has its output, then its counts (tests, passed, failed,
146
+ skipped, cancelled, todo), its exit status and its duration. One table of all
147
+ files ends the log, and the job's summary page repeats it.
148
+
149
+ **What fails a run:**
150
+
151
+ - A file fails when its exit status is not 0. A file that ends with status 0
152
+ and prints no summary, or whose summary counts a failed test, fails too. The
153
+ files after a failing one still run.
154
+ - A file that has not ended after its time is killed with its whole process
155
+ group and reported as timed out. The run goes on.
156
+ - **A tmux server left on the default socket.** A fixture that fails to
157
+ isolate its tmux server starts one at tmux's default place
158
+ (`<dir>/tmux-<uid>/oats`, where `<dir>` is `TMUX_TMPDIR`, else `/tmp`). On a
159
+ clean runner nothing fails because of it. So the script looks at that place
160
+ before the first file and after each file. The file after which a socket
161
+ first exists is marked in the table, and the run fails. Nothing is killed or
162
+ removed. If a socket is there before the first file, the header says so and
163
+ the check is skipped for that run.
164
+ - **On Linux, a user manager that does not answer**, at either step (above).
165
+
166
+ A file in which no test passed (every test skipped, for example without tmux,
167
+ or outside the workflow its guard names) is marked `no test passed`. It does
168
+ not fail the run.
169
+
170
+ ## Guard contract
171
+
172
+ A test that only means something on one of these runners guards on facts the
173
+ runner gives it. Each workflow's `name:` is part of that guard, and it is a
174
+ contract: the workflow tests pin it (`test/macos-tests-workflow.test.mjs`,
175
+ `test/linux-tests-workflow.test.mjs`). Renaming a workflow turns every test
176
+ that guards on it into a skip.
177
+
178
+ | runner | the test runs only when |
179
+ |---|---|
180
+ | macOS | `GITHUB_WORKFLOW` is `macOS tests`, `GITHUB_EVENT_NAME` is `workflow_dispatch`, and `process.platform` is `darwin` |
181
+ | Linux | `GITHUB_WORKFLOW` is `Linux tests`, `GITHUB_EVENT_NAME` is `workflow_dispatch`, and `process.platform` is `linux` |
182
+
183
+ The run passes its environment to each file as a developer's shell would. It
184
+ leaves out the request's two variables, the runner's command files
185
+ (`GITHUB_ENV`, `GITHUB_OUTPUT`, `GITHUB_PATH`, `GITHUB_STATE`,
186
+ `GITHUB_STEP_SUMMARY`) and node's nested-run marker. So `GITHUB_WORKFLOW` and
187
+ `GITHUB_EVENT_NAME` reach the test, and on Linux so does `XDG_RUNTIME_DIR`.
188
+
189
+ A guard is a switch, not a proof. Anyone can set these variables on their own
190
+ machine, and a guarded test that runs there drives that machine's real
191
+ manager. A guarded test outside its runner skips, and a run of only skips is
192
+ not evidence (below).
193
+
194
+ ## What counts as evidence
195
+
196
+ **A run is evidence for a commit whose diff has been reviewed.** A run of a
197
+ commit that nobody has read proves nothing about it: from its first install
198
+ step on, the commit's own code runs on the runner and can change what the run
199
+ prints (see [What a run does not protect](#what-a-run-does-not-protect)).
200
+
201
+ The pull request gives **the run's link**. From it, a run is evidence when all
202
+ of this holds. The first condition comes first because the others rest on it.
203
+
204
+ 1. **The workflow ran from `main`, by GitHub's own record of the run.** Not
205
+ by the log. A run started from another branch uses that branch's workflow
206
+ file and that branch's script, so every line of its log, the header
207
+ included, is text that branch wrote. Ask GitHub about the run instead:
208
+
209
+ ```bash
210
+ gh run view <run id> --json event,headSha,workflowName
211
+ git fetch origin
212
+ git merge-base --is-ancestor <headSha> origin/main; echo $?
213
+ ```
214
+
215
+ `event` must be `workflow_dispatch`. `workflowName` must be `macOS tests`
216
+ for macOS evidence, or `Linux tests` for Linux evidence. The last command
217
+ must print `0`: the workflow's commit is a commit of `main`. Anything else
218
+ is not a pass: `1` means the commit is not on `main`, and an error means
219
+ your clone does not have it. Check the commit, not the run's branch name: a
220
+ tag can carry a branch's name.
221
+ 2. The `commit under test` in the validation step's header **equals the head
222
+ under review**. A run of an earlier head is evidence for that head only.
223
+ 3. The run passed, and no file the pull request relies on is marked `no test
224
+ passed`. **A run of only skips is not evidence.**
225
+
226
+ The header also prints a `workflow ref` line. It is a convenience for
227
+ reading a run, and it is not proof of where the workflow ran from: the
228
+ workflow prints it.
229
+
230
+ With the first condition met, the log is the evidence. The job's summary page
231
+ is a convenience.
232
+
233
+ ## Bounds
234
+
235
+ These are constants of the scripts, not inputs:
236
+
237
+ | bound | macOS | Linux |
238
+ |---|---|---|
239
+ | files in one run | at most 10 files | at most 10 files |
240
+ | time for one file, then its process group is killed | 5 minutes | 20 minutes |
241
+ | time for the job | 60 minutes | 240 minutes |
242
+ | wait for the user manager | none | 60 seconds at the validation step |
243
+ | runner | `macos-14` (arm64), Node 22 | `ubuntu-24.04` (x64), Node 22 |
244
+
245
+ A Linux file gets 20 minutes because the files that qualify the record
246
+ capture at scale ([#1108](https://github.com/awebai/oats/issues/1108)) run
247
+ capture passes of up to 300 seconds, and a first upgrade may need several.
248
+ The Linux job's 240 minutes hold ten such files and the installs, under
249
+ GitHub's 6-hour limit for a job.
250
+
251
+ The workflows' steps upload no artifact and save or restore no cache: the log
252
+ and the summary are all they leave. The token can read the repository's
253
+ contents and do nothing more, and the workflows name no secret. The code a
254
+ run executes can reach more than the workflow's own steps do: see
255
+ [What a run does not protect](#what-a-run-does-not-protect).
256
+
257
+ ## What a run does not show
258
+
259
+ - **A clean machine, not yours.** The runner is a fresh image for each run.
260
+ It has no person's `PATH`, shell profile or login environment, and nothing
261
+ of an earlier run.
262
+ - **No real terminal.** No terminal emulator is attached to the tests.
263
+ - **One version and one architecture**: the image's, printed in the header.
264
+ - **A killed file's leftovers.** The script kills the process group of a file
265
+ that timed out. A tmux server has left that group by design, so a tmux
266
+ server of a killed file stays, and it is that test's to explain. On Linux,
267
+ a unit a test started under the user manager is not in the file's process
268
+ group either: it is the test's to stop.
269
+ - **Nothing about `main`.** A run started from `main` is listed among the
270
+ runs of main's head commit (`gh run list --commit <sha>`). It is also among
271
+ that commit's check runs, as `Named test files on macOS` or
272
+ `Named test files on Linux`. Its result is about the ref under test. It
273
+ says nothing about `main`, a failed run included. The commit's status
274
+ rollup does not hold these runs. A hand-started run can be the newest run
275
+ on `main`. So whoever checks `main` after a merge lists the runs by
276
+ workflow (`gh run list --workflow CI --branch main`) or reads the commit's
277
+ status rollup.
278
+
279
+ **What a Linux run does not show: a person's machine.** The user manager is
280
+ real systemd, but it runs on a hosted VM, not on a person's computer:
281
+
282
+ - The runner user has no login session. No PAM session was opened, no
283
+ graphical session exists, and no seat is assigned. Lingering, which the
284
+ workflow switched on, is what starts the manager. On a person's machine,
285
+ logging in starts it, and lingering is usually off: there, a user's timers
286
+ stop when the person logs out.
287
+ - The manager has none of a person's own units, environment
288
+ (`systemctl --user show-environment`) or `~/.config/systemd/user`. It has
289
+ only what the image and the test put there.
290
+ - It is one distribution, one systemd version and one image: Ubuntu 24.04,
291
+ as printed in the header. Another distribution's systemd or defaults are
292
+ not covered.
293
+ - Time is the VM's. A run shows what a timer does within the file's 20
294
+ minutes. It does not show a timer's behaviour across a suspend, a reboot or
295
+ days.
296
+
297
+ So a Linux run is evidence that the code drives `systemctl --user` correctly
298
+ on a working user manager. It is not evidence of how the code behaves on a
299
+ person's machine, which still needs a person's run there.
300
+
301
+ A pull request that asks for a person's run on a Mac still gets one.
302
+
303
+ ## What a run does not protect
304
+
305
+ After the validation, the ref's own code runs on the runner as the runner's
306
+ user: its install scripts, then its tests. Pull-request CI does the same for
307
+ every branch.
308
+
309
+ **Protected, in a run whose workflow commit is `main`'s: the validation and
310
+ the header.** They are printed from the workflow's own commit, before any
311
+ code of the ref has run. A commit under test cannot change which commit the
312
+ header names, or get a refused request past it. In a run started from another
313
+ branch nothing is protected: the workflow and the script are that branch's
314
+ (see [What counts as evidence](#what-counts-as-evidence)).
315
+
316
+ **Not protected: everything after the first install step**, the per-file
317
+ results and the table included. From there on, the ref's code can change
318
+ anything the runner's user can, and so what the later steps print. The result
319
+ is as honest as the commit. On Linux that includes the user manager: the
320
+ ref's code can start, stop or change its units.
321
+
322
+ **Where the check is.** A forgery has to be in the commit's own tree, its lock
323
+ file included. The evidence names that exact commit, and the reviewer of the
324
+ pull request reads its diff. Pull-request CI has the same property.
325
+
326
+ **The cache scope of the default branch.** A run started from the default
327
+ branch belongs to it, and executes another branch's code. Such code can reach
328
+ that run's Actions cache scope, which every branch reads. Only somebody with
329
+ write access can do it, by pushing the branch and starting the run. In the
330
+ maintainers' words: a run of this workflow can write the default branch's
331
+ cache scope; nothing reads it; a workflow that starts to restore a cache
332
+ reopens this. No workflow of this repository restores an Actions cache, and
333
+ the repository's tests keep it so.
334
+
335
+ **The job's token.** It is within reach of the code a run executes. It can
336
+ read this repository's contents and nothing else, and that is why neither
337
+ workflow's `permissions` ever grows.
338
+
339
+ **Root on the Linux runner.** The hosted Linux runner's user can use `sudo`
340
+ without a password: that is how the workflow enables lingering. The ref's
341
+ code can use it too. The VM is discarded after the run, and the token's
342
+ reach does not grow with root.
343
+
344
+ ## What the first Linux run from `main` proves
345
+
346
+ The Linux workflow can only be proven by a run of it from `main`. Its first
347
+ run, of an existing test file and a short check that `systemctl --user`
348
+ works, shows three things:
349
+
350
+ - the workflow starts and validates as the macOS one does;
351
+ - lingering and `XDG_RUNTIME_DIR` give that image a user manager that answers
352
+ within the bound. The header states the answer, how many times the manager
353
+ was asked and how long it took: an answer at the first ask, well under the
354
+ bound, shows that image did not need the wait;
355
+ - the installs and a per-file run work on that image.
356
+
357
+ It does not show what a guarded test sees, or does. The run's environment has
358
+ the facts the [Guard contract](#guard-contract) names, but no test file of
359
+ this repository guards on them yet. A guarded file's behaviour is shown when
360
+ such a file is dispatched, by its own run. The first run also does not show
361
+ that a later image still answers: every run's header says so again.
362
+
363
+ ## The terms
364
+
365
+ The maintainers agreed these terms, the same for both workflows.
366
+ `test/macos-tests-workflow.test.mjs` and `test/linux-tests-workflow.test.mjs`
367
+ read the workflow files and pin each one, through the checks they share in
368
+ `test/helpers/named-tests-workflow.mjs`:
369
+
370
+ - `workflow_dispatch` is the only trigger;
371
+ - `permissions` is exactly `contents: read`; no secret is named; no artifact
372
+ is uploaded; no step saves or restores a cache;
373
+ - no input is written into a shell line: the ref and the file list reach the
374
+ script as environment variables;
375
+ - the checkout is the workflow's own commit, without its token left in the
376
+ checkout's configuration;
377
+ - the job has a timeout, and its concurrency group is its own and one per
378
+ run, so no later run cancels an earlier one;
379
+ - each workflow's `name:` is the one in [Guard contract](#guard-contract).
380
+
381
+ Change a bound or a term with the maintainers' decision, and change this page
382
+ and those tests in the same commit.
@@ -9,7 +9,7 @@ or workspace membership alone does not make a package official.
9
9
 
10
10
  | package | release | capabilities | package souls |
11
11
  |---|---|---|---|
12
- | `oats.framework` | `oats-framework/v1.13.0` (this repository) | `oats.core`, `oats.setup`, `oats.support`, `oats.knowledge-theory` | `knowledge-theory-expert` |
12
+ | `oats.framework` | `oats-framework/v1.13.2` (this repository) | `oats.core`, `oats.setup`, `oats.support`, `oats.knowledge-theory` | `knowledge-theory-expert` |
13
13
  | `oats.okf` | `v5.2.1` | `oats.okf` (knowledge), `oats.okf-harvest`, `oats.okf-maintenance` | `knowledge-harvester`, `knowledge-maintainer` |
14
14
  | `oats.aweb` | `v1.29.0` | `oats.aweb` (messaging) | |
15
15
  | `oats.engineering` | `v1.10.0` | `oats.engineering-expert`, `oats.developer`, `oats.code-review`, `oats.maintainer` | `code-reviewer` |
package/docs/packages.md CHANGED
@@ -51,7 +51,7 @@ packages:
51
51
  - **Bare version** (`v5.2.1`, `5.2.1`, `1.0.0-rc.1`): the id is looked up in
52
52
  the official catalog — `package-catalog.json` in the `oats` repo, or the file
53
53
  named by `OATS_PACKAGE_CATALOG` — which supplies the repo url, the tag
54
- convention (`v5.2.1` or `oats-framework/v1.13.0`) and the payload path. An id
54
+ convention (`v5.2.1` or `oats-framework/v1.13.2`) and the payload path. An id
55
55
  the catalog does not know is `E_PACKAGE_MISSING` ("use `git:<repo>@<ref>` for
56
56
  a package outside the catalog"). The catalog is the reviewed official list
57
57
  ([official-catalog.md](official-catalog.md)) and the only way a
@@ -74,7 +74,7 @@ members:
74
74
  - git:github.com/acme/agents
75
75
  - git:github.com/acme/platform
76
76
  packages:
77
- oats.framework: v1.13.0
77
+ oats.framework: v1.13.2
78
78
  oats.okf: v5.2.1
79
79
  oats.aweb: v1.29.0
80
80
  teams:
@@ -353,11 +353,11 @@ A soul that names one of the package's capabilities with
353
353
  "policy": "docs/official-catalog.md",
354
354
  "packages": {
355
355
  "oats.okf": { "url": "https://github.com/awebai/oats-okf.git", "ref": "v5.2.1", "path": "oats-package" },
356
- "oats.framework": { "url": "https://github.com/awebai/oats.git", "ref": "oats-framework/v1.13.0", "path": "oats-package" }
356
+ "oats.framework": { "url": "https://github.com/awebai/oats.git", "ref": "oats-framework/v1.13.2", "path": "oats-package" }
357
357
  }
358
358
  }
359
359
  ```
360
360
 
361
- `ref` carries the tag convention: a workspace's `oats.framework: v1.13.0`
362
- resolves to tag `oats-framework/v1.13.0`. Resolving through the catalog never
361
+ `ref` carries the tag convention: a workspace's `oats.framework: v1.13.2`
362
+ resolves to tag `oats-framework/v1.13.2`. Resolving through the catalog never
363
363
  advances a lock by itself: `oats sync` does, and says so.
@@ -496,3 +496,16 @@ When the diff is not empty:
496
496
  not read it: it tags its own copy of `oats-package/`.) A green CI proves
497
497
  that the tag exists, not which commit it names: the pin's reviewer
498
498
  confirms on the remote that it names the first PR's merge commit.
499
+
500
+ ## Raising the Node version
501
+
502
+ A pull request that raises the Node version the workflows run
503
+ (`node-version` in `.github/workflows/`) also carries a macOS
504
+ [named run](named-test-runs.md) of `test/recovery-link-mode.test.mjs` at its
505
+ exact head. Every case in that file runs, with none skipped, and the run's
506
+ test warnings are read. The copier keeps each symbolic link's own mode with
507
+ `fs.lchmodSync`, which only macOS has and which Node lists as deprecated. If
508
+ a Node version drops it, the copy of a link whose mode differs from the new
509
+ link's must still be refused. Never take the link's mode out of the digest,
510
+ and never follow the link or change its target's mode, to make the upgrade
511
+ pass.
@@ -0,0 +1,223 @@
1
+ # OATS 0.62.0
2
+
3
+ Unreleased
4
+
5
+ ## Upgrade
6
+
7
+ - The bounded periodic capture service ships in this release; 0.56.0 through
8
+ 0.61.0 do not include it. Upgrade in this order: install this release's
9
+ compatible Desktop first, then this CLI and the matching pi bridge, then run
10
+ this CLI's setup conversion once per user on each host, from that user's
11
+ shell, then its health and doctor checks. Do not use these flags or diagnostics as
12
+ pre-upgrade checks on 0.61.0 or older binaries, which do not have them. See
13
+ [capture service release order](../upgrading.md#capture-service-release-order).
14
+ - Run `oats setup` (standalone: `turn-record setup`) once per user on each
15
+ host, from that user's shell, with the deployment's owner and root: setup
16
+ writes that user's `~/.claude*/settings.json` and user manager. Installing the CLI
17
+ alone converts nothing: persisted hooks and manager definitions change only
18
+ when setup runs. Owned capture watchers convert to the periodic service there.
19
+ A foreign, disabled or masked definition is refused before any change, with
20
+ two printed commands that keep the owner and root: `--replace-service` to
21
+ replace only the named capture service, or `--no-service` to keep the
22
+ operator's service and skip service files and manager calls. Repair unknown
23
+ or unreadable state before retrying; replacement never waives it.
24
+ `--dry-run` writes nothing and calls no manager. Keep the backups setup
25
+ reports under `<root>/service-backups/`.
26
+ - Setup rewrites the owned `Stop`/`SessionEnd` capture hooks in every
27
+ `~/.claude*/settings.json` from `async: true`, `timeout: 120` to the hook
28
+ entry `bin/capture-hook.mjs` with `async: false`, `timeout: 10`, even where
29
+ the command already matches. 0.61.0 cannot load that entry, so after a
30
+ rollback the hook exits 1 instead of blocking Claude's Stop; the upgrade
31
+ guide gives the rollback steps. Until setup runs, the persisted hooks keep
32
+ the old fields.
33
+ - Setup rewrites each `~/.claude*/settings.json` it changes with mode `0600`.
34
+ - The capture status file keeps `schemaVersion: 1` while it gains fields, a
35
+ contract decision: 0.61.0 reads a status written by this release as invalid
36
+ (see the rollback note in the upgrade guide).
37
+ - A service refusal changes nothing. A service failure after preflight no
38
+ longer skips capture: setup still writes the hooks and runs the first pass,
39
+ then exits 1 and ends with a fixed-format
40
+ setup summary naming the hooks written, the first pass's result, the
41
+ service failure, what that failed apply had already changed (the backup
42
+ directory, units stopped or disabled, definitions written, the pending
43
+ marker), and the remedy. Setup has no `--json`; see #1330.
44
+ - A plain rerun finishes setup's own failed activation: setup records it in a
45
+ private `<root>/capture-service-pending.json` marker, removed once activation
46
+ is verified, so a timer its failed enable left disabled is retried without a
47
+ flag (#1099). A disabled unit without that marker is still the operator's
48
+ choice and needs `--replace-service` or `--no-service`. Until the rerun,
49
+ `capture --health` and `oats doctor` warn `capture-service-pending`: the
50
+ periodic schedule is not verified. Apply has no atomic rollback: a manager
51
+ failure can leave backups, stopped jobs or written definitions in place.
52
+ - After conversion, check `oats capture --health` (or
53
+ `turn-record capture --health`) against the intended root, and the capture
54
+ warnings of `oats doctor`, keeping the same `HOME` and `TURN_RECORD_ROOT`.
55
+ The health rows are `capture-status-missing`, `capture-status-invalid`,
56
+ `capture-success-stale`, `capture-pass-failed`, `capture-input-set-aside`,
57
+ `capture-setup-required` (an owned capture watcher or hook that setup has
58
+ not converted) and `capture-service-pending` (setup's activation is not
59
+ verified); `oats doctor` reports a failed health command as
60
+ `capture-health-unavailable`. Definition files, an `ok` diagnosis or no
61
+ warnings do not prove the schedule is registered and enabled. A missing recent success can reflect sleep or an
62
+ off-host machine; progress or cleared issues alone are not success. Update
63
+ the CLI before relying on complete piped health output.
64
+
65
+ ## Added
66
+
67
+ - **Named test files can be run by hand on a Linux runner whose user manager
68
+ works** ([#1108](https://github.com/awebai/oats/issues/1108)). The workflow
69
+ `Linux tests` (`linux-tests.yml`) is the Linux counterpart of `macOS tests`.
70
+ It runs named test files of a named ref, each alone, on `ubuntu-24.04`:
71
+ at most 10 files, 20 minutes a file, 240 minutes a job. Before the
72
+ header, it enables lingering for the runner user and waits at most 60
73
+ seconds for `systemctl --user` to answer. The header says whether
74
+ the manager answered, and a manager that did not answer fails the run
75
+ before anything is installed. Its terms are the macOS workflow's:
76
+ dispatch only, a read-only token, no secret, artifact or cache, and no
77
+ input in a shell line. Both workflows run one script. The workflow's name
78
+ is a contract that guarded tests key on, together with `workflow_dispatch`
79
+ and `linux`. The macOS workflow's name, inputs and printed evidence are
80
+ unchanged ([named test runs](../named-test-runs.md)).
81
+
82
+ ## Changed
83
+
84
+ - **Capture runs as bounded supervised passes.** Ordinary capture,
85
+ `--sessions-only`, `--aw-only`, `--no-index` and `--retry-set-aside` run one
86
+ pass through a fixed supervisor and worker under the root lock. A partial
87
+ pass exits 0 with work pending; an operational failure exits nonzero; a
88
+ missing worker receipt is reported as unknown, never as zero counts. Ordinary
89
+ lock overlap exits 0 and keeps the holder's receipt; a recovery that overlaps
90
+ is not performed and exits 1. `index: updated` is printed only for a complete
91
+ pass with indexing enabled. `--no-index` appends without opening the search
92
+ index and never reports a fresh all-stage success. Ordinary `--json` stays a
93
+ usage error and retry JSON keeps `{appended, pending}`. `--reset-cursors`,
94
+ the exact `--home`/`--file` proofs and the public synchronous APIs keep their
95
+ contracts.
96
+ - A pass resumes across worker replacement: source discovery, the index
97
+ inventory, index reset (separately admitted, atomic), retained recovery
98
+ cleanup and run accounting all commit in bounded pages. Interrupted work keeps
99
+ its retry evidence and backs off 30, 60, 120, then 240 minutes while healthy
100
+ work continues. Success is recorded only after an all-stage audit of sources,
101
+ index, admissions, retries and cleanup.
102
+ - A raw source record over 8 MiB is never allocated whole or captured. The
103
+ complete records before it are captured, and the input is set aside at that
104
+ record's start offset. Records after it stay pending until the operator edits
105
+ the source or adds the whole input to `<root>/ignore`; `--retry-set-aside`
106
+ retries the input, but the oversized record still blocks it, in every case.
107
+ The limit applies only beyond what is already captured: native recovery
108
+ (after cache loss, a reset, a retry or a rewritten source) proves captured
109
+ records of any size by streamed per-line digests, so sessions that already
110
+ hold records over 8 MiB keep capturing their later growth. Known limitation:
111
+ an oversized line before a session's first timestamped line captures
112
+ nothing before it. `--health` names the edit-or-ignore remedy for these
113
+ entries.
114
+ - Foreground `--watch` keeps its two-second debounce and 15-minute fallback,
115
+ runs one pass at a time and coalesces notifications into one pending rerun.
116
+ - **Hooks launch capture detached.** `capture --hook` starts the supervisor in
117
+ its own session without the harness's pipes. Exit 0 means **capture
118
+ launched**, not initialization or completion; launch and usage errors exit 1,
119
+ never Claude's Stop-blocking 2. The harness waits only for the launcher;
120
+ no launch latency is measured. Foreign hooks, groups and unrelated fields are
121
+ kept, and `--install-hint` prints the same fields, keeping
122
+ `--sessions-only --no-index`.
123
+ - **Setup installs a periodic service.** On macOS,
124
+ `ai.aweb.turn-record-capture` runs at load and every 900 seconds
125
+ (`StartInterval`), with no `KeepAlive`. On Linux, a finite
126
+ `turn-record-capture.service` (`MemoryMax=2G`, `RuntimeMaxSec=300`, no
127
+ restart) is started by `turn-record-capture.timer` one minute after boot and
128
+ every 15 minutes. Each start runs one pass with all sources and indexing.
129
+ Setup reports "periodic capture scheduled (every 15 minutes)" only after
130
+ verifying registration and enablement, never a permanent PID. An unchanged,
131
+ enabled schedule is left running, preserving a pass in flight.
132
+ - Setup takes `--root`, `--replace-service`, `--no-service` and `--dry-run`, and
133
+ preflights every named service file, the manager state and the hook edits
134
+ before changing anything. `E_CAPTURE_SERVICE_FOREIGN` refuses definitions or
135
+ registrations it does not own; on macOS that includes a registration of the
136
+ capture label from another plist path, refused naming that path even with
137
+ `--replace-service`. `E_CAPTURE_SERVICE_DISABLED` states the manager state it
138
+ observed, without claiming the operator disabled it. Refusals and failures
139
+ print the user-facing command with the selected `--owner` and `--root`:
140
+ `oats setup …` when the kernel environment handed in a node (`OATS_NODE_BIN`,
141
+ also set in instance shells), otherwise `turn-record setup …`; both record
142
+ the same node.
143
+ - Setup's first pass runs through the same supervisor with `setup` attribution,
144
+ all sources and indexing, under the running Node; the hooks and service record
145
+ a stable Node. `--no-service` still runs it and `--dry-run` starts none.
146
+ Partial and overlapping passes exit 0 without claiming completion.
147
+ - **`capture --health [--json]`** distinguishes no installation (no capture
148
+ service definition, capture hook or setup pending marker: a record root
149
+ alone, as one recall or manual capture leaves it, is not one), unknown
150
+ evidence, missing or invalid status, stale or absent success, newer failures,
151
+ unfinished runs and owned capture watchers that need setup. It reads no source
152
+ content or manager state and changes nothing. A capture pass that finds the
153
+ status file unreadable moves it aside to `capture-status.json.invalid-<UTC
154
+ stamp>` and writes a new status; a foreground, setup, watch or service pass
155
+ names the moved-aside file on stderr, a hook pass leaves only the file. Reader failures exit nonzero;
156
+ warnings exit 0. Reads are bounded, with a cooperative deadline.
157
+ - **`oats doctor`** runs package capture health once, with a finite timeout and
158
+ output bound, and turns its rows into warnings. A failed, timed-out or invalid
159
+ health run is `capture-health-unavailable`. Remedies say `oats` under the
160
+ kernel and `turn-record` standalone. Doctor's exit and output contracts are
161
+ unchanged.
162
+ - `recall --thread <thread> --json` lists a thread in capture sequence. For a
163
+ thread spanning aweb accounts the cross-account sequence is unspecified, and
164
+ an ID window may contain turns from more than one account. Text thread
165
+ listings without a query stay timestamp-sorted.
166
+ - Real macOS and Linux manager lifecycle, storage and synchronization costs, and
167
+ full hosted service qualification are verified separately from the isolated
168
+ fixtures.
169
+ - **OATS Desktop accepts an `oats` CLI 0.62.x.** Its band is now
170
+ `>=0.26.0 <0.63.0`. 0.62 adds no new gate on the band: the Desktop does
171
+ not read the capture service, and `oats session attach` keeps the
172
+ arguments and exit codes the Desktop's viewer relies on.
173
+
174
+ ## Fixed
175
+
176
+ - Capture CLI reporting finishes naturally, so piped stdout and stderr drain
177
+ before exit: health samples, truncation banners, JSON receipts and error
178
+ messages are no longer cut off. Exit codes are unchanged, and health cannot
179
+ fall through to a capture.
180
+ - **A viewer from `oats session attach` no longer outlives an attach that
181
+ died without detaching** ([#1332](https://github.com/awebai/oats/issues/1332)).
182
+ A cgroup-wide `systemctl kill`, a stop timeout, oomd or a killed ssh used
183
+ to leave the `oatsview-*` session, and a keyed viewer's key table, on the
184
+ agents' tmux server. Now the viewer's client sets a `client-detached` hook
185
+ on its own session once it has attached, so the server removes the session
186
+ when the client's connection drops without a word from it (`oats` and its
187
+ client SIGKILLed together), and with it a keyed viewer's key table; a
188
+ client that ends any other way leaves the viewer to `oats`, or, if `oats`
189
+ has died, to the next attach's sweep. A
190
+ viewer session that ends another way has its table unbound by one global
191
+ hook entry, `session-closed[900]`, the only one OATS owns on the agents'
192
+ server: keep your own `session-closed` hooks at other indexes (tmux 3.2 and
193
+ later; before 3.2 the sweep removes the table). The viewer never uses tmux's
194
+ `destroy-unattached`, which crashes a tmux 3.4 server when another client
195
+ detaches the viewer ([#1336](https://github.com/awebai/oats/issues/1336)).
196
+ A global `client-detached` hook of yours does not run for a viewer's
197
+ client whose connection dropped: the viewer's own hook takes its place. Every attach also
198
+ sweeps what earlier attaches left before it opens its viewer.
199
+ It removes a viewer session that has no client and whose attach is gone,
200
+ and every viewer key table whose session is gone. When the sweep removed
201
+ something, the attach says so in one line on stderr. Exit statuses are
202
+ unchanged, a leave by the detach key's 20 included
203
+ ([execution targets](../execution-targets.md#inspect-input-and-attach)).
204
+
205
+ ## Provider updates
206
+
207
+ - **oats.framework 1.13.1, oats.setup 2.8.1: record capture setup is a
208
+ person-run step** ([#1108](https://github.com/awebai/oats/issues/1108)).
209
+ From OATS 0.62.0, `/oats-setup-model` and the setup briefing name the record
210
+ capture conversion `oats setup` as the third install the person runs from
211
+ their own shell, beside the host wake broker and the host timer, and say how
212
+ to read `turn-record capture --health` and `oats doctor`'s capture warnings.
213
+ Every new line names the release it starts in. Instruction text only; the
214
+ floors are unchanged. A deployment gets it by pinning
215
+ `oats.framework: v1.13.1` and running `oats sync`.
216
+
217
+ ## Documentation
218
+
219
+ - **`docs/macos-test-runs.md` is now
220
+ [`docs/named-test-runs.md`](../named-test-runs.md)** and covers both
221
+ runners. It adds each workflow's name as a guard contract and says what a
222
+ Linux run is not evidence of: a hosted VM's user manager with lingering is
223
+ not a person's login session.