@awebai/oats 0.61.0 → 0.62.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. package/README.md +1 -1
  2. package/bin/oats.mjs +26 -11
  3. package/docs/capabilities.md +1 -1
  4. package/docs/execution-targets.md +49 -3
  5. package/docs/implementation.md +2 -1
  6. package/docs/named-test-runs.md +382 -0
  7. package/docs/official-catalog.md +1 -1
  8. package/docs/packages.md +5 -5
  9. package/docs/release-notes/v0.62.0.md +223 -0
  10. package/docs/upgrading.md +133 -11
  11. package/docs/workspaces.md +1 -1
  12. package/lib/core.mjs +2 -2
  13. package/lib/doctor-capture.mjs +35 -0
  14. package/lib/session-viewer.mjs +86 -7
  15. package/package-catalog.json +1 -1
  16. package/package.json +1 -1
  17. package/packages/client/cli-locator.mjs +2 -2
  18. package/packages/record/README.md +87 -13
  19. package/packages/record/bin/capture-hook.mjs +8 -0
  20. package/packages/record/bin/capture.mjs +143 -124
  21. package/packages/record/bin/recall.mjs +8 -3
  22. package/packages/record/bin/setup.mjs +115 -225
  23. package/packages/record/bin/turn-record.mjs +2 -2
  24. package/packages/record/docs/capture-service.md +1443 -28
  25. package/packages/record/docs/capture-status.schema.json +429 -12
  26. package/packages/record/lib/capture-admission.mjs +82 -0
  27. package/packages/record/lib/capture-aw-unit.mjs +137 -0
  28. package/packages/record/lib/capture-aw.mjs +31 -12
  29. package/packages/record/lib/capture-budget.mjs +31 -0
  30. package/packages/record/lib/capture-cc.mjs +39 -6
  31. package/packages/record/lib/capture-cleanup-queue.mjs +135 -0
  32. package/packages/record/lib/capture-cleanup-work.mjs +77 -0
  33. package/packages/record/lib/capture-command-operation.mjs +37 -0
  34. package/packages/record/lib/capture-command-output.mjs +61 -0
  35. package/packages/record/lib/capture-command-worker.mjs +6 -0
  36. package/packages/record/lib/capture-command.mjs +48 -0
  37. package/packages/record/lib/capture-completion.mjs +152 -0
  38. package/packages/record/lib/capture-context.mjs +50 -0
  39. package/packages/record/lib/capture-default-roots.mjs +30 -0
  40. package/packages/record/lib/capture-discovery.mjs +114 -0
  41. package/packages/record/lib/capture-failures.mjs +45 -0
  42. package/packages/record/lib/capture-health-doctor.mjs +13 -0
  43. package/packages/record/lib/capture-health-installation.mjs +117 -0
  44. package/packages/record/lib/capture-health.mjs +116 -0
  45. package/packages/record/lib/capture-hook-ownership.mjs +58 -0
  46. package/packages/record/lib/capture-hook-runner.mjs +17 -0
  47. package/packages/record/lib/capture-hook.mjs +26 -0
  48. package/packages/record/lib/capture-index-inventory.mjs +205 -0
  49. package/packages/record/lib/capture-index-outcomes.mjs +113 -0
  50. package/packages/record/lib/capture-index-reset.mjs +109 -0
  51. package/packages/record/lib/capture-index-work.mjs +77 -0
  52. package/packages/record/lib/capture-issue-preparation.mjs +73 -0
  53. package/packages/record/lib/capture-issues.mjs +15 -2
  54. package/packages/record/lib/capture-lock.mjs +102 -16
  55. package/packages/record/lib/capture-native-unit.mjs +322 -0
  56. package/packages/record/lib/capture-pass.mjs +102 -0
  57. package/packages/record/lib/capture-recovery-snapshot.mjs +182 -0
  58. package/packages/record/lib/capture-retained-scratch.mjs +213 -0
  59. package/packages/record/lib/capture-retry.mjs +102 -0
  60. package/packages/record/lib/capture-run.mjs +125 -0
  61. package/packages/record/lib/capture-scratch.mjs +12 -3
  62. package/packages/record/lib/capture-service-definition.mjs +104 -0
  63. package/packages/record/lib/capture-service-install.mjs +362 -0
  64. package/packages/record/lib/capture-service-ownership.mjs +137 -0
  65. package/packages/record/lib/capture-service-runner.mjs +18 -0
  66. package/packages/record/lib/capture-small-snapshot.mjs +85 -0
  67. package/packages/record/lib/capture-source-discovery.mjs +319 -0
  68. package/packages/record/lib/capture-source-queue.mjs +199 -0
  69. package/packages/record/lib/capture-source-work.mjs +85 -0
  70. package/packages/record/lib/capture-state.mjs +16 -71
  71. package/packages/record/lib/capture-status.mjs +171 -0
  72. package/packages/record/lib/capture-supervisor.mjs +117 -0
  73. package/packages/record/lib/capture-watch.mjs +32 -0
  74. package/packages/record/lib/capture-work-session.mjs +238 -0
  75. package/packages/record/lib/capture-worker.mjs +58 -0
  76. package/packages/record/lib/derived-state.mjs +28 -15
  77. package/packages/record/lib/file-lines.mjs +22 -6
  78. package/packages/record/lib/formats.mjs +4 -3
  79. package/packages/record/lib/index-capture.mjs +97 -0
  80. package/packages/record/lib/index-db.mjs +3 -17
  81. package/packages/record/lib/index-reset.mjs +19 -0
  82. package/packages/record/lib/store.mjs +36 -11
  83. package/packages/record/lib/transcript-prefix.mjs +125 -0
  84. package/skills/oats-getting-started/SKILL.md +1 -1
  85. package/docs/macos-test-runs.md +0 -246
@@ -1,246 +0,0 @@
1
- # macOS test runs
2
-
3
- Pull-request CI runs the tests on Linux only. Some tests rest on what only
4
- macOS does: a file system that ignores case, `/var` and `/tmp` as links, long
5
- temporary paths, the BSD tools. The workflow
6
- [`macos-tests.yml`](../.github/workflows/macos-tests.yml) runs named test files
7
- of a named ref on a macOS runner. A person starts it by hand; no push and no
8
- pull request ever does. The link of its run goes on a pull request as that
9
- pull request's macOS evidence.
10
-
11
- It is not a second CI. It runs the files you name, each alone, and nothing
12
- else: not `npm test`, not a shard.
13
-
14
- ## Starting a run
15
-
16
- You need write access to the repository. From a checkout:
17
-
18
- ```bash
19
- gh workflow run macos-tests.yml --ref main \
20
- -f ref=my-branch \
21
- -f files='test/tui-terminal.test.mjs test/fixture-isolation.test.mjs' \
22
- -f tmux=true
23
- gh run list --workflow=macos-tests.yml --limit 1
24
- ```
25
-
26
- Or on the repository's Actions page: choose **macOS tests**, then **Run
27
- workflow**. Leave the workflow's own branch at `main` (see
28
- [What counts as evidence](#what-counts-as-evidence)).
29
-
30
- | input | what it is |
31
- |---|---|
32
- | `ref` | The commit to test: a branch name, a tag name or a full commit id (40 characters) of this repository. |
33
- | `files` | The test files, separated by spaces, commas or newlines. |
34
- | `tmux` | `true` installs tmux first (`brew install tmux`). Off by default. Tests that need tmux skip without it. |
35
-
36
- ## What a run accepts
37
-
38
- The script [`scripts/macos-tests.mjs`](../scripts/macos-tests.mjs) checks the
39
- whole request before anything of the ref is checked out, installed or run. A
40
- refusal names every entry it refuses and why, and the run fails having run
41
- nothing.
42
-
43
- **The ref** must be a commit of this repository: one reachable from a branch
44
- or a tag of `origin`.
45
-
46
- - A name is made of `A-Z a-z 0-9 . _ / -`, does not start with `-` and has no
47
- `..`. A branch whose name has another character is given by its full commit
48
- id. The text is taken as typed: a space before or after it is refused too.
49
- - A shortened commit id is refused: give all 40 characters.
50
- - **A pull request's number is not accepted.** A pull request from a fork has
51
- a head that is not this repository's commit, and nothing is ever fetched by
52
- what you typed. Name the branch, or give the commit id, of a pull request
53
- whose branch is in this repository.
54
- - A name that is both a branch and a tag is accepted when both are the same
55
- commit. Otherwise give the commit id of the one you mean.
56
-
57
- **Each file** must match `test/**/*.test.mjs` or
58
- `packages/<name>/test/**/*.test.mjs`, with path segments made of
59
- `A-Z a-z 0-9 . _ -`, none starting with a dot. It must be a regular file at
60
- that commit, not a symbolic link. A file named twice runs once.
61
-
62
- ## What a run does and prints
63
-
64
- The job checks out two trees. `runner/` is the workflow's own commit, and the
65
- script runs from it. `tested/` is the ref. Two trees are for two things: a
66
- ref older than the workflow can be tested, and an honest branch cannot change
67
- the runner by accident. They are not a defence against a commit that sets out
68
- to forge its result (see [What a run does not protect](#what-a-run-does-not-protect)).
69
-
70
- After the validation the job installs as pull-request CI does (`npm ci` at
71
- the root, then in `packages/desktop` without the Electron binary). Then it
72
- runs each file alone, one after another, as a developer does: `node --test
73
- <file>` from the root of the ref's tree.
74
-
75
- The step **Validate the request and check out its commit** prints a header,
76
- before any code of the ref has run:
77
-
78
- - the commit under test (full id) and the ref that was asked for;
79
- - the workflow's own ref and commit, as the script reads them (where a run
80
- really ran from is GitHub's record of it, not this line: see
81
- [What counts as evidence](#what-counts-as-evidence));
82
- - the macOS version, the runner image and the architecture;
83
- - the Node version;
84
- - whether the file system is case-sensitive, measured there (a file is made,
85
- then looked for by another case);
86
- - whether `tmux` is on `PATH`, and its version;
87
- - whether a socket is at tmux's default place for the server named `oats`.
88
-
89
- Its last line says so: `Everything above this line was printed before any code
90
- of the ref ran.` The step **Run each file alone** prints the header again
91
- before the first file, read after the installs, and says that it is the second
92
- reading.
93
-
94
- For each file the log has its output, then its counts (tests, passed, failed,
95
- skipped, cancelled, todo), its exit status and its duration. One table of all
96
- files ends the log, and the job's summary page repeats it.
97
-
98
- **What fails a run:**
99
-
100
- - A file fails when its exit status is not 0. A file that ends with status 0
101
- and prints no summary, or whose summary counts a failed test, fails too. The
102
- files after a failing one still run.
103
- - A file that has not ended after its time is killed with its whole process
104
- group and reported as timed out. The run goes on.
105
- - **A tmux server left on the default socket.** A fixture that fails to
106
- isolate its tmux server starts one at tmux's default place
107
- (`<dir>/tmux-<uid>/oats`, where `<dir>` is `TMUX_TMPDIR`, else `/tmp`). On a
108
- clean runner nothing fails because of it. So the script looks at that place
109
- before the first file and after each file. The file after which a socket
110
- first exists is marked in the table, and the run fails. Nothing is killed or
111
- removed. If a socket is there before the first file, the header says so and
112
- the check is skipped for that run.
113
-
114
- A file in which no test passed (every test skipped, for example without tmux)
115
- is marked `no test passed`. It does not fail the run.
116
-
117
- ## What counts as evidence
118
-
119
- **A run is evidence for a commit whose diff has been reviewed.** A run of a
120
- commit that nobody has read proves nothing about it: from its first install
121
- step on, the commit's own code runs on the runner and can change what the run
122
- prints (see [What a run does not protect](#what-a-run-does-not-protect)).
123
-
124
- The pull request gives **the run's link**. From it, a run is macOS evidence
125
- when all of this holds. The first condition comes first because the others
126
- rest on it.
127
-
128
- 1. **The workflow ran from `main`, by GitHub's own record of the run.** Not
129
- by the log. A run started from another branch uses that branch's workflow
130
- file and that branch's script, so every line of its log, the header
131
- included, is text that branch wrote. Ask GitHub about the run instead:
132
-
133
- ```bash
134
- gh run view <run id> --json event,headSha,workflowName
135
- git fetch origin
136
- git merge-base --is-ancestor <headSha> origin/main; echo $?
137
- ```
138
-
139
- `event` must be `workflow_dispatch` and `workflowName` must be `macOS
140
- tests`. The last command must print `0`: the workflow's commit is a commit
141
- of `main`. Anything else is not a pass: `1` means the commit is not on
142
- `main`, and an error means your clone does not have it. Check the commit,
143
- not the run's branch name: a tag can carry a branch's name.
144
- 2. The `commit under test` in the validation step's header **equals the head
145
- under review**. A run of an earlier head is evidence for that head only.
146
- 3. The run passed, and no file the pull request relies on is marked `no test
147
- passed`. **A run of only skips is not evidence.**
148
-
149
- The header also prints a `workflow ref` line. It is a convenience for
150
- reading a run, and it is not proof of where the workflow ran from: the
151
- workflow prints it.
152
-
153
- With the first condition met, the log is the evidence. The job's summary page
154
- is a convenience.
155
-
156
- ## Bounds
157
-
158
- These are constants of the script, not inputs:
159
-
160
- | bound | value |
161
- |---|---|
162
- | files in one run | at most 10 files |
163
- | time for one file | 5 minutes, then its process group is killed |
164
- | time for the job | 60 minutes |
165
- | runner | `macos-14` (arm64), Node 22 |
166
-
167
- The workflow's steps upload no artifact and save or restore no cache: the log
168
- and the summary are all they leave. The token can read the repository's
169
- contents and do nothing more, and the workflow names no secret. The code a run
170
- executes can reach more than the workflow's own steps do: see
171
- [What a run does not protect](#what-a-run-does-not-protect).
172
-
173
- ## What a run does not show
174
-
175
- - **A clean machine, not yours.** The runner is a fresh image for each run.
176
- It has no person's `PATH`, shell profile or login environment, and nothing
177
- of an earlier run.
178
- - **No real terminal.** No terminal emulator is attached to the tests.
179
- - **One macOS version and one architecture**: the image's, printed in the
180
- header.
181
- - **A killed file's leftovers.** The script kills the process group of a file
182
- that timed out. A tmux server has left that group by design, so a tmux
183
- server of a killed file stays, and it is that test's to explain.
184
- - **Nothing about `main`.** A run started from `main` is listed among the
185
- runs of main's head commit (`gh run list --commit <sha>`) and among that
186
- commit's check runs, as `Named test files on macOS`. Its result is about
187
- the ref under test. It says nothing about `main`, a failed run included.
188
- The commit's status rollup does not hold these runs. A macOS run can be the
189
- newest run on `main`, so whoever checks `main` after a merge lists the runs
190
- by workflow (`gh run list --workflow CI --branch main`) or reads the
191
- commit's status rollup.
192
-
193
- So a pull request that asks for a person's run on a Mac still gets one.
194
-
195
- ## What a run does not protect
196
-
197
- After the validation, the ref's own code runs on the runner as the runner's
198
- user: its install scripts, then its tests. Pull-request CI does the same for
199
- every branch.
200
-
201
- **Protected, in a run whose workflow commit is `main`'s: the validation and
202
- the header.** They are printed from the workflow's own commit, before any
203
- code of the ref has run. A commit under test cannot change which commit the
204
- header names, or get a refused request past it. In a run started from another
205
- branch nothing is protected: the workflow and the script are that branch's
206
- (see [What counts as evidence](#what-counts-as-evidence)).
207
-
208
- **Not protected: everything after the first install step**, the per-file
209
- results and the table included. From there on, the ref's code can change
210
- anything the runner's user can, and so what the later steps print. The result
211
- is as honest as the commit.
212
-
213
- **Where the check is.** A forgery has to be in the commit's own tree, its lock
214
- file included. The evidence names that exact commit, and the reviewer of the
215
- pull request reads its diff. Pull-request CI has the same property.
216
-
217
- **The cache scope of the default branch.** A run started from the default
218
- branch belongs to it, and executes another branch's code. Such code can reach
219
- that run's Actions cache scope, which every branch reads. Only somebody with
220
- write access can do it, by pushing the branch and starting the run. In the
221
- maintainers' words: a run of this workflow can write the default branch's
222
- cache scope; nothing reads it; a workflow that starts to restore a cache
223
- reopens this. No workflow of this repository restores an Actions cache, and
224
- the repository's tests keep it so.
225
-
226
- **The job's token.** It is within reach of the code a run executes. It can
227
- read this repository's contents and nothing else, and that is why this
228
- workflow's `permissions` never grows.
229
-
230
- ## The terms
231
-
232
- The maintainers agreed these terms. `test/macos-tests-workflow.test.mjs`
233
- reads the workflow file and pins each one:
234
-
235
- - `workflow_dispatch` is the only trigger;
236
- - `permissions` is exactly `contents: read`; no secret is named; no artifact
237
- is uploaded; no step saves or restores a cache;
238
- - no input is written into a shell line: the ref and the file list reach the
239
- script as environment variables;
240
- - the checkout is the workflow's own commit, without its token left in the
241
- checkout's configuration;
242
- - the job has a timeout, and its concurrency group is its own and one per
243
- run, so no later run cancels an earlier one.
244
-
245
- Change a bound or a term with the maintainers' decision, and change this page
246
- and that test in the same commit.