@awebai/oats 0.61.0 → 0.62.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/bin/oats.mjs +26 -11
- package/docs/capabilities.md +1 -1
- package/docs/execution-targets.md +49 -3
- package/docs/implementation.md +2 -1
- package/docs/named-test-runs.md +382 -0
- package/docs/official-catalog.md +1 -1
- package/docs/packages.md +5 -5
- package/docs/release-notes/v0.62.0.md +223 -0
- package/docs/upgrading.md +133 -11
- package/docs/workspaces.md +1 -1
- package/lib/core.mjs +2 -2
- package/lib/doctor-capture.mjs +35 -0
- package/lib/session-viewer.mjs +86 -7
- package/package-catalog.json +1 -1
- package/package.json +1 -1
- package/packages/client/cli-locator.mjs +2 -2
- package/packages/record/README.md +87 -13
- package/packages/record/bin/capture-hook.mjs +8 -0
- package/packages/record/bin/capture.mjs +143 -124
- package/packages/record/bin/recall.mjs +8 -3
- package/packages/record/bin/setup.mjs +115 -225
- package/packages/record/bin/turn-record.mjs +2 -2
- package/packages/record/docs/capture-service.md +1443 -28
- package/packages/record/docs/capture-status.schema.json +429 -12
- package/packages/record/lib/capture-admission.mjs +82 -0
- package/packages/record/lib/capture-aw-unit.mjs +137 -0
- package/packages/record/lib/capture-aw.mjs +31 -12
- package/packages/record/lib/capture-budget.mjs +31 -0
- package/packages/record/lib/capture-cc.mjs +39 -6
- package/packages/record/lib/capture-cleanup-queue.mjs +135 -0
- package/packages/record/lib/capture-cleanup-work.mjs +77 -0
- package/packages/record/lib/capture-command-operation.mjs +37 -0
- package/packages/record/lib/capture-command-output.mjs +61 -0
- package/packages/record/lib/capture-command-worker.mjs +6 -0
- package/packages/record/lib/capture-command.mjs +48 -0
- package/packages/record/lib/capture-completion.mjs +152 -0
- package/packages/record/lib/capture-context.mjs +50 -0
- package/packages/record/lib/capture-default-roots.mjs +30 -0
- package/packages/record/lib/capture-discovery.mjs +114 -0
- package/packages/record/lib/capture-failures.mjs +45 -0
- package/packages/record/lib/capture-health-doctor.mjs +13 -0
- package/packages/record/lib/capture-health-installation.mjs +117 -0
- package/packages/record/lib/capture-health.mjs +116 -0
- package/packages/record/lib/capture-hook-ownership.mjs +58 -0
- package/packages/record/lib/capture-hook-runner.mjs +17 -0
- package/packages/record/lib/capture-hook.mjs +26 -0
- package/packages/record/lib/capture-index-inventory.mjs +205 -0
- package/packages/record/lib/capture-index-outcomes.mjs +113 -0
- package/packages/record/lib/capture-index-reset.mjs +109 -0
- package/packages/record/lib/capture-index-work.mjs +77 -0
- package/packages/record/lib/capture-issue-preparation.mjs +73 -0
- package/packages/record/lib/capture-issues.mjs +15 -2
- package/packages/record/lib/capture-lock.mjs +102 -16
- package/packages/record/lib/capture-native-unit.mjs +322 -0
- package/packages/record/lib/capture-pass.mjs +102 -0
- package/packages/record/lib/capture-recovery-snapshot.mjs +182 -0
- package/packages/record/lib/capture-retained-scratch.mjs +213 -0
- package/packages/record/lib/capture-retry.mjs +102 -0
- package/packages/record/lib/capture-run.mjs +125 -0
- package/packages/record/lib/capture-scratch.mjs +12 -3
- package/packages/record/lib/capture-service-definition.mjs +104 -0
- package/packages/record/lib/capture-service-install.mjs +362 -0
- package/packages/record/lib/capture-service-ownership.mjs +137 -0
- package/packages/record/lib/capture-service-runner.mjs +18 -0
- package/packages/record/lib/capture-small-snapshot.mjs +85 -0
- package/packages/record/lib/capture-source-discovery.mjs +319 -0
- package/packages/record/lib/capture-source-queue.mjs +199 -0
- package/packages/record/lib/capture-source-work.mjs +85 -0
- package/packages/record/lib/capture-state.mjs +16 -71
- package/packages/record/lib/capture-status.mjs +171 -0
- package/packages/record/lib/capture-supervisor.mjs +117 -0
- package/packages/record/lib/capture-watch.mjs +32 -0
- package/packages/record/lib/capture-work-session.mjs +238 -0
- package/packages/record/lib/capture-worker.mjs +58 -0
- package/packages/record/lib/derived-state.mjs +28 -15
- package/packages/record/lib/file-lines.mjs +22 -6
- package/packages/record/lib/formats.mjs +4 -3
- package/packages/record/lib/index-capture.mjs +97 -0
- package/packages/record/lib/index-db.mjs +3 -17
- package/packages/record/lib/index-reset.mjs +19 -0
- package/packages/record/lib/store.mjs +36 -11
- package/packages/record/lib/transcript-prefix.mjs +125 -0
- package/skills/oats-getting-started/SKILL.md +1 -1
- package/docs/macos-test-runs.md +0 -246
package/docs/macos-test-runs.md
DELETED
|
@@ -1,246 +0,0 @@
|
|
|
1
|
-
# macOS test runs
|
|
2
|
-
|
|
3
|
-
Pull-request CI runs the tests on Linux only. Some tests rest on what only
|
|
4
|
-
macOS does: a file system that ignores case, `/var` and `/tmp` as links, long
|
|
5
|
-
temporary paths, the BSD tools. The workflow
|
|
6
|
-
[`macos-tests.yml`](../.github/workflows/macos-tests.yml) runs named test files
|
|
7
|
-
of a named ref on a macOS runner. A person starts it by hand; no push and no
|
|
8
|
-
pull request ever does. The link of its run goes on a pull request as that
|
|
9
|
-
pull request's macOS evidence.
|
|
10
|
-
|
|
11
|
-
It is not a second CI. It runs the files you name, each alone, and nothing
|
|
12
|
-
else: not `npm test`, not a shard.
|
|
13
|
-
|
|
14
|
-
## Starting a run
|
|
15
|
-
|
|
16
|
-
You need write access to the repository. From a checkout:
|
|
17
|
-
|
|
18
|
-
```bash
|
|
19
|
-
gh workflow run macos-tests.yml --ref main \
|
|
20
|
-
-f ref=my-branch \
|
|
21
|
-
-f files='test/tui-terminal.test.mjs test/fixture-isolation.test.mjs' \
|
|
22
|
-
-f tmux=true
|
|
23
|
-
gh run list --workflow=macos-tests.yml --limit 1
|
|
24
|
-
```
|
|
25
|
-
|
|
26
|
-
Or on the repository's Actions page: choose **macOS tests**, then **Run
|
|
27
|
-
workflow**. Leave the workflow's own branch at `main` (see
|
|
28
|
-
[What counts as evidence](#what-counts-as-evidence)).
|
|
29
|
-
|
|
30
|
-
| input | what it is |
|
|
31
|
-
|---|---|
|
|
32
|
-
| `ref` | The commit to test: a branch name, a tag name or a full commit id (40 characters) of this repository. |
|
|
33
|
-
| `files` | The test files, separated by spaces, commas or newlines. |
|
|
34
|
-
| `tmux` | `true` installs tmux first (`brew install tmux`). Off by default. Tests that need tmux skip without it. |
|
|
35
|
-
|
|
36
|
-
## What a run accepts
|
|
37
|
-
|
|
38
|
-
The script [`scripts/macos-tests.mjs`](../scripts/macos-tests.mjs) checks the
|
|
39
|
-
whole request before anything of the ref is checked out, installed or run. A
|
|
40
|
-
refusal names every entry it refuses and why, and the run fails having run
|
|
41
|
-
nothing.
|
|
42
|
-
|
|
43
|
-
**The ref** must be a commit of this repository: one reachable from a branch
|
|
44
|
-
or a tag of `origin`.
|
|
45
|
-
|
|
46
|
-
- A name is made of `A-Z a-z 0-9 . _ / -`, does not start with `-` and has no
|
|
47
|
-
`..`. A branch whose name has another character is given by its full commit
|
|
48
|
-
id. The text is taken as typed: a space before or after it is refused too.
|
|
49
|
-
- A shortened commit id is refused: give all 40 characters.
|
|
50
|
-
- **A pull request's number is not accepted.** A pull request from a fork has
|
|
51
|
-
a head that is not this repository's commit, and nothing is ever fetched by
|
|
52
|
-
what you typed. Name the branch, or give the commit id, of a pull request
|
|
53
|
-
whose branch is in this repository.
|
|
54
|
-
- A name that is both a branch and a tag is accepted when both are the same
|
|
55
|
-
commit. Otherwise give the commit id of the one you mean.
|
|
56
|
-
|
|
57
|
-
**Each file** must match `test/**/*.test.mjs` or
|
|
58
|
-
`packages/<name>/test/**/*.test.mjs`, with path segments made of
|
|
59
|
-
`A-Z a-z 0-9 . _ -`, none starting with a dot. It must be a regular file at
|
|
60
|
-
that commit, not a symbolic link. A file named twice runs once.
|
|
61
|
-
|
|
62
|
-
## What a run does and prints
|
|
63
|
-
|
|
64
|
-
The job checks out two trees. `runner/` is the workflow's own commit, and the
|
|
65
|
-
script runs from it. `tested/` is the ref. Two trees are for two things: a
|
|
66
|
-
ref older than the workflow can be tested, and an honest branch cannot change
|
|
67
|
-
the runner by accident. They are not a defence against a commit that sets out
|
|
68
|
-
to forge its result (see [What a run does not protect](#what-a-run-does-not-protect)).
|
|
69
|
-
|
|
70
|
-
After the validation the job installs as pull-request CI does (`npm ci` at
|
|
71
|
-
the root, then in `packages/desktop` without the Electron binary). Then it
|
|
72
|
-
runs each file alone, one after another, as a developer does: `node --test
|
|
73
|
-
<file>` from the root of the ref's tree.
|
|
74
|
-
|
|
75
|
-
The step **Validate the request and check out its commit** prints a header,
|
|
76
|
-
before any code of the ref has run:
|
|
77
|
-
|
|
78
|
-
- the commit under test (full id) and the ref that was asked for;
|
|
79
|
-
- the workflow's own ref and commit, as the script reads them (where a run
|
|
80
|
-
really ran from is GitHub's record of it, not this line: see
|
|
81
|
-
[What counts as evidence](#what-counts-as-evidence));
|
|
82
|
-
- the macOS version, the runner image and the architecture;
|
|
83
|
-
- the Node version;
|
|
84
|
-
- whether the file system is case-sensitive, measured there (a file is made,
|
|
85
|
-
then looked for by another case);
|
|
86
|
-
- whether `tmux` is on `PATH`, and its version;
|
|
87
|
-
- whether a socket is at tmux's default place for the server named `oats`.
|
|
88
|
-
|
|
89
|
-
Its last line says so: `Everything above this line was printed before any code
|
|
90
|
-
of the ref ran.` The step **Run each file alone** prints the header again
|
|
91
|
-
before the first file, read after the installs, and says that it is the second
|
|
92
|
-
reading.
|
|
93
|
-
|
|
94
|
-
For each file the log has its output, then its counts (tests, passed, failed,
|
|
95
|
-
skipped, cancelled, todo), its exit status and its duration. One table of all
|
|
96
|
-
files ends the log, and the job's summary page repeats it.
|
|
97
|
-
|
|
98
|
-
**What fails a run:**
|
|
99
|
-
|
|
100
|
-
- A file fails when its exit status is not 0. A file that ends with status 0
|
|
101
|
-
and prints no summary, or whose summary counts a failed test, fails too. The
|
|
102
|
-
files after a failing one still run.
|
|
103
|
-
- A file that has not ended after its time is killed with its whole process
|
|
104
|
-
group and reported as timed out. The run goes on.
|
|
105
|
-
- **A tmux server left on the default socket.** A fixture that fails to
|
|
106
|
-
isolate its tmux server starts one at tmux's default place
|
|
107
|
-
(`<dir>/tmux-<uid>/oats`, where `<dir>` is `TMUX_TMPDIR`, else `/tmp`). On a
|
|
108
|
-
clean runner nothing fails because of it. So the script looks at that place
|
|
109
|
-
before the first file and after each file. The file after which a socket
|
|
110
|
-
first exists is marked in the table, and the run fails. Nothing is killed or
|
|
111
|
-
removed. If a socket is there before the first file, the header says so and
|
|
112
|
-
the check is skipped for that run.
|
|
113
|
-
|
|
114
|
-
A file in which no test passed (every test skipped, for example without tmux)
|
|
115
|
-
is marked `no test passed`. It does not fail the run.
|
|
116
|
-
|
|
117
|
-
## What counts as evidence
|
|
118
|
-
|
|
119
|
-
**A run is evidence for a commit whose diff has been reviewed.** A run of a
|
|
120
|
-
commit that nobody has read proves nothing about it: from its first install
|
|
121
|
-
step on, the commit's own code runs on the runner and can change what the run
|
|
122
|
-
prints (see [What a run does not protect](#what-a-run-does-not-protect)).
|
|
123
|
-
|
|
124
|
-
The pull request gives **the run's link**. From it, a run is macOS evidence
|
|
125
|
-
when all of this holds. The first condition comes first because the others
|
|
126
|
-
rest on it.
|
|
127
|
-
|
|
128
|
-
1. **The workflow ran from `main`, by GitHub's own record of the run.** Not
|
|
129
|
-
by the log. A run started from another branch uses that branch's workflow
|
|
130
|
-
file and that branch's script, so every line of its log, the header
|
|
131
|
-
included, is text that branch wrote. Ask GitHub about the run instead:
|
|
132
|
-
|
|
133
|
-
```bash
|
|
134
|
-
gh run view <run id> --json event,headSha,workflowName
|
|
135
|
-
git fetch origin
|
|
136
|
-
git merge-base --is-ancestor <headSha> origin/main; echo $?
|
|
137
|
-
```
|
|
138
|
-
|
|
139
|
-
`event` must be `workflow_dispatch` and `workflowName` must be `macOS
|
|
140
|
-
tests`. The last command must print `0`: the workflow's commit is a commit
|
|
141
|
-
of `main`. Anything else is not a pass: `1` means the commit is not on
|
|
142
|
-
`main`, and an error means your clone does not have it. Check the commit,
|
|
143
|
-
not the run's branch name: a tag can carry a branch's name.
|
|
144
|
-
2. The `commit under test` in the validation step's header **equals the head
|
|
145
|
-
under review**. A run of an earlier head is evidence for that head only.
|
|
146
|
-
3. The run passed, and no file the pull request relies on is marked `no test
|
|
147
|
-
passed`. **A run of only skips is not evidence.**
|
|
148
|
-
|
|
149
|
-
The header also prints a `workflow ref` line. It is a convenience for
|
|
150
|
-
reading a run, and it is not proof of where the workflow ran from: the
|
|
151
|
-
workflow prints it.
|
|
152
|
-
|
|
153
|
-
With the first condition met, the log is the evidence. The job's summary page
|
|
154
|
-
is a convenience.
|
|
155
|
-
|
|
156
|
-
## Bounds
|
|
157
|
-
|
|
158
|
-
These are constants of the script, not inputs:
|
|
159
|
-
|
|
160
|
-
| bound | value |
|
|
161
|
-
|---|---|
|
|
162
|
-
| files in one run | at most 10 files |
|
|
163
|
-
| time for one file | 5 minutes, then its process group is killed |
|
|
164
|
-
| time for the job | 60 minutes |
|
|
165
|
-
| runner | `macos-14` (arm64), Node 22 |
|
|
166
|
-
|
|
167
|
-
The workflow's steps upload no artifact and save or restore no cache: the log
|
|
168
|
-
and the summary are all they leave. The token can read the repository's
|
|
169
|
-
contents and do nothing more, and the workflow names no secret. The code a run
|
|
170
|
-
executes can reach more than the workflow's own steps do: see
|
|
171
|
-
[What a run does not protect](#what-a-run-does-not-protect).
|
|
172
|
-
|
|
173
|
-
## What a run does not show
|
|
174
|
-
|
|
175
|
-
- **A clean machine, not yours.** The runner is a fresh image for each run.
|
|
176
|
-
It has no person's `PATH`, shell profile or login environment, and nothing
|
|
177
|
-
of an earlier run.
|
|
178
|
-
- **No real terminal.** No terminal emulator is attached to the tests.
|
|
179
|
-
- **One macOS version and one architecture**: the image's, printed in the
|
|
180
|
-
header.
|
|
181
|
-
- **A killed file's leftovers.** The script kills the process group of a file
|
|
182
|
-
that timed out. A tmux server has left that group by design, so a tmux
|
|
183
|
-
server of a killed file stays, and it is that test's to explain.
|
|
184
|
-
- **Nothing about `main`.** A run started from `main` is listed among the
|
|
185
|
-
runs of main's head commit (`gh run list --commit <sha>`) and among that
|
|
186
|
-
commit's check runs, as `Named test files on macOS`. Its result is about
|
|
187
|
-
the ref under test. It says nothing about `main`, a failed run included.
|
|
188
|
-
The commit's status rollup does not hold these runs. A macOS run can be the
|
|
189
|
-
newest run on `main`, so whoever checks `main` after a merge lists the runs
|
|
190
|
-
by workflow (`gh run list --workflow CI --branch main`) or reads the
|
|
191
|
-
commit's status rollup.
|
|
192
|
-
|
|
193
|
-
So a pull request that asks for a person's run on a Mac still gets one.
|
|
194
|
-
|
|
195
|
-
## What a run does not protect
|
|
196
|
-
|
|
197
|
-
After the validation, the ref's own code runs on the runner as the runner's
|
|
198
|
-
user: its install scripts, then its tests. Pull-request CI does the same for
|
|
199
|
-
every branch.
|
|
200
|
-
|
|
201
|
-
**Protected, in a run whose workflow commit is `main`'s: the validation and
|
|
202
|
-
the header.** They are printed from the workflow's own commit, before any
|
|
203
|
-
code of the ref has run. A commit under test cannot change which commit the
|
|
204
|
-
header names, or get a refused request past it. In a run started from another
|
|
205
|
-
branch nothing is protected: the workflow and the script are that branch's
|
|
206
|
-
(see [What counts as evidence](#what-counts-as-evidence)).
|
|
207
|
-
|
|
208
|
-
**Not protected: everything after the first install step**, the per-file
|
|
209
|
-
results and the table included. From there on, the ref's code can change
|
|
210
|
-
anything the runner's user can, and so what the later steps print. The result
|
|
211
|
-
is as honest as the commit.
|
|
212
|
-
|
|
213
|
-
**Where the check is.** A forgery has to be in the commit's own tree, its lock
|
|
214
|
-
file included. The evidence names that exact commit, and the reviewer of the
|
|
215
|
-
pull request reads its diff. Pull-request CI has the same property.
|
|
216
|
-
|
|
217
|
-
**The cache scope of the default branch.** A run started from the default
|
|
218
|
-
branch belongs to it, and executes another branch's code. Such code can reach
|
|
219
|
-
that run's Actions cache scope, which every branch reads. Only somebody with
|
|
220
|
-
write access can do it, by pushing the branch and starting the run. In the
|
|
221
|
-
maintainers' words: a run of this workflow can write the default branch's
|
|
222
|
-
cache scope; nothing reads it; a workflow that starts to restore a cache
|
|
223
|
-
reopens this. No workflow of this repository restores an Actions cache, and
|
|
224
|
-
the repository's tests keep it so.
|
|
225
|
-
|
|
226
|
-
**The job's token.** It is within reach of the code a run executes. It can
|
|
227
|
-
read this repository's contents and nothing else, and that is why this
|
|
228
|
-
workflow's `permissions` never grows.
|
|
229
|
-
|
|
230
|
-
## The terms
|
|
231
|
-
|
|
232
|
-
The maintainers agreed these terms. `test/macos-tests-workflow.test.mjs`
|
|
233
|
-
reads the workflow file and pins each one:
|
|
234
|
-
|
|
235
|
-
- `workflow_dispatch` is the only trigger;
|
|
236
|
-
- `permissions` is exactly `contents: read`; no secret is named; no artifact
|
|
237
|
-
is uploaded; no step saves or restores a cache;
|
|
238
|
-
- no input is written into a shell line: the ref and the file list reach the
|
|
239
|
-
script as environment variables;
|
|
240
|
-
- the checkout is the workflow's own commit, without its token left in the
|
|
241
|
-
checkout's configuration;
|
|
242
|
-
- the job has a timeout, and its concurrency group is its own and one per
|
|
243
|
-
run, so no later run cancels an earlier one.
|
|
244
|
-
|
|
245
|
-
Change a bound or a term with the maintainers' decision, and change this page
|
|
246
|
-
and that test in the same commit.
|