@lariocpt/crust-mongo 0.0.0-stage → 0.3.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,238 @@
1
+ ---
2
+ name: crust-builtins
3
+ description: crust has two separate namespaces — tool builtins (`mock-server`, `test-fixture`, `test-pipes`, `gen-fixtures`, `verify-web-links`, `logs`, `dotenv`, `skills`) that need a line of their own, and registered functions (`base64`, `salt`, `jwt`, `bundle`, `sql`, `wait`) that are space-separated stages usable as a source or mid-pipeline. Explains what the upstream item means to each one, why `bundle(5)` is a shell syntax error, and how to reach crypto/FS from a lambda. Use when a crust stage exits 2 with `syntax error near unexpected token`, when a mid-pipeline function eats the wrong argument, or when a lambda says `salt is not defined`.
4
+ ---
5
+
6
+ # crust's two namespaces
7
+
8
+ | | What it is | Where it runs | Parens? |
9
+ | --- | --- | --- | --- |
10
+ | **Tool builtin** | `mock-server`, `test-fixture`, `test-pipes`, `gen-fixtures`, `verify-web-links`, `logs`, `dotenv`, `skills` | dispatched **before** pipeline parsing, in-process, own line | subcommand style: `logs tail -F app.log` |
11
+ | **Registered function** | `base64`, `salt`, `jwt`, `bundle`, `sql`, `wait` + anything you add with `crust.fn` | an ordinary pipeline stage, or a source when nothing is upstream | **space-separated**: `salt 16`, not `salt(16)` |
12
+
13
+ `cd`, `export`, `alias`, `source`, `exit`, `history`, `help` and the job verbs
14
+ `jobs`, `fg`, `bg` are builtins too — the ones sh has its own version of, which
15
+ is why a redirect on *those* is left to sh. `procs`, `range`, `load`, `read`, `tail` and friends are **sources**,
16
+ not builtins.
17
+
18
+ `source` is the only session builtin with its own argument grammar:
19
+ `source <file> [args…]` — a `.ts`/`.js` file is imported; a `.sh`/`.bash`/`.zsh`
20
+ file (or one whose first line is a `#!` shebang naming a shell) runs **once in
21
+ a child shell** — its exported `KEY=value` delta comes back into the session,
22
+ its exit code is the line's, its side effects are real, but unsets don't
23
+ propagate and its cwd never comes back; anything else runs line-by-line in this
24
+ session (aliases, `export`s and `capture`s persist). An unmarked file that
25
+ looks like multi-line shell, or that doesn't parse as crust, is refused,
26
+ never half-run. In the line-by-line form the args are the *sourced file's*
27
+ positional arguments (`$1`, `$2`, … — `$0` is its own path), and the caller's
28
+ positionals are restored when the file ends; the shell form's child never sees
29
+ them — its `$1…` are its own shell's, not the file's:
30
+
31
+ ```crust
32
+ # demo.sh does `export DEMO=one` and changes cwd; only the export survives
33
+ source ./demo.sh
34
+ export # → … DEMO=one … — the env delta came back, the cwd did not
35
+ # line-by-line only: the args are the file's $1… ($0 its path)
36
+ source ./more.crust one two
37
+ ```
38
+
39
+ ## Job control is a third shape
40
+
41
+ `jobs`, `fg`, `bg` are ordinary builtins. `wait %1` and `kill %1` are **not** —
42
+ registering either name wholesale would take a word sh (or crust) already owns,
43
+ so only the `%job` operand form is intercepted and every other line keeps its
44
+ owner:
45
+
46
+ ```crust
47
+ wait :3000/health --timeout 5s # the registered `wait` function — readiness
48
+ kill -9 4242 # sh's: a PID is not a job spec
49
+ wait %1 # the job wait
50
+ kill %1 # the job signal — reports the SEND, not the death
51
+ ```
52
+
53
+ A trailing `&` on a shell line is crust's too (its own process group, stdin
54
+ `/dev/null`, outlives the session), and `jobs`/`fg`/`bg`/`wait %N` are how it
55
+ comes back. `Ctrl-Z` suspends a crust pipeline, builtin or `fg`/`wait` with
56
+ SIGSTOP; it cannot stop a foreground inherit-stdio child, because the runtime
57
+ ignores SIGTSTP and children inherit that ignore.
58
+
59
+ The single most common mistake is writing the JS form:
60
+
61
+ ```
62
+ bundle(5) → sh: -c: line 1: syntax error near unexpected token `(' (exit 2)
63
+ ```
64
+
65
+ Parens are crust syntax only for **lexer sources** (`range(0,9)`,
66
+ `read("…")`) — everything else in a stage is a shell word list.
67
+
68
+ A stage that *starts* with `{` or `[` is a JSON literal when it parses, or when
69
+ tight text still looks like someone typing one — a quote, colon, comma or a `$`
70
+ inside the brackets (`{"n":$N}`, `[1,$N]`). The shell's own bracket shapes are
71
+ always spaced, so `[ … ]` and `{ …; }` are shell stages, in any position — and a
72
+ mid-pipeline one works too, which it used not to:
73
+
74
+ ```crust
75
+ { echo a; echo b; } | cat
76
+ [ -f package.json ] && echo there
77
+ ```
78
+
79
+ ## What the item means, per function
80
+
81
+ A registered function is called `fn(item, ...lineArgs)` mid-pipeline. So the
82
+ first slot is **your data**, and each function decides what to do with it:
83
+
84
+ | Function | As a source | Mid-pipeline |
85
+ | --- | --- | --- |
86
+ | `base64 [-d\|decode] [-f\|--file <path>] [-o\|--out <path>]` | encodes the literal argument, or a file's bytes with `--file` | encodes (or decodes) **each item**; the item is data, never an option |
87
+ | `salt [bytes] [hex\|base64\|base64url]` | one salt, size = the argument | **one salt per item**; the size is the argument *trailing* the item |
88
+ | `jwt sign\|verify\|decode --secret <s>` | payload/token = the argument | the item **is** the payload (sign) or token (verify/decode) |
89
+ | `sql "<query>" [params…]` | streams rows | the query stays the one you wrote; the item **binds as the first parameter** when the line declares none |
90
+ | `mongo "<expr>"` | streams documents — an expression on `db` (`db.users.find({}).limit(10)`), cursor drained per item | the item binds as `item` in the expression — `… \| mongo "db.users.insertMany(item)"`; needs the mongo-bundled binary (`@lariocpt/crust-mongo`) |
91
+ | `wait <target> [--timeout <dur>]` | waits for the argument | the item is the target |
92
+ | `bundle <entry> [--outfile…]` | bundles the argument | a string item **is an entrypoint** |
93
+
94
+ ```crust
95
+ base64 hello
96
+ echo aGk= | base64 -d
97
+ salt 32 base64
98
+ lines users.csv | salt 16
99
+ jwt sign '{"sub":"42"}' --secret k
100
+ sql "select id, email from users limit 5" | (r => r.email)
101
+ range(2,2) | (n => "beta") | sql "SELECT name FROM t WHERE name = ?" | assert (r => r.name === "beta")
102
+ mongo "db.users.find({active: true}).limit(10)"
103
+ read docs.json | (s => JSON.parse(s)) | mongo "db.users.insertMany(item)"
104
+ wait :3001/health --timeout 30s
105
+ bundle src/index.ts --outfile dist/app.js --minify
106
+ ```
107
+
108
+ Options come from the line, never from the item: `printf '%s' -d | base64` is
109
+ `LWQ=`, the two characters encoded, not a mode flip. A dash that is not one of
110
+ that function's options is refused rather than encoded (`base64 --desc hello`
111
+ used to print `LS1kZXNj` and exit 0), and decoding refuses a character outside
112
+ the alphabet — Node's decoder skips unknown characters, which printed three
113
+ replacement characters at exit 0 for `not base64!!!`.
114
+
115
+ Each of the six also answers `-h`/`--help` with its own usage screen and exit
116
+ **0** — at parse time, **before any stage runs**, so `… | sql --help` prints
117
+ usage without making the request (the `touch` upstream of it never happens).
118
+ The scan stops at `--`, so `base64 -- --help` encodes the flag, and a value
119
+ that arrives through the pipe is data, never an option
120
+ (`printf '%s' --help | base64` encodes it). A `crust.fn` override of the name
121
+ keeps its own argument handling — `--help` is just another argument to it.
122
+
123
+ ```crust
124
+ base64 --help
125
+ ```
126
+
127
+ `wait` failing to become ready exits **1** — measured for every target form
128
+ (`:PORT`, `:PORT/path`, `port:PORT`) and for a `--timeout` that expires. crust
129
+ has no exit **124**: that number is `timeout(1)` killing crust from outside, so
130
+ wrapping a probe in `timeout` steals the code you were gating on.
131
+
132
+ ## A lambda cannot call them
133
+
134
+ Registered functions live in crust's function table, not in the JS scope a
135
+ lambda compiles into:
136
+
137
+ ```
138
+ range(2,2) | (x => salt(4)) → crust: salt is not defined (exit 1)
139
+ ```
140
+
141
+ That is not a dead end: a lambda body has Bun's own globals, and `await` works
142
+ in it, so reach the underlying capability directly.
143
+
144
+ ```crust
145
+ range(0,9) | (_ => crypto.getRandomValues(new Uint8Array(8)).toHex())
146
+ range(0,0) | (_ => JSON.parse(await Bun.file("package.json").text()).name)
147
+ ```
148
+
149
+ Prefer the registered function when it fits (`salt 16` per item is shorter than
150
+ the `crypto` line); drop to a lambda when you need something crust doesn't ship.
151
+
152
+ ## Session environment
153
+
154
+ A line of pure assignments *is* the session environment — it runs the same code
155
+ `export` runs, so the two spellings cannot disagree:
156
+
157
+ ```crust
158
+ FOO=bar
159
+ PATH=$PATH:/tmp/bin
160
+ CFLAGS+=$EXTRA_FLAGS
161
+ TOKEN='$literal'
162
+ ```
163
+
164
+ `FOO=bar` used to be `sh -c 'FOO=bar'` — exit **0**, value gone, and the next
165
+ line expanded `$FOO` to nothing. Quoting is sh's: `'…'` protects `$`, `"…"`
166
+ expands it, and a backslash protects exactly one character, so `FOO=C:\temp`
167
+ survives.
168
+
169
+ Two shapes are deliberately **not** session state:
170
+
171
+ - `FOO=bar cmd args` is sh's *transient* prefix — the variable lives for that
172
+ one command — so the line goes to sh untouched and later lines see nothing.
173
+ - `FOO=$(cmd)` is refused rather than run: its value would have to cross back
174
+ out of a child process. The honest route is a pipeline — `git rev-parse HEAD`
175
+ piped into a `capture` stage of that name. Quoting doesn't change that, and
176
+ neither does a pipe inside the substitution (`FOO=$(date | head -1)` is
177
+ refused too, not quietly dropped); a `$()` in a *shell* stage stays
178
+ sh's, so `echo "$(date)"` still prints the date — but a `$()` in a
179
+ *crust* stage is refused before anything runs, and the named route is
180
+ capture: `date +%s | capture TS`, then `GET :3000/?t=$TS`.
181
+
182
+ ## These are not shell commands
183
+
184
+ `base64`, `sort`, `head`, `wc` and friends exist as **real binaries** too. crust
185
+ runs its own when the name is registered (`base64 hello` encodes the *text*, not
186
+ a file), and sh's when it is not (`… | sort` is `/usr/bin/sort`). Neither one
187
+ takes a redirect: crust refuses it and tells you what to write instead — with
188
+ two different messages, because a function and a tool fail differently.
189
+
190
+ `base64` is the shadow that bites: `/usr/bin/base64 in.ts` encodes the FILE,
191
+ crust's encodes the five characters of the path (`aW4udHM=`) and now says so on
192
+ stderr. A file's bytes are `base64 --file logo.png` — crust's only binary-safe
193
+ route, because a pipeline decodes UTF-8 on the way (`read logo.png | base64`
194
+ gives `77+9…` for a PNG whose real answer is `iVBORw0…`) — and `--out <path>` is
195
+ how the answer reaches a file, since a function stage cannot be redirected.
196
+
197
+ A **registered function** with a redirect is refused as a stage, and pointed at
198
+ a shell tail:
199
+
200
+ ```
201
+ sql "select 1" > one.txt
202
+ → crust: `> one.txt`: crust has no redirect except on a shell stage, whose
203
+ text goes to sh as-is — a sql stage would read it as data. to save a
204
+ pipeline, end the line with a shell stage instead: | cat > one.txt
205
+ ```
206
+
207
+ The refusal also catches a stage the redirect **hides**. Stage patterns match the
208
+ whole text, so `range(1,3) > f` stopped matching as a `range` and fell through to
209
+ `sh -c`, where sh's own syntax error answered at exit 2 (and `parallel 5 > f`
210
+ ran GNU parallel and exited 0 with an empty file). crust strips the tail, sees
211
+ the stage, and refuses. Affected: `range`, `capture`, `parallel`, `expect`,
212
+ `stats`, `stdin`, and `load` — where the tail used to break load's parser first
213
+ and the complaint was about the rate plan. `grep` is the exception: its redirect
214
+ is the shell's and works, so `read **/*.log | grep ERROR > errors.txt` is a
215
+ supported way to save a filtered log.
216
+
217
+ ```crust
218
+ base64 hello | cat > out.b64
219
+ base64 --file logo.png --out logo.b64
220
+ base64 --file logo.b64 -d --out logo-copy.png
221
+ ```
222
+
223
+ A **tool builtin** is refused as a program, and the message prints the exact
224
+ whole-invocation redirect that works:
225
+
226
+ ```
227
+ verify-web-links --base-url https://example.com --json > report.json
228
+ → crust: verify-web-links is a crust builtin, it runs in-process and cannot
229
+ redirect. Redirect the whole invocation instead:
230
+ crust -c 'verify-web-links --base-url https://example.com --json' > report.json
231
+ ```
232
+
233
+ ```bash
234
+ crust -c 'verify-web-links --base-url https://example.com --json' > report.json
235
+ ```
236
+
237
+ Tools that produce a file themselves have a flag for it (`-o` / `--outfile` /
238
+ `--report`); check that before reaching for `>`.
@@ -0,0 +1,215 @@
1
+ ---
2
+ name: crust-cli
3
+ description: How to invoke crust and read its answer — the run modes (`-c`, script file, piped stdin, REPL, tool subcommands like `crust skills install`), what each exit code actually means (0/1/2/127/130, and why there is no 124), `--check` as a linter including what it cannot catch, `--env-file` vs `dotenv` vs `export`, the REPL's verified keybindings and Ctrl-C semantics, and where crust's globals (`GET`, `range`, `parallel`, `crust.fn`) do and do not exist. Use when a crust run's exit code is confusing, when a CI step passes on a failing line, when `--check` says ok but the line dies, when `GET is not defined` appears in a `.ts` file, or when driving the REPL from a script.
4
+ ---
5
+
6
+ # Running crust, and reading what it says back
7
+
8
+ ## Run modes
9
+
10
+ | Invocation | What happens |
11
+ | --- | --- |
12
+ | `crust` (stdin is a TTY) | Interactive REPL. |
13
+ | `crust` (stdin is piped) | Reads **lines of crust** from stdin to EOF and runs them like a script (fail-fast, same exit codes). stdin is the *command* source in this mode, so a shell stage inside those lines sees EOF. Want data on stdin? Use `-c`. |
14
+ | `crust run.crust [args…]` | Script mode: blank lines and `#` comments are skipped (so a `#!/usr/bin/env crust` shebang works), lines run in order, and it **fails fast** — the first failing line stops the file and becomes the process exit code. A line ending in `\` continues on the next. The extension doesn't matter. Extra arguments are the script's **positional arguments** — `$0` is the path, `$1`…`$9` the rest, `$#` the count, `$@`/`$*` all of them joined — expanding wherever an env var does (URLs, JSON literals, `-H` strings, fn args, `stats --out`), and empty, never literal, outside script mode. A leading `--` after the path is a separator, not an argument. An unreadable path is 127. |
15
+ | `crust -c '<line>'` | One line. A shell stage inherits the process stdin, so `docker logs app \| crust -c 'grep ERROR'` really does grep the logs. To feed a **crust** pipeline, name the source: `docker logs -f app \| crust -c 'stdin \| grep ERROR'`. To run several lines, put a **newline inside the one argument** — they then behave like a script (fail-fast) — or continue one line with a trailing `\`. Two lines as two arguments is refused. Quote the whole pipeline: `-c range(1,2) extra` splits into three argv words and is refused. |
16
+ | `crust <tool> [args…]` | A **tool builtin as a subcommand**: `crust skills install`, `crust mock-server openapi.yaml -p4000`, `crust test-fixture 'fixtures/*.crust.ts' -j8`. Same flags as the REPL line, and `crust <tool> --help` prints them. The names that work are the ones needing no live session: `skills`, `test-fixture`, `test-pipes`, `gen-fixtures`, `mock-server`, `verify-web-links`. The **session** builtins (`cd`, `export`, `alias`, `unalias`, `source`, `dotenv`, `exit`, `help`, `history`) are refused rather than silently doing nothing — a one-shot process has no session to change. `logs` is refused too, for a different reason: its argument is the rest of the line **verbatim** (a `procs({...})` literal), which argv cannot carry. Each refusal exits 2 and names the builtin, instead of the `cannot read cd` a fallthrough to script mode produced. A file that exists wins when it is the only argument, so `crust skills` runs `./skills`; any extra argument selects the tool. |
17
+ | `crust --check '<line>'` | Parse without running. See below. |
18
+ | `crust --env-file <path> …` | Loads `.env` before *any* run mode and before `init.ts`. The "loaded" note goes to stderr so a `-c` pipeline's stdout stays clean. A missing file exits **2, loudly** — this flag exists to replace shell shims that failed silently. Does not combine with `--check`. |
19
+ | `crust -h` / `-V` | Usage / version, exit 0. |
20
+
21
+ ## Exit codes (measured against the binary, not guessed)
22
+
23
+ | Code | Meaning | Examples |
24
+ | --- | --- | --- |
25
+ | 0 | Success. | `range(1,2)` |
26
+ | 1 | **crust** failed. Every error crust itself raises. | `assert` mismatch, a lambda that throws, `expect 200` vs a 500, `wait: <target> not ready after Ns`, a `GET` that can't connect, `read` on a missing file, a refused empty stage (`echo a \|\| echo b`), a builtin name that reached sh. |
27
+ | 2 | Invocation or syntax error — nothing ran. | `crust --nonsense`, `--check` with no line, `--env-file` on a missing file, `-c` given two lines, `mock-server --bogus` (prints the valid flags), sh's own parse errors on a pure-shell line (`bundle(5)`) — and the same parse error on any shell stage of a **mixed** line (named; before this a bad stage N surfaced only when its child exited, after stages 1..N-1 had run). |
28
+ | the stage's own | A shell stage's exit code, propagated. | `range(1,3) \| sh -c "exit 9"` → 9 |
29
+ | 127 | Command not found (shell stage), or an unreadable script file. | `nosuchbinary-xyz` |
30
+ | 130 / 143 | crust killed by SIGINT / SIGTERM. | Ctrl-C during a long `-c` run |
31
+
32
+ **A builtin name is not a missing binary.** `crust verify-web-links` with no
33
+ arguments exits **2** with the tool's own usage, and `crust skills install
34
+ --bogus` exits **2** naming the valid flags. 127 means a *script path* could not
35
+ be read, or sh could not find a command — if a builtin name produces 127, the
36
+ binary really is an older one that treated every positional as a path.
37
+
38
+ **There is no crust exit 124.** `wait` timing out is exit **1** with
39
+ `crust: wait: <target> not ready after Ns`. If a probe reports 124, then
40
+ `timeout(1)` — your harness — killed crust. Give `wait`/`load`/`procs` probes a
41
+ generous outer timeout, and when testing cancellation signal the real bun PID.
42
+
43
+ A **bare `wait :3000` probes `/`**, and readiness means a **2xx** — a mock server
44
+ or an API with no root route 404s forever, so the line fails after the full
45
+ timeout on a server that is perfectly up. Use a real health path (`wait
46
+ :3000/health`) or `wait port:3000` for a TCP connect; crust says so in the
47
+ failure message now, because "not ready" alone reads like a slow boot.
48
+
49
+ ### The last command wins (the one CI trap here)
50
+
51
+ A line made **entirely of shell stages** is handed to sh as one `sh -c` string,
52
+ so its code is the **last** command's — no `pipefail`:
53
+
54
+ ```bash
55
+ ls /nope | wc -l # 0 — wc succeeded
56
+ ls /nope # 2
57
+ ```
58
+
59
+ The moment a line contains a crust stage, crust owns the exit code and a
60
+ failing shell stage propagates. So a gate must **end on a crust stage**:
61
+
62
+ ```crust
63
+ # a run.crust: blanks and `#` are skipped, lines run in order, first failure stops the file
64
+ wait :3000/health
65
+ range(1,99) | parallel 10 | GET http://localhost:3000/items | expect 200 | stats
66
+ GET http://localhost:3000/items | assert (x => x.length > 0)
67
+ # a trailing `\` continues the line — in a script, in piped stdin, in a
68
+ # multi-line -c, or at the REPL (where an unbalanced quote or ( continues too)
69
+ load 10s 100/s | parallel 20 | GET http://localhost:3000/items \
70
+ | expect 200 | stats
71
+ ```
72
+
73
+ Or ask sh to gate, explicitly:
74
+
75
+ ```bash
76
+ sh -c 'set -o pipefail; ls /nope | wc -l' # 2 — would be 0 without pipefail
77
+ ```
78
+
79
+ ## `--check`: the linter, and its edges
80
+
81
+ `crust --check '<line>'` exits **0** if every line parses, **1** with the error
82
+ otherwise (`ok: N line(s) parse`). It splits on newlines and skips blank and
83
+ `#` lines, so a whole fenced block goes in as one argument. It checks three
84
+ things: crust's own grammar, a builtin-head line against **that CLI's flag
85
+ spec**, and every shell stage through `sh -n` (noexec — `cat > f` under `-n`
86
+ does not create `f`).
87
+
88
+ ```bash
89
+ crust --check 'read /nonexistent/*.json | POST :3000/x' # ok — sources are lazy
90
+ ```
91
+
92
+ ```
93
+ crust: shell stage does not parse (stage: head -(): sh: -c: line 1: syntax error near unexpected token `(`
94
+ in: range(1,3) | head -(
95
+ ```
96
+
97
+ The runtime runs the same preflight before executing any stage of a **mixed**
98
+ line, so a shell stage that does not parse exits 2 with this message and
99
+ nothing has run (measured pre-fix: `touch f | base64 | head -(` exited 2 and
100
+ `f` existed). A pure-shell line is one `sh -c`, which sh parses the whole line
101
+ for before executing anything, so its own stderr stays the answer there — and
102
+ 127 (command not found) is a runtime error the preflight cannot see.
103
+
104
+ What it does **not** check: anything needing I/O or a live process — a missing
105
+ fixture path, an unreachable URL, a wrong `output` matcher key, a semantic
106
+ refusal at runtime (`--env-file` + `--check` is refused for exactly this
107
+ reason). "Checks clean" means *parses*, never *will pass*.
108
+
109
+ ## Environment
110
+
111
+ | How | Scope | Reaches sh stages? |
112
+ | --- | --- | --- |
113
+ | `crust --env-file <path> …` | one process, before `init.ts` | yes |
114
+ | `dotenv <path>` / `dotenv --append` / `dotenv status` / `dotenv clear` | the session | yes |
115
+ | `export FOO=bar` | the session | yes |
116
+ | a bare `FOO=bar` line | the session — the same code as `export`, including `+=` | yes |
117
+ | `capture NAME` stage | writes `process.env.NAME` from the stream, for **later lines** | yes |
118
+
119
+ ## The REPL
120
+
121
+ Verified in a real pty, not read off the source:
122
+
123
+ - **Editing:** `Ctrl-A` / `Ctrl-E` home/end, arrow keys move (and `Home`/`End`/
124
+ `Delete`), `Ctrl-K` kill to end, `Ctrl-U` kill to start, `Ctrl-W` kill
125
+ previous word, `Ctrl-L` repaint (the buffer survives), `Backspace`, `Tab`
126
+ completes.
127
+ - **Tab** completes a **path** mid-stage (`ls sr` + Tab → `ls src/`) and lists
128
+ names from `$PATH` at the start of a stage. It does **not** know crust's own
129
+ builtins or registered functions — `verify-web-` + Tab lists nothing useful.
130
+ - **History:** up/down arrows. It is plain text at
131
+ `$XDG_DATA_HOME/crust/history` (default `~/.local/share/crust/history`), it
132
+ survives restarts, `history` prints it numbered, and re-running a recalled
133
+ line does not append a duplicate.
134
+ - **`Ctrl-R`** searches that history as you type: a `(crust-search):` line opens
135
+ below the one you are editing, showing the newest entry containing your query;
136
+ another `Ctrl-R` steps further back, `Backspace` shortens the query. `Enter`
137
+ **accepts** — the match lands on the prompt line, editable, and the **next**
138
+ `Enter` runs it (so a search at a `…` continuation prompt cannot lose the
139
+ pending line). `Ctrl-G` / `Esc` / `Ctrl-C` cancel and restore the draft
140
+ byte-exact; `Ctrl-C` mid-search does **not** clear the line. No match paints
141
+ `-- no match --` and `Enter` inserts the query as text — it never runs
142
+ nothing, and it never rewrites the history file.
143
+ - **`Ctrl-C`** at the prompt just clears the line. Mid-run it **cancels**:
144
+ children are killed, the line reports 130, and the prompt comes back so the
145
+ session continues. One item already produced by the source can still print
146
+ after the prompt redraws — cosmetic; the source is stopped.
147
+ - **`Ctrl-D`** (or `exit`) leaves; the exit code is 0 unless `exit <code>`.
148
+ - **`Ctrl-Z`** suspends what the line is waiting on — a crust pipeline, a
149
+ builtin, a `fg`/`wait` — as a job: SIGSTOP, `[2]+ Stopped sleep 12`, prompt
150
+ back, `bg` resumes a suspended pipeline as one unit. During a **foreground
151
+ inherit-stdio child** it does nothing at all (see Job control below).
152
+
153
+ ## Job control
154
+
155
+ `cmd &` is crust's own background job: its own process group (so a signal
156
+ reaches a forking `sh`'s whole tree), stdin `/dev/null`, output still on the
157
+ terminal, and it **outlives crust**. `%1` / `%` / `%+` name jobs; a bare number
158
+ names a job to `fg`/`bg`/`jobs` and a **PID** to `kill`.
159
+
160
+ | Line | What it does |
161
+ | --- | --- |
162
+ | `jobs`, `jobs -l` | the table; `-l` adds `(pid …)` and the code once there is one; an unknown option is refused, not ignored |
163
+ | `fg [spec]` | resume (if stopped) and **wait**; the job's own code becomes the line's |
164
+ | `bg [spec]` | resume a stopped job; an error if it was not stopped |
165
+ | `wait %1` | the job wait — **bare `wait` is the readiness stage**, `wait :3000/health` |
166
+ | `kill %1`, `kill -9 %1` | prints `Sent SIGTERM to job 1 (pid …)` — the send, not the death. `kill -9 4242` is sh's |
167
+
168
+ Statuses are `Running`, `Stopped`, `Done`, `Exit N`, `Terminated` (128+signal),
169
+ `Failed` (gone with no code — never a silent pass). Finished jobs stay listed for
170
+ the session, which is how `wait %1` still reports the code of a job that ended
171
+ minutes ago. `Ctrl-C` during `fg`/`wait` kills the job and returns 130.
172
+
173
+ SIGSTOP, never SIGTSTP: Bun's runtime ignores SIGTSTP and a child it spawns
174
+ inherits that ignore across `exec`, so **Ctrl-Z cannot stop a foreground
175
+ inherit-stdio child** (`vim`, a bare `sleep 20`) — the `^Z` echoes and the child
176
+ keeps running; Ctrl-C is the escape (child dies, line 130). Running jobs are left
177
+ alone at exit because outliving the session is what `&` means; **stopped** ones are
178
+ resumed with a notice, since SIGSTOP survives reparenting and no shell resumes a
179
+ process it did not stop.
180
+ Driving it non-interactively: crust checks for a TTY, so pipe bytes through
181
+ `script` and pace them — `{ printf 'range(1,3)\n'; sleep 1; printf '\004'; } |
182
+ script -qec 'crust' /dev/null`.
183
+
184
+ ## Where crust's globals exist
185
+
186
+ `GET`, `range`, `parallel`, `load`, `procs`, `$`, `crust.*` and everything you
187
+ define in `init.ts` are globals **wherever crust starts its runtime**: the REPL,
188
+ `-c`, piped stdin, a script file, `~/.config/crust/init.ts`, and any `.ts`/`.js`
189
+ file crust imports (`source hooks/warmup.ts` — inside it, `load` and `parallel`
190
+ are just there). A file run by plain `bun script.ts` never starts crust, so
191
+ those names are `undefined` there; import what you need from the modules
192
+ (`../src/sinks` today) or run the file under crust instead.
193
+
194
+ ## Auto-discovered stages
195
+
196
+ crust scans your **global bun install** at startup and turns opted-in packages
197
+ into pipeline stages — no `init.ts` entry needed.
198
+
199
+ - Reads `$CRUST_GLOBAL_PREFIX/package.json` (default `~/.bun/install/global`) and
200
+ inspects its dependencies. A package with a `bin` field and **no** `crust`
201
+ field is skipped, so it keeps working as a plain shell command; add
202
+ `"crust": { … }` and it becomes a stage *as well as* a command.
203
+ - `package.json` → `"crust": { "stage": "default" }` (or a named export) picks
204
+ which export becomes the stage; otherwise the default export, or the sole
205
+ function export. The stage is called `fn(item, ...args)` like any registered
206
+ function, so it works as a source *and* mid-pipeline.
207
+ - Scoped packages are exposed after the `/` (`@me/shout` → `shout`). A name
208
+ already registered (by `crust.fn` in `init.ts`) wins — discovery never
209
+ overwrites explicit config.
210
+ - The scan is cached at `$CRUST_CACHE_DIR/globals.json` (default
211
+ `~/.cache/crust`) keyed on the prefix `package.json` mtime; delete that file
212
+ after changing the install by hand.
213
+ - `CRUST_DEBUG=1` prints every decision: `skipping "x" — has bin field`,
214
+ `failed to inspect …`, `name collision on …`. That's where to look when a
215
+ package you expected just isn't a stage.
@@ -0,0 +1,161 @@
1
+ ---
2
+ name: crust-load-testing
3
+ description: Load-test and soak-test HTTP services with crust — the load rate source, parallel workers, timed verbs, stats windows, CI threshold gates via assert, and baseline comparison with stats --out. Use for smoke-load, latency percentiles, rps checks, performance regression gates in CI, or "is this endpoint fast/degrading" questions.
4
+ ---
5
+
6
+ # crust load testing
7
+
8
+ One line gives you a paced load run with real percentiles and a CI-ready
9
+ exit code. Tick pacing batches due slots per wakeup, so the generator
10
+ sustains 5000+ ticks/s — your real ceiling is downstream (`parallel` pool ×
11
+ service time). Single-process smoke-load and soak tooling, not a
12
+ distributed rig.
13
+
14
+ ## The shapes
15
+
16
+ ```crust
17
+ # fixed volume: N requests, K at a time
18
+ range(0, 999) | parallel 50 | GET :3000/health | expect 200 | stats
19
+
20
+ # paced: arrival RATE for a DURATION (load 30s 100/s), concurrency capped by parallel
21
+ load 30s 100/s | parallel 50 | GET :3000/health | expect 200 | stats
22
+
23
+ # ramp: comma-separated phases, one stats stream across all of them
24
+ load 10s 50/s, 30s 200/s | parallel 100 | GET :3000/health | stats --every 5
25
+
26
+ # POST load: ticks are {n, phase, scheduledAt, lagMs} — build bodies from them
27
+ load 10s 20/s | (t => ({name: "user" + t.n})) | parallel 8 | POST :3000/users | expect 2xx | stats
28
+ ```
29
+
30
+ > **Report availability:** `expect` throws at drain, so an `expect` placed
31
+ > BEFORE `stats` that fails prevents the summary from ever being emitted
32
+ > (the first two shapes above are illustrative of syntax, not of CI
33
+ > reporting). When the summary/report must survive a status failure, put
34
+ > `stats` (and `stats --out report.json`) before the gates and gate status
35
+ > on the summary instead — see CI gates below.
36
+
37
+ - `parallel` puts ANY http verb in load mode: output is `{status, ms, url}`
38
+ timing records, bodies drained, network errors become `status: 0` records
39
+ (they show in the histogram instead of killing the run). So a run whose
40
+ every request was refused prints `{"status":{"0":40}}` and still exits 0
41
+ unless a gate is present — `expect 200` over a dead port exits 1. `stats`
42
+ reports, it does not gate.
43
+ - **Completion ≠ success — three separate outcomes:** the run *completed*
44
+ (exit 0, no fatal stage error), *request correctness passed* (explicit
45
+ gates accepted the outcomes), *performance passed* (latency/throughput
46
+ thresholds held). A gate that only checks latency can be satisfied by fast
47
+ error responses — a 500 in 2ms is under any p95 budget. Latency-only
48
+ examples are NOT the definition of a passing load test; gate status too.
49
+ - The summary carries failure tallies `timedOut` and `redirectRefused`
50
+ (always present, 0 on a clean run): a `--timeout` budget hit or a
51
+ `--redirect error` refusal is a `{status: 0, timedOut/redirectRefused:
52
+ true}` RECORD, not a failure exit. Gate them — `expect 200` (status 0
53
+ fails it) or `stats | assert (s => s.redirectRefused === 0)`; exit 0
54
+ alone is never request success in load mode.
55
+ - Rate (`load`) and concurrency (`parallel N`) are independent knobs.
56
+ Little's law sizes the pool: `N ≥ rate × p99-in-seconds`.
57
+ - Durations `ms|s|m`; rates `N/s` or `N/m`, decimals allowed.
58
+
59
+ Canonical, version-pinned result-shape table: https://lariocpt.github.io/crust-website/docs/http-result-shapes/.
60
+
61
+ ## Honesty guarantees (read before quoting numbers)
62
+
63
+ - A gate that measured NOTHING fails: `stats` over an empty stream is tagged
64
+ `empty: true` and `assert` refuses it. `{count: 0, p95: 0}` used to satisfy
65
+ `s => s.p95 < 200`, so a run that issued zero requests reported success.
66
+ - Percentiles are bucketed (0.1ms below 100ms, 1ms below 1s) and report the
67
+ bucket's upper bound — never faster than reality. `count` and `meanMs` are
68
+ exact. Memory is constant, so a multi-hour soak is safe.
69
+
70
+ - `stats.rps` is MEASURED, never the target. If downstream saturates, stale
71
+ slots are skipped (never burst) and reported on stderr:
72
+ `load: target 3000 ticks — emitted 2868, dropped 132 … achieved 95.6/s`.
73
+ A drop report means the target rate was NOT sustained — raise `parallel N`
74
+ or lower the rate before quoting percentiles.
75
+ - Percentiles come only from timed records; they include body download (a
76
+ route that drips its body over 100ms reports p50 102, not 0.4).
77
+ - With `--every`, a slow request lands in the window it FINISHED in — and
78
+ because stats waits for a completion before it can flush, a window's
79
+ `wallMs` can exceed N (measured: `--every 2` → window 1 `wallMs: 3006`).
80
+ - `--out` takes `.json` only — any other extension is a loud error, never a
81
+ text file written under a json-looking name.
82
+
83
+ ## CI gates — thresholds are assert composition
84
+
85
+ The primary shape gates **status first, then performance**, with `stats`
86
+ (and the report) upstream of every gate so the summary survives a failing
87
+ gate. When failure artifacts must survive request-status failures, collect
88
+ statistics and write the report BEFORE applying aggregate gates:
89
+
90
+ ```crust
91
+ load 30s 100/s | parallel 50 | GET :3000/health | stats --out load/last.json | assert (s => Object.keys(s.status).every(k => k.startsWith("2"))) | assert (s => s.timedOut === 0) | assert (s => s.redirectRefused === 0) | assert (s => s.p95 < 200) | assert (s => s.rps > 80)
92
+ ```
93
+
94
+ One assert per threshold: the failure names the predicate and prints the
95
+ actual summary, and the line exits 1 (`crust -c` stops there). The gates
96
+ above cover all four things a load gate must mean: every response was 2xx
97
+ (a fast 500 is not a pass), no timeout or redirect-refusal records, the
98
+ latency budget held, and the achieved rate held. An empty run is refused by
99
+ `assert` itself (`empty: true`), so "measured nothing" never passes.
100
+ `rps` is the ACHIEVED rate — if the target was not sustained, stderr already
101
+ said so (`emitted … dropped … achieved …`); a drop report means the number
102
+ below the rate gate is the honest one.
103
+
104
+ Baseline gate — `--out` writes a versioned artifact
105
+ (`{crustStats: 1, startedAt, urls, summary, windows?}`), an async assert
106
+ reads it back:
107
+
108
+ ```crust
109
+ load 10s 100/s | parallel 50 | GET :3000/health | stats --out load/last.json | assert (async s => { const b = await Bun.file("load/baseline.json").json(); return s.p95 < 2 * b.summary.p95 })
110
+ ```
111
+
112
+ The artifact is written BEFORE the gate runs, so a failed gate still leaves
113
+ the file for CI upload.
114
+
115
+ Soak with fail-fast windows (`--every N` emits `{window: k, …}` deltas then
116
+ `{final: true, …}`; a bare predicate gates every one of them):
117
+
118
+ ```crust
119
+ load 60s 25/s | parallel 25 | GET :3000/health | stats --every 5 | assert (s => !s.window || s.p95 < 400)
120
+ ```
121
+
122
+ Guard `!s.window ||` scopes a threshold to window objects, `!s.final ||` to
123
+ the final summary — and a guard needs the object it guards. Without `--every`
124
+ the plain summary carries neither key, so `!s.window` is true and `||`
125
+ short-circuits: the threshold never runs. `stats | assert (s => !s.window ||
126
+ s.p95 < 0.1)` exits 0 with p95 at 4.2. Drop the guard when you drop `--every`.
127
+
128
+ ## Warmup
129
+
130
+ A separate line in the same script — its stats simply aren't gated:
131
+
132
+ ```crust
133
+ range(0, 99) | parallel 10 | GET :3000/health | stats
134
+ load 30s 100/s | parallel 50 | GET :3000/health | stats | assert (s => s.p95 < 200)
135
+ ```
136
+
137
+ ## TS API equivalent
138
+
139
+ The same surface as globals inside `init.ts` (and crust-run config code) —
140
+ `load`, `parallel`, `timedGet`, `timedHttpItem`, `statsStage`, `Pipeline`:
141
+
142
+ ```ts
143
+ const out = await load([{ durMs: 10_000, rps: 100 }])
144
+ .pipe(parallel(50, timedGet("http://localhost:3000/health")))
145
+ .pipe(statsStage(undefined, "load/last.json"))
146
+ .collect();
147
+ ```
148
+
149
+ `timedHttpItem(method, url, opts?)` is the any-verb per-item timer when you
150
+ need POST bodies from upstream items.
151
+
152
+ ## Version and evidence
153
+
154
+ Behavior here was verified against **crust v0.3.5** — summary schema
155
+ (`{count, wallMs, rps, status, timedOut, redirectRefused, p50, p95, p99,
156
+ meanMs}`, `empty: true` on a zero-request run) and the `--out`-before-gates
157
+ write order are source-verified; throughput claims (5000+ ticks/s) are
158
+ single-process measurements on the author's hardware — environment-dependent,
159
+ not a promise. Pick the lightest correct layer for the job (see the
160
+ crust-api-testing skill's shared principle), and keep approved targets and
161
+ execution restrictions in repository policy, not in this skill.