@lariocpt/crust-mongo 0.0.0-stage → 0.3.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +174 -2
- package/bin/crust +64 -0
- package/lib/install.mjs +330 -0
- package/lib/skills-prompt.mjs +134 -0
- package/package.json +55 -4
- package/scripts/postinstall.mjs +36 -0
- package/skills/crust-api-testing/SKILL.md +154 -0
- package/skills/crust-builtins/SKILL.md +238 -0
- package/skills/crust-cli/SKILL.md +215 -0
- package/skills/crust-load-testing/SKILL.md +123 -0
- package/skills/crust-logs/SKILL.md +99 -0
- package/skills/crust-mcp/SKILL.md +108 -0
- package/skills/crust-mock-server/SKILL.md +138 -0
- package/skills/crust-pipelines/SKILL.md +161 -0
- package/skills/crust-procs/SKILL.md +108 -0
- package/skills/crust-verify-web-links/SKILL.md +126 -0
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: crust-cli
|
|
3
|
+
description: How to invoke crust and read its answer — the run modes (`-c`, script file, piped stdin, REPL, tool subcommands like `crust skills install`), what each exit code actually means (0/1/2/127/130, and why there is no 124), `--check` as a linter including what it cannot catch, `--env-file` vs `dotenv` vs `export`, the REPL's verified keybindings and Ctrl-C semantics, and where crust's globals (`GET`, `range`, `parallel`, `crust.fn`) do and do not exist. Use when a crust run's exit code is confusing, when a CI step passes on a failing line, when `--check` says ok but the line dies, when `GET is not defined` appears in a `.ts` file, or when driving the REPL from a script.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Running crust, and reading what it says back
|
|
7
|
+
|
|
8
|
+
## Run modes
|
|
9
|
+
|
|
10
|
+
| Invocation | What happens |
|
|
11
|
+
| --- | --- |
|
|
12
|
+
| `crust` (stdin is a TTY) | Interactive REPL. |
|
|
13
|
+
| `crust` (stdin is piped) | Reads **lines of crust** from stdin to EOF and runs them like a script (fail-fast, same exit codes). stdin is the *command* source in this mode, so a shell stage inside those lines sees EOF. Want data on stdin? Use `-c`. |
|
|
14
|
+
| `crust run.crust [args…]` | Script mode: blank lines and `#` comments are skipped (so a `#!/usr/bin/env crust` shebang works), lines run in order, and it **fails fast** — the first failing line stops the file and becomes the process exit code. A line ending in `\` continues on the next. The extension doesn't matter. Extra arguments are the script's **positional arguments** — `$0` is the path, `$1`…`$9` the rest, `$#` the count, `$@`/`$*` all of them joined — expanding wherever an env var does (URLs, JSON literals, `-H` strings, fn args, `stats --out`), and empty, never literal, outside script mode. A leading `--` after the path is a separator, not an argument. An unreadable path is 127. |
|
|
15
|
+
| `crust -c '<line>'` | One line. A shell stage inherits the process stdin, so `docker logs app \| crust -c 'grep ERROR'` really does grep the logs. To feed a **crust** pipeline, name the source: `docker logs -f app \| crust -c 'stdin \| grep ERROR'`. To run several lines, put a **newline inside the one argument** — they then behave like a script (fail-fast) — or continue one line with a trailing `\`. Two lines as two arguments is refused. Quote the whole pipeline: `-c range(1,2) extra` splits into three argv words and is refused. |
|
|
16
|
+
| `crust <tool> [args…]` | A **tool builtin as a subcommand**: `crust skills install`, `crust mock-server openapi.yaml -p4000`, `crust test-fixture 'fixtures/*.crust.ts' -j8`. Same flags as the REPL line, and `crust <tool> --help` prints them. The names that work are the ones needing no live session: `skills`, `test-fixture`, `test-pipes`, `gen-fixtures`, `mock-server`, `verify-web-links`. The **session** builtins (`cd`, `export`, `alias`, `unalias`, `source`, `dotenv`, `exit`, `help`, `history`) are refused rather than silently doing nothing — a one-shot process has no session to change. `logs` is refused too, for a different reason: its argument is the rest of the line **verbatim** (a `procs({...})` literal), which argv cannot carry. Each refusal exits 2 and names the builtin, instead of the `cannot read cd` a fallthrough to script mode produced. A file that exists wins when it is the only argument, so `crust skills` runs `./skills`; any extra argument selects the tool. |
|
|
17
|
+
| `crust --check '<line>'` | Parse without running. See below. |
|
|
18
|
+
| `crust --env-file <path> …` | Loads `.env` before *any* run mode and before `init.ts`. The "loaded" note goes to stderr so a `-c` pipeline's stdout stays clean. A missing file exits **2, loudly** — this flag exists to replace shell shims that failed silently. Does not combine with `--check`. |
|
|
19
|
+
| `crust -h` / `-V` | Usage / version, exit 0. |
|
|
20
|
+
|
|
21
|
+
## Exit codes (measured against the binary, not guessed)
|
|
22
|
+
|
|
23
|
+
| Code | Meaning | Examples |
|
|
24
|
+
| --- | --- | --- |
|
|
25
|
+
| 0 | Success. | `range(1,2)` |
|
|
26
|
+
| 1 | **crust** failed. Every error crust itself raises. | `assert` mismatch, a lambda that throws, `expect 200` vs a 500, `wait: <target> not ready after Ns`, a `GET` that can't connect, `read` on a missing file, a refused empty stage (`echo a \|\| echo b`), a builtin name that reached sh. |
|
|
27
|
+
| 2 | Invocation or syntax error — nothing ran. | `crust --nonsense`, `--check` with no line, `--env-file` on a missing file, `-c` given two lines, `mock-server --bogus` (prints the valid flags), sh's own parse errors on a pure-shell line (`bundle(5)`) — and the same parse error on any shell stage of a **mixed** line (named; before this a bad stage N surfaced only when its child exited, after stages 1..N-1 had run). |
|
|
28
|
+
| the stage's own | A shell stage's exit code, propagated. | `range(1,3) \| sh -c "exit 9"` → 9 |
|
|
29
|
+
| 127 | Command not found (shell stage), or an unreadable script file. | `nosuchbinary-xyz` |
|
|
30
|
+
| 130 / 143 | crust killed by SIGINT / SIGTERM. | Ctrl-C during a long `-c` run |
|
|
31
|
+
|
|
32
|
+
**A builtin name is not a missing binary.** `crust verify-web-links` with no
|
|
33
|
+
arguments exits **2** with the tool's own usage, and `crust skills install
|
|
34
|
+
--bogus` exits **2** naming the valid flags. 127 means a *script path* could not
|
|
35
|
+
be read, or sh could not find a command — if a builtin name produces 127, the
|
|
36
|
+
binary really is an older one that treated every positional as a path.
|
|
37
|
+
|
|
38
|
+
**There is no crust exit 124.** `wait` timing out is exit **1** with
|
|
39
|
+
`crust: wait: <target> not ready after Ns`. If a probe reports 124, then
|
|
40
|
+
`timeout(1)` — your harness — killed crust. Give `wait`/`load`/`procs` probes a
|
|
41
|
+
generous outer timeout, and when testing cancellation signal the real bun PID.
|
|
42
|
+
|
|
43
|
+
A **bare `wait :3000` probes `/`**, and readiness means a **2xx** — a mock server
|
|
44
|
+
or an API with no root route 404s forever, so the line fails after the full
|
|
45
|
+
timeout on a server that is perfectly up. Use a real health path (`wait
|
|
46
|
+
:3000/health`) or `wait port:3000` for a TCP connect; crust says so in the
|
|
47
|
+
failure message now, because "not ready" alone reads like a slow boot.
|
|
48
|
+
|
|
49
|
+
### The last command wins (the one CI trap here)
|
|
50
|
+
|
|
51
|
+
A line made **entirely of shell stages** is handed to sh as one `sh -c` string,
|
|
52
|
+
so its code is the **last** command's — no `pipefail`:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
ls /nope | wc -l # 0 — wc succeeded
|
|
56
|
+
ls /nope # 2
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The moment a line contains a crust stage, crust owns the exit code and a
|
|
60
|
+
failing shell stage propagates. So a gate must **end on a crust stage**:
|
|
61
|
+
|
|
62
|
+
```crust
|
|
63
|
+
# a run.crust: blanks and `#` are skipped, lines run in order, first failure stops the file
|
|
64
|
+
wait :3000/health
|
|
65
|
+
range(1,99) | parallel 10 | GET http://localhost:3000/items | expect 200 | stats
|
|
66
|
+
GET http://localhost:3000/items | assert (x => x.length > 0)
|
|
67
|
+
# a trailing `\` continues the line — in a script, in piped stdin, in a
|
|
68
|
+
# multi-line -c, or at the REPL (where an unbalanced quote or ( continues too)
|
|
69
|
+
load 10s 100/s | parallel 20 | GET http://localhost:3000/items \
|
|
70
|
+
| expect 200 | stats
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Or ask sh to gate, explicitly:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
sh -c 'set -o pipefail; ls /nope | wc -l' # 2 — would be 0 without pipefail
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## `--check`: the linter, and its edges
|
|
80
|
+
|
|
81
|
+
`crust --check '<line>'` exits **0** if every line parses, **1** with the error
|
|
82
|
+
otherwise (`ok: N line(s) parse`). It splits on newlines and skips blank and
|
|
83
|
+
`#` lines, so a whole fenced block goes in as one argument. It checks three
|
|
84
|
+
things: crust's own grammar, a builtin-head line against **that CLI's flag
|
|
85
|
+
spec**, and every shell stage through `sh -n` (noexec — `cat > f` under `-n`
|
|
86
|
+
does not create `f`).
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
crust --check 'read /nonexistent/*.json | POST :3000/x' # ok — sources are lazy
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
crust: shell stage does not parse (stage: head -(): sh: -c: line 1: syntax error near unexpected token `(`
|
|
94
|
+
in: range(1,3) | head -(
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
The runtime runs the same preflight before executing any stage of a **mixed**
|
|
98
|
+
line, so a shell stage that does not parse exits 2 with this message and
|
|
99
|
+
nothing has run (measured pre-fix: `touch f | base64 | head -(` exited 2 and
|
|
100
|
+
`f` existed). A pure-shell line is one `sh -c`, which sh parses the whole line
|
|
101
|
+
for before executing anything, so its own stderr stays the answer there — and
|
|
102
|
+
127 (command not found) is a runtime error the preflight cannot see.
|
|
103
|
+
|
|
104
|
+
What it does **not** check: anything needing I/O or a live process — a missing
|
|
105
|
+
fixture path, an unreachable URL, a wrong `output` matcher key, a semantic
|
|
106
|
+
refusal at runtime (`--env-file` + `--check` is refused for exactly this
|
|
107
|
+
reason). "Checks clean" means *parses*, never *will pass*.
|
|
108
|
+
|
|
109
|
+
## Environment
|
|
110
|
+
|
|
111
|
+
| How | Scope | Reaches sh stages? |
|
|
112
|
+
| --- | --- | --- |
|
|
113
|
+
| `crust --env-file <path> …` | one process, before `init.ts` | yes |
|
|
114
|
+
| `dotenv <path>` / `dotenv --append` / `dotenv status` / `dotenv clear` | the session | yes |
|
|
115
|
+
| `export FOO=bar` | the session | yes |
|
|
116
|
+
| a bare `FOO=bar` line | the session — the same code as `export`, including `+=` | yes |
|
|
117
|
+
| `capture NAME` stage | writes `process.env.NAME` from the stream, for **later lines** | yes |
|
|
118
|
+
|
|
119
|
+
## The REPL
|
|
120
|
+
|
|
121
|
+
Verified in a real pty, not read off the source:
|
|
122
|
+
|
|
123
|
+
- **Editing:** `Ctrl-A` / `Ctrl-E` home/end, arrow keys move (and `Home`/`End`/
|
|
124
|
+
`Delete`), `Ctrl-K` kill to end, `Ctrl-U` kill to start, `Ctrl-W` kill
|
|
125
|
+
previous word, `Ctrl-L` repaint (the buffer survives), `Backspace`, `Tab`
|
|
126
|
+
completes.
|
|
127
|
+
- **Tab** completes a **path** mid-stage (`ls sr` + Tab → `ls src/`) and lists
|
|
128
|
+
names from `$PATH` at the start of a stage. It does **not** know crust's own
|
|
129
|
+
builtins or registered functions — `verify-web-` + Tab lists nothing useful.
|
|
130
|
+
- **History:** up/down arrows. It is plain text at
|
|
131
|
+
`$XDG_DATA_HOME/crust/history` (default `~/.local/share/crust/history`), it
|
|
132
|
+
survives restarts, `history` prints it numbered, and re-running a recalled
|
|
133
|
+
line does not append a duplicate.
|
|
134
|
+
- **`Ctrl-R`** searches that history as you type: a `(crust-search):` line opens
|
|
135
|
+
below the one you are editing, showing the newest entry containing your query;
|
|
136
|
+
another `Ctrl-R` steps further back, `Backspace` shortens the query. `Enter`
|
|
137
|
+
**accepts** — the match lands on the prompt line, editable, and the **next**
|
|
138
|
+
`Enter` runs it (so a search at a `…` continuation prompt cannot lose the
|
|
139
|
+
pending line). `Ctrl-G` / `Esc` / `Ctrl-C` cancel and restore the draft
|
|
140
|
+
byte-exact; `Ctrl-C` mid-search does **not** clear the line. No match paints
|
|
141
|
+
`-- no match --` and `Enter` inserts the query as text — it never runs
|
|
142
|
+
nothing, and it never rewrites the history file.
|
|
143
|
+
- **`Ctrl-C`** at the prompt just clears the line. Mid-run it **cancels**:
|
|
144
|
+
children are killed, the line reports 130, and the prompt comes back so the
|
|
145
|
+
session continues. One item already produced by the source can still print
|
|
146
|
+
after the prompt redraws — cosmetic; the source is stopped.
|
|
147
|
+
- **`Ctrl-D`** (or `exit`) leaves; the exit code is 0 unless `exit <code>`.
|
|
148
|
+
- **`Ctrl-Z`** suspends what the line is waiting on — a crust pipeline, a
|
|
149
|
+
builtin, a `fg`/`wait` — as a job: SIGSTOP, `[2]+ Stopped sleep 12`, prompt
|
|
150
|
+
back, `bg` resumes a suspended pipeline as one unit. During a **foreground
|
|
151
|
+
inherit-stdio child** it does nothing at all (see Job control below).
|
|
152
|
+
|
|
153
|
+
## Job control
|
|
154
|
+
|
|
155
|
+
`cmd &` is crust's own background job: its own process group (so a signal
|
|
156
|
+
reaches a forking `sh`'s whole tree), stdin `/dev/null`, output still on the
|
|
157
|
+
terminal, and it **outlives crust**. `%1` / `%` / `%+` name jobs; a bare number
|
|
158
|
+
names a job to `fg`/`bg`/`jobs` and a **PID** to `kill`.
|
|
159
|
+
|
|
160
|
+
| Line | What it does |
|
|
161
|
+
| --- | --- |
|
|
162
|
+
| `jobs`, `jobs -l` | the table; `-l` adds `(pid …)` and the code once there is one; an unknown option is refused, not ignored |
|
|
163
|
+
| `fg [spec]` | resume (if stopped) and **wait**; the job's own code becomes the line's |
|
|
164
|
+
| `bg [spec]` | resume a stopped job; an error if it was not stopped |
|
|
165
|
+
| `wait %1` | the job wait — **bare `wait` is the readiness stage**, `wait :3000/health` |
|
|
166
|
+
| `kill %1`, `kill -9 %1` | prints `Sent SIGTERM to job 1 (pid …)` — the send, not the death. `kill -9 4242` is sh's |
|
|
167
|
+
|
|
168
|
+
Statuses are `Running`, `Stopped`, `Done`, `Exit N`, `Terminated` (128+signal),
|
|
169
|
+
`Failed` (gone with no code — never a silent pass). Finished jobs stay listed for
|
|
170
|
+
the session, which is how `wait %1` still reports the code of a job that ended
|
|
171
|
+
minutes ago. `Ctrl-C` during `fg`/`wait` kills the job and returns 130.
|
|
172
|
+
|
|
173
|
+
SIGSTOP, never SIGTSTP: Bun's runtime ignores SIGTSTP and a child it spawns
|
|
174
|
+
inherits that ignore across `exec`, so **Ctrl-Z cannot stop a foreground
|
|
175
|
+
inherit-stdio child** (`vim`, a bare `sleep 20`) — the `^Z` echoes and the child
|
|
176
|
+
keeps running; Ctrl-C is the escape (child dies, line 130). Running jobs are left
|
|
177
|
+
alone at exit because outliving the session is what `&` means; **stopped** ones are
|
|
178
|
+
resumed with a notice, since SIGSTOP survives reparenting and no shell resumes a
|
|
179
|
+
process it did not stop.
|
|
180
|
+
Driving it non-interactively: crust checks for a TTY, so pipe bytes through
|
|
181
|
+
`script` and pace them — `{ printf 'range(1,3)\n'; sleep 1; printf '\004'; } |
|
|
182
|
+
script -qec 'crust' /dev/null`.
|
|
183
|
+
|
|
184
|
+
## Where crust's globals exist
|
|
185
|
+
|
|
186
|
+
`GET`, `range`, `parallel`, `load`, `procs`, `$`, `crust.*` and everything you
|
|
187
|
+
define in `init.ts` are globals **wherever crust starts its runtime**: the REPL,
|
|
188
|
+
`-c`, piped stdin, a script file, `~/.config/crust/init.ts`, and any `.ts`/`.js`
|
|
189
|
+
file crust imports (`source hooks/warmup.ts` — inside it, `load` and `parallel`
|
|
190
|
+
are just there). A file run by plain `bun script.ts` never starts crust, so
|
|
191
|
+
those names are `undefined` there; import what you need from the modules
|
|
192
|
+
(`../src/sinks` today) or run the file under crust instead.
|
|
193
|
+
|
|
194
|
+
## Auto-discovered stages
|
|
195
|
+
|
|
196
|
+
crust scans your **global bun install** at startup and turns opted-in packages
|
|
197
|
+
into pipeline stages — no `init.ts` entry needed.
|
|
198
|
+
|
|
199
|
+
- Reads `$CRUST_GLOBAL_PREFIX/package.json` (default `~/.bun/install/global`) and
|
|
200
|
+
inspects its dependencies. A package with a `bin` field and **no** `crust`
|
|
201
|
+
field is skipped, so it keeps working as a plain shell command; add
|
|
202
|
+
`"crust": { … }` and it becomes a stage *as well as* a command.
|
|
203
|
+
- `package.json` → `"crust": { "stage": "default" }` (or a named export) picks
|
|
204
|
+
which export becomes the stage; otherwise the default export, or the sole
|
|
205
|
+
function export. The stage is called `fn(item, ...args)` like any registered
|
|
206
|
+
function, so it works as a source *and* mid-pipeline.
|
|
207
|
+
- Scoped packages are exposed after the `/` (`@me/shout` → `shout`). A name
|
|
208
|
+
already registered (by `crust.fn` in `init.ts`) wins — discovery never
|
|
209
|
+
overwrites explicit config.
|
|
210
|
+
- The scan is cached at `$CRUST_CACHE_DIR/globals.json` (default
|
|
211
|
+
`~/.cache/crust`) keyed on the prefix `package.json` mtime; delete that file
|
|
212
|
+
after changing the install by hand.
|
|
213
|
+
- `CRUST_DEBUG=1` prints every decision: `skipping "x" — has bin field`,
|
|
214
|
+
`failed to inspect …`, `name collision on …`. That's where to look when a
|
|
215
|
+
package you expected just isn't a stage.
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: crust-load-testing
|
|
3
|
+
description: Load-test and soak-test HTTP services with crust — the load rate source, parallel workers, timed verbs, stats windows, CI threshold gates via assert, and baseline comparison with stats --out. Use for smoke-load, latency percentiles, rps checks, performance regression gates in CI, or "is this endpoint fast/degrading" questions.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# crust load testing
|
|
7
|
+
|
|
8
|
+
One line gives you a paced load run with real percentiles and a CI-ready
|
|
9
|
+
exit code. Tick pacing batches due slots per wakeup, so the generator
|
|
10
|
+
sustains 5000+ ticks/s — your real ceiling is downstream (`parallel` pool ×
|
|
11
|
+
service time). Single-process smoke-load and soak tooling, not a
|
|
12
|
+
distributed rig.
|
|
13
|
+
|
|
14
|
+
## The shapes
|
|
15
|
+
|
|
16
|
+
```crust
|
|
17
|
+
# fixed volume: N requests, K at a time
|
|
18
|
+
range(0, 999) | parallel 50 | GET :3000/health | expect 200 | stats
|
|
19
|
+
|
|
20
|
+
# paced: arrival RATE for a DURATION (load 30s 100/s), concurrency capped by parallel
|
|
21
|
+
load 30s 100/s | parallel 50 | GET :3000/health | expect 200 | stats
|
|
22
|
+
|
|
23
|
+
# ramp: comma-separated phases, one stats stream across all of them
|
|
24
|
+
load 10s 50/s, 30s 200/s | parallel 100 | GET :3000/health | stats --every 5
|
|
25
|
+
|
|
26
|
+
# POST load: ticks are {n, phase, scheduledAt, lagMs} — build bodies from them
|
|
27
|
+
load 10s 20/s | (t => ({name: "user" + t.n})) | parallel 8 | POST :3000/users | expect 2xx | stats
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
- `parallel` puts ANY http verb in load mode: output is `{status, ms, url}`
|
|
31
|
+
timing records, bodies drained, network errors become `status: 0` records
|
|
32
|
+
(they show in the histogram instead of killing the run). So a run whose
|
|
33
|
+
every request was refused prints `{"status":{"0":40}}` and still exits 0
|
|
34
|
+
unless a gate is present — `expect 200` over a dead port exits 1. `stats`
|
|
35
|
+
reports, it does not gate.
|
|
36
|
+
- The summary carries failure tallies `timedOut` and `redirectRefused`
|
|
37
|
+
(always present, 0 on a clean run): a `--timeout` budget hit or a
|
|
38
|
+
`--redirect error` refusal is a `{status: 0, timedOut/redirectRefused:
|
|
39
|
+
true}` RECORD, not a failure exit. Gate them — `expect 200` (status 0
|
|
40
|
+
fails it) or `stats | assert (s => s.redirectRefused === 0)`; exit 0
|
|
41
|
+
alone is never request success in load mode.
|
|
42
|
+
- Rate (`load`) and concurrency (`parallel N`) are independent knobs.
|
|
43
|
+
Little's law sizes the pool: `N ≥ rate × p99-in-seconds`.
|
|
44
|
+
- Durations `ms|s|m`; rates `N/s` or `N/m`, decimals allowed.
|
|
45
|
+
|
|
46
|
+
## Honesty guarantees (read before quoting numbers)
|
|
47
|
+
|
|
48
|
+
- A gate that measured NOTHING fails: `stats` over an empty stream is tagged
|
|
49
|
+
`empty: true` and `assert` refuses it. `{count: 0, p95: 0}` used to satisfy
|
|
50
|
+
`s => s.p95 < 200`, so a run that issued zero requests reported success.
|
|
51
|
+
- Percentiles are bucketed (0.1ms below 100ms, 1ms below 1s) and report the
|
|
52
|
+
bucket's upper bound — never faster than reality. `count` and `meanMs` are
|
|
53
|
+
exact. Memory is constant, so a multi-hour soak is safe.
|
|
54
|
+
|
|
55
|
+
- `stats.rps` is MEASURED, never the target. If downstream saturates, stale
|
|
56
|
+
slots are skipped (never burst) and reported on stderr:
|
|
57
|
+
`load: target 3000 ticks — emitted 2868, dropped 132 … achieved 95.6/s`.
|
|
58
|
+
A drop report means the target rate was NOT sustained — raise `parallel N`
|
|
59
|
+
or lower the rate before quoting percentiles.
|
|
60
|
+
- Percentiles come only from timed records; they include body download (a
|
|
61
|
+
route that drips its body over 100ms reports p50 102, not 0.4).
|
|
62
|
+
- With `--every`, a slow request lands in the window it FINISHED in — and
|
|
63
|
+
because stats waits for a completion before it can flush, a window's
|
|
64
|
+
`wallMs` can exceed N (measured: `--every 2` → window 1 `wallMs: 3006`).
|
|
65
|
+
- `--out` takes `.json` only — any other extension is a loud error, never a
|
|
66
|
+
text file written under a json-looking name.
|
|
67
|
+
|
|
68
|
+
## CI gates — thresholds are assert composition
|
|
69
|
+
|
|
70
|
+
```crust
|
|
71
|
+
load 30s 100/s | parallel 50 | GET :3000/health | stats | assert (s => s.p95 < 200) | assert (s => s.rps > 80)
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
One assert per threshold: the failure names the predicate and prints the
|
|
75
|
+
actual summary, and the line exits 1 (`crust -c` stops there).
|
|
76
|
+
|
|
77
|
+
Baseline gate — `--out` writes a versioned artifact
|
|
78
|
+
(`{crustStats: 1, startedAt, urls, summary, windows?}`), an async assert
|
|
79
|
+
reads it back:
|
|
80
|
+
|
|
81
|
+
```crust
|
|
82
|
+
load 10s 100/s | parallel 50 | GET :3000/health | stats --out load/last.json | assert (async s => { const b = await Bun.file("load/baseline.json").json(); return s.p95 < 2 * b.summary.p95 })
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The artifact is written BEFORE the gate runs, so a failed gate still leaves
|
|
86
|
+
the file for CI upload.
|
|
87
|
+
|
|
88
|
+
Soak with fail-fast windows (`--every N` emits `{window: k, …}` deltas then
|
|
89
|
+
`{final: true, …}`; a bare predicate gates every one of them):
|
|
90
|
+
|
|
91
|
+
```crust
|
|
92
|
+
load 60s 25/s | parallel 25 | GET :3000/health | stats --every 5 | assert (s => !s.window || s.p95 < 400)
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Guard `!s.window ||` scopes a threshold to window objects, `!s.final ||` to
|
|
96
|
+
the final summary — and a guard needs the object it guards. Without `--every`
|
|
97
|
+
the plain summary carries neither key, so `!s.window` is true and `||`
|
|
98
|
+
short-circuits: the threshold never runs. `stats | assert (s => !s.window ||
|
|
99
|
+
s.p95 < 0.1)` exits 0 with p95 at 4.2. Drop the guard when you drop `--every`.
|
|
100
|
+
|
|
101
|
+
## Warmup
|
|
102
|
+
|
|
103
|
+
A separate line in the same script — its stats simply aren't gated:
|
|
104
|
+
|
|
105
|
+
```crust
|
|
106
|
+
range(0, 99) | parallel 10 | GET :3000/health | stats
|
|
107
|
+
load 30s 100/s | parallel 50 | GET :3000/health | stats | assert (s => s.p95 < 200)
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## TS API equivalent
|
|
111
|
+
|
|
112
|
+
The same surface as globals inside `init.ts` (and crust-run config code) —
|
|
113
|
+
`load`, `parallel`, `timedGet`, `timedHttpItem`, `statsStage`, `Pipeline`:
|
|
114
|
+
|
|
115
|
+
```ts
|
|
116
|
+
const out = await load([{ durMs: 10_000, rps: 100 }])
|
|
117
|
+
.pipe(parallel(50, timedGet("http://localhost:3000/health")))
|
|
118
|
+
.pipe(statsStage(undefined, "load/last.json"))
|
|
119
|
+
.collect();
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`timedHttpItem(method, url, opts?)` is the any-verb per-item timer when you
|
|
123
|
+
need POST bodies from upstream items.
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: crust-logs
|
|
3
|
+
description: Search a live log stream with crust's `logs` session — it holds a tail/procs/shell source open, keeps a ring buffer, and re-runs every typed pipeline fragment over the buffered past and then the live stream; `json on` parses NDJSON at query time, `search` counts the buffer instantly, `buffer N` resizes without restarting. Use when tailing dev services or containers, when a grep workflow keeps losing history, or when you want to widen a filter over logs you already saw.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# crust logs
|
|
7
|
+
|
|
8
|
+
## What it is
|
|
9
|
+
|
|
10
|
+
`logs <source>` keeps the source open and a ring buffer (default 10,000 items)
|
|
11
|
+
behind it, so one query can look at the PAST you already streamed and stay
|
|
12
|
+
attached to the live firehose. Every line you type at `logs>` is an ordinary
|
|
13
|
+
crust pipeline fragment — the whole shell grammar is the query language.
|
|
14
|
+
|
|
15
|
+
```crust
|
|
16
|
+
logs tail -n 0 -F app.log
|
|
17
|
+
logs procs({web: "bun run dev", api: "bun api.ts"})
|
|
18
|
+
logs docker logs -f my-container
|
|
19
|
+
logs --buffer 50000 tail -F big.log
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Boot prints what it holds: `logs: tail -n 0 -F app.log — buffering last 10000
|
|
23
|
+
items (`help` for commands, `exit` to leave)`.
|
|
24
|
+
|
|
25
|
+
## A fragment runs TWICE
|
|
26
|
+
|
|
27
|
+
Once over the buffer snapshot, then over the live stream until Ctrl-C, past a
|
|
28
|
+
`-- live --` marker. So side effects fire twice: append with `>>`, never `>`,
|
|
29
|
+
and a `POST` fragment re-sends buffered matches.
|
|
30
|
+
|
|
31
|
+
- **Ctrl-C once** ends the live view *gracefully* — the stream finishes, so
|
|
32
|
+
terminal stages flush (a query ending in `stats` prints its summary right
|
|
33
|
+
then; `wc -l` prints its count). **Ctrl-C twice** hard-cancels a stuck query.
|
|
34
|
+
The session survives both.
|
|
35
|
+
- A query that fails on the retro pass prints its error and re-prompts. Errors
|
|
36
|
+
never kill the session or the held source.
|
|
37
|
+
|
|
38
|
+
## Session commands
|
|
39
|
+
|
|
40
|
+
| command | what it does |
|
|
41
|
+
| --- | --- |
|
|
42
|
+
| `search <text>` | fixed substring over the BUFFER only — matches highlighted, and a count that always prints, zero included (`search: 5 matching item(s) of 36 buffered`). No live view, no Ctrl-C. For regex/case-folding/live, run a `grep` query |
|
|
43
|
+
| `json on` / `json off` | parse string items at QUERY time (see below) |
|
|
44
|
+
| `buffer` / `buffer N` | report usage (`buffer: 57/10000 items (pushed 57, evicted 0)`) or resize live, newest kept — widen a too-small window without losing the held source |
|
|
45
|
+
| `clear` | empty the buffer (`buffer cleared`) |
|
|
46
|
+
| `help` | the command list, including the double-run note |
|
|
47
|
+
| `exit` / Ctrl-D | tear the source down and leave (procs get SIGTERM→SIGKILL) |
|
|
48
|
+
|
|
49
|
+
Query errors print and re-prompt; up-arrow recalls earlier queries.
|
|
50
|
+
|
|
51
|
+
## json on — and the silent zero it prevents
|
|
52
|
+
|
|
53
|
+
With `json off` (the default) items are the raw strings. A lambda that reads
|
|
54
|
+
fields therefore matches NOTHING, quietly: 50 buffered NDJSON lines, and
|
|
55
|
+
`filter (e => e.level >= 30) | wc -l` reported `0`. `json on` fixes that at
|
|
56
|
+
query time: a string item parsing to a JSON **object or array** reaches your
|
|
57
|
+
lambdas parsed; plain text, strings that parse to primitives (`"42"` stays a
|
|
58
|
+
string), and non-string items flow through unchanged and are counted out loud.
|
|
59
|
+
The ring stores raw items, so `json off` is a lossless revert.
|
|
60
|
+
|
|
61
|
+
When the buffer looks like NDJSON the session prints, once, on your first
|
|
62
|
+
*pipeline fragment* (not on `help` or `search`):
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
hint: buffer looks like NDJSON — `json on` parses object lines at query time (`json off` reverts)
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
It never auto-enables — changing the item type under a lambda you already typed
|
|
69
|
+
would be a false picture.
|
|
70
|
+
|
|
71
|
+
## Item types by source
|
|
72
|
+
|
|
73
|
+
- `tail`/shell sources: plain **strings**.
|
|
74
|
+
- `procs({...})`: `{proc, stream, line}` **objects** — the session tells you so
|
|
75
|
+
at boot (`logs: items are {proc, stream, line} objects — try (l => l.line) or
|
|
76
|
+
filter (l => l.proc === "web")`). `(l => l.line)` extracts text,
|
|
77
|
+
`filter (l => l.proc === "web")` selects one process, and `search` still works
|
|
78
|
+
(it matches each item's rendered form).
|
|
79
|
+
- Pretty-printing is just another stage: `grep api_request | pino-pretty
|
|
80
|
+
--colorize --singleLine` (`--colorize` forces ANSI through the pipe; local
|
|
81
|
+
`node_modules/.bin` is on PATH). Normalize `procs` objects first.
|
|
82
|
+
|
|
83
|
+
## Traps
|
|
84
|
+
|
|
85
|
+
- `logs` is interactive-only. No tty → exit 2 and a pointer:
|
|
86
|
+
`logs: interactive session needs a tty — for piped data use `cmd | crust -c 'stdin | …'``.
|
|
87
|
+
For piped data there is no session — build the pipeline directly.
|
|
88
|
+
- The `logs` line owns its line: `logs tail -F app.log | grep ERROR` refuses
|
|
89
|
+
(`logs is a crust builtin — it runs in-process, so sh cannot run it`) rather
|
|
90
|
+
than letting sh report `command not found`. Type the filter at `logs>`.
|
|
91
|
+
- A gate query (`count 1`, `expect 200`) runs against the BUFFER first — a
|
|
92
|
+
retro failure prints before a single live item arrives. That is the honest
|
|
93
|
+
order, but it means `logs`+gate reflects history, not just what follows.
|
|
94
|
+
- `search` is buffer-only and fixed-substring: it never sees live items and
|
|
95
|
+
never interprets regex.
|
|
96
|
+
- Ctrl-C at an idle `logs>` prompt is nothing: it prints `^C` and re-prompts,
|
|
97
|
+
the source keeps running. `exit` (or Ctrl-D) is the teardown — verified with a
|
|
98
|
+
`procs` source, including after a Ctrl-C: the group gets SIGTERM→SIGKILL and
|
|
99
|
+
no `sleep` survives the session.
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: crust-mcp
|
|
3
|
+
description: Drive crust directly from an MCP-capable agent (OpenCode, Claude Code, Codex, Cursor) instead of a shell — `crust mcp` exposes every capability as a first-class tool: run_line, check_line, test_fixture/test_pipes, gen_fixtures, load_run, verify_web_links, plus lifecycle mock servers, supervised procs, and buffered log sessions that stay open across calls. Use when configuring crust as an MCP server, choosing which crust_* tool fits a task, or reasoning about what persists between tool calls and what does not.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# crust over MCP
|
|
7
|
+
|
|
8
|
+
`crust mcp` speaks JSON-RPC 2.0 over stdio — one message per line. It is a
|
|
9
|
+
whole-process MODE, not a REPL builtin: the MCP host owns stdin and stdout.
|
|
10
|
+
|
|
11
|
+
## Install & configure
|
|
12
|
+
|
|
13
|
+
`crust` must be on PATH (either channel works — same binary):
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
curl -fsSL https://raw.githubusercontent.com/lariocpt/crust/main/install.sh | bash
|
|
17
|
+
# or: npm i -g @lariocpt/crust
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Then register it with the agent host — e.g. OpenCode / Claude Code style:
|
|
21
|
+
|
|
22
|
+
```json
|
|
23
|
+
{ "mcpServers": { "crust": { "command": "crust", "args": ["mcp"] } } }
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
VS Code uses a DIFFERENT schema in `.vscode/mcp.json` (workspace) or the
|
|
27
|
+
user-level MCP settings — the `mcpServers` shape above silently fails there
|
|
28
|
+
("command is required" or the server just never appears):
|
|
29
|
+
|
|
30
|
+
```json
|
|
31
|
+
{
|
|
32
|
+
"servers": {
|
|
33
|
+
"crust": { "type": "stdio", "command": "/home/user/.local/bin/crust", "args": ["mcp"] }
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Two traps, both verified: (1) `"command"` must be the ABSOLUTE path — a
|
|
39
|
+
GUI-launched VS Code does not inherit the shell's PATH, so bare `"crust"`
|
|
40
|
+
dies `spawn crust ENOENT` if the launcher lives outside the default
|
|
41
|
+
lookup dirs; (2) VS Code's MCP support ships inside the Chat/Copilot Chat
|
|
42
|
+
extension — Code - OSS / VSCodium has no MCP host without an extension
|
|
43
|
+
(Cline, Roo Code or Continue each carry their own MCP settings).
|
|
44
|
+
|
|
45
|
+
Grammar lands automatically: the one-shot tools pass line/fixture text to a
|
|
46
|
+
fresh crust child untouched, so anything the lexer accepts (`--redirect`,
|
|
47
|
+
new builtins) is an MCP tool call the moment the installed binary has it —
|
|
48
|
+
no MCP-side change per feature.
|
|
49
|
+
|
|
50
|
+
Verify by hand — the transport is just lines on stdin:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}\n{"jsonrpc":"2.0","method":"notifications/initialized"}\n{"jsonrpc":"2.0","id":2,"method":"tools/list"}\n' | crust mcp
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Two execution planes — what persists
|
|
57
|
+
|
|
58
|
+
| Plane | Tools | State |
|
|
59
|
+
| --- | --- | --- |
|
|
60
|
+
| **One-shot** | `crust_run_line`, `crust_check_line`, `crust_test_fixture`, `crust_test_pipes`, `crust_gen_fixtures`, `crust_verify_web_links`, `crust_load_run` | a fresh crust CHILD per call — `cwd`/`env` apply per call and NEVER carry over; pass them explicitly |
|
|
61
|
+
| **Lifecycle** | `crust_mock_server_start/stop/list/violations`, `crust_procs_start/stop`, `crust_logs_open/query/close` | held IN the server across calls; every held job dies when the server exits |
|
|
62
|
+
|
|
63
|
+
`crust_run_line` runs ordinary crust grammar — one line, `|`-stages, lambdas:
|
|
64
|
+
|
|
65
|
+
```crust
|
|
66
|
+
range(1,5) | (n => n * n) | stats
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Honest failure — never a false pass over MCP
|
|
70
|
+
|
|
71
|
+
A nonzero child exit, a killed timeout, a failed `assert`, a mock validation
|
|
72
|
+
error is an **`isError` result** carrying the verbatim exit code and output.
|
|
73
|
+
Zero matches is answered with an explicit "zero items", never silence. Trust
|
|
74
|
+
`isError`; the text is evidence, not a verdict.
|
|
75
|
+
|
|
76
|
+
## Picking the right tool
|
|
77
|
+
|
|
78
|
+
- Validate generated crust first — `crust_check_line` parses without running
|
|
79
|
+
(files and ports don't need to exist).
|
|
80
|
+
- `crust_procs_start` supervises dev services (env/after/ready/restart per
|
|
81
|
+
proc). Its stream is queried through `crust_logs_query` — items are
|
|
82
|
+
`{proc, stream, line}` objects. Gate readiness in a follow-up:
|
|
83
|
+
|
|
84
|
+
```crust
|
|
85
|
+
wait :3000/health
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
- `crust_logs_open` holds a tail/shell/`procs({…})` source with a ring buffer;
|
|
89
|
+
`crust_logs_query` runs a pipeline fragment over the buffered past, then
|
|
90
|
+
live for `live_ms` — `json: true` parses NDJSON items before the fragment:
|
|
91
|
+
|
|
92
|
+
```crust
|
|
93
|
+
filter (l => l.proc === "api") | (l => l.line) | grep -i timeout
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
- `crust_mock_server_start` defaults `port: 0` (OS-assigned) — always take
|
|
97
|
+
the returned `url`. `proxy: "<upstream>"` turns it into a conformance audit;
|
|
98
|
+
read findings with `crust_mock_server_violations`.
|
|
99
|
+
- `crust_load_run` composes the documented pipeline (`load … | parallel … |
|
|
100
|
+
… | stats | assert`); an `assert` failure is `isError` — a run that
|
|
101
|
+
measured nothing is refused (`empty`), never a green gate.
|
|
102
|
+
|
|
103
|
+
## Limits, stated
|
|
104
|
+
|
|
105
|
+
- One tool call runs at a time (the interrupt bus is process-global).
|
|
106
|
+
- Output is tail-capped per stream and truncation is always announced.
|
|
107
|
+
- `crust_procs_stop` reports "teardown still in flight past 10s" honestly
|
|
108
|
+
when a child ignores SIGTERM — crust kills it on server exit.
|