@lariocpt/crust-mongo 0.0.0-stage → 0.3.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,215 @@
1
+ ---
2
+ name: crust-cli
3
+ description: How to invoke crust and read its answer — the run modes (`-c`, script file, piped stdin, REPL, tool subcommands like `crust skills install`), what each exit code actually means (0/1/2/127/130, and why there is no 124), `--check` as a linter including what it cannot catch, `--env-file` vs `dotenv` vs `export`, the REPL's verified keybindings and Ctrl-C semantics, and where crust's globals (`GET`, `range`, `parallel`, `crust.fn`) do and do not exist. Use when a crust run's exit code is confusing, when a CI step passes on a failing line, when `--check` says ok but the line dies, when `GET is not defined` appears in a `.ts` file, or when driving the REPL from a script.
4
+ ---
5
+
6
+ # Running crust, and reading what it says back
7
+
8
+ ## Run modes
9
+
10
+ | Invocation | What happens |
11
+ | --- | --- |
12
+ | `crust` (stdin is a TTY) | Interactive REPL. |
13
+ | `crust` (stdin is piped) | Reads **lines of crust** from stdin to EOF and runs them like a script (fail-fast, same exit codes). stdin is the *command* source in this mode, so a shell stage inside those lines sees EOF. Want data on stdin? Use `-c`. |
14
+ | `crust run.crust [args…]` | Script mode: blank lines and `#` comments are skipped (so a `#!/usr/bin/env crust` shebang works), lines run in order, and it **fails fast** — the first failing line stops the file and becomes the process exit code. A line ending in `\` continues on the next. The extension doesn't matter. Extra arguments are the script's **positional arguments** — `$0` is the path, `$1`…`$9` the rest, `$#` the count, `$@`/`$*` all of them joined — expanding wherever an env var does (URLs, JSON literals, `-H` strings, fn args, `stats --out`), and empty, never literal, outside script mode. A leading `--` after the path is a separator, not an argument. An unreadable path is 127. |
15
+ | `crust -c '<line>'` | One line. A shell stage inherits the process stdin, so `docker logs app \| crust -c 'grep ERROR'` really does grep the logs. To feed a **crust** pipeline, name the source: `docker logs -f app \| crust -c 'stdin \| grep ERROR'`. To run several lines, put a **newline inside the one argument** — they then behave like a script (fail-fast) — or continue one line with a trailing `\`. Two lines as two arguments is refused. Quote the whole pipeline: `-c range(1,2) extra` splits into three argv words and is refused. |
16
+ | `crust <tool> [args…]` | A **tool builtin as a subcommand**: `crust skills install`, `crust mock-server openapi.yaml -p4000`, `crust test-fixture 'fixtures/*.crust.ts' -j8`. Same flags as the REPL line, and `crust <tool> --help` prints them. The names that work are the ones needing no live session: `skills`, `test-fixture`, `test-pipes`, `gen-fixtures`, `mock-server`, `verify-web-links`. The **session** builtins (`cd`, `export`, `alias`, `unalias`, `source`, `dotenv`, `exit`, `help`, `history`) are refused rather than silently doing nothing — a one-shot process has no session to change. `logs` is refused too, for a different reason: its argument is the rest of the line **verbatim** (a `procs({...})` literal), which argv cannot carry. Each refusal exits 2 and names the builtin, instead of the `cannot read cd` a fallthrough to script mode produced. A file that exists wins when it is the only argument, so `crust skills` runs `./skills`; any extra argument selects the tool. |
17
+ | `crust --check '<line>'` | Parse without running. See below. |
18
+ | `crust --env-file <path> …` | Loads `.env` before *any* run mode and before `init.ts`. The "loaded" note goes to stderr so a `-c` pipeline's stdout stays clean. A missing file exits **2, loudly** — this flag exists to replace shell shims that failed silently. Does not combine with `--check`. |
19
+ | `crust -h` / `-V` | Usage / version, exit 0. |
20
+
21
+ ## Exit codes (measured against the binary, not guessed)
22
+
23
+ | Code | Meaning | Examples |
24
+ | --- | --- | --- |
25
+ | 0 | Success. | `range(1,2)` |
26
+ | 1 | **crust** failed. Every error crust itself raises. | `assert` mismatch, a lambda that throws, `expect 200` vs a 500, `wait: <target> not ready after Ns`, a `GET` that can't connect, `read` on a missing file, a refused empty stage (`echo a \|\| echo b`), a builtin name that reached sh. |
27
+ | 2 | Invocation or syntax error — nothing ran. | `crust --nonsense`, `--check` with no line, `--env-file` on a missing file, `-c` given two lines, `mock-server --bogus` (prints the valid flags), sh's own parse errors on a pure-shell line (`bundle(5)`) — and the same parse error on any shell stage of a **mixed** line (named; before this a bad stage N surfaced only when its child exited, after stages 1..N-1 had run). |
28
+ | the stage's own | A shell stage's exit code, propagated. | `range(1,3) \| sh -c "exit 9"` → 9 |
29
+ | 127 | Command not found (shell stage), or an unreadable script file. | `nosuchbinary-xyz` |
30
+ | 130 / 143 | crust killed by SIGINT / SIGTERM. | Ctrl-C during a long `-c` run |
31
+
32
+ **A builtin name is not a missing binary.** `crust verify-web-links` with no
33
+ arguments exits **2** with the tool's own usage, and `crust skills install
34
+ --bogus` exits **2** naming the valid flags. 127 means a *script path* could not
35
+ be read, or sh could not find a command — if a builtin name produces 127, the
36
+ binary really is an older one that treated every positional as a path.
37
+
38
+ **There is no crust exit 124.** `wait` timing out is exit **1** with
39
+ `crust: wait: <target> not ready after Ns`. If a probe reports 124, then
40
+ `timeout(1)` — your harness — killed crust. Give `wait`/`load`/`procs` probes a
41
+ generous outer timeout, and when testing cancellation signal the real bun PID.
42
+
43
+ A **bare `wait :3000` probes `/`**, and readiness means a **2xx** — a mock server
44
+ or an API with no root route 404s forever, so the line fails after the full
45
+ timeout on a server that is perfectly up. Use a real health path (`wait
46
+ :3000/health`) or `wait port:3000` for a TCP connect; crust says so in the
47
+ failure message now, because "not ready" alone reads like a slow boot.
48
+
49
+ ### The last command wins (the one CI trap here)
50
+
51
+ A line made **entirely of shell stages** is handed to sh as one `sh -c` string,
52
+ so its code is the **last** command's — no `pipefail`:
53
+
54
+ ```bash
55
+ ls /nope | wc -l # 0 — wc succeeded
56
+ ls /nope # 2
57
+ ```
58
+
59
+ The moment a line contains a crust stage, crust owns the exit code and a
60
+ failing shell stage propagates. So a gate must **end on a crust stage**:
61
+
62
+ ```crust
63
+ # a run.crust: blanks and `#` are skipped, lines run in order, first failure stops the file
64
+ wait :3000/health
65
+ range(1,99) | parallel 10 | GET http://localhost:3000/items | expect 200 | stats
66
+ GET http://localhost:3000/items | assert (x => x.length > 0)
67
+ # a trailing `\` continues the line — in a script, in piped stdin, in a
68
+ # multi-line -c, or at the REPL (where an unbalanced quote or ( continues too)
69
+ load 10s 100/s | parallel 20 | GET http://localhost:3000/items \
70
+ | expect 200 | stats
71
+ ```
72
+
73
+ Or ask sh to gate, explicitly:
74
+
75
+ ```bash
76
+ sh -c 'set -o pipefail; ls /nope | wc -l' # 2 — would be 0 without pipefail
77
+ ```
78
+
79
+ ## `--check`: the linter, and its edges
80
+
81
+ `crust --check '<line>'` exits **0** if every line parses, **1** with the error
82
+ otherwise (`ok: N line(s) parse`). It splits on newlines and skips blank and
83
+ `#` lines, so a whole fenced block goes in as one argument. It checks three
84
+ things: crust's own grammar, a builtin-head line against **that CLI's flag
85
+ spec**, and every shell stage through `sh -n` (noexec — `cat > f` under `-n`
86
+ does not create `f`).
87
+
88
+ ```bash
89
+ crust --check 'read /nonexistent/*.json | POST :3000/x' # ok — sources are lazy
90
+ ```
91
+
92
+ ```
93
+ crust: shell stage does not parse (stage: head -(): sh: -c: line 1: syntax error near unexpected token `(`
94
+ in: range(1,3) | head -(
95
+ ```
96
+
97
+ The runtime runs the same preflight before executing any stage of a **mixed**
98
+ line, so a shell stage that does not parse exits 2 with this message and
99
+ nothing has run (measured pre-fix: `touch f | base64 | head -(` exited 2 and
100
+ `f` existed). A pure-shell line is one `sh -c`, which sh parses the whole line
101
+ for before executing anything, so its own stderr stays the answer there — and
102
+ 127 (command not found) is a runtime error the preflight cannot see.
103
+
104
+ What it does **not** check: anything needing I/O or a live process — a missing
105
+ fixture path, an unreachable URL, a wrong `output` matcher key, a semantic
106
+ refusal at runtime (`--env-file` + `--check` is refused for exactly this
107
+ reason). "Checks clean" means *parses*, never *will pass*.
108
+
109
+ ## Environment
110
+
111
+ | How | Scope | Reaches sh stages? |
112
+ | --- | --- | --- |
113
+ | `crust --env-file <path> …` | one process, before `init.ts` | yes |
114
+ | `dotenv <path>` / `dotenv --append` / `dotenv status` / `dotenv clear` | the session | yes |
115
+ | `export FOO=bar` | the session | yes |
116
+ | a bare `FOO=bar` line | the session — the same code as `export`, including `+=` | yes |
117
+ | `capture NAME` stage | writes `process.env.NAME` from the stream, for **later lines** | yes |
118
+
119
+ ## The REPL
120
+
121
+ Verified in a real pty, not read off the source:
122
+
123
+ - **Editing:** `Ctrl-A` / `Ctrl-E` home/end, arrow keys move (and `Home`/`End`/
124
+ `Delete`), `Ctrl-K` kill to end, `Ctrl-U` kill to start, `Ctrl-W` kill
125
+ previous word, `Ctrl-L` repaint (the buffer survives), `Backspace`, `Tab`
126
+ completes.
127
+ - **Tab** completes a **path** mid-stage (`ls sr` + Tab → `ls src/`) and lists
128
+ names from `$PATH` at the start of a stage. It does **not** know crust's own
129
+ builtins or registered functions — `verify-web-` + Tab lists nothing useful.
130
+ - **History:** up/down arrows. It is plain text at
131
+ `$XDG_DATA_HOME/crust/history` (default `~/.local/share/crust/history`), it
132
+ survives restarts, `history` prints it numbered, and re-running a recalled
133
+ line does not append a duplicate.
134
+ - **`Ctrl-R`** searches that history as you type: a `(crust-search):` line opens
135
+ below the one you are editing, showing the newest entry containing your query;
136
+ another `Ctrl-R` steps further back, `Backspace` shortens the query. `Enter`
137
+ **accepts** — the match lands on the prompt line, editable, and the **next**
138
+ `Enter` runs it (so a search at a `…` continuation prompt cannot lose the
139
+ pending line). `Ctrl-G` / `Esc` / `Ctrl-C` cancel and restore the draft
140
+ byte-exact; `Ctrl-C` mid-search does **not** clear the line. No match paints
141
+ `-- no match --` and `Enter` inserts the query as text — it never runs
142
+ nothing, and it never rewrites the history file.
143
+ - **`Ctrl-C`** at the prompt just clears the line. Mid-run it **cancels**:
144
+ children are killed, the line reports 130, and the prompt comes back so the
145
+ session continues. One item already produced by the source can still print
146
+ after the prompt redraws — cosmetic; the source is stopped.
147
+ - **`Ctrl-D`** (or `exit`) leaves; the exit code is 0 unless `exit <code>`.
148
+ - **`Ctrl-Z`** suspends what the line is waiting on — a crust pipeline, a
149
+ builtin, a `fg`/`wait` — as a job: SIGSTOP, `[2]+ Stopped sleep 12`, prompt
150
+ back, `bg` resumes a suspended pipeline as one unit. During a **foreground
151
+ inherit-stdio child** it does nothing at all (see Job control below).
152
+
153
+ ## Job control
154
+
155
+ `cmd &` is crust's own background job: its own process group (so a signal
156
+ reaches a forking `sh`'s whole tree), stdin `/dev/null`, output still on the
157
+ terminal, and it **outlives crust**. `%1` / `%` / `%+` name jobs; a bare number
158
+ names a job to `fg`/`bg`/`jobs` and a **PID** to `kill`.
159
+
160
+ | Line | What it does |
161
+ | --- | --- |
162
+ | `jobs`, `jobs -l` | the table; `-l` adds `(pid …)` and the code once there is one; an unknown option is refused, not ignored |
163
+ | `fg [spec]` | resume (if stopped) and **wait**; the job's own code becomes the line's |
164
+ | `bg [spec]` | resume a stopped job; an error if it was not stopped |
165
+ | `wait %1` | the job wait — **bare `wait` is the readiness stage**, `wait :3000/health` |
166
+ | `kill %1`, `kill -9 %1` | prints `Sent SIGTERM to job 1 (pid …)` — the send, not the death. `kill -9 4242` is sh's |
167
+
168
+ Statuses are `Running`, `Stopped`, `Done`, `Exit N`, `Terminated` (128+signal),
169
+ `Failed` (gone with no code — never a silent pass). Finished jobs stay listed for
170
+ the session, which is how `wait %1` still reports the code of a job that ended
171
+ minutes ago. `Ctrl-C` during `fg`/`wait` kills the job and returns 130.
172
+
173
+ SIGSTOP, never SIGTSTP: Bun's runtime ignores SIGTSTP and a child it spawns
174
+ inherits that ignore across `exec`, so **Ctrl-Z cannot stop a foreground
175
+ inherit-stdio child** (`vim`, a bare `sleep 20`) — the `^Z` echoes and the child
176
+ keeps running; Ctrl-C is the escape (child dies, line 130). Running jobs are left
177
+ alone at exit because outliving the session is what `&` means; **stopped** ones are
178
+ resumed with a notice, since SIGSTOP survives reparenting and no shell resumes a
179
+ process it did not stop.
180
+ Driving it non-interactively: crust checks for a TTY, so pipe bytes through
181
+ `script` and pace them — `{ printf 'range(1,3)\n'; sleep 1; printf '\004'; } |
182
+ script -qec 'crust' /dev/null`.
183
+
184
+ ## Where crust's globals exist
185
+
186
+ `GET`, `range`, `parallel`, `load`, `procs`, `$`, `crust.*` and everything you
187
+ define in `init.ts` are globals **wherever crust starts its runtime**: the REPL,
188
+ `-c`, piped stdin, a script file, `~/.config/crust/init.ts`, and any `.ts`/`.js`
189
+ file crust imports (`source hooks/warmup.ts` — inside it, `load` and `parallel`
190
+ are just there). A file run by plain `bun script.ts` never starts crust, so
191
+ those names are `undefined` there; import what you need from the modules
192
+ (`../src/sinks` today) or run the file under crust instead.
193
+
194
+ ## Auto-discovered stages
195
+
196
+ crust scans your **global bun install** at startup and turns opted-in packages
197
+ into pipeline stages — no `init.ts` entry needed.
198
+
199
+ - Reads `$CRUST_GLOBAL_PREFIX/package.json` (default `~/.bun/install/global`) and
200
+ inspects its dependencies. A package with a `bin` field and **no** `crust`
201
+ field is skipped, so it keeps working as a plain shell command; add
202
+ `"crust": { … }` and it becomes a stage *as well as* a command.
203
+ - `package.json` → `"crust": { "stage": "default" }` (or a named export) picks
204
+ which export becomes the stage; otherwise the default export, or the sole
205
+ function export. The stage is called `fn(item, ...args)` like any registered
206
+ function, so it works as a source *and* mid-pipeline.
207
+ - Scoped packages are exposed after the `/` (`@me/shout` → `shout`). A name
208
+ already registered (by `crust.fn` in `init.ts`) wins — discovery never
209
+ overwrites explicit config.
210
+ - The scan is cached at `$CRUST_CACHE_DIR/globals.json` (default
211
+ `~/.cache/crust`) keyed on the prefix `package.json` mtime; delete that file
212
+ after changing the install by hand.
213
+ - `CRUST_DEBUG=1` prints every decision: `skipping "x" — has bin field`,
214
+ `failed to inspect …`, `name collision on …`. That's where to look when a
215
+ package you expected just isn't a stage.
@@ -0,0 +1,123 @@
1
+ ---
2
+ name: crust-load-testing
3
+ description: Load-test and soak-test HTTP services with crust — the load rate source, parallel workers, timed verbs, stats windows, CI threshold gates via assert, and baseline comparison with stats --out. Use for smoke-load, latency percentiles, rps checks, performance regression gates in CI, or "is this endpoint fast/degrading" questions.
4
+ ---
5
+
6
+ # crust load testing
7
+
8
+ One line gives you a paced load run with real percentiles and a CI-ready
9
+ exit code. Tick pacing batches due slots per wakeup, so the generator
10
+ sustains 5000+ ticks/s — your real ceiling is downstream (`parallel` pool ×
11
+ service time). Single-process smoke-load and soak tooling, not a
12
+ distributed rig.
13
+
14
+ ## The shapes
15
+
16
+ ```crust
17
+ # fixed volume: N requests, K at a time
18
+ range(0, 999) | parallel 50 | GET :3000/health | expect 200 | stats
19
+
20
+ # paced: arrival RATE for a DURATION (load 30s 100/s), concurrency capped by parallel
21
+ load 30s 100/s | parallel 50 | GET :3000/health | expect 200 | stats
22
+
23
+ # ramp: comma-separated phases, one stats stream across all of them
24
+ load 10s 50/s, 30s 200/s | parallel 100 | GET :3000/health | stats --every 5
25
+
26
+ # POST load: ticks are {n, phase, scheduledAt, lagMs} — build bodies from them
27
+ load 10s 20/s | (t => ({name: "user" + t.n})) | parallel 8 | POST :3000/users | expect 2xx | stats
28
+ ```
29
+
30
+ - `parallel` puts ANY http verb in load mode: output is `{status, ms, url}`
31
+ timing records, bodies drained, network errors become `status: 0` records
32
+ (they show in the histogram instead of killing the run). So a run whose
33
+ every request was refused prints `{"status":{"0":40}}` and still exits 0
34
+ unless a gate is present — `expect 200` over a dead port exits 1. `stats`
35
+ reports, it does not gate.
36
+ - The summary carries failure tallies `timedOut` and `redirectRefused`
37
+ (always present, 0 on a clean run): a `--timeout` budget hit or a
38
+ `--redirect error` refusal is a `{status: 0, timedOut/redirectRefused:
39
+ true}` RECORD, not a failure exit. Gate them — `expect 200` (status 0
40
+ fails it) or `stats | assert (s => s.redirectRefused === 0)`; exit 0
41
+ alone is never request success in load mode.
42
+ - Rate (`load`) and concurrency (`parallel N`) are independent knobs.
43
+ Little's law sizes the pool: `N ≥ rate × p99-in-seconds`.
44
+ - Durations `ms|s|m`; rates `N/s` or `N/m`, decimals allowed.
45
+
46
+ ## Honesty guarantees (read before quoting numbers)
47
+
48
+ - A gate that measured NOTHING fails: `stats` over an empty stream is tagged
49
+ `empty: true` and `assert` refuses it. `{count: 0, p95: 0}` used to satisfy
50
+ `s => s.p95 < 200`, so a run that issued zero requests reported success.
51
+ - Percentiles are bucketed (0.1ms below 100ms, 1ms below 1s) and report the
52
+ bucket's upper bound — never faster than reality. `count` and `meanMs` are
53
+ exact. Memory is constant, so a multi-hour soak is safe.
54
+
55
+ - `stats.rps` is MEASURED, never the target. If downstream saturates, stale
56
+ slots are skipped (never burst) and reported on stderr:
57
+ `load: target 3000 ticks — emitted 2868, dropped 132 … achieved 95.6/s`.
58
+ A drop report means the target rate was NOT sustained — raise `parallel N`
59
+ or lower the rate before quoting percentiles.
60
+ - Percentiles come only from timed records; they include body download (a
61
+ route that drips its body over 100ms reports p50 102, not 0.4).
62
+ - With `--every`, a slow request lands in the window it FINISHED in — and
63
+ because stats waits for a completion before it can flush, a window's
64
+ `wallMs` can exceed N (measured: `--every 2` → window 1 `wallMs: 3006`).
65
+ - `--out` takes `.json` only — any other extension is a loud error, never a
66
+ text file written under a json-looking name.
67
+
68
+ ## CI gates — thresholds are assert composition
69
+
70
+ ```crust
71
+ load 30s 100/s | parallel 50 | GET :3000/health | stats | assert (s => s.p95 < 200) | assert (s => s.rps > 80)
72
+ ```
73
+
74
+ One assert per threshold: the failure names the predicate and prints the
75
+ actual summary, and the line exits 1 (`crust -c` stops there).
76
+
77
+ Baseline gate — `--out` writes a versioned artifact
78
+ (`{crustStats: 1, startedAt, urls, summary, windows?}`), an async assert
79
+ reads it back:
80
+
81
+ ```crust
82
+ load 10s 100/s | parallel 50 | GET :3000/health | stats --out load/last.json | assert (async s => { const b = await Bun.file("load/baseline.json").json(); return s.p95 < 2 * b.summary.p95 })
83
+ ```
84
+
85
+ The artifact is written BEFORE the gate runs, so a failed gate still leaves
86
+ the file for CI upload.
87
+
88
+ Soak with fail-fast windows (`--every N` emits `{window: k, …}` deltas then
89
+ `{final: true, …}`; a bare predicate gates every one of them):
90
+
91
+ ```crust
92
+ load 60s 25/s | parallel 25 | GET :3000/health | stats --every 5 | assert (s => !s.window || s.p95 < 400)
93
+ ```
94
+
95
+ Guard `!s.window ||` scopes a threshold to window objects, `!s.final ||` to
96
+ the final summary — and a guard needs the object it guards. Without `--every`
97
+ the plain summary carries neither key, so `!s.window` is true and `||`
98
+ short-circuits: the threshold never runs. `stats | assert (s => !s.window ||
99
+ s.p95 < 0.1)` exits 0 with p95 at 4.2. Drop the guard when you drop `--every`.
100
+
101
+ ## Warmup
102
+
103
+ A separate line in the same script — its stats simply aren't gated:
104
+
105
+ ```crust
106
+ range(0, 99) | parallel 10 | GET :3000/health | stats
107
+ load 30s 100/s | parallel 50 | GET :3000/health | stats | assert (s => s.p95 < 200)
108
+ ```
109
+
110
+ ## TS API equivalent
111
+
112
+ The same surface as globals inside `init.ts` (and crust-run config code) —
113
+ `load`, `parallel`, `timedGet`, `timedHttpItem`, `statsStage`, `Pipeline`:
114
+
115
+ ```ts
116
+ const out = await load([{ durMs: 10_000, rps: 100 }])
117
+ .pipe(parallel(50, timedGet("http://localhost:3000/health")))
118
+ .pipe(statsStage(undefined, "load/last.json"))
119
+ .collect();
120
+ ```
121
+
122
+ `timedHttpItem(method, url, opts?)` is the any-verb per-item timer when you
123
+ need POST bodies from upstream items.
@@ -0,0 +1,99 @@
1
+ ---
2
+ name: crust-logs
3
+ description: Search a live log stream with crust's `logs` session — it holds a tail/procs/shell source open, keeps a ring buffer, and re-runs every typed pipeline fragment over the buffered past and then the live stream; `json on` parses NDJSON at query time, `search` counts the buffer instantly, `buffer N` resizes without restarting. Use when tailing dev services or containers, when a grep workflow keeps losing history, or when you want to widen a filter over logs you already saw.
4
+ ---
5
+
6
+ # crust logs
7
+
8
+ ## What it is
9
+
10
+ `logs <source>` keeps the source open and a ring buffer (default 10,000 items)
11
+ behind it, so one query can look at the PAST you already streamed and stay
12
+ attached to the live firehose. Every line you type at `logs>` is an ordinary
13
+ crust pipeline fragment — the whole shell grammar is the query language.
14
+
15
+ ```crust
16
+ logs tail -n 0 -F app.log
17
+ logs procs({web: "bun run dev", api: "bun api.ts"})
18
+ logs docker logs -f my-container
19
+ logs --buffer 50000 tail -F big.log
20
+ ```
21
+
22
+ Boot prints what it holds: `logs: tail -n 0 -F app.log — buffering last 10000
23
+ items (`help` for commands, `exit` to leave)`.
24
+
25
+ ## A fragment runs TWICE
26
+
27
+ Once over the buffer snapshot, then over the live stream until Ctrl-C, past a
28
+ `-- live --` marker. So side effects fire twice: append with `>>`, never `>`,
29
+ and a `POST` fragment re-sends buffered matches.
30
+
31
+ - **Ctrl-C once** ends the live view *gracefully* — the stream finishes, so
32
+ terminal stages flush (a query ending in `stats` prints its summary right
33
+ then; `wc -l` prints its count). **Ctrl-C twice** hard-cancels a stuck query.
34
+ The session survives both.
35
+ - A query that fails on the retro pass prints its error and re-prompts. Errors
36
+ never kill the session or the held source.
37
+
38
+ ## Session commands
39
+
40
+ | command | what it does |
41
+ | --- | --- |
42
+ | `search <text>` | fixed substring over the BUFFER only — matches highlighted, and a count that always prints, zero included (`search: 5 matching item(s) of 36 buffered`). No live view, no Ctrl-C. For regex/case-folding/live, run a `grep` query |
43
+ | `json on` / `json off` | parse string items at QUERY time (see below) |
44
+ | `buffer` / `buffer N` | report usage (`buffer: 57/10000 items (pushed 57, evicted 0)`) or resize live, newest kept — widen a too-small window without losing the held source |
45
+ | `clear` | empty the buffer (`buffer cleared`) |
46
+ | `help` | the command list, including the double-run note |
47
+ | `exit` / Ctrl-D | tear the source down and leave (procs get SIGTERM→SIGKILL) |
48
+
49
+ Query errors print and re-prompt; up-arrow recalls earlier queries.
50
+
51
+ ## json on — and the silent zero it prevents
52
+
53
+ With `json off` (the default) items are the raw strings. A lambda that reads
54
+ fields therefore matches NOTHING, quietly: 50 buffered NDJSON lines, and
55
+ `filter (e => e.level >= 30) | wc -l` reported `0`. `json on` fixes that at
56
+ query time: a string item parsing to a JSON **object or array** reaches your
57
+ lambdas parsed; plain text, strings that parse to primitives (`"42"` stays a
58
+ string), and non-string items flow through unchanged and are counted out loud.
59
+ The ring stores raw items, so `json off` is a lossless revert.
60
+
61
+ When the buffer looks like NDJSON the session prints, once, on your first
62
+ *pipeline fragment* (not on `help` or `search`):
63
+
64
+ ```
65
+ hint: buffer looks like NDJSON — `json on` parses object lines at query time (`json off` reverts)
66
+ ```
67
+
68
+ It never auto-enables — changing the item type under a lambda you already typed
69
+ would be a false picture.
70
+
71
+ ## Item types by source
72
+
73
+ - `tail`/shell sources: plain **strings**.
74
+ - `procs({...})`: `{proc, stream, line}` **objects** — the session tells you so
75
+ at boot (`logs: items are {proc, stream, line} objects — try (l => l.line) or
76
+ filter (l => l.proc === "web")`). `(l => l.line)` extracts text,
77
+ `filter (l => l.proc === "web")` selects one process, and `search` still works
78
+ (it matches each item's rendered form).
79
+ - Pretty-printing is just another stage: `grep api_request | pino-pretty
80
+ --colorize --singleLine` (`--colorize` forces ANSI through the pipe; local
81
+ `node_modules/.bin` is on PATH). Normalize `procs` objects first.
82
+
83
+ ## Traps
84
+
85
+ - `logs` is interactive-only. No tty → exit 2 and a pointer:
86
+ `logs: interactive session needs a tty — for piped data use `cmd | crust -c 'stdin | …'``.
87
+ For piped data there is no session — build the pipeline directly.
88
+ - The `logs` line owns its line: `logs tail -F app.log | grep ERROR` refuses
89
+ (`logs is a crust builtin — it runs in-process, so sh cannot run it`) rather
90
+ than letting sh report `command not found`. Type the filter at `logs>`.
91
+ - A gate query (`count 1`, `expect 200`) runs against the BUFFER first — a
92
+ retro failure prints before a single live item arrives. That is the honest
93
+ order, but it means `logs`+gate reflects history, not just what follows.
94
+ - `search` is buffer-only and fixed-substring: it never sees live items and
95
+ never interprets regex.
96
+ - Ctrl-C at an idle `logs>` prompt is nothing: it prints `^C` and re-prompts,
97
+ the source keeps running. `exit` (or Ctrl-D) is the teardown — verified with a
98
+ `procs` source, including after a Ctrl-C: the group gets SIGTERM→SIGKILL and
99
+ no `sleep` survives the session.
@@ -0,0 +1,108 @@
1
+ ---
2
+ name: crust-mcp
3
+ description: Drive crust directly from an MCP-capable agent (OpenCode, Claude Code, Codex, Cursor) instead of a shell — `crust mcp` exposes every capability as a first-class tool: run_line, check_line, test_fixture/test_pipes, gen_fixtures, load_run, verify_web_links, plus lifecycle mock servers, supervised procs, and buffered log sessions that stay open across calls. Use when configuring crust as an MCP server, choosing which crust_* tool fits a task, or reasoning about what persists between tool calls and what does not.
4
+ ---
5
+
6
+ # crust over MCP
7
+
8
+ `crust mcp` speaks JSON-RPC 2.0 over stdio — one message per line. It is a
9
+ whole-process MODE, not a REPL builtin: the MCP host owns stdin and stdout.
10
+
11
+ ## Install & configure
12
+
13
+ `crust` must be on PATH (either channel works — same binary):
14
+
15
+ ```bash
16
+ curl -fsSL https://raw.githubusercontent.com/lariocpt/crust/main/install.sh | bash
17
+ # or: npm i -g @lariocpt/crust
18
+ ```
19
+
20
+ Then register it with the agent host — e.g. OpenCode / Claude Code style:
21
+
22
+ ```json
23
+ { "mcpServers": { "crust": { "command": "crust", "args": ["mcp"] } } }
24
+ ```
25
+
26
+ VS Code uses a DIFFERENT schema in `.vscode/mcp.json` (workspace) or the
27
+ user-level MCP settings — the `mcpServers` shape above silently fails there
28
+ ("command is required" or the server just never appears):
29
+
30
+ ```json
31
+ {
32
+ "servers": {
33
+ "crust": { "type": "stdio", "command": "/home/user/.local/bin/crust", "args": ["mcp"] }
34
+ }
35
+ }
36
+ ```
37
+
38
+ Two traps, both verified: (1) `"command"` must be the ABSOLUTE path — a
39
+ GUI-launched VS Code does not inherit the shell's PATH, so bare `"crust"`
40
+ dies `spawn crust ENOENT` if the launcher lives outside the default
41
+ lookup dirs; (2) VS Code's MCP support ships inside the Chat/Copilot Chat
42
+ extension — Code - OSS / VSCodium has no MCP host without an extension
43
+ (Cline, Roo Code or Continue each carry their own MCP settings).
44
+
45
+ Grammar lands automatically: the one-shot tools pass line/fixture text to a
46
+ fresh crust child untouched, so anything the lexer accepts (`--redirect`,
47
+ new builtins) is an MCP tool call the moment the installed binary has it —
48
+ no MCP-side change per feature.
49
+
50
+ Verify by hand — the transport is just lines on stdin:
51
+
52
+ ```bash
53
+ printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"smoke","version":"0"}}}\n{"jsonrpc":"2.0","method":"notifications/initialized"}\n{"jsonrpc":"2.0","id":2,"method":"tools/list"}\n' | crust mcp
54
+ ```
55
+
56
+ ## Two execution planes — what persists
57
+
58
+ | Plane | Tools | State |
59
+ | --- | --- | --- |
60
+ | **One-shot** | `crust_run_line`, `crust_check_line`, `crust_test_fixture`, `crust_test_pipes`, `crust_gen_fixtures`, `crust_verify_web_links`, `crust_load_run` | a fresh crust CHILD per call — `cwd`/`env` apply per call and NEVER carry over; pass them explicitly |
61
+ | **Lifecycle** | `crust_mock_server_start/stop/list/violations`, `crust_procs_start/stop`, `crust_logs_open/query/close` | held IN the server across calls; every held job dies when the server exits |
62
+
63
+ `crust_run_line` runs ordinary crust grammar — one line, `|`-stages, lambdas:
64
+
65
+ ```crust
66
+ range(1,5) | (n => n * n) | stats
67
+ ```
68
+
69
+ ## Honest failure — never a false pass over MCP
70
+
71
+ A nonzero child exit, a killed timeout, a failed `assert`, a mock validation
72
+ error is an **`isError` result** carrying the verbatim exit code and output.
73
+ Zero matches is answered with an explicit "zero items", never silence. Trust
74
+ `isError`; the text is evidence, not a verdict.
75
+
76
+ ## Picking the right tool
77
+
78
+ - Validate generated crust first — `crust_check_line` parses without running
79
+ (files and ports don't need to exist).
80
+ - `crust_procs_start` supervises dev services (env/after/ready/restart per
81
+ proc). Its stream is queried through `crust_logs_query` — items are
82
+ `{proc, stream, line}` objects. Gate readiness in a follow-up:
83
+
84
+ ```crust
85
+ wait :3000/health
86
+ ```
87
+
88
+ - `crust_logs_open` holds a tail/shell/`procs({…})` source with a ring buffer;
89
+ `crust_logs_query` runs a pipeline fragment over the buffered past, then
90
+ live for `live_ms` — `json: true` parses NDJSON items before the fragment:
91
+
92
+ ```crust
93
+ filter (l => l.proc === "api") | (l => l.line) | grep -i timeout
94
+ ```
95
+
96
+ - `crust_mock_server_start` defaults `port: 0` (OS-assigned) — always take
97
+ the returned `url`. `proxy: "<upstream>"` turns it into a conformance audit;
98
+ read findings with `crust_mock_server_violations`.
99
+ - `crust_load_run` composes the documented pipeline (`load … | parallel … |
100
+ … | stats | assert`); an `assert` failure is `isError` — a run that
101
+ measured nothing is refused (`empty`), never a green gate.
102
+
103
+ ## Limits, stated
104
+
105
+ - One tool call runs at a time (the interrupt bus is process-global).
106
+ - Output is tail-capped per stream and truncation is always announced.
107
+ - `crust_procs_stop` reports "teardown still in flight past 10s" honestly
108
+ when a child ignores SIGTERM — crust kills it on server exit.