@sdxc/spec 0.0.0-pre.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/LICENSE.md +21 -0
  2. package/README.md +924 -0
  3. package/dist/ast.d.ts +193 -0
  4. package/dist/ast.js +9 -0
  5. package/dist/builtins.d.ts +29 -0
  6. package/dist/builtins.js +66 -0
  7. package/dist/cli.d.ts +21 -0
  8. package/dist/cli.js +297 -0
  9. package/dist/diagnostics.d.ts +47 -0
  10. package/dist/diagnostics.js +8 -0
  11. package/dist/errors.d.ts +131 -0
  12. package/dist/errors.js +159 -0
  13. package/dist/executor.d.ts +66 -0
  14. package/dist/executor.js +320 -0
  15. package/dist/expectation.d.ts +61 -0
  16. package/dist/expectation.js +222 -0
  17. package/dist/index.d.ts +51 -0
  18. package/dist/index.js +36 -0
  19. package/dist/lexer.d.ts +22 -0
  20. package/dist/lexer.js +284 -0
  21. package/dist/loader.d.ts +21 -0
  22. package/dist/loader.js +81 -0
  23. package/dist/parser.d.ts +24 -0
  24. package/dist/parser.js +502 -0
  25. package/dist/permissions.d.ts +139 -0
  26. package/dist/permissions.js +325 -0
  27. package/dist/plugin.d.ts +90 -0
  28. package/dist/plugin.js +9 -0
  29. package/dist/plugins/browser.d.ts +24 -0
  30. package/dist/plugins/browser.js +896 -0
  31. package/dist/plugins/cli.d.ts +17 -0
  32. package/dist/plugins/cli.js +134 -0
  33. package/dist/plugins/db-e2e-probe.d.ts +14 -0
  34. package/dist/plugins/db-e2e-probe.js +112 -0
  35. package/dist/plugins/db.d.ts +19 -0
  36. package/dist/plugins/db.js +199 -0
  37. package/dist/plugins/demo.d.ts +17 -0
  38. package/dist/plugins/demo.js +70 -0
  39. package/dist/plugins/env.d.ts +18 -0
  40. package/dist/plugins/env.js +87 -0
  41. package/dist/plugins/fs.d.ts +16 -0
  42. package/dist/plugins/fs.js +415 -0
  43. package/dist/plugins/http.d.ts +19 -0
  44. package/dist/plugins/http.js +505 -0
  45. package/dist/plugins/jwt.d.ts +17 -0
  46. package/dist/plugins/jwt.js +342 -0
  47. package/dist/plugins/sample.d.ts +27 -0
  48. package/dist/plugins/sample.js +400 -0
  49. package/dist/plugins/url.d.ts +18 -0
  50. package/dist/plugins/url.js +126 -0
  51. package/dist/project-config.d.ts +163 -0
  52. package/dist/project-config.js +497 -0
  53. package/dist/registry.d.ts +56 -0
  54. package/dist/registry.js +110 -0
  55. package/dist/reporter.d.ts +30 -0
  56. package/dist/reporter.js +237 -0
  57. package/dist/run.d.ts +74 -0
  58. package/dist/run.js +179 -0
  59. package/dist/runner.d.ts +52 -0
  60. package/dist/runner.js +38 -0
  61. package/dist/source.d.ts +37 -0
  62. package/dist/source.js +31 -0
  63. package/dist/sources.d.ts +45 -0
  64. package/dist/sources.js +54 -0
  65. package/dist/tokens.d.ts +34 -0
  66. package/dist/tokens.js +25 -0
  67. package/dist/transport-stdio.d.ts +34 -0
  68. package/dist/transport-stdio.js +400 -0
  69. package/dist/values.d.ts +48 -0
  70. package/dist/values.js +52 -0
  71. package/dist/workers.d.ts +40 -0
  72. package/dist/workers.js +26 -0
  73. package/dist/workspace-none.d.ts +23 -0
  74. package/dist/workspace-none.js +33 -0
  75. package/dist/workspace.d.ts +47 -0
  76. package/dist/workspace.js +116 -0
  77. package/package.json +28 -0
package/README.md ADDED
@@ -0,0 +1,924 @@
1
+ # @sdxc/spec
2
+
3
+ Write down how your app should behave, then run it. `spec` is an **executable
4
+ specification runner**: you describe behavior in `.spec` files — setup, action,
5
+ expectation — and it runs each one against your real app, in an isolated
6
+ workspace, under permissions you grant explicitly. The specs don't care _how_
7
+ the app is built (a CLI, an HTTP server, a database, a web page); they only
8
+ describe what it should do, so they stay true as the implementation changes.
9
+
10
+ ## Overview
11
+
12
+ A suite is a directory of `.spec` files (conventionally `spec/`). Each file is
13
+ written in a deliberately tiny language — no `if`, no loops, no operators — so a
14
+ spec reads as a linear, diffable list of steps. A `test` declares its setup in
15
+ `given`, its action in `when`, and its checks in `then`. The `spec` CLI loads
16
+ the suite, runs every test in its **own fresh temporary directory**, and prints
17
+ one line per test plus a summary.
18
+
19
+ Two rules shape everything:
20
+
21
+ - **Deny by default.** A test that spawns a process, reaches the network, reads
22
+ an environment variable, or touches files outside its workspace needs an
23
+ explicit `--allow-*` grant. Every denial tells you the exact flag that would
24
+ allow it.
25
+ - **Isolated per test.** Each test gets its own workspace, created before it
26
+ runs and removed after. Tests never see each other's files, so order never
27
+ matters.
28
+
29
+ Exit codes are the contract to script against: **0** everything passed, **1** a
30
+ test failed, **2** a usage or load error (a bad flag, an unreadable suite, a
31
+ parse error).
32
+
33
+ ## Quickstart
34
+
35
+ ### Get the CLI
36
+
37
+ Inside this repo you can run the CLI straight from source — the dev entry point:
38
+
39
+ ```sh
40
+ bun packages/spec/src/cli.ts run spec
41
+ ```
42
+
43
+ For everyday use, compile a single self-contained executable that starts fast
44
+ and runs anywhere (no repo, no `node_modules` beside it):
45
+
46
+ ```sh
47
+ cd packages/spec
48
+ bun run build # → packages/spec/bin/spec
49
+ ./bin/spec run spec
50
+ ```
51
+
52
+ Put `bin/spec` on your `PATH` and it's just `spec`. The rest of this guide
53
+ writes `spec`; use whichever launcher you have.
54
+
55
+ ### Write a suite
56
+
57
+ Create `spec/greeting.spec`:
58
+
59
+ ```
60
+ use fs
61
+ use cli
62
+
63
+ test "the script prints its greeting" {
64
+ given {
65
+ write "index.js" "console.log(\"hello from the workspace\")"
66
+ }
67
+ when {
68
+ let result = run "bun" "index.js"
69
+ }
70
+ then {
71
+ expect result.exit_code 0
72
+ expect result.stdout "hello from the workspace\n"
73
+ }
74
+ }
75
+
76
+ test "the generated config is on disk" {
77
+ given {
78
+ write "package.json" { name: "demo", type: "module" }
79
+ }
80
+ then {
81
+ expect file "package.json" exists
82
+ expect file "package.json" contains "\"type\": \"module\""
83
+ }
84
+ }
85
+ ```
86
+
87
+ ### Run it
88
+
89
+ `write` creates files in the test's workspace and `run` starts processes there.
90
+ Spawning `bun` is a privileged act, so grant exactly that one executable:
91
+
92
+ ```sh
93
+ spec run spec --allow-run=bun
94
+ ```
95
+
96
+ ```
97
+ ✓ the script prints its greeting
98
+ ✓ the generated config is on disk
99
+
100
+ 2 passed, 0 failed (21ms)
101
+ ```
102
+
103
+ Drop the grant and the process-spawning test fails **before** `bun` is ever
104
+ launched. Every test denied for the same missing grant collapses into one block
105
+ that names the flag to add and lists the tests it affected:
106
+
107
+ ```
108
+ ✓ the generated config is on disk
109
+
110
+ ✗ Permission denied: run (1 test)
111
+
112
+ The spec attempted to reach:
113
+ > cli.run
114
+
115
+ Re-run with an appropriate permission, for example:
116
+ > spec run --allow-run
117
+
118
+ Affected tests:
119
+ - the script prints its greeting (spec/greeting.spec:9)
120
+
121
+ 1 passed, 1 failed (1ms)
122
+ ```
123
+
124
+ `spec run` takes a **directory** (default `./spec`); it scans it recursively for
125
+ `.spec` files.
126
+
127
+ ## The language by example
128
+
129
+ ### Tests and phases
130
+
131
+ A `test` has up to three phase blocks, always in this order: `given` (arrange),
132
+ `when` (act), `then` (assert). Each is optional, but you can't reorder them.
133
+ Lines end statements — there are no semicolons. `#` starts a comment to
134
+ end of line (a `#` inside a string is just text).
135
+
136
+ ```
137
+ test "a write is read back verbatim" {
138
+ given {
139
+ write "notes.txt" "remember the milk" # arrange
140
+ }
141
+ when {
142
+ let content = read "notes.txt" # act, capture a value
143
+ }
144
+ then {
145
+ expect content "remember the milk" # assert
146
+ }
147
+ }
148
+ ```
149
+
150
+ ### `expect`
151
+
152
+ `expect` has three forms:
153
+
154
+ ```
155
+ expect content "remember the milk" # two values: deep structural equality
156
+ expect true # one value: it must be true
157
+ expect file "notes.txt" exists # observable: assert straight from a capability
158
+ ```
159
+
160
+ The observable form reads the world through a capability (`file … exists`,
161
+ `file … contains`, `directory … exists`, `browser.heading …`) and passes when
162
+ that observation holds.
163
+
164
+ ### `let` and references
165
+
166
+ `let` binds the result of a step. Reach into a returned object with a dotted
167
+ reference:
168
+
169
+ ```
170
+ when {
171
+ let result = run "bun" "build.js"
172
+ }
173
+ then {
174
+ expect result.exit_code 0
175
+ expect result.stdout "built\n"
176
+ }
177
+ ```
178
+
179
+ One thing to know: a **bare word** in tool-argument position is a symbol, not a
180
+ variable. `write "f" content` hands the tool the literal word `content`. To pass
181
+ a bound value to a tool, use a dotted reference (`result.stdout`), boxing it in
182
+ an object if needed (`let x = { path: p }` then `write x.path …`).
183
+
184
+ A bare path on the right of `let`/`return` is a reference when its head names a
185
+ binding, but when the head is not a binding and the path resolves to a tool that
186
+ needs no arguments, it is a **zero-argument tool call** — so `let current =
187
+ browser.url` binds that tool's observed value. This works for any argument-less
188
+ tool, and the call is permission-gated like any other.
189
+
190
+ ### `eventually`
191
+
192
+ Wrap an observable assertion in `eventually` to retry it until it holds or the
193
+ window ends — for anything that becomes true a moment later (a server coming up,
194
+ an async write landing). A plain assertion checks exactly once.
195
+
196
+ ```
197
+ then {
198
+ eventually within 2s {
199
+ expect file "ready.txt" exists
200
+ }
201
+ }
202
+ ```
203
+
204
+ Only assertions may be retried — an action (a mutation) inside `eventually` is
205
+ an error, since a retried mutation is not a retried check.
206
+
207
+ ### `command` and `fixture`
208
+
209
+ A **command** is a reusable step; a **fixture** is reusable data (its value is
210
+ whatever it `return`s). Define them in the same file, or share them suite-wide
211
+ by putting them under `spec/commands/` and `spec/fixtures/`. Either way they
212
+ resolve **by name** from anywhere — no import, no path — because every
213
+ definition is registered before any test runs.
214
+
215
+ ```
216
+ # spec/fixtures/book.spec
217
+ fixture book {
218
+ return { title: "Dune", author: "Herbert", year: 1965 }
219
+ }
220
+ ```
221
+
222
+ ```
223
+ # spec/commands/seed.spec
224
+ use fs
225
+
226
+ command seed_file(path) {
227
+ let target = { path: path }
228
+ write target.path "seeded"
229
+ }
230
+ ```
231
+
232
+ ```
233
+ # spec/tour.spec
234
+ test "commands and fixtures compose by name" {
235
+ given {
236
+ seed_file "out.txt" # run a command for its effect
237
+ }
238
+ when {
239
+ let record = fixture book # run a fixture for its value
240
+ }
241
+ then {
242
+ expect record.title "Dune"
243
+ expect file "out.txt" contains "seeded"
244
+ }
245
+ }
246
+ ```
247
+
248
+ ### `use` and namespaces
249
+
250
+ Capabilities live in namespaces (`fs`, `cli`, `http`, …). Use a fully qualified
251
+ name anywhere (`fs.write`, `cli.run`), or `use` a namespace at the top of a file
252
+ to call its tools by their bare names (`write`, `run`). `use` is **per file**;
253
+ suite commands and fixtures need no `use` at all. If a bare name could mean two
254
+ things, that's an error naming both candidates — the runtime never guesses.
255
+
256
+ ```
257
+ use fs # now `write`, `read`, `file`, … are available unqualified
258
+ use cli # now `run` is too
259
+ ```
260
+
261
+ ## Capabilities
262
+
263
+ Capabilities are the built-in namespaces. `fs` needs no grant (it's confined to
264
+ the workspace); the rest are privileged and denied until you grant them.
265
+
266
+ ### `fs` — the workspace filesystem
267
+
268
+ Every path is resolved inside the test's workspace, so `fs` needs no permission.
269
+ Tools: `write`, `read`, `mkdir`, `copy`, `remove`, and the observables `exists`,
270
+ `file`, `directory`. Strings are written verbatim; objects/arrays are written as
271
+ JSON.
272
+
273
+ ```
274
+ use fs
275
+
276
+ test "mkdir, copy, and remove move files around the workspace" {
277
+ given {
278
+ mkdir "src"
279
+ write "src/index.ts" "export const answer = 42"
280
+ }
281
+ when {
282
+ copy "src/index.ts" "dist/index.ts"
283
+ remove "src/index.ts"
284
+ }
285
+ then {
286
+ expect directory "src" exists
287
+ expect file "dist/index.ts" contains "answer = 42"
288
+ }
289
+ }
290
+ ```
291
+
292
+ ### `cli` — run processes · `--allow-run`
293
+
294
+ `run` spawns a program in the workspace and returns `{ stdout, stderr,
295
+ exit_code }`. It needs `--allow-run`, scoped by executable basename. The child
296
+ gets a minimal environment (`PATH`/`HOME`/`TMPDIR` plus only the vars you granted
297
+ with `--allow-env`), so your host environment never leaks in.
298
+
299
+ ```
300
+ use cli
301
+
302
+ test "run captures stdout and the exit code" {
303
+ when {
304
+ let result = run "echo" "hello"
305
+ }
306
+ then {
307
+ expect result.exit_code 0
308
+ expect result.stdout "hello\n"
309
+ }
310
+ }
311
+ ```
312
+
313
+ ```sh
314
+ spec run spec --allow-run=echo
315
+ ```
316
+
317
+ ### `http` — call an HTTP API · `--allow-net`
318
+
319
+ `get`, `post`, `put`, `patch`, `delete`, each needing `--allow-net` for the
320
+ URL's host (and port, if you scope one). URLs must be **absolute**. An optional
321
+ bare body travels as `text/plain` when it's a string, JSON otherwise. Each returns
322
+ `{ status, ok, headers, text, json }`; an HTTP error status is a normal value —
323
+ only a network-level failure is an error.
324
+
325
+ ```
326
+ use http
327
+
328
+ test "creating a post returns 201" {
329
+ when {
330
+ let response = http.post "http://localhost:3000/api/posts" { title: "Hello" }
331
+ }
332
+ then {
333
+ expect response.status 201
334
+ }
335
+ }
336
+ ```
337
+
338
+ ```sh
339
+ spec run spec --allow-net=localhost:3000
340
+ ```
341
+
342
+ #### Request options: headers, bodies, and credentials
343
+
344
+ After the URL, a request takes optional **word-tagged options**, in any order:
345
+
346
+ - `headers { Name: "value", … }` — request headers (an `Authorization`, an
347
+ `Accept`, a cookie). Numbers and booleans stringify; header names are
348
+ case-insensitive, and an explicit `content-type` here overrides the body's.
349
+ - `form { field: "value", … }` — a body sent as
350
+ `application/x-www-form-urlencoded` (the shape OAuth token endpoints and
351
+ classic form posts expect).
352
+ - `json <value>` — a body sent as `application/json`; the explicit form of a
353
+ bare non-string body.
354
+ - `text "<string>"` — a body sent as `text/plain`; the explicit form of a bare
355
+ string body.
356
+ - `bearer <token>` — sets `Authorization: Bearer <token>`, so a resource-server
357
+ call passes the access token, not a hand-built header.
358
+ - `basic <user> <pass>` — sets `Authorization: Basic base64(user:pass)`, the
359
+ `client_secret_basic` shape OAuth introspection and revocation expect. It is
360
+ the one option that takes two values.
361
+
362
+ They combine, so an authenticated form post is one call:
363
+
364
+ ```
365
+ use http
366
+
367
+ test "the token endpoint rejects a bad code" {
368
+ when {
369
+ let response = http.post "http://localhost:3000/oauth/token" form {
370
+ grant_type: "authorization_code"
371
+ code: "bogus"
372
+ } headers { authorization: "Basic dXNlcjpwYXNz" }
373
+ }
374
+ then {
375
+ expect response.status 400
376
+ expect response.json.error "invalid_grant"
377
+ }
378
+ }
379
+
380
+ test "a bogus bearer token is rejected" {
381
+ when {
382
+ let who = http.get "http://localhost:3000/userinfo" bearer "bogus"
383
+ }
384
+ then {
385
+ expect who.status 401
386
+ }
387
+ }
388
+ ```
389
+
390
+ A request carries **at most one body** (the bare body, or one of
391
+ `json`/`form`/`text`), **at most one** `headers` block, and **at most one** auth
392
+ option (`bearer` or `basic`); a second body, a body on a `GET`, both `bearer` and
393
+ `basic`, or an unknown tag is an error. An explicit `headers.authorization`
394
+ overrides `bearer`/`basic`. The two original forms — `http.get url` and
395
+ `http.<verb> url <body>` — are unchanged, so existing specs keep working exactly
396
+ as before.
397
+
398
+ ### `browser` — drive a real browser · `--allow-net`
399
+
400
+ Drive a browser through its **accessibility tree**, not DOM internals: address
401
+ elements by role (a bare word) and accessible name (a string). Actions include
402
+ `open`, `navigate`, `cookie`, `ua`, `click`, `fill … with …`, `check`, `press`;
403
+ observables include `heading`, `link`, `button`, `text`, `checkbox`, `url`,
404
+ `title`. Reaching the page is the privileged act, so each tool needs
405
+ `--allow-net` for the target host. It's backed by a globally installed `agent-browser` CLI, loaded lazily —
406
+ a suite that never touches `browser.*` needs neither the grant nor the binary.
407
+
408
+ ```
409
+ use browser
410
+
411
+ test "the sign-in form authenticates" {
412
+ when {
413
+ browser.open "http://localhost:3000/login"
414
+ browser.fill textbox "Email" with "user@example.com"
415
+ browser.fill textbox "Password" with "correct horse"
416
+ browser.click button "Sign in"
417
+ }
418
+ then {
419
+ expect browser.heading "Welcome back"
420
+ expect browser.title "My App"
421
+ expect browser.url "http://localhost:3000/home"
422
+ }
423
+ }
424
+ ```
425
+
426
+ ```sh
427
+ spec run spec --allow-net=localhost:3000
428
+ ```
429
+
430
+ `browser.heading` also takes a level, for when the rung of the document outline
431
+ is part of the behavior: `level 3` matches an `<h3>` and equally a
432
+ `role="heading"` with `aria-level="3"`, because both reach the accessibility
433
+ tree the same way. A heading of that name at another level fails with the levels
434
+ it did find.
435
+
436
+ ```
437
+ then {
438
+ expect browser.heading "Billing" level 2
439
+ }
440
+ ```
441
+
442
+ Like `browser.url`, `browser.title` reads as a value too — `let name =
443
+ browser.title` with no argument binds the current title instead of asserting one.
444
+
445
+ `browser.cookie` seeds the session's cookie jar, so a test that isn't about
446
+ signing in can arrive already signed in. The `for` clause names the URL the
447
+ cookie belongs to, which is what lets it be set _before_ the first navigation;
448
+ drop the clause to set it on the page already open. Pair it with
449
+ [`env.get`](#env--read-a-granted-variable----allow-env) — the token belongs in
450
+ the environment, not in the document.
451
+
452
+ ```
453
+ given {
454
+ let token = env.get "SESSION_COOKIE"
455
+ let jar = { session: token }
456
+ browser.cookie "session" jar.session for "http://localhost:3000/app"
457
+ }
458
+ ```
459
+
460
+ `browser.ua` sets the `User-Agent` the session sends, so the app can recognize
461
+ its own spec run — skip a rate limiter, tag analytics, take a test-only branch.
462
+ Set it before `open`; it applies to requests made after it. It changes the
463
+ request header only: `navigator.userAgent` inside the page still reports the
464
+ real browser.
465
+
466
+ ```
467
+ given {
468
+ browser.ua "spec-runner/1.0"
469
+ }
470
+ ```
471
+
472
+ `browser.url` also reads as a value: `let current = browser.url` binds the
473
+ session's current URL, so a spec can pull the authorization `code` out of the
474
+ page the browser landed on. (A bare binding reaches a tool through a dotted
475
+ reference, so box it first — see [`let` and references](#let-and-references).)
476
+
477
+ ```
478
+ when {
479
+ browser.click button "Authorize"
480
+ let landing = browser.url # capture the redirect URL
481
+ let where = { url: landing }
482
+ let code = url.query where.url "code" # read ?code=… out of it
483
+ }
484
+ ```
485
+
486
+ ### `db` — query a database · `--allow-env=DATABASE_URL`
487
+
488
+ `db.query` runs raw SQL on Bun's SQL client and returns `{ rows, affected_rows,
489
+ count }`. It reads the connection string from the `DATABASE_URL` environment
490
+ variable, so granting that one variable is the whole authorization — a spec can
491
+ never redirect the connection elsewhere. The connection opens lazily on the
492
+ first query and closes at the end of the run.
493
+
494
+ ```
495
+ use db
496
+
497
+ test "an INSERT reports exactly one affected row" {
498
+ when {
499
+ let result = db.query "INSERT INTO ledger (entry) VALUES ('opening balance')"
500
+ }
501
+ then {
502
+ expect result.affected_rows 1
503
+ expect result.count 0
504
+ }
505
+ }
506
+ ```
507
+
508
+ ```sh
509
+ DATABASE_URL=postgres://localhost/test spec run spec --allow-env=DATABASE_URL
510
+ ```
511
+
512
+ ### `env` — read a granted variable · `--allow-env`
513
+
514
+ `env.get NAME` reads one environment variable, and only one the caller granted
515
+ by name. It is how a spec names a secret without containing one: the document
516
+ says _which_ variable holds the session token, the environment says what it is,
517
+ and the same spec runs against local, staging, and CI. An unset variable is an
518
+ error unless you give `env.get` a fallback — its optional second argument — which
519
+ covers an absent value, never an absent grant.
520
+
521
+ ```
522
+ use env
523
+ use browser
524
+
525
+ test "the dashboard renders for a signed-in session" {
526
+ given {
527
+ let token = env.get "SESSION_COOKIE"
528
+ let jar = { session: token }
529
+ browser.cookie "session" jar.session for "http://localhost:3000/app"
530
+ }
531
+ when {
532
+ browser.open "http://localhost:3000/app"
533
+ }
534
+ then {
535
+ # Without the cookie, this would have redirected to /login.
536
+ expect browser.url "http://localhost:3000/app"
537
+ }
538
+ }
539
+ ```
540
+
541
+ ```sh
542
+ SESSION_COOKIE=… spec run spec --allow-env=SESSION_COOKIE --allow-net=localhost:3000
543
+ ```
544
+
545
+ ### `url` — parse a URL · no grant
546
+
547
+ Pure, permissionless URL parsing — no network, no filesystem — so a spec can pull
548
+ a value out of a URL it already holds instead of doing string surgery the language
549
+ deliberately omits. Its typical job is reading the authorization `code` out of the
550
+ redirect URL an OAuth authorize step lands on.
551
+
552
+ - `url.query <url> <name>` — the value of a query-string parameter.
553
+ - `url.fragment <url> <name>` — the value of a parameter after the `#` (the
554
+ implicit/hybrid OAuth response shape).
555
+ - `url.path <url>` — the URL's pathname; `url.host <url>` — its host and port.
556
+
557
+ A missing parameter, a non-string argument, or an unparseable URL is an error —
558
+ `query`/`fragment` never bind a silent null.
559
+
560
+ ```
561
+ use url
562
+
563
+ test "the authorization code is read from the redirect URL" {
564
+ when {
565
+ let code = url.query "http://localhost:3000/callback?code=abc123&state=s" "code"
566
+ }
567
+ then {
568
+ expect code "abc123"
569
+ }
570
+ }
571
+ ```
572
+
573
+ ### `sample` — generate a suite's input · no grant
574
+
575
+ Names, addresses, identifiers, and numbers drawn per test, instead of literals
576
+ typed into the suite. Pure computation over the test's own stream — no network,
577
+ no filesystem, no grant — and every value reproduces: the same test draws the
578
+ same data on every run, so a failure on generated input is replayable.
579
+
580
+ A call target carries at most one dot, so **each module is one tool** returning a
581
+ record of everything that module generates. Bind it, then read fields by path:
582
+
583
+ ```
584
+ use sample
585
+
586
+ test "a visitor signs up from somewhere" {
587
+ given {
588
+ let who = sample.person
589
+ let where = sample.location
590
+ }
591
+ when {
592
+ let created = http.post "http://localhost:3000/signup" json {
593
+ email: who.email
594
+ name: who.full_name
595
+ job: who.job_title
596
+ city: where.city
597
+ country: where.country
598
+ }
599
+ }
600
+ then {
601
+ expect created.status 201
602
+ }
603
+ }
604
+ ```
605
+
606
+ | Tool | The record holds |
607
+ | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
608
+ | `sample.person` | `first_name`, `last_name`, `full_name`, `prefix`, `suffix`, `sex`, `gender`, `zodiac_sign`, `job_title`, `bio`, `email`, `username`, `phone` |
609
+ | `sample.internet` | `email`, `username`, `url`, `domain_name`, `password`, `ip`, `ipv4`, `ipv6`, `mac`, `port`, `protocol`, `http_method`, `http_status_code`, `jwt`, `user_agent`, `emoji` |
610
+ | `sample.location` | `country`, `city`, `country_code`, `state`, `county`, `street`, `street_address`, `zip_code`, `postal_address`, `latitude`, `longitude`, `time_zone` |
611
+ | `sample.company` | `name`, `catch_phrase`, `buzz_phrase`, and their parts |
612
+ | `sample.lorem` | `word`, `words`, `sentence`, `paragraph`, `lines`, `slug`, `text` |
613
+ | `sample.date` | `past`, `future`, `recent`, `soon`, `anytime`, `birthdate`, `month`, `weekday`, `time_zone` — dates as ISO timestamps |
614
+ | `sample.string` | `uuid`, `ulid`, `nanoid`, `alpha`, `alphanumeric`, `numeric`, `hexadecimal`, `binary`, `octal`, `symbol` |
615
+ | `sample.number` | `int`, `float`, `hex`, `binary`, `octal`, `roman_numeral`, `big_int` |
616
+ | `sample.color` | `human`, `hex`, `rgb`, `hsl`, `space`, `css_function` |
617
+ | `sample.datatype` | `boolean` |
618
+ | `sample.git` | `branch`, `commit_sha`, `short_sha`, `commit_message`, `commit_date`, `commit_entry` |
619
+ | `sample.hacker` | `abbreviation`, `adjective`, `noun`, `verb`, `ingverb`, `phrase` |
620
+ | `sample.phone` | `number`, `national`, `international`, `imei` |
621
+ | `sample.system` | `file_name`, `file_ext`, `file_type`, `mime_type`, `directory_path`, `file_path`, `network_interface`, `semver`, `cron` |
622
+
623
+ A record's fields agree with each other: a person's address matches their name,
624
+ and a location's postal address names its own city and country.
625
+
626
+ The generators that take an argument keep a tool of their own, alongside two
627
+ shortcuts for the values a suite reaches for most:
628
+
629
+ - `sample.int <min> <max>` — an integer, both bounds included.
630
+ - `sample.float <min> <max>` — a number between the bounds, two decimals.
631
+ - `sample.words <count>` — that many words of placeholder prose.
632
+ - `sample.pick <list>` — one element of a list another tool returned.
633
+ - `sample.email`, `sample.uuid` — the two single values worth their own name.
634
+
635
+ Every tool is an **action**, not an observation: a draw advances the stream, so
636
+ `sample` may not head an `eventually` — polling until a random value matches is
637
+ never what an author meant.
638
+
639
+ A test's data follows its **identity** — the run's seed, the test's file inside
640
+ the suite, and its title — and nothing else. Two runs of a suite generate the
641
+ same data, whatever the concurrency, whatever order the tests ran in, and
642
+ wherever the suite is checked out. Adding or removing a neighboring test does
643
+ not move it either.
644
+
645
+ ```sh
646
+ spec run spec --seed=checkout # a different suite-wide seed
647
+ spec run spec --seed=random # draw one, printed so it can be replayed
648
+ ```
649
+
650
+ `--seed=random` prints the seed it drew before the run:
651
+
652
+ ```
653
+ seed 1007223771 (replay with --seed=1007223771)
654
+ ```
655
+
656
+ That is how a suite is shaken for hidden dependence on particular values while
657
+ keeping any failure it turns up reproducible.
658
+
659
+ ### `jwt` — read and verify tokens · `--allow-net` (verify only)
660
+
661
+ Read and verify JSON Web Tokens, the heart of specifying an OIDC server.
662
+
663
+ - `jwt.decode <token>` — split a token into `{ header, payload }` with **no**
664
+ signature check; permissionless, for asserting on claims (`decoded.payload.sub`,
665
+ `decoded.header.alg`, …).
666
+ - `jwt.verify <token> <jwks_url>` — fetch the issuer's JWKS, select the key the
667
+ token names by `kid`, verify its **ES256** signature and expiry, and return the
668
+ verified payload — so a spec proves an id_token is genuinely issuer-signed, not
669
+ just well-formed. It reaches the network to read the JWKS, so it needs
670
+ `--allow-net` for that host; a bad signature, an unknown key, an expired token,
671
+ or a non-ES256 algorithm is an error.
672
+
673
+ ```
674
+ use jwt
675
+
676
+ test "the id_token is genuinely signed and names the right subject" {
677
+ given {
678
+ let tokens = fixture issued_tokens
679
+ }
680
+ when {
681
+ let claims = jwt.verify tokens.id_token "http://localhost:3000/.well-known/jwks.json"
682
+ }
683
+ then {
684
+ expect claims.iss "https://id.example.com"
685
+ expect claims.aud "the-client-id"
686
+ }
687
+ }
688
+ ```
689
+
690
+ ```sh
691
+ spec run spec --allow-net=localhost:3000
692
+ ```
693
+
694
+ ## Permissions
695
+
696
+ Nothing privileged runs without a grant, and any attempt beyond your grants
697
+ fails with the exact flag it needs — so start with no flags and let the denials
698
+ tell you the suite's true footprint.
699
+
700
+ | Flag | Grants |
701
+ | ----------------------------- | ------------------------------------------------------------- |
702
+ | `--allow-run[=name,…]` | Spawn processes (scoped by executable basename) |
703
+ | `--allow-net[=host[:port],…]` | Reach the network (scoped by host, optionally port) |
704
+ | `--allow-env[=VAR,…]` | Read environment variables (scoped by name) |
705
+ | `--allow-host-fs[=dir,…]` | Touch files outside the workspace (path-prefix scoped) |
706
+ | `--allow-plugins[=ns,…]` | Launch project-declared plugins (see below) |
707
+ | `--allow-config` | Apply the permissions the suite's config declares (see below) |
708
+
709
+ A bare flag grants the whole family (`--allow-net`); a scoped flag grants only
710
+ what it lists (`--allow-run=bun,git`); repeated scoped flags union. Some recipes:
711
+
712
+ ```sh
713
+ # Pure filesystem suite — the workspace is always writable, so no flags at all:
714
+ spec run spec
715
+
716
+ # One known tool, one local server, and two named env vars:
717
+ spec run spec --allow-run=bun --allow-net=localhost:3000 --allow-env=CI,NODE_ENV
718
+
719
+ # Read shared fixture data from outside the workspace (path prefix):
720
+ spec run spec --allow-host-fs=/opt/fixtures
721
+ ```
722
+
723
+ When several tests fail for the same missing grant, they collapse into one
724
+ grouped block — the `(N tests)` count and `Affected tests:` list make it obvious
725
+ what a single flag would unblock:
726
+
727
+ ```
728
+ ✗ Permission denied: run (3 tests)
729
+
730
+ The spec attempted to reach:
731
+ > cli.run
732
+
733
+ Re-run with an appropriate permission, for example:
734
+ > spec run --allow-run
735
+
736
+ Affected tests:
737
+ - first needs run (spec/denials.spec:3)
738
+ - second needs run (spec/denials.spec:9)
739
+ - third needs run (spec/denials.spec:15)
740
+ ```
741
+
742
+ ## The `spec/config.jsonc` file
743
+
744
+ Rather than reciting the same `--allow-*` line every run, a suite can carry its
745
+ own configuration in `spec/config.jsonc` (JSONC: comments and trailing commas
746
+ allowed). It has two keys, both optional: `permissions` and `plugins`.
747
+
748
+ ```jsonc
749
+ // spec/config.jsonc
750
+ {
751
+ "permissions": {
752
+ // A bare string is a whole-family grant (like a bare --allow-<family>);
753
+ // a [family, ...scopes] tuple is a scoped grant (like --allow-<family>=…).
754
+ "allow": ["run", "plugins", ["env", "DATABASE_URL"]],
755
+ },
756
+ }
757
+ ```
758
+
759
+ The families match the flags: `"run"` ≡ `--allow-run`, `["env","DATABASE_URL"]`
760
+ ≡ `--allow-env=DATABASE_URL`, `["net","localhost:3000"]` ≡
761
+ `--allow-net=localhost:3000`, and `"plugins"` (or `["plugins","greet"]`) ≡
762
+ `--allow-plugins`. A malformed or unknown entry is a load error naming it — a
763
+ broken declaration is never silently ignored.
764
+
765
+ Crucially, the declaration is **declare + opt-in**, never ambient authority.
766
+ With no opt-in the file grants **nothing**: the suite still fails closed with
767
+ the normal denials. The operator opts in with one flag:
768
+
769
+ ```sh
770
+ spec run spec --allow-config
771
+ ```
772
+
773
+ Then the effective grants are the config's declared set **unioned** with any
774
+ explicit `--allow-*` flags you also pass — flags only ever add. Because nothing
775
+ in the file takes effect until someone who read it adds `--allow-config`, a
776
+ cloned or untrusted repo can't self-grant. As a convenience, when a denial
777
+ _would_ be covered by the config, the denial adds one line pointing at
778
+ `--allow-config`.
779
+
780
+ ### Loading project plugins
781
+
782
+ The `plugins` key maps a namespace to the command that launches its plugin, so
783
+ specs name `greet.hello` and never a path:
784
+
785
+ ```jsonc
786
+ // spec/config.jsonc
787
+ {
788
+ "plugins": {
789
+ // A relative "." path resolves against this file's directory.
790
+ "greet": { "command": ["bun", "./greeter.ts"] },
791
+ },
792
+ }
793
+ ```
794
+
795
+ Declaring a plugin is **not** permission to run it — launching one executes code
796
+ the project ships, so it's deny-by-default too. `spec run` starts a declared
797
+ plugin only with `--allow-plugins` (all) or `--allow-plugins=greet` (named); a
798
+ suite that imports an unauthorized plugin is refused before any test runs. The
799
+ built-in namespaces (`fs`, `cli`, `http`, `browser`, `db`, `env`, `url`, `jwt`)
800
+ are
801
+ never affected.
802
+
803
+ ## Custom and third-party plugins
804
+
805
+ Every capability is just a plugin — one namespace exposing typed tools behind a
806
+ single interface — so your own is a first-class citizen. You can build one
807
+ in-process (for embedders), as an external executable speaking a small
808
+ NDJSON-over-stdio protocol (any language), or install a third party's. The full
809
+ authoring guide — the `Plugin` interface, all three shapes, project loading, and
810
+ the trust model — is [docs/writing-plugins.md](./docs/writing-plugins.md), with
811
+ a runnable showcase under `examples/plugin-loading/`.
812
+
813
+ ## Performance
814
+
815
+ The runner itself is cheap — roughly a millisecond per test on top of whatever
816
+ the app under test costs — so wall-time is dominated by your app, not the
817
+ harness. Two levers keep it that way. Use the **compiled binary** (`bun run
818
+ build` → `./bin/spec`) instead of `bun src/cli.ts` to skip the per-launch
819
+ transpile cost, which matters most for suites that shell out to `spec` many
820
+ times. And for suites whose tests spend their time waiting — a browser, an HTTP
821
+ round-trip, a slow process — run them with **`--concurrency=N`** (alias
822
+ `--jobs=N`) to overlap that waiting:
823
+
824
+ ```sh
825
+ spec run spec --concurrency=8
826
+ ```
827
+
828
+ Concurrency defaults to `1` (strictly sequential — today's behavior). At `N` the
829
+ runner executes up to `N` tests at once but still reports results in **source
830
+ order**, so the output, counts, and exit code are byte-for-byte identical
831
+ regardless of how the schedule shook out; only the wall-time changes. The runner
832
+ isolates each test's **workspace**, not the app under test — so a suite that
833
+ shares one mutable backend (the same database rows, one stateful server) may
834
+ still need `--concurrency=1`.
835
+
836
+ ## Beyond the CLI
837
+
838
+ The CLI is the product surface, and its exit codes (0 pass, 1 test failure, 2
839
+ usage/load error) are what you script against. For embedding the runner in
840
+ another program, the package also exports a programmatic `runSuite` and the
841
+ supporting types from `@sdxc/spec`; every fallible export returns a
842
+ [`@sdxc/result`](/packages/result) `Result`, so parse errors, permission denials,
843
+ and tool failures are values you branch on, never thrown exceptions.
844
+
845
+ ### Choosing which capabilities exist
846
+
847
+ `runSuite` registers all eight built-in namespaces. Pass `builtins` to register
848
+ only some:
849
+
850
+ ```ts
851
+ import { runSuite } from "@sdxc/spec";
852
+
853
+ let run = await runSuite({ root: "spec", grants, builtins: ["http", "url", "jwt"] });
854
+ ```
855
+
856
+ This is **not** a permission decision, and the difference matters. A denied
857
+ capability still exists — the denial names the flag that would allow it. A
858
+ namespace left out of `builtins` does not exist: a spec calling `fs.write` fails
859
+ with an unknown name, because there is no flag that would ever lift it. Use
860
+ grants to say "not now", and `builtins` to say "not here".
861
+
862
+ `createBuiltinPlugins(only?)` builds the same list on its own, for callers that
863
+ assemble a plugin set by hand.
864
+
865
+ ### Running without a filesystem or a process
866
+
867
+ `runSuite` assumes a Bun or Node process: it reads the suite off a disk, gives
868
+ each test a temp directory, and can spawn `cli`, `browser` and `db`. Underneath
869
+ it is `runTests`, which assumes nothing — the suite, the plugin set, the grants,
870
+ and the workspace factory all arrive as arguments:
871
+
872
+ ```ts
873
+ import { isFailure } from "@sdxc/result";
874
+ import {
875
+ createHttpPlugin,
876
+ createJwtPlugin,
877
+ createNoFilesystemWorkspace,
878
+ createUrlPlugin,
879
+ loadSources,
880
+ parseGrants,
881
+ runTests,
882
+ } from "@sdxc/spec/workers";
883
+
884
+ let loaded = loadSources([{ path: "flow.spec", text: source }]);
885
+ if (isFailure(loaded)) return loaded;
886
+
887
+ let grants = parseGrants(["--allow-net=app.example.com"]);
888
+ if (isFailure(grants)) return grants;
889
+
890
+ let outcome = await runTests({
891
+ suite: loaded.data,
892
+ plugins: [createHttpPlugin(), createUrlPlugin(), createJwtPlugin()],
893
+ grants: grants.data,
894
+ createWorkspace: createNoFilesystemWorkspace,
895
+ });
896
+ ```
897
+
898
+ `loadSources` is the half of loading that has no filesystem in it: hand it
899
+ `{ path, text }` pairs from wherever the specs live — a database row, an HTTP
900
+ body, a bundled string — and it parses and registers them exactly as
901
+ `loadSuite` does after its directory walk. `createNoFilesystemWorkspace` refuses
902
+ every path, which a run without `fs` and `cli` never asks it to resolve.
903
+
904
+ The `@sdxc/spec/workers` entry point exists because of what a module may
905
+ **import**, not what a run may do: `db` imports Bun's SQL client, and `cli`,
906
+ `browser` and the stdio plugin transport reach for the `Bun` global, so a module
907
+ importing them cannot load in a V8-isolate runtime however carefully the run is
908
+ permissioned. That entry point exports the language core plus the three
909
+ capabilities that are already pure — `http`, `url`, `jwt` — and a test in the
910
+ package walks its import graph to keep it that way. It still needs Node
911
+ compatibility enabled for `node:path` and `node:fs`, which the permission set
912
+ reaches only through a host-filesystem grant.
913
+
914
+ There is no browser capability there, deliberately. Driving a browser without a
915
+ local binary means calling a remote service over HTTP, and which service that is
916
+ belongs to the host, not to this package: implement the same tool surface as a
917
+ [plugin](./docs/writing-plugins.md) and pass it to `runTests` beside the others.
918
+
919
+ ## Related packages
920
+
921
+ - [`@sdxc/result`](/packages/result) — the `Result` type every fallible export
922
+ returns; errors are values, never throws.
923
+ - [`@sdxc/duration`](/packages/duration) — parses and validates the duration
924
+ literals (`10s`, `500ms`) the language accepts.