@rhost/testkit 0.1.0 → 1.3.1

package/README.md

@@ -23,13 +23,20 @@ npm install @rhost/testkit
 - [Quick start](#quick-start)
 - [API reference — RhostRunner](#api-reference--rhostrunner)
 - [API reference — RhostExpect](#api-reference--rhostexpect)
+- [Snapshot testing](#snapshot-testing)
 - [API reference — RhostWorld](#api-reference--rhostworld)
 - [API reference — RhostClient](#api-reference--rhostclient)
 - [API reference — RhostContainer](#api-reference--rhostcontainer)
+- [Offline validator](#offline-validator)
+- [Softcode formatter](#softcode-formatter)
+- [Benchmark mode](#benchmark-mode)
+- [Watch mode](#watch-mode)
+- [CI/CD templates](#cicd-templates)
 - [MUSH output format](#mush-output-format)
 - [Using with LLM skills](#using-with-llm-skills)
 - [Environment variables](#environment-variables)
 - [Examples](#examples)
+- [Roadmap](#roadmap)
 
 ---
 
@@ -39,7 +46,14 @@ npm install @rhost/testkit
 
 - **Eval** softcode expressions and capture their output
 - **Assert** results with a Jest-like `expect()` API and MUSH-aware matchers
-- **Manage fixtures** — create objects, set attributes, and auto-destroy everything after each test
+- **Snapshot test** softcode output — first run writes, subsequent runs compare with a diff on mismatch
+- **Preview** raw server output exactly as a MUSH client sees it — ANSI colours and all
+- **Manage fixtures** — create/destroy objects, set attributes, force commands, send mail, and more
+- **Validate softcode offline** — catch syntax errors, wrong arg counts, register clobber risks, and dialect compatibility without a server
+- **Format softcode** — normalize whitespace and optionally indent nested calls (`rhost-testkit fmt`)
+- **Benchmark** expressions — measure median / p95 / p99 latency per softcode call against a live server
+- **Watch mode** — re-run tests on save with a 300ms debounce
+- **Generate CI/CD workflows** — one command to get GitHub Actions or GitLab CI configured
 - **Spin up RhostMUSH in Docker** for isolated, reproducible CI runs
 - **Report** results with a pretty, indented pass/fail tree
 
@@ -251,9 +265,10 @@ interface SuiteContext {
 
 ```typescript
 interface TestContext {
-    expect(expression: string): RhostExpect   // create an expect for a softcode expression
-    client: RhostClient                        // the live MUSH connection
-    world:  RhostWorld                         // fresh per-test fixture manager (auto-cleaned)
+    expect(expression: string): RhostExpect              // create an expect for a softcode expression
+    preview(input: string, opts?: PreviewOptions): Promise<string>  // print raw server output
+    client: RhostClient                                  // the live MUSH connection
+    world:  RhostWorld                                   // fresh per-test fixture manager (auto-cleaned)
 }
 ```
 
@@ -323,6 +338,7 @@ await expect('add(1,1)').not.toBeError();
 | `.toBeDbref()` | Result matches `/^#\d+$/` (a positive object reference). |
 | `.toContainWord(word: string, sep?: string)` | Word is present in the space-delimited list (or custom separator). |
 | `.toHaveWordCount(n: number, sep?: string)` | List has exactly `n` words. Empty string has 0. |
+| `.toMatchSnapshot()` | Compare output against a stored snapshot; writes on first run. |
 
 ### Failure message format
 
@@ -353,6 +369,45 @@ await client.disconnect();
 
 ---
 
+## Snapshot testing
+
+Snapshot tests lock in the output of a softcode expression. The first run writes the value to a `.snap` file; subsequent runs compare against it and diff on mismatch.
+
+```typescript
+runner.describe('iter output', ({ it }) => {
+    it('produces the right sequence', async ({ expect }) => {
+        await expect('iter(lnum(1,5),##)').toMatchSnapshot();
+    });
+});
+```
+
+On first run, a file like `__snapshots__/my-tests.test.ts.snap` is created:
+```json
+{
+  "iter output > produces the right sequence: 1": "1 2 3 4 5"
+}
+```
+
+On subsequent runs the stored value is compared. If it differs, the test fails with a line-by-line diff:
+```
+- 1 2 3 4 5
++ 1 2 3 4 5 6
+```
+
+To update all snapshots (e.g. after an intentional change):
+
+```bash
+RHOST_UPDATE_SNAPSHOTS=1 npx ts-node my-tests.test.ts
+```
+
+Or pass `updateSnapshots: true` to `runner.run()`.
+
+The runner prints a snapshot summary after each run: `Snapshots: 3 passed, 1 written, 0 updated`.
+
+**Snapshot key format:** `"Suite > Sub-suite > Test name: N"` — where `N` is the 1-based call count within the test. Calling `.toMatchSnapshot()` twice in one test produces two separate keys.
+
+---
+
 ## API reference — RhostWorld
 
 `RhostWorld` manages MUSH object fixtures. Objects created through `world` are registered and destroyed automatically in `world.cleanup()`. In the runner, `cleanup()` is called after every `it()` test — even on failure.
@@ -400,6 +455,51 @@ await world.trigger(dbref: string, attr: string, args?: string): Promise<string[
 ```
 Triggers `@trigger dbref/attr=args`. Returns all output lines captured before the sentinel.
 
+```typescript
+await world.pemit(target: string, msg: string): Promise<void>
+```
+Emits a message to a target: `@pemit target=msg`.
+
+```typescript
+await world.remit(room: string, msg: string): Promise<void>
+```
+Emits a message to all objects in a room: `@remit room=msg`.
+
+```typescript
+await world.force(actor: string, cmd: string): Promise<void>
+```
+Forces an object to run a command: `@force actor=cmd`.
+
+```typescript
+await world.parent(child: string, parent: string): Promise<void>
+```
+Sets a parent object: `@parent child=parent`.
+
+```typescript
+await world.zone(name: string): Promise<string>
+```
+Creates a zone room via `@dig name` and sets `INHERIT_ZONE`. Returns the room dbref. Registers for cleanup.
+
+```typescript
+await world.addToChannel(dbref: string, chan: string): Promise<void>
+```
+Adds an object to a channel: `@channel/add chan=dbref`.
+
+```typescript
+await world.grantQuota(dbref: string, n: number): Promise<void>
+```
+Sets the build quota for an object: `@quota/set dbref=n`.
+
+```typescript
+await world.wait(ms: number): Promise<void>
+```
+Pauses the test for `ms` milliseconds. Plain JavaScript delay — not a MUSH `@wait`.
+
+```typescript
+await world.mail(to: string, subj: string, body: string): Promise<void>
+```
+Sends in-game mail: `@mail to=subj/body`.
+
 ```typescript
 await world.destroy(dbref: string): Promise<void>
 ```
@@ -466,7 +566,7 @@ Establishes the TCP connection and drains the welcome banner.
 ```typescript
 await client.login(username: string, password: string): Promise<void>
 ```
-Sends `connect <username> <password>` and waits for the login sentinel. Throws `RangeError` if either credential contains `\n` or `\r`.
+Sends `connect <username> <password>` and waits for the login sentinel. Throws `RangeError` if the username contains `\n`, `\r`, spaces, or tabs (which would split the MUSH connect command), or if the password contains `\n` or `\r`.
 
 ```typescript
 await client.eval(expression: string, timeout?: number): Promise<string>
@@ -478,6 +578,32 @@ await client.command(cmd: string, timeout?: number): Promise<string[]>
 ```
 Sends a MUSH command and captures all output lines until the sentinel. Returns an array of lines (may be empty).
 
+```typescript
+await client.preview(input: string, options?: PreviewOptions): Promise<string>
+```
+Evaluate an expression or run a command and print the raw server output to stdout exactly as a MUSH client receives it — ANSI colours, formatting codes, and all. Output is rendered in a labelled frame. Returns the raw string so you can still assert on it.
+
+**`PreviewOptions`:**
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `mode` | `'eval' \| 'command'` | `'eval'` | `'eval'` wraps in `think`; `'command'` sends as a raw MUSH command |
+| `label` | `string` | the input string | Custom frame header label |
+| `timeout` | `number` | client default | Per-call timeout (ms) |
+| `print` | `boolean` | `true` | Set `false` to suppress stdout and only use the return value |
+
+```typescript
+// Softcode expression — see the raw colour output
+await preview('ansi(rh,CRITICAL HIT!)');
+
+// Room description as a player sees it
+await preview('look here', { mode: 'command' });
+
+// Assert without printing
+const raw = await preview('ansi(b,test)', { print: false });
+expect(stripAnsi(raw)).toBe('test');
+```
+
 ```typescript
 client.onLine(handler: (line: string) => void): void
 client.offLine(handler: (line: string) => void): void
@@ -579,6 +705,242 @@ process.exit(result.failed > 0 ? 1 : 0);
 
 ---
 
+## Offline validator
+
+Validate softcode expressions without a server connection. Catches structural errors (unbalanced parens/brackets), wrong argument counts, and unknown built-in functions.
+
+### Programmatic API
+
+```typescript
+import { validate, validateFile } from '@rhost/testkit/validator';
+
+const result = validate('add(2,3)');
+// result.valid       => true
+// result.diagnostics => []
+
+const bad = validate('add(2,3');
+// bad.valid                 => false
+// bad.diagnostics[0].code   => 'E001'
+// bad.diagnostics[0].message => "Unclosed '(' ..."
+```
+
+```typescript
+const result = validateFile('./mycode.mush');
+```
+
+### CLI
+
+```bash
+npx rhost-testkit validate "add(2,3)"
+npx rhost-testkit validate --file mycode.mush
+npx rhost-testkit validate --json "abs(1,2)"   # machine-readable output
+```
+
+Exit code `0` = valid (warnings allowed), `1` = one or more errors.
+
+### Diagnostic codes
+
+| Code | Severity | Meaning |
+|------|----------|---------|
+| `E001` | error | Unclosed `(` |
+| `E002` | error | Unexpected `)` |
+| `E003` | error | Unclosed `[` |
+| `E004` | error | Unexpected `]` |
+| `E006` | error | Too few arguments for known built-in |
+| `E007` | error | Too many arguments for known built-in |
+| `W001` | warning | Empty expression |
+| `W002` | warning | Empty argument (e.g. `add(,3)`) |
+| `W003` | warning | Deprecated function |
+| `W005` | warning | Unknown function name (may be a UDF) |
+| `W006` | warning | Register clobber risk — `setq()` inside a loop body may overwrite registers in concurrent iterations |
+
+### Dialect compatibility report
+
+Report which functions are portable across MUSH platforms (RhostMUSH, PennMUSH, TinyMUX):
+
+```typescript
+import { compatibilityReport } from '@rhost/testkit';
+
+const report = compatibilityReport('json(get,key)');
+// report.portable  => false
+// report.restricted => [{ name: 'json', platforms: ['rhost'] }]
+```
+
+```bash
+npx rhost-testkit validate --compat mycode.mush
+```
+
+### Register clobber analysis
+
+`%q0`–`%q9` registers are scoped per queue entry. The validator warns when `setq()` appears inside a loop body (`iter()`, `parse()`, `map()`, etc.) where concurrent invocations can silently overwrite each other's registers:
+
+```bash
+$ rhost-testkit validate --file combat.mush
+
+WARNING W006: register clobber risk
+  setq() writes %q0 inside iter() at offset 14.
+  Wrap in localize() to scope registers to each iteration.
+```
+
+---
+
+## Softcode formatter
+
+Normalize whitespace in softcode files — strips extra spaces around `(`, `,`, `)` while preserving interior argument text.
+
+### CLI
+
+```bash
+# Format a file in-place
+npx rhost-testkit fmt mycode.mush
+
+# Check without writing (exit 1 if not formatted — useful in CI)
+npx rhost-testkit fmt --check mycode.mush
+
+# Indent nested function calls for human readability
+npx rhost-testkit fmt --pretty mycode.mush
+
+# Normalize function names to lowercase
+npx rhost-testkit fmt --lowercase mycode.mush
+
+# Format from stdin
+echo "add( 2, 3 )" | npx rhost-testkit fmt
+```
+
+### Programmatic API
+
+```typescript
+import { format } from '@rhost/testkit';
+
+const result = format('add( 2, 3 )');
+// result.formatted => 'add(2,3)'
+// result.changed   => true
+
+// Pretty mode — indents nested calls for readability (not for upload)
+const pretty = format('add(mul(2,3),4)', { pretty: true });
+// pretty.formatted => 'add(\n  mul(2,3),\n  4\n)'
+
+// Lowercase function names
+const lower = format('ADD(2,3)', { lowercase: true });
+// lower.formatted => 'add(2,3)'
+```
+
+Interior whitespace within argument text is preserved — `pemit(%#,hello world)` is not changed.
+
+---
+
+## Benchmark mode
+
+Profile softcode performance against a live server. Reports median, p95, and p99 latency per expression.
+
+### Programmatic API
+
+```typescript
+import { RhostBenchmark, formatBenchResults } from '@rhost/testkit';
+
+const bench = new RhostBenchmark(client);
+
+bench
+  .add('add(2,3)', { name: 'addition', iterations: 100, warmup: 10 })
+  .add('iter(lnum(1,100),##)', { name: 'heavy iter', iterations: 50, warmup: 5 });
+
+const results = await bench.run();
+console.log(formatBenchResults(results));
+```
+
+Output:
+
+```
+Benchmark Results
+────────────────────────────────────────────────────────────────────────
+  addition
+  iterations: 100  warmup: 10
+  median: 4.231ms  mean: 4.512ms  p95: 7.820ms  p99: 12.003ms
+  min: 3.901ms  max: 14.221ms
+
+  heavy iter
+  iterations: 50  warmup: 5
+  median: 18.440ms  mean: 19.201ms  p95: 31.100ms  p99: 38.500ms
+  min: 16.002ms  max: 41.300ms
+────────────────────────────────────────────────────────────────────────
+```
+
+### `runBench` — single expression
+
+```typescript
+import { runBench } from '@rhost/testkit';
+
+const result = await runBench(client, 'iter(lnum(1,1000),##)', {
+  name: 'heavy iter',
+  iterations: 100,
+  warmup: 10,
+});
+
+console.log(`median: ${result.median.toFixed(2)}ms`);
+console.log(`p95: ${result.p95.toFixed(2)}ms`);
+```
+
+### `BenchmarkResult` shape
+
+```typescript
+interface BenchmarkResult {
+  name:       string;
+  iterations: number;
+  warmup:     number;
+  samples:    number[];   // raw timings in ms, in run order
+  mean:       number;
+  median:     number;
+  p95:        number;
+  p99:        number;
+  min:        number;
+  max:        number;
+}
+```
+
+---
+
+## Watch mode
+
+Re-run test files automatically on save.
+
+```bash
+# Auto-discover *.test.ts / *.spec.ts under the current directory
+npx rhost-testkit watch
+
+# Watch specific files
+npx rhost-testkit watch src/__tests__/math.test.ts
+
+# Options
+npx rhost-testkit watch --debounce 500   # longer debounce (default: 300ms)
+npx rhost-testkit watch --no-clear       # don't clear terminal between runs
+```
+
+TypeScript files are run with `ts-node --transpile-only`. Plain JS files are run with Node directly. Watch mode exits cleanly on Ctrl+C.
+
+---
+
+## CI/CD templates
+
+Generate a ready-to-use workflow file for your CI platform with one command:
+
+```bash
+# GitHub Actions → .github/workflows/mush-tests.yml
+npx rhost-testkit init --ci github
+
+# GitLab CI → .gitlab-ci.yml
+npx rhost-testkit init --ci gitlab
+
+# Overwrite an existing file
+npx rhost-testkit init --ci github --force
+```
+
+The generated file includes:
+- Node.js 20 setup
+- `npm ci` + `npm test`
+- A commented-out block for optional integration tests against a `rhostmush/rhostmush` Docker container — uncomment and set `RHOST_PASS` in your secrets to enable
+
+---
+
 ## MUSH output format
 
 Understanding what `client.eval()` returns is essential for writing correct assertions.
@@ -856,6 +1218,32 @@ npm run example:01
 
 ---
 
+## Roadmap
+
+See [ROADMAP.md](./ROADMAP.md) for full details and implementation notes.
+
+### ✅ Shipped
+
+| Version | Features |
+|---------|----------|
+| v0.2.0 | Offline validator · Watch mode · Snapshot testing |
+| v1.0.0 | Extended world API · CI/CD templates |
+| v1.1.0 | Server pre-flight assertions · Multi-persona test matrix · Side-effect assertion mode |
+| v1.2.0 | Register clobber analyzer · Deploy pipeline with rollback · Dialect compatibility report |
+| v1.3.0 | **Softcode formatter** (`rhost-testkit fmt`) · **Benchmark mode** (`RhostBenchmark`) |
+
+### Planned
+
+**Test coverage tracking** — report which attributes on which objects were exercised during a run.
+
+**Interactive REPL** — persistent connection with readline history and tab-complete for built-in functions.
+
+**Parallel test execution** — run independent `describe` blocks concurrently on separate connections.
+
+**Recursion depth profiler** — track max call depth per expression, warn before hitting server recursion limits.
+
+---
+
 ## License
 
 MIT