@typecad/expect 1.0.0-alpha.13 → 1.0.0-alpha.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,345 +1,417 @@
1
- # @typecad/expect
2
-
3
- Hardware test runner for [TypeCAD](../../README.md). Write vitest-style assertions in TypeScript; the framework compiles them to firmware, uploads to your board, reads the results over serial, and reports pass/fail in one command.
4
-
5
- ```
6
- cuttlefish-test v0.1.0
7
-
8
- ✓ A0 analog read (2 tests)
9
- ✓ reads a value in valid ADC range
10
- ✓ reads less than mid-scale when grounded
11
-
12
- ✓ A1 analog read (2 tests)
13
- ✓ returns a non-negative value
14
- ✓ is within 10-bit ADC range
15
-
16
-
17
- Tests 4 passed (4)
18
- Board @typecad/board-arduino-uno @ COM4
19
- Time 18.97s
20
-
21
- PASS All tests passed
22
- ```
23
-
24
- ---
25
-
26
- ## Table of Contents
27
-
28
- - [How it works](#how-it-works)
29
- - [Installation](#installation)
30
- - [Writing tests](#writing-tests)
31
- - [describe / it](#describe--it)
32
- - [expect](#expect)
33
- - [done](#done)
34
- - [Numeric matchers](#numeric-matchers)
35
- - [String matchers](#string-matchers)
36
- - [Running tests](#running-tests)
37
- - [CLI flags](#cli-flags)
38
- - [cuttlefish.config.ts](#cuttlefishconfigts)
39
- - [Architecture](#architecture)
40
- - [Pipeline](#pipeline)
41
- - [Serial protocol](#serial-protocol)
42
- - [AST preprocessor](#ast-preprocessor)
43
- - [Limitations](#limitations)
44
-
45
- ---
46
-
47
- ## How it works
48
-
49
- 1. **Preprocess** — an AST transform rewrites the fluent test syntax into `Serial.print()` calls.
50
- 2. **Transpile** — Cuttlefish converts the rewritten TypeScript to a C++ Arduino sketch.
51
- 3. **Compile** — `arduino-cli compile` builds the sketch for the target board.
52
- 4. **Upload** — `arduino-cli upload` flashes the firmware over serial.
53
- 5. **Capture** — the host reads structured protocol lines from the serial port.
54
- 6. **Evaluate** — assertion math runs on the host; the firmware only sends raw values.
55
- 7. **Report** — results are printed in vitest-style output.
56
-
57
- ---
58
-
59
- ## Installation
60
-
61
- ```bash
62
- npm install --save-dev @typecad/expect @typecad/cuttlefish
63
- ```
64
-
65
- `@typecad/expect` ships the `cuttlefish-test` CLI. It pairs with [`@typecad/cuttlefish`](https://cuttlefish.typecad.net), which transpiles your TypeScript test files to C++ for upload to hardware.
66
-
67
- **Prerequisites:**
68
-
69
- - [`arduino-cli`](https://arduino.github.io/arduino-cli/) installed and on `PATH`
70
- - The target board core installed (`arduino-cli core install arduino:avr` for Uno)
71
- - A USB serial port available
72
-
73
- ---
74
-
75
- ## Writing tests
76
-
77
- Test files follow a fluent chaining style. Expectations can use direct values or zero-argument functions/IIFEs that return the value to assert.
78
-
79
- ```typescript
80
- // examples/my-sensor.test.ts
81
- import { describe, done } from '@typecad/expect';
82
- import { A0 } from '@typecad/board';
83
-
84
- describe("A0 analog read")
85
- .it("reads a value in valid ADC range")
86
- .expect(A0.readAnalog()).toBeWithinRange(0, 1023)
87
- .it("reads less than mid-scale when grounded")
88
- .expect(A0.readAnalog()).toBeLessThan(512);
89
-
90
- done();
91
- ```
92
-
93
- ### describe / it
94
-
95
- `describe(name: string): Suite`
96
-
97
- Opens a named test group. Returns a `Suite` that you chain `.it()` calls onto.
98
-
99
- `suite.it(name: string): Suite`
100
-
101
- Opens a named test case within the current group. Returns the same `Suite` for further chaining.
102
-
103
- ### expect
104
-
105
- `suite.expect(value: number): Expectation`
106
-
107
- Captures a hardware value to be asserted. The argument must be a TypeCAD hardware expression (e.g. `A0.readAnalog()`, `pin.read()`) or a zero-argument function returning one. The preprocessor hoists hardware expressions so they are evaluated exactly once.
108
-
109
- `suite.expectString(value: string): StringExpectation`
110
-
111
- Same as `expect`, for string-producing expressions.
112
-
113
- ### done
114
-
115
- `done(): void`
116
-
117
- Must be the **last statement** in every test file. Emits the `[TC:SUITE_END]` sentinel over serial and puts the firmware into an idle loop so the host runner knows collection is complete.
118
-
119
- ---
120
-
121
- ### Numeric matchers
122
-
123
- All numeric matchers return the parent `Suite`, so you can continue the chain with `.it()`.
124
-
125
- | Matcher | Passes when |
126
- |---|---|
127
- | `.toBe(n)` | `actual === n` |
128
- | `.toBeGreaterThan(n)` | `actual > n` |
129
- | `.toBeGreaterThanOrEqual(n)` | `actual >= n` |
130
- | `.toBeLessThan(n)` | `actual < n` |
131
- | `.toBeLessThanOrEqual(n)` | `actual <= n` |
132
- | `.toBeCloseTo(n, precision)` | `\|actual n\| < 10^(−precision)` |
133
- | `.toBeWithinRange(min, max)` | `actual >= min && actual <= max` |
134
- | `.toBeTruthy()` | `actual !== 0` |
135
- | `.toBeFalsy()` | `actual === 0` |
136
- | `.toNotBe(n)` | `actual !== n` |
137
-
138
- ### String matchers
139
-
140
- | Matcher | Passes when |
141
- |---|---|
142
- | `.toBe(s)` | `actual === s` |
143
- | `.toContain(sub)` | `actual` contains `sub` |
144
- | `.toHaveLength(n)` | `actual.length === n` |
145
- | `.toNotBe(s)` | `actual !== s` |
146
-
147
- ---
148
-
149
- ## Running tests
150
-
151
- ### CLI
152
-
153
- ```bash
154
- # Run a specific file
155
- npx --package=@typecad/expect cuttlefish-test examples/my-sensor.test.ts
156
-
157
- # Run all test files matched by config include patterns
158
- npx --package=@typecad/expect cuttlefish-test
159
-
160
- # Override the port at run-time
161
- npx --package=@typecad/expect cuttlefish-test --port /dev/ttyACM0 examples/my-sensor.test.ts
162
- ```
163
-
164
- Or via the npm script defined in the root `package.json`:
165
-
166
- ```bash
167
- npm run test:hw
168
- npm run test:hw -- --port COM4
169
- ```
170
-
171
- ### CLI flags
172
-
173
- | Flag | Short | Default | Description |
174
- |---|---|---|---|
175
- | `--port <port>` | `-p` | from config | Serial port (e.g. `COM4`, `/dev/ttyACM0`) |
176
- | `--board <pkg>` | `-b` | from config | Board package name override |
177
- | `--build-target <fqbn>` | | from config | Framework-specific build target / FQBN override |
178
- | `--baud <rate>` | | `115200` | Serial baud rate |
179
- | `--timeout <ms>` | `-t` | `30000` | Serial read timeout in milliseconds |
180
- | `--include <glob>` | `-i` | from config | Test file glob pattern (repeatable) |
181
- | `--exclude <glob>` | `-x` | from config | Test file glob pattern to skip (repeatable) |
182
- | `--verbose` | `-v` | `false` | Show raw serial output and per-assertion detail |
183
- | `--help` | `-h` | | Print help and exit |
184
-
185
- ### cuttlefish.config.ts
186
-
187
- Add a `test` section to your project's `cuttlefish.config.ts` to avoid passing flags every time:
188
-
189
- ```typescript
190
- // cuttlefish.config.ts
191
- import type { CuttlefishConfig } from '@typecad/cuttlefish/api';
192
-
193
- const config: CuttlefishConfig = {
194
- target: 'avr',
195
- board: '@typecad/board-arduino-uno',
196
- framework: '@typecad/framework-arduino',
197
- frameworkData: {
198
- buildTarget: 'arduino:avr:uno',
199
- },
200
-
201
- test: {
202
- port: 'COM4', // serial port of the connected board
203
- baudRate: 115200, // must match Serial.begin() in firmware
204
- timeout: 30000, // ms to wait for SUITE_END before giving up
205
- include: [ // glob patterns for test discovery
206
- 'examples/**/*.test.ts',
207
- 'tests/hardware/**/*.test.ts',
208
- ],
209
- exclude: [ // optional glob patterns to skip after discovery
210
- 'tests/hardware/avr-only/**/*.test.ts',
211
- ],
212
- },
213
- };
214
-
215
- export default config;
216
- ```
217
-
218
- All `test` fields are optional and can be overridden by CLI flags.
219
-
220
- ### Target-specific skips
221
-
222
- Use a file-level comment when a test is valid only for some MCUs or framework targets. The runner checks these comments before preprocessing, compiling, or uploading.
223
-
224
- ```typescript
225
- // @typecad-skip-target esp32: ESP32 does not expose the AVR watchdog API.
226
- ```
227
-
228
- The inverse form skips every target except the listed ones:
229
-
230
- ```typescript
231
- // @typecad-only-target avr,megaavr: uses AVR watchdog registers.
232
- ```
233
-
234
- Targets are matched against `target`, the FQBN parts from `frameworkData.buildTarget` such as `esp32` in `esp32:esp32:esp32`, the full FQBN, and the board package name.
235
-
236
- Skipped files are reported in the same style as Vitest:
237
-
238
- ```text
239
- ↓ tests/32-wdt.test.ts (skipped)
240
-
241
- Tests 1 skipped (1)
242
- Test Files 1 skipped (1)
243
- PASS All tests passed
244
- ```
245
-
246
- Run with `--verbose` to print the skip reason from the directive.
247
-
248
- ### Uno showcase validation example
249
-
250
- Use the stock-Uno showcase in [examples/23-transpiler-showcase.ts](../../examples/23-transpiler-showcase.ts) for manual serial confirmation, then run the companion hardware test in [examples/24-uno-validation.test.ts](../../examples/24-uno-validation.test.ts) for automated checks.
251
-
252
- Example flow:
253
-
254
- ```bash
255
- # 1. Compile and upload the serial-output showcase
256
- npx @typecad/cuttlefish src/23-transpiler-showcase.ts --compile --upload --port COM4
257
-
258
- # 2. Run the on-hardware expect test against the connected Uno
259
- npx --package=@typecad/expect cuttlefish-test examples/24-uno-validation.test.ts --port COM4
260
- ```
261
-
262
- This hybrid workflow is the recommended way to confirm that simple variables, arithmetic, arrays, enums, functions, GPIO, and analog input are behaving correctly on real Uno hardware.
263
-
264
- ---
265
-
266
- ## Architecture
267
-
268
- ### Pipeline
269
-
270
- ```
271
- ┌─────────────────┐
272
- │ test file .ts │ (user-authored TypeScript)
273
- └────────┬────────┘
274
- │ AST preprocessor (host, Node.js)
275
-
276
- ┌─────────────────┐
277
- rewritten .ts │ (Serial.print calls, hoisted hardware vars)
278
- └────────┬────────┘
279
- │ Cuttlefish transpiler
280
-
281
- ┌─────────────────┐
282
- │ .ino sketch │ (Arduino C++)
283
- └────────┬────────┘
284
- │ arduino-cli compile + upload
285
-
286
- ┌─────────────────┐
287
- │ running board │
288
- └────────┬────────┘
289
- │ serial port (structured text lines)
290
-
291
- ┌─────────────────┐
292
- │ host parser │ Node.js builds result tree
293
- └────────┬────────┘
294
- │ evaluator
295
-
296
- ┌─────────────────┐
297
- │ pass / fail │ printed by reporter
298
- └─────────────────┘
299
- ```
300
-
301
- ### Serial protocol
302
-
303
- The firmware emits structured lines that the host runner filters from any other debug output:
304
-
305
- ```
306
- [TC:SUITE_START]
307
- [TC:DESCRIBE:A0 analog read]
308
- [TC:IT:reads a value in valid ADC range]
309
- [TC:EXPECT:toBeWithinRange:0,1023:487]
310
- [TC:IT:reads less than mid-scale when grounded]
311
- [TC:EXPECT:toBeLessThan:512:487]
312
- [TC:SUITE_END]
313
- ```
314
-
315
- All lines not beginning with `[TC:` are ignored, so `Serial.print()` debug statements in imported board libraries do not interfere with results.
316
-
317
- **Assertion line format:** `[TC:EXPECT:<matcher>:<expected>:<actual>]`
318
-
319
- - `expected` — the value(s) from the test source (e.g. `0,1023` for a range)
320
- - `actual` the raw value read from hardware
321
-
322
- Assertion math (pass/fail, formatting) is computed entirely on the host, not in firmware.
323
-
324
- ### AST preprocessor
325
-
326
- The Cuttlefish transpiler cannot evaluate hardware calls (like `A0.readAnalog()`) when they are nested inside non-TypeCAD function calls — they lose their structured IR and become plain text. The preprocessor solves this before transpilation:
327
-
328
- 1. Removes the `import { describe, done } from '@typecad/expect'` statement.
329
- 2. Emits a `Serial.begin(...)` + `[TC:SUITE_START]` preamble once.
330
- 3. Walks the fluent chain `describe(...).it(...).expect(expr).matcher(args)`.
331
- 4. **Hoists** hardware expressions out of `.expect()` into a `const __tc_vN: number = expr;` declaration at the surrounding statement level.
332
- 5. Replaces the `.expect(...).matcher(...)` chain with the appropriate `Serial.print("[TC:EXPECT:...]")` calls.
333
- 6. Rewrites `done()` to `Serial.println("[TC:SUITE_END]") + while(true){delay(1000)}`.
334
-
335
- The result is valid TypeCAD TypeScript with no nested hardware calls, ready for the standard transpiler.
336
-
337
- ---
338
-
339
- ## Limitations
340
-
341
- - **No vitest-style callback suites** — groups and cases are defined by fluent chaining, not by `describe("name", () => { ... })`.
342
- - **No async tests** — all timing is implicit (the board executes sequentially, the host waits on serial output).
343
- - **Sequential execution only** — all describes in a file run once, in order, inside `setup()`. There is no `beforeEach`/`afterEach`.
344
- - **One file per upload** — each test file produces one sketch and one upload cycle. Multiple test files run as separate upload+execute passes.
345
- - **Number types only for hardware values** — TypeCAD maps numeric hardware readings to `int`/`float`. String expectations are for software string variables, not raw hardware reads.
1
+ # @typecad/expect
2
+
3
+ Hardware test runner for [TypeCAD](../../README.md). Write vitest-style assertions in TypeScript; the framework compiles them to firmware, uploads to your board, reads the results over serial, and reports pass/fail in one command.
4
+
5
+ ```
6
+ cuttlefish-test v0.1.0
7
+
8
+ ✓ A0 analog read (2 tests)
9
+ ✓ reads a value in valid ADC range
10
+ ✓ reads less than mid-scale when grounded
11
+
12
+ ✓ A1 analog read (2 tests)
13
+ ✓ returns a non-negative value
14
+ ✓ is within 10-bit ADC range
15
+
16
+
17
+ Tests 4 passed (4)
18
+ Board blackpill_f411ce/stm32f411xe @ COM4
19
+ Time 18.97s
20
+
21
+ PASS All tests passed
22
+ ```
23
+
24
+ ---
25
+
26
+ ## Table of Contents
27
+
28
+ - [How it works](#how-it-works)
29
+ - [Installation](#installation)
30
+ - [Writing tests](#writing-tests)
31
+ - [describe / it](#describe--it)
32
+ - [expect](#expect)
33
+ - [done](#done)
34
+ - [Numeric matchers](#numeric-matchers)
35
+ - [String matchers](#string-matchers)
36
+ - [Running tests](#running-tests)
37
+ - [CLI flags](#cli-flags)
38
+ - [cuttlefish.config.ts](#cuttlefishconfigts)
39
+ - [Architecture](#architecture)
40
+ - [Pipeline](#pipeline)
41
+ - [Serial protocol](#serial-protocol)
42
+ - [AST preprocessor](#ast-preprocessor)
43
+ - [Limitations](#limitations)
44
+
45
+ ---
46
+
47
+ ## How it works
48
+
49
+ 1. **Preprocess** — an AST transform rewrites the fluent test syntax into console `print()` calls.
50
+ 2. **Transpile** — Cuttlefish converts the rewritten TypeScript to a C++ Zephyr program.
51
+ 3. **Compile** — `west build` compiles the program for the target board.
52
+ 4. **Upload** — `west flash` flashes the firmware over serial.
53
+ 5. **Capture** — the host reads structured protocol lines from the serial port.
54
+ 6. **Evaluate** — assertion math runs on the host; the firmware only sends raw values.
55
+ 7. **Report** — results are printed in vitest-style output.
56
+
57
+ ---
58
+
59
+ ## Installation
60
+
61
+ ```bash
62
+ npm install --save-dev @typecad/expect @typecad/cuttlefish
63
+ ```
64
+
65
+ `@typecad/expect` ships the `cuttlefish-test` CLI. It pairs with [`@typecad/cuttlefish`](https://cuttlefish.typecad.net), which transpiles your TypeScript test files to C++ for upload to hardware.
66
+
67
+ **Prerequisites:**
68
+
69
+ - A Zephyr build environment — install it with the bundled installer:
70
+ `npx --package @typecad/framework-zephyr zephyr-installer`
71
+ - A project `cuttlefish.config.ts` naming a board target
72
+ (`board: 'blackpill_f411ce/stm32f411xe'`) and `@typecad/framework-zephyr`
73
+ - A USB serial port available
74
+
75
+ ---
76
+
77
+ ## Writing tests
78
+
79
+ Test files follow a fluent chaining style. Expectations can use direct values or zero-argument functions/IIFEs that return the value to assert.
80
+
81
+ ```typescript
82
+ // examples/my-sensor.test.ts
83
+ import { describe, done } from '@typecad/expect';
84
+ import { ADC_PIN, ADC_MAX } from '@typecad/test-pins';
85
+ import { ADC } from '@typecad/board';
86
+
87
+ describe("A0 analog read")
88
+ .it("reads a value in valid ADC range")
89
+ .expect((() => { const sense = new ADC(ADC_PIN); return sense.read(); })())
90
+ .toBeWithinRange(0, ADC_MAX)
91
+ .it("reads less than mid-scale when grounded")
92
+ .expect((() => { const sense = new ADC(ADC_PIN); return sense.read(); })())
93
+ .toBeLessThan(ADC_MAX / 2);
94
+
95
+ done();
96
+ ```
97
+
98
+ ### describe / it
99
+
100
+ `describe(name: string): Suite`
101
+
102
+ Opens a named test group. Returns a `Suite` that you chain `.it()` calls onto.
103
+
104
+ `suite.it(name: string): Suite`
105
+
106
+ Opens a named test case within the current group. Returns the same `Suite` for further chaining.
107
+
108
+ ### expect
109
+
110
+ `suite.expect(value: number): Expectation`
111
+
112
+ Captures a hardware value to be asserted. The argument must be a TypeCAD hardware expression (e.g. `sense.read()`, `pin.get()`) or a zero-argument function returning one. The preprocessor hoists hardware expressions so they are evaluated exactly once.
113
+
114
+ `suite.expectString(value: string): StringExpectation`
115
+
116
+ Same as `expect`, for string-producing expressions.
117
+
118
+ ### done
119
+
120
+ `done(): void`
121
+
122
+ Must be the **last statement** in every test file. Emits the `[TC:SUITE_END]` sentinel over serial and puts the firmware into an idle loop so the host runner knows collection is complete.
123
+
124
+ ---
125
+
126
+ ### Numeric matchers
127
+
128
+ All numeric matchers return the parent `Suite`, so you can continue the chain with `.it()`.
129
+
130
+ | Matcher | Passes when |
131
+ |---|---|
132
+ | `.toBe(n)` | `actual === n` |
133
+ | `.toBeGreaterThan(n)` | `actual > n` |
134
+ | `.toBeGreaterThanOrEqual(n)` | `actual >= n` |
135
+ | `.toBeLessThan(n)` | `actual < n` |
136
+ | `.toBeLessThanOrEqual(n)` | `actual <= n` |
137
+ | `.toBeCloseTo(n, precision)` | `\|actual − n\| < 10^(−precision)` |
138
+ | `.toBeWithinRange(min, max)` | `actual >= min && actual <= max` |
139
+ | `.toBeTruthy()` | `actual !== 0` |
140
+ | `.toBeFalsy()` | `actual === 0` |
141
+ | `.toNotBe(n)` | `actual !== n` |
142
+
143
+ ### String matchers
144
+
145
+ | Matcher | Passes when |
146
+ |---|---|
147
+ | `.toBe(s)` | `actual === s` |
148
+ | `.toContain(sub)` | `actual` contains `sub` |
149
+ | `.toHaveLength(n)` | `actual.length === n` |
150
+ | `.toNotBe(s)` | `actual !== s` |
151
+
152
+ ---
153
+
154
+ ## Running tests
155
+
156
+ ### CLI
157
+
158
+ ```bash
159
+ # Run a specific file
160
+ npx --package=@typecad/expect cuttlefish-test examples/my-sensor.test.ts
161
+
162
+ # Run all test files matched by config include patterns
163
+ npx --package=@typecad/expect cuttlefish-test
164
+
165
+ # Override the port at run-time
166
+ npx --package=@typecad/expect cuttlefish-test --port /dev/ttyACM0 examples/my-sensor.test.ts
167
+ ```
168
+
169
+ Or via the npm script defined in the root `package.json`:
170
+
171
+ ```bash
172
+ npm run test:hw
173
+ npm run test:hw -- --port COM4
174
+ ```
175
+
176
+ ### CLI flags
177
+
178
+ | Flag | Short | Default | Description |
179
+ |---|---|---|---|
180
+ | `--port <port>` | `-p` | from config | Serial port (e.g. `COM4`, `/dev/ttyACM0`) |
181
+ | `--board <pkg>` | `-b` | from config | Board target override |
182
+ | `--build-target <id>` | | from config | Framework build target override (e.g. `blackpill_f411ce/stm32f411xe`) |
183
+ | `--baud <rate>` | | `115200` | Serial baud rate |
184
+ | `--timeout <ms>` | `-t` | `30000` | Serial read timeout in milliseconds |
185
+ | `--include <glob>` | `-i` | from config | Test file glob pattern (repeatable) |
186
+ | `--exclude <glob>` | `-x` | from config | Test file glob pattern to skip (repeatable) |
187
+ | `--verbose` | `-v` | `false` | Show raw serial output and per-assertion detail |
188
+ | `--config <path>` | | `cuttlefish.config.ts` | Config file to load (suite dirs pass a board-specific config) |
189
+ | `--discover` | | | List attached USB serial ports (VID:PID, serial, manufacturer) and which one the active board identity matches, then exit |
190
+ | `--dry-run` | | | Compile only — skip upload and serial execution (pipeline check without hardware) |
191
+ | `--bail` | | | Stop after the first failing file |
192
+ | `--help` | `-h` | | Print help and exit |
193
+
194
+ ### USB port discovery (multi-board rigs)
195
+
196
+ COM/tty numbers reshuffle on every replug and on every CDC re-enumeration
197
+ after a flash, so a test box with several boards identifies them by USB
198
+ VID/PID (+ optional serial number) instead:
199
+
200
+ ```typescript
201
+ test: {
202
+ usb: { vid: '2FE3', pid: '0001' }, // resolves the port by identity
203
+ },
204
+ ```
205
+
206
+ Board packages that ship a `test-pins.json` can carry the same `usb` block —
207
+ then no config change is needed at all (an explicit `test.usb` in the config
208
+ wins). Zephyr CDC consoles enumerate at the Zephyr test IDs `2FE3:0001` for
209
+ every board (per-board PIDs are no longer assigned), so vid/pid alone cannot
210
+ distinguish several attached CDC boards — run one CDC board at a time.
211
+ UART-bridge boards identify by their bridge chip (Uno 16U2 `2341:0043`,
212
+ CH340 clones `1A86:7523`; ESP32 DevKitC CP2102 `10C4:EA60`).
213
+ Several identical devkits disambiguate with the bridge's USB serial number:
214
+ `usb: { vid: '10C4', pid: 'EA60', serial: '0001' }`.
215
+
216
+ When a USB identity is active the runner re-resolves the port **after every
217
+ upload** — a CDC console that comes back under a different COM number is
218
+ found again automatically. Discovery failures are loud and list every
219
+ attached port (what a nightly log wants). An explicit `--port` flag or
220
+ `test.port` always overrides discovery.
221
+
222
+ Bring a rig up with `--discover`: it prints every attached port's
223
+ VID:PID/serial/manufacturer and marks which one the current config matches
224
+ (exit code 1 when the identity has no unique match, so scripts can gate).
225
+
226
+ ```bash
227
+ cuttlefish-test --config boards/blackpill/cuttlefish.config.ts --discover
228
+ # USB serial ports:
229
+ # COM7 2FE3:0001 serial … <-- matches this config
230
+ # COM4 10C4:EA60 serial 0001 [Silicon Labs]
231
+ # config identity 2FE3:0001 -> COM7
232
+ ```
233
+
234
+ ### cuttlefish.config.ts
235
+
236
+ Add a `test` section to your project's `cuttlefish.config.ts` to avoid passing flags every time:
237
+
238
+ ```typescript
239
+ // cuttlefish.config.ts
240
+ import type { CuttlefishConfig } from '@typecad/cuttlefish/api';
241
+
242
+ const config: CuttlefishConfig = {
243
+ entry: './src/main.ts',
244
+ board: 'blackpill_f411ce/stm32f411xe',
245
+ framework: '@typecad/framework-zephyr',
246
+
247
+ test: {
248
+ port: 'COM4', // serial port of the connected board
249
+ baudRate: 115200, // must match the console's baud rate
250
+ timeout: 30000, // ms to wait for SUITE_END before giving up
251
+ include: [ // glob patterns for test discovery
252
+ 'examples/**/*.test.ts',
253
+ 'tests/hardware/**/*.test.ts',
254
+ ],
255
+ exclude: [ // optional glob patterns to skip after discovery
256
+ 'tests/hardware/network/**/*.test.ts',
257
+ ],
258
+ },
259
+ };
260
+
261
+ export default config;
262
+ ```
263
+
264
+ All `test` fields are optional and can be overridden by CLI flags.
265
+
266
+ ### Target-specific skips
267
+
268
+ Use a file-level comment when a test is valid only for some MCUs or framework targets. The runner checks these comments before preprocessing, compiling, or uploading.
269
+
270
+ ```typescript
271
+ // @typecad-skip-target native: this group drives Zephyr console timing.
272
+ ```
273
+
274
+ The inverse form skips every target except the listed ones:
275
+
276
+ ```typescript
277
+ // @typecad-only-target esp32s3_devkitc,blackpill_f411ce: pins live in test-pins.json for these boards.
278
+ ```
279
+
280
+ Targets are matched against `target`, the `buildTarget` id, and the final `/`-segment of the board target (e.g. `stm32f411xe` from `blackpill_f411ce/stm32f411xe`) — `*` matches everything.
281
+
282
+ Skipped files are reported in the same style as Vitest:
283
+
284
+ ```text
285
+ ↓ tests/32-wdt.test.ts (skipped)
286
+
287
+ Tests 1 skipped (1)
288
+ Test Files 1 skipped (1)
289
+ PASS All tests passed
290
+ ```
291
+
292
+ Run with `--verbose` to print the skip reason from the directive.
293
+
294
+ ### Board test-pin roles (`@typecad/test-pins`)
295
+
296
+ Each board config ships a `test-pins.json` (co-located with its
297
+ `cuttlefish.config.ts`, e.g. `packages/hal/boards/<name>/`) declaring which
298
+ pins a hardware suite may use and the board's numeric facts:
299
+
300
+ ```jsonc
301
+ {
302
+ "pins": {
303
+ "gpioOut": "PB5",
304
+ "gpioIn": "PB0",
305
+ "pwm": "PB6", "pwmAlt": "PB7",
306
+ "cs": "PA4", "interrupt": "PA0",
307
+ "led": "LED", "button": "BUTTON",
308
+ "adcPin": "PA1", "adcPinAlt": "PA2",
309
+ "i2cBus": "'I2C0'"
310
+ },
311
+ "facts": {
312
+ "adcMax": 4095
313
+ }
314
+ }
315
+ ```
316
+
317
+ Test files import stable role names instead of board-specific pin symbols:
318
+
319
+ ```typescript
320
+ import { GPIO_OUT, PWM_PIN, ADC_PIN, ADC_MAX } from '@typecad/test-pins';
321
+ ```
322
+
323
+ During preprocessing the runner substitutes each role with the configured board's pin symbol (facts become numeric literals) and rewrites the import to `@typecad/board` — the exact lowering path hand-written per-board tests use. Files declare the roles they need so they skip cleanly on boards that cannot provide them:
324
+
325
+ ```typescript
326
+ // @typecad-requires-roles adcPin, adcMax
327
+ ```
328
+
329
+ Because expect matcher arguments must be literals, compare facts on-device inside the `expect()` IIFE:
330
+
331
+ ```typescript
332
+ describe("ADC upper bound")
333
+ .expect((() => { const sense = new ADC(ADC_PIN); return sense.read() <= ADC_MAX ? 1 : 0; })()).toBe(1)
334
+ ```
335
+
336
+ ---
337
+
338
+ ## Architecture
339
+
340
+ ### Pipeline
341
+
342
+ ```
343
+ ┌─────────────────┐
344
+ test file .ts │ (user-authored TypeScript)
345
+ └────────┬────────┘
346
+ │ AST preprocessor (host, Node.js)
347
+
348
+ ┌─────────────────┐
349
+ │ rewritten .ts │ (console print calls, hoisted hardware vars)
350
+ └────────┬────────┘
351
+ │ Cuttlefish transpiler
352
+
353
+ ┌─────────────────┐
354
+ │ .cpp program │ (Zephyr C++)
355
+ └────────┬────────┘
356
+ │ west build + flash
357
+
358
+ ┌─────────────────┐
359
+ │ running board │
360
+ └────────┬────────┘
361
+ │ serial port (structured text lines)
362
+
363
+ ┌─────────────────┐
364
+ │ host parser │ Node.js — builds result tree
365
+ └────────┬────────┘
366
+ │ evaluator
367
+
368
+ ┌─────────────────┐
369
+ │ pass / fail │ printed by reporter
370
+ └─────────────────┘
371
+ ```
372
+
373
+ ### Serial protocol
374
+
375
+ The firmware emits structured lines that the host runner filters from any other debug output:
376
+
377
+ ```
378
+ [TC:SUITE_START]
379
+ [TC:DESCRIBE:A0 analog read]
380
+ [TC:IT:reads a value in valid ADC range]
381
+ [TC:EXPECT:toBeWithinRange:0,1023:487]
382
+ [TC:IT:reads less than mid-scale when grounded]
383
+ [TC:EXPECT:toBeLessThan:512:487]
384
+ [TC:SUITE_END]
385
+ ```
386
+
387
+ All lines not beginning with `[TC:` are ignored, so any other console output (`printk` debug prints, shell output) does not interfere with results.
388
+
389
+ **Assertion line format:** `[TC:EXPECT:<matcher>:<expected>:<actual>]`
390
+
391
+ - `expected` — the value(s) from the test source (e.g. `0,1023` for a range)
392
+ - `actual` — the raw value read from hardware
393
+
394
+ Assertion math (pass/fail, formatting) is computed entirely on the host, not in firmware.
395
+
396
+ ### AST preprocessor
397
+
398
+ The Cuttlefish transpiler cannot evaluate hardware calls (like `sense.read()`) when they are nested inside non-TypeCAD function calls — they lose their structured IR and become plain text. The preprocessor solves this before transpilation:
399
+
400
+ 1. Removes the `import { describe, done } from '@typecad/expect'` statement.
401
+ 2. Emits the console init + `[TC:SUITE_START]` preamble once (the Zephyr console self-initializes — no init call is needed).
402
+ 3. Walks the fluent chain `describe(...).it(...).expect(expr).matcher(args)`.
403
+ 4. **Hoists** hardware expressions out of `.expect()` into a `const __tc_vN: number = expr;` declaration at the surrounding statement level.
404
+ 5. Replaces the `.expect(...).matcher(...)` chain with the appropriate `__tc_print("[TC:EXPECT:...]")` calls.
405
+ 6. Rewrites `done()` to `__tc_println("[TC:SUITE_END]")` + `while (true) { k_msleep(1000); }`.
406
+
407
+ The result is valid TypeCAD TypeScript with no nested hardware calls, ready for the standard transpiler.
408
+
409
+ ---
410
+
411
+ ## Limitations
412
+
413
+ - **No vitest-style callback suites** — groups and cases are defined by fluent chaining, not by `describe("name", () => { ... })`.
414
+ - **No async tests** — all timing is implicit (the board executes sequentially, the host waits on serial output).
415
+ - **Sequential execution only** — all describes in a file run once, in order, from the program's top-level statements. There is no `beforeEach`/`afterEach`.
416
+ - **One file per upload** — each test file produces one program and one upload cycle. Multiple test files run as separate upload+execute passes.
417
+ - **Number types only for hardware values** — TypeCAD maps numeric hardware readings to `int`/`float`. String expectations are for software string variables, not raw hardware reads.