sounding 0.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +10 -0
- package/RESEARCH.md +426 -0
- package/index.js +4 -0
- package/package.json +23 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# Sounding
|
|
2
|
+
|
|
3
|
+
Sounding is a testing framework for Sails applications and The Boring JavaScript Stack.
|
|
4
|
+
|
|
5
|
+
This repository currently starts with research and product design work for an elegant testing story built on:
|
|
6
|
+
- the native Node.js test runner
|
|
7
|
+
- Playwright for browser testing
|
|
8
|
+
- Captain Vane for factories and scenarios
|
|
9
|
+
|
|
10
|
+
See `RESEARCH.md`.
|
package/RESEARCH.md
ADDED
|
@@ -0,0 +1,426 @@
|
|
|
1
|
+
# Sounding
|
|
2
|
+
|
|
3
|
+
## Working thesis
|
|
4
|
+
|
|
5
|
+
Sounding is a testing framework for Sails applications and The Boring JavaScript Stack.
|
|
6
|
+
|
|
7
|
+
It should make it feel natural to test:
|
|
8
|
+
- helpers and business logic
|
|
9
|
+
- actions and endpoints
|
|
10
|
+
- Inertia responses
|
|
11
|
+
- authentication flows
|
|
12
|
+
- email flows
|
|
13
|
+
- browser journeys
|
|
14
|
+
- sockets, jobs, payments, and webhooks over time
|
|
15
|
+
|
|
16
|
+
The key idea is simple:
|
|
17
|
+
|
|
18
|
+
**Use the native Node.js test runner, use Playwright for browser work, and wrap both in a Sails-aware runtime that makes realistic tests easy to write and easy to trust.**
|
|
19
|
+
|
|
20
|
+
This should feel less like "yet another framework" and more like the missing test home for everything TBJS already does.
|
|
21
|
+
|
|
22
|
+
## Why the name
|
|
23
|
+
|
|
24
|
+
Sounding is the act of measuring depth before you commit the ship.
|
|
25
|
+
|
|
26
|
+
It is how mariners probe unknown waters, verify what is safe, and learn what lies beneath the surface.
|
|
27
|
+
|
|
28
|
+
That maps naturally to testing.
|
|
29
|
+
|
|
30
|
+
Sounding gives us a name that suggests:
|
|
31
|
+
- probing the system
|
|
32
|
+
- measuring the unknown
|
|
33
|
+
- learning before committing
|
|
34
|
+
- confidence before open water
|
|
35
|
+
|
|
36
|
+
It is also:
|
|
37
|
+
- one word
|
|
38
|
+
- maritime without being too cute
|
|
39
|
+
- broad enough for unit, integration, endpoint, and browser testing
|
|
40
|
+
- distinct from Captain Vane, which can remain the data/scenario engine underneath
|
|
41
|
+
|
|
42
|
+
## The problem we are actually solving
|
|
43
|
+
|
|
44
|
+
The TBJS testing story is close, but still fragmented.
|
|
45
|
+
|
|
46
|
+
Today we have pieces:
|
|
47
|
+
- `node:test` for unit-style tests
|
|
48
|
+
- Playwright for browser flows
|
|
49
|
+
- `inertia-sails/test` for response assertions
|
|
50
|
+
- `getSails()` patterns for loading the app in tests
|
|
51
|
+
- ad hoc seeding and fixture code per application
|
|
52
|
+
|
|
53
|
+
What we do **not** have yet is a coherent testing runtime that feels native to Sails.
|
|
54
|
+
|
|
55
|
+
Current pain points:
|
|
56
|
+
- tests often need too much setup ceremony
|
|
57
|
+
- data setup is repetitive and not expressive enough
|
|
58
|
+
- E2E and app boot orchestration can get ugly fast
|
|
59
|
+
- `sails-disk` and multi-process test setups collide in painful ways
|
|
60
|
+
- realistic auth/email/payment flows are still too manual to test cleanly
|
|
61
|
+
- people are tempted to add app-code test hooks just to make tests possible
|
|
62
|
+
|
|
63
|
+
The missing thing is not just a test runner.
|
|
64
|
+
|
|
65
|
+
The missing thing is an **elegant testing story**.
|
|
66
|
+
|
|
67
|
+
## Design principles
|
|
68
|
+
|
|
69
|
+
### 1. Native first
|
|
70
|
+
Sounding should build on the native Node.js test runner, not compete with it.
|
|
71
|
+
|
|
72
|
+
### 2. Sails-aware, not Sails-entangled
|
|
73
|
+
It should understand helpers, actions, policies, sessions, Waterline, Inertia, mail, sockets, and jobs.
|
|
74
|
+
But it should not force awkward test-only app code.
|
|
75
|
+
|
|
76
|
+
### 3. Tests own test data
|
|
77
|
+
Factories, traits, scenarios, and fixtures should live under `tests/`, not in the app runtime.
|
|
78
|
+
|
|
79
|
+
### 4. One live runtime per browser flow
|
|
80
|
+
For E2E, there should be one real app instance and one isolated test database. No shadow app instances fighting the same datastore.
|
|
81
|
+
|
|
82
|
+
### 5. Disposable databases by default
|
|
83
|
+
For serious end-to-end or endpoint testing, the default should be a temporary SQLite database per run or per worker, stored under `/tmp`, not `sails-disk`.
|
|
84
|
+
|
|
85
|
+
### 6. Realistic over synthetic
|
|
86
|
+
The goal is not mocking everything. The goal is real flows with as little fake plumbing as possible.
|
|
87
|
+
|
|
88
|
+
### 7. Minimal magic
|
|
89
|
+
The best APIs should feel obvious. The framework should save time, not hide too much.
|
|
90
|
+
|
|
91
|
+
### 8. Great failure output
|
|
92
|
+
When a test fails, the developer should know:
|
|
93
|
+
- what world was created
|
|
94
|
+
- what request or browser step failed
|
|
95
|
+
- what the relevant app state was
|
|
96
|
+
|
|
97
|
+
## What Sounding should cover
|
|
98
|
+
|
|
99
|
+
### Unit / helper tests
|
|
100
|
+
- helpers
|
|
101
|
+
- pure business logic
|
|
102
|
+
- model-adjacent logic
|
|
103
|
+
- policies when run in isolation
|
|
104
|
+
|
|
105
|
+
### Endpoint / action tests
|
|
106
|
+
- guest vs authenticated access
|
|
107
|
+
- redirects
|
|
108
|
+
- JSON and HTML responses
|
|
109
|
+
- action inputs/exits
|
|
110
|
+
- policy interaction
|
|
111
|
+
|
|
112
|
+
### Inertia integration tests
|
|
113
|
+
- component name assertions
|
|
114
|
+
- prop assertions
|
|
115
|
+
- partial reload behavior
|
|
116
|
+
- validation and redirect behavior
|
|
117
|
+
|
|
118
|
+
### Browser / E2E tests
|
|
119
|
+
- sign in flows
|
|
120
|
+
- onboarding
|
|
121
|
+
- editor flows
|
|
122
|
+
- gated-content flows
|
|
123
|
+
- checkout and subscription handoff
|
|
124
|
+
- mobile navigation
|
|
125
|
+
|
|
126
|
+
### Mail tests
|
|
127
|
+
- magic link emails
|
|
128
|
+
- password reset emails
|
|
129
|
+
- invite emails
|
|
130
|
+
- webhook-triggered notifications
|
|
131
|
+
|
|
132
|
+
### Future layers
|
|
133
|
+
- sockets
|
|
134
|
+
- quest jobs
|
|
135
|
+
- webhook simulation
|
|
136
|
+
- uploads
|
|
137
|
+
- passkey/WebAuthn flows
|
|
138
|
+
|
|
139
|
+
## The core mental model
|
|
140
|
+
|
|
141
|
+
Sounding should feel like this:
|
|
142
|
+
|
|
143
|
+
- **App**: a booted Sails application under test
|
|
144
|
+
- **World**: a realistic set of data created for a test
|
|
145
|
+
- **Actor**: a user role in that world
|
|
146
|
+
- **Trial**: the test itself
|
|
147
|
+
- **Mailbox**: captured outbound mail for assertions
|
|
148
|
+
- **Browser**: Playwright page/context helpers
|
|
149
|
+
|
|
150
|
+
The tests should read like behavior, not setup plumbing.
|
|
151
|
+
|
|
152
|
+
## What Captain Vane should become
|
|
153
|
+
|
|
154
|
+
Captain Vane should not disappear.
|
|
155
|
+
|
|
156
|
+
Captain Vane should become the data and scenario engine that powers Sounding.
|
|
157
|
+
|
|
158
|
+
### Captain Vane should own
|
|
159
|
+
- factories
|
|
160
|
+
- traits / states
|
|
161
|
+
- sequences
|
|
162
|
+
- deterministic seeds
|
|
163
|
+
- build vs create APIs
|
|
164
|
+
- relationship graphs
|
|
165
|
+
- scenarios that return readable world objects
|
|
166
|
+
|
|
167
|
+
### Captain Vane v2 should support
|
|
168
|
+
- `tests/factories`
|
|
169
|
+
- `tests/scenarios`
|
|
170
|
+
- `build()`
|
|
171
|
+
- `buildMany()`
|
|
172
|
+
- `create()`
|
|
173
|
+
- `createMany()`
|
|
174
|
+
- `state()` / `trait()`
|
|
175
|
+
- `seed()`
|
|
176
|
+
- `afterBuild()` / `afterCreate()`
|
|
177
|
+
|
|
178
|
+
### Captain Vane should not own
|
|
179
|
+
- Sails app boot lifecycle
|
|
180
|
+
- request clients
|
|
181
|
+
- Playwright lifecycle
|
|
182
|
+
- mail capture runtime
|
|
183
|
+
- worker/database orchestration
|
|
184
|
+
|
|
185
|
+
That is Sounding’s job.
|
|
186
|
+
|
|
187
|
+
## What Sounding should own
|
|
188
|
+
|
|
189
|
+
### App lifecycle
|
|
190
|
+
- boot Sails once for the test mode being used
|
|
191
|
+
- manage ports and process lifecycle
|
|
192
|
+
- expose helpers for in-process and browser-driven testing
|
|
193
|
+
|
|
194
|
+
### Database lifecycle
|
|
195
|
+
- create one isolated SQLite database per run or per worker by default
|
|
196
|
+
- configure Sails to use it automatically in test mode
|
|
197
|
+
- tear it down cleanly
|
|
198
|
+
- support Postgres later for heavier projects
|
|
199
|
+
|
|
200
|
+
### Runtime adapters
|
|
201
|
+
- request client for endpoint/action tests
|
|
202
|
+
- Inertia assertion helpers
|
|
203
|
+
- Playwright integration for browser tests
|
|
204
|
+
- mailbox capture
|
|
205
|
+
- auth/session helpers
|
|
206
|
+
- socket client helpers later
|
|
207
|
+
- job helpers later
|
|
208
|
+
|
|
209
|
+
### Ergonomic API surface
|
|
210
|
+
The top-level API should make Sails concepts first-class without inventing a giant DSL.
|
|
211
|
+
|
|
212
|
+
## Proposed API direction
|
|
213
|
+
|
|
214
|
+
### Helper test
|
|
215
|
+
```js
|
|
216
|
+
import { test } from 'drydock'
|
|
217
|
+
|
|
218
|
+
test.helper('signupWithTeam creates a team and membership', async ({ helper, expect }) => {
|
|
219
|
+
const result = await helper('user.signupWithTeam', {
|
|
220
|
+
fullName: 'Kelvin O',
|
|
221
|
+
email: 'kelvin@example.com',
|
|
222
|
+
tosAcceptedByIp: '127.0.0.1',
|
|
223
|
+
})
|
|
224
|
+
|
|
225
|
+
expect(result.user.email).toBe('kelvin@example.com')
|
|
226
|
+
})
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
### Endpoint test
|
|
230
|
+
```js
|
|
231
|
+
import { test } from 'drydock'
|
|
232
|
+
|
|
233
|
+
test.endpoint('guest is redirected from dashboard', async ({ request, expect }) => {
|
|
234
|
+
const response = await request.get('/dashboard')
|
|
235
|
+
expect(response).toRedirectTo('/login')
|
|
236
|
+
})
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
### Inertia test
|
|
240
|
+
```js
|
|
241
|
+
import { test } from 'drydock'
|
|
242
|
+
|
|
243
|
+
test.inertia('pricing page returns the expected component and props', async ({ visit, expect }) => {
|
|
244
|
+
const page = await visit('/pricing')
|
|
245
|
+
expect(page).toBeInertiaPage('billing/pricing')
|
|
246
|
+
})
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
### Browser test
|
|
250
|
+
```js
|
|
251
|
+
import { test } from 'drydock'
|
|
252
|
+
|
|
253
|
+
test.browser('subscriber can read a members-only issue', async ({ page, world, login, expect }) => {
|
|
254
|
+
await world.use('issue-access')
|
|
255
|
+
await login.as('subscriber', page)
|
|
256
|
+
|
|
257
|
+
await page.goto(world.issues.gated.url)
|
|
258
|
+
|
|
259
|
+
await expect(page.getByText(world.issues.gated.fullText)).toBeVisible()
|
|
260
|
+
})
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
### Mail test
|
|
264
|
+
```js
|
|
265
|
+
import { test } from 'drydock'
|
|
266
|
+
|
|
267
|
+
test.mail('magic link sends a usable email', async ({ mailbox, expect }) => {
|
|
268
|
+
const email = await mailbox.latest()
|
|
269
|
+
expect(email.subject).toContain('Sign in')
|
|
270
|
+
expect(email.ctaUrl).toMatch(/magic-link/)
|
|
271
|
+
})
|
|
272
|
+
```
|
|
273
|
+
|
|
274
|
+
## Database strategy
|
|
275
|
+
|
|
276
|
+
This is the most important runtime choice.
|
|
277
|
+
|
|
278
|
+
### What we should avoid
|
|
279
|
+
- `sails-disk` for serious endpoint/E2E tests
|
|
280
|
+
- multi-process setups that touch the same datastore files
|
|
281
|
+
- test-only HTTP routes just for seeding state
|
|
282
|
+
|
|
283
|
+
### Recommended default
|
|
284
|
+
For `0.0.1`, Sounding should default to:
|
|
285
|
+
- **temporary SQLite**
|
|
286
|
+
- one database file per run or per worker
|
|
287
|
+
- stored under something like `/tmp/drydock/<run-id>/<worker-id>.sqlite`
|
|
288
|
+
|
|
289
|
+
Why SQLite first:
|
|
290
|
+
- TBJS already leans on SQLite as a sensible default
|
|
291
|
+
- no external services required
|
|
292
|
+
- much more realistic than `sails-disk`
|
|
293
|
+
- transactions and relational behavior are available
|
|
294
|
+
- dramatically better for E2E orchestration
|
|
295
|
+
|
|
296
|
+
Later, Sounding can support Postgres for teams that want parity with production.
|
|
297
|
+
|
|
298
|
+
## Test data location
|
|
299
|
+
|
|
300
|
+
A strong rule:
|
|
301
|
+
|
|
302
|
+
**All test data definitions live under `tests/`.**
|
|
303
|
+
|
|
304
|
+
Suggested structure:
|
|
305
|
+
|
|
306
|
+
```text
|
|
307
|
+
tests/
|
|
308
|
+
factories/
|
|
309
|
+
user.js
|
|
310
|
+
issue.js
|
|
311
|
+
subscription.js
|
|
312
|
+
scenarios/
|
|
313
|
+
issue-access.js
|
|
314
|
+
publisher-editor.js
|
|
315
|
+
reader-dashboard.js
|
|
316
|
+
e2e/
|
|
317
|
+
integration/
|
|
318
|
+
unit/
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
This keeps product code clean and keeps the testing world owned by the tests.
|
|
322
|
+
|
|
323
|
+
## What 0.0.1 should actually ship
|
|
324
|
+
|
|
325
|
+
The first release should be sharp, not huge.
|
|
326
|
+
|
|
327
|
+
### 0.0.1 goals
|
|
328
|
+
- native Node test runner integration
|
|
329
|
+
- Playwright browser integration
|
|
330
|
+
- Sails app boot manager
|
|
331
|
+
- temporary SQLite database lifecycle
|
|
332
|
+
- `test.helper()`
|
|
333
|
+
- `test.endpoint()`
|
|
334
|
+
- `test.browser()`
|
|
335
|
+
- basic auth helpers for common roles
|
|
336
|
+
- mailbox capture for log mailers / test mailers
|
|
337
|
+
- Captain Vane adapter for factories/scenarios
|
|
338
|
+
- one reference TBJS example app
|
|
339
|
+
|
|
340
|
+
### 0.0.1 non-goals
|
|
341
|
+
- custom assertion engine from scratch
|
|
342
|
+
- sockets and jobs on day one
|
|
343
|
+
- full payment/webhook simulation on day one
|
|
344
|
+
- WebAuthn passkey coverage on day one
|
|
345
|
+
- every possible Sails hook abstraction immediately
|
|
346
|
+
|
|
347
|
+
The 0.0.1 bar is: **credible, elegant, useful, and real.**
|
|
348
|
+
|
|
349
|
+
## What a good 0.0.1 feels like
|
|
350
|
+
|
|
351
|
+
A developer should be able to:
|
|
352
|
+
- install Sounding
|
|
353
|
+
- point it at a Sails app
|
|
354
|
+
- define factories and scenarios under `tests/`
|
|
355
|
+
- run helper tests, endpoint tests, and browser tests with one coherent mental model
|
|
356
|
+
- avoid touching product code to make tests possible
|
|
357
|
+
|
|
358
|
+
If we achieve that, the story is already strong.
|
|
359
|
+
|
|
360
|
+
## Roadmap after 0.0.1
|
|
361
|
+
|
|
362
|
+
### 0.1.x
|
|
363
|
+
- richer Captain Vane trait/state system
|
|
364
|
+
- `test.inertia()`
|
|
365
|
+
- storage-state auth helpers
|
|
366
|
+
- mobile/browser project presets
|
|
367
|
+
- better response and redirect assertions
|
|
368
|
+
|
|
369
|
+
### 0.2.x
|
|
370
|
+
- sockets
|
|
371
|
+
- quest jobs
|
|
372
|
+
- webhooks
|
|
373
|
+
- upload helpers
|
|
374
|
+
- payment flow harnesses
|
|
375
|
+
|
|
376
|
+
### 0.3.x
|
|
377
|
+
- passkey/WebAuthn helpers
|
|
378
|
+
- scenario debugger / world inspector
|
|
379
|
+
- better watch mode and failure reports
|
|
380
|
+
- richer CI output
|
|
381
|
+
|
|
382
|
+
## Risks and sharp edges
|
|
383
|
+
|
|
384
|
+
### 1. Too much magic
|
|
385
|
+
If Sounding tries to hide too much, it will become hard to trust.
|
|
386
|
+
|
|
387
|
+
### 2. App boot complexity
|
|
388
|
+
Sails boot and teardown need to be extremely predictable.
|
|
389
|
+
|
|
390
|
+
### 3. Database abstraction drift
|
|
391
|
+
We should not pretend SQLite and Postgres are identical. We should be honest about the tradeoffs.
|
|
392
|
+
|
|
393
|
+
### 4. Overcoupling to one stack shape
|
|
394
|
+
It should be opinionated for TBJS, but still flexible enough for real Sails apps.
|
|
395
|
+
|
|
396
|
+
### 5. Captain Vane boundary blur
|
|
397
|
+
If Sounding and Captain Vane overlap too much, both products get muddy.
|
|
398
|
+
|
|
399
|
+
## Success criteria
|
|
400
|
+
|
|
401
|
+
Sounding is working if:
|
|
402
|
+
- a new TBJS user can write meaningful tests quickly
|
|
403
|
+
- endpoint and E2E tests do not require test-only app routes
|
|
404
|
+
- browser tests can run against one isolated app + one isolated database cleanly
|
|
405
|
+
- data setup reads like business scenarios, not SQL dumps
|
|
406
|
+
- failures are easier to understand than today
|
|
407
|
+
|
|
408
|
+
## Short naming shortlist
|
|
409
|
+
|
|
410
|
+
### Chosen: Sounding
|
|
411
|
+
Best balance of elegance, meaning, and distinctiveness.
|
|
412
|
+
|
|
413
|
+
### Alternatives considered
|
|
414
|
+
- **Drydock** — strong and concrete, but more about environment than probing behavior
|
|
415
|
+
- **Seatrial** — very direct, but less elegant as a brand
|
|
416
|
+
- **Harbor** — strong, but feels more like infra than testing
|
|
417
|
+
- **Trials** — clear, but less distinctive
|
|
418
|
+
|
|
419
|
+
## Final take
|
|
420
|
+
|
|
421
|
+
Sounding should become the elegant testing story for TBJS.
|
|
422
|
+
|
|
423
|
+
Captain Vane should power the world-building underneath it.
|
|
424
|
+
|
|
425
|
+
If we get the boundaries right, we do not just end up with a nicer API.
|
|
426
|
+
We end up with a testing system that actually matches how Sails applications are built.
|
package/index.js
ADDED
package/package.json
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "sounding",
|
|
3
|
+
"version": "0.0.0",
|
|
4
|
+
"description": "Testing framework for Sails applications and The Boring JavaScript Stack.",
|
|
5
|
+
"main": "index.js",
|
|
6
|
+
"license": "MIT",
|
|
7
|
+
"keywords": [
|
|
8
|
+
"sails",
|
|
9
|
+
"testing",
|
|
10
|
+
"node:test",
|
|
11
|
+
"playwright",
|
|
12
|
+
"tbjs"
|
|
13
|
+
],
|
|
14
|
+
"files": [
|
|
15
|
+
"index.js",
|
|
16
|
+
"README.md",
|
|
17
|
+
"RESEARCH.md",
|
|
18
|
+
"LICENSE"
|
|
19
|
+
],
|
|
20
|
+
"publishConfig": {
|
|
21
|
+
"access": "public"
|
|
22
|
+
}
|
|
23
|
+
}
|