@ajnart/self-host-testkit 1.2.1 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,204 +1,235 @@
1
1
  # @ajnart/self-host-testkit
2
2
 
3
- Real, isolated Docker App fixtures for TypeScript and JavaScript integration
4
- tests. Start an App through self-host's existing Setup, call its API, and close
5
- the fixture. ESM only: use `import`, not CommonJS `require`. No runtime npm
6
- dependencies and no test-runner dependency.
3
+ Real App fixtures for integration tests, using self-host's existing headless
4
+ Setup. ESM only, Node.js 22+, no runtime npm dependencies or test-runner plugin.
7
5
 
8
- ## Before you start
6
+ This guide describes **1.3.0**: `withApp`, ordered Pairs, recovery paths, the
7
+ Sonarr witness and the JavaScript Action's automatic post cleanup are not in
8
+ 1.2.1. For 1.2.1, see its
9
+ [immutable guide](https://github.com/ajnart/self-host/blob/5bd44f7f4cd3c350d6cf449e958eae6c0df28a69/testkit/node/README.md).
9
10
 
10
- You need **Node.js 22+**, a running local **Docker Engine** and the **Docker
11
- Compose v2 plugin** (`docker compose version`). Linux and macOS x64/arm64 are
12
- supported; Docker must use a local Unix socket and enforce memory, CPU and PID
13
- limits. Allow up to 5 GiB RAM and five CPUs per fixture, plus image/storage space.
14
- First startup pulls images and can take several minutes.
11
+ ## Requirements and setup
15
12
 
16
- For a published release, install the SDK as a dev dependency. Version 1.2.1 must
17
- exist on npm and GitHub before using these commands; for an unpublished source
18
- checkout, use the source workflow linked below instead:
13
+ Run tests directly on a Linux/macOS x64/arm64 Docker host with Node.js 22+, local
14
+ Unix socket access and Docker Compose v2. Docker Desktop must be running. Docker
15
+ must enforce memory, CPU and PID caps: up to five containers per fixture, each
16
+ limited to 1 GiB, one CPU and 512 PIDs. Budget up to 5 GiB and five CPUs per fixture,
17
+ plus runner overhead, disk space and network access for images.
19
18
 
20
- ```bash
21
- npm install --save-dev @ajnart/self-host-testkit@1.2.1
22
- ```
23
-
24
- This guide describes version 1.2.1. The older v1.2.0 tarball does not contain the setup command,
25
- `TestkitError` export or safe startup reasons. For that release, extract
26
- `self-host-testkit` from its platform archive, verify `checksums.txt`, and set
27
- `SELF_HOST_TESTKIT_BINARY` to the helper's absolute path. For the improved source,
28
- follow the [checkout quickstart](../README.md#typescript-install).
19
+ GitHub-hosted Ubuntu is the simplest headless environment. Self-hosted runners
20
+ need the same limits and permissions. Containerized test runners, GitHub job
21
+ containers, Docker-in-Docker, remote/TCP Docker and native Windows are unsupported.
22
+ A mounted socket is insufficient: App loopback Addresses and bind-mount paths
23
+ belong to the daemon host, not the test container. See the [support matrix](../README.md#headless-and-container-requirements).
29
24
 
30
- Version 1.2.0 used the unpublished package name `@self-host/testkit`. When
31
- upgrading, change imports to `@ajnart/self-host-testkit`. The exact npm package
32
- is also available as `self-host-testkit-1.2.1.tgz` on the matching GitHub release.
25
+ **Source:** follow the [checkout build](../README.md#try-this-source-checkout),
26
+ which builds generated SDK exports with Bun and the helper with Go, then selects
27
+ that helper using `SELF_HOST_TESTKIT_BINARY`. Run the included demo from this
28
+ checkout so the package self-import resolves to its matching SDK.
33
29
 
34
- ## Install the matching helper
35
-
36
- After installing the SDK, run:
30
+ **Published release:** version 1.3.0 must exist on npm and GitHub before using
31
+ these commands. Install it and explicitly prepare its matching helper:
37
32
 
38
33
  ```bash
34
+ npm install --save-dev @ajnart/self-host-testkit@1.3.0
39
35
  npm exec -- self-host-testkit-setup
40
36
  ```
41
37
 
42
- This explicit setup downloads only the matching Linux/macOS helper, requires one
43
- SHA-256 checksum entry and verifies the archive before extraction. It caches the
44
- helper privately under `$XDG_CACHE_HOME/self-host-testkit/`, or
45
- `~/.cache/self-host-testkit/`. It does not run Docker, modify PATH, require Go, or
46
- download anything during package installation or import. Repeating setup reuses
47
- the cached helper. `createSelfHost()` discovers it automatically.
48
-
49
- To use a source-built or offline helper instead:
50
-
51
- ```bash
52
- export SELF_HOST_TESTKIT_BINARY=/absolute/path/to/self-host-testkit
53
- ```
38
+ Released setup requires network access and `tar`, verifies the archive's SHA-256
39
+ checksum and reuses a private versioned helper cache. It needs **no Go** and does
40
+ not start Docker or modify PATH. Neither npm installation nor import downloads
41
+ anything.
54
42
 
55
- Resolution order: `binary` option, `SELF_HOST_TESTKIT_BINARY`, the versioned
56
- private cache, then `self-host-testkit` on PATH. Keep SDK, helper and Action on
57
- the same release or immutable source commit. Never substitute the normal
58
- `self-host` CLI: the helper is a separate executable.
43
+ Helper resolution: `binary` option, `SELF_HOST_TESTKIT_BINARY`, private versioned
44
+ cache, then `self-host-testkit` on PATH. Keep SDK and helper on the same release
45
+ or source commit; matching version strings alone do not prove source identity.
46
+ The normal `self-host` CLI is not the testkit helper.
59
47
 
60
- ## A small demo
48
+ ## Replace a mock, not your adapter
61
49
 
62
- Save this as `demo.mjs` and run `node demo.mjs` after installing the SDK/helper:
50
+ Pass the real App's `url` and key to the adapter you already test. For example,
51
+ if your existing test helper is `testSonarrAdapter({ baseUrl, apiKey })`, replace
52
+ its mock configuration with:
63
53
 
64
54
  ```javascript
65
- import { createSelfHost } from "@ajnart/self-host-testkit";
55
+ import { withApp } from "@ajnart/self-host-testkit";
66
56
 
67
- const selfHost = createSelfHost();
68
- try {
69
- const sonarr = await selfHost.spinup("sonarr");
70
- const response = await fetch(`${sonarr.url}/api/v3/system/status`, {
71
- headers: { "X-Api-Key": await sonarr.getApiKey() },
72
- signal: AbortSignal.timeout(10_000),
57
+ await withApp("sonarr", async (sonarr) => {
58
+ await testSonarrAdapter({
59
+ baseUrl: sonarr.url,
60
+ apiKey: await sonarr.getApiKey(),
73
61
  });
74
- if (!response.ok) throw new Error("Sonarr rejected the request");
75
- const status = await response.json();
76
- if (status.appName !== "Sonarr") throw new Error("Unexpected App identity");
77
- console.log(`Connected to Sonarr ${status.version}`);
78
- } finally {
79
- await selfHost.close();
80
- }
62
+ });
81
63
  ```
82
64
 
83
- The package also includes `examples/sonarr.mjs`. No API key appears in its output.
84
- Cleanup uses ordinary `try/finally` or test-runner hooks, without requiring
85
- TypeScript disposable libraries or newer JavaScript syntax.
65
+ `testSonarrAdapter` denotes your own existing test code, not another package to
66
+ install. Keep its real assertions for meaningful data and rejected credentials;
67
+ HTTP 200 or a running container alone is not integration acceptance. Never print
68
+ API keys, credentials or raw response bodies in assertion failures.
86
69
 
87
- ## Vitest
70
+ For a standalone example without an adapter, run `examples/sonarr.mjs` after the
71
+ source build. It checks Sonarr's authenticated identity and prints no secrets.
88
72
 
89
- Use your project's existing Vitest dependency. Set hook timeouts explicitly;
90
- the default hook timeout is too short for image pulls and Setup.
73
+ ## Scoped fixtures and timeouts
91
74
 
92
- ```typescript
93
- import { afterAll, beforeAll, expect, test } from "vitest";
94
- import { createSelfHost, type App } from "@ajnart/self-host-testkit";
95
-
96
- const selfHost = createSelfHost();
97
- let sonarr: App;
75
+ `withApp(id, callback, options?)` returns the awaited callback result and always
76
+ attempts cleanup after startup or callback failure. The callback receives the
77
+ existing `App` API, including `url` and `getApiKey()`; it is runner-independent.
78
+ Options combine `binary`, `timeoutMs`, `manifest`, an optional `image` override
79
+ and the same optional `onCleanupError` as `withPair`.
98
80
 
99
- beforeAll(async () => {
100
- sonarr = await selfHost.spinup("sonarr");
101
- }, 780_000);
81
+ Use your runner's existing async-test API and increase its timeout for image
82
+ pulls, Setup, assertions **and cleanup**. For Node's built-in runner:
102
83
 
103
- afterAll(async () => {
104
- await selfHost.close();
105
- }, 160_000);
106
-
107
- test("Sonarr returns its authenticated identity", async () => {
108
- const response = await fetch(`${sonarr.url}/api/v3/system/status`, {
109
- headers: { "X-Api-Key": await sonarr.getApiKey() },
110
- signal: AbortSignal.timeout(10_000),
84
+ ```javascript
85
+ import assert from "node:assert/strict";
86
+ import { test } from "node:test";
87
+ import { withApp } from "@ajnart/self-host-testkit";
88
+
89
+ test("Sonarr authenticates", { timeout: 900_000 }, async () => {
90
+ await withApp("sonarr", async (sonarr) => {
91
+ const response = await fetch(`${sonarr.url}/api/v3/system/status`, {
92
+ headers: { "X-Api-Key": await sonarr.getApiKey() },
93
+ signal: AbortSignal.timeout(10_000),
94
+ });
95
+ assert.equal(response.ok, true);
96
+ const status = await response.json();
97
+ assert.equal(status.appName, "Sonarr");
111
98
  });
112
- expect(response.ok).toBe(true);
113
- const status = await response.json();
114
- expect(status.appName).toBe("Sonarr");
115
- expect(status.version).toMatch(/^\d+\./);
116
99
  });
117
100
  ```
118
101
 
119
- This verifies identity, not complete integration acceptance. Your adapter tests
120
- should also check meaningful data and invalid-key behavior. Do not print raw
121
- response bodies, credentials or API keys in failing assertions.
102
+ `timeoutMs` defaults to 720,000 (12 minutes), accepts 1–840,000 (14 minutes) and
103
+ bounds the **App lifetime**, not just startup. The helper's hard maximum is 15
104
+ minutes, with time reserved for cleanup. Cold pulls can take minutes; avoid
105
+ sharing one fixture across a suite longer than its lifetime. Bound concurrency
106
+ and allow cleanup time in your test hooks and CI job timeout.
122
107
 
123
- ## API
108
+ When startup or your callback throws, `withApp` preserves that exact thrown value,
109
+ including primitives or `undefined`, even if cleanup also fails. If there is no
110
+ primary failure, cleanup failure is thrown. For recovery after a primary failure,
111
+ use the Action's shared manifest or select your own private manifest in advance.
124
112
 
125
- ```typescript
126
- const selfHost = createSelfHost({
127
- timeoutMs: 720_000,
128
- });
129
- const app = await selfHost.spinup("sonarr", {
130
- image: "lscr.io/linuxserver/sonarr:latest",
131
- });
132
- const apiKey = await app.getApiKey();
133
- const credentials = await app.credentials();
134
- const info = await app.info();
135
- await app.close();
136
- await selfHost.close();
137
- ```
113
+ ## Advanced: manage fixture lifetime yourself
138
114
 
139
- - `createSelfHost({ binary?, timeoutMs?, manifest? })`: creates a fixture manager
140
- without starting Docker. `timeoutMs` defaults to 720,000 and accepts 1–840,000.
141
- It bounds the App lifetime, not just startup; cleanup time is reserved within
142
- that budget. Do not share a fixture across a longer-running suite.
143
- - `spinup(id, { image? })`: awaits Setup and returns an App with its loopback
144
- `url`. Each call gets a separate helper and Project; image overrides apply to
145
- the selected App, not its dependency images.
146
- - `getApiKey(name?)`: explicitly reads a private key; the default name is
147
- `api_key`. Use the App's declared key name when it differs.
148
- - `credentials()`: `username`, `password`, `email`, `apiKey`. Fields are
149
- non-enumerable and JSON/inspection redact the object, but reading a field
150
- returns the actual secret. Never log it or upload it.
151
- - `info()`: public `id`, `url`, `image`, `imageId`, `repoDigests`. Record image
152
- digests when reproducibility matters; `latest` is deliberately mutable.
153
- - `app.close()` closes one fixture; `selfHost.close()` closes all its fixtures.
154
- Both are idempotent. Successful cleanup removes owned containers, networks,
155
- anonymous container volumes and private storage. Named volumes are rejected.
156
- - `manifest`: optional absolute recovery filename in an existing owned mode-0700
157
- directory. Otherwise the SDK creates a private temporary parent. The helper
158
- records ownership before startup. Keep recovery manifests and Project data
159
- private; never upload them as public CI artifacts.
160
-
161
- ## Errors and recovery
115
+ Use `createSelfHost` when existing hooks need to own several App lifetimes:
162
116
 
163
- ```typescript
164
- import { TestkitError } from "@ajnart/self-host-testkit";
117
+ ```javascript
118
+ import { createSelfHost } from "@ajnart/self-host-testkit";
165
119
 
120
+ const selfHost = createSelfHost({ timeoutMs: 720_000 });
166
121
  try {
167
- await selfHost.spinup("sonarr");
168
- } catch (error: unknown) {
169
- if (error instanceof TestkitError && error.code === "BINARY") {
170
- throw new Error(
171
- "Install the matching testkit helper before running this suite",
172
- );
173
- }
174
- throw error;
122
+ const sonarr = await selfHost.spinup("sonarr");
123
+ const apiKey = await sonarr.getApiKey();
124
+ await testSonarrAdapter({ baseUrl: sonarr.url, apiKey });
125
+ } finally {
126
+ await selfHost.close();
175
127
  }
176
128
  ```
177
129
 
178
- Errors expose a sanitized `code`: `BINARY`, `PROTOCOL`, `STATE`, `START`,
179
- `CREDENTIALS`, `INFO`, `CLEANUP`, `TIMEOUT` or `TRANSPORT`. They never include raw
180
- provider output, helper stderr or executable paths. A `START` error may also
181
- have an allowlisted `reason` and a corresponding safe hint: `APP`, `OPTIONS`,
182
- `DOCKER`, `CAPABILITIES`, `PORT`, `OWNERSHIP`, `LIMITS`, `LOCK`, `UNSUPPORTED`,
183
- `STORAGE`, `CONTAINER` or `SETUP`. These distinguish unsupported fixtures from
184
- failed Setup without forwarding provider messages.
185
-
186
- - **BINARY:** run the explicit setup command or set the helper path.
187
- - **START:** check local Docker/Compose, supported App prerequisites, resource
188
- limits and image availability. Apps needing a Manual step, hardware, external
189
- account, host networking, socket mount or named volume fail rather than skip.
190
- - **TIMEOUT:** allow cold-pull time within the bounded lifetime, lower concurrent
191
- fixture count, or use a smaller supported App.
192
- - **CLEANUP:** retain the private manifest and retry scoped recovery after helper
193
- exit. For CI, use the always-run step below. For local recovery, pass the
194
- matching helper `--cleanup-manifest /absolute/private/manifest.jsonl`.
195
-
196
- If you supply a manifest, retain its path outside the test for recovery. SIGKILL,
197
- runner loss and daemon outages can prevent cleanup; disposable runners are best.
198
- Do not run Docker prune or broad container/volume removal to recover fixtures.
130
+ - `createSelfHost({ binary?, timeoutMs?, manifest? })` creates a manager without
131
+ starting Docker. `spinup(id, { image? })` awaits Setup. Every App has its own
132
+ temporary Project and random loopback-only Address.
133
+ - `getApiKey(name?)` explicitly reads a key; the default name is `api_key`.
134
+ - `credentials()` returns private `username`, `password`, `email`, `apiKey` fields.
135
+ JSON/inspection redact the object, but reading a field returns the real secret.
136
+ - `info()` returns public `id`, `url`, `image`, `imageId`, `repoDigests`. Record
137
+ digests for reproducibility; `latest` is mutable. An `image` override changes
138
+ the selected App's image, not dependency images.
139
+ - `app.close()` closes one fixture; `selfHost.close()` closes its fixtures.
140
+ Concurrent calls share one attempt; successful cleanup is idempotent. A failed
141
+ close can be retried, but does not reopen the manager for new starts.
142
+ - Successful cleanup removes owned containers, networks, anonymous container
143
+ volumes and private storage. Named volumes are rejected.
144
+
145
+ The advanced example uses ordinary `finally` semantics: a cleanup error can
146
+ replace an earlier error. Prefer `withApp` when exact primary-error preservation
147
+ matters.
148
+
149
+ ## Ordered pair API
150
+
151
+ Pair calls require the helper matching this SDK; 1.2.1 does not contain them. Existing `spinup`/`withApp` calls are unchanged.
152
+
153
+ ```javascript
154
+ import { withPair } from "@ajnart/self-host-testkit";
155
+
156
+ await withPair(
157
+ "homarr",
158
+ "sonarr",
159
+ async (pair) => {
160
+ const result = await pair.inspect();
161
+ if (result.outcome !== "complete") throw new Error("Pair needs attention");
162
+ const key = await pair.target.getApiKey();
163
+ await testSonarrAdapter({ baseUrl: pair.target.url, apiKey: key });
164
+ },
165
+ { order: "consumer-first" },
166
+ );
167
+ ```
168
+
169
+ `createSelfHost().spinupPair(consumer, target, { order, images? })` returns the
170
+ same Pair for explicit lifetime management. `order` must be `consumer-first` or
171
+ `target-first`; both are sequential production installs into one fresh Project.
172
+ `images` may override only selected main App images, keyed by App ID.
173
+ Pairs need web Apps whose required Inputs have defaults or are host ports, and
174
+ one declared consumer Link. Unsupported Compose/account/hardware/private-input boundaries
175
+ remain explicit errors or Manual outcomes, not automatic skips.
176
+
177
+ - `consumer`/`target` expose `url`, `getApiKey`, `credentials`, `info` and private
178
+ recovery access, but **no close method**. Only Pair/manager close owns cleanup.
179
+ - `initial` records first-install absence. `inspect()` returns selected public
180
+ observations and `complete`, `manual`, `prerequisite` or `failed`. Missing,
181
+ skipped or unready Links are never complete. Completion is the production
182
+ recorded outcome, not a claim of rendered acceptance for every peer.
183
+ Selected Info and inspection results must retain the selected IDs and exact
184
+ loopback Addresses established at startup, including their frozen ports/paths.
185
+ Inspections must also retain the startup-declared Link ID; response steps cannot
186
+ nominate a replacement Link to satisfy completion.
187
+ Step IDs must be unique within each App. Summaries must agree with observed
188
+ failures, Manual Setup and prerequisite facts; contradictory results reject
189
+ through the same awaited whole-owner cleanup boundary.
190
+ - `setup(app?)` retries production Setup for one/both selected Apps;
191
+ `remove(app)` is non-purge and retains data; `install(app)` reinstalls only a
192
+ selected admitted App with its original ports/options. No action renews the
193
+ lifetime. An absent App cannot expose stale Info/credentials.
194
+ - One owner/admission budget covers both Apps plus every backing service:
195
+ five containers, 5 GiB, five CPUs; first-creation 1 GiB/one CPU/512-PID caps.
196
+ Local Access/private binds/volume/socket restrictions remain; no override or
197
+ arbitrary command/secret-write interface is exposed.
198
+ - An infrastructure action/inspection or selected-view key/credential/Info failure
199
+ immediately attempts independent-context whole-Pair cleanup and disables further actions.
200
+ Expected Manual/incomplete Setup remains inspectable. `TestkitError` preserves
201
+ an `OPERATION` reason (such as `LIMITS`), or `TIMEOUT`/`CANCELED`, and an additive
202
+ `cleanupFailed` flag. A failed cleanup retains private recovery ownership and
203
+ can be retried with close/manifest
204
+ recovery; it does not replace the original action cause.
205
+ - Selected-view key/credential/Info reads share the Pair's original work deadline.
206
+ Ordinary missing keys, absent Apps and Manual credentials remain nonfatal;
207
+ cancellation, deadline and storage failures disable the whole owner. Malformed
208
+ action/inspection or selected-view results likewise disable the owner and await
209
+ owned cleanup before rejecting with `PROTOCOL`, retaining any secondary `cleanupFailed` flag.
210
+ Retrying cleanup never re-enables Pair or selected-view actions.
211
+ - Beginning explicit Pair close also permanently disables live actions and reads,
212
+ even if cleanup fails. Close remains retryable within the original lifetime.
213
+ - `withPair` preserves exact primary failures, including `undefined`, over
214
+ cleanup failure. Cleanup-only failures throw. Optional `onCleanupError` receives
215
+ a sanitized cleanup error when there is already a primary failure; failure of
216
+ that notification does not replace the original. Recovery remains manifest-
217
+ scoped and uses the existing Action post hook.
218
+
219
+ `examples/homarr-sonarr.mjs` demonstrates one order with authenticated Sonarr
220
+ identity and production-Setup-driven advancing Stats. Run a second fresh scenario
221
+ with `target-first` to test the other order. It never configures Homarr in Node,
222
+ does not require a downloader or Kuma, and must wait for integrated Sonarr leaf
223
+ registration plus a centrally approved runtime slot. Empty/native/error and
224
+ rendered/lifecycle acceptance remain separate evidence; no runtime is implied by
225
+ building or importing the SDK.
199
226
 
200
227
  ## GitHub Actions
201
228
 
229
+ **Source workflow:** the local Action builds its helper and uses the generated
230
+ SDK from the same checkout. The demo starts a real Sonarr App; run it only when
231
+ you intend to use Docker.
232
+
202
233
  ```yaml
203
234
  jobs:
204
235
  integration:
@@ -206,82 +237,135 @@ jobs:
206
237
  timeout-minutes: 20
207
238
  steps:
208
239
  - uses: actions/checkout@v7
209
- - uses: actions/setup-node@v6
240
+ - uses: ./testkit
210
241
  with:
211
- node-version: 22
212
- - run: npm ci
213
- - id: testkit
214
- uses: ajnart/self-host/testkit@v1.2.1
215
- - run: npx vitest run path/to/live-integration.test.ts
216
- - name: Clean up owned fixtures
217
- if: always() && steps.testkit.outcome == 'success'
218
- run: '"$SELF_HOST_TESTKIT_BINARY" --cleanup-manifest "$SELF_HOST_TESTKIT_MANIFEST"'
242
+ source: true
243
+ - run: node "$SELF_HOST_TESTKIT_SDK/examples/sonarr.mjs"
219
244
  ```
220
245
 
221
- Pin the Action to a full commit SHA and install the SDK from the same source or
222
- release. The Action requires Bash, Python 3 and tar; it verifies and extracts a
223
- patched Go toolchain privately for its own build without changing the caller's
224
- PATH or `GOTOOLCHAIN`. It does not require a preinstalled Go launcher and works
225
- when the caller sets `GOSUMDB=off`. Standard GitHub-hosted Ubuntu runners supply
226
- these prerequisites.
227
- The original v1.2.0 Action does not yet preserve the caller's Go environment.
228
-
229
- Outputs: `binary-path`, `sdk-path`, `manifest-path`. Job environment:
230
- `SELF_HOST_TESTKIT_BINARY`, `SELF_HOST_TESTKIT_SDK`, `SELF_HOST_TESTKIT_MANIFEST`.
231
- The Action does not install the npm dependency or start Apps. Its committed SDK
232
- exports and source-built helper match its ref. Set bounded matrix concurrency;
233
- each simultaneous fixture adds its own resource budget. Always-run cleanup is
234
- mandatory because composite Actions have no finally hook.
235
-
236
- ## Go
246
+ The **JavaScript Action** registers automatic post cleanup, including after a
247
+ test step fails. Keep ordinary fixture cleanup too; post cleanup is a safety net,
248
+ not a guarantee after runner loss. No separate cleanup step is needed for this
249
+ new Action. The published 1.2.1 composite Action does not have this post hook.
250
+
251
+ In default release mode, install the consumer SDK with `npm ci` before the Action;
252
+ it resolves that installed SDK and downloads its exact matching checksummed
253
+ helper, without Go. Pin a reviewed Action commit. For monorepos, set `working-directory` to the
254
+ consumer directory and run installation/tests there. The Action does not install
255
+ npm dependencies or start Apps. Outputs remain `binary-path`, `sdk-path`, `manifest-path`; environment
256
+ variables remain `SELF_HOST_TESTKIT_BINARY`, `SELF_HOST_TESTKIT_SDK`,
257
+ `SELF_HOST_TESTKIT_MANIFEST`.
258
+
259
+ With `source: true`, the Action privately obtains Go and builds its helper,
260
+ using that checkout's committed generated SDK, with no release fallback. Source
261
+ mode also requires Python 3 and `tar`. Tests must import **that** SDK, not an older
262
+ installed npm package; for the included demo, run
263
+ `node "$SELF_HOST_TESTKIT_SDK/examples/sonarr.mjs"`. The consumer's working
264
+ directory does not change which source checkout supplies the helper and SDK.
265
+
266
+ ## Errors and private recovery
267
+
268
+ `TestkitError` has sanitized `code`, optional startup/operation `reason` with safe
269
+ hints, and a `cleanupFailed` flag for secondary cleanup trouble.
270
+ Codes: `BINARY`, `PROTOCOL`, `STATE`, `START`, `CREDENTIALS`, `INFO`, `CLEANUP`,
271
+ `TIMEOUT`, `CANCELED`, `TRANSPORT`, `OPERATION`. Raw provider output and helper stderr
272
+ are not forwarded.
273
+
274
+ - **BINARY:** explicitly prepare a matching helper or set its absolute path.
275
+ - **START:** check Docker/Compose, resource limits and image availability. Apps
276
+ needing Manual steps, external accounts, hardware, host networking, socket
277
+ mounts or named volumes fail explicitly, rather than silently skipping Setup.
278
+ - **TIMEOUT:** budget cold pulls within the lifetime or reduce concurrency.
279
+ - **CLEANUP:** retain private recovery storage and retry once Docker is available.
280
+
281
+ `selfHost.recoveryPath` and `app.recoveryPath` explicitly expose the private
282
+ manifest filename; `TestkitError.recoveryPath` may expose it when available.
283
+ These properties are non-enumerable and must not be logged or uploaded. To choose a
284
+ manifest yourself, provide an absolute filename inside an existing owned
285
+ mode-0700 directory. The helper records ownership before starting containers.
286
+
287
+ Retry `close()` on a retained handle, or after helper exit use the matching helper:
237
288
 
238
289
  ```bash
239
- go get github.com/ajnart/self-host@v1.2.1
240
- ```
241
-
242
- ```go
243
- peer, err := testkit.Start(ctx, testkit.Options{App: "sonarr"})
244
- if err != nil {
245
- return err
246
- }
247
- defer func() {
248
- cleanup, cancel := context.WithTimeout(context.Background(), time.Minute)
249
- defer cancel()
250
- if err := peer.Close(cleanup); err != nil {
251
- log.Print("fixture cleanup failed; retain its private recovery path")
252
- }
253
- }()
254
- key, err := peer.GetAPIKey(ctx, "")
290
+ "$SELF_HOST_TESTKIT_BINARY" --cleanup-manifest "$SELF_HOST_TESTKIT_MANIFEST"
255
291
  ```
256
292
 
257
- Import `github.com/ajnart/self-host/testkit`; it is part of the root module, not a
258
- separate Go module. `RecoveryPath()` is private operational metadata. Go startup
259
- errors preserve known safe testkit causes and context cancellation while hiding
260
- raw provider errors. The current source removes storage on successful `Close`;
261
- v1.2.0 retains it until a dead-process `Cleanup` pass.
293
+ The Action sets both variables; outside CI, set the manifest variable privately
294
+ from your chosen manifest or recovery getter. Failed cleanup retains the marker
295
+ for retry. Do not upload manifests, Project files or credentials as artifacts;
296
+ do not use Docker prune or broad container/volume deletion for recovery. SIGKILL,
297
+ runner loss and daemon outages can still prevent cleanup; prefer disposable runners.
298
+
299
+ ## Sonarr data-preservation witness
300
+
301
+ This capability is new in 1.3.0. Keep the SDK and helper on the same release or
302
+ reviewed checkout. For a fresh ordered
303
+ `spinupPair("homarr", "sonarr", options)` or `withPair`, explicitly set
304
+ `sonarrFilterWitness: true` and provide `images.sonarr` as a reviewed immutable
305
+ `repository@sha256:<64 lowercase hex digits>` reference. Omission preserves the
306
+ legacy startup sequence and grants no witness permission. The two installation
307
+ orders use the same methods in separate fresh Pairs.
308
+
309
+ Before opted-in Pair startup, the SDK requires protocol-v1 static capability
310
+ `sonarrFilterWitness: 1` from its helper and then the matching startup echo.
311
+ An old or incompatible helper fails safely before `spinupPair`; there is no
312
+ fallback to Node HTTP. Capability support is not image/source approval: review
313
+ the resolved Sonarr release and image separately before any approved native run.
314
+ The current witness contract accepts authenticated stable Sonarr v4 identity,
315
+ requires a complete/running owned Pair and refuses an existing saved-filter
316
+ collection. Default/standalone APIs and resource/lifetime limits are unchanged.
317
+
318
+ An opted-in Pair exposes exactly two zero-argument methods:
262
319
 
263
- ## Developing this package
264
-
265
- From the repository root:
266
-
267
- ```bash
268
- bun install --frozen-lockfile
269
- bun run --cwd testkit/node build
270
- bun run --cwd testkit/node check
271
- bun run --cwd testkit/node lint
272
- bun test testkit/node/tests
273
- npm pack ./testkit/node --ignore-scripts --dry-run
320
+ ```typescript
321
+ const original = await pair.createSonarrFilterWitness();
322
+ const before = await pair.readSonarrFilterWitness();
323
+ if (
324
+ before.kind !== original.kind ||
325
+ before.objectId !== original.objectId ||
326
+ before.contentSha256 !== original.contentSha256
327
+ ) {
328
+ throw new Error("The original Sonarr witness changed");
329
+ }
274
330
  ```
275
331
 
276
- `src/*.mts` is the source of truth. TypeScript generates the committed `.mjs`
277
- exports and `.d.mts` declarations; CI verifies they remain current. No hand-kept
278
- parallel declarations. The tarball includes only runtime modules, public types,
279
- license, README and the demo; it does not ship development dependencies or tests.
280
- `SELF_HOST_RELEASE_VERSION=X.Y.Z node testkit/node/pack.mjs` validates the package
281
- allowlist and adds its checksum to release artifacts. Build first. Version bumps
282
- and npm trusted publishing require a separate release decision; never overwrite
283
- v1.2.0 assets with a modified package.
284
-
285
- Backup/Restore caching and live Docker acceptance are deferred, not implied by
286
- passing SDK transport tests. The downstream Homarr weekly checklist is tracked
287
- in [draft PR #7446](https://github.com/homarr-labs/homarr/pull/7446).
332
+ Each returns an immutable `SonarrFilterWitness` with `kind` equal to
333
+ `"sonarr-custom-filter-v1"`, the original positive `objectId`, and a 64-character
334
+ lowercase `contentSha256`. The Go helper generates the opaque synthetic label;
335
+ Node never supplies or receives its content, resource selector, request body or
336
+ credential. The saved object is an unselected series filter with no predicates,
337
+ not a series, profile, tag, provider or job. Creation uses the official Sonarr v3
338
+ custom-filter API once and confirms the new identity by readback. Tags would not
339
+ be durable because Sonarr housekeeping deletes unused tags.
340
+
341
+ Create once after the production Pair Link is complete. Read before removal and
342
+ after each of two ordinary `pair.remove("sonarr")` / `pair.install("sonarr")`
343
+ cycles; compare all three fields with the original receipt. Do not call Create
344
+ inside a cycle. Repeated creation, read-before-create, ambiguous POST results,
345
+ changed/missing data, unsupported schemas and image drift fail rather than adopt
346
+ or recreate a witness. Owner work deadlines, mutex, credential owner and scoped
347
+ failure/cleanup behavior are reused. A timeout or failed write cannot be retried
348
+ within that Pair. Receipt IDs/hashes may be recorded; never export credentials,
349
+ nonce-bearing bodies, App logs or private fixture files.
350
+
351
+ An invalid witness receipt or startup capability echo disables that helper's
352
+ whole owner and awaits scoped cleanup before rejecting with `PROTOCOL`.
353
+ Secondary cleanup failure is retained as `cleanupFailed: true`; it does not
354
+ replace the protocol failure or expose raw causes. Subsequent actions are
355
+ rejected, while explicit close/manifest recovery may retry failed cleanup.
356
+
357
+ Both cycles must retain data through non-purge target removal. Final Pair/manager
358
+ close still cleans the complete owned disposable fixture; do not add an API
359
+ delete, broad Docker teardown or attach/reseed recovery. A helper lost before
360
+ verification can be recovered for cleanup, not resumed as a successful proof.
361
+ Fake transport tests do not establish native preservation, populated-media
362
+ semantics, rendering, cross-kind survivors or stored-key corruption/recovery.
363
+
364
+ ## Source maintenance
365
+
366
+ `src/*.mts` owns the generated `.mjs` and `.d.mts` files. From the repository root,
367
+ run `bun run --cwd testkit/node build`, then `check` and `lint` with the same
368
+ command prefix. Never mix source SDK output with an unrelated released helper.
369
+ For Go callers, see the [Go API](https://pkg.go.dev/github.com/ajnart/self-host/testkit).
370
+ Live App acceptance and Backup/Restore caching remain deferred; transport tests
371
+ do not establish either.