@botiverse/k-carrier 0.1.7 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/NOTICE +5 -2
- package/README.md +72 -26
- package/core/src/artifact/download.ts +23 -1
- package/core/src/artifact/gzip.ts +26 -0
- package/core/src/artifact/source.ts +2 -0
- package/core/src/{createUpgrader.ts → createRunner.ts} +15 -14
- package/core/src/index.ts +12 -3
- package/core/src/launcher/launch.ts +15 -0
- package/core/src/launcher/supervise.ts +170 -0
- package/core/src/lifecycle/commandHost.ts +111 -0
- package/core/src/lifecycle/hostAdapter.ts +28 -16
- package/core/src/operation.ts +47 -25
- package/core/src/operationLifecycle.ts +2 -7
- package/core/src/platform/ops.ts +7 -0
- package/core/src/platform/posix.ts +26 -6
- package/core/src/platform/windows.ts +8 -2
- package/core/src/protocol/runner.ts +81 -0
- package/core/src/provenance/journal.ts +1 -1
- package/core/src/quarantine.ts +167 -0
- package/core/src/runner/cli.ts +27 -0
- package/core/src/runner/execute.ts +68 -0
- package/core/src/txn/engine.ts +41 -85
- package/core/src/txn/fileEffects.ts +15 -1
- package/core/src/txn/hostCallBudget.ts +4 -1
- package/core/src/txn/hostCallUncertain.ts +2 -0
- package/core/src/txn/lock.ts +81 -37
- package/core/src/txn/state.ts +1 -1
- package/core/src/upgrade/drive.ts +31 -2
- package/core/src/upgrade/outcome.ts +1 -1
- package/core/src/upgrade/recover.ts +22 -1
- package/core/src/upgrade/retire.ts +1 -1
- package/core/src/upgrader.ts +7 -8
- package/docs/design.md +173 -0
- package/docs/guide.md +196 -0
- package/docs/harness-design.md +75 -170
- package/docs/integration.md +221 -354
- package/docs/prior-art/design-influences.md +26 -0
- package/docs/prior-art/external-runner-research.md +49 -0
- package/docs/reference.md +209 -0
- package/docs/test-plan.md +89 -92
- package/harness/src/adapter/releaseKnob.ts +1 -1
- package/harness/src/adapter/serviceChecks.ts +5 -5
- package/harness/src/artifact/m1.ts +8 -8
- package/harness/src/artifact/m1Resume.ts +2 -2
- package/harness/src/artifact/m3.ts +25 -104
- package/harness/src/artifact/m3Hosts.ts +9 -61
- package/harness/src/artifact/m4.ts +3 -3
- package/harness/src/artifact/m5.ts +5 -5
- package/harness/src/artifact/m6.ts +6 -6
- package/harness/src/artifact/m6Status.ts +1 -1
- package/harness/src/examples/checks.ts +10 -13
- package/harness/src/fixtures/cliToolSource.ts +166 -0
- package/harness/src/fixtures/externalCrashAdapter.ts +19 -0
- package/harness/src/fixtures/managedHost.ts +100 -0
- package/harness/src/fixtures/serviceSource.ts +181 -0
- package/harness/src/fixtures/supervisedAdapter.ts +57 -0
- package/harness/src/scenario/processScan.ts +3 -1
- package/harness/src/scenario/sandbox.ts +2 -2
- package/harness/src/teeth/artifact.ts +3 -3
- package/harness/src/teeth/examples.ts +1 -1
- package/package.json +5 -3
- package/docs/design-v1.md +0 -246
- package/docs/prior-art.md +0 -150
package/docs/integration.md
CHANGED
|
@@ -1,378 +1,245 @@
|
|
|
1
|
-
# Integrating K
|
|
2
|
-
|
|
3
|
-
A from-zero guide. If you already know updaters, skim §2 (concepts) and jump
|
|
4
|
-
to your profile in §3.
|
|
5
|
-
|
|
6
|
-
## 0. The premise (read this first)
|
|
7
|
-
|
|
8
|
-
**K assumes restarting your service is not expensive.** It guarantees you
|
|
9
|
-
**come back up** — not that you never went down. A short interruption during a
|
|
10
|
-
version change is accepted by design; what K refuses to accept is an upgrade
|
|
11
|
-
that leaves you unrunnable, half-migrated, or claiming success it cannot prove.
|
|
12
|
-
|
|
13
|
-
If you need strict continuous availability, **K is the wrong tool** — better
|
|
14
|
-
said here than discovered from behaviour later.
|
|
15
|
-
|
|
16
|
-
## 1. What problem does K solve? (plain words)
|
|
17
|
-
|
|
18
|
-
Making a program update itself sounds trivial — download the new version,
|
|
19
|
-
replace the file. For a simple CLI tool, it almost is. It stops being trivial
|
|
20
|
-
the moment your program is a **service that keeps running**:
|
|
21
|
-
|
|
22
|
-
- You must swap the binary **under a live process** and hand control to the
|
|
23
|
-
new version without dropping what it was doing.
|
|
24
|
-
- If the new version is broken, you need a way **back** — and "the machine
|
|
25
|
-
crashed halfway through" must never leave the user with nothing runnable.
|
|
26
|
-
- "It updated" is easy to *claim* and surprisingly hard to *prove*. A version
|
|
27
|
-
string can say `2.0` while the old process is still running, or while the
|
|
28
|
-
OS still auto-starts the old copy at boot. (This exact failure — new
|
|
29
|
-
version number, old behavior — is the production incident K grew out of.)
|
|
30
|
-
- On a **person's own machine** (not a company server), you also can't just
|
|
31
|
-
change things silently: the owner decides whether upgrades are automatic,
|
|
32
|
-
confirmed, or notify-only.
|
|
33
|
-
|
|
34
|
-
K packages the solutions to all of these as a library, so an app adopts them
|
|
35
|
-
instead of re-discovering the failure modes one incident at a time.
|
|
36
|
-
|
|
37
|
-
## 2. The concepts, in one paragraph each
|
|
38
|
-
|
|
39
|
-
**Release source** — the one place K asks *your* product two questions:
|
|
40
|
-
"what should this install be on?" (`checkForUpdate`) and "give me exactly
|
|
41
|
-
this version" (`fetchRelease`). K holds **no versioning policy of its own** —
|
|
42
|
-
what your streams are called ("stable", "nightly", "lts-2024"), which version
|
|
43
|
-
counts as newest, whether you use semver or dates, and long-term pinning all
|
|
44
|
-
live inside your source. A ready-made `staticManifestSource({ baseUrl })` covers
|
|
45
|
-
the common case (static host, semver, no automatic downgrade) as *one policy*,
|
|
46
|
-
not as a rule of the framework.
|
|
47
|
-
|
|
48
|
-
**Two slots: `stable` and `experiment`** — K never overwrites your only copy.
|
|
49
|
-
The running, trusted version sits in the *stable* slot. A new version is
|
|
50
|
-
downloaded into the *experiment* slot and run **as a trial**. Only after it
|
|
51
|
-
proves itself is it *promoted* to stable; if anything fails, K *rolls back*
|
|
52
|
-
to the untouched stable copy. Think blue/green deployment, on one machine.
|
|
53
|
-
|
|
54
|
-
**Journal** — before K does anything (download, stop, swap, promote), it
|
|
55
|
-
writes what it is *about* to do to an append-only log, then does it. If the
|
|
56
|
-
machine dies mid-upgrade, the next start replays the journal and either
|
|
57
|
-
finishes the job or rolls back — decided by the log, not by guesswork. This
|
|
58
|
-
is why "kill it at any moment" is a test we run, not a fear.
|
|
59
|
-
|
|
60
|
-
**HostAdapter** — the small interface *you* implement so K can drive *your*
|
|
61
|
-
service without knowing anything about it: pause your workloads
|
|
62
|
-
(`quiesce`), stop/start the service, report health from the live process,
|
|
63
|
-
resume workloads. It's the entire integration surface — K core contains
|
|
64
|
-
zero concepts from any particular app.
|
|
65
|
-
|
|
66
|
-
**Predicates (proof of upgrade)** — instead of trusting a version string, K
|
|
67
|
-
checks two facts and calls the upgrade done only when both hold:
|
|
68
|
-
`binary_at_target` ("the *live process* — same PID that answered — reports
|
|
69
|
-
the new version") and `host_lifecycle_converged` ("OS-level state like
|
|
70
|
-
launch-at-login was written AND read back consistent from its one true
|
|
71
|
-
source"). Metadata like version fields or channel names is *banned* as
|
|
72
|
-
evidence — it has been wrong in the wild.
|
|
73
|
-
|
|
74
|
-
**Policy** — who decides an upgrade happens: `auto` (just do it),
|
|
75
|
-
`confirm` (ask the owner first), `notify-only` (tell, don't act). On
|
|
76
|
-
personal devices the owner always wins; even server-pushed upgrades pass
|
|
77
|
-
this gate.
|
|
78
|
-
|
|
79
|
-
**Install ownership** — if your binary was installed by something else (an
|
|
80
|
-
OS package manager, or a parent service that injects its own copy), that
|
|
81
|
-
manager owns upgrades. K detects this and refuses to self-upgrade a managed
|
|
82
|
-
copy — returning a typed `held: managed-elsewhere` instead of silently
|
|
83
|
-
creating a version mismatch.
|
|
84
|
-
|
|
85
|
-
## 3. Adoption: two process models, plus capabilities you opt into
|
|
86
|
-
|
|
87
|
-
A profile is a **process model**, and the model is defined by one number:
|
|
88
|
-
**how many live incarnations K itself manages.**
|
|
89
|
-
|
|
90
|
-
| Profile | K-managed live processes | Who hands over | Examples |
|
|
91
|
-
|---|---|---|---|
|
|
92
|
-
| **`swap`** | **0** | nobody — new bytes take effect on the next start | a one-shot CLI, `rustup`, **and a long-running interactive session like Claude Code** |
|
|
93
|
-
| **`service`** | **1** (briefly 0 mid-handover) | K stops the old, starts the new, and proves it | a resident daemon, Raft Computer |
|
|
94
|
-
|
|
95
|
-
That a quick CLI and an hours-long agent session share a profile is surprising
|
|
96
|
-
at first and correct on reflection: **neither has a process K hands over.**
|
|
97
|
-
Several old-version processes may keep running in the `swap` model — normal,
|
|
98
|
-
and invisible to K.
|
|
99
|
-
|
|
100
|
-
There is **no third model**. OS lifecycle convergence
|
|
101
|
-
and fleet drive are **capabilities** you opt into on top of `service`; bundling
|
|
102
|
-
them into a "profile" confused *what your app does* with *what K does*, and
|
|
103
|
-
what your app does is none of K's business.
|
|
104
|
-
|
|
105
|
-
```ts
|
|
106
|
-
// a service that also wants its sessions preserved and its OS lifecycle proven
|
|
107
|
-
createUpgrader({ host, source, policy: "auto", /* ... */ });
|
|
108
|
-
// capabilities are declared by implementing the corresponding host duties:
|
|
109
|
-
// named readback surfaces -> lifecycle-convergence
|
|
110
|
-
// attach the drive module -> fleet-drive
|
|
111
|
-
```
|
|
1
|
+
# Integrating K
|
|
112
2
|
|
|
113
|
-
|
|
3
|
+
Build an independent installer from K and a trusted product adapter. The
|
|
4
|
+
application exposes lifecycle/health controls; it does not run K's transaction
|
|
5
|
+
engine. Read [how an upgrade works](guide.md) first if you have not, then
|
|
6
|
+
start with the [runnable example](../examples/external-service/README.md).
|
|
7
|
+
The [design](design.md) states the obligations; the [reference](reference.md)
|
|
8
|
+
has protocols, exit codes and file layout.
|
|
114
9
|
|
|
115
|
-
|
|
116
|
-
application's **semantic** compatibility across versions — and being clear
|
|
117
|
-
about that line is part of the contract.
|
|
10
|
+
## Publish three deliverables
|
|
118
11
|
|
|
119
|
-
|
|
|
12
|
+
| Deliverable | Responsibility |
|
|
120
13
|
|---|---|
|
|
121
|
-
|
|
|
122
|
-
|
|
|
123
|
-
|
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
Rather than leaving compatibility as a documentation promise, declare it —
|
|
135
|
-
K turns your declaration into a mechanical gate:
|
|
136
|
-
|
|
137
|
-
```ts
|
|
138
|
-
class MyHost implements HostAdapter {
|
|
139
|
-
// Optional. Called BEFORE staging and BEFORE promote.
|
|
140
|
-
// Return a refusal string to stop the transition; null to allow.
|
|
141
|
-
async checkCompatibility(from: string, to: string): Promise<string | null> {
|
|
142
|
-
if (schemaGeneration(to) > schemaGeneration(from) && !hasDownMigration(to, from)) {
|
|
143
|
-
return `no down-migration from schema ${to} to ${from}`;
|
|
144
|
-
}
|
|
145
|
-
return null;
|
|
146
|
-
}
|
|
147
|
-
}
|
|
14
|
+
| Bootstrap script (`install.sh`) | Select, download, verify and launch the installer |
|
|
15
|
+
| Installer/runner | K plus your product adapter, published with its own version |
|
|
16
|
+
| Product release | The application executable selected by the adapter's ReleaseSource |
|
|
17
|
+
|
|
18
|
+
All three can share a CDN and build repository. For example:
|
|
19
|
+
|
|
20
|
+
```text
|
|
21
|
+
https://downloads.example.com/my-service/
|
|
22
|
+
install.sh
|
|
23
|
+
installers/1.4.1/linux-x64/installer
|
|
24
|
+
installers/1.4.1/manifest.json
|
|
25
|
+
releases/2.8.0/linux-x64/service
|
|
26
|
+
releases/2.8.0/manifest.json
|
|
148
27
|
```
|
|
149
28
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
out-of-band responsibility, and K says so in `status --json`
|
|
155
|
-
(`compatibility: "undeclared"`), so nobody mistakes silence for a
|
|
156
|
-
guarantee.
|
|
157
|
-
|
|
158
|
-
### Invariants are shipped, not hidden
|
|
29
|
+
These paths and manifest names are illustrative, not a K schema. The bootstrap
|
|
30
|
+
verifies installer 1.4.1, then asks it to install product 2.8.0. The adapter
|
|
31
|
+
resolves the product's URL, SHA-256 and size independently. Authenticate both
|
|
32
|
+
sets of metadata; hashes alone do not establish publisher identity.
|
|
159
33
|
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
shape:
|
|
34
|
+
`install.sh` and `self upgrade` use the same runner protocol and installation
|
|
35
|
+
state. To ship an installer-only fix, publish a new installer and update their
|
|
36
|
+
selection mechanism. Keep versioned artifacts immutable. The controller is an
|
|
37
|
+
execution role, not a mandatory fourth deliverable.
|
|
165
38
|
|
|
166
|
-
|
|
167
|
-
import { BUILT_IN_INVARIANTS, checkInvariants, type Invariant } from "@botiverse/k-carrier";
|
|
39
|
+
## Distribute a built installer
|
|
168
40
|
|
|
169
|
-
|
|
170
|
-
id: "myapp.no-orphaned-jobs",
|
|
171
|
-
description: "no job is left claimed by a dead worker",
|
|
172
|
-
check: (s) => (orphanCount(s) > 0 ? `${orphanCount(s)} orphaned jobs` : null),
|
|
173
|
-
};
|
|
41
|
+
End users download finished artifacts; these build choices belong to publishers.
|
|
174
42
|
|
|
175
|
-
|
|
176
|
-
|
|
43
|
+
| Form | Delivered artifact | Runtime requirement |
|
|
44
|
+
|---|---|---|
|
|
45
|
+
| Single executable per OS/architecture | Node SEA with the runner embedded, built with `--cjs` | No external Node for the worker; supervisor and controller dependencies are separate |
|
|
46
|
+
| Cross-platform JavaScript | K and adapter bundled into one `.mjs` | Independently available Node 24 and adapter dependencies |
|
|
177
47
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
from a seed hours later. Your invariants ride the simulator's seeded fault
|
|
183
|
-
injection for free — that is the practical answer to "who guarantees my
|
|
184
|
-
semantics": **you state them, K's machinery exercises them.**
|
|
48
|
+
One JS file is portable only if its adapter and dependencies support the targets.
|
|
49
|
+
`scripts/build-runner.mjs` produces both bundle forms; SEA injection, signing
|
|
50
|
+
and publication are publisher responsibilities. Whichever form you ship must
|
|
51
|
+
survive stopping and replacing the application.
|
|
185
52
|
|
|
186
|
-
|
|
187
|
-
of the versions.** Where you can express the meaning as a predicate, hand it
|
|
188
|
-
to K and it becomes enforced rather than hoped for.
|
|
53
|
+
### Build a single executable
|
|
189
54
|
|
|
190
|
-
|
|
55
|
+
Node's single-executable-application (SEA) tooling embeds the runner into a
|
|
56
|
+
copy of the Node binary. This recipe uses Node 24.15.0 and a CommonJS entry.
|
|
57
|
+
The supervisor executes the result directly, with no
|
|
58
|
+
`interpreter` option. This procedure was verified end to end: a SEA runner
|
|
59
|
+
promoted the example service under `launchRunner`.
|
|
191
60
|
|
|
192
|
-
```
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
CLI `myapp self upgrade` ──┤ construct| ├─ (no signature client — see §Trust)
|
|
197
|
-
install script ──┘ the same | ├─ txn (two-slot + journal + state machine)
|
|
198
|
-
┌────────────┐ | ├─ lifecycle (handoff orchestration)
|
|
199
|
-
│ Upgrader │──┼──►├─ converge (predicates + readback)
|
|
200
|
-
└────────────┘ | ├─ policy (consent/notify gating)
|
|
201
|
-
your HostAdapter ◄────────────────────┼───┤
|
|
202
|
-
your notificationSink ◄───────────────┼───┤
|
|
203
|
-
your onProgress ◄─────────────────────┼───┘ (calls back into your code only)
|
|
204
|
-
```
|
|
61
|
+
```sh
|
|
62
|
+
# 1. CommonJS entry for this Node 24 SEA recipe.
|
|
63
|
+
# This demo adapter needs the controller change described below before upgrading.
|
|
64
|
+
node scripts/build-runner.mjs --cjs examples/external-service/adapter.ts dist/runner.cjs
|
|
205
65
|
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
entrypoint upgrades correctly and another silently doesn't.
|
|
210
|
-
|
|
211
|
-
## 4.5 Showing progress
|
|
212
|
-
|
|
213
|
-
Pass `onProgress` and K reports where an upgrade is:
|
|
214
|
-
|
|
215
|
-
```ts
|
|
216
|
-
createUpgrader({
|
|
217
|
-
...,
|
|
218
|
-
onProgress: (p) => {
|
|
219
|
-
// p.stage: checking | downloading | verifying | staging
|
|
220
|
-
// | handing-over | probing | promoted | rolled-back
|
|
221
|
-
// p.downloaded / p.total: bytes, present during `downloading` only
|
|
222
|
-
render(p);
|
|
223
|
-
},
|
|
224
|
-
});
|
|
225
|
-
```
|
|
66
|
+
# 2. Prepare the blob.
|
|
67
|
+
printf '%s' '{"main":"dist/runner.cjs","output":"dist/sea-prep.blob","disableExperimentalSEAWarning":true}' > dist/sea-config.json
|
|
68
|
+
node --experimental-sea-config dist/sea-config.json
|
|
226
69
|
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
a bar that restarts from zero after a network blip reads as "it lost my
|
|
235
|
-
download".
|
|
236
|
-
- **Your sink cannot fail the upgrade.** K calls it inside a `try`/`catch`
|
|
237
|
-
and discards anything it throws. An observation surface must never become
|
|
238
|
-
a failure mode — if your renderer breaks, the upgrade still completes.
|
|
239
|
-
|
|
240
|
-
Artifact transfer has three independent fail-closed budgets. Response headers
|
|
241
|
-
must arrive promptly, body progress must not go silent, and the full transfer
|
|
242
|
-
has a hard ceiling derived from the release source's declared byte size. The
|
|
243
|
-
defaults accept a Computer-sized binary that takes longer than ten seconds
|
|
244
|
-
while still bounding an unreachable server and a wedged mid-body stream. An
|
|
245
|
-
adopter with stricter network requirements may provide all four policy fields:
|
|
246
|
-
|
|
247
|
-
```ts
|
|
248
|
-
createUpgrader({
|
|
249
|
-
...,
|
|
250
|
-
artifactTransferPolicy: {
|
|
251
|
-
responseTimeoutMs: 20_000,
|
|
252
|
-
idleTimeoutMs: 30_000,
|
|
253
|
-
minimumBytesPerSecond: 128 * 1024,
|
|
254
|
-
maximumOverallTimeoutMs: 20 * 60_000,
|
|
255
|
-
},
|
|
256
|
-
});
|
|
257
|
-
```
|
|
70
|
+
# 3. Inject into a Node binary for the target platform.
|
|
71
|
+
cp "$(command -v node)" dist/runner
|
|
72
|
+
# macOS only: codesign --remove-signature dist/runner
|
|
73
|
+
npx postject@1.0.0-alpha.6 dist/runner NODE_SEA_BLOB dist/sea-prep.blob \
|
|
74
|
+
--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2
|
|
75
|
+
# macOS: add --macho-segment-name NODE_SEA to the postject command
|
|
76
|
+
# macOS only: codesign --sign - dist/runner (use your release identity)
|
|
258
77
|
|
|
259
|
-
|
|
260
|
-
by `maximumOverallTimeoutMs`. Invalid, zero, or effectively unbounded policies
|
|
261
|
-
are rejected before the byte request starts.
|
|
262
|
-
|
|
263
|
-
The stages are not a parallel state machine: they are derived from the L1
|
|
264
|
-
transaction phases (`stageForPhase`), so a progress display can never show a
|
|
265
|
-
state the transaction does not have.
|
|
266
|
-
|
|
267
|
-
## 4.6 One durable operation receipt
|
|
268
|
-
|
|
269
|
-
When a host detaches the transaction driver from the service it replaces,
|
|
270
|
-
pass an exact operation descriptor to `upgradeTo`. K then owns the only
|
|
271
|
-
durable operation state, including the previous stable version and terminal
|
|
272
|
-
outcome:
|
|
273
|
-
|
|
274
|
-
```ts
|
|
275
|
-
await upgrader.upgradeTo("2.0.0", {
|
|
276
|
-
consented: true,
|
|
277
|
-
operation: {
|
|
278
|
-
id: requestId,
|
|
279
|
-
startedAtMs: Date.now(),
|
|
280
|
-
metadata: { originServerId }, // non-secret host correlation only
|
|
281
|
-
},
|
|
282
|
-
});
|
|
283
|
-
|
|
284
|
-
const receipt = await upgrader.operation();
|
|
285
|
-
if (receipt.kind === "observed" && receipt.operation.outcome !== null) {
|
|
286
|
-
await deliver(receipt.operation);
|
|
287
|
-
await upgrader.acknowledgeOperation(receipt.operation.id);
|
|
288
|
-
}
|
|
78
|
+
# 4. Publish dist/runner with its sha256 and size; launch it with no interpreter.
|
|
289
79
|
```
|
|
290
80
|
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
81
|
+
Build one SEA per target OS and architecture with that platform's Node
|
|
82
|
+
binary. Binary size depends on the target Node build. Installer and product
|
|
83
|
+
release metadata can include optional gzip transport; both paths use the
|
|
84
|
+
same verified downloader.
|
|
85
|
+
|
|
86
|
+
**The SEA pitfall.** Inside a SEA, `process.execPath` is the SEA itself. An
|
|
87
|
+
adapter or controller that spawns `process.execPath some-script.mjs` re-runs
|
|
88
|
+
the embedded runner instead of the script, and the upgrade fails at the first
|
|
89
|
+
controller call. Make the controller a native executable or its own SEA, or
|
|
90
|
+
pass an explicit interpreter path into the adapter at build time. The example
|
|
91
|
+
adapter uses `process.execPath` and therefore only works under an external
|
|
92
|
+
Node.
|
|
93
|
+
|
|
94
|
+
This packages the worker only. `launchRunner` is a Node API, and the demo
|
|
95
|
+
controller also needs Node. To ship an installation chain that needs no
|
|
96
|
+
preinstalled runtime, package the supervisor and controller dependencies too.
|
|
97
|
+
|
|
98
|
+
The bootstrap selects a compatible installer, downloads and verifies it, passes
|
|
99
|
+
the request, supervises settlement, then cleans temporary code. It must not contain
|
|
100
|
+
another swap/rollback algorithm. K provides `launchRunner` for Node callers;
|
|
101
|
+
there is not yet a complete product-ready shell bootstrap template.
|
|
102
|
+
|
|
103
|
+
## 1. Define the adapter and state
|
|
104
|
+
|
|
105
|
+
`createRunner` requires a HostAdapter. Supply release lookup, installation
|
|
106
|
+
ownership, consent policy,
|
|
107
|
+
notification handling and lifecycle operations through trusted build-time code.
|
|
108
|
+
Use `checkCompatibility(from, to)` for transitions constrained by data/protocol
|
|
109
|
+
compatibility. Another package manager's installation is `managed-elsewhere`.
|
|
110
|
+
|
|
111
|
+
The controller implements fence, quiesce, stop, start, healthProbe and resume, either
|
|
112
|
+
directly or through `createCommandHost`. Stop confirms termination; start is
|
|
113
|
+
idempotent; probe returns version, pid and startId from one live instance. Work
|
|
114
|
+
promised by quiesce must be restorable on both the candidate and rollback slots.
|
|
115
|
+
`fence` must confirm that earlier queued or detached controller actions cannot
|
|
116
|
+
later mutate the installation. `createCommandHost` drains recorded controller
|
|
117
|
+
processes first. Adapters with no effects surviving their worker may omit fence;
|
|
118
|
+
all other adapters must supply it and test it against their real service manager.
|
|
119
|
+
A command controller always receives `fence`; if it queues nothing, it
|
|
120
|
+
acknowledges. A stateless service implements `quiesce` and `resume` as
|
|
121
|
+
acknowledgements too. The full obligations only apply to workloads your
|
|
122
|
+
product promises to preserve across an upgrade.
|
|
123
|
+
|
|
124
|
+
Choose one persistent `stateDir` per installation for slots, journal and receipts.
|
|
125
|
+
Keep application data, installer scratch code and interpreter outside the slots.
|
|
126
|
+
Run the installer outside the application's service-management boundary: spawning
|
|
127
|
+
a child does not escape a systemd cgroup or Windows job.
|
|
128
|
+
|
|
129
|
+
## 2. Establish the initial installation
|
|
130
|
+
|
|
131
|
+
K's upgrade flow requires trusted, usable bytes in stable. For an existing
|
|
132
|
+
installation, a trusted setup step calls
|
|
133
|
+
`bootstrapStable({stateDir, version, artifactPath})` with its current executable.
|
|
134
|
+
This seeds a fallback; it does not authenticate/download those bytes or start a
|
|
135
|
+
service. It refuses conflicting state and does not overwrite initialized stable.
|
|
136
|
+
Fresh installation and historical-state repair remain product setup work.
|
|
137
|
+
|
|
138
|
+
The controller starts the K-selected artifact via `slotArtifactPath` or the path
|
|
139
|
+
provided by `createCommandHost`. Each slot contains one `artifact.bin`; package
|
|
140
|
+
layouts and additional install hooks need a product contract.
|
|
141
|
+
|
|
142
|
+
Promotion renames the slot directories. On Windows a running executable
|
|
143
|
+
locks its directory against rename, so a controller there must copy or
|
|
144
|
+
hard-link the artifact to a runtime path outside the slots before starting
|
|
145
|
+
it, and must not execute `artifact.bin` in place. On POSIX running from the
|
|
146
|
+
slot works because open files survive a rename, but copying keeps the
|
|
147
|
+
runtime path stable across promotion on every platform. The example copies
|
|
148
|
+
the selected artifact to `active.mjs` for both reasons (Node also needs the
|
|
149
|
+
extension).
|
|
150
|
+
|
|
151
|
+
K restores executables, not data migrations. Keep repair/cleanup limited to owned
|
|
152
|
+
installation state and provide backup/restore for destructive data changes.
|
|
153
|
+
|
|
154
|
+
## 3. Build and launch
|
|
155
|
+
|
|
156
|
+
On the build machine, bundle your trusted adapter with K:
|
|
157
|
+
|
|
158
|
+
```sh
|
|
159
|
+
pnpm install --frozen-lockfile
|
|
160
|
+
node scripts/build-runner.mjs examples/external-service/adapter.ts /tmp/k-runner.mjs
|
|
303
161
|
```
|
|
304
162
|
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
**This layout belongs to that source, not to K.** Publishing from a private
|
|
309
|
-
API, date-stamped paths, or an OCI registry means writing your own
|
|
310
|
-
`ReleaseSource`; K only ever learns `{ version, url, sha256, size }` and never
|
|
311
|
-
parses a manifest itself. Multiple streams are usually one base URL each
|
|
312
|
-
(`.../stable`, `.../nightly`), which also keeps their blast radius separate.
|
|
313
|
-
|
|
314
|
-
### Trust: what K checks, and what it does not
|
|
315
|
-
|
|
316
|
-
**K verifies integrity, not authenticity.** It checks `sha256` + `size` on the
|
|
317
|
-
assembled bytes. It does **not** verify who produced them: there is no
|
|
318
|
-
signature chain and no trust root (removed 2026-08-06 — `docs/design-v1.md`
|
|
319
|
-
§L0.5 has the decision).
|
|
320
|
-
|
|
321
|
-
A digest is not a signature. `sha256` proves the bytes you received are the
|
|
322
|
-
bytes the manifest described — but the manifest comes from the same place the
|
|
323
|
-
bytes do, so a source serving malicious bytes will serve a matching digest for
|
|
324
|
-
them just as happily.
|
|
325
|
-
|
|
326
|
-
⚠️ **So this is yours to think about, not K's:**
|
|
327
|
-
|
|
328
|
-
| Threat | Covered by K? |
|
|
329
|
-
|--------|---------------|
|
|
330
|
-
| corruption in transit | ✅ (and your HTTPS already covers it) |
|
|
331
|
-
| a wrong artifact on your CDN — leaked publish credentials, misconfigured bucket, poisoned pipeline | ❌ **not covered** — the check passes and every client installs |
|
|
332
|
-
|
|
333
|
-
**OS code signing is a different guarantee, not a substitute.** Authenticode /
|
|
334
|
-
codesign / notarization answer "is this program signed by a recognisable
|
|
335
|
-
vendor", enforced by the OS on the install paths it controls. A distribution
|
|
336
|
-
signature answers "**is this the exact artifact we published**", enforced by
|
|
337
|
-
your app before the bytes reach a slot. If you ship through an app store or a
|
|
338
|
-
platform installer you get some of the former for free; if you ship a plain
|
|
339
|
-
binary from a CDN, as the example host does, you get neither automatically.
|
|
340
|
-
|
|
341
|
-
If you need authenticity today, do it in your own `ReleaseSource`: verify
|
|
342
|
-
before returning the `Release`, and refuse rather than return unverified bytes.
|
|
343
|
-
⚠️ And if you build it, remember the trap this project already hit: **"accept
|
|
344
|
-
unsigned" may only be declared by YOUR code, never by a field in the manifest**
|
|
345
|
-
— the manifest is served by the very party a signature chain exists to distrust.
|
|
346
|
-
|
|
347
|
-
## 6. Testing your integration
|
|
348
|
-
|
|
349
|
-
Three beliefs shape how K is tested — knowing them explains what the harness
|
|
350
|
-
will and won't do with your app (full design: `harness-design.md`):
|
|
351
|
-
|
|
352
|
-
1. **Test like a user.** The primary tests spawn your *real binary* and drive
|
|
353
|
-
it through its *CLI commands*, asserting from outside (exit codes, files,
|
|
354
|
-
what version actually runs next). Library-level tests are the exception,
|
|
355
|
-
not the rule — a green that only exists inside an import is not proof.
|
|
356
|
-
2. **The tests are the spec.** Every guarantee K claims (never dual-run,
|
|
357
|
-
never bricked, sessions survive rollback…) exists as a registered tooth
|
|
358
|
-
with a declared way to make it fail. A claim without a runnable red case
|
|
359
|
-
doesn't count — that includes profile support ("K supports CLIs" is
|
|
360
|
-
backed by a runnable example, not a sentence).
|
|
361
|
-
3. **No test backdoors.** K core contains zero test-awareness — no test
|
|
362
|
-
modes, no "skip verification" flags. Everything the harness uses is a
|
|
363
|
-
product surface you also get (status command, injected clock, config).
|
|
364
|
-
So passing the harness means the *shipping* code path works, not a
|
|
365
|
-
test-shaped variant of it.
|
|
366
|
-
|
|
367
|
-
Run the harness against **your** adapter, at your profile:
|
|
163
|
+
Follow the [example setup](../examples/external-service/README.md) before invoking
|
|
164
|
+
that example runner. After authenticating the caller and obtaining approval for
|
|
165
|
+
the target, submit a request from an operator shell or independent supervisor:
|
|
368
166
|
|
|
369
|
-
```
|
|
370
|
-
|
|
167
|
+
```sh
|
|
168
|
+
printf '%s' '{"protocolVersion":1,"action":"upgrade","id":"install-2","targetVersion":"2.0.0","consented":true}' | node /tmp/k-runner.mjs
|
|
371
169
|
```
|
|
372
170
|
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
171
|
+
`consented` records approval; it is not authorization supplied by an untrusted
|
|
172
|
+
network client. Requests cannot select adapter modules, commands or release URLs.
|
|
173
|
+
Logs go to stderr and the response to stdout. Web entrypoints need an external
|
|
174
|
+
launch facility and a way to retrieve the result after reconnection.
|
|
175
|
+
|
|
176
|
+
## 4. Observe and recover
|
|
177
|
+
|
|
178
|
+
Inspect both the operation outcome and exit code; the
|
|
179
|
+
[walkthrough](guide.md#reading-the-result) explains the two results that
|
|
180
|
+
surprise people, and the [reference](reference.md#protocol-v1) lists every
|
|
181
|
+
code.
|
|
182
|
+
|
|
183
|
+
Retry the same id/target to replay a terminal result. Use a new id for a new
|
|
184
|
+
attempt; there is no by-id status or archive-list action. Terminal receipts are
|
|
185
|
+
archived without an acknowledgement gate. Active work, corrupt state and a live
|
|
186
|
+
lock owner still prevent conflicting transactions. See
|
|
187
|
+
[receipts and retries](reference.md#receipts-and-retries).
|
|
188
|
+
|
|
189
|
+
After installer failure, run a compatible verified installer with `recover` over
|
|
190
|
+
the same state. Recovery needs no release lookup: before durable promote intent
|
|
191
|
+
it restores stable; after it, it replays commit. An external supervisor or
|
|
192
|
+
operator must trigger this after power loss. Never clear a lock or receipt merely
|
|
193
|
+
to bypass unresolved work.
|
|
194
|
+
|
|
195
|
+
Use `launchRunner` for a supervised install, or `superviseRunner` when the caller
|
|
196
|
+
needs a structured result. Both enforce execution/recovery deadlines and recover
|
|
197
|
+
only the original operation. Exit 3 leaves a verified runner and `recovery.json`;
|
|
198
|
+
call `resumeRunner(path)` to retry offline. Keep that directory until recovery
|
|
199
|
+
settles. The [example installer](../examples/external-service/install.mjs) wires
|
|
200
|
+
this flow. Directly invoking a worker does not supervise it.
|
|
201
|
+
|
|
202
|
+
If the whole invocation dies, start a compatible installer against the same
|
|
203
|
+
state; it settles unfinished work before executing a new request. A live earlier
|
|
204
|
+
worker still blocks takeover. Product OS startup hooks and service-unit isolation
|
|
205
|
+
must be validated separately; K does not install a permanent watchdog. If a
|
|
206
|
+
request died before its operation was recorded, bound recovery refuses rather
|
|
207
|
+
than guessing which earlier operation it owns. Inspect `status` and explicitly
|
|
208
|
+
run operator `recover` on the retained runner when current-state repair is needed.
|
|
209
|
+
|
|
210
|
+
## 5. Validate the product
|
|
211
|
+
|
|
212
|
+
Use the [test plan](test-plan.md), then test your real installer and controller on
|
|
213
|
+
each target platform. Cover baseline setup, running-service upgrade, bad-candidate
|
|
214
|
+
rollback, installer death, offline recovery, workload/data retention and service
|
|
215
|
+
isolation. Observe declared OS lifecycle surfaces before retiring their previous
|
|
216
|
+
manager. A green framework test is not product acceptance.
|
|
217
|
+
|
|
218
|
+
## Using Hands as the release platform
|
|
219
|
+
|
|
220
|
+
Hands is the release-management platform K's authors use to publish installers
|
|
221
|
+
and application releases. Any platform that answers "which version, at which
|
|
222
|
+
URL, with which SHA-256 and size" fits the same way; nothing here is specific
|
|
223
|
+
to Hands.
|
|
224
|
+
|
|
225
|
+
Hands supplies publication, channel/platform selection and artifact metadata.
|
|
226
|
+
Your adapter maps its response to a K ReleaseSource with exact version, URL,
|
|
227
|
+
SHA-256 and size; K performs the local transaction. A launcher may separately
|
|
228
|
+
obtain the installer from Hands. Keep installer and product identities distinct.
|
|
229
|
+
|
|
230
|
+
K has no built-in Hands connector or result uploader. Product authentication,
|
|
231
|
+
channel/cohort policy and remote reporting belong to the integration. Forward the
|
|
232
|
+
actual operation id/outcome; publication or process launch is not installation
|
|
233
|
+
success, and local promotion does not prove cloud reconnection.
|
|
234
|
+
|
|
235
|
+
Withdrawing a release affects future distribution. It does not roll back already
|
|
236
|
+
installed machines. K can recover existing local slots offline; downloading an
|
|
237
|
+
older release still depends on the source authorizing and serving it.
|
|
238
|
+
|
|
239
|
+
### Optional gzip release transport
|
|
240
|
+
|
|
241
|
+
A `Release` may include a `gzip` URL, compressed size and SHA-256. K verifies
|
|
242
|
+
compressed bytes, bounds decompression, then checks the canonical size and hash.
|
|
243
|
+
Missing gzip metadata uses the canonical URL. Failure of a selected gzip object
|
|
244
|
+
is terminal; K does not silently switch representations. Resume offsets refer to
|
|
245
|
+
the compressed object.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# Prior art and design influences
|
|
2
|
+
|
|
3
|
+
K draws on existing installer designs. This directory records the ideas and their
|
|
4
|
+
limits; it is not a claim that K supersedes every updater or that another project
|
|
5
|
+
lacks a feature. The original surveys used upstream source snapshots or moving
|
|
6
|
+
`main` links and were not exhaustive product audits. Attribution is summarized
|
|
7
|
+
in [NOTICE](../../NOTICE).
|
|
8
|
+
|
|
9
|
+
| Source | Idea used in K | Boundary |
|
|
10
|
+
|---|---|---|
|
|
11
|
+
| [Rustup](https://github.com/rust-lang/rustup) | Thin bootstrap and an installer that can execute independently of the installed program | Helper code still has a version, trust requirements and platform-specific replacement constraints |
|
|
12
|
+
| [Tailscale clientupdate](https://github.com/tailscale/tailscale/tree/main/clientupdate) | Respect installation ownership; distinguish replacing bytes from restarting the service | K uses an external runner; it does not embed the transaction in the application daemon |
|
|
13
|
+
| [Datadog installer](https://github.com/DataDog/datadog-agent/tree/main/pkg/fleet) | Stable/experiment slots with promotion and rollback | K does not adopt a package catalog, permanent installer daemon or fleet control plane |
|
|
14
|
+
| [Tailscale distsign](https://github.com/tailscale/tailscale/tree/main/clientupdate/distsign) | A studied example of distribution authentication | K currently checks SHA-256 and size, not publisher signatures; no signing roadmap is implied |
|
|
15
|
+
|
|
16
|
+
The [external installer research](external-runner-research.md) gives source links
|
|
17
|
+
and explains the execution-boundary decision. K's supported contract is defined
|
|
18
|
+
by its [design](../design.md), not by feature comparisons with other projects.
|
|
19
|
+
|
|
20
|
+
The original test survey also informed table-driven platform tests and observing
|
|
21
|
+
events after a known marker. K adds its own fault injection, seeded simulation
|
|
22
|
+
and real-runner tests; these do not establish exhaustive platform reliability.
|
|
23
|
+
See the [harness guide](../harness-design.md) for what each layer proves.
|
|
24
|
+
|
|
25
|
+
These are conceptual influences. The project's attribution record states that
|
|
26
|
+
no upstream implementation code was copied.
|