@botiverse/k-carrier 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/NOTICE +24 -0
- package/README.md +62 -0
- package/core/src/artifact/README.md +1 -0
- package/core/src/artifact/collectStream.ts +49 -0
- package/core/src/artifact/download.ts +300 -0
- package/core/src/artifact/errors.ts +24 -0
- package/core/src/artifact/partialPath.ts +18 -0
- package/core/src/artifact/source.ts +59 -0
- package/core/src/artifact/staticManifestSource.ts +185 -0
- package/core/src/artifact/swap.ts +21 -0
- package/core/src/clock.ts +22 -0
- package/core/src/converge/lifecycle.ts +114 -0
- package/core/src/converge/predicates.ts +57 -0
- package/core/src/converge/report.ts +38 -0
- package/core/src/createUpgrader.ts +300 -0
- package/core/src/drive/README.md +1 -0
- package/core/src/index.ts +17 -0
- package/core/src/invariants.ts +243 -0
- package/core/src/lifecycle/hostAdapter.ts +77 -0
- package/core/src/liveness.ts +80 -0
- package/core/src/platform/README.md +1 -0
- package/core/src/platform/index.ts +11 -0
- package/core/src/platform/ops.ts +53 -0
- package/core/src/platform/posix.ts +99 -0
- package/core/src/platform/windows.ts +73 -0
- package/core/src/policy/README.md +1 -0
- package/core/src/progress.ts +54 -0
- package/core/src/provenance/journal.ts +216 -0
- package/core/src/status/report.ts +76 -0
- package/core/src/status/reportStore.ts +67 -0
- package/core/src/txn/effects.ts +40 -0
- package/core/src/txn/engine.ts +275 -0
- package/core/src/txn/fileEffects.ts +129 -0
- package/core/src/txn/hostCallBudget.ts +28 -0
- package/core/src/txn/lock.ts +124 -0
- package/core/src/txn/state.ts +77 -0
- package/core/src/txn/transitions.ts +49 -0
- package/core/src/upgrade/outcome.ts +53 -0
- package/core/src/upgrade/retire.ts +44 -0
- package/core/src/upgrader.ts +124 -0
- package/docs/design-v1.md +246 -0
- package/docs/harness-design.md +170 -0
- package/docs/integration.md +326 -0
- package/docs/prior-art.md +150 -0
- package/docs/test-plan.md +100 -0
- package/harness/README.md +20 -0
- package/harness/src/adapter/probeChecks.ts +93 -0
- package/harness/src/adapter/releaseKnob.ts +65 -0
- package/harness/src/adapter/serviceChecks.ts +260 -0
- package/harness/src/artifact/checks.ts +232 -0
- package/harness/src/artifact/downloadHoles.ts +300 -0
- package/harness/src/artifact/downloadHolesMutations.ts +170 -0
- package/harness/src/artifact/m1.ts +204 -0
- package/harness/src/artifact/m1Resume.ts +141 -0
- package/harness/src/artifact/m3.ts +296 -0
- package/harness/src/artifact/m3Hosts.ts +241 -0
- package/harness/src/artifact/m4.ts +201 -0
- package/harness/src/artifact/m5.ts +239 -0
- package/harness/src/artifact/m6.ts +274 -0
- package/harness/src/artifact/m6Drive.ts +296 -0
- package/harness/src/artifact/m6DriveMutations.ts +70 -0
- package/harness/src/artifact/m6Mutations.ts +136 -0
- package/harness/src/artifact/m6Status.ts +224 -0
- package/harness/src/artifact-factory/checks.ts +68 -0
- package/harness/src/artifact-factory/demo.ts +70 -0
- package/harness/src/artifact-factory/factory.ts +134 -0
- package/harness/src/artifact-factory/run.ts +107 -0
- package/harness/src/blackbox.ts +284 -0
- package/harness/src/cli.ts +281 -0
- package/harness/src/crash/enumerate.ts +66 -0
- package/harness/src/crash/runner.ts +162 -0
- package/harness/src/examples/checks.ts +286 -0
- package/harness/src/fake-host/checks.ts +188 -0
- package/harness/src/fake-host/daemon-entry.ts +39 -0
- package/harness/src/fake-host/daemon.ts +159 -0
- package/harness/src/fake-host/inproc.ts +256 -0
- package/harness/src/fake-server/manifest.ts +68 -0
- package/harness/src/fake-server/server.ts +205 -0
- package/harness/src/fake-server/store.ts +208 -0
- package/harness/src/fixtures/minimal-adapter.ts +108 -0
- package/harness/src/fixtures/service-adapter.ts +187 -0
- package/harness/src/index.ts +11 -0
- package/harness/src/receipt.ts +83 -0
- package/harness/src/runner.ts +268 -0
- package/harness/src/scenario/processScan.ts +55 -0
- package/harness/src/scenario/sandbox.ts +259 -0
- package/harness/src/scenario/virtualClock.ts +56 -0
- package/harness/src/selfverify/selfVerify.ts +136 -0
- package/harness/src/sim/checks.ts +46 -0
- package/harness/src/sim/corpus.ts +36 -0
- package/harness/src/sim/effectRuntime.ts +113 -0
- package/harness/src/sim/error.ts +14 -0
- package/harness/src/sim/prng.ts +29 -0
- package/harness/src/sim/record.ts +76 -0
- package/harness/src/sim/run.ts +148 -0
- package/harness/src/sim/scheduler.ts +72 -0
- package/harness/src/sim/world.ts +284 -0
- package/harness/src/target.ts +86 -0
- package/harness/src/targetCheck.ts +47 -0
- package/harness/src/teeth/artifact.ts +215 -0
- package/harness/src/teeth/artifactDownload.ts +136 -0
- package/harness/src/teeth/artifactFactory.ts +41 -0
- package/harness/src/teeth/artifactM5.ts +68 -0
- package/harness/src/teeth/artifactM6.ts +264 -0
- package/harness/src/teeth/blackbox.ts +22 -0
- package/harness/src/teeth/checks.ts +278 -0
- package/harness/src/teeth/examples.ts +57 -0
- package/harness/src/teeth/fakeHost.ts +120 -0
- package/harness/src/teeth/index.ts +16 -0
- package/harness/src/teeth/m0.ts +135 -0
- package/harness/src/teeth/registry.ts +267 -0
- package/harness/src/teeth/selfCheck.ts +45 -0
- package/harness/src/teeth/sim.ts +61 -0
- package/package.json +43 -0
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
# K 测试框架(harness)设计 v1
|
|
2
|
+
|
|
3
|
+
08-05:测试框架要**提前设计成完整的一块**,不是随层补测试。本文是 harness 的架构设计;`test-plan.md` 是跑在它上面的计划。
|
|
4
|
+
|
|
5
|
+
## 0. 定位:harness = 框架的可执行规格(executable spec)
|
|
6
|
+
|
|
7
|
+
顺序反转:**先有 harness,后有功能层**。每个功能层落地的定义 = "它让 harness 里预先写好的那组齿从 RED 变 GREEN"。测试不是功能的附件,是功能的规格。三个推论:
|
|
8
|
+
1. M0(harness 自举)先于一切层实现;
|
|
9
|
+
2. **不允许 harness 外的 ad-hoc 测试**——新齿必须进 registry(否则齿的 must-red/分档/自验纪律管不到它);
|
|
10
|
+
3. harness 只吃 core 的公共 API + HostAdapter ⇒ 它同时是 **API 的第一个消费者**(dogfood:API 不好用,harness 先痛)。
|
|
11
|
+
|
|
12
|
+
## 1. 组件架构
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
harness/
|
|
16
|
+
├─ fake-host/ 假宿主(两种形态)
|
|
17
|
+
│ ├─ inproc.ts 进程内 HostAdapter 实现(快速单元级)
|
|
18
|
+
│ └─ daemon.ts 可 spawn 的真进程假 daemon(kill -9 是真的)
|
|
19
|
+
├─ fake-server/ 本地静态发布服务器 + 篡改 API
|
|
20
|
+
├─ artifact-factory/ 版本工件工厂(一次构建多次盖戳 + behavior 旋钮)
|
|
21
|
+
├─ scenario/ 场景运行器(隔离沙箱 + 虚拟时钟)
|
|
22
|
+
├─ crash/ 崩溃注入编排器(枚举生成,禁手列)
|
|
23
|
+
├─ teeth/ 齿注册表 + 分档 + 自验
|
|
24
|
+
└─ cli.ts `k-harness` 入口(含 --adapter 接入方模式)
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
### 1.1 fake-host(假宿主)
|
|
28
|
+
- **两形态**:`inproc`(进程内实现,跑快速逻辑齿)+ `process`(编译成真二进制、真 spawn、真 PID/startId —— kill -9、双跑检测、probe 活性都必须在真进程上验,mock 验不了崩溃)。
|
|
29
|
+
- **故障注入开关**(per 方法):`fail-on-quiesce / hang-on-stop / wrong-version-probe / stale-startId-probe / crash-during-start ...`——每颗齿测"故障被抓",开关关掉齿必须转绿(证明齿测的是故障不是常态)。
|
|
30
|
+
- **虚拟负载账本**:假宿主维护一个确定性"会话状态"文件(计数器+校验和);`quiesce↔resume` 等价断言 = 账本逐字节比对(**含 rolled-back 后 resume**)。这是"会话保留"的可机械判定形态。
|
|
31
|
+
|
|
32
|
+
### 1.2 fake-server(假发布端)
|
|
33
|
+
- 本地静态文件服务 + manifest 构造器(含 Range 续传——不认 Range 的桩会让"续传"悄悄退化成普通下载)。
|
|
34
|
+
- **篡改 API**:`corruptByte(file, offset) / swapFiles / serveOlderVersion / dropFile` —— 完整性齿全部走"真篡改→真拒绝",不 mock 校验函数。判据是 sha256:K 验完整性不验来源(design-v1 §L0.5),所以篡改的判据也只能是"服务的字节还对不对得上 manifest 的摘要"。
|
|
35
|
+
|
|
36
|
+
### 1.3 scenario(场景运行器)
|
|
37
|
+
- **一场景一沙箱**:独立 temp stateDir + 独立 fake-server 端口 → 全部并行安全、可重复。
|
|
38
|
+
- **虚拟时钟注入**:core 的超时/重试全走注入 clock(框架级 clock seam —— 我们 web 侧 clock-ratchet 的同款纪律),场景可快进;无真实 sleep。
|
|
39
|
+
- 场景 = 声明式脚本(步骤 + 期望 outcome + 期望 journal 尾部),跑完输出结构化 receipt(给 CI 和人两用)。
|
|
40
|
+
|
|
41
|
+
### 1.4 crash(崩溃注入编排器,承重件)
|
|
42
|
+
- **覆盖面由代码生成**:从 core 导入状态机迁移表,自动枚举 `迁移边 × kill 点`(每个动作的 journal-写前/写后至少两点)→ 生成场景矩阵。**手列 kill 点非法**。
|
|
43
|
+
- **完备性齿**:core 新增一个 phase/迁移而矩阵没覆盖 ⇒ harness 自身 RED(枚举器数量对账)。防"加了状态忘了测崩溃"。
|
|
44
|
+
- 每格断言同一组不变式:重启后 = 恢复 stable 或完成迁移;**永不双跑**(真进程存活探测);**永不砖**(stable 可再启动);journal 可重放。
|
|
45
|
+
|
|
46
|
+
### 1.45 DST:确定性模拟测试(与崩溃矩阵互补)
|
|
47
|
+
|
|
48
|
+
**实现状态:已落地。** 入口为 `k-harness sim`;实现位于
|
|
49
|
+
`harness/src/sim/`,三颗注册齿为 `sim.seed-replay-identical`、
|
|
50
|
+
`sim.smoke-invariants`、`sim.fault-surface-covered`。
|
|
51
|
+
|
|
52
|
+
**两条腿**:崩溃矩阵 = **枚举穷举**(在它的粒度上完备,给保证);DST = **种子随机深探**(FoundationDB/TigerBeetle 手法,找枚举想不到的交错,给发现)。
|
|
53
|
+
|
|
54
|
+
- **前提(对 txn 引擎的架构约束,写引擎前就定)**:L1 引擎 = **纯状态机 + Effects 接口**——journal append/fsync、槽操作、宿主调用、时钟全部经注入的 effects 层,引擎本体零直接 IO/时间/随机。这**不是**测试后门(§1.8 自洽):effects 层就是平台适配器的天然挂点(各平台 fsync/swap 本来就不同实现),是产品级抽象。
|
|
55
|
+
- **模拟器**:SimEffects = 内存盘(可模拟 partial write / fsync 丢失 / 重排)+ VirtualClock + **种子 PRNG 故障调度器**(在任意 effect 点注入 crash/fail/delay,按种子决定)。跑 N 千个种子 × 每种子一条完整升级/回滚剧本 → 断言同一组不变式(永不双跑/永不砖/journal 可重放/谓词诚实)。
|
|
56
|
+
- **可复现**:任何失败 = 一个种子号,`k-harness sim --seed X` 逐字节重放。**失败种子沉淀为枚举矩阵的新固定格**(发现→保证的转化管道)。
|
|
57
|
+
- **跑法**:PR 门跑固定 smoke 种子集(快、确定);夜跑扩大随机种子量;语料库(历史失败种子)永久保留。
|
|
58
|
+
- PR/本地:`k-harness sim`(固定、评审可见的 smoke seeds)。
|
|
59
|
+
- 单种子重放:`k-harness sim --seed X --json`(同 seed receipt 逐字节相同)。
|
|
60
|
+
- 扩量:`k-harness sim --start-seed X --seeds N`;nightly 默认 50,000 seeds。
|
|
61
|
+
- 失败自动原子合并进 `.k-harness/sim-failures.json`,workflow 保存为 artifact;记录含 seed、失败原因、transcript SHA-256 与一条可直接运行的 replay command。
|
|
62
|
+
- 范围诚实:DST 覆盖 txn/converge 的逻辑交错;真进程/真 OS 面(信号、真 fsync 语义)仍归崩溃矩阵与真机轮——**模拟不替代真进程层,两者叠加**。
|
|
63
|
+
|
|
64
|
+
### 1.5 teeth(齿注册表)
|
|
65
|
+
每颗齿是一条注册记录,声明即纪律(缺任一字段注册失败):
|
|
66
|
+
```ts
|
|
67
|
+
registerTooth({
|
|
68
|
+
id: "txn.no-dual-run",
|
|
69
|
+
profiles: ["service", "hosted"], // 分档
|
|
70
|
+
kind: "invariant", // 或 { kind: "baseline", failureCondition: "..." }
|
|
71
|
+
mustRed: [ // mutation 契约:≥1 条,且答得出"不被我抓还会被谁抓"
|
|
72
|
+
{ mutate: "skip journal fsync before handover", caughtOnlyBy: "this" },
|
|
73
|
+
],
|
|
74
|
+
run: async (ctx) => { ... },
|
|
75
|
+
});
|
|
76
|
+
```
|
|
77
|
+
- **分档执行**:`--profile swap|daemon|managed` 选齿集;cli 档误挂 L2 齿 ⇒ 注册期报错(档界齿)。
|
|
78
|
+
- **断言二分机械化**:`kind` 必填 invariant 或 baseline-带失效条件;CI 扫无标注断言(断言纪律的执行器)。
|
|
79
|
+
- **mutation-runner 对接**:registry 导出齿清单 + must-red 表,Lincan 的 runner 直接消费(未变异 baseline 0 失败 / 每齿变异必红 / 全红也不发结论)。
|
|
80
|
+
|
|
81
|
+
### 1.6 自验(M0 出口,harness 的上岗证)
|
|
82
|
+
harness 判定别人之前先判自己,三样本缺一不可:
|
|
83
|
+
- **known-green**:正确实现走完整升级 → 必须全绿;
|
|
84
|
+
- **known-red**:注入一个已知故障 → 对应齿必须红、且只红该红的;
|
|
85
|
+
- **对抗样本**:**结构上能过齿的检查、但违反真实 oracle** 的假实现(例:probe 换 pid 不换 startId 报新版本;quiesce 把账本备份再恢复伪装等价)→ 必须被抓。对抗样本清单随齿长(每次真实逃逸事后加一条)。
|
|
86
|
+
自验不过 ⇒ harness 拒绝运行任何评审(exit 非零 + typed 原因)。
|
|
87
|
+
|
|
88
|
+
### 1.7 接入方模式(`k-harness --profile X --adapter path`)
|
|
89
|
+
同一套齿对**外部真 adapter** 跑合规子集(不跑需要故障开关的齿,跑契约齿:quiesce↔resume 等价、probe 活性、ownership 响应)。绿 = 接入方契约达标;这也是 examples 三 demo 的验收方式——**demo 和接入方走同一道门**。
|
|
90
|
+
|
|
91
|
+
## 1.75 两个测试平面:黑盒优先(08-05:"就像启动一个 CLI、跑它的命令")
|
|
92
|
+
|
|
93
|
+
harness 有两个平面,**默认用外面那个**:
|
|
94
|
+
|
|
95
|
+
- **黑盒平面(主平面)**:spawn **真实打包好的二进制**,只通过它的命令行驱动(`mytool self upgrade` / `mytool status`),从外面断言:exit code、输出、盘上文件、进程状态、下次运行的版本。**就是用户的用法** —— 它顺带真正验证了"每个入口构造同一 Upgrader"这类 claim(library 平面验不了打包/入口接线)。examples 三 demo 都是真 CLI,端到端齿全在这层写。
|
|
96
|
+
- **library 平面(辅助)**:import core API 直驱 Upgrader——只留给黑盒够不着的内部齿(如 journal 重放细节)。
|
|
97
|
+
|
|
98
|
+
**规则:能在黑盒层表达的齿必须写在黑盒层**;library 层是例外、要说明为什么外面够不着。(同我们 symptom-layer 教义:用户层的红是最不可伪造的 oracle。)
|
|
99
|
+
|
|
100
|
+
接入方黑盒模式随之而来:`k-harness --profile swap --bin ./mytool` —— **零代码集成**:给你的真二进制,harness 起 fake-server、跑你的升级命令、断言下次运行版本/回滚/held。比 `--adapter` 还轻(cli 档接入方连 adapter 都不用给)。
|
|
101
|
+
|
|
102
|
+
## 1.76 黑盒 CLI 契约(08-05:"需要定义查版本子命令、状态 predicate 之类")
|
|
103
|
+
|
|
104
|
+
黑盒平面要从外面问二进制三类问题,因此有一个**小的 CLI 契约** —— 但它是"声明"不是"每家自己发明":
|
|
105
|
+
|
|
106
|
+
**① core 白送命令实现**(app 只做一行接线):
|
|
107
|
+
- `versionCommand()` → 打印二进制自身版本(cli 档够用);
|
|
108
|
+
- `statusCommand()` → **问活进程**(走宿主 socket,同 same-PID 纪律)输出机读 JSON:`{ProcessEvidence, TxnState, ConvergenceReport}` —— 就是 core 已有的三个类型,不另造 schema;
|
|
109
|
+
- `selfUpgradeCommand()` → 包装 `upgrader.upgrade()`,outcome 四态按结构化输出。
|
|
110
|
+
|
|
111
|
+
**② app 声明命令名**(K 不规定你的 CLI 长相,但声明是**必须的**——harness 不猜命令,缺声明立即 typed FAIL):二进制旁边放一个 `k.target.ts`(或 `--target <path>`),default export typed `BlackBoxTarget`,命令名全部显式声明:
|
|
112
|
+
```ts
|
|
113
|
+
import type { BlackBoxTarget } from "@k-carrier/harness";
|
|
114
|
+
|
|
115
|
+
export default {
|
|
116
|
+
version: ["--version"],
|
|
117
|
+
status: ["k-status", "--json"], // 可选
|
|
118
|
+
selfUpgrade: ["self", "upgrade"],
|
|
119
|
+
env: { K_RELEASE_BASE: "..." }, // 可选
|
|
120
|
+
} satisfies BlackBoxTarget;
|
|
121
|
+
```
|
|
122
|
+
`k-harness --bin ./mytool` 动态 import 它来驱动(node 原生跑 TS,不用 build)。**没有 `k.target.ts` = 立即 typed FAIL(`BLACKBOX_TARGET_REQUIRED`),不探测、不猜**——猜对了省一行配置,猜错了给出的是一个可信的错误结论(不确定时要求显式声明,不替用户发明规矩)。
|
|
123
|
+
|
|
124
|
+
**③ 与透明性原则自洽**:这些不是测试后门,是**产品本来就该有的面**(用户和 support 一样需要 `status --json`)——harness 只是恰好消费它们。cli 档最小契约 = `version` + `selfUpgrade` 两条;daemon/managed 档 + `status`(活进程 JSON)。
|
|
125
|
+
|
|
126
|
+
**齿**:契约自身可验 —— `k.target.ts` 声明的命令跑不通 / status 输出不合 schema ⇒ 黑盒验收直接 FAIL(typed,不进齿评审);缺 target 文件 ⇒ 必 FAIL 且信息可操作(齿 `blackbox.missing-target-fails`)。
|
|
127
|
+
|
|
128
|
+
## 1.77 版本工件工厂 + 清场(08-05:"准备相应版本的二进制?删除清空?")
|
|
129
|
+
|
|
130
|
+
**① artifact-factory(升级测试需要"同一个 app 的 vX 和 vY")**:
|
|
131
|
+
- `makeRelease({version, behavior})` → 产出**盖了版本戳的真二进制** + manifest,落到 fake-server。实现 = **一次构建、多次盖戳**(构建 demo 源码一次,post-build 往二进制里注入版本串——与真 SEA 嵌版本同型,快且真实;不用"版本写在旁边文件"那种假形态)。
|
|
132
|
+
- `behavior` 旋钮让某个"新版本"**故意坏**:`crash-on-start / wrong-probe / hang-on-quiesce ...` —— 回滚齿、known-red、对抗样本的 fixture 都从这来("升到坏版本→自动回滚→stable 完好"整条链可黑盒复现)。
|
|
133
|
+
- 内容寻址缓存(key = demo 源 hash × version × behavior),跨场景复用,不重复构建。
|
|
134
|
+
|
|
135
|
+
**② 清场(teardown)**:
|
|
136
|
+
- 沙箱边界即清场边界:install dir + stateDir + fake-server 存储全在场景沙箱内。teardown = **杀进程树并确认真死**(按沙箱标记 pgrep 复核零残留——僵尸 `__service` 是我们的真实产线教训,"发了 kill"≠"死了")→ 删沙箱目录。崩溃场景故意留下的中间态也被同一动作清干净(一切都在沙箱里,所以 rm 恒有效)。
|
|
137
|
+
- **越界写齿**:core 在任何场景中写沙箱外任何路径 ⇒ RED。这颗齿顺带保证了产品级卫生(升级器不污染全局 HOME/系统目录),也让"清空"永远可信——**能一键删干净,是因为先机械保证了它只写在自己地盘**。
|
|
138
|
+
- 顺带的产品映射:沙箱清单 = 将来"干净卸载"要删的东西的权威地图(卸载功能本身另立,不在本期)。
|
|
139
|
+
|
|
140
|
+
## 1.8 透明性原则(08-05:测试框架对升级框架透明)
|
|
141
|
+
|
|
142
|
+
**core 对 harness 零感知,机械强制**:
|
|
143
|
+
- **禁 test-conditional**:core 内不得存在 "if under test" 任何形态(环境变量开关/测试模式 flag/AllowUnsigned 之类后门)。CI ratchet 扫 core 源码(同我们 clock-ratchet 手法),出现即红。
|
|
144
|
+
- harness 需要的一切必须走**产品级注入面**——这些面是产品本来就需要的,不是为测试开的:
|
|
145
|
+
- `HostAdapter`:产品 API 本体,fake-host 只是又一个 adapter;
|
|
146
|
+
- `releaseBase`:指向 localhost 是配置,不是测试感知;
|
|
147
|
+
- `clock`:时钟 seam 是正当的生产抽象(默认真时钟),不是测试后门;
|
|
148
|
+
- `stateDir`:本就按 app 配置。
|
|
149
|
+
- 崩溃注入 = 对真进程 kill -9,零 core 配合;故障注入全在 fake-host(harness 侧代码);对抗样本 = 假 adapter——全部外部。
|
|
150
|
+
- **反向信号**:若某颗齿写不出来、除非给 core 开后门 ⇒ 判定为**公共 API 不足**(dogfood 信号),修 API 而不是开门。透明性由此与 forcing-function 同构:测试框架也只能是 core 的一个普通消费者。
|
|
151
|
+
|
|
152
|
+
## 2. 关键设计决定(为什么这样)
|
|
153
|
+
1. **真进程优先**:崩溃/双跑/probe 活性只在真 spawn 的假 daemon 上验——mock 崩溃 = 没测崩溃。inproc 只服务快速逻辑齿。
|
|
154
|
+
2. **枚举生成覆盖面**:kill 矩阵、齿-档对账、断言标注扫描全由代码生成/校验,**人列清单在这三处非法**(人会漏,且漏的方向总是"看起来覆盖够了")。
|
|
155
|
+
3. **虚拟时钟 + 沙箱**:决定论优先;flaky 即 bug。
|
|
156
|
+
4. **齿注册表是唯一入口**:declaration = 纪律载体(分档/must-red/二分标注都在注册时强制),绕开注册表的测试 CI 拒收。
|
|
157
|
+
5. **对抗样本制度化**:自验含"骗过检查但违反 oracle"的样本,且逃逸事后必须沉淀为新对抗样本——今天 uninstall 设计 那套 BLIND 对抗采样的教训直接机械化。
|
|
158
|
+
|
|
159
|
+
## 3. 实现顺序(M0 内部)
|
|
160
|
+
1. teeth 注册表 + 分档执行器 + 二分标注检查(纯逻辑,先立规矩);
|
|
161
|
+
2. scenario 沙箱 + 虚拟时钟;
|
|
162
|
+
3. fake-server(静态服务 + 篡改 API);
|
|
163
|
+
4. fake-host inproc → fake-host daemon(真进程);
|
|
164
|
+
5. crash 枚举器(吃 core 状态机表——此时 core 只需 `txn/state.ts` 的类型,已存在);
|
|
165
|
+
6. 自验三样本 → **M0 出口**。
|
|
166
|
+
此后每个功能层(M1–M6)的落地 = 先在 registry 写该层的齿(RED)→ 实现层 → 齿转 GREEN。
|
|
167
|
+
|
|
168
|
+
## 4. 边界
|
|
169
|
+
- harness 不测 UI/产品语义(壳仓库自己的事);只测 core 契约 + 接入方 adapter 合规。
|
|
170
|
+
- 真机/平台矩阵(mac launchd、Windows 服务)走 Testbed 轮,harness 出可移植齿、Testbed 供真床。
|
|
@@ -0,0 +1,326 @@
|
|
|
1
|
+
# Integrating K into your application
|
|
2
|
+
|
|
3
|
+
A from-zero guide. If you already know updaters, skim §2 (concepts) and jump
|
|
4
|
+
to your profile in §3.
|
|
5
|
+
|
|
6
|
+
## 0. The premise (read this first)
|
|
7
|
+
|
|
8
|
+
**K assumes restarting your service is not expensive.** It guarantees you
|
|
9
|
+
**come back up** — not that you never went down. A short interruption during a
|
|
10
|
+
version change is accepted by design; what K refuses to accept is an upgrade
|
|
11
|
+
that leaves you unrunnable, half-migrated, or claiming success it cannot prove.
|
|
12
|
+
|
|
13
|
+
If you need strict continuous availability, **K is the wrong tool** — better
|
|
14
|
+
said here than discovered from behaviour later.
|
|
15
|
+
|
|
16
|
+
## 1. What problem does K solve? (plain words)
|
|
17
|
+
|
|
18
|
+
Making a program update itself sounds trivial — download the new version,
|
|
19
|
+
replace the file. For a simple CLI tool, it almost is. It stops being trivial
|
|
20
|
+
the moment your program is a **service that keeps running**:
|
|
21
|
+
|
|
22
|
+
- You must swap the binary **under a live process** and hand control to the
|
|
23
|
+
new version without dropping what it was doing.
|
|
24
|
+
- If the new version is broken, you need a way **back** — and "the machine
|
|
25
|
+
crashed halfway through" must never leave the user with nothing runnable.
|
|
26
|
+
- "It updated" is easy to *claim* and surprisingly hard to *prove*. A version
|
|
27
|
+
string can say `2.0` while the old process is still running, or while the
|
|
28
|
+
OS still auto-starts the old copy at boot. (This exact failure — new
|
|
29
|
+
version number, old behavior — is the production incident K grew out of.)
|
|
30
|
+
- On a **person's own machine** (not a company server), you also can't just
|
|
31
|
+
change things silently: the owner decides whether upgrades are automatic,
|
|
32
|
+
confirmed, or notify-only.
|
|
33
|
+
|
|
34
|
+
K packages the solutions to all of these as a library, so an app adopts them
|
|
35
|
+
instead of re-discovering the failure modes one incident at a time.
|
|
36
|
+
|
|
37
|
+
## 2. The concepts, in one paragraph each
|
|
38
|
+
|
|
39
|
+
**Release source** — the one place K asks *your* product two questions:
|
|
40
|
+
"what should this install be on?" (`checkForUpdate`) and "give me exactly
|
|
41
|
+
this version" (`fetchRelease`). K holds **no versioning policy of its own** —
|
|
42
|
+
what your streams are called ("stable", "nightly", "lts-2024"), which version
|
|
43
|
+
counts as newest, whether you use semver or dates, and long-term pinning all
|
|
44
|
+
live inside your source. A ready-made `staticManifestSource({ baseUrl })` covers
|
|
45
|
+
the common case (static host, semver, no automatic downgrade) as *one policy*,
|
|
46
|
+
not as a rule of the framework.
|
|
47
|
+
|
|
48
|
+
**Two slots: `stable` and `experiment`** — K never overwrites your only copy.
|
|
49
|
+
The running, trusted version sits in the *stable* slot. A new version is
|
|
50
|
+
downloaded into the *experiment* slot and run **as a trial**. Only after it
|
|
51
|
+
proves itself is it *promoted* to stable; if anything fails, K *rolls back*
|
|
52
|
+
to the untouched stable copy. Think blue/green deployment, on one machine.
|
|
53
|
+
|
|
54
|
+
**Journal** — before K does anything (download, stop, swap, promote), it
|
|
55
|
+
writes what it is *about* to do to an append-only log, then does it. If the
|
|
56
|
+
machine dies mid-upgrade, the next start replays the journal and either
|
|
57
|
+
finishes the job or rolls back — decided by the log, not by guesswork. This
|
|
58
|
+
is why "kill it at any moment" is a test we run, not a fear.
|
|
59
|
+
|
|
60
|
+
**HostAdapter** — the small interface *you* implement so K can drive *your*
|
|
61
|
+
service without knowing anything about it: pause your workloads
|
|
62
|
+
(`quiesce`), stop/start the service, report health from the live process,
|
|
63
|
+
resume workloads. It's the entire integration surface — K core contains
|
|
64
|
+
zero concepts from any particular app.
|
|
65
|
+
|
|
66
|
+
**Predicates (proof of upgrade)** — instead of trusting a version string, K
|
|
67
|
+
checks two facts and calls the upgrade done only when both hold:
|
|
68
|
+
`binary_at_target` ("the *live process* — same PID that answered — reports
|
|
69
|
+
the new version") and `host_lifecycle_converged` ("OS-level state like
|
|
70
|
+
launch-at-login was written AND read back consistent from its one true
|
|
71
|
+
source"). Metadata like version fields or channel names is *banned* as
|
|
72
|
+
evidence — it has been wrong in the wild.
|
|
73
|
+
|
|
74
|
+
**Policy** — who decides an upgrade happens: `auto` (just do it),
|
|
75
|
+
`confirm` (ask the owner first), `notify-only` (tell, don't act). On
|
|
76
|
+
personal devices the owner always wins; even server-pushed upgrades pass
|
|
77
|
+
this gate.
|
|
78
|
+
|
|
79
|
+
**Install ownership** — if your binary was installed by something else (an
|
|
80
|
+
OS package manager, or a parent service that injects its own copy), that
|
|
81
|
+
manager owns upgrades. K detects this and refuses to self-upgrade a managed
|
|
82
|
+
copy — returning a typed `held: managed-elsewhere` instead of silently
|
|
83
|
+
creating a version mismatch.
|
|
84
|
+
|
|
85
|
+
## 3. Adoption: two process models, plus capabilities you opt into
|
|
86
|
+
|
|
87
|
+
A profile is a **process model**, and the model is defined by one number:
|
|
88
|
+
**how many live incarnations K itself manages.**
|
|
89
|
+
|
|
90
|
+
| Profile | K-managed live processes | Who hands over | Examples |
|
|
91
|
+
|---|---|---|---|
|
|
92
|
+
| **`swap`** | **0** | nobody — new bytes take effect on the next start | a one-shot CLI, `rustup`, **and a long-running interactive session like Claude Code** |
|
|
93
|
+
| **`service`** | **1** (briefly 0 mid-handover) | K stops the old, starts the new, and proves it | a resident daemon, Raft Computer |
|
|
94
|
+
|
|
95
|
+
That a quick CLI and an hours-long agent session share a profile is surprising
|
|
96
|
+
at first and correct on reflection: **neither has a process K hands over.**
|
|
97
|
+
Several old-version processes may keep running in the `swap` model — normal,
|
|
98
|
+
and invisible to K.
|
|
99
|
+
|
|
100
|
+
There is **no third model**. OS lifecycle convergence
|
|
101
|
+
and fleet drive are **capabilities** you opt into on top of `service`; bundling
|
|
102
|
+
them into a "profile" confused *what your app does* with *what K does*, and
|
|
103
|
+
what your app does is none of K's business.
|
|
104
|
+
|
|
105
|
+
```ts
|
|
106
|
+
// a service that also wants its sessions preserved and its OS lifecycle proven
|
|
107
|
+
createUpgrader({ host, source, policy: "auto", /* ... */ });
|
|
108
|
+
// capabilities are declared by implementing the corresponding host duties:
|
|
109
|
+
// named readback surfaces -> lifecycle-convergence
|
|
110
|
+
// attach the drive module -> fleet-drive
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
## 3.5 Responsibility boundary: what K guarantees vs what you must
|
|
114
|
+
|
|
115
|
+
K guarantees **mechanical** properties. It cannot guarantee your
|
|
116
|
+
application's **semantic** compatibility across versions — and being clear
|
|
117
|
+
about that line is part of the contract.
|
|
118
|
+
|
|
119
|
+
| K guarantees (mechanically, with teeth) | You must guarantee (K can't see it) |
|
|
120
|
+
|---|---|
|
|
121
|
+
| the transition itself: never two incarnations live, never an unbootable host, crash at any step recovers | that version N+1 can *read* what version N wrote (your data, DB schema, caches) |
|
|
122
|
+
| the artifact is byte-complete (sha256 + size) — **authenticity is NOT checked; see §Trust** | that N+1 speaks a protocol your server still accepts (and N does too, if you may roll back) |
|
|
123
|
+
| the *binary* is restorable — rollback returns the exact bytes that were running | that rolling the binary back is *meaningful* — **K restores your binary, not your data**. If N+1 migrated the user's database, rolling back to N leaves N facing N+1-shaped data |
|
|
124
|
+
| proof the new version is actually live and OS lifecycle converged | what `quiesce` must park durably, and what `resume` must bring back |
|
|
125
|
+
| the owner's consent policy is honored | whether this upgrade is *safe to offer* at all (feature flags, in-flight work, licence state) |
|
|
126
|
+
|
|
127
|
+
**The sharpest case is rollback**, and it is the same trap as downgrade:
|
|
128
|
+
a rollback that restores the binary while leaving forward-migrated data
|
|
129
|
+
behind is not a rollback, it's a new failure. K refuses to pretend
|
|
130
|
+
otherwise — which is why it gives you a place to say so:
|
|
131
|
+
|
|
132
|
+
### Declare it, and K enforces it for you
|
|
133
|
+
|
|
134
|
+
Rather than leaving compatibility as a documentation promise, declare it —
|
|
135
|
+
K turns your declaration into a mechanical gate:
|
|
136
|
+
|
|
137
|
+
```ts
|
|
138
|
+
class MyHost implements HostAdapter {
|
|
139
|
+
// Optional. Called BEFORE staging and BEFORE promote.
|
|
140
|
+
// Return a refusal string to stop the transition; null to allow.
|
|
141
|
+
async checkCompatibility(from: string, to: string): Promise<string | null> {
|
|
142
|
+
if (schemaGeneration(to) > schemaGeneration(from) && !hasDownMigration(to, from)) {
|
|
143
|
+
return `no down-migration from schema ${to} to ${from}`;
|
|
144
|
+
}
|
|
145
|
+
return null;
|
|
146
|
+
}
|
|
147
|
+
}
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
- Refusing at **stage** time means the upgrade never starts (typed
|
|
151
|
+
`held: incompatible`).
|
|
152
|
+
- Refusing at **promote** time means K rolls back instead of committing.
|
|
153
|
+
- Not implementing it is allowed — then compatibility is entirely your
|
|
154
|
+
out-of-band responsibility, and K says so in `status --json`
|
|
155
|
+
(`compatibility: "undeclared"`), so nobody mistakes silence for a
|
|
156
|
+
guarantee.
|
|
157
|
+
|
|
158
|
+
### Invariants are shipped, not hidden
|
|
159
|
+
|
|
160
|
+
K's guarantees exist as an **exported invariant library** (`core/src/invariants.ts`),
|
|
161
|
+
not as private test assertions. One definition, three consumers: K's own
|
|
162
|
+
teeth, the deterministic simulator (checked after *every* effect, on every
|
|
163
|
+
seed), and **your** tests — plus any app invariants you write in the same
|
|
164
|
+
shape:
|
|
165
|
+
|
|
166
|
+
```ts
|
|
167
|
+
import { BUILT_IN_INVARIANTS, checkInvariants, type Invariant } from "@botiverse/k-carrier";
|
|
168
|
+
|
|
169
|
+
const myAppInvariant: Invariant = {
|
|
170
|
+
id: "myapp.no-orphaned-jobs",
|
|
171
|
+
description: "no job is left claimed by a dead worker",
|
|
172
|
+
check: (s) => (orphanCount(s) > 0 ? `${orphanCount(s)} orphaned jobs` : null),
|
|
173
|
+
};
|
|
174
|
+
|
|
175
|
+
const violations = checkInvariants(snapshot, [...BUILT_IN_INVARIANTS, myAppInvariant]);
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
An invariant is a pure predicate over an observable snapshot (the same
|
|
179
|
+
shape `status --json` emits), so the identical check runs in-process, in
|
|
180
|
+
simulation, and black-box against a real binary. Violations return a
|
|
181
|
+
*reason*, so a failure explains itself even when a simulator replays it
|
|
182
|
+
from a seed hours later. Your invariants ride the simulator's seeded fault
|
|
183
|
+
injection for free — that is the practical answer to "who guarantees my
|
|
184
|
+
semantics": **you state them, K's machinery exercises them.**
|
|
185
|
+
|
|
186
|
+
Rule of thumb: **K owns the mechanics of the transition; you own the meaning
|
|
187
|
+
of the versions.** Where you can express the meaning as a predicate, hand it
|
|
188
|
+
to K and it becomes enforced rather than hoped for.
|
|
189
|
+
|
|
190
|
+
## 4. The boundary in one picture
|
|
191
|
+
|
|
192
|
+
```
|
|
193
|
+
YOUR APP | K CORE
|
|
194
|
+
|
|
|
195
|
+
daemon ──┐ | ┌─ artifact (download/resume/verify/swap)
|
|
196
|
+
CLI `myapp self upgrade` ──┤ construct| ├─ (no signature client — see §Trust)
|
|
197
|
+
install script ──┘ the same | ├─ txn (two-slot + journal + state machine)
|
|
198
|
+
┌────────────┐ | ├─ lifecycle (handoff orchestration)
|
|
199
|
+
│ Upgrader │──┼──►├─ converge (predicates + readback)
|
|
200
|
+
└────────────┘ | ├─ policy (consent/notify gating)
|
|
201
|
+
your HostAdapter ◄────────────────────┼───┤
|
|
202
|
+
your notificationSink ◄───────────────┼───┤
|
|
203
|
+
your onProgress ◄─────────────────────┼───┘ (calls back into your code only)
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
One rule regardless of profile: **every entrypoint constructs the same
|
|
207
|
+
Upgrader.** Your daemon's auto-update loop, your CLI subcommand, your
|
|
208
|
+
install script — same object, same path. This kills the bug class where one
|
|
209
|
+
entrypoint upgrades correctly and another silently doesn't.
|
|
210
|
+
|
|
211
|
+
## 4.5 Showing progress
|
|
212
|
+
|
|
213
|
+
Pass `onProgress` and K reports where an upgrade is:
|
|
214
|
+
|
|
215
|
+
```ts
|
|
216
|
+
createUpgrader({
|
|
217
|
+
...,
|
|
218
|
+
onProgress: (p) => {
|
|
219
|
+
// p.stage: checking | downloading | verifying | staging
|
|
220
|
+
// | handing-over | probing | promoted | rolled-back
|
|
221
|
+
// p.downloaded / p.total: bytes, present during `downloading` only
|
|
222
|
+
render(p);
|
|
223
|
+
},
|
|
224
|
+
});
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Three things worth knowing before you draw a bar with it:
|
|
228
|
+
|
|
229
|
+
- **Only `downloading` has a denominator.** Every other stage reports a
|
|
230
|
+
stage and nothing else, because K does not know how long staging or
|
|
231
|
+
probing will take and will not invent a number.
|
|
232
|
+
- **`downloaded` counts bytes on disk, not bytes fetched this attempt.** A
|
|
233
|
+
resumed download starts at the size of the partial file. That is deliberate:
|
|
234
|
+
a bar that restarts from zero after a network blip reads as "it lost my
|
|
235
|
+
download".
|
|
236
|
+
- **Your sink cannot fail the upgrade.** K calls it inside a `try`/`catch`
|
|
237
|
+
and discards anything it throws. An observation surface must never become
|
|
238
|
+
a failure mode — if your renderer breaks, the upgrade still completes.
|
|
239
|
+
|
|
240
|
+
The stages are not a parallel state machine: they are derived from the L1
|
|
241
|
+
transaction phases (`stageForPhase`), so a progress display can never show a
|
|
242
|
+
state the transaction does not have.
|
|
243
|
+
|
|
244
|
+
## 5. Publishing releases
|
|
245
|
+
|
|
246
|
+
If you use the built-in `staticManifestSource`, its layout is:
|
|
247
|
+
|
|
248
|
+
```
|
|
249
|
+
<baseUrl>/manifest.json version, per-target {file, sha256, size}
|
|
250
|
+
<baseUrl>/<artifact> the binaries
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
Root private keys stay offline; root public keys are compiled into your app.
|
|
254
|
+
Any static file host works — there is no server-side logic.
|
|
255
|
+
|
|
256
|
+
**This layout belongs to that source, not to K.** Publishing from a private
|
|
257
|
+
API, date-stamped paths, or an OCI registry means writing your own
|
|
258
|
+
`ReleaseSource`; K only ever learns `{ version, url, sha256, size }` and never
|
|
259
|
+
parses a manifest itself. Multiple streams are usually one base URL each
|
|
260
|
+
(`.../stable`, `.../nightly`), which also keeps their blast radius separate.
|
|
261
|
+
|
|
262
|
+
### Trust: what K checks, and what it does not
|
|
263
|
+
|
|
264
|
+
**K verifies integrity, not authenticity.** It checks `sha256` + `size` on the
|
|
265
|
+
assembled bytes. It does **not** verify who produced them: there is no
|
|
266
|
+
signature chain and no trust root (removed 2026-08-06 — `docs/design-v1.md`
|
|
267
|
+
§L0.5 has the decision).
|
|
268
|
+
|
|
269
|
+
A digest is not a signature. `sha256` proves the bytes you received are the
|
|
270
|
+
bytes the manifest described — but the manifest comes from the same place the
|
|
271
|
+
bytes do, so a source serving malicious bytes will serve a matching digest for
|
|
272
|
+
them just as happily.
|
|
273
|
+
|
|
274
|
+
⚠️ **So this is yours to think about, not K's:**
|
|
275
|
+
|
|
276
|
+
| Threat | Covered by K? |
|
|
277
|
+
|--------|---------------|
|
|
278
|
+
| corruption in transit | ✅ (and your HTTPS already covers it) |
|
|
279
|
+
| a wrong artifact on your CDN — leaked publish credentials, misconfigured bucket, poisoned pipeline | ❌ **not covered** — the check passes and every client installs |
|
|
280
|
+
|
|
281
|
+
**OS code signing is a different guarantee, not a substitute.** Authenticode /
|
|
282
|
+
codesign / notarization answer "is this program signed by a recognisable
|
|
283
|
+
vendor", enforced by the OS on the install paths it controls. A distribution
|
|
284
|
+
signature answers "**is this the exact artifact we published**", enforced by
|
|
285
|
+
your app before the bytes reach a slot. If you ship through an app store or a
|
|
286
|
+
platform installer you get some of the former for free; if you ship a plain
|
|
287
|
+
binary from a CDN, as the example host does, you get neither automatically.
|
|
288
|
+
|
|
289
|
+
If you need authenticity today, do it in your own `ReleaseSource`: verify
|
|
290
|
+
before returning the `Release`, and refuse rather than return unverified bytes.
|
|
291
|
+
⚠️ And if you build it, remember the trap this project already hit: **"accept
|
|
292
|
+
unsigned" may only be declared by YOUR code, never by a field in the manifest**
|
|
293
|
+
— the manifest is served by the very party a signature chain exists to distrust.
|
|
294
|
+
|
|
295
|
+
## 6. Testing your integration
|
|
296
|
+
|
|
297
|
+
Three beliefs shape how K is tested — knowing them explains what the harness
|
|
298
|
+
will and won't do with your app (full design: `harness-design.md`):
|
|
299
|
+
|
|
300
|
+
1. **Test like a user.** The primary tests spawn your *real binary* and drive
|
|
301
|
+
it through its *CLI commands*, asserting from outside (exit codes, files,
|
|
302
|
+
what version actually runs next). Library-level tests are the exception,
|
|
303
|
+
not the rule — a green that only exists inside an import is not proof.
|
|
304
|
+
2. **The tests are the spec.** Every guarantee K claims (never dual-run,
|
|
305
|
+
never bricked, sessions survive rollback…) exists as a registered tooth
|
|
306
|
+
with a declared way to make it fail. A claim without a runnable red case
|
|
307
|
+
doesn't count — that includes profile support ("K supports CLIs" is
|
|
308
|
+
backed by a runnable example, not a sentence).
|
|
309
|
+
3. **No test backdoors.** K core contains zero test-awareness — no test
|
|
310
|
+
modes, no "skip verification" flags. Everything the harness uses is a
|
|
311
|
+
product surface you also get (status command, injected clock, config).
|
|
312
|
+
So passing the harness means the *shipping* code path works, not a
|
|
313
|
+
test-shaped variant of it.
|
|
314
|
+
|
|
315
|
+
Run the harness against **your** adapter, at your profile:
|
|
316
|
+
|
|
317
|
+
```
|
|
318
|
+
k-harness --profile service --adapter ./dist/myHost.js
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
Same teeth K tests itself with, tiered to your profile: crash-injection per
|
|
322
|
+
state-machine edge, quiesce/resume equivalence (including post-rollback),
|
|
323
|
+
probe liveness, predicate readback. Green here means your integration honors
|
|
324
|
+
the contract — it is the same bar the built-in examples must pass
|
|
325
|
+
(`examples/`: one runnable app per profile; a profile without a green
|
|
326
|
+
example has no support claim).
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Prior art: how k-carrier relates to Tailscale and Datadog updaters
|
|
2
|
+
|
|
3
|
+
This document is the design-rationale and license-defense record behind the
|
|
4
|
+
`NOTICE` file. k-carrier's architecture was informed by studying two mature,
|
|
5
|
+
unrelated self-updating systems. **Only architectural concepts were studied; no
|
|
6
|
+
source code was copied**, and the implementation here is original. This note
|
|
7
|
+
records what each project does, what k-carrier deliberately reuses, and where it
|
|
8
|
+
does something neither project attempts.
|
|
9
|
+
|
|
10
|
+
Both projects were read first-hand from their upstream `main` branches:
|
|
11
|
+
|
|
12
|
+
- **Tailscale** — `tailscale/tailscale`: `clientupdate/` (`clientupdate.go`,
|
|
13
|
+
`clientupdate_windows.go`, `clientupdate_downloads.go`) and
|
|
14
|
+
`clientupdate/distsign/` (`distsign.go`).
|
|
15
|
+
- **Datadog** — `DataDog/datadog-agent`: `pkg/fleet/installer/`
|
|
16
|
+
(`installer.go`, `oci/`, `packages/`, `db/`) and `pkg/fleet/daemon/`
|
|
17
|
+
(`daemon.go`, `remote_config.go`, `local_api.go`, `task_db.go`).
|
|
18
|
+
|
|
19
|
+
## Tailscale `clientupdate`
|
|
20
|
+
|
|
21
|
+
- **Single package, dual entry point.** One `clientupdate` package is shared by
|
|
22
|
+
the `tailscaled` daemon and the `tailscale` CLI — a single canonical executor
|
|
23
|
+
reached from two binaries.
|
|
24
|
+
- **Track model.** stable / unstable / release-candidate; the minor version's
|
|
25
|
+
parity selects the track; a `Confirm(newVersion)` callback gates the upgrade.
|
|
26
|
+
- **Platform matrix = "defer to whoever owns the install."** A dispatch table
|
|
27
|
+
routes apt / dnf-yum / apk / Synology / QNAP / FreeBSD pkg each to its native
|
|
28
|
+
package manager; macOS GUI builds go through Sparkle; the Mac App Store build
|
|
29
|
+
cannot self-update; Arch only prints guidance (it respects pacman's
|
|
30
|
+
ownership). Every branch carries a `canAutoUpdate` flag.
|
|
31
|
+
- **Bare-binary line (Linux).** require-root → resolve version → confirm →
|
|
32
|
+
download + verify a tarball → unpack over the install → restart via the
|
|
33
|
+
service manager.
|
|
34
|
+
- **Windows.** MSI plus a self-copy trick (copy the running executable to a temp
|
|
35
|
+
path and run the installer from there to avoid self-locking) + Authenticode
|
|
36
|
+
verification + a re-entry environment variable.
|
|
37
|
+
- **`distsign` signing (the strongest supply-chain idea).** A two-tier Ed25519
|
|
38
|
+
scheme: offline root keys are compiled into the client and sign rotating
|
|
39
|
+
signing keys, which in turn sign the distributed files; the server is just
|
|
40
|
+
static files (`$file` + `$file.sig`); signing keys are fetched dynamically
|
|
41
|
+
before each download; root rotation ships a new client.
|
|
42
|
+
|
|
43
|
+
**What Tailscale does *not* do** (and k-carrier does): the restart is
|
|
44
|
+
best-effort — on failure it prints "please restart manually" rather than
|
|
45
|
+
failing closed; there is no post-upgrade read-back (the restart command
|
|
46
|
+
returning is treated as success, with no probe of the new daemon's version); and
|
|
47
|
+
there is no rollback (the old binary is simply overwritten).
|
|
48
|
+
|
|
49
|
+
## Datadog fleet installer
|
|
50
|
+
|
|
51
|
+
- **A self-managing package manager.** It installs and uninstalls packages,
|
|
52
|
+
including the installer itself — the installer is one of the managed packages.
|
|
53
|
+
- **Two-slot upgrade transaction (the core idea k-carrier reuses).** Each
|
|
54
|
+
package keeps a `{Stable, Experiment}` pair of slots: `InstallExperiment`
|
|
55
|
+
stages the new version in the experiment slot (with pre/post start hooks), then
|
|
56
|
+
either `PromoteExperiment` (make it the new stable) or `RemoveExperiment`
|
|
57
|
+
(roll back to stable). **Upgrade is a blue/green transaction with rollback**,
|
|
58
|
+
and slot state is readable back. Configuration changes flow through the same
|
|
59
|
+
experiment / promote / rollback machinery.
|
|
60
|
+
- **Daemon drive plane.** A remote-config channel pushes a catalog and upgrade
|
|
61
|
+
tasks; the daemon executes them; a task database records the work; `GetState`
|
|
62
|
+
reads back `{Stable, Experiment}` per package — so a fleet's "who is on what
|
|
63
|
+
version / mid-experiment" is observable. The local API is a unix socket /
|
|
64
|
+
named pipe.
|
|
65
|
+
- **Distribution = OCI images**, content-addressed by digest (layers verified by
|
|
66
|
+
digest).
|
|
67
|
+
- It explicitly sequences self-teardown (on Linux a pre-stop step would kill its
|
|
68
|
+
own process, so it removes the experiment before stopping).
|
|
69
|
+
|
|
70
|
+
## Dimension-by-dimension
|
|
71
|
+
|
|
72
|
+
| Dimension | Tailscale | Datadog | k-carrier |
|
|
73
|
+
|---|---|---|---|
|
|
74
|
+
| Executor shape | single package, dual entry ✅ | standalone installer + daemon drive | one core, shared by multiple entry points — the Tailscale shape |
|
|
75
|
+
| Upgrade transaction / rollback | ❌ overwrite, no rollback | ★ two-slot + promote/rollback | fail-closed + rollback — the Datadog model |
|
|
76
|
+
| Post-upgrade read-back | ❌ (restart = success) | `GetState` per package | convergence read-back with a version probe — stricter predicate layer |
|
|
77
|
+
| Fleet observability | ❌ | ★ remote-config + task-db + state | server-push + read-back drive plane — the Datadog shape |
|
|
78
|
+
| Channels / pinning | track + confirm | catalog + remote config | latest / alpha / pinned channel file |
|
|
79
|
+
| Signing / supply chain | ★ two-tier offline-root Ed25519 | OCI digest addressing | **integrity-only today (sha256 + size); no signing implemented** — distsign studied as a future key layer |
|
|
80
|
+
| Service handoff | Windows self-copy trick | explicit self-kill sequencing | detached service handoff **with in-flight workload/session preservation** |
|
|
81
|
+
| OS-supervisor convergence | ❌ (native per-platform services) | ❌ | supervisor retirement + login-item migration + read-back |
|
|
82
|
+
| Generic framework | ❌ coupled to Tailscale | ❌ coupled to Datadog's package ecosystem | generic core + a thin host shell |
|
|
83
|
+
|
|
84
|
+
## The three concepts k-carrier reuses
|
|
85
|
+
|
|
86
|
+
1. **Core transaction — from Datadog's two slots.** k-carrier's upgrade is a
|
|
87
|
+
`stable` / `experiment` pair with promote / rollback, which naturally gives
|
|
88
|
+
fail-closed behavior, rollback, and read-back state.
|
|
89
|
+
2. **Executor shape — from Tailscale's single package.** One core reached from
|
|
90
|
+
several entry points (daemon / CLI / installer), rather than duplicated logic.
|
|
91
|
+
3. **Supply-chain layer — from Tailscale's `distsign`.** The two-tier
|
|
92
|
+
offline-root Ed25519 idea is the intended future key layer over the existing
|
|
93
|
+
content-addressed distribution. **It is a studied concept, not present in this
|
|
94
|
+
repository** — k-carrier currently verifies artifact integrity by sha256 +
|
|
95
|
+
size only.
|
|
96
|
+
|
|
97
|
+
## Why k-carrier exists (the gap)
|
|
98
|
+
|
|
99
|
+
The two projects sit at opposite ends of a spectrum. Tailscale is a system
|
|
100
|
+
service on mostly *personal* devices, so it is deferential to install ownership
|
|
101
|
+
(package-manager routing, confirm callbacks, Arch's print-only path). Datadog
|
|
102
|
+
manages *fleet* servers/nodes an organization operates, so it can push central
|
|
103
|
+
remote-config and read the fleet back (the machines have no "personal will").
|
|
104
|
+
|
|
105
|
+
k-carrier targets **a managed service running on a personal device**, which is
|
|
106
|
+
the *union*, not a midpoint: it wants Datadog's management capabilities
|
|
107
|
+
(transaction / rollback / read-back / remote drive / fleet observability) **and**
|
|
108
|
+
Tailscale's device-respect (consent / notification / ownership deference), plus
|
|
109
|
+
capabilities neither project has — **preserving the host's in-flight workload and
|
|
110
|
+
sessions across the swap, and an un-fakeable convergence read-back**. This union
|
|
111
|
+
is strictly harder than either project's problem, which is the cleanest
|
|
112
|
+
explanation for why no off-the-shelf framework fits: nobody has needed exactly
|
|
113
|
+
this superset. That gap is k-carrier's reason to exist as a generic core.
|
|
114
|
+
|
|
115
|
+
**Layer 0 is a commodity.** Fetching, verifying, and atomically swapping a
|
|
116
|
+
binary is already solved by self-update libraries (e.g. Rust `self_update`, Go
|
|
117
|
+
`selfupdate`), and by the updaters shipped with CLI agents — because a CLI has no
|
|
118
|
+
long-running service to hand off. k-carrier's Layer 0 looks like those and could
|
|
119
|
+
even sit on top of one; its value is entirely in the layers above (transaction,
|
|
120
|
+
handoff, read-back), which is exactly the list that separates a CLI self-updater
|
|
121
|
+
from a managed-service carrier.
|
|
122
|
+
|
|
123
|
+
## Licenses
|
|
124
|
+
|
|
125
|
+
Tailscale is **BSD-3-Clause** and the Datadog agent is **Apache-2.0** — both
|
|
126
|
+
permissive. k-carrier borrows concepts only and copies no code; the signing idea
|
|
127
|
+
is described from the concept and would be written independently if implemented.
|
|
128
|
+
k-carrier itself is released under **Apache-2.0** (see `LICENSE` and `NOTICE`).
|
|
129
|
+
|
|
130
|
+
## Test-design influences
|
|
131
|
+
|
|
132
|
+
- **Tailscale is unit-only, zero e2e.** Its tests are pure-function / file
|
|
133
|
+
operations (sources.list rewriting as table-driven byte-in/byte-out, repo
|
|
134
|
+
track edits, version parsing, tarball unpack, the confirm callback) plus key
|
|
135
|
+
rotation and a local-server download test. There are no upgrade-e2e, handoff,
|
|
136
|
+
crash, or rollback tests — **the test shape mirrors the design's gaps**: with
|
|
137
|
+
no rollback or read-back, nothing needs an e2e. k-carrier borrows the
|
|
138
|
+
table-driven pure-function style for its platform adapters.
|
|
139
|
+
- **Datadog has real-VM e2e.** e2e runs on freshly provisioned cloud VMs (real
|
|
140
|
+
systemd / package managers), with a `host.State()` full-snapshot assertion,
|
|
141
|
+
journald timestamp anchoring (`LastJournaldTimestamp()` → "assert X happened
|
|
142
|
+
*after* the marker", an event-shaped oracle that stops stale events from
|
|
143
|
+
passing as new evidence), ownership-migration scenarios, and failure suites.
|
|
144
|
+
It is slow and expensive. k-carrier borrows the timestamp-anchored assertion
|
|
145
|
+
and the ownership-migration scenario.
|
|
146
|
+
- **What neither has** (and k-carrier's harness adds): a crash-injection matrix,
|
|
147
|
+
adversarial self-verification, a mutation contract, a black-box zero-integration
|
|
148
|
+
mode, and a teeth registry with registration discipline. Tailscale's unit-only
|
|
149
|
+
suite restates the lesson: **the test shape follows the design — a design with
|
|
150
|
+
no rollback or read-back never demands tests for them.**
|