@twinsearth/w2m-dsh-plugin 0.6.1 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +91 -0
- package/README.md +17 -0
- package/TROUBLESHOOTING.md +29 -0
- package/bin/w2m-localside.mjs +5 -0
- package/package.json +1 -1
- package/src/agent/agent.mjs +91 -19
- package/src/agent/crypto.mjs +14 -0
- package/src/agent/p2p-node.mjs +96 -7
- package/src/agent/p2p.mjs +19 -7
- package/src/plugin/tools.mjs +1 -1
- package/src/relay/persistence.mjs +72 -2
- package/src/relay/server.mjs +10 -0
- package/src/relay/state.mjs +9 -2
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,97 @@ All notable changes to this project are documented here. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
|
|
5
5
|
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.6.2] — 2026-10-13
|
|
8
|
+
|
|
9
|
+
The release that stops a broken *relay* from being reported as an out-of-date *peer*, and stops a
|
|
10
|
+
relay that has quietly stopped recording from looking healthy.
|
|
11
|
+
|
|
12
|
+
The incident, measured on this fleet: every machine at v0.6.1 held a valid device key and announced
|
|
13
|
+
it, `peers_known` stayed `0` on all of them, no handshake was ever armed, and every direct channel
|
|
14
|
+
came back `P2P_CRYPTO_PLAINTEXT_PEER` — documented as *"the peer never learned a key (an older
|
|
15
|
+
build)"*. Both ends were the same build, on the same host. The component that was actually behind was
|
|
16
|
+
the relay: it started `2026-10-10T18:01:40Z`, and v0.6.0 and v0.6.1 shipped on `2026-10-11`. It had
|
|
17
|
+
never heard of `static_key`, so it accepted every announcement, dropped the field, and served entries
|
|
18
|
+
without it. The reason code blamed the one component that was up to date.
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
* **`encryption.relay_serves_keys`** — whether the relay keeps the key this machine announces. `null`
|
|
23
|
+
until an announcement proves it either way; `false` when the relay dropped what was sent; `null` for
|
|
24
|
+
a node with no key of its own, because such a node never asked and must not conclude "no". Detected
|
|
25
|
+
from this node's **own** announcement — it POSTs a key and the relay echoes the entry it stored — so
|
|
26
|
+
it needs no relay cooperation, no version negotiation and no capability call. That is what makes it
|
|
27
|
+
work against the relay that is actually deployed, which is the whole point.
|
|
28
|
+
* **`P2P_CRYPTO_RELAY_NO_KEYS`** — a new reason, used when a peer announced, this machine holds a key
|
|
29
|
+
of its own, no key came back for that peer, and this node has already proved the relay drops keys.
|
|
30
|
+
* **`/healthz` reports durability**: top-level `degraded` and `degraded_reason`, plus `degraded`,
|
|
31
|
+
`degraded_since`, `last_append_error` and `append_failures` inside `persistence`. `ok` deliberately
|
|
32
|
+
stays `true`: the relay still serves the fleet, and flipping it would turn a recording problem into
|
|
33
|
+
an outage. Persistence disabled is not degraded — there is nothing that can fail.
|
|
34
|
+
|
|
35
|
+
### Fixed
|
|
36
|
+
|
|
37
|
+
* **The reason a direct channel is plaintext now names the real cause.** `punch()` computed it as
|
|
38
|
+
`request === null ? PLAINTEXT_PEER : NO_KEY`, and `dial()` sends no crypto request in three
|
|
39
|
+
situations that are all *local* — this machine has no device key, it holds no key for that peer, or
|
|
40
|
+
it is dialling itself. Every one of them was reported as the peer being old. The code already
|
|
41
|
+
disagreed with itself about this: `dial()`'s own comment claims the reason "then says
|
|
42
|
+
`P2P_CRYPTO_NO_KEY`". The reason is now decided where it is known and travels with the call;
|
|
43
|
+
`PLAINTEXT_PEER` keeps its real meaning and keeps firing for the one case that is genuinely about
|
|
44
|
+
the peer — it publishes no key, so there is nothing to encrypt with.
|
|
45
|
+
* **`dial()` learns the peer key it was throwing away.** The peer's `static_key` has ridden in the
|
|
46
|
+
relay's `/v1/peer/{id}` record since v0.6.0, and `dial()` was fetching that very record for the
|
|
47
|
+
candidate list and discarding the key. A dispatcher that reported `PLAINTEXT_PEER` against a
|
|
48
|
+
same-version peer could have encrypted; it had an empty store and never asked. Reporting alone would
|
|
49
|
+
have been a better-worded lie.
|
|
50
|
+
* **`device.key` is loaded from beside `device.json`, not from `--state`.** The plugin followed the
|
|
51
|
+
documented rule; the agent did not, on the stated grounds that "`identity.mjs` puts `device.json`
|
|
52
|
+
when the agent is run with `--state`" — which is false. `device.json` is fixed at
|
|
53
|
+
`<DSH_HOME>/xclient/device.json` by the identity contract and `--state` does not move it. Measured
|
|
54
|
+
consequence on one machine running both halves: two different keypairs under one `machine_id`, both
|
|
55
|
+
announcing to the one `/v1/peer/{machine_id}` slot, the relay's record alternating between the two
|
|
56
|
+
nodes' ports every ~8s, each announcement overwriting the other. A dispatcher reading that record
|
|
57
|
+
learned one node's key or the other's, never reliably the key of the node it was about to dial. The
|
|
58
|
+
fix takes the path the caller actually resolved — `bin/w2m-localside.mjs` already had it in
|
|
59
|
+
`identityPath` and simply did not pass it down — and deliberately does **not** re-resolve `DSH_HOME`
|
|
60
|
+
inside the agent, which would make a test or an embedder read and *create* a key in the real user's
|
|
61
|
+
home.
|
|
62
|
+
* **A key already generated in `--state` is adopted, not abandoned.** `DeviceKey.load` warns that
|
|
63
|
+
"silently generating a replacement key would make every peer that knows the old one refuse this
|
|
64
|
+
machine", so the canonical location may create and a legacy file may only be adopted. An adopted key
|
|
65
|
+
is logged, because the public key is what a peer authenticates this machine by.
|
|
66
|
+
* **A relay that serves without recording now says so.** On a full disk the relay answered every
|
|
67
|
+
request normally and appended nothing; the only trace was one warning per failed append, and this
|
|
68
|
+
fleet's relay had reached **22 byte-identical `ENOSPC` entries**, so the signal was buried in its own
|
|
69
|
+
repetition. Identical warnings are now counted rather than repeated, and durability is a first-class
|
|
70
|
+
state: entered by the first failed append, cleared by the next success. `append_failures` is
|
|
71
|
+
cumulative and deliberately never reset — "how much was lost" is the first question after a disk
|
|
72
|
+
fills, and a counter that returns to zero cannot answer it.
|
|
73
|
+
* **A patch override that asserts the wrong `name:` now has a documented failure mode.** In a patch,
|
|
74
|
+
`name` is an *assertion* about the entry being targeted, not a rename, and the Loader drops the whole
|
|
75
|
+
patch on mismatch — leaving the plugin mounted on the bundle's empty defaults: eight tools present,
|
|
76
|
+
`w2m_status` reading `paired: false` and a null `rabbitUrl`, with the profile's settings sitting in
|
|
77
|
+
the file the whole time, unread. README now carries the hazard where the config block is copied,
|
|
78
|
+
`TROUBLESHOOTING.md` §2 names it as the third mechanism for "the patch lost my configuration" (and
|
|
79
|
+
the only one where the file looks intact), and §17 documents the reason codes this release adds and
|
|
80
|
+
corrects.
|
|
81
|
+
* **`test/agent.test.mjs`'s sub-path CLI test no longer depends on the host's `PATH`.** This is what
|
|
82
|
+
made v0.6.1's own pre-flight **RED** — "do not bump" — on a machine where node exists only as DSH's
|
|
83
|
+
bundled runtime. `runCli` points the child's `DSH_HOME` at a scratch directory, so
|
|
84
|
+
`bundledBinDirs()` correctly finds no bundled runtime there; the command under test is built from
|
|
85
|
+
`process.execPath` while the allow-list entry is the bare name `node -e`; and with no bundled runtime
|
|
86
|
+
and no `node` on `PATH`, that name does not resolve, so the absolute path is refused with
|
|
87
|
+
`COMMAND_EXECUTABLE_PATH_MISMATCH` — the allow-list behaving exactly as documented. CI runs
|
|
88
|
+
`actions/setup-node`, which is why it was green there. The child now gets the executing node's own
|
|
89
|
+
directory on its `PATH`, which restores the identity the check exists to confirm rather than
|
|
90
|
+
weakening it: a mismatched absolute path is still refused with the same code.
|
|
91
|
+
|
|
92
|
+
### Changed
|
|
93
|
+
|
|
94
|
+
* `scripts/preflight.mjs` reports **11 pass, 0 fail, 1 unverifiable — GREEN, cleared to bump** on a
|
|
95
|
+
clean macOS checkout. Before this release the same gate reported "10 pass, 1 fail — RED".
|
|
96
|
+
* The local suite reads **557 tests, 555 pass, 0 fail**.
|
|
97
|
+
|
|
7
98
|
## [0.6.1] — 2026-10-11
|
|
8
99
|
|
|
9
100
|
### Fixed
|
package/README.md
CHANGED
|
@@ -283,6 +283,23 @@ Then give it the relay URL in that profile's `cordis.patch.yml` — or leave
|
|
|
283
283
|
p2pMode: auto
|
|
284
284
|
```
|
|
285
285
|
|
|
286
|
+
> **Do not add a `name:` line to that row.** In a patch, `name` is not a rename — it is an
|
|
287
|
+
> *assertion* about the entry being patched, and the Loader skips the whole patch when it does not
|
|
288
|
+
> match. The row is already inserted by the bundle with the package name, so an override that
|
|
289
|
+
> asserts something else is discarded, and the plugin mounts with the bundle's **empty** defaults:
|
|
290
|
+
>
|
|
291
|
+
> ```
|
|
292
|
+
> patch: name mismatch for "w2m" (expected "@twinsearth/w2m-dsh-plugin",
|
|
293
|
+
> got "@twinsearth/w2m-dsh-plugin/tools"), skipping
|
|
294
|
+
> ```
|
|
295
|
+
>
|
|
296
|
+
> That is a silent failure by construction — the install looks right, eight tools appear, and
|
|
297
|
+
> `w2m_status` reports `paired: false` and a null `rabbitUrl` because none of your values were ever
|
|
298
|
+
> applied. It is not hypothetical: this row's expected name moved from the `/tools` subpath to the
|
|
299
|
+
> package root in v0.5.4 (see `cordis.patch.yml`'s header for why), and every override that still
|
|
300
|
+
> asserted the old subpath stopped being applied on upgrade. `id` already identifies the row, so the
|
|
301
|
+
> safest `name:` is no `name:` at all.
|
|
302
|
+
|
|
286
303
|
Restart DSH. You should see eight tools: `w2m_devices`, `w2m_run`, `w2m_wait`,
|
|
287
304
|
`w2m_report`, `w2m_status`, `w2m_update`, `w2m_history`, `w2m_stats`.
|
|
288
305
|
|
package/TROUBLESHOOTING.md
CHANGED
|
@@ -104,6 +104,29 @@ YAML, and the loader fails on the file as a whole — one bad row takes the enti
|
|
|
104
104
|
just this plugin. Both cases are why this file is treated as hand-edited state, not as a generated
|
|
105
105
|
artifact.
|
|
106
106
|
|
|
107
|
+
A third mechanism, and the only one where the file looks *perfect* and is still being ignored: **a
|
|
108
|
+
`name:` line in your override is an assertion, not a rename.** The bundle inserts this row itself with
|
|
109
|
+
the package name, so your row targets it by `id` and a `name:` only has to agree. When it does not, the
|
|
110
|
+
Loader drops the whole patch:
|
|
111
|
+
|
|
112
|
+
```
|
|
113
|
+
patch: name mismatch for "w2m" (expected "@twinsearth/w2m-dsh-plugin",
|
|
114
|
+
got "@twinsearth/w2m-dsh-plugin/tools"), skipping
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
Every value in that block is then absent — not overridden with something else, simply never applied —
|
|
118
|
+
and the plugin mounts on the bundle's empty defaults. This is the failure mode where `cat` shows your
|
|
119
|
+
settings sitting right there and `w2m_status` reports `paired: false` anyway.
|
|
120
|
+
|
|
121
|
+
It is also the one that arrives **on upgrade** rather than on a crash: the entry's expected name moved
|
|
122
|
+
from the `/tools` subpath to the package root in v0.5.4 (`cordis.patch.yml`'s header records why), and
|
|
123
|
+
every override still asserting the old subpath stopped being applied the moment the profile was
|
|
124
|
+
upgraded — with no edit to the profile and no error. If you keep a `name:`, an upgrade can silently
|
|
125
|
+
disconnect you from your own configuration.
|
|
126
|
+
|
|
127
|
+
**The fix is to delete the line.** `id: w2m` already identifies the row; asserting the name buys
|
|
128
|
+
nothing and turns any future rename by the bundle into a silent misconfiguration.
|
|
129
|
+
|
|
107
130
|
**Decision procedure.**
|
|
108
131
|
|
|
109
132
|
```bash
|
|
@@ -113,6 +136,9 @@ cat "$DSH_HOME/profiles/<profile>/cordis.patch.yml"
|
|
|
113
136
|
|
|
114
137
|
- First characters are `[]` → the placeholder case; see the safe-edit rule below.
|
|
115
138
|
- The `w2m` row is absent → it was dropped; re-add it (§2 fix 3).
|
|
139
|
+
- The row carries a `name:` line → **delete that line first**, then re-check the values. Everything
|
|
140
|
+
else in this section assumes the patch is being applied at all, and a mismatched `name:` means it
|
|
141
|
+
is not.
|
|
116
142
|
- The row is present but values are empty → it was reset; re-apply them.
|
|
117
143
|
|
|
118
144
|
```bash
|
|
@@ -827,6 +853,9 @@ elsewhere. `<kind>` is the sub-directory a program is started with: `localside`
|
|
|
827
853
|
| `LEASE_EXPIRED` | task status | §4/§5 — the executor stopped renewing its lease |
|
|
828
854
|
| `P2P_UNAVAILABLE` | `refused` result in `p2pMode: direct` | §6 — the punch did not land and direct-only refuses the fallback |
|
|
829
855
|
| `P2P_PUNCH_TIMEOUT` | reported transport reason | §6, §13 |
|
|
856
|
+
| `P2P_CRYPTO_RELAY_NO_KEYS` | `p2p.encryption.reason` on a direct channel that came back plaintext | **the relay, not the peer.** This machine holds a key and the peer announced one, yet the relay served no key for it — this node has proved, from its own announcement, that the relay drops the `static_key` field. Almost always a relay older than v0.6.0: upgrade and restart it. `w2m_status` shows the same fact as `encryption.relay_serves_keys: false`. Replaced a misleading `PLAINTEXT_PEER` in v0.6.2 |
|
|
857
|
+
| `P2P_CRYPTO_PLAINTEXT_PEER` | `p2p.encryption.reason` | the peer genuinely publishes no key, so there is nothing to encrypt with. Check the peer's version before assuming it is merely misconfigured — and note that this code used to be reported for local causes too, which is why `P2P_CRYPTO_NO_KEY` and `..._RELAY_NO_KEYS` now exist |
|
|
858
|
+
| `P2P_CRYPTO_NO_KEY` | `p2p.encryption.reason` | **this** machine's gap, not the peer's: no device key of its own, no key for that peer, or a self-dial (a machine keeps no key for itself). `device.key` lives beside `device.json`, not in `--state` |
|
|
830
859
|
| `RUNNER_THREW` | the **auto-updater**'s spawn result (`w2m_update`), not an envelope status | the updater's own runner threw before or instead of running: read the message beside it, and check the install path. An envelope never carries this — `src/agent/exec.mjs` defines only `ok`, `nonzero_exit`, `timeout`, `crashed`, `refused` and `unverifiable` |
|
|
831
860
|
| `SIGNATURE_*` (`MISMATCH`, `EXPIRED`, `REPLAY`, `REQUIRED`, `INCOMPLETE`) | relay, when signing is configured | the signing secret is rotated or the two sides disagree; `SIGNING_NOT_CONFIGURED` means signing is off |
|
|
832
861
|
|
package/bin/w2m-localside.mjs
CHANGED
|
@@ -230,6 +230,11 @@ async function main() {
|
|
|
230
230
|
project,
|
|
231
231
|
stateDir,
|
|
232
232
|
identity,
|
|
233
|
+
// v0.6.2: the device key belongs beside the `device.json` this process actually loaded, and this
|
|
234
|
+
// is the only place that knows where that is. Without it the agent fell back to `--state`, a
|
|
235
|
+
// different directory — so on one machine the plugin and the agent each minted their own keypair
|
|
236
|
+
// and then overwrote each other's announcement under the one shared machine_id.
|
|
237
|
+
identityPath,
|
|
233
238
|
caps,
|
|
234
239
|
platform,
|
|
235
240
|
allowedCommands,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@twinsearth/w2m-dsh-plugin",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.2",
|
|
4
4
|
"description": "DSH: Windows2MacOS — drive the same project on online Windows and macOS machines from one DeepSeek account, across networks and regions. P2P hole punching is the default path (relay + shared server as rendezvous, ledger and fallback), with a bundled STUN responder, Tailscale/public-VPS/tunnel shapes, Rabbit relay + per-machine Localside + eight DSH tools (replicate / split), nine aggregate verdicts and signed requests.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
package/src/agent/agent.mjs
CHANGED
|
@@ -23,8 +23,8 @@
|
|
|
23
23
|
|
|
24
24
|
import { Readable } from 'node:stream';
|
|
25
25
|
import { access, constants as fsConstants } from 'node:fs/promises';
|
|
26
|
-
import { mkdirSync, renameSync, unlinkSync, writeFileSync } from 'node:fs';
|
|
27
|
-
import { join } from 'node:path';
|
|
26
|
+
import { existsSync, mkdirSync, renameSync, unlinkSync, writeFileSync } from 'node:fs';
|
|
27
|
+
import { dirname, join } from 'node:path';
|
|
28
28
|
import { randomUUID } from 'node:crypto';
|
|
29
29
|
|
|
30
30
|
import {
|
|
@@ -49,7 +49,7 @@ import {
|
|
|
49
49
|
} from './git.mjs';
|
|
50
50
|
import { createSpool } from './spool.mjs';
|
|
51
51
|
import { matchAllowedCommand, parseAllowedCommands } from './allowed-commands.mjs';
|
|
52
|
-
import { CRYPTO_REASON, DeviceKey, PeerKeyStore } from './crypto.mjs';
|
|
52
|
+
import { CRYPTO_REASON, DEVICE_KEY_FILE, DeviceKey, PeerKeyStore } from './crypto.mjs';
|
|
53
53
|
import { DEFAULT_P2P_MODE, P2P_DEFAULTS, P2PNode, normalizeP2PMode, normalizeP2PPort } from './p2p-node.mjs';
|
|
54
54
|
import { punchSession, reverseDialSession } from './p2p.mjs';
|
|
55
55
|
import { createSigner } from '../signing.mjs';
|
|
@@ -988,6 +988,7 @@ export function createAgent(options) {
|
|
|
988
988
|
project,
|
|
989
989
|
stateDir,
|
|
990
990
|
identity,
|
|
991
|
+
identityPath = null,
|
|
991
992
|
caps,
|
|
992
993
|
platform,
|
|
993
994
|
once = false,
|
|
@@ -1093,11 +1094,17 @@ export function createAgent(options) {
|
|
|
1093
1094
|
// *dispatcher* publishes in an offer, and "which key will this machine present" must not depend on
|
|
1094
1095
|
// whether a particular code path happened to have run yet. `loadOrCreate` is the documented rule
|
|
1095
1096
|
// ("generated the first time a machine pairs"), and a failure is never fatal — see `deviceKeyError`.
|
|
1096
|
-
const deviceKeyOutcome = loadDeviceKey(options.deviceKey ?? null, stateDir, identity.machine_id);
|
|
1097
|
+
const deviceKeyOutcome = loadDeviceKey(options.deviceKey ?? null, stateDir, identity.machine_id, identityPath);
|
|
1097
1098
|
/** @type {object|null} This machine's static keypair, or null when there is none. */
|
|
1098
1099
|
const deviceKey = deviceKeyOutcome.key;
|
|
1099
1100
|
/** @type {string|null} Why there is no key, when there is none. Reported, never thrown. */
|
|
1100
1101
|
const deviceKeyError = deviceKeyOutcome.error;
|
|
1102
|
+
/**
|
|
1103
|
+
* A key adopted from the pre-v0.6.2 location (`--state/device.key`). Reported once at startup,
|
|
1104
|
+
* because a machine whose key moved is a machine whose public key a peer may have learned under the
|
|
1105
|
+
* old file — silence here would make "my peers stopped talking to me" have no visible cause.
|
|
1106
|
+
*/
|
|
1107
|
+
const deviceKeyMigratedFrom = deviceKeyOutcome.migratedFrom ?? null;
|
|
1101
1108
|
/**
|
|
1102
1109
|
* Peer static keys, learned from offers.
|
|
1103
1110
|
*
|
|
@@ -1359,6 +1366,15 @@ export function createAgent(options) {
|
|
|
1359
1366
|
log(result.ok ? 'info' : 'warn', `p2p ${p2pMode}: ${result.ok ? 'node started' : 'node did not start'}`, {
|
|
1360
1367
|
error: result.error ?? null,
|
|
1361
1368
|
});
|
|
1369
|
+
// v0.6.2: a key adopted from the pre-fix location. Said out loud because the public key is what
|
|
1370
|
+
// a peer authenticates this machine by, and a silent move would look like a peer going deaf.
|
|
1371
|
+
if (deviceKeyMigratedFrom !== null) {
|
|
1372
|
+
log('warn', 'p2p: adopting a device key from the old --state location', {
|
|
1373
|
+
from: deviceKeyMigratedFrom,
|
|
1374
|
+
public_key: deviceKey === null ? null : deviceKey.publicKeyBase64,
|
|
1375
|
+
note: 'v0.6.2 moved device.key beside device.json; delete the old file once every peer has seen the new announcement',
|
|
1376
|
+
});
|
|
1377
|
+
}
|
|
1362
1378
|
refreshP2PStatus();
|
|
1363
1379
|
return result;
|
|
1364
1380
|
})();
|
|
@@ -4067,32 +4083,88 @@ export function createAgent(options) {
|
|
|
4067
4083
|
* be read so an operator can fix it. Refusing to start an agent because a *security upgrade* could not
|
|
4068
4084
|
* complete is exactly the fleet-losing failure `docs/P2P-CRYPTO.md` §4 forbids.
|
|
4069
4085
|
*
|
|
4070
|
-
*
|
|
4071
|
-
*
|
|
4072
|
-
* `DeviceKey`
|
|
4073
|
-
*
|
|
4086
|
+
* WHERE THE KEY LIVES, AND THE BUG THAT CAME FROM GETTING IT WRONG (v0.6.2)
|
|
4087
|
+
*
|
|
4088
|
+
* `DeviceKey.load` documents the rule as "`device.key` **beside `device.json`**", and this function
|
|
4089
|
+
* used to break it: it loaded from the `--state` directory, justified by a comment claiming that
|
|
4090
|
+
* `identity.mjs` puts `device.json` there "when the agent is run with `--state`". That claim is false.
|
|
4091
|
+
* `device.json` is fixed at `<DSH_HOME>/xclient/device.json` by the identity contract and `--state`
|
|
4092
|
+
* does not move it — which is why an agent run as `--state ~/.dsh/xclient/localside-mac` had its
|
|
4093
|
+
* identity one directory *above* its key.
|
|
4094
|
+
*
|
|
4095
|
+
* The measured consequence, on one machine running both halves of W2M: the plugin (which loads the key
|
|
4096
|
+
* correctly, from `dirname(device.json)`) and the agent each generated a *different* pair, and both
|
|
4097
|
+
* announced to the one `/v1/peer/{machine_id}` slot — the relay's record alternated between the two
|
|
4098
|
+
* nodes' ports every few seconds, and the `static_key` a dispatcher read was one node's or the other's.
|
|
4099
|
+
* A dispatcher that learns the wrong key cannot arm a handshake, so every direct channel came back
|
|
4100
|
+
* `P2P_CRYPTO_PLAINTEXT_PEER` — reported as "the peer is an older build" when both ends were the same
|
|
4101
|
+
* build on the same host.
|
|
4074
4102
|
*
|
|
4075
|
-
*
|
|
4076
|
-
*
|
|
4103
|
+
* MIGRATION. `DeviceKey.load` warns that "silently generating a replacement key would make every peer
|
|
4104
|
+
* that knows the old one refuse this machine". So a key already generated in the old location is
|
|
4105
|
+
* adopted rather than abandoned: canonical first, then a copy forward from `--state`, then generate.
|
|
4106
|
+
*
|
|
4107
|
+
* @param {object|null} injected A `DeviceKey`, or null to load or generate one.
|
|
4108
|
+
* @param {string} stateDir The `--state` directory; the key's home only when `deviceFile` is absent.
|
|
4077
4109
|
* @param {string} machineId
|
|
4078
|
-
* @
|
|
4110
|
+
* @param {string|null} [deviceFile] Absolute path of the `device.json` this agent loaded. The key
|
|
4111
|
+
* belongs beside it; pass the path the caller resolved rather than re-resolving `DSH_HOME` here.
|
|
4112
|
+
* @returns {{key: object|null, error: string|null, migratedFrom: string|null}}
|
|
4079
4113
|
*/
|
|
4080
|
-
function loadDeviceKey(injected, stateDir, machineId) {
|
|
4114
|
+
function loadDeviceKey(injected, stateDir, machineId, deviceFile = undefined) {
|
|
4081
4115
|
if (injected !== null && injected !== undefined) {
|
|
4082
4116
|
if (typeof injected.sharedSecret !== 'function' || typeof injected.publicKeyBase64 !== 'string') {
|
|
4083
|
-
return { key: null, error: 'P2P_CRYPTO_NO_KEY: the injected deviceKey is not a DeviceKey' };
|
|
4117
|
+
return { key: null, error: 'P2P_CRYPTO_NO_KEY: the injected deviceKey is not a DeviceKey', migratedFrom: null };
|
|
4084
4118
|
}
|
|
4085
|
-
return { key: injected, error: null };
|
|
4119
|
+
return { key: injected, error: null, migratedFrom: null };
|
|
4086
4120
|
}
|
|
4087
|
-
|
|
4088
|
-
|
|
4089
|
-
|
|
4090
|
-
|
|
4121
|
+
|
|
4122
|
+
// Beside `device.json` — the documented rule. `identityPath` is the path the caller actually
|
|
4123
|
+
// loaded, never a fresh global resolution: resolving `DSH_HOME` here would make a test (or any
|
|
4124
|
+
// caller with an injected identity) read and *create* a key in the real user's home.
|
|
4125
|
+
let canonicalDir = null;
|
|
4126
|
+
if (typeof deviceFile === 'string' && deviceFile !== '') canonicalDir = dirname(deviceFile);
|
|
4127
|
+
|
|
4128
|
+
const legacyPath = join(stateDir, DEVICE_KEY_FILE);
|
|
4129
|
+
|
|
4130
|
+
/**
|
|
4131
|
+
* Candidates in order. `mayCreate` is the whole subtlety: only a location that is *the* documented
|
|
4132
|
+
* home may mint a key. A legacy directory is consulted to adopt a key that is already there, never
|
|
4133
|
+
* to create one — `DeviceKey.load` warns that generating a replacement "would make every peer that
|
|
4134
|
+
* knows the old one refuse this machine", so an upgrade must not silently change this machine's
|
|
4135
|
+
* public key.
|
|
4136
|
+
*
|
|
4137
|
+
* With no `identityPath` the caller injected its own identity and there is no canonical home, so
|
|
4138
|
+
* `--state` keeps the pre-v0.6.2 behaviour of being able to create. That is what stops a test or an
|
|
4139
|
+
* embedder from reading and writing the real user's `~/.dsh/xclient`.
|
|
4140
|
+
*/
|
|
4141
|
+
const candidates =
|
|
4142
|
+
canonicalDir === null
|
|
4143
|
+
? [{ dir: stateDir, mayCreate: true, from: null }]
|
|
4144
|
+
: [
|
|
4145
|
+
{ dir: canonicalDir, mayCreate: true, from: null },
|
|
4146
|
+
...(existsSync(legacyPath) ? [{ dir: stateDir, mayCreate: false, from: legacyPath }] : []),
|
|
4147
|
+
];
|
|
4148
|
+
|
|
4149
|
+
let lastError = null;
|
|
4150
|
+
for (const candidate of candidates) {
|
|
4151
|
+
try {
|
|
4152
|
+
const loaded = DeviceKey.load({ dir: candidate.dir, machineId });
|
|
4153
|
+
if (candidate.mayCreate === false && loaded.created) continue;
|
|
4154
|
+
return { key: loaded.key, error: null, migratedFrom: candidate.from };
|
|
4155
|
+
} catch (error) {
|
|
4156
|
+
lastError = error;
|
|
4157
|
+
}
|
|
4158
|
+
}
|
|
4159
|
+
|
|
4160
|
+
if (lastError !== null) {
|
|
4091
4161
|
return {
|
|
4092
4162
|
key: null,
|
|
4093
|
-
error: `P2P_CRYPTO_KEY_UNREADABLE: ${
|
|
4163
|
+
error: `P2P_CRYPTO_KEY_UNREADABLE: ${lastError instanceof Error ? lastError.message : String(lastError)}`,
|
|
4164
|
+
migratedFrom: null,
|
|
4094
4165
|
};
|
|
4095
4166
|
}
|
|
4167
|
+
return { key: null, error: null, migratedFrom: null };
|
|
4096
4168
|
}
|
|
4097
4169
|
|
|
4098
4170
|
/**
|
package/src/agent/crypto.mjs
CHANGED
|
@@ -209,6 +209,20 @@ export const CRYPTO_REASON = Object.freeze({
|
|
|
209
209
|
MALFORMED: 'P2P_CRYPTO_MALFORMED',
|
|
210
210
|
/** The peer never learned a key (an older build), so a direct channel with it is plaintext. */
|
|
211
211
|
PLAINTEXT_PEER: 'P2P_CRYPTO_PLAINTEXT_PEER',
|
|
212
|
+
/**
|
|
213
|
+
* The relay is not serving peer keys, so no handshake can be armed by anyone (v0.6.2).
|
|
214
|
+
*
|
|
215
|
+
* Distinct from {@link PLAINTEXT_PEER} on purpose, and the distinction is the whole reason this code
|
|
216
|
+
* exists. A relay older than v0.6.0 has never heard of `static_key`: it accepts an announcement,
|
|
217
|
+
* silently drops the field, and serves an entry without it. Every peer then looks exactly like a
|
|
218
|
+
* machine that has no key — which is how a stale *relay* came to be reported for hours as "the peer
|
|
219
|
+
* is an older build", pointing the investigation at the one component that was up to date.
|
|
220
|
+
*
|
|
221
|
+
* Detected from this node's own announcement: we POST a key, so the entry the relay echoes back says
|
|
222
|
+
* whether it kept it. No relay cooperation and no version negotiation required, which is what makes
|
|
223
|
+
* it work against the relay that is actually deployed.
|
|
224
|
+
*/
|
|
225
|
+
RELAY_NO_KEYS: 'P2P_CRYPTO_RELAY_NO_KEYS',
|
|
212
226
|
/**
|
|
213
227
|
* A `REFUSE` datagram was **dropped**, not honoured: it named a session this punch did not agree.
|
|
214
228
|
*
|
package/src/agent/p2p-node.mjs
CHANGED
|
@@ -509,6 +509,14 @@ export class P2PNode extends EventEmitter {
|
|
|
509
509
|
this.deviceKey = deviceKey ?? null;
|
|
510
510
|
/** @type {PeerKeyStore} Peer static keys, learned from offers and from the relay. */
|
|
511
511
|
this.peerKeys = peerKeys instanceof PeerKeyStore ? peerKeys : new PeerKeyStore();
|
|
512
|
+
/**
|
|
513
|
+
* Does the relay keep the static key this machine announces? `null` until an announcement proves
|
|
514
|
+
* it either way. See the note in `performAnnounce` — a relay older than v0.6.0 drops the field
|
|
515
|
+
* silently, and that is otherwise indistinguishable from every peer having no key.
|
|
516
|
+
*
|
|
517
|
+
* @type {boolean|null}
|
|
518
|
+
*/
|
|
519
|
+
this.relayServesKeys = null;
|
|
512
520
|
/** @type {object|null} `CryptoSession` overrides; null means the documented thresholds. */
|
|
513
521
|
this.cryptoTuning = cryptoTuning ?? null;
|
|
514
522
|
/** @type {NonceCache} Replay tracking for inbound handshakes. Outlives one `accept()`. */
|
|
@@ -654,6 +662,12 @@ export class P2PNode extends EventEmitter {
|
|
|
654
662
|
enabled: this.deviceKey !== null,
|
|
655
663
|
public_key: this.deviceKey === null ? null : (this.deviceKey.publicKeyBase64 ?? null),
|
|
656
664
|
peers_known: this.peerKeys.size,
|
|
665
|
+
/**
|
|
666
|
+
* `null` until this node's own announcement proves it either way; `false` means the relay
|
|
667
|
+
* dropped the key we sent, which is the fact that explains an otherwise mysterious
|
|
668
|
+
* `peers_known: 0` on a fleet whose machines all hold keys (v0.6.2).
|
|
669
|
+
*/
|
|
670
|
+
relay_serves_keys: this.relayServesKeys,
|
|
657
671
|
handshakes: this.cryptoCounts.handshakes,
|
|
658
672
|
encrypted_channels: this.cryptoCounts.encrypted_channels,
|
|
659
673
|
plaintext_channels: this.cryptoCounts.plaintext_channels,
|
|
@@ -951,6 +965,28 @@ export class P2PNode extends EventEmitter {
|
|
|
951
965
|
const posted = response?.ok === true;
|
|
952
966
|
if (!posted && postError === null) postError = describeAnnounceFailure(response);
|
|
953
967
|
|
|
968
|
+
/**
|
|
969
|
+
* Does the relay keep the static key we send it? (v0.6.2)
|
|
970
|
+
*
|
|
971
|
+
* This is knowable from our own announcement, and only from our own announcement: we POST
|
|
972
|
+
* `static_key`, and the relay echoes the entry it stored. A relay that predates v0.6.0 has never
|
|
973
|
+
* heard of the field, drops it silently, and echoes an entry without it — indistinguishable, from
|
|
974
|
+
* any peer's point of view, from a machine that has no key at all.
|
|
975
|
+
*
|
|
976
|
+
* That ambiguity is what turned a stale relay into hours of "the peer is an older build". The
|
|
977
|
+
* deployed relay in this fleet started 2026-10-10T18:01:40Z; v0.6.0 and v0.6.1 shipped on
|
|
978
|
+
* 2026-10-11. Every announcement carried a valid key, every stored entry lost it, no dispatcher
|
|
979
|
+
* could ever arm a handshake, and `PLAINTEXT_PEER` blamed the one component that was up to date.
|
|
980
|
+
*
|
|
981
|
+
* Recorded rather than reported here: one announce proves the relay's behaviour, and `dial()`
|
|
982
|
+
* turns it into a named reason only where it explains something. `null` means "not yet known" —
|
|
983
|
+
* a machine with no key of its own cannot ask the question at all, and must not conclude "no".
|
|
984
|
+
*/
|
|
985
|
+
if (posted && this.deviceKey !== null) {
|
|
986
|
+
const echoed = response?.json?.static_key;
|
|
987
|
+
this.relayServesKeys = typeof echoed === 'string' && echoed !== '' ? true : false;
|
|
988
|
+
}
|
|
989
|
+
|
|
954
990
|
// A POST failure is the headline when there is one: the announcement is not live at
|
|
955
991
|
// all. Otherwise a discovery failure is what makes this announcement incomplete.
|
|
956
992
|
const error = !posted ? postError : discoveryOk ? null : discoveryError;
|
|
@@ -1142,6 +1178,9 @@ export class P2PNode extends EventEmitter {
|
|
|
1142
1178
|
const fromOffer = Array.isArray(options.candidates) && options.candidates.length > 0;
|
|
1143
1179
|
|
|
1144
1180
|
let announced = [];
|
|
1181
|
+
// v0.6.2: the peer's static key as published in the same record the candidates came from. Lifted
|
|
1182
|
+
// out of the lookup block because `lookup` is scoped to it, and the key is needed further down.
|
|
1183
|
+
let announcedStaticKey = null;
|
|
1145
1184
|
if (fromOffer) {
|
|
1146
1185
|
announced = options.candidates;
|
|
1147
1186
|
} else {
|
|
@@ -1166,6 +1205,8 @@ export class P2PNode extends EventEmitter {
|
|
|
1166
1205
|
return fail(`P2P_LOOKUP_FAILED: ${describeLookupFailure(lookup)}`);
|
|
1167
1206
|
}
|
|
1168
1207
|
announced = Array.isArray(lookup.json?.peer?.candidates) ? lookup.json.peer.candidates : [];
|
|
1208
|
+
const key = lookup.json?.peer?.static_key;
|
|
1209
|
+
if (typeof key === 'string' && key !== '') announcedStaticKey = key;
|
|
1169
1210
|
}
|
|
1170
1211
|
|
|
1171
1212
|
const remote = [];
|
|
@@ -1176,6 +1217,52 @@ export class P2PNode extends EventEmitter {
|
|
|
1176
1217
|
remote.push(usable);
|
|
1177
1218
|
if (remote.length >= this.tuning.maxCandidates) break;
|
|
1178
1219
|
}
|
|
1220
|
+
|
|
1221
|
+
/**
|
|
1222
|
+
* The peer's static key, learned from its own announcement when this machine does not hold one.
|
|
1223
|
+
*
|
|
1224
|
+
* v0.6.2. This is the other half of the plaintext-downgrade fix, and without it the reason code
|
|
1225
|
+
* alone would have been a better-worded lie: the dispatcher that reported
|
|
1226
|
+
* `P2P_CRYPTO_PLAINTEXT_PEER` against a same-version peer *could* have encrypted, it simply had an
|
|
1227
|
+
* empty key store and never asked. The relay has carried the peer's `static_key` in this very
|
|
1228
|
+
* record since v0.6.0 (`relay/peers.mjs`), the plugin already read it, and the agent's own `dial()`
|
|
1229
|
+
* was fetching the same record for the candidate list and then throwing the key away.
|
|
1230
|
+
*
|
|
1231
|
+
* Adopting it here is what makes the local/remote distinction possible as well as useful: a peer
|
|
1232
|
+
* that publishes no key is genuinely an older build and `PLAINTEXT_PEER` is honest; a peer that
|
|
1233
|
+
* publishes one we had not learned is *our* gap, and after this it is not a gap at all.
|
|
1234
|
+
*/
|
|
1235
|
+
let peerKey = this.peerKeys.get(machineId);
|
|
1236
|
+
if (peerKey === null && announcedStaticKey !== null) {
|
|
1237
|
+
this.peerKeys.set(machineId, announcedStaticKey, 'relay');
|
|
1238
|
+
peerKey = announcedStaticKey;
|
|
1239
|
+
}
|
|
1240
|
+
|
|
1241
|
+
/**
|
|
1242
|
+
* Why this dial cannot be encrypted, when it cannot. Named here because this is where the reason
|
|
1243
|
+
* is actually known: `punch()` sees only "no crypto request" and used to guess, blaming the peer
|
|
1244
|
+
* for what is usually a local condition.
|
|
1245
|
+
*
|
|
1246
|
+
* `null` means a handshake will be armed. `PLAINTEXT_PEER` is reserved for the one case that is
|
|
1247
|
+
* about the peer -- it publishes no key, so there is nothing to encrypt with. Everything else is
|
|
1248
|
+
* `NO_KEY`: this machine has no device key of its own, holds no key for that peer, or is dialling
|
|
1249
|
+
* itself (a machine keeps no key for itself, so its own `respondHandshake` would refuse it with
|
|
1250
|
+
* `P2P_CRYPTO_UNKNOWN_PEER`).
|
|
1251
|
+
*/
|
|
1252
|
+
const cryptoBlockedReason =
|
|
1253
|
+
this.deviceKey === null
|
|
1254
|
+
? CRYPTO_REASON.NO_KEY
|
|
1255
|
+
: machineId === this.machineId
|
|
1256
|
+
? CRYPTO_REASON.NO_KEY
|
|
1257
|
+
: peerKey === null
|
|
1258
|
+
? this.relayServesKeys === false
|
|
1259
|
+
// The peer announced, we hold a key of our own, and no key came back for it. If this
|
|
1260
|
+
// node has already proved the relay drops keys, the relay is the cause and saying
|
|
1261
|
+
// otherwise sends the operator to the wrong machine.
|
|
1262
|
+
? CRYPTO_REASON.RELAY_NO_KEYS
|
|
1263
|
+
: CRYPTO_REASON.PLAINTEXT_PEER
|
|
1264
|
+
: null;
|
|
1265
|
+
|
|
1179
1266
|
if (remote.length === 0) {
|
|
1180
1267
|
// `peersAnnounced` still counts what was offered, not what survived validation: a caller
|
|
1181
1268
|
// reading "the punch failed" needs to see that there *was* a list, and that every entry in it
|
|
@@ -1210,17 +1297,19 @@ export class P2PNode extends EventEmitter {
|
|
|
1210
1297
|
// sends would be refused by its own `respondHandshake` (`P2P_CRYPTO_UNKNOWN_PEER`) and the
|
|
1211
1298
|
// task it dispatched to itself would be lost. `machineId !== this.machineId` is what makes a
|
|
1212
1299
|
// self-lease survive a security upgrade, which is the same rule as every other fallback here.
|
|
1213
|
-
...(
|
|
1214
|
-
|
|
1215
|
-
this.peerKeys.get(machineId) === null
|
|
1216
|
-
? {}
|
|
1217
|
-
: {
|
|
1300
|
+
...(cryptoBlockedReason === null && peerKey !== null
|
|
1301
|
+
? {
|
|
1218
1302
|
crypto: {
|
|
1219
1303
|
deviceKey: this.deviceKey,
|
|
1220
|
-
peerKey
|
|
1304
|
+
peerKey,
|
|
1221
1305
|
peerMachineId: machineId,
|
|
1222
1306
|
},
|
|
1223
|
-
}
|
|
1307
|
+
}
|
|
1308
|
+
: {}),
|
|
1309
|
+
// v0.6.2: the reason travels with the call instead of being guessed downstream. `punch()`
|
|
1310
|
+
// cannot tell a local "we hold no key" from the peer being an older build, and it used to
|
|
1311
|
+
// report the peer-blaming one for both.
|
|
1312
|
+
cryptoBlockedReason,
|
|
1224
1313
|
...(this.cryptoTuning === null ? {} : { cryptoTuning: this.cryptoTuning }),
|
|
1225
1314
|
});
|
|
1226
1315
|
} catch (error) {
|
package/src/agent/p2p.mjs
CHANGED
|
@@ -558,13 +558,25 @@ export async function punch(options) {
|
|
|
558
558
|
* the refusal MEDIUM-4 was about.
|
|
559
559
|
*/
|
|
560
560
|
const refusalKey = handshake === null ? null : handshake.refusalMacKey;
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
|
|
567
|
-
|
|
561
|
+
/**
|
|
562
|
+
* Why there is no handshake, when there is none. `null` means one is in flight.
|
|
563
|
+
*
|
|
564
|
+
* v0.6.2: `request === null` is *this end's* decision, not a fact about the peer. `P2PNode.dial`
|
|
565
|
+
* sends no crypto request in three situations and every one of them is local — this machine has no
|
|
566
|
+
* device key, it holds no key for that peer, or it is dialling itself (a machine keeps no key for
|
|
567
|
+
* itself, so its own `respondHandshake` would refuse it with `P2P_CRYPTO_UNKNOWN_PEER`).
|
|
568
|
+
*
|
|
569
|
+
* It used to report `PLAINTEXT_PEER` here, documented as "the peer never learned a key (an older
|
|
570
|
+
* build)". Measured consequence: a dispatcher with an empty peer-key store reported every direct
|
|
571
|
+
* channel as the peer being out of date, when both ends were the same build on the same host — an
|
|
572
|
+
* encryption downgrade recorded as normal compatibility. `dial()`'s own comment has always claimed
|
|
573
|
+
* the reason would be `NO_KEY`; this is that behaviour, and it is what the objective asks for: a
|
|
574
|
+
* punch that did not encrypt says whether the cause was ours or theirs.
|
|
575
|
+
*
|
|
576
|
+
* `PLAINTEXT_PEER` keeps its real meaning and its real sites: a handshake *was* armed and the peer
|
|
577
|
+
* answered in the clear, which is the only case where the peer is genuinely the reason.
|
|
578
|
+
*/
|
|
579
|
+
const noCryptoReason = handshake !== null ? null : (options.cryptoBlockedReason ?? CRYPTO_REASON.NO_KEY);
|
|
568
580
|
/** Whether this punch is trying to encrypt at all. */
|
|
569
581
|
const wantCrypto = handshake !== null;
|
|
570
582
|
|
package/src/plugin/tools.mjs
CHANGED
|
@@ -177,7 +177,7 @@ const MAX_STAGES = 16;
|
|
|
177
177
|
* `package.json` during `scripts/pack.mjs` rather than hand-maintained, and `W2M_PLUGIN_VERSION`
|
|
178
178
|
* overrides it at runtime for tests and packagers.
|
|
179
179
|
*/
|
|
180
|
-
const PLUGIN_VERSION = '0.6.
|
|
180
|
+
const PLUGIN_VERSION = '0.6.2';
|
|
181
181
|
|
|
182
182
|
/** How long any single HTTP call may take before it is aborted, in milliseconds. */
|
|
183
183
|
const DEFAULT_REQUEST_TIMEOUT_MS = 15_000;
|
|
@@ -36,6 +36,15 @@ export class Persistence {
|
|
|
36
36
|
this._now = now;
|
|
37
37
|
/** @type {{file:string, message:string, line?:number}[]} */
|
|
38
38
|
this.warnings = [];
|
|
39
|
+
/**
|
|
40
|
+
* Durability state (v0.6.2): whether the ledger is actually being written.
|
|
41
|
+
*
|
|
42
|
+
* `appendFailures` is cumulative and never reset, so "how much was lost" stays answerable after a
|
|
43
|
+
* disk fills and is freed again. `degradedSinceMs` is the live condition.
|
|
44
|
+
*/
|
|
45
|
+
this.appendFailures = 0;
|
|
46
|
+
this.lastAppendError = null;
|
|
47
|
+
this.degradedSinceMs = null;
|
|
39
48
|
this.devicesCorrupt = false;
|
|
40
49
|
this.devicesMissing = true;
|
|
41
50
|
this.revivedDevices = 0;
|
|
@@ -50,8 +59,61 @@ export class Persistence {
|
|
|
50
59
|
get operatorTokenFile() { return this.enabled ? path.join(this.dir, OPERATOR_TOKEN_FILE) : null; }
|
|
51
60
|
|
|
52
61
|
warn(file, message, extra = {}) {
|
|
53
|
-
|
|
54
|
-
|
|
62
|
+
// Deduplicated (v0.6.2). A full disk fails *every* append, and appending an identical entry each
|
|
63
|
+
// time buried the signal in its own repetition: this fleet's relay reached 22 byte-identical
|
|
64
|
+
// "ENOSPC" warnings and no way to ask "is this relay still recording?" without counting them. One
|
|
65
|
+
// entry that counts is what an operator can act on.
|
|
66
|
+
const existing = this.warnings.find((entry) => entry.file === file && entry.message === message);
|
|
67
|
+
if (existing !== undefined) {
|
|
68
|
+
existing.count = (existing.count ?? 1) + 1;
|
|
69
|
+
existing.last_at = rfc3339(this._now());
|
|
70
|
+
return existing;
|
|
71
|
+
}
|
|
72
|
+
const entry = { file, message, count: 1, ...extra };
|
|
73
|
+
this.warnings.push(entry);
|
|
74
|
+
return entry;
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
/**
|
|
78
|
+
* One ledger append failed: the relay is still serving, but it is no longer recording (v0.6.2).
|
|
79
|
+
*
|
|
80
|
+
* This is a first-class state rather than a log line because the ledger *is* the relay's durability
|
|
81
|
+
* story. Every machine's result, every lease, everything a later question would be answered from —
|
|
82
|
+
* none of it survives a restart once appends stop landing. A relay in this state accepts work
|
|
83
|
+
* exactly as before, which is precisely why it has to say so out loud.
|
|
84
|
+
*
|
|
85
|
+
* @param {string} message
|
|
86
|
+
* @returns {false}
|
|
87
|
+
*/
|
|
88
|
+
noteAppendFailure(message) {
|
|
89
|
+
this.appendFailures += 1;
|
|
90
|
+
this.lastAppendError = message;
|
|
91
|
+
if (this.degradedSinceMs === null) this.degradedSinceMs = this._now();
|
|
92
|
+
this.warn(LEDGER_FILE, `append failed: ${message}`);
|
|
93
|
+
return false;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
/**
|
|
97
|
+
* The ledger is being written again.
|
|
98
|
+
*
|
|
99
|
+
* The history is kept — `append_failures` is not reset — because "how much was lost" is the first
|
|
100
|
+
* question after a disk fills, and a counter that silently returns to zero cannot answer it.
|
|
101
|
+
*
|
|
102
|
+
* @returns {true}
|
|
103
|
+
*/
|
|
104
|
+
noteAppendSuccess() {
|
|
105
|
+
if (this.degradedSinceMs !== null) {
|
|
106
|
+
const forMs = Math.max(0, this._now() - this.degradedSinceMs);
|
|
107
|
+
this.degradedSinceMs = null;
|
|
108
|
+
this.lastAppendError = null;
|
|
109
|
+
this.warn(LEDGER_FILE, `append recovered after ${this.appendFailures} failure(s) over ${forMs}ms`);
|
|
110
|
+
}
|
|
111
|
+
return true;
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
/** Is the relay serving without recording? @returns {boolean} */
|
|
115
|
+
get degraded() {
|
|
116
|
+
return this.degradedSinceMs !== null;
|
|
55
117
|
}
|
|
56
118
|
|
|
57
119
|
ensureDir() {
|
|
@@ -199,6 +261,14 @@ export class Persistence {
|
|
|
199
261
|
devices_missing: this.devicesMissing,
|
|
200
262
|
ledger_lines_read: this.ledgerLinesRead,
|
|
201
263
|
ledger_lines_skipped: this.ledgerLinesSkipped,
|
|
264
|
+
/**
|
|
265
|
+
* v0.6.2. `degraded: true` means the relay is answering requests while failing to record them,
|
|
266
|
+
* which is the one relay failure that looks exactly like health from the outside.
|
|
267
|
+
*/
|
|
268
|
+
degraded: this.degraded,
|
|
269
|
+
degraded_since: this.degradedSinceMs === null ? null : rfc3339(this.degradedSinceMs),
|
|
270
|
+
last_append_error: this.lastAppendError,
|
|
271
|
+
append_failures: this.appendFailures,
|
|
202
272
|
warnings: this.warnings.map((w) => ({ ...w })),
|
|
203
273
|
};
|
|
204
274
|
}
|
package/src/relay/server.mjs
CHANGED
|
@@ -902,6 +902,16 @@ export class RelayServer {
|
|
|
902
902
|
// so this drops to 0 by itself when nobody is refreshing.
|
|
903
903
|
...this.peers.stats(),
|
|
904
904
|
ok: true,
|
|
905
|
+
/**
|
|
906
|
+
* v0.6.2: hoisted out of `persistence` so it cannot be missed by a reader that only looks at
|
|
907
|
+
* the top level. `degraded: true` means this relay is answering requests while failing to record
|
|
908
|
+
* them -- the one relay failure that presents as health. `ok` deliberately stays true: the relay
|
|
909
|
+
* still serves the fleet, and flipping it would turn a recording problem into an outage.
|
|
910
|
+
*/
|
|
911
|
+
degraded: this.persistence.enabled && this.persistence.degraded,
|
|
912
|
+
degraded_reason: this.persistence.degraded
|
|
913
|
+
? `ledger appends are failing (${this.persistence.lastAppendError ?? 'unknown error'}); results are served but not durably recorded`
|
|
914
|
+
: null,
|
|
905
915
|
protocol_version: PROTOCOL_VERSION,
|
|
906
916
|
rabbit_time: this.state.nowIso(),
|
|
907
917
|
uptime_ms: this.state.nowMs() - this._startedAtMs,
|
package/src/relay/state.mjs
CHANGED
|
@@ -470,9 +470,16 @@ export class RabbitState {
|
|
|
470
470
|
_ledger(entry) {
|
|
471
471
|
if (this._restoring || !this._persist) return false;
|
|
472
472
|
try {
|
|
473
|
-
|
|
473
|
+
const appended = this._persist.append(entry);
|
|
474
|
+
// v0.6.2: a successful append is what clears a degraded relay, so the outcome has to be
|
|
475
|
+
// reported back rather than assumed -- otherwise the state could only ever be entered.
|
|
476
|
+
if (appended) this._persist.noteAppendSuccess?.();
|
|
477
|
+
return appended;
|
|
474
478
|
} catch (err) {
|
|
475
|
-
|
|
479
|
+
// Recorded as a state, not merely as a line: this is the one relay failure that keeps serving,
|
|
480
|
+
// and so the one that is invisible from every other angle (v0.6.2).
|
|
481
|
+
if (typeof this._persist.noteAppendFailure === 'function') this._persist.noteAppendFailure(err.message);
|
|
482
|
+
else this._persist.warn?.('ledger.jsonl', `append failed: ${err.message}`);
|
|
476
483
|
return false;
|
|
477
484
|
}
|
|
478
485
|
}
|