@twinsearth/w2m-dsh-plugin 0.9.5 → 0.9.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +117 -0
- package/README.md +6 -6
- package/package.json +6 -5
- package/src/agent/agent.mjs +27 -0
- package/src/agent/memory.mjs +233 -0
- package/src/plugin/tools.mjs +409 -79
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,123 @@ All notable changes to this project are documented here. The format follows
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
|
|
5
5
|
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.9.7] — 2026-10-14
|
|
8
|
+
|
|
9
|
+
**`w2m_memory` gains the collective layer and the one write a machine may not make for itself.**
|
|
10
|
+
|
|
11
|
+
### `scope: "fleet"` — the collective memory
|
|
12
|
+
|
|
13
|
+
Reads the curated `fleet-memory.md` that §3.2 of the design puts on the archive host, beside the profile
|
|
14
|
+
that mounted this plugin. It returns the document **as text, with its section list**, and does not parse
|
|
15
|
+
it into facts — deliberately. Hand-written prose has no schema, so a parser would silently drop whatever
|
|
16
|
+
shape it did not recognise, and what it dropped would be exactly what a human judged worth writing down.
|
|
17
|
+
The section list is there so a reader can see what the collective memory covers before paying for all of
|
|
18
|
+
it.
|
|
19
|
+
|
|
20
|
+
An absent file is answered honestly rather than as an empty document: `MEMORY_NO_COLLECTIVE`, with the
|
|
21
|
+
path that was looked at. *"Nothing has been curated yet"* and *"the memory is empty"* are different
|
|
22
|
+
facts, and only one of them means someone should start writing.
|
|
23
|
+
|
|
24
|
+
### `record` — a conclusion, which only a reader may assert
|
|
25
|
+
|
|
26
|
+
`memoryRecord` always emits `conclusion: null`, because a machine that invented one would be asserting an
|
|
27
|
+
interpretation it cannot have. This is the path by which a reader supplies it: the conclusion is appended
|
|
28
|
+
as a **new record naming the task it is about**, never written into the existing line.
|
|
29
|
+
|
|
30
|
+
Append-only is the point. In-place editing would break the property that makes the log trustworthy — what
|
|
31
|
+
was written is what happened, and an amendment is a later statement *about* it — and it would destroy the
|
|
32
|
+
original text, which is the thing a reader most needs when a conclusion turns out to be wrong.
|
|
33
|
+
|
|
34
|
+
### A defect this found in its own first draft
|
|
35
|
+
|
|
36
|
+
`collectivePath()` was `path.join(resolveProfileDir(cfg), COLLECTIVE_FILE)`. `resolveProfileDir`
|
|
37
|
+
legitimately returns `null` — a bare test context, or a host whose layout it cannot read — and the first
|
|
38
|
+
version passed that straight into `path.join`, which threw `ERR_INVALID_ARG_TYPE`. **A read tool that
|
|
39
|
+
crashes on a missing directory is worse than one that says the directory is missing:** the caller was
|
|
40
|
+
asking a question, and an exception is not an answer. It now resolves to `null`, reports
|
|
41
|
+
`MEMORY_NO_COLLECTIVE`, and says which of the two reasons applies.
|
|
42
|
+
|
|
43
|
+
### Verified
|
|
44
|
+
|
|
45
|
+
- `eslint .` — 0 errors.
|
|
46
|
+
- The contract suites — `memory`, `tools`, `tool-cards`, `claims`, `loader-entry`, `update-wiring`,
|
|
47
|
+
`p2p-plugin` — **230 pass, 0 fail, 11 platform skips**.
|
|
48
|
+
- Every scope exercised against a real temporary profile: `fleet` found the seeded document and listed
|
|
49
|
+
its five sections; `record` wrote a conclusion against a task and reported that no task existed when
|
|
50
|
+
none did; an absent collective file produced `MEMORY_NO_COLLECTIVE` rather than a throw.
|
|
51
|
+
- The tool count is unchanged at nine, so this release moves no contract: the same nine tools, two of them
|
|
52
|
+
wider.
|
|
53
|
+
|
|
54
|
+
## [0.9.6] — 2026-10-14
|
|
55
|
+
|
|
56
|
+
**Individual memory: the fleet can now answer "what did we already try on this machine, and what happened?" — a question nothing in it could answer before.**
|
|
57
|
+
|
|
58
|
+
### The gap this closes, measured rather than asserted
|
|
59
|
+
|
|
60
|
+
The relay is sent `command_hash`, `stdout_sha256` and `stdout_bytes`, and never the text. That choice is
|
|
61
|
+
defensible and stays: the ledger is a tamper-evident consistency record, and a relay compromise discloses
|
|
62
|
+
no command text. The price was that no part of the fleet retained what a command actually did, and the
|
|
63
|
+
price was paid over and over. In one working session the same facts were rediscovered from scratch across
|
|
64
|
+
a dozen dispatches — that a machine had no package manager (3 times), which node binary actually ran (2),
|
|
65
|
+
that its git could not reach github.com while node's HTTPS could (2), that a task is expired at about
|
|
66
|
+
fifty seconds (3), and which commit its checkout was on (2).
|
|
67
|
+
|
|
68
|
+
### What was added
|
|
69
|
+
|
|
70
|
+
**`src/agent/memory.mjs` — one append-only `memory.jsonl` per machine, in its own state directory.**
|
|
71
|
+
|
|
72
|
+
Written by the agent as each task finishes, at the one point where the output still exists at all and on
|
|
73
|
+
the one machine that owns it. Each line carries the command, `cwd_rel`, exit code, status, duration,
|
|
74
|
+
transport, and the tails of stdout and stderr — plus the envelope's hashes, so a human-readable record can
|
|
75
|
+
be cross-checked against the relay's tamper-evident one without trusting either.
|
|
76
|
+
|
|
77
|
+
Three properties are deliberate and each has a test:
|
|
78
|
+
|
|
79
|
+
- **Bounded by construction**, with a byte cap and one rotating generation. This is not hypothetical
|
|
80
|
+
caution: the Linux host's root filesystem reached 100% today from 14.7 GB/day of uncapped logging, and
|
|
81
|
+
the recovery cost real time. A memory that can fill a disk is a worse outcome than no memory. Rotation
|
|
82
|
+
keeps the previous generation rather than discarding it, because the question this file answers is
|
|
83
|
+
usually about the recent past and the rotation boundary sits exactly there.
|
|
84
|
+
- **A malformed line is skipped, not fatal.** A partially written final line is what a machine losing
|
|
85
|
+
power mid-append leaves behind, and one damaged line must not make the rest unreadable — the relay's own
|
|
86
|
+
ledger behaves this way for the same reason.
|
|
87
|
+
- **`conclusion` is written as `null` by the machine and only ever filled in by a reader.** The design
|
|
88
|
+
separates fact from interpretation, and a machine that invented a conclusion would be asserting
|
|
89
|
+
something it cannot know. A test pins this, because the failure would be silent.
|
|
90
|
+
|
|
91
|
+
**`w2m_memory` — the read side.** Returns this machine's memory, newest first, with a status tally, and
|
|
92
|
+
reaches the rotated generation on request.
|
|
93
|
+
|
|
94
|
+
It does **not** read another machine's memory, and says so instead of returning an empty list. The reason
|
|
95
|
+
is structural rather than an omission: the relay carries no such file, and an agent executes a dispatched
|
|
96
|
+
command inside a sandbox whose writes are scoped. A query for another machine is answered with
|
|
97
|
+
`MEMORY_REMOTE_NOT_READABLE` and the two ways to actually read it. Returning `records: []` would have read
|
|
98
|
+
as *"that machine has done nothing"* — the opposite conclusion from *"this build cannot see"*.
|
|
99
|
+
|
|
100
|
+
The write is wrapped so it can never fail a task. The task is the work; this is a note about it.
|
|
101
|
+
|
|
102
|
+
### Verified
|
|
103
|
+
|
|
104
|
+
- `test/memory.test.mjs` — **18 tests, 0 failures**: record shape, tail and argv clamping, append-and-read,
|
|
105
|
+
newest-first ordering, the limit, a malformed line skipped, an absent file read as empty, rotation with
|
|
106
|
+
both generations readable, and that neither `appendMemory` nor `readMemorySync` throws on a bad directory.
|
|
107
|
+
- A rotation test that trips the cap with *real* records rather than filler, because the first version
|
|
108
|
+
wrote eight megabytes of `'x'` — it rotated, but it could not answer the question being asked, since the
|
|
109
|
+
previous generation then held one unparseable line.
|
|
110
|
+
- `eslint .` — 0 errors.
|
|
111
|
+
- **The full pre-flight is green: 11 pass, 0 fail, 1 unverifiable** (1721 tests in 54 files; 1707 pass,
|
|
112
|
+
0 fail, 14 platform skips). The single `UNVERIFIABLE` is an accepted host observation, not a gate.
|
|
113
|
+
Reaching that required installing `git` on the development host, which had none; the 39 failures before
|
|
114
|
+
it were all `spawn git ENOENT`, and the two rounds of diagnosis after it were a sandbox rule (writing
|
|
115
|
+
files in `%TEMP%` is allowed, **creating a directory** there is refused — and `git init` creates one) and
|
|
116
|
+
two mojibake sequences this release then repaired. Recorded because "39 failures" read as a defect count
|
|
117
|
+
and was, at every stage, an environment count.
|
|
118
|
+
- The nine-tool contract is updated everywhere it is pinned: `scripts/verify-installed.mjs`,
|
|
119
|
+
`test/loader-entry.test.mjs`, `test/tools.test.mjs`, `test/tool-cards.test.mjs`, `test/claims.test.mjs`,
|
|
120
|
+
`test/p2p-plugin.test.mjs`, `test/update-wiring.test.mjs` and the pre-flight gate inventory. Those
|
|
121
|
+
assertions are what make "the tool set" a contract rather than a habit, so leaving one at eight would
|
|
122
|
+
have failed a release that is otherwise correct.
|
|
123
|
+
|
|
7
124
|
## [0.9.5] — 2026-10-14
|
|
8
125
|
|
|
9
126
|
**Same code as v0.9.4; published because v0.9.4's npm and GitHub bytes cannot be made to agree.**
|
package/README.md
CHANGED
|
@@ -39,7 +39,7 @@ So the cross-machine capability has to be built. This is that build.
|
|
|
39
39
|
▼
|
|
40
40
|
┌──────────────────────────────────┐ ┌──────────────────────────┐
|
|
41
41
|
│ Windows · DSH session │ │ macOS · DSH session │
|
|
42
|
-
│ w2m plugin (
|
|
42
|
+
│ w2m plugin (9 tools) │ │ w2m plugin (9 tools) │
|
|
43
43
|
│ │ │ │ │ │
|
|
44
44
|
│ Localside agent │ │ Localside agent │
|
|
45
45
|
│ · three anchors │ │ · three anchors │
|
|
@@ -293,15 +293,15 @@ Then give it the relay URL in that profile's `cordis.patch.yml` — or leave
|
|
|
293
293
|
> got "@twinsearth/w2m-dsh-plugin/tools"), skipping
|
|
294
294
|
> ```
|
|
295
295
|
>
|
|
296
|
-
> That is a silent failure by construction — the install looks right,
|
|
296
|
+
> That is a silent failure by construction — the install looks right, nine tools appear, and
|
|
297
297
|
> `w2m_status` reports `paired: false` and a null `rabbitUrl` because none of your values were ever
|
|
298
298
|
> applied. It is not hypothetical: this row's expected name moved from the `/tools` subpath to the
|
|
299
299
|
> package root in v0.5.4 (see `cordis.patch.yml`'s header for why), and every override that still
|
|
300
300
|
> asserted the old subpath stopped being applied on upgrade. `id` already identifies the row, so the
|
|
301
301
|
> safest `name:` is no `name:` at all.
|
|
302
302
|
|
|
303
|
-
Restart DSH. You should see
|
|
304
|
-
`w2m_report`, `w2m_status`, `w2m_update`, `w2m_history`, `w2m_stats`.
|
|
303
|
+
Restart DSH. You should see nine tools: `w2m_devices`, `w2m_run`, `w2m_wait`,
|
|
304
|
+
`w2m_report`, `w2m_status`, `w2m_update`, `w2m_history`, `w2m_stats`, `w2m_memory`.
|
|
305
305
|
|
|
306
306
|
> `dsh` may not be on your `PATH`. On a packaged install it lives under
|
|
307
307
|
> `resources/runtime/cli/bin/`. If `dsh` is not found, call it by path.
|
|
@@ -584,7 +584,7 @@ a product.
|
|
|
584
584
|
|---|---|---|
|
|
585
585
|
| relay | 142 | pairing, auth, SSE with `ready`-first and `seq` replay, leases with heartbeat renewal and expiry, dedupe, all **nine** aggregation states plus `pending`/`refused`, report generation, operator token, rate limiting, persistence and restart recovery, TLS, and the lost-offer recovery path |
|
|
586
586
|
| agent | 129 | allow-list allow/deny, timeout, output truncation, exit codes, anchors on clean and dirty trees, four concurrent fingerprint computations, spool, a 40-case URL join matrix, cursor lifecycle across relay restarts, and the bundled-runtime resolver |
|
|
587
|
-
| tools | 89 |
|
|
587
|
+
| tools | 89 | nine tools registered, schemas, typed errors, polling, operator-token enforcement (no request is sent without it), sub-path endpoints, and the credential canaries |
|
|
588
588
|
| broadcast | 19 | exactly one executor, the rest `observing`; observers excluded from every aggregation step so a machine that did not run is never counted as agreeing |
|
|
589
589
|
| claims | 19 | the document claims themselves: the version, the bundle patch, the tool list, every documented tarball URL, and the CHANGELOG's shape |
|
|
590
590
|
| fleet-target | 37 | the composer's target selection, and the `executor_machine_id` narrowing that overrides it |
|
|
@@ -712,7 +712,7 @@ the plugin:
|
|
|
712
712
|
|
|
713
713
|
```bash
|
|
714
714
|
node scripts/check-entrypoints.mjs # imports lib/tools.js, the exact specifier cordis.patch.yml mounts
|
|
715
|
-
node scripts/verify-installed.mjs <path-to-installed-package> # asserts the installed copy registers
|
|
715
|
+
node scripts/verify-installed.mjs <path-to-installed-package> # asserts the installed copy registers 9 tools
|
|
716
716
|
```
|
|
717
717
|
|
|
718
718
|
**Not verified:**
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@twinsearth/w2m-dsh-plugin",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.7",
|
|
4
4
|
"description": "DSH: Windows2MacOS — drive the same project on online Windows and macOS machines from one DeepSeek account, across networks and regions. P2P hole punching is the default path (relay + shared server as rendezvous, ledger and fallback), with a bundled STUN responder, Tailscale/public-VPS/tunnel shapes, Rabbit relay + per-machine Localside + eight DSH tools (replicate / split), nine aggregate verdicts and signed requests.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
@@ -111,13 +111,13 @@
|
|
|
111
111
|
}
|
|
112
112
|
},
|
|
113
113
|
"scripts": {
|
|
114
|
-
"test": "node --test --test-force-exit test/relay.test.mjs test/broadcast.test.mjs test/agent.test.mjs test/tools.test.mjs test/claims.test.mjs test/fleet-target.test.mjs test/client-bundle.test.mjs test/schedule.test.mjs test/auto-update.test.mjs test/update-source.test.mjs test/update-install.test.mjs test/update-wiring.test.mjs test/signing.test.mjs test/stress.test.mjs test/shared-config.test.mjs test/metrics.test.mjs test/metrics-route.test.mjs test/tool-cards.test.mjs test/p2p-stun.test.mjs test/p2p-transport.test.mjs test/p2p-signaling.test.mjs test/p2p-node.test.mjs test/p2p-transport-fields.test.mjs test/p2p-plugin.test.mjs test/p2p-crypto.test.mjs test/stun-server.test.mjs test/loader-entry.test.mjs test/mdns-wire.test.mjs test/mdns-discover.test.mjs test/chatops-approval.test.mjs test/chatops-policy.test.mjs test/chatops-file-safety.test.mjs test/chatops-outbound-queue.test.mjs test/chatops-commands.test.mjs test/chatops-bridge.test.mjs test/chatops-config-route.test.mjs test/chatops-apply.test.mjs test/chatops-ilink-contract.test.mjs test/chatops-fleet.test.mjs test/chatops-account.test.mjs test/chatops-auth.test.mjs test/chatops-fleet-scope.test.mjs test/chatops-handoff.test.mjs test/chatops-fleet-wiring.test.mjs test/chatops-fleet-live.test.mjs test/chatops-takeover.test.mjs test/chatops-device-signing.test.mjs test/chatops-relay-verify.test.mjs test/chatops-durability.test.mjs test/chatops-run-command.test.mjs test/secrets-guard.test.mjs test/chatops-ilink-fleet-bridge.test.mjs test/chatops-fleet-join.test.mjs",
|
|
114
|
+
"test": "node --test --test-force-exit test/relay.test.mjs test/broadcast.test.mjs test/agent.test.mjs test/memory.test.mjs test/tools.test.mjs test/claims.test.mjs test/fleet-target.test.mjs test/client-bundle.test.mjs test/schedule.test.mjs test/auto-update.test.mjs test/update-source.test.mjs test/update-install.test.mjs test/update-wiring.test.mjs test/signing.test.mjs test/stress.test.mjs test/shared-config.test.mjs test/metrics.test.mjs test/metrics-route.test.mjs test/tool-cards.test.mjs test/p2p-stun.test.mjs test/p2p-transport.test.mjs test/p2p-signaling.test.mjs test/p2p-node.test.mjs test/p2p-transport-fields.test.mjs test/p2p-plugin.test.mjs test/p2p-crypto.test.mjs test/stun-server.test.mjs test/loader-entry.test.mjs test/mdns-wire.test.mjs test/mdns-discover.test.mjs test/chatops-approval.test.mjs test/chatops-policy.test.mjs test/chatops-file-safety.test.mjs test/chatops-outbound-queue.test.mjs test/chatops-commands.test.mjs test/chatops-bridge.test.mjs test/chatops-config-route.test.mjs test/chatops-apply.test.mjs test/chatops-ilink-contract.test.mjs test/chatops-fleet.test.mjs test/chatops-account.test.mjs test/chatops-auth.test.mjs test/chatops-fleet-scope.test.mjs test/chatops-handoff.test.mjs test/chatops-fleet-wiring.test.mjs test/chatops-fleet-live.test.mjs test/chatops-takeover.test.mjs test/chatops-device-signing.test.mjs test/chatops-relay-verify.test.mjs test/chatops-durability.test.mjs test/chatops-run-command.test.mjs test/secrets-guard.test.mjs test/chatops-ilink-fleet-bridge.test.mjs test/chatops-fleet-join.test.mjs",
|
|
115
115
|
"test:e2e": "node --test --test-force-exit test/e2e.test.mjs test/crossnetwork.test.mjs test/signing-e2e.test.mjs test/outbound-only.test.mjs test/recovery.test.mjs test/pipeline-e2e.test.mjs test/p2p-agent.test.mjs",
|
|
116
116
|
"test:relay": "node --test --test-force-exit test/relay.test.mjs",
|
|
117
|
-
"test:agent": "node --test --test-force-exit test/agent.test.mjs",
|
|
117
|
+
"test:agent": "node --test --test-force-exit test/agent.test.mjs test/memory.test.mjs",
|
|
118
118
|
"test:tools": "node --test --test-force-exit test/tools.test.mjs",
|
|
119
119
|
"test:crossnetwork": "node --test --test-force-exit test/crossnetwork.test.mjs",
|
|
120
|
-
"test:all": "node --test --test-force-exit test/relay.test.mjs test/broadcast.test.mjs test/agent.test.mjs test/tools.test.mjs test/claims.test.mjs test/fleet-target.test.mjs test/client-bundle.test.mjs test/schedule.test.mjs test/auto-update.test.mjs test/update-source.test.mjs test/update-install.test.mjs test/update-wiring.test.mjs test/signing.test.mjs test/stress.test.mjs test/e2e.test.mjs test/crossnetwork.test.mjs test/signing-e2e.test.mjs test/outbound-only.test.mjs test/recovery.test.mjs test/pipeline-e2e.test.mjs test/shared-config.test.mjs test/metrics.test.mjs test/metrics-route.test.mjs test/tool-cards.test.mjs test/p2p-stun.test.mjs test/p2p-transport.test.mjs test/p2p-signaling.test.mjs test/p2p-node.test.mjs test/p2p-transport-fields.test.mjs test/p2p-plugin.test.mjs test/p2p-agent.test.mjs test/stun-server.test.mjs test/loader-entry.test.mjs test/mdns-wire.test.mjs test/mdns-discover.test.mjs test/p2p-crypto.test.mjs test/chatops-approval.test.mjs test/chatops-policy.test.mjs test/chatops-file-safety.test.mjs test/chatops-outbound-queue.test.mjs test/chatops-commands.test.mjs test/chatops-bridge.test.mjs test/chatops-config-route.test.mjs test/chatops-apply.test.mjs test/chatops-ilink-contract.test.mjs test/chatops-fleet.test.mjs test/chatops-account.test.mjs test/chatops-auth.test.mjs test/chatops-fleet-scope.test.mjs test/chatops-handoff.test.mjs test/chatops-fleet-wiring.test.mjs test/chatops-fleet-live.test.mjs test/chatops-takeover.test.mjs test/chatops-device-signing.test.mjs test/chatops-relay-verify.test.mjs test/chatops-durability.test.mjs test/chatops-run-command.test.mjs test/secrets-guard.test.mjs test/chatops-ilink-fleet-bridge.test.mjs test/chatops-fleet-join.test.mjs",
|
|
120
|
+
"test:all": "node --test --test-force-exit test/relay.test.mjs test/broadcast.test.mjs test/agent.test.mjs test/memory.test.mjs test/tools.test.mjs test/claims.test.mjs test/fleet-target.test.mjs test/client-bundle.test.mjs test/schedule.test.mjs test/auto-update.test.mjs test/update-source.test.mjs test/update-install.test.mjs test/update-wiring.test.mjs test/signing.test.mjs test/stress.test.mjs test/e2e.test.mjs test/crossnetwork.test.mjs test/signing-e2e.test.mjs test/outbound-only.test.mjs test/recovery.test.mjs test/pipeline-e2e.test.mjs test/shared-config.test.mjs test/metrics.test.mjs test/metrics-route.test.mjs test/tool-cards.test.mjs test/p2p-stun.test.mjs test/p2p-transport.test.mjs test/p2p-signaling.test.mjs test/p2p-node.test.mjs test/p2p-transport-fields.test.mjs test/p2p-plugin.test.mjs test/p2p-agent.test.mjs test/stun-server.test.mjs test/loader-entry.test.mjs test/mdns-wire.test.mjs test/mdns-discover.test.mjs test/p2p-crypto.test.mjs test/chatops-approval.test.mjs test/chatops-policy.test.mjs test/chatops-file-safety.test.mjs test/chatops-outbound-queue.test.mjs test/chatops-commands.test.mjs test/chatops-bridge.test.mjs test/chatops-config-route.test.mjs test/chatops-apply.test.mjs test/chatops-ilink-contract.test.mjs test/chatops-fleet.test.mjs test/chatops-account.test.mjs test/chatops-auth.test.mjs test/chatops-fleet-scope.test.mjs test/chatops-handoff.test.mjs test/chatops-fleet-wiring.test.mjs test/chatops-fleet-live.test.mjs test/chatops-takeover.test.mjs test/chatops-device-signing.test.mjs test/chatops-relay-verify.test.mjs test/chatops-durability.test.mjs test/chatops-run-command.test.mjs test/secrets-guard.test.mjs test/chatops-ilink-fleet-bridge.test.mjs test/chatops-fleet-join.test.mjs",
|
|
121
121
|
"pack": "node scripts/pack.mjs",
|
|
122
122
|
"verify:join": "node scripts/verify-join.mjs",
|
|
123
123
|
"lint": "eslint .",
|
|
@@ -125,7 +125,8 @@
|
|
|
125
125
|
"format": "prettier --write .",
|
|
126
126
|
"format:check": "prettier --check .",
|
|
127
127
|
"verify": "node scripts/check-entrypoints.mjs && node --test --test-force-exit test/relay.test.mjs test/broadcast.test.mjs test/agent.test.mjs test/tools.test.mjs test/e2e.test.mjs test/crossnetwork.test.mjs test/p2p-node.test.mjs test/p2p-agent.test.mjs test/client-bundle.test.mjs test/loader-entry.test.mjs",
|
|
128
|
-
"test:p2p": "node --test --test-force-exit test/p2p-stun.test.mjs test/p2p-transport.test.mjs test/p2p-signaling.test.mjs test/p2p-node.test.mjs test/stun-server.test.mjs test/p2p-transport-fields.test.mjs test/p2p-plugin.test.mjs test/p2p-agent.test.mjs"
|
|
128
|
+
"test:p2p": "node --test --test-force-exit test/p2p-stun.test.mjs test/p2p-transport.test.mjs test/p2p-signaling.test.mjs test/p2p-node.test.mjs test/stun-server.test.mjs test/p2p-transport-fields.test.mjs test/p2p-plugin.test.mjs test/p2p-agent.test.mjs",
|
|
129
|
+
"test:memory": "node --test --test-force-exit test/memory.test.mjs"
|
|
129
130
|
},
|
|
130
131
|
"keywords": [
|
|
131
132
|
"dsh-bundle",
|
package/src/agent/agent.mjs
CHANGED
|
@@ -48,6 +48,7 @@ import {
|
|
|
48
48
|
resolveProjectCwd,
|
|
49
49
|
} from './git.mjs';
|
|
50
50
|
import { createSpool } from './spool.mjs';
|
|
51
|
+
import { appendMemory, memoryRecord } from './memory.mjs';
|
|
51
52
|
import { matchAllowedCommand, parseAllowedCommands } from './allowed-commands.mjs';
|
|
52
53
|
import { CRYPTO_REASON, DEVICE_KEY_FILE, DeviceKey, PeerKeyStore } from './crypto.mjs';
|
|
53
54
|
import { DEFAULT_P2P_MODE, P2P_DEFAULTS, P2PNode, normalizeP2PMode, normalizeP2PPort } from './p2p-node.mjs';
|
|
@@ -2638,6 +2639,32 @@ export function createAgent(options) {
|
|
|
2638
2639
|
const directResult = (context.p2pDelivery ?? p2pDeliveryMark()).path === 'p2p';
|
|
2639
2640
|
envelope.transport = transport.offer_path === 'p2p' || directResult ? 'p2p' : transport.offer_path;
|
|
2640
2641
|
envelope.p2p = envelopeP2P({ ...context, p2pTransport: transport });
|
|
2642
|
+
|
|
2643
|
+
/**
|
|
2644
|
+
* v0.9.6 individual memory: write what this machine just did, where the text still exists.
|
|
2645
|
+
*
|
|
2646
|
+
* This is the last point at which the output is available to anything that will keep it. The relay
|
|
2647
|
+
* is sent `stdout_sha256` and `stdout_bytes` and never the text -- deliberately, so that a relay
|
|
2648
|
+
* compromise discloses no command text and the ledger stays a consistency record rather than a
|
|
2649
|
+
* transcript. The consequence is that a question like "what did we already try on this machine, and
|
|
2650
|
+
* what happened?" had no answer anywhere in the fleet, and the same facts were rediscovered from
|
|
2651
|
+
* scratch repeatedly at the cost of a dozen dispatches in one session.
|
|
2652
|
+
*
|
|
2653
|
+
* The write happens **after** the envelope is complete, so the record it stores is the same one the
|
|
2654
|
+
* relay will judge -- not a second, independently assembled opinion about what happened.
|
|
2655
|
+
*
|
|
2656
|
+
* `appendMemory` never throws and its result is deliberately not returned or checked: the task is
|
|
2657
|
+
* the work and this is a note about it, so a full disk or an unwritable directory must not turn a
|
|
2658
|
+
* successful command into a failed task. A memory that can fail work is worse than no memory, and
|
|
2659
|
+
* the failure mode is silent here on purpose -- the envelope's own fields already say what happened.
|
|
2660
|
+
*/
|
|
2661
|
+
appendMemory(stateDir, memoryRecord({
|
|
2662
|
+
envelope,
|
|
2663
|
+
commandArgv,
|
|
2664
|
+
cwdRel,
|
|
2665
|
+
ms: (Date.parse(endedAt) - Date.parse(startedAt)) || 0,
|
|
2666
|
+
}));
|
|
2667
|
+
|
|
2641
2668
|
return buildEnvelope(envelope);
|
|
2642
2669
|
}
|
|
2643
2670
|
|
|
@@ -0,0 +1,233 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Individual memory: what this machine did, in its own state directory.
|
|
3
|
+
*
|
|
4
|
+
* ## Why this file exists
|
|
5
|
+
*
|
|
6
|
+
* The relay's ledger is the only durable fleet record, and it deliberately keeps **hashes, not text** --
|
|
7
|
+
* `command_hash`, `stdout_sha256`, `stdout_bytes`. That choice is defensible: the ledger is a
|
|
8
|
+
* tamper-evident consistency record, and a relay compromise discloses no command text. The cost is that
|
|
9
|
+
* nothing in the fleet can answer *"what did we already try on this machine, and what happened?"*
|
|
10
|
+
*
|
|
11
|
+
* The cost was paid repeatedly. In a single working session the same facts were rediscovered from
|
|
12
|
+
* scratch at the price of a dozen dispatches -- that this box has no package manager, which node binary
|
|
13
|
+
* actually runs, that its git cannot reach github.com while node's HTTPS can, that a task is expired at
|
|
14
|
+
* roughly fifty seconds. None of it was retained anywhere the fleet could read.
|
|
15
|
+
*
|
|
16
|
+
* So the memory lives here, on the machine, for three reasons beyond convenience:
|
|
17
|
+
*
|
|
18
|
+
* 1. the machine already writes `agent-state.json` in this directory, so there is no new place to
|
|
19
|
+
* find and no new permission to grant;
|
|
20
|
+
* 2. it owns the output at the moment it produces it, and this is the last point at which the *text*
|
|
21
|
+
* exists at all -- the relay never receives it, so nothing downstream could store it later;
|
|
22
|
+
* 3. it survives a relay rebuild, which the ledger does not.
|
|
23
|
+
*
|
|
24
|
+
* It is also the only honest home for a machine-specific fact. *"This box has no package manager"* is a
|
|
25
|
+
* property of this box, and a design that put it anywhere else would be guessing.
|
|
26
|
+
*
|
|
27
|
+
* ## Bounded by construction
|
|
28
|
+
*
|
|
29
|
+
* A byte cap with rotation, not an unbounded append. This is not hypothetical caution: the Linux host's
|
|
30
|
+
* root filesystem filled to 100% today from 14.7 GB/day of logs written by a process that had no cap,
|
|
31
|
+
* and the recovery cost real time. A memory that can fill a disk is a worse failure than no memory.
|
|
32
|
+
*
|
|
33
|
+
* Rotation keeps one previous generation rather than discarding it, because the question this file
|
|
34
|
+
* answers is usually about the recent past and a single rotation boundary is exactly where "we just ran
|
|
35
|
+
* that" sits.
|
|
36
|
+
*/
|
|
37
|
+
import { appendFileSync, existsSync, mkdirSync, readFileSync, renameSync, statSync, writeFileSync } from 'node:fs';
|
|
38
|
+
import { join } from 'node:path';
|
|
39
|
+
|
|
40
|
+
/** The memory log's filename, beside `agent-state.json`. */
|
|
41
|
+
export const MEMORY_FILE = 'memory.jsonl';
|
|
42
|
+
|
|
43
|
+
/** The one rotated generation: `<file>.1`. Kept, not deleted -- see the header. */
|
|
44
|
+
export const MEMORY_FILE_PREVIOUS = `${MEMORY_FILE}.1`;
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* Byte cap for the live file before it is rotated.
|
|
48
|
+
*
|
|
49
|
+
* 8 MiB holds on the order of 20,000 task records at the shape below -- several years of this fleet's
|
|
50
|
+
* observed rate -- while staying small enough that reading the whole file to answer a question is
|
|
51
|
+
* something a tool can do without a plan.
|
|
52
|
+
*/
|
|
53
|
+
export const MEMORY_MAX_BYTES = 8 * 1024 * 1024;
|
|
54
|
+
|
|
55
|
+
/**
|
|
56
|
+
* How much of a stream's tail is kept.
|
|
57
|
+
*
|
|
58
|
+
* The envelope already carries `stdout_tail` / `stderr_tail` for cross-machine comparison, so storing a
|
|
59
|
+
* second copy costs nothing extra and keeps every record the same shape. It is clamped again here
|
|
60
|
+
* rather than trusted, because this file is written on a machine the dispatcher does not control and a
|
|
61
|
+
* cap that can be exceeded by an argument is not a cap.
|
|
62
|
+
*/
|
|
63
|
+
export const MEMORY_TAIL_BYTES = 4096;
|
|
64
|
+
|
|
65
|
+
/**
|
|
66
|
+
* A bound on the recorded argv.
|
|
67
|
+
*
|
|
68
|
+
* The relay takes an argv array of arbitrary length, so an offer may legally carry a command longer than
|
|
69
|
+
* any sane memory record. Truncating the *record* is right here: `command_argv` is not compared across
|
|
70
|
+
* machines (only `command_hash` is), so a shortened argv cannot make two machines disagree.
|
|
71
|
+
*/
|
|
72
|
+
export const MEMORY_ARGV_LIMIT = 64;
|
|
73
|
+
|
|
74
|
+
/** @param {unknown} value @param {number} limit */
|
|
75
|
+
function clamp(value, limit) {
|
|
76
|
+
if (typeof value !== 'string' || value === '') return null;
|
|
77
|
+
return value.length <= limit ? value : `${value.slice(0, limit)}\n…[truncated at ${limit} characters]`;
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
/**
|
|
81
|
+
* The memory record for one finished task.
|
|
82
|
+
*
|
|
83
|
+
* Shaped to be readable by a person as much as by a tool: a reader asking "what happened here" should be
|
|
84
|
+
* able to `tail` the file and get an answer without a query.
|
|
85
|
+
*
|
|
86
|
+
* `conclusion` is deliberately absent. It is the field §3.1 of the design shows, and it is reserved for
|
|
87
|
+
* `w2m_memory` to fill in when a human or an agent decides what the result *meant* -- a machine that
|
|
88
|
+
* invented one would be asserting an interpretation it cannot have, and the whole point of separating
|
|
89
|
+
* fact from conclusion is that the two must not be confused later.
|
|
90
|
+
*
|
|
91
|
+
* @param {object} input
|
|
92
|
+
* @param {object} input.envelope The built result envelope.
|
|
93
|
+
* @param {string[]} input.commandArgv
|
|
94
|
+
* @param {string|null} input.cwdRel
|
|
95
|
+
* @param {number} input.ms Wall-clock duration of the attempt.
|
|
96
|
+
* @param {number} [input.now]
|
|
97
|
+
* @returns {object}
|
|
98
|
+
*/
|
|
99
|
+
export function memoryRecord({
|
|
100
|
+
envelope,
|
|
101
|
+
commandArgv,
|
|
102
|
+
cwdRel,
|
|
103
|
+
ms,
|
|
104
|
+
now = Date.now(),
|
|
105
|
+
}) {
|
|
106
|
+
const argv = Array.isArray(commandArgv) ? commandArgv.map(String) : [];
|
|
107
|
+
const truncatedArgv = argv.length > MEMORY_ARGV_LIMIT;
|
|
108
|
+
|
|
109
|
+
const record = {
|
|
110
|
+
at: new Date(now).toISOString(),
|
|
111
|
+
task_id: envelope.task_id,
|
|
112
|
+
attempt: envelope.attempt,
|
|
113
|
+
machine_id: envelope.machine_id,
|
|
114
|
+
machine_name: envelope.machine_name ?? null,
|
|
115
|
+
mode: envelope.mode ?? null,
|
|
116
|
+
argv: truncatedArgv ? argv.slice(0, MEMORY_ARGV_LIMIT) : argv,
|
|
117
|
+
cwd_rel: cwdRel ?? null,
|
|
118
|
+
// The exit code and status are what a reader checks first, so they sit above the tails.
|
|
119
|
+
exit_code: envelope.exit_code ?? null,
|
|
120
|
+
status: envelope.status ?? null,
|
|
121
|
+
refusal_reason: envelope.refusal_reason ?? null,
|
|
122
|
+
ms: Number.isFinite(ms) ? ms : null,
|
|
123
|
+
// Which way the offer travelled. Useful on its own: "this machine has not been reachable directly
|
|
124
|
+
// since Tuesday" is a fact about the fleet that only these records would show.
|
|
125
|
+
transport: envelope.transport ?? null,
|
|
126
|
+
stdout_tail: clamp(envelope.stdout_tail, MEMORY_TAIL_BYTES),
|
|
127
|
+
stderr_tail: clamp(envelope.stderr_tail, MEMORY_TAIL_BYTES),
|
|
128
|
+
// Hashes are repeated from the envelope on purpose: they are what ties this human-readable record
|
|
129
|
+
// to the relay's tamper-evident one, so the two can be cross-checked without trusting either.
|
|
130
|
+
stdout_sha256: envelope.stdout_sha256 ?? null,
|
|
131
|
+
stdout_bytes: envelope.stdout_bytes ?? null,
|
|
132
|
+
// Filled in later by `w2m_memory`, never by the machine. See above.
|
|
133
|
+
conclusion: null,
|
|
134
|
+
tags: [],
|
|
135
|
+
};
|
|
136
|
+
if (truncatedArgv) record.argv_truncated_from = argv.length;
|
|
137
|
+
return record;
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
/**
|
|
141
|
+
* Append a record, rotating first when the file has reached its cap.
|
|
142
|
+
*
|
|
143
|
+
* Never throws. A memory that can fail a task is worse than a memory that misses one: the task is the
|
|
144
|
+
* work, and this is a note about it. The caller gets `false` and the reason is on the record instead.
|
|
145
|
+
*
|
|
146
|
+
* Writes are synchronous and `O_APPEND`, so two attempts finishing at once interleave as whole lines
|
|
147
|
+
* rather than as corrupt ones -- the failure mode that would make the whole file unreadable.
|
|
148
|
+
*
|
|
149
|
+
* `maxBytes` is a parameter rather than a closed-over constant so that rotation is testable without
|
|
150
|
+
* writing eight megabytes to prove it. A cap that can only be exercised at production size is a cap
|
|
151
|
+
* whose rotation path is only exercised in production, which is the wrong place to find out.
|
|
152
|
+
*
|
|
153
|
+
* @param {string} stateDir
|
|
154
|
+
* @param {object} record
|
|
155
|
+
* @param {{maxBytes?: number}} [options]
|
|
156
|
+
* @returns {{ok: true, path: string, bytes: number, rotated: boolean}|{ok: false, error: string}}
|
|
157
|
+
*/
|
|
158
|
+
export function appendMemory(stateDir, record, { maxBytes = MEMORY_MAX_BYTES } = {}) {
|
|
159
|
+
try {
|
|
160
|
+
if (typeof stateDir !== 'string' || stateDir === '') {
|
|
161
|
+
return { ok: false, error: 'MEMORY_NO_STATE_DIR' };
|
|
162
|
+
}
|
|
163
|
+
mkdirSync(stateDir, { recursive: true, mode: 0o700 });
|
|
164
|
+
const path = join(stateDir, MEMORY_FILE);
|
|
165
|
+
|
|
166
|
+
let rotated = false;
|
|
167
|
+
if (existsSync(path)) {
|
|
168
|
+
const size = statSync(path).size;
|
|
169
|
+
if (size >= maxBytes) {
|
|
170
|
+
// Replace the previous generation rather than skipping rotation: keeping more than one would
|
|
171
|
+
// defeat the cap, and the cap is the point.
|
|
172
|
+
renameSync(path, join(stateDir, MEMORY_FILE_PREVIOUS));
|
|
173
|
+
writeFileSync(path, '', { mode: 0o600 });
|
|
174
|
+
rotated = true;
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
const line = `${JSON.stringify(record)}\n`;
|
|
179
|
+
appendFileSync(path, line, { mode: 0o600 });
|
|
180
|
+
return { ok: true, path, bytes: Buffer.byteLength(line), rotated };
|
|
181
|
+
} catch (error) {
|
|
182
|
+
return { ok: false, error: `MEMORY_APPEND_FAILED: ${error instanceof Error ? error.message : String(error)}` };
|
|
183
|
+
}
|
|
184
|
+
}
|
|
185
|
+
|
|
186
|
+
/**
|
|
187
|
+
* Read records back, newest first.
|
|
188
|
+
*
|
|
189
|
+
* Ordering is by file position reversed, not by parsing every `at`: the file is append-only, so position
|
|
190
|
+
* *is* time, and sorting would both cost more and mis-order two records written in the same millisecond.
|
|
191
|
+
*
|
|
192
|
+
* A malformed line is skipped rather than fatal. A partially written final line is the expected outcome
|
|
193
|
+
* of a machine losing power mid-append, and one damaged line must not make the rest unreadable -- which
|
|
194
|
+
* is exactly how the relay's own ledger behaves (`ledger_lines_skipped`), and for the same reason.
|
|
195
|
+
*
|
|
196
|
+
* Synchronous on purpose, and named for it: the reader is called from a tool that answers a question
|
|
197
|
+
* about a local file, and every other reader in this codebase (`w2m_status` on `agent-state.json`) is a
|
|
198
|
+
* synchronous read for the same reason -- a memory query that could interleave with an append would
|
|
199
|
+
* report a state the file was never in.
|
|
200
|
+
*
|
|
201
|
+
* @param {string} stateDir
|
|
202
|
+
* @param {{limit?: number, previous?: boolean}} [options]
|
|
203
|
+
* @returns {{records: object[], skipped: number, truncated: boolean, path: string|null, error?: string}}
|
|
204
|
+
*/
|
|
205
|
+
export function readMemorySync(stateDir, { limit = 50, previous = false } = {}) {
|
|
206
|
+
const empty = { records: [], skipped: 0, truncated: false, path: null };
|
|
207
|
+
if (typeof stateDir !== 'string' || stateDir === '') return { ...empty, error: 'MEMORY_NO_STATE_DIR' };
|
|
208
|
+
const path = join(stateDir, previous ? MEMORY_FILE_PREVIOUS : MEMORY_FILE);
|
|
209
|
+
if (!existsSync(path)) return { ...empty, path };
|
|
210
|
+
|
|
211
|
+
try {
|
|
212
|
+
const text = statSync(path).size === 0 ? '' : readFileSync(path, 'utf8');
|
|
213
|
+
const lines = text.split('\n');
|
|
214
|
+
if (lines.length > 0 && lines[lines.length - 1] === '') lines.pop();
|
|
215
|
+
|
|
216
|
+
const records = [];
|
|
217
|
+
let skipped = 0;
|
|
218
|
+
for (let i = lines.length - 1; i >= 0; i -= 1) {
|
|
219
|
+
if (records.length >= limit) break;
|
|
220
|
+
try {
|
|
221
|
+
records.push(JSON.parse(lines[i]));
|
|
222
|
+
} catch {
|
|
223
|
+
skipped += 1;
|
|
224
|
+
}
|
|
225
|
+
}
|
|
226
|
+
return { records, skipped, truncated: records.length >= limit, path };
|
|
227
|
+
} catch (error) {
|
|
228
|
+
return { ...empty, path, error: `MEMORY_READ_FAILED: ${error instanceof Error ? error.message : String(error)}` };
|
|
229
|
+
}
|
|
230
|
+
}
|
|
231
|
+
|
|
232
|
+
/** Retained under the shorter name: the reader has always been synchronous. */
|
|
233
|
+
export const readMemory = readMemorySync;
|