worktrust 0.10.36 → 0.10.37
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +1 -1
- package/README.md +14 -1
- package/SECURITY.md +63 -1
- package/chat-exports.mjs +4 -4
- package/config-edits.mjs +25 -0
- package/engine-release.json +1 -1
- package/history-outbox.mjs +44 -11
- package/history-progress.mjs +141 -0
- package/log-session.mjs +25 -25
- package/package-release.json +1 -1
- package/package.json +4 -2
- package/preserve.mjs +4 -3
- package/self-update.mjs +274 -0
- package/setup-mcp.mjs +3 -3
- package/worktrust.mjs +167 -31
package/AGENTS.md
CHANGED
|
@@ -7,7 +7,7 @@ the rules of this package.
|
|
|
7
7
|
|
|
8
8
|
## Do not run it for the person
|
|
9
9
|
|
|
10
|
-
The bare `worktrust` command, `connect`, `update`, `history` (and `--rebuild`), `disconnect`, `protect-key`, `preserve`, `codex`, `antigravity` and the
|
|
10
|
+
The bare `worktrust` command, `connect`, `update`, `self-update`, `history` (and `--rebuild`), `disconnect`, `protect-key`, `preserve`, `codex`, `antigravity` and the
|
|
11
11
|
import commands change what this computer sends or couples. **Only the person runs them, in their own terminal.** In an
|
|
12
12
|
agent's shell the CLI refuses these actions, `--yes` included. Version 0.10.12 adds the missing default-command
|
|
13
13
|
guard and tests both fresh and coupled synthetic homes. Do not work around the policy: do not unset
|
package/README.md
CHANGED
|
@@ -1,5 +1,16 @@
|
|
|
1
1
|
# worktrust
|
|
2
2
|
|
|
3
|
+
## 0.10.37: an export arrives whole; progress, Ctrl+C and the reason in WorkTrust's words; CLI active and updating itself
|
|
4
|
+
|
|
5
|
+
An export send no longer stops at WorkTrust's upload budget per minute: the person's own send waits it out (at most 90
|
|
6
|
+
seconds a wait) and goes on, and a batch WorkTrust stored in part is sent again at once until it holds every line; a line
|
|
7
|
+
already received is kept once. Every long step shows how far it is, and Ctrl+C stops cleanly, keeping what has not arrived
|
|
8
|
+
for the next run. A batch that did not arrive is said in WorkTrust's own words instead of "Not all of it arrived". The
|
|
9
|
+
receipt after an export names its hours as reconstructed by WorkTrust, beside the measured ones, never as measured.
|
|
10
|
+
CLI active per computer: a pause set in Sources › Devices is kept by the computer at its next session, and turning it back
|
|
11
|
+
on lets the computer update itself, verified (SECURITY.md, "Updating itself when you turn it on"). A computer on 0.10.36
|
|
12
|
+
or older updates once by hand: `npx worktrust@latest update`. What is measured is unchanged: the measure rules stay 3.
|
|
13
|
+
|
|
3
14
|
## 0.10.36: large and cut-off ChatGPT exports; hours only with `mcp off`; a re-measure reported whole or not at all
|
|
4
15
|
|
|
5
16
|
`npx worktrust history --export` now reads a ChatGPT export of any size: a zip64 archive (an export over 4 GB), and a
|
|
@@ -359,7 +370,9 @@ agent runs it makes no difference: `preserve` writes to your own user's places (
|
|
|
359
370
|
`~/AI-Evidence`), so every terminal and agent on this computer under your user reaches the same archive, and lines
|
|
360
371
|
from different versions sit side by side, each naming the version that measured it. Another user on the same computer
|
|
361
372
|
gets an archive of their own. A coupled computer's session hook runs from its own copy in `~/.worktrust` and changes
|
|
362
|
-
|
|
373
|
+
with `npx worktrust@latest update`, or (from 0.10.37, installed from npm) when you turn CLI active back on for that
|
|
374
|
+
computer in WorkTrust › Sources: it then updates itself, verified, the next time WorkTrust runs there (SECURITY.md,
|
|
375
|
+
"Updating itself when you turn it on"). The WorkTrust app says when a newer version is out.
|
|
363
376
|
|
|
364
377
|
**One command walks it all.** `npx worktrust@latest preserve` shows the report, asks `Keep these settings? [y/N]`
|
|
365
378
|
(then does what `--apply` does), asks `Keep a local archive in ~/AI-Evidence? [y/N]` (then does what `--archive`
|
package/SECURITY.md
CHANGED
|
@@ -12,7 +12,8 @@ private transcripts or account data in a public issue.
|
|
|
12
12
|
The complete official release is required. Individual script download fallbacks are removed;
|
|
13
13
|
interactive commands may check for a newer release.
|
|
14
14
|
- Submissions go to the configured WorkTrust MCP endpoint. Interactive commands can also contact
|
|
15
|
-
the npm registry for a version notice. Release notices never execute a different version
|
|
15
|
+
the npm registry for a version notice. Release notices never execute a different version; the one exception is the verified update when
|
|
16
|
+
you turn the CLI back on in Sources (below). The npm package bundles its runtime
|
|
16
17
|
modules, but package-manager installation and explicit upgrades still use the network. `preserve` itself is local.
|
|
17
18
|
A custom `--origin`, URL or registry override changes whom you trust.
|
|
18
19
|
- The key it receives is a WorkTrust coupling for this computer: it can submit work metadata to your
|
|
@@ -132,6 +133,67 @@ artifact was built from reviewed source. `npm audit signatures` can inspect an i
|
|
|
132
133
|
tree; verification coverage and failures must be reported for that actual installation.
|
|
133
134
|
|
|
134
135
|
|
|
136
|
+
## Updating itself when you turn it on (0.10.37)
|
|
137
|
+
|
|
138
|
+
This is the one place the CLI installs and runs a newer version without you typing a command. It exists because you
|
|
139
|
+
asked for it in the app: when you switch **CLI active** back on for a computer in WorkTrust › Sources, that computer
|
|
140
|
+
brings itself to the newest published CLI. A registry version string alone never starts it (the interactive notice
|
|
141
|
+
above still only tells you), and `npx worktrust update` is unchanged.
|
|
142
|
+
|
|
143
|
+
- **When.** WorkTrust asks for it only after your own act: turning CLI active on, under your signed-in session (with
|
|
144
|
+
your second factor when your account has one), for a computer whose CLI had confirmed it was paused. WorkTrust then
|
|
145
|
+
records one instruction for that computer; a pause clears it, a computer that never paused gets none. The hook reads it
|
|
146
|
+
with the computer's own signed key (`api/cli/door`) as `update: { version, at }`; `at`, the moment you turned it on,
|
|
147
|
+
is its id. The CLI takes an id once (`~/.worktrust/update.json`, 0600), never while paused, never one dated more than
|
|
148
|
+
five minutes past the door's own clock (its `Date` header), and, after an attempt that did not install, none for 24
|
|
149
|
+
hours. The "only after your turn-on" rule is WorkTrust's server's: the computer cannot see your act, so it does not
|
|
150
|
+
rely on it (next point).
|
|
151
|
+
- **Which version, decided on the computer.** It installs only a version that is newer than itself (`current`
|
|
152
|
+
otherwise: no downgrade) AND that npm's own registry record lists as `dist-tags.latest` and does not deprecate,
|
|
153
|
+
whatever version the door names. A version the console classes critical is deprecated on npm as well
|
|
154
|
+
(tools/worktrust-release/README.md), so a computer refuses it even if a server named it.
|
|
155
|
+
- **npm 11.12.0 or later.** `npm audit signatures --include-attestations` first appears there. The CLI uses the npm beside
|
|
156
|
+
the running Node (its own installation), else the first one on PATH, asks `npm --version` first, and with an older
|
|
157
|
+
npm, or none, reports `unsupported` before anything is fetched: update npm, or run the command.
|
|
158
|
+
- **What runs, implicitly.** Detached from the hook (which returns at once and prints nothing), under a lock:
|
|
159
|
+
1. `npm --version` and `npm view worktrust@<version> --json --registry=https://registry.npmjs.org/` (the policy above);
|
|
160
|
+
2. `npm install --ignore-scripts --registry=https://registry.npmjs.org/ worktrust@<version>` into a new staging folder
|
|
161
|
+
under `~/.worktrust/update-stage` (no lifecycle script of the package runs);
|
|
162
|
+
3. `npm audit signatures --json --include-attestations` in that folder;
|
|
163
|
+
4. after all checks below hold: the package is copied into `~/.worktrust/verified/<identity>` (as every update does)
|
|
164
|
+
and that new launcher, which verifies itself before importing anything, runs `update --yes`. That command may renew
|
|
165
|
+
this computer's key (the weekly signed renewal), takes WorkTrust's door out of and back into each app you agreed to
|
|
166
|
+
(Claude Code through `claude mcp remove` and `claude mcp add`; the other apps' config files rewritten; if the add
|
|
167
|
+
does not finish, the files it took out are written back as they were and Claude Code's entry is put back), refreshes
|
|
168
|
+
the session hook and the Codex and Antigravity wake lines you agreed to, reports its version to WorkTrust, and, when
|
|
169
|
+
the measure rules changed, counts this computer's sessions for the re-measure offer and sends none of them (the
|
|
170
|
+
offer's default is No). Nothing new is coupled and no history is sent.
|
|
171
|
+
The staging folder is removed afterwards (a folder that cannot be removed is logged, never a failure). Agent-shell
|
|
172
|
+
markers are left out of these processes' environment because the act is yours from Sources; `self-update` typed in
|
|
173
|
+
an agent's shell is refused like every person-only command.
|
|
174
|
+
- **The trust root, none of it WorkTrust's server.** (a) npm's registry signature for that exact version and integrity,
|
|
175
|
+
and (b) the npm provenance attestation: both VERIFIED BY NPM (registry keys; Sigstore's root, certificate chain and
|
|
176
|
+
transparency log) and relayed to the CLI by `npm audit signatures`; the CLI then binds what npm verified: the
|
|
177
|
+
certificate must name `https://github.com/fhomey/worktrust-cli/.github/workflows/publish.yml@refs/tags/v<version>`, its
|
|
178
|
+
Fulcio extension 1.3.6.1.4.1.57264.1.8 must hold GitHub Actions' OIDC issuer exactly, its DSSE signature must verify
|
|
179
|
+
under that certificate, its subject must be this tarball's sha512, and its workflow, repository and tag must match.
|
|
180
|
+
(c) WorkTrust's own release signature (`package-release.json`) under the key pinned in the CLI already installed is
|
|
181
|
+
the binding root: it does not depend on npm's word or on WorkTrust's server. The registry is forced to
|
|
182
|
+
registry.npmjs.org for every npm call, so a configured mirror cannot supply its own keys or record. A compromised
|
|
183
|
+
WorkTrust server can still send an instruction at a moment of its choosing; what a computer then installs is only the
|
|
184
|
+
release npm currently lists as latest and does not deprecate, newer than its own, with that provenance and that
|
|
185
|
+
release signature.
|
|
186
|
+
- **What falls back to the command.** A Python installation (pip, pipx, uv, uvx) does not update itself (`python`), and
|
|
187
|
+
neither does a computer without npm 11.12 or later, or a CLI not started from its verified npm copy (`unsupported`).
|
|
188
|
+
Sources then says what to do, with the command. A computer on 0.10.36 or older has no updater: update it once by hand.
|
|
189
|
+
- **When something fails.** Nothing new runs and the running CLI stays as it was. The next door call reports a code
|
|
190
|
+
beside the version, `worktrust-update/<installed|current|python|unsupported|unverified|failed>.<id>`, never content;
|
|
191
|
+
the reason stays in `~/.worktrust/update.log` on this computer (versions, codes, a short npm message; no key). An
|
|
192
|
+
outcome is written only for the instruction that started it. An attempt that has not finished in half an hour is
|
|
193
|
+
reported `failed`, once. `worktrust status` names the outcome.
|
|
194
|
+
- **What it does not establish.** As above: signatures and provenance tie the bytes to the reviewed publish workflow;
|
|
195
|
+
they do not prove truthful activity, and a process running as you can still change files after a check.
|
|
196
|
+
|
|
135
197
|
## The WorkTrust mod for Claude Code (0.10.15, opt-in)
|
|
136
198
|
|
|
137
199
|
`worktrust mod install` asks, then writes a Claude Code plugin to `~/.worktrust/claude-mod` and names
|
package/chat-exports.mjs
CHANGED
|
@@ -620,7 +620,7 @@ async function* exportRecords(given, { expect = null, excluded = () => false, sk
|
|
|
620
620
|
const totals = {};
|
|
621
621
|
const { parts, close, cuts = [] } = exportParts(given);
|
|
622
622
|
const path = parts[0].path, digest = createHash("sha256");
|
|
623
|
-
let vendor = null, count = 0, oversized = 0, empty = 0, bytes = 0, next = PROGRESS_BYTES, offset = 0;
|
|
623
|
+
let vendor = null, count = 0, oversized = 0, empty = 0, bytes = 0, next = PROGRESS_BYTES, offset = 0, tick = 0, at = 0; const size = parts.reduce((n, one) => n + (Number(one.bytes) || 0), 0);
|
|
624
624
|
if (report) { report.path = path; report.vendor = expect; report.digest = null; report.conversations = 0; report.oversized = 0; report.empty = 0; report.parts = parts.length; report.cut = cuts.length > 0 ? cuts : null; }
|
|
625
625
|
const one = (kept, index) => {
|
|
626
626
|
limit(count < EXPORT_LIMITS.conversations, "conversation count");
|
|
@@ -642,14 +642,14 @@ async function* exportRecords(given, { expect = null, excluded = () => false, sk
|
|
|
642
642
|
};
|
|
643
643
|
try {
|
|
644
644
|
for (const part of parts) {
|
|
645
|
-
let queue = [], seen = 0;
|
|
645
|
+
let queue = [], seen = 0; at += 1;
|
|
646
646
|
const walk = walker(metadata ? META_CONVERSATION : CONVERSATION, { text, budget: text ? EXPORT_LIMITS.conversationTextBytes : EXPORT_LIMITS.conversationBytes, onElement: (kept, index) => queue.push([kept, offset + index]) });
|
|
647
647
|
const drain = function* () { const due = queue; queue = []; for (const [kept, index] of due) { seen = index + 1 - offset; yield one(kept, index); } };
|
|
648
648
|
try {
|
|
649
649
|
for await (const chunk of part.chunks()) {
|
|
650
650
|
digest.update(chunk); bytes += chunk.length; walk.push(chunk);
|
|
651
651
|
if (bytes >= next) { process.stderr.write(`export: ${Math.round(bytes / 1024 ** 2)} MB read, ${count} conversations\n`); next += PROGRESS_BYTES; }
|
|
652
|
-
yield* drain();
|
|
652
|
+
yield* drain(); if (report?.onProgress && !(Date.now() < tick)) { tick = Date.now() + 250; report.onProgress({ part: at, parts: parts.length, bytes, size, conversations: count }); } // the read as it goes (2026-11-05), counts only
|
|
653
653
|
}
|
|
654
654
|
walk.end();
|
|
655
655
|
yield* drain();
|
|
@@ -662,7 +662,7 @@ async function* exportRecords(given, { expect = null, excluded = () => false, sk
|
|
|
662
662
|
}
|
|
663
663
|
} finally { close(); }
|
|
664
664
|
if (vendor === null && empty > 0) throw Error(`${path} is neither a ChatGPT nor a Claude export (or mixes the two)`);
|
|
665
|
-
if (report) { report.vendor = vendor ?? expect; report.digest = digest.digest("hex"); report.conversations = count; report.oversized = oversized; report.empty = empty; if (metadata) report.metadata = vendor === "chatgpt" ? totals : null; }
|
|
665
|
+
report?.onProgress?.({ part: at, parts: parts.length, bytes, size, conversations: count }); if (report) { report.vendor = vendor ?? expect; report.digest = digest.digest("hex"); report.conversations = count; report.oversized = oversized; report.empty = empty; if (metadata) report.metadata = vendor === "chatgpt" ? totals : null; }
|
|
666
666
|
}
|
|
667
667
|
|
|
668
668
|
/** One export, read whole but compact: its vendor by shape, one record per conversation (its kept lines; none for an excluded or oversized one), and the digest of its bytes (what `sent` is remembered by). */
|
package/config-edits.mjs
CHANGED
|
@@ -7,6 +7,7 @@
|
|
|
7
7
|
* Only block style is handled; a section written inline (`key: { … }`) makes placeUnder return false, and the caller
|
|
8
8
|
* leaves the file alone. No dependency: it ships beside setup-mcp.mjs.
|
|
9
9
|
*/
|
|
10
|
+
import { existsSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
|
|
10
11
|
const indentOf = (line) => line.length - line.trimStart().length;
|
|
11
12
|
const meaningful = (line) => line.trim() !== "" && !line.trimStart().startsWith("#");
|
|
12
13
|
/** `key:` at `indent` between start and stop: its line and the end of its block, or the inline value it carries. */
|
|
@@ -85,3 +86,27 @@ export const isWorkTrustDoor = (url) => {
|
|
|
85
86
|
return parsed.protocol === "https:" && (parsed.hostname === "worktrust.io" || parsed.hostname.endsWith(".worktrust.io")) || (own !== null && parsed.host === own) || ((parsed.hostname === "localhost" || parsed.hostname === "127.0.0.1") && LOCAL_DOOR_PORTS.has(parsed.port));
|
|
86
87
|
} catch { return false; }
|
|
87
88
|
};
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* PUT BACK WHAT A FAILED RUN TOOK OUT (security review L3, 2026-11-05). `doorSnapshot`: every config FILE setup-mcp may write
|
|
92
|
+
* (its bytes, or null when absent) and Claude Code's own user-scope entry, read only. `putDoorsBack(text, name, claude)`:
|
|
93
|
+
* writes those files back as they were and, with `claude` (a runner of Claude Code's own command), puts its entry back
|
|
94
|
+
* through `claude mcp add-json`. Claude Code's file is never rewritten (Claude Code writes it all the time); Goose's plugin
|
|
95
|
+
* folder and OpenClaw (its own command) are not kept. The snapshot travels in memory and on stdin, never in a file.
|
|
96
|
+
*/
|
|
97
|
+
export function doorSnapshot(files, claudeFile, name) {
|
|
98
|
+
const regular = (file) => { try { return statSync(file).isFile(); } catch { return false; } };
|
|
99
|
+
let claude = null;
|
|
100
|
+
try { claude = JSON.parse(readFileSync(claudeFile, "utf8")).mcpServers?.[name] ?? null; } catch { /* no Claude Code here */ }
|
|
101
|
+
return { files: Object.fromEntries([...new Set(files)].filter((file) => regular(file) || !existsSync(file)).map((file) => [file, regular(file) ? readFileSync(file).toString("base64") : null])), claude };
|
|
102
|
+
}
|
|
103
|
+
export function putDoorsBack(text, name, claude) {
|
|
104
|
+
let snapshot = null, back = 0;
|
|
105
|
+
try { snapshot = JSON.parse(text); } catch { snapshot = null; }
|
|
106
|
+
for (const [file, data] of Object.entries(snapshot?.files ?? {})) {
|
|
107
|
+
try { if (data === null) { if (existsSync(file)) rmSync(file); } else writeFileSync(file, Buffer.from(String(data), "base64")); back += 1; }
|
|
108
|
+
catch { console.log(` ! ${file} could not be put back as it was`); }
|
|
109
|
+
}
|
|
110
|
+
if (snapshot?.claude && claude) { claude(["mcp", "remove", "-s", "user", name]); console.log(claude(["mcp", "add-json", "-s", "user", name, JSON.stringify(snapshot.claude)]) ? " ↺ Claude Code: its WorkTrust entry put back" : " ! Claude Code: its WorkTrust entry could not be put back"); }
|
|
111
|
+
console.log(` ↺ ${back} config file(s) put back as they were before this run`);
|
|
112
|
+
}
|
package/engine-release.json
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"keyId":"608168ccdef72562fc91f9c0f9f2c2d6876bece7f7362cb545b818f7947da6b5","payload":{"files":{"chat-exports.mjs":"
|
|
1
|
+
{"keyId":"608168ccdef72562fc91f9c0f9f2c2d6876bece7f7362cb545b818f7947da6b5","payload":{"files":{"chat-exports.mjs":"1dc109a971f88605ee3fa3a7564bcd57bce2047b86bfcd92eb3e5acf865d8f11","claude-mod.mjs":"02d86b0cdddf39879efc5194c0a5be6b41456faedec28963dc123a4e88dc3942","commit-rules.mjs":"e94cc32a5415f6d9e978145ab9505ac4ae0bac9431e6e9104b9b37155260b51f","config-edits.mjs":"527a7674f9d52b284bfda3b8089eb4b930a709bf14aedbec7f6eef1e59ef52ee","count-behaviour.mjs":"179baf829eda488bab3f17ae6917473ec01d0d6346f47de7bc144351274f19c8","credential-store.mjs":"a4b3540a61f4ce19e535c7f1d260c0d817686cd066d559a76db2ab108144abba","credential-vault-scripts.mjs":"7e42fe1db3885c2dec6f1de87bebfbefe2292d29179a68f3da87d19eb6c7704b","credential-vault.mjs":"279baabb217e2a3f170e8d7f2b271a88484531f9031c441360ea91561a70da99","git-commits.mjs":"f9c084c54937a86df8fe30a0db2a58e04eeef0dbae69c54b8e368efc9964d6d8","history-outbox.mjs":"85c70389effc5b4ec9d9b33af9192314e67f47af2752515b42a7959344ce2354","history-progress.mjs":"dcc25ee94b727f2605a763349a4d8764640335d6cf0043fb6108cebeed16dcb9","log-session.mjs":"f7e8f247edcc5a3ad17c2c05b83452a938d0e9e20cde5f806a54c1fb04f28958","orchestration-contract.mjs":"ec08f303a166365b99e70b675de4cfe76c6156c2567e14b5569f6d992f0231b7","orchestration-evidence.mjs":"d0d06cecf0eb08e5c6e653c62a4f45b1eddf2119c2acb8341056b34f488d214a","orchestration-report.mjs":"d4e0a9ad8e4a01b462d22c6b1866aac291565c533ff476085d3367cb6d59bc1b","package-integrity.mjs":"a8210c9494a56bfb9a5ea3d10064c7a269f0b6330c0af7c0700e706c3c0224e3","preserve-lines.mjs":"2df64a69687aa51219bf7b86b88d175cc50990fa29574b31897e41a9ebeecf43","preserve-summary.mjs":"137f781ed7021596a3f0ea67afcf2ab8616ac293f833dbb8f176e838e0ed8321","preserve.mjs":"cd111cae0f53a25a2d878a158479abfb1ace353361454843f91a16d241101d46","runtime-install.mjs":"aab47b52c79d726ef428a24f982c648fa3f1c2dc2b574f633611d02924864e5f","self-update.mjs":"378e3620d48c72cf0bf812be408883b995cdb2223de34a20f762dd73eeffb258","session-databases.mjs":"b9fe20d7445205bdbf4cf2eaaf9cc770dad01e23d41e644b04930ca34e10426d","setup-mcp.mjs":"b732e3dd9b3e522b7cc8acf167ce28ba8f1a8dee797ee15bd7c3b02919b13e6f","stack-rules.mjs":"b23c56e72ec56ba02b17ed895f869172d988c468a697e8d6026b0e268f61e489","stretch-evidence.mjs":"5f5667b92adf11b1a5a6330a0f26dd6db73d56fe60823d9a5246dab71d27d380","transcript-readers.mjs":"e9f20415c2b5d62a09f35a63075002ebd1da56edb32f242d52822d58ecbd8ac3","worktrust-launch.mjs":"84801bd29d472d48188c126b9ebe3989a3132b54e13cc4fe534ff7c383a294ee","worktrust.mjs":"c417a0cabf02318cb3200bda514a3d4ca39dc95f5528d36ff59307f2f045b839"},"format":"worktrust-package/1","kind":"engine","version":"0.10.37"},"signature":"Hg_9vD73TXqtvh1_o8PHDBitqG8xsXHtwqr06dL6jDc7eOuOQLT-ZgKmPhw1TLojSJ9dG5nhWLfvZN7X9o2fBQ"}
|
package/history-outbox.mjs
CHANGED
|
@@ -77,9 +77,39 @@ export function historyRefusal(text,response) {
|
|
|
77
77
|
// A 413 (0.10.29) is the batch, not the queue: it is cut smaller, never sent again as it was, and nothing else waits for it.
|
|
78
78
|
// THE SERVER'S OWN SENTENCE (0.10.35; every final refusal since 2026-10-31): why WorkTrust refused the batch, printed instead of
|
|
79
79
|
// a guess (0.10.35 said "it missed a batch for seven days" of a batch refused for its lines). One line, control characters
|
|
80
|
-
// taken out, at most 400 characters.
|
|
81
|
-
|
|
82
|
-
|
|
80
|
+
// taken out, at most 400 characters. AND OF A BATCH THAT WAITS (0.10.37, incident 2026-11-05): an 'unconfirmed' reply that
|
|
81
|
+
// carries WorkTrust's own error ("could not import (57014)") hands its sentence on too, so a person reads why, not a bare
|
|
82
|
+
// "Not all of it arrived" that needed a database to explain. A review is said by the caller in its own words.
|
|
83
|
+
const message=rpc?.id===1&&typeof rpc?.error?.message==='string'&&rpc.error.message.trim()?rpc.error.message:null;
|
|
84
|
+
const said=message&&!review?message.replace(/[\u0000-\u001f\u007f-\u009f]/g,' ').slice(0,400):null;
|
|
85
|
+
// WHICH BUDGET AND HOW MUCH LANDED (2026-11-05): a 429's data.reason (minute_budget is waited out, day_budget stops), and the
|
|
86
|
+
// door's "ran out of time" counts, so a resend that lands more is told from one that lands nothing (deliverHistory).
|
|
87
|
+
const budget=rpc?.id===1&&typeof rpc?.error?.data?.reason==='string'&&/^[a-z_]{1,40}$/.test(rpc.error.data.reason)?rpc.error.data.reason:null,partial=said?partialOf(said):null;
|
|
88
|
+
return {reason:review?'review':closed?'closed':response.status===413?'oversize':response.status===429?'capacity':'unconfirmed',seconds:Number.isFinite(raw)&&raw>0?Math.min(MAX_RETRY_SECONDS,raw):0,...(said?{said}:{}),...(budget?{budget}:{}),...(partial?{partial}:{})};
|
|
89
|
+
}
|
|
90
|
+
/** import-history.ts's "the batch ran out of time: N new lines were stored in whole slices and M were not sent" as { stored, unsent }. */
|
|
91
|
+
export function partialOf(said){
|
|
92
|
+
if(typeof said!=='string'||!/\bran out of time\b/i.test(said))return null;
|
|
93
|
+
const stored=/(\d{1,7})\s+new\s+lines?\s+(?:were\s+|was\s+)?stored/i.exec(said),unsent=/(\d{1,7})\s+(?:were\s+|was\s+)?not\s+sent/i.exec(said);
|
|
94
|
+
return stored?{stored:Number(stored[1]),unsent:unsent?Number(unsent[1]):null}:null;
|
|
95
|
+
}
|
|
96
|
+
/**
|
|
97
|
+
* A BATCH THAT LANDED IN PART GOES AGAIN AT ONCE (incident 2026-11-05: 0.10.36 paused at the door's "ran out of time" and left 26
|
|
98
|
+
* of 45 export batches waiting). A line already received is kept once, so a PATIENT send (the person's own; never a hook) resends
|
|
99
|
+
* the batch while each resend lands more (a stored count above 0; a sentence without counts earns one resend), at most ten (each
|
|
100
|
+
* lands a whole slice of 20 of at most 200), and waits out the upload budget per minute (≤ 90 s a wait, 5 a batch). Anything else
|
|
101
|
+
* is answered as before. `confirm(text)` throws unless the text is a receipt for every line; a network error is the caller's.
|
|
102
|
+
*/
|
|
103
|
+
export const HISTORY_RESENDS=10,BUDGET_WAIT_SECONDS=90,BUDGET_WAITS=5;
|
|
104
|
+
export async function deliverHistory(send,confirm,{patient=false,lines=0,progress=null,halt=null,sleep=(ms)=>new Promise((done)=>setTimeout(done,ms))}={}){ // `halt`: a stop asked, so nothing is sent again
|
|
105
|
+
for(let resends=0,waits=0;;){
|
|
106
|
+
const response=await send(),text=await response.text();
|
|
107
|
+
try{if(!response.ok)throw Error();confirm(text);return {ok:true,response,text};}catch{/* a refusal, read below */}
|
|
108
|
+
const refusal=historyRefusal(text,response),part=refusal.partial;
|
|
109
|
+
if(patient&&!halt?.()&&refusal.reason==='unconfirmed'&&response.status===200&&refusal.said&&!(refusal.seconds>0)&&resends<HISTORY_RESENDS&&(part?part.stored>0:resends===0)){resends++;progress?.({phase:'resend',lines,...(part?{stored:part.stored,...(Number.isInteger(part.unsent)?{unsent:part.unsent}:{})}:{})});continue;}
|
|
110
|
+
if(patient&&!halt?.()&&refusal.budget==='minute_budget'&&refusal.reason==='capacity'&&refusal.seconds>0&&refusal.seconds<=BUDGET_WAIT_SECONDS&&waits<BUDGET_WAITS){waits++;progress?.({phase:'wait',seconds:refusal.seconds});await sleep(refusal.seconds*1000);continue;}
|
|
111
|
+
return {ok:false,response,text,refusal};
|
|
112
|
+
}
|
|
83
113
|
}
|
|
84
114
|
|
|
85
115
|
/** Explicit recovery, even after the AI client removed its source files. A changed account/key
|
|
@@ -92,8 +122,10 @@ export function historyRefusal(text,response) {
|
|
|
92
122
|
* of its batches is still on this computer; once WorkTrust holds part of a run, the rest can never complete it, and is let go
|
|
93
123
|
* (`refused`: WorkTrust keeps the earlier lines counting, and `npx worktrust@latest history` measures again). A single line too
|
|
94
124
|
* large for a call is let go the same way. `only` limits a pass to those batches (and their pieces): the send that just queued them.
|
|
125
|
+
* `patient` (2026-11-05): the person's own pass resends a batch landed in part and waits out a minute's budget (deliverHistory);
|
|
126
|
+
* a hook's pass never does. `progress` hears each batch's counts, for the command's display.
|
|
95
127
|
*/
|
|
96
|
-
export async function resumeHistory(target, {send, confirmed, note, closed=null, retryReview=false, dry=false, limit=Infinity, cutter=null, only=null}) {
|
|
128
|
+
export async function resumeHistory(target, {send, confirmed, note, closed=null, retryReview=false, dry=false, limit=Infinity, cutter=null, only=null, patient=false, progress=null, halt=null, sleep=undefined}) {
|
|
97
129
|
const root=join(homedir(),'.worktrust','history-outbox'),scope=scopeOf(target);
|
|
98
130
|
const totals={received:0,review:0,waiting:0,batches:0,refused:0,held:0,split:0};let paused=null,sent=0;
|
|
99
131
|
if(!existsSync(root))return totals;
|
|
@@ -141,7 +173,7 @@ export async function resumeHistory(target, {send, confirmed, note, closed=null,
|
|
|
141
173
|
if(queued.status==='acknowledged'||queued.status==='closed')continue;
|
|
142
174
|
if(queued.status==='replaced'){queue.unshift(...queued.replacedBy);continue;}
|
|
143
175
|
// `limit` (0.10.27): the hook sends a few a run by itself; the rest wait, untouched, for the next run.
|
|
144
|
-
if(sent>=limit){totals.waiting+=item.entries.length;continue;}
|
|
176
|
+
if(sent>=limit||halt?.()){totals.waiting+=item.entries.length;continue;} // or a stop was asked (2026-11-05)
|
|
145
177
|
if(queued.status==='review'&&!retryReview){totals.review+=item.entries.length;continue;}
|
|
146
178
|
if(cutter){
|
|
147
179
|
const tighter=queued.status==='oversize',runTooLarge=Boolean(item.run)&&runOf(item.run.id).some(batch=>!cutter.fits(batch.entries)||marked(batch.id));
|
|
@@ -154,17 +186,18 @@ export async function resumeHistory(target, {send, confirmed, note, closed=null,
|
|
|
154
186
|
if(queued.status==='waiting'&&!paused)paused={reason:'unconfirmed',seconds:Math.max(1,Math.ceil((queued.retryAt-Date.now())/1000))};
|
|
155
187
|
if(paused){if(queued.status==='review'){totals.review+=item.entries.length;continue;}queued.defer(paused.reason,paused.seconds);totals.waiting+=item.entries.length;continue;}
|
|
156
188
|
try {
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
const refusal=
|
|
189
|
+
// 2026-11-05: through deliverHistory, so a patient pass (`--resume`, the send's own cut) resends a batch landed in part at once.
|
|
190
|
+
sent++;const delivered=await deliverHistory(()=>send(item.body,item.source),(text)=>confirmed(text,item.entries.length),{patient,lines:item.entries.length,progress,halt,sleep}),text=delivered.text;
|
|
191
|
+
if(!delivered.ok){
|
|
192
|
+
const refusal=delivered.refusal;if(refusal.reason!=='oversize')progress?.({phase:'batch',lines:item.entries.length,landed:false});
|
|
161
193
|
if(refusal.reason==='closed'){queued.close();totals.refused+=item.entries.length;if(refusal.said)totals.said=refusal.said;closed?.(item.run,item.entries.length);continue;}
|
|
162
194
|
// Too large (0.10.29): marked and taken again at once, by the cut, never resent as it was. Without a cutter it waits.
|
|
163
195
|
if(refusal.reason==='oversize'){queued.oversize();if(cutter){seen.delete(id);queue.unshift(id);}else totals.waiting+=item.entries.length;continue;}
|
|
196
|
+
if(refusal.said)totals.reason=refusal.said; // 0.10.37: why the batch waits, in WorkTrust's words (the last one heard)
|
|
164
197
|
if(refusal.reason!=='review')paused={reason:refusal.reason,seconds:refusal.seconds>0?refusal.seconds:5};queued.defer(refusal.reason,paused?.seconds??refusal.seconds);totals[refusal.reason==='review'?'review':'waiting']+=item.entries.length;continue;
|
|
165
198
|
}
|
|
166
|
-
queued.confirm(item.entries.length);totals.received+=item.entries.length;note(text,item.run);
|
|
167
|
-
}catch{paused={reason:'unconfirmed',seconds:5};queued.defer(paused.reason,paused.seconds);totals.waiting+=item.entries.length;}
|
|
199
|
+
queued.confirm(item.entries.length);totals.received+=item.entries.length;note(text,item.run);progress?.({phase:'batch',lines:item.entries.length,landed:true});
|
|
200
|
+
}catch{paused={reason:'unconfirmed',seconds:5};queued.defer(paused.reason,paused.seconds);totals.waiting+=item.entries.length;progress?.({phase:'batch',lines:item.entries.length,landed:false});}
|
|
168
201
|
}
|
|
169
202
|
return totals;
|
|
170
203
|
}
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* THE LONG STEPS, AS THEY GO, AND A STOP THAT LEAVES NOTHING HALF DONE (owner, incident 2026-11-05). `npx worktrust history
|
|
3
|
+
* --export` of 5,003 conversations read a 4 GB archive and sent about 45 batches for many minutes with nothing on the screen,
|
|
4
|
+
* and Ctrl+C had no handler. Reads no file and no network; counts only, never a title, a text or a file's name.
|
|
5
|
+
*
|
|
6
|
+
* THE EVENTS. The hook, asked by WORKTRUST_HISTORY_PROGRESS, hands each step's counts on (`progressSink`): "1" prints one JSON
|
|
7
|
+
* line each on stdout ({"history_progress":{phase,…}}) for the command to draw; "stderr" draws them here, on stderr (preserve
|
|
8
|
+
* reads the hook's stdout as the archive's lines). Written with writeSync, so a long synchronous read still shows as it goes.
|
|
9
|
+
*
|
|
10
|
+
* phase read { unit, part, parts, bytes?, size?, conversations? } reading an export's parts, or the sessions here
|
|
11
|
+
* phase plan { batches, lines } what the send will be (the read is closed, the stop hint said once)
|
|
12
|
+
* phase batch { lines, landed } a batch confirmed (or already held), or one that did not arrive
|
|
13
|
+
* phase resend { lines, stored?, unsent? } a batch the door landed in part, sent again at once
|
|
14
|
+
* phase wait { seconds } the door's upload limit per minute, waited out
|
|
15
|
+
*
|
|
16
|
+
* THE DISPLAY. On a terminal ONE line rewritten in place (\r, no spinner), redrawn each second: a bar, the counts, percent,
|
|
17
|
+
* elapsed and, sending, the time left from the rate so far, waits included (said once two batches landed). A resend and a wait
|
|
18
|
+
* are said on lines of their own. Not a terminal (piped, CI): a reading line per tenth, a line per batch, never a carriage
|
|
19
|
+
* return or an escape. `end()` closes with the totals; `stop()` with where it stopped and how to go on.
|
|
20
|
+
*/
|
|
21
|
+
import { writeSync } from "node:fs";
|
|
22
|
+
|
|
23
|
+
const thousands = (n) => Number(n).toLocaleString("en-US");
|
|
24
|
+
const count = (n) => (Number.isSafeInteger(n) && n >= 0 ? n : 0);
|
|
25
|
+
/** 12s, 4m10s, 1h04m. */
|
|
26
|
+
export const took = (ms) => {
|
|
27
|
+
const s = Math.max(0, Math.round(ms / 1000));
|
|
28
|
+
if (s < 60) return `${s}s`;
|
|
29
|
+
const m = Math.floor(s / 60);
|
|
30
|
+
return m < 60 ? `${m}m${String(s % 60).padStart(2, "0")}s` : `${Math.floor(m / 60)}h${String(m % 60).padStart(2, "0")}m`;
|
|
31
|
+
};
|
|
32
|
+
const UNITS = { parts: ["part", "parts"], sessions: ["session", "sessions"] };
|
|
33
|
+
const bar = (percent) => { const full = Math.min(16, Math.floor((16 * percent) / 100)); return "▓".repeat(full) + "░".repeat(16 - full); };
|
|
34
|
+
export const CONTINUE = "Run the same command again to continue: a line already received is kept once.";
|
|
35
|
+
|
|
36
|
+
export function historyProgress({ stream = process.stdout, now = Date.now } = {}) {
|
|
37
|
+
const tty = stream?.isTTY === true;
|
|
38
|
+
let plan = null, started = null, landed = 0, lines = 0, tried = 0, open = false, ticking = null, read = null, ended = false;
|
|
39
|
+
const of = () => Math.max(plan?.lines ?? 0, lines);
|
|
40
|
+
const percent = () => (of() ? Math.floor((100 * lines) / of()) : 0);
|
|
41
|
+
const elapsed = (from = started) => took(now() - (from ?? now()));
|
|
42
|
+
// The rate so far, over the whole time since the send began (a wait or a resend included), never a guess before two batches.
|
|
43
|
+
const left = () => {
|
|
44
|
+
if (landed < 2 || lines >= of()) return "";
|
|
45
|
+
const spent = now() - started;
|
|
46
|
+
if (!(spent > 0)) return "";
|
|
47
|
+
const seconds = ((of() - lines) * spent) / lines / 1000;
|
|
48
|
+
return seconds < 60 ? " · <1 min left" : ` · ~${Math.round(seconds / 60)} min left`;
|
|
49
|
+
};
|
|
50
|
+
const totals = () => `${landed}/${Math.max(plan?.batches ?? 0, landed)} batches · ${thousands(lines)}/${thousands(of())} lines · ${percent()}% · ${elapsed()}`;
|
|
51
|
+
// Reading: by bytes when the size is known (an export's parts), else by the count (the sessions here).
|
|
52
|
+
const readPercent = () => (read.size > 0 ? Math.min(100, Math.floor((100 * read.bytes) / read.size)) : read.parts > 0 ? Math.floor((100 * read.part) / read.parts) : 0);
|
|
53
|
+
const readCounts = () => `${thousands(read.part)}/${thousands(read.parts)} ${UNITS[read.unit]?.[read.parts === 1 ? 0 : 1] ?? "parts"}${read.conversations !== null ? ` · ${thousands(read.conversations)} conversations` : ""}`;
|
|
54
|
+
const line = () => (read && !plan ? ` Reading ${bar(readPercent())} ${readCounts()} · ${readPercent()}% · ${elapsed(read.started)}` : ` Sending ${bar(percent())} ${totals()}${left()}`);
|
|
55
|
+
const draw = () => { if (!tty || (!plan && !read)) return; stream.write(`\r\x1b[K${line()}`); open = true; };
|
|
56
|
+
const clear = () => { if (open) stream.write("\r\x1b[K"); open = false; };
|
|
57
|
+
const note = (text) => { if (tty) { clear(); stream.write(` ${text}\n`); draw(); } else stream.write(` ${text}\n`); };
|
|
58
|
+
const tick = () => { if (tty && !ticking) { ticking = setInterval(() => { if (open) draw(); }, 1000); ticking.unref?.(); } };
|
|
59
|
+
const closeRead = () => { if (!read || read.closed) return; read.closed = true; clear(); stream.write(` Read ${readCounts()} · ${elapsed(read.started)}\n`); };
|
|
60
|
+
const handle = (event) => {
|
|
61
|
+
if (event.phase === "read") {
|
|
62
|
+
if (plan) return;
|
|
63
|
+
const unit = event.unit === "sessions" ? "sessions" : "parts";
|
|
64
|
+
if (!read || read.unit !== unit) read = { unit, started: now(), step: -1, closed: false };
|
|
65
|
+
Object.assign(read, { part: count(event.part), parts: Math.max(count(event.parts), count(event.part)), bytes: count(event.bytes), size: count(event.size), conversations: Number.isSafeInteger(event.conversations) ? event.conversations : null });
|
|
66
|
+
if (tty) { draw(); tick(); return; }
|
|
67
|
+
const step = Math.floor(readPercent() / 10);
|
|
68
|
+
if (step > read.step) { read.step = step; stream.write(` reading ${readCounts()} · ${readPercent()}% · ${elapsed(read.started)}\n`); }
|
|
69
|
+
return;
|
|
70
|
+
}
|
|
71
|
+
if (event.phase === "plan") {
|
|
72
|
+
closeRead();
|
|
73
|
+
plan = { batches: count(event.batches), lines: count(event.lines) };
|
|
74
|
+
started ??= now();
|
|
75
|
+
stream.write(tty ? "\x1b[2m Ctrl+C stops; run it again to continue.\x1b[22m\n" : " Ctrl+C stops; run it again to continue.\n");
|
|
76
|
+
draw(); tick();
|
|
77
|
+
return;
|
|
78
|
+
}
|
|
79
|
+
started ??= now();
|
|
80
|
+
if (event.phase === "batch") {
|
|
81
|
+
tried += 1;
|
|
82
|
+
if (event.landed === true) { landed += 1; lines += count(event.lines); }
|
|
83
|
+
if (tty) draw();
|
|
84
|
+
else stream.write(` batch ${tried}/${Math.max(plan?.batches ?? 0, tried)} · ${thousands(lines)}/${thousands(of())} lines · ${percent()}% · ${elapsed()}${left()}${event.landed === true ? "" : " · not confirmed"}\n`);
|
|
85
|
+
return;
|
|
86
|
+
}
|
|
87
|
+
if (event.phase === "resend") {
|
|
88
|
+
const n = count(event.lines), unsent = Number.isSafeInteger(event.unsent) ? event.unsent : null, stored = Number.isSafeInteger(event.stored) ? event.stored : null;
|
|
89
|
+
const what = unsent !== null && n >= unsent ? `${n - unsent} of ${n} stored` : stored !== null ? `${stored} new lines stored` : "not confirmed";
|
|
90
|
+
note(`batch ${tried + 1}: ${what}, ${unsent !== null || stored !== null ? "resending the rest" : "sending it once more"}`);
|
|
91
|
+
return;
|
|
92
|
+
}
|
|
93
|
+
if (event.phase === "wait") note(`waiting ${count(event.seconds)} s: upload limit per minute`);
|
|
94
|
+
};
|
|
95
|
+
const quiet = () => { if (ticking) { clearInterval(ticking); ticking = null; } };
|
|
96
|
+
return {
|
|
97
|
+
event(event) { if (event && typeof event === "object") handle(event); },
|
|
98
|
+
line(text) {
|
|
99
|
+
if (typeof text !== "string" || !text.startsWith('{"history_progress":')) return false;
|
|
100
|
+
try { const event = JSON.parse(text).history_progress; if (event && typeof event === "object") handle(event); } catch { /* a torn event is dropped, never printed */ }
|
|
101
|
+
return true;
|
|
102
|
+
},
|
|
103
|
+
end() {
|
|
104
|
+
quiet();
|
|
105
|
+
if (ended) return; ended = true;
|
|
106
|
+
if (!plan) { closeRead(); return; }
|
|
107
|
+
clear();
|
|
108
|
+
stream.write(` ${lines >= of() && of() > 0 ? "Sent" : "Stopped"} ${totals()}\n`);
|
|
109
|
+
},
|
|
110
|
+
/** Ctrl+C: the line restored, where it stopped, and how to go on. */
|
|
111
|
+
stop({ sending = CONTINUE, reading = "Nothing was sent; run the same command again to start again." } = {}) {
|
|
112
|
+
quiet(); clear(); ended = true;
|
|
113
|
+
if (plan) stream.write(` Stopped at ${landed}/${Math.max(plan.batches, landed)} batches · ${thousands(lines)}/${thousands(of())} lines. ${sending}\n`);
|
|
114
|
+
else stream.write(` Stopped${read ? ` while reading, at ${readCounts()}` : ""}. ${reading}\n`);
|
|
115
|
+
},
|
|
116
|
+
};
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
/** The hook's side of the events (see above); null when nobody asked. */
|
|
120
|
+
export function progressSink(mode = process.env.WORKTRUST_HISTORY_PROGRESS) {
|
|
121
|
+
const out = (fd) => (text) => { try { writeSync(fd, text); } catch { /* a full or closed pipe drops an event, never the run */ } return true; };
|
|
122
|
+
if (mode === "1") { const write = out(1); return { event: (event) => write(`${JSON.stringify({ history_progress: event })}\n`), end() {} }; }
|
|
123
|
+
if (mode !== "stderr") return null;
|
|
124
|
+
const shown = historyProgress({ stream: { isTTY: process.stderr.isTTY === true, write: out(2) } });
|
|
125
|
+
process.once("exit", () => shown.end()); // the totals, however the hook ends (writeSync: an exit handler may write)
|
|
126
|
+
return { event: (event) => shown.event(event), end: () => shown.end() };
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
/**
|
|
130
|
+
* STOPPING SAFELY (owner, 2026-11-05). The first SIGINT or SIGTERM asks a send to stop: the call in flight is abandoned through
|
|
131
|
+
* `signal` (a line it carried that WorkTrust already received is kept once, so the next run sends it again safely), no batch
|
|
132
|
+
* starts after it, the outbox keeps every batch not confirmed (its files are written to a temporary name and renamed, and a
|
|
133
|
+
* signal is handled between, never inside, those synchronous writes), and the hook exits 130. A second one exits at once.
|
|
134
|
+
*/
|
|
135
|
+
export function stopSignals() {
|
|
136
|
+
const controller = new AbortController();
|
|
137
|
+
let asked = 0;
|
|
138
|
+
const stop = () => { asked += 1; if (asked > 1) process.exit(130); controller.abort(); };
|
|
139
|
+
process.on("SIGINT", stop); process.on("SIGTERM", stop);
|
|
140
|
+
return { get asked() { return asked > 0; }, signal: controller.signal };
|
|
141
|
+
}
|