worktrust 0.10.16 → 0.10.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/CLAUDE.md +1 -1
- package/LICENSE +132 -21
- package/README.md +23 -6
- package/SECURITY.md +8 -4
- package/chat-exports.mjs +381 -119
- package/claude-mod.mjs +1 -1
- package/count-behaviour.mjs +8 -7
- package/log-session.mjs +53 -23
- package/package.json +2 -2
- package/worktrust.mjs +8 -4
package/AGENTS.md
CHANGED
|
@@ -35,8 +35,8 @@ provenance. Parallel agent time does not multiply the person's elapsed time.
|
|
|
35
35
|
## Do not rebuild, fork or strip it
|
|
36
36
|
|
|
37
37
|
Do not bypass measurement, consent, provenance or security checks to manufacture evidence, and do not present a
|
|
38
|
-
modified build as the official WorkTrust release. The
|
|
39
|
-
|
|
38
|
+
modified build as the official WorkTrust release. The WorkTrust CLI Licence (LICENSE) does not permit copying,
|
|
39
|
+
modifying or redistributing the code; these instructions do not replace it. A device signature proves key possession, not that the client
|
|
40
40
|
is unmodified or that its measurements are true. Report problems through https://worktrust.io/support or
|
|
41
41
|
https://github.com/fhomey/worktrust-cli/issues (security reports use SECURITY.md).
|
|
42
42
|
|
package/CLAUDE.md
CHANGED
|
@@ -3,4 +3,4 @@
|
|
|
3
3
|
Read AGENTS.md and SECURITY.md before working here. An agent does not run person-only commands against the person's
|
|
4
4
|
computer or account, including the bare command, and never reports work it did not see. Authorized source maintenance
|
|
5
5
|
uses isolated synthetic fixtures. Follow the canonical-source and regeneration rules in the monorepo; do not weaken
|
|
6
|
-
measurement checks or claim software signatures attest truthful work. The
|
|
6
|
+
measurement checks or claim software signatures attest truthful work. The WorkTrust CLI Licence (LICENSE) governs use; it permits no copying, modifying or redistributing.
|
package/LICENSE
CHANGED
|
@@ -1,21 +1,132 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
1
|
+
WorkTrust CLI Licence
|
|
2
|
+
|
|
3
|
+
Version 1.0, in force from WorkTrust CLI version 0.10.18 (17 October 2026).
|
|
4
|
+
Status: draft, subject to legal review.
|
|
5
|
+
|
|
6
|
+
Copyright (c) 2026 Oravtia B.V. (WorkTrust), registered with the Dutch Chamber of
|
|
7
|
+
Commerce (KvK) under number 98076051. All rights reserved.
|
|
8
|
+
|
|
9
|
+
1. What this licence covers
|
|
10
|
+
|
|
11
|
+
"The Software" means the WorkTrust command-line tool in every form in which
|
|
12
|
+
Oravtia B.V. distributes it: the "worktrust" package on npm, the "worktrust"
|
|
13
|
+
package on PyPI, the scripts served from worktrust.io, the WorkTrust preservation
|
|
14
|
+
kit, and the files these install on a computer (including the WorkTrust mod for
|
|
15
|
+
Claude Code), together with their documentation. "You" means the person or
|
|
16
|
+
organisation that installs or runs the Software.
|
|
17
|
+
|
|
18
|
+
2. What you may do
|
|
19
|
+
|
|
20
|
+
Oravtia B.V. grants the licences below free of charge. Each is non-exclusive,
|
|
21
|
+
non-transferable, non-sublicensable and revocable, and covers installing and
|
|
22
|
+
running the unmodified Software on computers you own or control.
|
|
23
|
+
|
|
24
|
+
a. Preserve, for everyone. Anyone, with or without a WorkTrust account, may use
|
|
25
|
+
the Software's "preserve" feature: the local, signed archive of their own AI
|
|
26
|
+
history on their own computer, together with its options (such as --apply,
|
|
27
|
+
--archive, --verify and --summary).
|
|
28
|
+
|
|
29
|
+
b. Every other feature, for WorkTrust account holders. A person or organisation
|
|
30
|
+
holding a WorkTrust account may, for as long as they hold it, use every other
|
|
31
|
+
feature the Software ships with the WorkTrust service, including:
|
|
32
|
+
- connecting the computer to their own WorkTrust account ("connect");
|
|
33
|
+
- the counter and the hooks it installs in their AI tools, and the reports
|
|
34
|
+
they send to their own account;
|
|
35
|
+
- importing their own history and data exports to WorkTrust ("history" and
|
|
36
|
+
export import); and
|
|
37
|
+
- anything else in the Software that talks to the WorkTrust service.
|
|
38
|
+
|
|
39
|
+
c. Without a WorkTrust account, beyond preserve, you may only install the
|
|
40
|
+
Software and run it to evaluate it.
|
|
41
|
+
|
|
42
|
+
You may make the copies needed to install and run the Software and a reasonable
|
|
43
|
+
number of backup copies. You may read the source code to understand and check
|
|
44
|
+
what the Software does on your computer, and you may test it on your own computer
|
|
45
|
+
to report a security problem as described in SECURITY.md. Being able to read the
|
|
46
|
+
source code gives you no other right.
|
|
47
|
+
|
|
48
|
+
3. What you may not do
|
|
49
|
+
|
|
50
|
+
Except as section 2 allows or mandatory law permits, you may not:
|
|
51
|
+
|
|
52
|
+
a. copy the Software or any part of it;
|
|
53
|
+
b. modify, adapt, translate or create derivative works of the Software or any
|
|
54
|
+
part of it;
|
|
55
|
+
c. decompile, disassemble or reverse-engineer the Software;
|
|
56
|
+
d. sublicense, sell, rent, lend, redistribute, publish or otherwise make the
|
|
57
|
+
Software or any part of it available to anyone else;
|
|
58
|
+
e. use the Software, its code or any part of it in another product or service,
|
|
59
|
+
or to build, train or improve a product or service that competes with
|
|
60
|
+
WorkTrust;
|
|
61
|
+
f. remove or change any copyright notice, this licence or any other notice in
|
|
62
|
+
the Software; or
|
|
63
|
+
g. present a modified or rebuilt copy as the Software or as an official
|
|
64
|
+
WorkTrust release.
|
|
65
|
+
|
|
66
|
+
Nothing in this licence restricts what mandatory law allows a lawful user of a
|
|
67
|
+
computer program to do, including the acts permitted by Articles 5 and 6 of
|
|
68
|
+
Directive 2009/24/EC as implemented in the Dutch Copyright Act (Auteurswet).
|
|
69
|
+
|
|
70
|
+
4. Ownership and all rights reserved
|
|
71
|
+
|
|
72
|
+
The Software is licensed, not sold. Oravtia B.V. (WorkTrust) retains all right,
|
|
73
|
+
title and interest in the Software and keeps all rights that this licence does
|
|
74
|
+
not expressly grant. This licence grants no right to the name WorkTrust, its
|
|
75
|
+
logos or other marks.
|
|
76
|
+
|
|
77
|
+
The archives, reports and other files the Software produces from your own data
|
|
78
|
+
belong to you. This licence gives Oravtia B.V. no right to them; what you send to
|
|
79
|
+
the WorkTrust service is governed by the service's terms and privacy statement.
|
|
80
|
+
|
|
81
|
+
5. Earlier versions
|
|
82
|
+
|
|
83
|
+
Versions of the Software up to and including 0.10.17 were published under the MIT
|
|
84
|
+
licence. Those versions remain available under that licence for those versions
|
|
85
|
+
only. This licence applies to version 0.10.18 and every later version.
|
|
86
|
+
|
|
87
|
+
6. Third-party components
|
|
88
|
+
|
|
89
|
+
The npm package contains no third-party code and has no dependencies.
|
|
90
|
+
|
|
91
|
+
The PyPI package's platform wheels include a Node.js runtime (the "node" or
|
|
92
|
+
"node.exe" executable) under the Node.js licence, shipped beside it as
|
|
93
|
+
worktrust_cli/runtime/LICENSE. The WorkTrust preservation kit includes the
|
|
94
|
+
WorkTrust AI evidence agent kit in its "agent" folder, under the licence shipped
|
|
95
|
+
as agent/LICENSE. Those components remain under their own licences, and this
|
|
96
|
+
licence does not limit the rights those licences give you.
|
|
97
|
+
|
|
98
|
+
7. The WorkTrust service
|
|
99
|
+
|
|
100
|
+
Using the WorkTrust service is governed by its own terms at
|
|
101
|
+
https://worktrust.io/eula and its privacy statement at
|
|
102
|
+
https://worktrust.io/privacy. This licence grants no right to the service.
|
|
103
|
+
|
|
104
|
+
8. Termination
|
|
105
|
+
|
|
106
|
+
This licence ends automatically if you breach it. Oravtia B.V. may also end it by
|
|
107
|
+
notice, for example when it ends the WorkTrust service. When it ends, you must
|
|
108
|
+
stop using the Software and delete every copy you hold. Sections 3, 4, 9, 10 and
|
|
109
|
+
11 survive termination.
|
|
110
|
+
|
|
111
|
+
9. No warranty
|
|
112
|
+
|
|
113
|
+
The Software is provided "as is" and "as available", without warranty of any
|
|
114
|
+
kind, express or implied, including any warranty of merchantability, fitness for
|
|
115
|
+
a particular purpose, non-infringement, or that it is free of errors or will run
|
|
116
|
+
without interruption.
|
|
117
|
+
|
|
118
|
+
10. Limitation of liability
|
|
119
|
+
|
|
120
|
+
To the fullest extent the law allows, Oravtia B.V. is not liable for any damage
|
|
121
|
+
arising from or in connection with the Software or its use, whether in contract,
|
|
122
|
+
tort or otherwise, including loss of data, profit or business. This does not
|
|
123
|
+
limit liability for intent or deliberate recklessness, or any liability that
|
|
124
|
+
cannot be limited under mandatory law, including consumer protection law.
|
|
125
|
+
|
|
126
|
+
11. Governing law
|
|
127
|
+
|
|
128
|
+
This licence is governed by the laws of the Netherlands. Disputes are submitted
|
|
129
|
+
to the competent court in the Netherlands, without limiting any mandatory right
|
|
130
|
+
a consumer has to bring proceedings in, or rely on the law of, their own country.
|
|
131
|
+
|
|
132
|
+
Contact: hello@worktrust.io
|
package/README.md
CHANGED
|
@@ -120,7 +120,7 @@ npx worktrust@latest history --rebuild re-measure them with the current rules;
|
|
|
120
120
|
npx worktrust@latest history --from-archive [dir] send what your local archive holds of sessions your AI apps
|
|
121
121
|
already deleted, marked as from your archive: a day this computer's key signed as
|
|
122
122
|
measured, every other day as reconstructed (shows the plan, asks first)
|
|
123
|
-
npx worktrust@latest history --export <path> a ChatGPT or Claude data export (the zip, its folder
|
|
123
|
+
npx worktrust@latest history --export <path> a ChatGPT or Claude data export (the zip, its folder, conversations.json, a conversations part or the manifest)
|
|
124
124
|
as history: hours reconstructed from its clocks, never verified (shows the plan, asks first)
|
|
125
125
|
npx worktrust@latest history --from-folder <dir> your own copy of an AI app's session folder (a backed-up ~/.claude/projects,
|
|
126
126
|
~/.codex/sessions): the days this computer's sessions lack, reconstructed (asks first)
|
|
@@ -300,9 +300,23 @@ sends them with one difference: WorkTrust **replaces** this computer's earlier l
|
|
|
300
300
|
Export data) or a Claude data export (Settings → Privacy → Export data) onto your record as history, so years of
|
|
301
301
|
browser chats appear with their months, days and rhythm. The path is the zip the vendor sent you, the folder it was
|
|
302
302
|
unpacked into, or `conversations.json` itself; the vendor is told by the file's shape. Of the zip only
|
|
303
|
-
`conversations.json` is opened: the profile, memories and files beside it are never inflated.
|
|
304
|
-
|
|
305
|
-
|
|
303
|
+
`conversations.json` is opened: the profile, memories and files beside it are never inflated. Claude on the web now
|
|
304
|
+
sends a manifest and one zip per category (0.10.17): give a `conversations-000.zip` part, the folder holding the
|
|
305
|
+
`conversations-*.zip` parts (read in part order), or the manifest json, which is read only for the file names of its
|
|
306
|
+
conversations parts lying beside it; its download links are never followed. The other parts (memories, projects,
|
|
307
|
+
artifacts, frames, design chats, the light metadata) are refused by name and never opened.
|
|
308
|
+
|
|
309
|
+
Any size (0.10.18): the file is read as a stream, a conversation at a time, so a gigabyte export reads in a bounded
|
|
310
|
+
memory; it keeps of each conversation only the clocks, roles, ids and a model name it needs (the counter, the text it
|
|
311
|
+
reads here). Bounds: a file to 64 GB, two million conversations, nesting to 64; a single conversation whose kept
|
|
312
|
+
fields pass 8 MB is left out and counted, never the export. A zip over 4 GB (zip64) is refused by name: unpack it and
|
|
313
|
+
give `conversations.json`. Every 64 MB a line on stderr says how many megabytes and conversations were read, nothing else.
|
|
314
|
+
|
|
315
|
+
- **What goes.** Per run of activity in a conversation (0.10.17; before, one line per conversation and day): a run
|
|
316
|
+
ends where two messages lie more than thirty minutes apart, the gap after which the hook calls a stretch finished, so a
|
|
317
|
+
conversation left open for hours is two windows, not one. Cutting changes the windows and never the total: the five
|
|
318
|
+
minutes the clock counts after a message stay with the run before the gap. A day of one run keeps the name it had; a
|
|
319
|
+
cut day replaces the one line an earlier send made of it. Each run: a neutral line (`A conversation in ChatGPT`, `A conversation in Claude on
|
|
306
320
|
the web`), the seconds between the messages' clocks counted the way a measured day is (a pause over five minutes is
|
|
307
321
|
not work; a day is cut at eight hours), your own turns as a count, the model's name where the export carries one, when
|
|
308
322
|
it began and ended, and a stable name made of the conversation's id and the day, so the same export sent twice, or
|
|
@@ -429,5 +443,8 @@ Security reports: see [SECURITY.md](SECURITY.md).
|
|
|
429
443
|
|
|
430
444
|
## Licence
|
|
431
445
|
|
|
432
|
-
|
|
433
|
-
|
|
446
|
+
Use is allowed under the WorkTrust CLI Licence; copying, modifying or redistributing the code is not. See LICENSE.
|
|
447
|
+
The licence is free of charge. `preserve` (with `--apply`, `--archive`, `--verify` and `--summary`) is for everyone,
|
|
448
|
+
with or without an account; every other feature is for WorkTrust account holders, and without an account the rest may
|
|
449
|
+
only be installed and evaluated. The archives and files the CLI makes from your own data are yours.
|
|
450
|
+
From 0.10.18 on; versions up to and including 0.10.17 were published under the MIT licence and stay under it.
|
package/SECURITY.md
CHANGED
|
@@ -59,10 +59,14 @@ potentially identifying metadata; see the README's complete payload list.
|
|
|
59
59
|
|
|
60
60
|
Agent-shell detection is a workflow safeguard, not protection from malicious same-user software.
|
|
61
61
|
Agents must follow AGENTS.md. From 0.10.12 the bare invocation is covered by the same guard as explicit
|
|
62
|
-
person-only commands; help and version are safe informational paths.
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
62
|
+
person-only commands; help and version are safe informational paths. An export is read as a stream
|
|
63
|
+
(0.10.18) through a JSON walk of our own that keeps a fixed set of fields per conversation and skips the
|
|
64
|
+
rest unbuilt; a zip entry is inflated as a stream. Bounds: a file (bare or zip) to 64 GiB, a zip directory
|
|
65
|
+
to 8 MiB, nesting to 64, conversations to 2,000,000, a conversation's kept fields to 8 MiB (the counter's
|
|
66
|
+
text, read locally, to 64 MiB; a conversation past its budget is left out and counted), an export held whole
|
|
67
|
+
to 512 MiB of kept fields; zip64 is refused. Peak memory is bounded by the chunk in flight and one
|
|
68
|
+
conversation's kept fields (`scripts/test-export-memory.mjs` holds it under 200 MiB for a gigabyte). These
|
|
69
|
+
are resource bounds, not a sandbox or independent validation of the export's truth.
|
|
66
70
|
|
|
67
71
|
## Verifying a version
|
|
68
72
|
|
package/chat-exports.mjs
CHANGED
|
@@ -4,22 +4,35 @@
|
|
|
4
4
|
* `conversations.json` in it. The counter reads it for signals; the session hook reads it as HISTORY, so a person's
|
|
5
5
|
* years of browser chats appear on their record. One reader here, imported by both, so the two cannot drift.
|
|
6
6
|
*
|
|
7
|
+
* READ AS A STREAM (0.10.18, owner 2026-10-17: a real 375 MB export was refused at the old 64 MB ceiling, and some
|
|
8
|
+
* exports run past 10 GB). The file is never read whole: its bytes pass through a small JSON walk of our own that keeps,
|
|
9
|
+
* per conversation, only the fields the two readers use (the conversation's id and clock; per message the role, the
|
|
10
|
+
* clock, ChatGPT's model name and, for ChatGPT, the tree's parent links to find the kept branch) and skips every other
|
|
11
|
+
* value as it passes, never building it. A zip entry is inflated as a stream the same way. One conversation lives in
|
|
12
|
+
* memory at a time; a conversation whose kept fields outgrow their budget is left out and counted, never the export.
|
|
13
|
+
* The counter alone asks for the messages' text (`text: true`), read on this computer for its rubric as it always was;
|
|
14
|
+
* the hook never does. A title is read for the --exclude test alone and dropped there.
|
|
15
|
+
*
|
|
7
16
|
* WHAT IT NEVER DOES. It opens nothing of the zip but `conversations.json` (a vendor's export carries the account's
|
|
8
|
-
* profile, memories and files beside it: those are never inflated).
|
|
9
|
-
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
17
|
+
* profile, memories and files beside it: those are never inflated). Claude's split export (2026-10-17: a manifest and one
|
|
18
|
+
* zip per category) is read through its `conversations-NNN.zip` parts alone, in part order; the manifest only names them,
|
|
19
|
+
* its links are never fetched, and every other category's zip is refused by name. To the hook it hands clocks, roles and
|
|
20
|
+
* a model name only: the conversation's title and every message stay here. Hours from an export are RECONSTRUCTED, never
|
|
21
|
+
* measured: the file can be edited before it is read and no device key witnessed it, so the record shows them with their
|
|
22
|
+
* origin and never in verified hours.
|
|
12
23
|
*
|
|
13
|
-
* readExport(path) { vendor, conversations, digest }
|
|
14
|
-
* exportSessions(path, …) one session per conversation, in the transcript's own shape
|
|
24
|
+
* readExport(path) Promise of { vendor, conversations, digest }: every conversation's kept fields, bounded
|
|
25
|
+
* exportSessions(path, …) async: one session per conversation as it streams, in the transcript's own shape
|
|
15
26
|
* exportRef(vendor, id, day) the stretch's stable name: the same on any computer, so a second send is the same line
|
|
16
|
-
*
|
|
27
|
+
* exportRunRef(vendor, id, day, index) one run's name, when the hook cuts a conversation-day into runs (0.10.17)
|
|
28
|
+
* findExports(dir) Promise of the exports lying in one folder (its top level only), by path and vendor
|
|
17
29
|
*/
|
|
18
30
|
import { createHash } from "node:crypto";
|
|
31
|
+
import { once } from "node:events";
|
|
19
32
|
import { existsSync, openSync, readSync, closeSync, fstatSync, readdirSync, statSync, constants } from "node:fs";
|
|
20
33
|
import { homedir } from "node:os";
|
|
21
|
-
import { basename, join } from "node:path";
|
|
22
|
-
import {
|
|
34
|
+
import { basename, dirname, join } from "node:path";
|
|
35
|
+
import { createInflateRaw } from "node:zlib";
|
|
23
36
|
|
|
24
37
|
/** The two vendors, by the client name the door reads as their product (chatgpt → chatgpt_web, claude-web → claude_web). */
|
|
25
38
|
export const EXPORT_CLIENTS = ["chatgpt", "claude-web"];
|
|
@@ -30,10 +43,14 @@ export const exportProductName = (vendor) => PRODUCT_NAMES[vendor] ?? vendor;
|
|
|
30
43
|
const FILE = "conversations.json";
|
|
31
44
|
const expand = (path) => String(path).replace(/^~(?=\/|$)/, homedir());
|
|
32
45
|
|
|
33
|
-
//
|
|
34
|
-
//
|
|
35
|
-
|
|
36
|
-
|
|
46
|
+
// THE BOUNDS (0.10.18). A file, bare or a zip, up to 64 GB, read as a stream; a zip's directory 8 MB; up to 64 parts; a
|
|
47
|
+
// manifest 1 MB; nesting 64 deep; up to 2,000,000 conversations. Per conversation the KEPT fields (never the file's bytes)
|
|
48
|
+
// up to 8 MB (the counter's text, read here alone, 64 MB): past that the conversation is left out and counted. An export
|
|
49
|
+
// held whole (readExport) stops at 512 MB of kept fields. A zip64 archive (one over 4 GB) is refused by name: unpack it
|
|
50
|
+
// and give conversations.json itself. No limit override.
|
|
51
|
+
export const EXPORT_LIMITS = Object.freeze({ fileBytes: 64 * 1024 ** 3, directoryBytes: 8 * 1024 ** 2, depth: 64, conversations: 2_000_000,
|
|
52
|
+
conversationBytes: 8 * 1024 ** 2, conversationTextBytes: 64 * 1024 ** 2, keptBytes: 512 * 1024 ** 2, keyBytes: 256, parts: 64, manifestBytes: 1024 ** 2 });
|
|
53
|
+
const PROGRESS_BYTES = 64 * 1024 ** 2, CHUNK_BYTES = 1024 ** 2;
|
|
37
54
|
const limit = (condition, label) => { if (!condition) throw Error(`export limit exceeded: ${label}`); };
|
|
38
55
|
const damaged = (condition) => { if (!condition) throw Error("a damaged zip archive"); };
|
|
39
56
|
/** One descriptor, bounded reads: reject special files/short reads and bound concurrent file growth. */
|
|
@@ -45,22 +62,48 @@ function openExport(path, maximum) {
|
|
|
45
62
|
limit(stat.size <= maximum, "file bytes");
|
|
46
63
|
const readAt = (at, length, end = stat.size) => {
|
|
47
64
|
damaged(Number.isSafeInteger(at) && Number.isSafeInteger(length) && at >= 0 && length >= 0 && at + length <= end && end <= stat.size);
|
|
48
|
-
const buffer = Buffer.
|
|
65
|
+
const buffer = Buffer.allocUnsafe(length);
|
|
49
66
|
let got = 0;
|
|
50
67
|
while (got < length) { const count = readSync(fd, buffer, got, length - got, at + got); damaged(count > 0); got += count; }
|
|
51
68
|
return buffer;
|
|
52
69
|
};
|
|
53
|
-
|
|
70
|
+
/** The bytes from `at`, a megabyte at a time into ONE buffer (the walk copies what it keeps, so the resident set stays flat), under the same checks: a file that shrinks while it is read is damaged. */
|
|
71
|
+
const chunksOf = function* (at, length, end) {
|
|
72
|
+
const buffer = Buffer.allocUnsafe(Math.min(CHUNK_BYTES, length));
|
|
73
|
+
for (let done = 0; done < length; done += CHUNK_BYTES) {
|
|
74
|
+
const count = Math.min(CHUNK_BYTES, length - done);
|
|
75
|
+
damaged(Number.isSafeInteger(at + done) && at >= 0 && at + done + count <= end && end <= stat.size);
|
|
76
|
+
let got = 0;
|
|
77
|
+
while (got < count) { const read = readSync(fd, buffer, got, count - got, at + done + got); damaged(read > 0); got += read; }
|
|
78
|
+
yield buffer.subarray(0, count);
|
|
79
|
+
}
|
|
80
|
+
};
|
|
81
|
+
return { fd, size: stat.size, readAt, chunksOf, close: () => closeSync(fd) };
|
|
54
82
|
} catch (error) { closeSync(fd); throw error; }
|
|
55
83
|
}
|
|
84
|
+
/** A deflated entry, inflated as it is read: never more than a chunk of it in memory, its length held to the directory's word. */
|
|
85
|
+
async function* inflated(chunks, uncompressed) {
|
|
86
|
+
const inflate = createInflateRaw();
|
|
87
|
+
let total = 0;
|
|
88
|
+
const feed = (async () => { for (const chunk of chunks) { if (inflate.destroyed) return; if (!inflate.write(chunk)) await once(inflate, "drain"); } inflate.end(); })().catch(() => null);
|
|
89
|
+
try {
|
|
90
|
+
for await (const piece of inflate) { total += piece.length; damaged(total <= uncompressed); yield piece; }
|
|
91
|
+
} catch (error) {
|
|
92
|
+
if (/^a damaged zip|^export limit/.test(error?.message ?? "")) throw error;
|
|
93
|
+
throw Error("a damaged zip archive or export limit exceeded: expanded entry bytes");
|
|
94
|
+
} finally { inflate.destroy(); await feed; }
|
|
95
|
+
damaged(total === uncompressed);
|
|
96
|
+
}
|
|
56
97
|
|
|
57
98
|
/**
|
|
58
|
-
* A ZIP, READ FOR
|
|
59
|
-
* directory walked for the
|
|
60
|
-
* zip64 archive, is refused by name). Nothing else in the archive is touched.
|
|
99
|
+
* A ZIP, READ FOR ITS CONVERSATIONS ENTRIES. No dependency: the end-of-central-directory record is found from the back, the
|
|
100
|
+
* central directory walked for the names `accept` takes, and only those entries' bytes read (stored or deflated, as a
|
|
101
|
+
* stream; anything else, and a zip64 archive, is refused by name), in part order. Nothing else in the archive is touched.
|
|
102
|
+
* Two entries of one name are ambiguous and refused. The descriptor stays open until `close`, so the bytes read are the
|
|
103
|
+
* bytes the directory described.
|
|
61
104
|
*/
|
|
62
|
-
function
|
|
63
|
-
const {
|
|
105
|
+
function zipEntries(path, accept) {
|
|
106
|
+
const { size, readAt, chunksOf, close } = openExport(path, EXPORT_LIMITS.fileBytes);
|
|
64
107
|
try {
|
|
65
108
|
const tailLength = Math.min(size, 65_557);
|
|
66
109
|
const tail = readAt(size - tailLength, tailLength);
|
|
@@ -68,13 +111,14 @@ function zipEntry(path, wanted) {
|
|
|
68
111
|
for (let at = tail.length - 22; at >= 0; at -= 1) if (tail.readUInt32LE(at) === 0x06054b50 && at + 22 + tail.readUInt16LE(at + 20) === tail.length) { eocd = at; break; }
|
|
69
112
|
if (eocd < 0) throw Error(`not a zip archive: ${path}`);
|
|
70
113
|
const entries = tail.readUInt16LE(eocd + 10), directorySize = tail.readUInt32LE(eocd + 12), directoryAt = tail.readUInt32LE(eocd + 16);
|
|
71
|
-
if (entries === 0xffff || directorySize === 0xffffffff || directoryAt === 0xffffffff) throw Error(`a zip64 archive is not read: ${path}`);
|
|
114
|
+
if (entries === 0xffff || directorySize === 0xffffffff || directoryAt === 0xffffffff) throw Error(`a zip64 archive is not read: ${path} (unpack it and give its ${FILE})`);
|
|
72
115
|
if (tail.readUInt16LE(eocd + 4) !== 0 || tail.readUInt16LE(eocd + 6) !== 0) throw Error("unsupported zip: multiple disks");
|
|
73
116
|
damaged(tail.readUInt16LE(eocd + 8) === entries);
|
|
74
117
|
limit(directorySize <= EXPORT_LIMITS.directoryBytes, "zip directory bytes");
|
|
75
118
|
damaged(directoryAt + directorySize === size - tailLength + eocd);
|
|
76
119
|
const directory = readAt(directoryAt, directorySize);
|
|
77
|
-
let at = 0
|
|
120
|
+
let at = 0;
|
|
121
|
+
const selected = new Map();
|
|
78
122
|
for (let index = 0; index < entries; index += 1) {
|
|
79
123
|
damaged(at + 46 <= directory.length && directory.readUInt32LE(at) === 0x02014b50);
|
|
80
124
|
const method = directory.readUInt16LE(at + 10), compressed = directory.readUInt32LE(at + 20), uncompressed = directory.readUInt32LE(at + 24);
|
|
@@ -83,130 +127,341 @@ function zipEntry(path, wanted) {
|
|
|
83
127
|
damaged(at + 46 + nameLength + extraLength + commentLength <= directory.length);
|
|
84
128
|
const name = directory.toString("utf8", at + 46, at + 46 + nameLength);
|
|
85
129
|
at += 46 + nameLength + extraLength + commentLength;
|
|
86
|
-
if (basename(name)
|
|
87
|
-
if (compressed === 0xffffffff || uncompressed === 0xffffffff || localAt === 0xffffffff) throw Error(`a zip64 entry is not read: ${path}`);
|
|
88
|
-
limit(compressed <= EXPORT_LIMITS.compressedBytes, "compressed entry bytes");
|
|
89
|
-
limit(uncompressed <= EXPORT_LIMITS.jsonBytes, "expanded entry bytes");
|
|
130
|
+
if (!accept(basename(name)) || name.endsWith("/")) continue;
|
|
131
|
+
if (compressed === 0xffffffff || uncompressed === 0xffffffff || localAt === 0xffffffff) throw Error(`a zip64 entry is not read: ${path} (unpack it and give its ${FILE})`);
|
|
90
132
|
if ((flags & (1 | 64)) !== 0 || disk !== 0) throw Error("unsupported zip: encrypted or multi-disk entry");
|
|
91
|
-
damaged(selected
|
|
92
|
-
selected
|
|
133
|
+
damaged(!selected.has(basename(name)));
|
|
134
|
+
selected.set(basename(name), { name, method, flags, compressed, uncompressed, localAt });
|
|
93
135
|
}
|
|
94
136
|
damaged(at === directory.length);
|
|
95
|
-
|
|
96
|
-
const
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
} finally { closeSync(fd); }
|
|
137
|
+
limit(selected.size <= EXPORT_LIMITS.parts, "conversations parts");
|
|
138
|
+
const parts = [...selected.keys()].sort(byPart).map((key) => {
|
|
139
|
+
const { name, method, flags, compressed, uncompressed, localAt } = selected.get(key);
|
|
140
|
+
const local = readAt(localAt, 30, directoryAt);
|
|
141
|
+
damaged(local.readUInt32LE(0) === 0x04034b50 && local.readUInt16LE(8) === method && local.readUInt16LE(6) === flags);
|
|
142
|
+
const nameLength = local.readUInt16LE(26), dataAt = localAt + 30 + nameLength + local.readUInt16LE(28);
|
|
143
|
+
damaged(readAt(localAt + 30, nameLength, directoryAt).toString("utf8") === name);
|
|
144
|
+
if (!(flags & 8)) damaged(local.readUInt32LE(18) === compressed && local.readUInt32LE(22) === uncompressed);
|
|
145
|
+
damaged(dataAt + compressed <= directoryAt);
|
|
146
|
+
if (method === 0) damaged(compressed === uncompressed);
|
|
147
|
+
else if (method !== 8) throw Error(`${basename(name)} in ${path} is compressed in a way this reader does not open (method ${method})`);
|
|
148
|
+
return { path, bytes: uncompressed, chunks: () => (method === 0 ? chunksOf(dataAt, compressed, directoryAt) : inflated(chunksOf(dataAt, compressed, directoryAt), uncompressed)) };
|
|
149
|
+
});
|
|
150
|
+
return { parts, close };
|
|
151
|
+
} catch (error) { close(); throw error; }
|
|
111
152
|
}
|
|
112
153
|
|
|
113
|
-
/**
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
154
|
+
/**
|
|
155
|
+
* THE CONVERSATIONS, AND ONLY THEM (owner, 2026-10-17). Inside a zip: `conversations.json`, or a part-numbered
|
|
156
|
+
* `conversations-000.json`. Claude on the web now hands a person a manifest and one zip per category; of those only the
|
|
157
|
+
* `conversations-NNN.zip` parts are read. The other categories (memories, projects, artifacts, frames, design chats, the
|
|
158
|
+
* light metadata) are refused by their name, never opened.
|
|
159
|
+
*/
|
|
160
|
+
const CONVERSATIONS_ENTRY = /^conversations(?:[-_]\d{1,6})?\.json$/i;
|
|
161
|
+
const CONVERSATIONS_PART = /^conversations[-_](\d{1,6})\.zip$/i;
|
|
162
|
+
const OTHER_PART = /^(light_metadata|projects|memories|design_chats|frames|artifact_data|users?|attachments?)[-_]\d{1,6}\.zip$/i;
|
|
163
|
+
const partOf = (name) => Number(/(\d{1,6})\.(?:json|zip)$/i.exec(name)?.[1] ?? -1);
|
|
164
|
+
const byPart = (a, b) => partOf(a) - partOf(b) || a.localeCompare(b);
|
|
165
|
+
const refuseOther = (path) => { const other = OTHER_PART.exec(basename(path)); if (other) throw Error(`${basename(path)} is the export's ${other[1].toLowerCase()} part: only its conversations parts are read`); };
|
|
166
|
+
/** One conversations zip: its conversations entries, or a refusal that names the zip. */
|
|
167
|
+
function zipParts(path, given) {
|
|
168
|
+
refuseOther(path);
|
|
169
|
+
const { parts, close } = zipEntries(path, (name) => CONVERSATIONS_ENTRY.test(name));
|
|
170
|
+
if (parts.length === 0) { close(); throw Error(`no ${FILE} in ${given}`); }
|
|
171
|
+
return { parts, close };
|
|
121
172
|
}
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
173
|
+
/** A folder's conversations zips (conversations-000.zip, -001.zip …), in part order; nothing else of the folder is read. */
|
|
174
|
+
const folderParts = (dir) => { let names = []; try { names = readdirSync(dir); } catch { return []; } return names.filter((name) => CONVERSATIONS_PART.test(name)).sort(byPart).map((name) => join(dir, name)); };
|
|
175
|
+
/**
|
|
176
|
+
* THE MANIFEST (Claude on the web, version "1.0"): a list of `data_files` with a category, a part, a file name and a link.
|
|
177
|
+
* It is read for the file names of its conversations parts alone, which must lie beside it; a link in it is NEVER fetched
|
|
178
|
+
* (this reader opens no network), and a name that is not a plain conversations zip in the same folder is refused.
|
|
179
|
+
*/
|
|
180
|
+
function manifestParts(manifest, path) {
|
|
181
|
+
const files = Array.isArray(manifest?.data_files) ? manifest.data_files : null;
|
|
182
|
+
if (!files) throw Error(`${path} is neither an export nor its manifest`);
|
|
183
|
+
limit(files.length <= 10_000, "manifest entries");
|
|
184
|
+
const wanted = files.filter((file) => file && typeof file === "object" && file.category === "conversations").map((file) => String(file.filename ?? ""));
|
|
185
|
+
if (wanted.length === 0) throw Error(`${path} names no conversations part`);
|
|
186
|
+
const missing = [], found = [];
|
|
187
|
+
for (const name of wanted.sort(byPart)) {
|
|
188
|
+
if (!/^[\w.-]{1,200}$/.test(name) || !CONVERSATIONS_PART.test(name)) throw Error(`${path} names a conversations part that is not a plain conversations zip beside it`);
|
|
189
|
+
const at = join(dirname(path), name);
|
|
190
|
+
if (existsSync(at)) found.push(at); else missing.push(name);
|
|
134
191
|
}
|
|
192
|
+
if (missing.length > 0) throw Error(`the manifest names conversations parts that are not beside it: ${missing.join(", ")} (download them into the same folder; the links in the manifest are never followed)`);
|
|
193
|
+
return found;
|
|
194
|
+
}
|
|
195
|
+
/** Several zips' parts together: one export, inside one budget, every descriptor closed together. */
|
|
196
|
+
function zipsParts(zips, given) {
|
|
197
|
+
limit(zips.length <= EXPORT_LIMITS.parts, "conversations parts");
|
|
198
|
+
const opened = [];
|
|
199
|
+
try {
|
|
200
|
+
for (const zip of zips) opened.push(zipParts(zip, given));
|
|
201
|
+
const parts = opened.flatMap((one) => one.parts);
|
|
202
|
+
limit(parts.length <= EXPORT_LIMITS.parts && parts.reduce((sum, part) => sum + part.bytes, 0) <= EXPORT_LIMITS.fileBytes, "conversations parts");
|
|
203
|
+
return { parts, close: () => { for (const one of opened) one.close(); } };
|
|
204
|
+
} catch (error) { for (const one of opened) one.close(); throw error; }
|
|
135
205
|
}
|
|
136
206
|
|
|
137
|
-
/**
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
207
|
+
/** Every conversations part, wherever the given path keeps them (a folder, the file itself, a zip, a part, or the manifest), each a source of chunks. */
|
|
208
|
+
function exportParts(given) {
|
|
209
|
+
let path = expand(given);
|
|
210
|
+
let stat; try { stat = statSync(path); } catch { throw Error(`export not found: ${given}`); }
|
|
211
|
+
if (stat.isDirectory()) {
|
|
212
|
+
if (existsSync(join(path, FILE))) path = join(path, FILE);
|
|
213
|
+
else {
|
|
214
|
+
const zips = folderParts(path);
|
|
215
|
+
if (zips.length === 0) throw Error(`no ${FILE} in ${given}`);
|
|
216
|
+
return zipsParts(zips, given);
|
|
217
|
+
}
|
|
218
|
+
}
|
|
219
|
+
if (/\.zip$/i.test(path)) return zipParts(path, given);
|
|
220
|
+
const file = openExport(path, EXPORT_LIMITS.fileBytes);
|
|
221
|
+
let manifest;
|
|
222
|
+
try {
|
|
223
|
+
// A json holding an object, not an array of conversations, may be the vendor's manifest: read for its conversations parts alone.
|
|
224
|
+
if (basename(path) === FILE || file.size > EXPORT_LIMITS.manifestBytes || !/^\s*\{/.test(file.readAt(0, Math.min(64, file.size)).toString("utf8"))) return { parts: [{ path, bytes: file.size, chunks: () => file.chunksOf(0, file.size, file.size) }], close: file.close };
|
|
225
|
+
try { manifest = JSON.parse(file.readAt(0, file.size).toString("utf8")); } catch { throw Error(`export unreadable: ${path}`); }
|
|
226
|
+
} catch (error) { file.close(); throw error; }
|
|
227
|
+
file.close();
|
|
228
|
+
return zipsParts(manifestParts(manifest, path), given);
|
|
142
229
|
}
|
|
143
230
|
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
231
|
+
// ─── THE WALK: an incremental JSON tokenizer that keeps a plan's fields and skips the rest without building it ───
|
|
232
|
+
/** A kept field that held an object or array where a scalar was expected: String() and Number() of it read as JSON.parse's object would. */
|
|
233
|
+
const NONSCALAR = Object.freeze({});
|
|
234
|
+
/** An element whose kept fields outgrew their budget: handed over as this, counted, never built. */
|
|
235
|
+
const OVERSIZED = Symbol("oversized");
|
|
236
|
+
const S = "scalar", T = "text"; // a scalar kept as it is; a text kept as a string for the counter, as its byte length for the hook
|
|
237
|
+
const NODE = { keys: { parent: S, message: { keys: { author: { keys: { role: S } }, create_time: S, content: { keys: { parts: { items: T } } }, metadata: { keys: { model_slug: S, is_visually_hidden_from_conversation: S } } } } } };
|
|
238
|
+
const CONVERSATION = { keys: { title: S, name: S, conversation_id: S, id: S, uuid: S, create_time: S, created_at: S, current_node: S, mapping: { any: NODE },
|
|
239
|
+
chat_messages: { items: { keys: { sender: S, created_at: S, text: T, content: { items: { keys: { type: S, text: T } } } } } } } };
|
|
240
|
+
const malformed = () => { throw Object.assign(Error("malformed"), { malformed: true }); };
|
|
241
|
+
const NUMBER = /^-?(?:0|[1-9]\d*)(?:\.\d+)?(?:[eE][+-]?\d+)?$/;
|
|
242
|
+
const VALUE = 0, KEY = 1, COLON = 2, AFTER = 3, STRING = 4, TOKEN = 5, DONE = 6;
|
|
243
|
+
/**
|
|
244
|
+
* The walk over one part's bytes, chunk by chunk, in any chunking. The top level must be an array; each of its elements is
|
|
245
|
+
* handed to `onElement` as the fields of `plan` (a tree of `keys`, `any` for a dictionary, `items` for a list; `scalar`
|
|
246
|
+
* and `text` at the leaves), with what `budget` allows: past it the element's kept fields are dropped and it is handed over
|
|
247
|
+
* as OVERSIZED. Depth is bounded; a string's escapes and a number's shape are checked as JSON.parse would, kept or skipped.
|
|
248
|
+
*/
|
|
249
|
+
function walker(plan, { text, budget, onElement }) {
|
|
250
|
+
const stack = []; // { out, plan, key, kind, fresh, root }: out is the kept container, null while skipping
|
|
251
|
+
let state = VALUE, used = 0, over = false, index = 0, started = false, vPlan = null, tok = "";
|
|
252
|
+
let sRaw = null, sLen = 0, sEsc = 0, sKey = false, sKeep = false, sEscaped = false;
|
|
253
|
+
const top = () => stack[stack.length - 1];
|
|
254
|
+
const oversize = () => { over = true; for (let i = 1; i < stack.length; i += 1) { stack[i].out = null; stack[i].plan = null; stack[i].key = null; } sRaw = null; sKeep = false; };
|
|
255
|
+
const spend = (cost) => { used += cost; if (used > budget) oversize(); };
|
|
256
|
+
const store = (value, cost) => {
|
|
257
|
+
const frame = top();
|
|
258
|
+
if (frame.root) { onElement(over ? OVERSIZED : value, index); index += 1; used = 0; over = false; }
|
|
259
|
+
else if (frame.out !== null) {
|
|
260
|
+
if (frame.out instanceof Map) { if (frame.key !== null) frame.out.set(frame.key, value); }
|
|
261
|
+
else if (Array.isArray(frame.out)) frame.out.push(value);
|
|
262
|
+
else if (frame.key !== null) frame.out[frame.key] = value;
|
|
263
|
+
spend(cost + 16);
|
|
264
|
+
}
|
|
265
|
+
frame.key = null; vPlan = null; state = AFTER;
|
|
266
|
+
};
|
|
267
|
+
const open = (kind) => {
|
|
268
|
+
limit(stack.length < EXPORT_LIMITS.depth, "JSON depth");
|
|
269
|
+
const root = stack.length === 0, wanted = root ? { items: plan } : vPlan && typeof vPlan === "object" ? vPlan : null;
|
|
270
|
+
const out = root ? null : kind === "{" && wanted?.keys ? {} : kind === "{" && wanted?.any ? new Map() : kind === "[" && wanted?.items ? [] : null;
|
|
271
|
+
stack.push({ out, plan: root || out !== null ? wanted : null, key: null, kind, fresh: true, root });
|
|
272
|
+
vPlan = kind === "[" ? top().plan?.items ?? null : null; state = kind === "{" ? KEY : VALUE;
|
|
273
|
+
};
|
|
274
|
+
const close = (kind) => { const frame = stack.pop(); if (frame.kind !== kind) malformed(); if (frame.root) state = DONE; else store(frame.out ?? NONSCALAR, 0); };
|
|
275
|
+
const beginString = (key) => { state = STRING; sKey = key; sKeep = key ? top().out !== null : vPlan === S || (vPlan === T && text); sRaw = sKeep ? [] : null; sLen = 0; sEsc = 0; sEscaped = false; };
|
|
276
|
+
/** Kept bytes of the string in flight: a key past its bound stops being one; a value past the budget takes the element with it. */
|
|
277
|
+
const take = (chunk, from, to) => {
|
|
278
|
+
sLen += to - from;
|
|
279
|
+
if (!sKeep) return;
|
|
280
|
+
if (sKey) { if (sLen > EXPORT_LIMITS.keyBytes) { sKeep = false; sRaw = null; return; } } else spend(to - from);
|
|
281
|
+
if (sKeep) sRaw.push(Buffer.from(chunk.subarray(from, to))); // a copy: the chunk's buffer is reused for the next read
|
|
282
|
+
};
|
|
283
|
+
const finishString = () => {
|
|
284
|
+
const raw = sRaw ? Buffer.concat(sRaw) : null;
|
|
285
|
+
sRaw = null;
|
|
286
|
+
const decode = () => (sEscaped ? JSON.parse(`"${raw.toString("utf8")}"`) : raw.toString("utf8"));
|
|
287
|
+
if (sKey) {
|
|
288
|
+
const frame = top(), key = raw ? decode() : null;
|
|
289
|
+
if (frame.plan?.any && key !== null) { frame.key = key; vPlan = frame.plan.any; spend(raw.length); }
|
|
290
|
+
else if (frame.plan?.keys && key !== null && Object.hasOwn(frame.plan.keys, key)) { frame.key = key; vPlan = frame.plan.keys[key]; }
|
|
291
|
+
else { frame.key = null; vPlan = null; }
|
|
292
|
+
state = COLON;
|
|
293
|
+
} else if (sKeep) store(decode(), 0);
|
|
294
|
+
else store(vPlan === T ? sLen : NONSCALAR, 8); // a text the hook never reads: its byte length says whether there was one
|
|
295
|
+
};
|
|
296
|
+
const finishToken = () => {
|
|
297
|
+
let value;
|
|
298
|
+
if (tok === "true") value = true; else if (tok === "false") value = false; else if (tok === "null") value = null;
|
|
299
|
+
else if (NUMBER.test(tok)) value = Number(tok); else malformed();
|
|
300
|
+
tok = ""; store(value, 8);
|
|
301
|
+
};
|
|
302
|
+
const push = (chunk) => {
|
|
303
|
+
const n = chunk.length;
|
|
304
|
+
for (let i = 0; i < n; i += 1) {
|
|
305
|
+
const b = chunk[i];
|
|
306
|
+
if (state === STRING) {
|
|
307
|
+
if (sEsc === 0) {
|
|
308
|
+
let j = i; // the common run: bytes that are neither a quote, a backslash nor a control character, taken at once
|
|
309
|
+
while (j < n && chunk[j] !== 34 && chunk[j] !== 92 && chunk[j] >= 32) j += 1;
|
|
310
|
+
if (j > i) { take(chunk, i, j); if (j === n) break; i = j; }
|
|
311
|
+
if (chunk[i] === 34) { finishString(); continue; }
|
|
312
|
+
if (chunk[i] !== 92) malformed();
|
|
313
|
+
sEsc = 1; sEscaped = true; take(chunk, i, i + 1); continue;
|
|
314
|
+
}
|
|
315
|
+
if (sEsc === 1) { if (b === 117) sEsc = 2; else if (b === 34 || b === 92 || b === 47 || b === 98 || b === 102 || b === 110 || b === 114 || b === 116) sEsc = 0; else malformed(); }
|
|
316
|
+
else { if (!((b >= 48 && b <= 57) || (b >= 65 && b <= 70) || (b >= 97 && b <= 102))) malformed(); sEsc = sEsc === 5 ? 0 : sEsc + 1; }
|
|
317
|
+
take(chunk, i, i + 1);
|
|
318
|
+
continue;
|
|
319
|
+
}
|
|
320
|
+
if (state === TOKEN) {
|
|
321
|
+
if ((b >= 48 && b <= 57) || b === 45 || b === 43 || b === 46 || (b >= 97 && b <= 122) || b === 69) { tok += String.fromCharCode(b); if (tok.length > 4096) malformed(); continue; }
|
|
322
|
+
finishToken();
|
|
323
|
+
}
|
|
324
|
+
if (b === 32 || b === 10 || b === 13 || b === 9) continue;
|
|
325
|
+
if (state === VALUE) {
|
|
326
|
+
if (!started) { started = true; if (b === 123) throw Object.assign(Error("not an array"), { notArray: true }); if (b !== 91) malformed(); }
|
|
327
|
+
const frame = top();
|
|
328
|
+
if (b === 93) { if (frame?.kind === "[" && frame.fresh) { close("["); continue; } malformed(); }
|
|
329
|
+
if (frame) frame.fresh = false;
|
|
330
|
+
if (b === 34) beginString(false);
|
|
331
|
+
else if (b === 123) open("{");
|
|
332
|
+
else if (b === 91) open("[");
|
|
333
|
+
else if ((b >= 48 && b <= 57) || b === 45 || b === 116 || b === 102 || b === 110) { state = TOKEN; tok = String.fromCharCode(b); }
|
|
334
|
+
else malformed();
|
|
335
|
+
} else if (state === KEY) {
|
|
336
|
+
const frame = top();
|
|
337
|
+
if (b === 34) { frame.fresh = false; beginString(true); }
|
|
338
|
+
else if (b === 125 && frame.fresh) close("{");
|
|
339
|
+
else malformed();
|
|
340
|
+
} else if (state === COLON) { if (b !== 58) malformed(); state = VALUE; }
|
|
341
|
+
else if (state === AFTER) {
|
|
342
|
+
const frame = top();
|
|
343
|
+
if (b === 44) { if (frame.kind === "{") state = KEY; else { state = VALUE; vPlan = frame.plan?.items ?? null; } }
|
|
344
|
+
else if (b === 93) close("[");
|
|
345
|
+
else if (b === 125) close("{");
|
|
346
|
+
else malformed();
|
|
347
|
+
} else malformed(); // DONE: nothing may follow the array
|
|
348
|
+
}
|
|
349
|
+
};
|
|
350
|
+
const end = () => { if (state === TOKEN) finishToken(); if (state !== DONE) malformed(); };
|
|
351
|
+
return { push, end };
|
|
155
352
|
}
|
|
156
353
|
|
|
354
|
+
// ─── ONE CONVERSATION'S KEPT FIELDS → ITS LINES (the shapes the old whole-file reader produced, line for line) ───
|
|
157
355
|
/** The kept branch of one ChatGPT conversation, oldest first: from `current_node` up through the parents. */
|
|
158
|
-
function chatgptBranch(
|
|
159
|
-
const mapping =
|
|
160
|
-
const branch = [];
|
|
161
|
-
|
|
162
|
-
for (let id = conversation?.current_node; id && mapping[id] && !seen.has(id); id = mapping[id].parent) { seen.add(id); branch.push(mapping[id]); }
|
|
356
|
+
function chatgptBranch(kept) {
|
|
357
|
+
const mapping = kept.mapping instanceof Map ? kept.mapping : new Map();
|
|
358
|
+
const branch = [], seen = new Set();
|
|
359
|
+
for (let id = kept.current_node; id && mapping.get(String(id)) && !seen.has(String(id)); id = mapping.get(String(id))?.parent) { seen.add(String(id)); branch.push(mapping.get(String(id))); }
|
|
163
360
|
return branch.reverse();
|
|
164
361
|
}
|
|
165
362
|
/** A ChatGPT conversation's lines: the person and the model only; system prompts, tools and what the app hides are neither. */
|
|
166
|
-
function chatgptLines(
|
|
363
|
+
function chatgptLines(kept, text) {
|
|
167
364
|
const lines = [];
|
|
168
|
-
for (const node of chatgptBranch(
|
|
169
|
-
const m = node?.message;
|
|
170
|
-
const role = m?.author?.role;
|
|
365
|
+
for (const node of chatgptBranch(kept)) {
|
|
366
|
+
const m = node?.message, role = m?.author?.role;
|
|
171
367
|
if ((role !== "user" && role !== "assistant") || m?.metadata?.is_visually_hidden_from_conversation) continue;
|
|
172
|
-
const parts = Array.isArray(m?.content?.parts) ? m.content.parts
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
368
|
+
const parts = Array.isArray(m?.content?.parts) ? m.content.parts : [];
|
|
369
|
+
// With the text: the string parts joined. Without: the parts' byte lengths say whether the join is empty.
|
|
370
|
+
let body;
|
|
371
|
+
if (text) { body = parts.filter((part) => typeof part === "string").join("\n"); if (body.length === 0) continue; }
|
|
372
|
+
else { const lengths = parts.filter((part) => typeof part === "number"); if (lengths.length === 0 || (lengths.length === 1 && lengths[0] === 0)) continue; }
|
|
373
|
+
const seconds = Number(m?.create_time ?? kept.create_time);
|
|
176
374
|
const at = Number.isFinite(seconds) && seconds > 0 ? new Date(seconds * 1000).toISOString() : null;
|
|
177
375
|
const model = typeof m?.metadata?.model_slug === "string" ? m.metadata.model_slug : undefined;
|
|
178
|
-
lines.push({ type: role === "user" ? "user" : "assistant", timestamp: at,
|
|
376
|
+
lines.push({ type: role === "user" ? "user" : "assistant", timestamp: at, ...(role === "assistant" && model ? { model } : {}), ...(text ? { text: body } : {}) });
|
|
179
377
|
}
|
|
180
378
|
return lines;
|
|
181
379
|
}
|
|
182
|
-
/** A claude.ai conversation's messages
|
|
183
|
-
function claudeLines(
|
|
184
|
-
const messages = Array.isArray(
|
|
380
|
+
/** A claude.ai conversation's messages; the export names no model. */
|
|
381
|
+
function claudeLines(kept, text) {
|
|
382
|
+
const messages = Array.isArray(kept.chat_messages) ? kept.chat_messages : [];
|
|
185
383
|
return messages.map((m) => {
|
|
186
|
-
const
|
|
187
|
-
|
|
188
|
-
return { type: m?.sender === "human" ? "user" : "assistant", timestamp: at, message: { content: [{ type: "text", text }] } };
|
|
384
|
+
const body = !text ? undefined : typeof m?.text === "string" && m.text.length > 0 ? m.text : (Array.isArray(m?.content) ? m.content.filter((c) => c?.type === "text" && typeof c.text === "string").map((c) => c.text).join("\n") : "");
|
|
385
|
+
return { type: m?.sender === "human" ? "user" : "assistant", timestamp: m?.created_at ?? kept.created_at ?? null, ...(text ? { text: body } : {}) };
|
|
189
386
|
});
|
|
190
387
|
}
|
|
191
|
-
|
|
192
|
-
const
|
|
388
|
+
/** A kept line in the transcript's own shape, as the counter and the hook read every client's lines. */
|
|
389
|
+
const transcriptLine = ({ type, timestamp, model, text }) => ({ type, timestamp, message: { ...(model ? { model } : {}), content: [{ type: "text", text: text ?? "" }] } });
|
|
193
390
|
|
|
194
391
|
/**
|
|
195
|
-
*
|
|
196
|
-
* the
|
|
197
|
-
*
|
|
198
|
-
*
|
|
392
|
+
* THE EXPORT, ONE CONVERSATION AT A TIME: each part's bytes through the walk, the digest taken as they pass, the vendor
|
|
393
|
+
* told from the first conversation's shape and held to (a second shape, or an unknown one, refuses the export; `expect`
|
|
394
|
+
* refuses the other vendor by name), the title read for `excluded` and dropped there, a conversation whose kept fields
|
|
395
|
+
* outgrow their budget counted and left out. Yields one { vendor, id, lines } per conversation (an excluded or oversized
|
|
396
|
+
* one without an id, marked); `report` receives the path and vendor before the first conversation, the digest, the counts
|
|
397
|
+
* and the number of parts when the stream ends. Counts only on stderr.
|
|
199
398
|
*/
|
|
200
|
-
|
|
201
|
-
const {
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
399
|
+
async function* exportRecords(given, { expect = null, excluded = () => false, skipped = null, text = false, report = null } = {}) {
|
|
400
|
+
const { parts, close } = exportParts(given);
|
|
401
|
+
const path = parts[0].path, digest = createHash("sha256");
|
|
402
|
+
let vendor = null, count = 0, oversized = 0, bytes = 0, next = PROGRESS_BYTES, offset = 0;
|
|
403
|
+
if (report) { report.path = path; report.vendor = expect; report.digest = null; report.conversations = 0; report.oversized = 0; report.parts = parts.length; }
|
|
404
|
+
const one = (kept, index) => {
|
|
405
|
+
limit(count < EXPORT_LIMITS.conversations, "conversation count");
|
|
406
|
+
count += 1;
|
|
407
|
+
if (kept === OVERSIZED) { oversized += 1; return { vendor, oversized: true, lines: [] }; }
|
|
408
|
+
const shape = kept && typeof kept === "object" && kept !== NONSCALAR ? (kept.mapping instanceof Map || kept.mapping === NONSCALAR ? "chatgpt" : Array.isArray(kept.chat_messages) ? "claude-web" : "unknown") : "unknown";
|
|
409
|
+
if (vendor === null) vendor = shape;
|
|
410
|
+
if (shape === "unknown" || shape !== vendor) throw Error(`${path} is neither a ChatGPT nor a Claude export (or mixes the two)`);
|
|
411
|
+
if (expect && vendor !== expect) throw Error(`${path} is a ${exportProductName(vendor)} export, not ${exportProductName(expect)}`);
|
|
412
|
+
if (report) report.vendor = vendor;
|
|
413
|
+
const title = String((vendor === "chatgpt" ? kept.title : kept.name) ?? "");
|
|
414
|
+
if (excluded(title)) { skipped?.push(title.slice(0, 40)); return { vendor, excluded: true, lines: [] }; }
|
|
415
|
+
return { vendor, id: String((vendor === "chatgpt" ? kept.conversation_id ?? kept.id : kept.uuid) ?? `#${index}`), lines: vendor === "chatgpt" ? chatgptLines(kept, text) : claudeLines(kept, text) };
|
|
416
|
+
};
|
|
417
|
+
try {
|
|
418
|
+
for (const part of parts) {
|
|
419
|
+
let queue = [], seen = 0;
|
|
420
|
+
const walk = walker(CONVERSATION, { text, budget: text ? EXPORT_LIMITS.conversationTextBytes : EXPORT_LIMITS.conversationBytes, onElement: (kept, index) => queue.push([kept, offset + index]) });
|
|
421
|
+
const drain = function* () { const due = queue; queue = []; for (const [kept, index] of due) { seen = index + 1 - offset; yield one(kept, index); } };
|
|
422
|
+
try {
|
|
423
|
+
for await (const chunk of part.chunks()) {
|
|
424
|
+
digest.update(chunk); bytes += chunk.length; walk.push(chunk);
|
|
425
|
+
if (bytes >= next) { process.stderr.write(`export: ${Math.round(bytes / 1024 ** 2)} MB read, ${count} conversations\n`); next += PROGRESS_BYTES; }
|
|
426
|
+
yield* drain();
|
|
427
|
+
}
|
|
428
|
+
walk.end();
|
|
429
|
+
yield* drain();
|
|
430
|
+
} catch (error) {
|
|
431
|
+
if (error?.malformed) throw Error(`export unreadable: ${part.path}`);
|
|
432
|
+
if (error?.notArray) throw Error(`${part.path} must contain an array of conversations`);
|
|
433
|
+
throw error;
|
|
434
|
+
}
|
|
435
|
+
offset += seen;
|
|
436
|
+
}
|
|
437
|
+
} finally { close(); }
|
|
438
|
+
if (report) { report.vendor = vendor ?? expect; report.digest = digest.digest("hex"); report.conversations = count; report.oversized = oversized; }
|
|
439
|
+
}
|
|
440
|
+
|
|
441
|
+
/** One export, read whole but compact: its vendor by shape, one record per conversation (its kept lines; none for an excluded or oversized one), and the digest of its bytes (what `sent` is remembered by). */
|
|
442
|
+
export async function readExport(given, expect = null, { excluded = () => false, skipped = null, text = false } = {}) {
|
|
443
|
+
const report = {}, conversations = [];
|
|
444
|
+
let kept = 0;
|
|
445
|
+
for await (const record of exportRecords(given, { expect, excluded, skipped, text, report })) {
|
|
446
|
+
kept += 64 + (record.id?.length ?? 0) + record.lines.reduce((sum, line) => sum + 64 + (line.model?.length ?? 0) + (line.text?.length ?? 0), 0);
|
|
447
|
+
limit(kept <= EXPORT_LIMITS.keptBytes, "kept bytes");
|
|
448
|
+
conversations.push(record);
|
|
449
|
+
}
|
|
450
|
+
return { vendor: report.vendor, conversations, digest: report.digest, path: report.path, parts: report.parts, oversized: report.oversized };
|
|
451
|
+
}
|
|
452
|
+
|
|
453
|
+
/**
|
|
454
|
+
* Every conversation of an export as one session, as it streams: `key` names the vendor and the conversation's id,
|
|
455
|
+
* `lines` carry the transcript shape both readers measure. `excluded(title)` skips a conversation by its title before
|
|
456
|
+
* anything else of it is kept (the same `--exclude` as a project directory); the title itself goes no further than that
|
|
457
|
+
* call. `text` hands the counter the messages' text (read here, for its rubric); the hook leaves it out and gets clocks,
|
|
458
|
+
* roles and a model name. `given` is a path, or an export `readExport` already returned; `report` is filled as above.
|
|
459
|
+
*/
|
|
460
|
+
export async function* exportSessions(given, { expect = null, excluded = () => false, skipped = null, text = false, report = null } = {}) {
|
|
461
|
+
const records = typeof given === "string" ? exportRecords(given, { expect, excluded, skipped, text, report }) : given.conversations;
|
|
462
|
+
for await (const { vendor, id, lines } of records) {
|
|
463
|
+
if (lines.length === 0) continue; // excluded, left out as too large, or without a message
|
|
464
|
+
yield { key: `${vendor === "chatgpt" ? "chatgpt" : "claude.ai"}:${id}`, projectDir: vendor === "chatgpt" ? "chatgpt" : "claude-ai", basename: vendor === "chatgpt" ? "chatgpt" : "claude.ai", vendor, id, lines: lines.map(transcriptLine) };
|
|
210
465
|
}
|
|
211
466
|
}
|
|
212
467
|
|
|
@@ -216,23 +471,30 @@ export function* exportSessions(given, { expect = null, excluded = () => false,
|
|
|
216
471
|
* computers, is each line once. The conversation's id is a vendor's opaque id; the hash carries nothing readable.
|
|
217
472
|
*/
|
|
218
473
|
export const exportRef = (vendor, id, day) => createHash("sha256").update(`worktrust-export|${vendor}|${id}|${day}`).digest("base64url");
|
|
474
|
+
/**
|
|
475
|
+
* ONE RUN'S NAME (CLI 0.10.17, owner 2026-10-17): a conversation-day the hook cuts into runs of activity names each run by
|
|
476
|
+
* the same three and the run's index, in order from the day's first message. A day of one run keeps `exportRef`, the name
|
|
477
|
+
* it always had, so a day sent before the cut is the same line; a day of several runs names that old line as the one its
|
|
478
|
+
* runs replace (the hook's `supersedes_ref`).
|
|
479
|
+
*/
|
|
480
|
+
export const exportRunRef = (vendor, id, day, index) => createHash("sha256").update(`worktrust-export|${vendor}|${id}|${day}|run|${index}`).digest("base64url");
|
|
219
481
|
|
|
220
482
|
/**
|
|
221
483
|
* THE EXPORTS IN ONE FOLDER, its top level only (the Downloads folder, where a vendor's export lands): a zip holding
|
|
222
484
|
* `conversations.json`, a folder holding one, or the file itself. Each is read to tell its vendor and digest; a file
|
|
223
485
|
* that is none of these, or not an export, is passed over without a word.
|
|
224
486
|
*/
|
|
225
|
-
export function findExports(dir = join(homedir(), "Downloads")) {
|
|
487
|
+
export async function findExports(dir = join(homedir(), "Downloads")) {
|
|
226
488
|
const root = expand(dir);
|
|
227
489
|
let names = []; try { names = readdirSync(root); } catch { return []; }
|
|
228
490
|
const found = [];
|
|
229
491
|
for (const name of names.sort()) {
|
|
230
492
|
const path = join(root, name);
|
|
231
493
|
let stat; try { stat = statSync(path); } catch { continue; }
|
|
232
|
-
const candidate = stat.isDirectory() ? existsSync(join(path, FILE)) : /\.zip$/i.test(name) || name === FILE;
|
|
494
|
+
const candidate = stat.isDirectory() ? existsSync(join(path, FILE)) || folderParts(path).length > 0 : (/\.zip$/i.test(name) && !OTHER_PART.test(name)) || name === FILE;
|
|
233
495
|
if (!candidate) continue;
|
|
234
496
|
try {
|
|
235
|
-
const { vendor, conversations, digest } = readExport(path);
|
|
497
|
+
const { vendor, conversations, digest } = await readExport(path);
|
|
236
498
|
if (vendor && conversations.length > 0) found.push({ path, vendor, digest, conversations: conversations.length });
|
|
237
499
|
} catch { /* not an export, or not one this reader opens */ }
|
|
238
500
|
}
|
package/claude-mod.mjs
CHANGED
|
@@ -146,7 +146,7 @@ export function register(on) {
|
|
|
146
146
|
}
|
|
147
147
|
`;
|
|
148
148
|
|
|
149
|
-
export const PLUGIN_JSON = { name: MOD_NAME, version: MOD_VERSION, description: "WorkTrust: counts how you work with Claude Code (permission answers, effort levels, context peaks, subagents), never a word of the session. Counts stay on this computer until the WorkTrust CLI reports a session.", author: { name: "WorkTrust", url: "https://worktrust.io" }, homepage: "https://worktrust.io", license: "
|
|
149
|
+
export const PLUGIN_JSON = { name: MOD_NAME, version: MOD_VERSION, description: "WorkTrust: counts how you work with Claude Code (permission answers, effort levels, context peaks, subagents), never a word of the session. Counts stay on this computer until the WorkTrust CLI reports a session.", author: { name: "WorkTrust", url: "https://worktrust.io" }, homepage: "https://worktrust.io", license: "LicenseRef-WorkTrust-CLI" };
|
|
150
150
|
export const HOOKS_JSON = { modules: ["./register.js"] };
|
|
151
151
|
|
|
152
152
|
const dirs = (home) => ({ plugin: join(home, ".worktrust", "claude-mod"), counts: join(home, ".worktrust", "claude-mod-counts"), settings: join(home, ".claude", "settings.json") });
|
package/count-behaviour.mjs
CHANGED
|
@@ -622,16 +622,17 @@ const skippedConversations = [];
|
|
|
622
622
|
* both; a vendor's zip is opened for its conversations.json alone. Without the module an export flag is refused by name.
|
|
623
623
|
*/
|
|
624
624
|
const { exportSessions = null } = (await import("./chat-exports.mjs").catch(() => null)) ?? {};
|
|
625
|
-
|
|
625
|
+
// 0.10.18: the export streams, one conversation in memory at a time, with its text (`text: true`: this rubric reads it here).
|
|
626
|
+
async function* vendorExportSessions(paths, expect) {
|
|
626
627
|
if (paths.length > 0 && !exportSessions) throw Error("chat-exports.mjs is not beside this file: a data export cannot be read");
|
|
627
|
-
for (const given of paths) for (const session of exportSessions(given, { expect, excluded, skipped: skippedConversations })) {
|
|
628
|
+
for (const given of paths) for await (const session of exportSessions(given, { expect, excluded, skipped: skippedConversations, text: true })) {
|
|
628
629
|
if (expect === "chatgpt") chatgptConversations += 1; else exportConversations += 1;
|
|
629
630
|
yield session;
|
|
630
631
|
}
|
|
631
632
|
}
|
|
632
|
-
function* claudeExportSessions() { yield* vendorExportSessions(EXPORTS, "claude-web"); }
|
|
633
|
+
async function* claudeExportSessions() { yield* vendorExportSessions(EXPORTS, "claude-web"); }
|
|
633
634
|
let chatgptConversations = 0;
|
|
634
|
-
function* chatgptExportSessions() { yield* vendorExportSessions(CHATGPT_EXPORTS, "chatgpt"); }
|
|
635
|
+
async function* chatgptExportSessions() { yield* vendorExportSessions(CHATGPT_EXPORTS, "chatgpt"); }
|
|
635
636
|
/**
|
|
636
637
|
* THE READERS FOR CODEX AND ANTIGRAVITY live in transcript-readers.mjs beside this file (2026-10-04;
|
|
637
638
|
* the session hook imports the same file). Run alone, without it, the counter reads Claude Code,
|
|
@@ -709,7 +710,7 @@ const decodeLine = (buffer, start, end) => { try { return buffer.toString("utf8"
|
|
|
709
710
|
/** The digest of a transcript file, streamed: its path and every byte. */
|
|
710
711
|
function fileDigest(path) { const hash = createHash("sha256").update(`file:${path}\n`); for (const _ of fileLines(path, hash)) { /* the bytes are the input */ } return hash.digest("hex"); }
|
|
711
712
|
const sessionDigest = (session) => session.path ? fileDigest(session.path) : fingerprint(session);
|
|
712
|
-
function* sessions() {
|
|
713
|
+
async function* sessions() { // async since 0.10.18: an export streams through it
|
|
713
714
|
for (const file of transcriptFiles()) {
|
|
714
715
|
yield { key: file, projectDir: file.split("/").slice(-2, -1)[0] ?? "", basename: null, path: file };
|
|
715
716
|
}
|
|
@@ -849,7 +850,7 @@ const seenLines = new Set(); // v0.10: line identities already counted (a uuid o
|
|
|
849
850
|
// Validate the processed prefix before reuse. Edited/truncated/deleted sessions restart the
|
|
850
851
|
// scan; complete session boundaries preserve signals that depend on earlier turns.
|
|
851
852
|
const manifest = [];
|
|
852
|
-
for(const session of sessions())manifest.push(sessionDigest(session));
|
|
853
|
+
for await (const session of sessions())manifest.push(sessionDigest(session));
|
|
853
854
|
exportConversations = 0; chatgptConversations = 0; copilotSessionCount = 0; codexRolloutCount = 0; antigravityCount = 0; skippedDirs.length = 0; countedDirs.length = 0; skippedConversations.length = 0;
|
|
854
855
|
const checkpoint = flag("no-checkpoint") || flag("stretch-signals") || LABEL_OUT ? null : checkpointStore(
|
|
855
856
|
value("checkpoint") ?? join(homedir(), ".worktrust", "counter-checkpoint.json"),
|
|
@@ -867,7 +868,7 @@ const resumeAt=checkpoint?.saved?.next??0;
|
|
|
867
868
|
let sessionIndex=0, processed=0;
|
|
868
869
|
const maxSessions=Number(value("batch-sessions")??Number.POSITIVE_INFINITY);
|
|
869
870
|
if (value("batch-sessions") !== undefined && (!Number.isSafeInteger(maxSessions) || maxSessions < 1)) throw Error("batch-sessions must be a positive integer");
|
|
870
|
-
for (const session of sessions()) {
|
|
871
|
+
for await (const session of sessions()) {
|
|
871
872
|
// A file's digest is taken while its lines stream past and compared when the session ends.
|
|
872
873
|
if(!session.path && fingerprint(session)!==manifest[sessionIndex]) throw Error("Source changed during counting; rerun to resume safely");
|
|
873
874
|
if(sessionIndex++<resumeAt)continue;
|
package/log-session.mjs
CHANGED
|
@@ -270,7 +270,7 @@ const { callOf = (block) => ({ id: block.id, kinds: [], family: String(block.nam
|
|
|
270
270
|
const { DATABASE_SESSION = /(?!)/, databaseLines = () => null, databaseSessions = function* () {} } = (await import("./session-databases.mjs").catch(() => null)) ?? {};
|
|
271
271
|
// A WEB CHAT'S DATA EXPORT AS HISTORY (0.10.10, owner 2026-10-10): chat-exports.mjs reads a ChatGPT or Claude export (the counter
|
|
272
272
|
// imports the same reader); `--history --export <path>` sends its conversations as RECONSTRUCTED days, never measured, never tokens.
|
|
273
|
-
const { EXPORT_CLIENTS = [], EXPORT_TITLES = {}, exportSessions = null, exportRef = null,
|
|
273
|
+
const { EXPORT_CLIENTS = [], EXPORT_TITLES = {}, exportSessions = null, exportRef = null, exportRunRef = null, findExports = null } = (await import("./chat-exports.mjs").catch(() => null)) ?? {};
|
|
274
274
|
const { modCounts = () => null } = (await import("./claude-mod.mjs").catch(() => null)) ?? {}; // 0.10.15: the WorkTrust mod's live counts per Claude Code session and day; absent → no `live` group, never zeros
|
|
275
275
|
const liveOf = (file, day) => modCounts(String(file).split(/[\\/]/).pop().replace(/\.jsonl$/, ""), day) ?? codexLive(file, day); // 0.10.16: a Codex rollout's own
|
|
276
276
|
const { stackOfFiles = () => [], displayStackOfFiles = () => [] } = (await import("./stack-rules.mjs").catch(() => null)) ?? {}; // 0.10.11: core's stack rules and the display-only keys (tools_seen), generated (build-stack-rules --check); absent → nothing named, never a guess
|
|
@@ -736,7 +736,7 @@ function batchesOf(group, byDay) {
|
|
|
736
736
|
return out;
|
|
737
737
|
}
|
|
738
738
|
/** THE RECEIPTS' COUNTS (2026-10-10): what an export send replaced, how many AI-app lines lie on its days, how many were withdrawn; summed over the batches, read for the counts alone. */
|
|
739
|
-
const RECEIPTS = { new: 0, alreadyHeld: 0, filled: 0, replaced: 0, superseded: 0, unknownOnDays: 0, unknownWithdrawn: 0 }; // 2026-10-10: and what a re-measure did (new, held, filled, replaced), so the command can say it
|
|
739
|
+
const RECEIPTS = { new: 0, alreadyHeld: 0, filled: 0, replaced: 0, superseded: 0, recut: 0, unknownOnDays: 0, unknownWithdrawn: 0 }; // 0.10.17: recut, the one-envelope export lines its runs replaced // 2026-10-10: and what a re-measure did (new, held, filled, replaced), so the command can say it
|
|
740
740
|
const noteReceipt = (answer) => {
|
|
741
741
|
try {
|
|
742
742
|
const json = JSON.parse(answer.trim().startsWith("{") ? answer : answer.split("\n").find((row) => row.startsWith("data:"))?.slice(5) ?? "null");
|
|
@@ -862,7 +862,8 @@ function historyEntries(floor = null, all = false) {
|
|
|
862
862
|
|
|
863
863
|
/**
|
|
864
864
|
* A DATA EXPORT'S CONVERSATIONS AS HISTORY (0.10.10, owner 2026-10-10: "hours from an export count as reconstructed, shown,
|
|
865
|
-
* never in verified hours").
|
|
865
|
+
* never in verified hours"). Since 0.10.17 one line per RUN of activity in a conversation-day (runsOf, below), no longer one
|
|
866
|
+
* envelope per conversation and day. The same clock cuts them (a day, gaps capped at IDLE_CAP) and the same measure counts the
|
|
866
867
|
* person's own turns and the model, but the line says `reconstructed`: the file can be edited before it is read and no
|
|
867
868
|
* device key witnessed it. No tokens (the export carries none and none are estimated), no layers, no evidence; the
|
|
868
869
|
* title names the product and never the conversation; the stretch's name is the vendor, the conversation's id and the
|
|
@@ -870,37 +871,66 @@ function historyEntries(floor = null, all = false) {
|
|
|
870
871
|
* (20261009170000), so the line is cut there before it leaves.
|
|
871
872
|
*/
|
|
872
873
|
const RECONSTRUCTED_MAX = 28800;
|
|
874
|
+
/**
|
|
875
|
+
* WHERE A CONVERSATION-DAY IS CUT (CLI 0.10.17, owner 2026-10-17): the hook's own stretch-finished gap, IDLE_BEFORE_SWEEP
|
|
876
|
+
* (half an hour of quiet says a stretch is finished), never a new constant. Counting is unchanged: every gap between two
|
|
877
|
+
* messages, the person's or the model's, counts up to IDLE_CAP, and a gap the cut falls in is counted, as it always was,
|
|
878
|
+
* with the run BEFORE it (the five minutes the clock credits after a message). So the runs' seconds sum to exactly what
|
|
879
|
+
* the one envelope carried, and only the windows sharpen: a long-idle conversation no longer spans its own gap.
|
|
880
|
+
*/
|
|
881
|
+
const RUN_GAP = IDLE_BEFORE_SWEEP;
|
|
882
|
+
function runsOf(messages) {
|
|
883
|
+
const runs = [];
|
|
884
|
+
let current = null;
|
|
885
|
+
for (let i = 0; i < messages.length; i += 1) {
|
|
886
|
+
const gap = i === 0 ? null : messages[i].at - messages[i - 1].at;
|
|
887
|
+
if (current && gap > RUN_GAP) { current.seconds += Math.min(IDLE_CAP, Math.round(gap / 1000)); current = null; }
|
|
888
|
+
if (!current) { current = { messages: [], seconds: 0 }; runs.push(current); }
|
|
889
|
+
else current.seconds += Math.min(IDLE_CAP, Math.round(gap / 1000));
|
|
890
|
+
current.messages.push(messages[i]);
|
|
891
|
+
}
|
|
892
|
+
return runs;
|
|
893
|
+
}
|
|
873
894
|
const EXPORTS_STATE = join(homedir(), ".worktrust", "exports.json"); // per export digest: sent, or offered and declined; nothing of a conversation
|
|
874
895
|
const readExportsState = () => { try { return JSON.parse(readFileSync(EXPORTS_STATE, "utf8")); } catch { return {}; } };
|
|
875
896
|
const rememberExport = (digest, note) => { const state = readExportsState(); state[digest] = { ...(state[digest] ?? {}), ...note }; mkdirSync(dirname(EXPORTS_STATE), { recursive: true }); writeFileSync(EXPORTS_STATE, JSON.stringify(state, null, 1)); };
|
|
876
|
-
function exportEntries() {
|
|
877
|
-
if (!exportSessions) throw Error("chat-exports.mjs is not beside this file: a data export cannot be read");
|
|
897
|
+
async function exportEntries() {
|
|
898
|
+
if (!exportSessions || !exportRunRef) throw Error("chat-exports.mjs is not beside this file, or is older than this hook: a data export cannot be read");
|
|
878
899
|
const entries = [], reports = [];
|
|
879
900
|
for (const given of EXPORT_PATHS) {
|
|
880
|
-
const read =
|
|
881
|
-
|
|
882
|
-
for (const { id, lines } of exportSessions(read, { excluded })) {
|
|
901
|
+
const read = {}, report = { days: new Set(), seconds: 0 }; // 0.10.18: the reader streams the export and fills `read` as it goes (path and vendor first; digest, count and the conversations left out as too large at the end)
|
|
902
|
+
for await (const { id, lines } of exportSessions(given, { excluded, report: read })) {
|
|
883
903
|
const messages = [];
|
|
884
904
|
for (const line of lines) { const at = line.timestamp ? Date.parse(line.timestamp) : NaN; if (Number.isFinite(at)) messages.push(messageOf(line, at)); }
|
|
885
905
|
messages.sort((a, b) => a.at - b.at);
|
|
886
906
|
for (const [day, ofDay] of byDayOf(messages)) {
|
|
887
907
|
if (ofDay.length === 0) continue;
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
|
|
908
|
+
// ONE STRETCH PER RUN OF ACTIVITY (0.10.17): the day's messages cut where the hook calls a stretch finished; one run
|
|
909
|
+
// is the day as before, under the name it always had; several name the one-envelope line they replace.
|
|
910
|
+
const runs = runsOf(ofDay), whole = measure(null, ofDay), old = exportRef(read.vendor, id, day);
|
|
911
|
+
runs.forEach(({ messages: run, seconds }, index) => {
|
|
912
|
+
const stretch = measure(null, run), ref = runs.length === 1 ? old : exportRunRef(read.vendor, id, day, index);
|
|
913
|
+
// The day as a whole decides whether there is a duration (under a minute is none, as before); a run then carries
|
|
914
|
+
// its own share of the day's count, and a run with nothing to count carries none.
|
|
915
|
+
const own = runs.length === 1 ? whole.seconds : whole.seconds !== null && seconds > 0 ? seconds : null;
|
|
916
|
+
const entry = {
|
|
917
|
+
title: EXPORT_TITLES[read.vendor], kind: "researched",
|
|
918
|
+
...(own !== null ? { seconds: Math.min(RECONSTRUCTED_MAX, own), duration_basis: "reconstructed" } : {}),
|
|
919
|
+
...(stretch.exchanges > 0 ? { exchanges: stretch.exchanges, turn_basis: "humanOnly" } : {}),
|
|
920
|
+
...(stretch.model ? { model: stretch.model } : {}),
|
|
921
|
+
stretch_ref: ref, source_event_id: `exp_${ref}`,
|
|
922
|
+
...(runs.length > 1 ? { supersedes_ref: old } : {}),
|
|
923
|
+
started_at: new Date(stretch.from).toISOString(), utc_offset: stretch.offset, at: new Date(stretch.to).toISOString(),
|
|
924
|
+
};
|
|
925
|
+
entrySource.set(entry, read.vendor);
|
|
926
|
+
entryFile.set(entry, { file: `${read.vendor}:${id}`, from: stretch.from, folder: null });
|
|
927
|
+
report.seconds += entry.seconds ?? 0; entries.push(entry);
|
|
928
|
+
});
|
|
929
|
+
report.days.add(day);
|
|
900
930
|
}
|
|
901
931
|
}
|
|
902
932
|
const days = [...report.days].sort();
|
|
903
|
-
reports.push({
|
|
933
|
+
reports.push({ path: read.path, vendor: read.vendor, digest: read.digest, conversations: read.conversations, days: days.length, seconds: report.seconds, hours: Math.round(report.seconds / 360) / 10, first: days[0] ?? null, last: days.at(-1) ?? null, oversized: read.oversized });
|
|
904
934
|
}
|
|
905
935
|
const seen = new Set();
|
|
906
936
|
return { entries: entries.filter((entry) => !seen.has(entry.stretch_ref) && seen.add(entry.stretch_ref)).sort((a, b) => a.at.localeCompare(b.at)), reports };
|
|
@@ -1172,7 +1202,7 @@ if (flag("evidence-summary")) {
|
|
|
1172
1202
|
if (flag("archive-lines")) { const lines = historyEntries(null, true).map((entry) => JSON.stringify({ ...entry, ...(DERIVED.get(entry) ?? {}), client: entrySource.get(entry) ?? "claude", started_at: new Date(entryFile.get(entry).from).toISOString(), folder: entryFile.get(entry).folder })); process.stdout.write(`${[...lines, JSON.stringify({ archive_end: lines.length })].join("\n")}\n`); await exitFlushed(0); } // one write and a count, so the reader tells a whole answer from a cut one
|
|
1173
1203
|
// THE EXPORTS IN A FOLDER (0.10.10, for the bare `npx worktrust`): which lie there, by vendor and digest, and which this computer
|
|
1174
1204
|
// already sent or was asked about; reads, prints one JSON line, sends nothing. `--export-offered <digest>` records a No.
|
|
1175
|
-
if (flag("find-exports")) { const named = value("find-exports"), state = readExportsState(); const found = (findExports ? findExports(named && !named.startsWith("--") ? named : undefined) : []).map((one) => ({ ...one, sent: state[one.digest]?.sent === true, offered: Boolean(state[one.digest]) })); process.stdout.write(`${JSON.stringify(found)}\n`); await exitFlushed(0); }
|
|
1205
|
+
if (flag("find-exports")) { const named = value("find-exports"), state = readExportsState(); const found = (findExports ? await findExports(named && !named.startsWith("--") ? named : undefined) : []).map((one) => ({ ...one, sent: state[one.digest]?.sent === true, offered: Boolean(state[one.digest]) })); process.stdout.write(`${JSON.stringify(found)}\n`); await exitFlushed(0); }
|
|
1176
1206
|
if (value("export-offered")) { rememberExport(value("export-offered"), { offered: new Date().toISOString() }); await exitFlushed(0); }
|
|
1177
1207
|
if (!door && !codexDoor && !DRY) { process.exit(0); } // Not coupled on this machine: nothing to do, quietly.
|
|
1178
1208
|
|
|
@@ -1214,7 +1244,7 @@ if (flag("history")) {
|
|
|
1214
1244
|
const historyFloor = floorOf(readState(), "__historyFloor");
|
|
1215
1245
|
// AN EXPORT IS ITS OWN HISTORY (0.10.10): its lines alone, reconstructed, and never a rebuild (a rebuild replaces this computer's own measured days).
|
|
1216
1246
|
let fromExports = null, fromFolder = null;
|
|
1217
|
-
if (EXPORT_PATHS.length > 0) { try { fromExports = exportEntries(); } catch (error) { console.log(`export: ${error.message}`); await exitFlushed(1); } }
|
|
1247
|
+
if (EXPORT_PATHS.length > 0) { try { fromExports = await exportEntries(); } catch (error) { console.log(`export: ${error.message}`); await exitFlushed(1); } }
|
|
1218
1248
|
// A COPIED SESSION FOLDER (0.10.10): its own history, reconstructed, the live folders' stretches skipped; never a rebuild.
|
|
1219
1249
|
if (flag("from-folder")) {
|
|
1220
1250
|
const given = value("from-folder");
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "worktrust",
|
|
3
|
-
"version": "0.10.
|
|
3
|
+
"version": "0.10.18",
|
|
4
4
|
"description": "Couple this computer to WorkTrust: approve a code in your browser, and every AI app here reports what you built. Metadata only, no dependencies.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
"engines": {
|
|
35
35
|
"node": ">=18"
|
|
36
36
|
},
|
|
37
|
-
"license": "
|
|
37
|
+
"license": "SEE LICENSE IN LICENSE",
|
|
38
38
|
"homepage": "https://worktrust.io",
|
|
39
39
|
"keywords": [
|
|
40
40
|
"worktrust",
|
package/worktrust.mjs
CHANGED
|
@@ -20,7 +20,7 @@
|
|
|
20
20
|
* npx worktrust history --from-folder <dir> your own copy of an AI app's session folder (a backed-up ~/.claude/projects,
|
|
21
21
|
* ~/.codex/sessions): the days this computer's own sessions lack, reconstructed,
|
|
22
22
|
* never verified hours (shows the plan, asks first; 0.10.10)
|
|
23
|
-
* npx worktrust history --export <path> a ChatGPT or Claude data export (the zip, its folder
|
|
23
|
+
* npx worktrust history --export <path> a ChatGPT or Claude data export (the zip, its folder, conversations.json, a conversations part or the manifest)
|
|
24
24
|
* as history: hours RECONSTRUCTED from its clocks, shown with their origin,
|
|
25
25
|
* never verified hours (shows the plan, asks first; 0.10.10). The bare
|
|
26
26
|
* command on a coupled computer offers an export lying in ~/Downloads once.
|
|
@@ -83,7 +83,7 @@ const command = args.includes("--help") ? "help" : args.includes("--version") ?
|
|
|
83
83
|
const flag = (name) => { const at = args.indexOf(`--${name}`); return at >= 0 ? args[at + 1] : undefined; };
|
|
84
84
|
const has = (name) => args.includes(`--${name}`);
|
|
85
85
|
/** This CLI's version, said to the door so the app can tell which computer runs an old one (check-cli-package holds it equal to package.json). */
|
|
86
|
-
const CLI_VERSION = "0.10.
|
|
86
|
+
const CLI_VERSION = "0.10.18";
|
|
87
87
|
const ORIGIN = (flag("origin") ?? process.env.WORKTRUST_ORIGIN ?? "https://app.worktrust.io").replace(/\/$/, "");
|
|
88
88
|
const MCP = flag("url") ?? process.env.WORKTRUST_MCP_URL ?? `${ORIGIN}/api/mcp`;
|
|
89
89
|
const HOME_DIR = join(homedir(), ".worktrust");
|
|
@@ -767,7 +767,7 @@ async function history() {
|
|
|
767
767
|
const key = await freshKey();
|
|
768
768
|
if (!key) fail("this computer is not coupled through the key file. Run npx worktrust first.");
|
|
769
769
|
if (has("from-archive")) return offerArchive(await scripts(), key);
|
|
770
|
-
if (has("export")) { if (!flag("export") || flag("export").startsWith("--")) fail("--export needs the export's path: the zip, its folder,
|
|
770
|
+
if (has("export")) { if (!flag("export") || flag("export").startsWith("--")) fail("--export needs the export's path: the zip, its folder, conversations.json, a conversations-000.zip part, or the manifest"); await offerExport(await scripts(), key, flag("export")); return; }
|
|
771
771
|
if (has("from-folder")) { if (!flag("from-folder") || flag("from-folder").startsWith("--")) fail("--from-folder needs the copied folder's path"); await offerFolder(await scripts(), key, flag("from-folder")); return; }
|
|
772
772
|
// ONE PATH (0.10.5): sending is re-measuring. On a computer WorkTrust has never heard from nothing is replaced, so it is a
|
|
773
773
|
// plain send; after a failure, a merge or an older CLI, the same command replaces day by day. `--rebuild` still reads.
|
|
@@ -812,6 +812,8 @@ async function offerExport(paths, key, path) {
|
|
|
812
812
|
if (!report.days) { say(` ${product} export · ${report.conversations} conversations, none with a clock to place: nothing to send.`); return true; }
|
|
813
813
|
say(` ${product} export · ${report.conversations} conversations · ${report.days} days · ${report.hours} reconstructed hours · ${String(report.first).slice(0, 7)} … ${String(report.last).slice(0, 7)}`);
|
|
814
814
|
say(" Hours reconstructed from the export's own clocks: shown with their origin, never verified hours. Never a title or a message.");
|
|
815
|
+
// LEFT OUT AND COUNTED (0.10.18): a conversation whose kept fields outgrew the reader's budget is passed over, never the export.
|
|
816
|
+
if (report.oversized > 0) say(` ${report.oversized} conversation${report.oversized === 1 ? "" : "s"} too large to read ${report.oversized === 1 ? "is" : "are"} left out; the rest is read.`);
|
|
815
817
|
if (has("dry-run")) { say(" Dry run: nothing was sent."); return false; }
|
|
816
818
|
if (!has("history") && !(await ask("Send it as history (reconstructed hours per day; never text)?", false))) { say(` Not sent. Send it later with: npx worktrust history --export ${path}`); return false; }
|
|
817
819
|
const sent = await run(paths["log-session.mjs"], ["--history", "--export", path], key.token, true, env);
|
|
@@ -821,6 +823,8 @@ async function offerExport(paths, key, path) {
|
|
|
821
823
|
// says so; the lines an AI app reported under no product on those days may describe the same conversations, and go only on a yes.
|
|
822
824
|
const counts = lastCounts(sent.out);
|
|
823
825
|
if (counts.superseded > 0) say(` ${counts.superseded} earlier line${counts.superseded === 1 ? "" : "s"} of ${product} on those days, brought in by a connector, ${counts.superseded === 1 ? "is" : "are"} withdrawn and kept (each with a correction row): the export's lines count, a day counts once.`);
|
|
826
|
+
// ONE STRETCH PER RUN (0.10.17): a conversation-day sent before as one envelope is now its runs; the door withdrew the envelope.
|
|
827
|
+
if (counts.recut > 0) say(` ${counts.recut} earlier line${counts.recut === 1 ? "" : "s"} of this export, one per conversation and day, ${counts.recut === 1 ? "is" : "are"} replaced by one line per run of activity (withdrawn and kept, the same total).`);
|
|
824
828
|
if (counts.unknownOnDays > 0) {
|
|
825
829
|
const n = counts.unknownOnDays;
|
|
826
830
|
say(` ${n} earlier line${n === 1 ? "" : "s"} an AI app reported for these days, under no product, may describe the same conversations.`);
|
|
@@ -1150,7 +1154,7 @@ function help() {
|
|
|
1150
1154
|
say(" --from-archive [dir] send what your archive (preserve --archive) holds of sessions your AI apps already deleted,");
|
|
1151
1155
|
say(" marked as from your archive: a day this computer's key signed as measured, every other day");
|
|
1152
1156
|
say(" as reconstructed, never verified hours (shows the plan, asks first)");
|
|
1153
|
-
say(" --export <path> a ChatGPT or Claude data export (the zip, its folder
|
|
1157
|
+
say(" --export <path> a ChatGPT or Claude data export (the zip, its folder, conversations.json, a conversations part or the manifest) as history:");
|
|
1154
1158
|
say(" hours reconstructed from its clocks, shown with their origin, never verified hours (asks first);");
|
|
1155
1159
|
say(" on its days the export replaces the product's earlier connector lines (withdrawn, kept)");
|
|
1156
1160
|
say(" --withdraw-unknown with --export: yes in advance to withdrawing the earlier lines an AI app reported for those days");
|