tokenusage-cli 0.2.4 → 0.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,42 @@
3
3
  本包(`tokenusage-cli`)的用户可见变更记录。发版时在顶部新增一节,
4
4
  `## <版本号>` 行会被 release workflow 取作 GitHub Release 正文。
5
5
 
6
+ ## 0.2.6
7
+
8
+ 移除从未接线的错误上报 SDK,安装体积与运行时副本明显变小;采集行为不变。
9
+
10
+ - 删除 `@sentry/node` 与 `src/lib/sentry.js`、`src/lib/telemetry-pref.js`:
11
+ CLI 只在启动时 init 了 Sentry,`captureException` 没有任何调用点,实际从不上报;
12
+ 但它在运行时副本里占 44 MB(67.5 MB 的约 65%),全局安装还带入
13
+ `@sentry/bundler-plugins` 的构建期依赖链(安装总量 93.1 MB 中约 59 MB)。
14
+ 移除后运行时副本 22.9 MB(实测 462 个文件)、干净 prefix 安装 23 MB
15
+ (此前 93.1 MB),且不再有第三方上报入口。
16
+ - `bin/tokenusage.js` 未捕获异常的兜底里还留着一个 `require('../src/lib/sentry')`
17
+ (包在 try/catch 里,删模块后永不生效的残留);一并删除,守卫测试改为同时扫描
18
+ `src/` 与 `bin/`。
19
+ - README 隐私边界同步:CLI 不做匿名心跳、也不做任何第三方错误上报。
20
+ - `THIRD_PARTY_NOTICES.md` 的运行时依赖示例去掉 `@sentry/node`。
21
+ - 新增 `test/packaging.test.js` 守卫:`@sentry/*` 不得重新出现在依赖或源码中;
22
+ 运行时副本回归测试改用 `yauzl` 的传递依赖 `pend` 验证扁平闭包复制。
23
+ - 未随本版改动:macOS/Linux 原生调度、`init` 逐工具集成选择(见
24
+ `docs/cli-release.md` §四,等真机验证/形态确认后再做)。
25
+
26
+ ## 0.2.5
27
+
28
+ 0.2.4 验收后的小修:去掉安装警告、修正初始化重复文案,并补 Windows CI。
29
+
30
+ - 移除 `@mongodb-js/zstd` 与两处 Node 20 zstd 回落分支:包 `engines` 已是
31
+ `>=22.19`,`node:zlib` 自带 zstd。`npm i -g` 不再出现 `prebuild-install`
32
+ 弃用警告,运行时副本也少一个 native 依赖。
33
+ - 修 `init` 跳过行重复文案:detail 已以 `Skipped` 开头时不再叠加
34
+ `Skipped - ` 前缀(此前显示 `Skipped - Skipped (--no-integrations)`)。
35
+ - 顺手去掉 `--dry-run` 收尾的重复 `Dry run complete` 行,只保留
36
+ `Run init without --dry-run to apply changes.`。
37
+ - README 补齐 `init` 的 `--yes` / `--dry-run` / `--no-open` / `--link-code`
38
+ 参数说明。
39
+ - CI 新增 `windows-latest` job:跑 CLI 测试 + `init --dry-run` 烟测并断言
40
+ 输出不含 `Skipped - Skipped`,避免 VBS / 入口类回归只靠发版前人工验收。
41
+
6
42
  ## 0.2.4
7
43
 
8
44
  计划任务抗 Node 版本切换 + 自检一次性收口的采集可靠性修复(发 0.2.3 后立即补;
package/README.md CHANGED
@@ -30,7 +30,7 @@ tokenusage sync --drain # 首次/积压补传(更大的批处理预算)
30
30
 
31
31
  | 命令 | 说明 |
32
32
  |---|---|
33
- | `tokenusage init` | 本地初始化:扫描根配置、各工具 hook 安装、注册每小时计划任务(Windows 隐藏窗口;`--no-task` 跳过任务;`--run-report=off` 关闭运行记录上报;`--no-integrations` 不装任何 hook/notify/插件,只做纯定时采集)。不含配对 |
33
+ | `tokenusage init` | 本地初始化:扫描根配置、各工具 hook 安装、注册每小时计划任务(Windows 隐藏窗口)。`--no-task` 跳过任务;`--run-report=off` 关闭运行记录上报;`--no-integrations` 不装任何 hook/notify/插件,只做纯定时采集;`--yes` 非交互;`--dry-run` 只预览不落盘;`--no-open` 不打开浏览器;`--link-code <code>` 用已有配对码。不含配对 |
34
34
  | `tokenusage device-login` | 设备流配对:输出 `user_code`,在站点 `/device` 页批准后本机拿到设备令牌 |
35
35
  | `tokenusage sync` | 解析本地 AI 工具日志 → 上报小时桶 + 会话元数据 |
36
36
  | `tokenusage sync --watch` | 常驻模式,默认每 15 分钟增量一轮(`--interval <ms>` 可调,≥60000) |
@@ -103,11 +103,25 @@ CLI 未配对也能正常用,只是**降级为“只采集、不上报”**:
103
103
 
104
104
  **永不上传**:prompt / 助手正文 / 工具参数 / 命令输出 / diff / 会话标题 / 本地绝对路径 / resume 命令。测试(`test/session-upload.test.js`)用字段白名单锁定该边界。
105
105
 
106
- CLI 也不发送匿名安装心跳:上游的 `tokenusage-telemetry` 端点在自托管后端从未实现,0.2.3 起已移除。可选的 Sentry 错误上报必须显式配置 DSN,并受 `TOKENUSAGE_NO_TELEMETRY` / `DO_NOT_TRACK` 约束。
106
+ CLI 不发送匿名安装心跳,也不做任何第三方错误上报:上游的 `tokenusage-telemetry` 端点在自托管后端从未实现(0.2.3 移除);从未接线的 Sentry SDK 也已在 0.2.6 删除。除你自己配对的实例外,CLI 不向任何第三方发送数据。
107
107
 
108
108
  ## 数据目录
109
109
 
110
- `~/.tokenusage/`(`0700` 目录 + `0600` 文件):`tracker/config.json`(配对与运行配置)、上传队列(`queue.jsonl` + offset)、会话分析 sidecar、`session-upload-watermarks.json`(增量水位)、`tracker/logs/sync.log`(自动采集滚动日志)、暂停标志。与旧版 tokentracker CLI 的 `~/.tokentracker/` 互不干扰,迁移期两套 CLI 可在同一台机器共存。
110
+ `~/.tokenusage/`(`0700` 目录 + `0600` 文件):`tracker/config.json`(配对与运行配置)、`tracker/app/`(自包含运行时副本,`.runtime-ready` 记录版本)、上传队列(`queue.jsonl` + offset)、会话分析 sidecar、`session-upload-watermarks.json`(增量水位)、`tracker/logs/sync.log`(自动采集滚动日志)、暂停标志。与旧版 tokentracker CLI 的 `~/.tokentracker/` 互不干扰,迁移期两套 CLI 可在同一台机器共存。
111
+
112
+ ## 自包含运行时副本(0.2.4+)
113
+
114
+ init 会把当前 CLI 及其依赖闭包复制到 `~/.tokenusage/tracker/app`,并写入
115
+ `.runtime-ready`(版本 + 安装时间)。计划任务 / notify / hook 优先启动已安装包;
116
+ 包路径失效时(典型是 nvm-windows 切换 Node 版本后全局前缀变化)自动回落到副本,
117
+ 小时任务不再 `file not found`。副本先复制到 `app.new` 暂存,`require` 全部直接
118
+ 依赖且 `--version` 实跑通过后才替换并写标记;没有标记的副本不会被任何自动采集
119
+ 入口使用。
120
+
121
+ 升级 CLI 或切换 Node 版本后重跑一次
122
+ `tokenusage init --yes --no-integrations --no-open` 刷新副本;`tokenusage doctor`
123
+ 的 `runtime.copy` 检查会报告副本缺失、缺标记或版本落后,`uninstall --purge`
124
+ 连同数据目录一并清理。
111
125
 
112
126
  ## provider 支持面
113
127
 
@@ -11,6 +11,6 @@ upstream repository and were then modified for the token-usage contract
11
11
  (`docs/cli-contract.md` in the repository root). The upstream MIT license text
12
12
  and copyright notice are preserved in this package's `LICENSE` file.
13
13
 
14
- 本包不含其他第三方资产:运行时依赖(`undici`、`@sentry/node` 等)的许可随
14
+ 本包不含其他第三方资产:运行时依赖(如 `undici`)的许可随
15
15
  `node_modules` 一同分发;品牌图标、字体与定价快照属于 Web 端,登记在仓库根目录的
16
16
  `THIRD_PARTY_NOTICES.md`。
package/bin/tokenusage.js CHANGED
@@ -43,12 +43,6 @@ if (applyResult && applyResult.unprotected === true) {
43
43
  }
44
44
 
45
45
  run(argv).catch((err) => {
46
- try {
47
- const { captureException } = require('../src/lib/sentry');
48
- captureException(err, { tags: { surface: 'cli-unhandled' } });
49
- } catch {
50
- /* ignore */
51
- }
52
46
  console.error(err?.stack || String(err));
53
47
  if (debug) {
54
48
  if (typeof err?.status === 'number') {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tokenusage-cli",
3
- "version": "0.2.4",
3
+ "version": "0.2.6",
4
4
  "description": "Self-hosted AI coding token usage collector for the Token Usage dashboard.",
5
5
  "keywords": [
6
6
  "token-usage",
@@ -43,15 +43,12 @@
43
43
  },
44
44
  "dependencies": {
45
45
  "@journeyapps/wa-sqlite": "2.0.7",
46
- "@mongodb-js/zstd": "^2.0.1",
47
- "@sentry/node": "^11.8.0",
48
46
  "smol-toml": "^1.9.0",
49
47
  "undici": "8.12.2",
50
48
  "yauzl": "^3.4.0"
51
49
  },
52
50
  "pnpm": {
53
51
  "onlyBuiltDependencies": [
54
- "@mongodb-js/zstd",
55
52
  "@journeyapps/wa-sqlite",
56
53
  "esbuild"
57
54
  ]
package/src/cli.js CHANGED
@@ -9,16 +9,6 @@ const { cmdSessions } = require("./commands/sessions");
9
9
  const { cmdPause, cmdResume } = require("./commands/pause");
10
10
 
11
11
  async function run(argv) {
12
- // Best-effort Sentry init (no-op without DSN / when telemetry opted out).
13
- try {
14
- const { initSentry } = require("./lib/sentry");
15
- const { resolveTrackerPaths } = require("./lib/tracker-paths");
16
- const { trackerDir } = await resolveTrackerPaths().catch(() => ({ trackerDir: null }));
17
- await initSentry({ trackerDir });
18
- } catch {
19
- /* never block CLI startup on telemetry */
20
- }
21
-
22
12
  const [command, ...rest] = argv;
23
13
 
24
14
  // No args → short help (the upstream local dashboard was removed in this
@@ -332,7 +332,7 @@ function renderAccountNotLinked({ context } = {}) {
332
332
  process.stdout.write(
333
333
  [
334
334
  "",
335
- "Dry run complete. Run init without --dry-run to apply changes.",
335
+ "Run init without --dry-run to apply changes.",
336
336
  "",
337
337
  ].join("\n"),
338
338
  );
@@ -2474,8 +2474,6 @@ function probeRuntimeCopy(appDir) {
2474
2474
  "if (missing.length > 0) { console.error(missing.join(',')); process.exit(3); }",
2475
2475
  ].join("\n");
2476
2476
  const env = { ...process.env };
2477
- // The probe must not trigger telemetry or hook spawning.
2478
- env.TOKENUSAGE_NO_TELEMETRY = "1";
2479
2477
  try {
2480
2478
  const probe = cp.spawnSync(process.execPath, ["-e", script], {
2481
2479
  cwd: appDir,
package/src/lib/cli-ui.js CHANGED
@@ -155,7 +155,13 @@ function createSpinner({ text, intervalMs = 80 }) {
155
155
  function formatSummaryLine({ label, status, detail }) {
156
156
  const isSuccess = status === "updated" || status === "set" || status === "installed";
157
157
  const bullet = isSuccess ? color("*", GREEN) : "o";
158
- const statusLabel = isSuccess ? detail || status : detail ? `Skipped - ${detail}` : "Skipped";
158
+ const statusLabel = isSuccess
159
+ ? detail || status
160
+ : !detail
161
+ ? "Skipped"
162
+ : /^Skipped\b/i.test(detail)
163
+ ? detail
164
+ : `Skipped - ${detail}`;
159
165
  const line = ` ${bullet} ${label.padEnd(22)} [${statusLabel}]`;
160
166
  return isSuccess ? line : color(line, DIM);
161
167
  }
@@ -190,8 +190,8 @@ function isRolloutFileName(name) {
190
190
  // Compressed rollout archives cannot be tailed by byte offset: the cursor
191
191
  // stores compressed bytes while the parser consumes decompressed text. Treat
192
192
  // them as content-addressed snapshots — one parse per (inode, size) version —
193
- // reusing the DSH session's zstd machinery (built-in zlib first,
194
- // `@mongodb-js/zstd` fallback) including its multi-frame handling. Returns
193
+ // reusing the DSH session's built-in zlib zstd machinery, including its
194
+ // multi-frame handling. Returns
195
195
  // null for a missing/non-file path; an empty archive yields no lines (the
196
196
  // same outcome as an empty uncompressed rollout).
197
197
  const CODEX_ROLLOUT_COMPRESSED_PATTERN = /^rollout-.*\.jsonl\.zstd?$/;
@@ -13174,10 +13174,18 @@ function resolveZedDbPath(env = process.env) {
13174
13174
  return path.join(xdg, "zed", "threads", "threads.db");
13175
13175
  }
13176
13176
 
13177
+ function decompressBuiltinZstd(data) {
13178
+ const zlib = require("node:zlib");
13179
+ if (typeof zlib.zstdDecompressSync !== "function") {
13180
+ throw new Error("built-in zstd is unavailable; Node.js >= 22.19 is required");
13181
+ }
13182
+ return Buffer.from(zlib.zstdDecompressSync(data));
13183
+ }
13184
+
13177
13185
  // Decode a row's BLOB payload into UTF-8 JSON text. Zed marks zstd-compressed
13178
13186
  // blobs with data_type="zstd"; older / smaller threads use data_type="json"
13179
- // and store the bytes verbatim. Node 24+ has native zstd; Node 20 needs the
13180
- // @mongodb-js/zstd fallback. Cap decoded size to mirror tokscale's safety net.
13187
+ // and store the bytes verbatim. Node >=22.19 already ships zstd; the package
13188
+ // engine floor guarantees it. Cap decoded size to mirror tokscale's safety net.
13181
13189
  async function decodeZedThreadBlob({ dataType, data }) {
13182
13190
  const type = (dataType || "").trim().toLowerCase();
13183
13191
  if (type === "json") {
@@ -13187,11 +13195,7 @@ async function decodeZedThreadBlob({ dataType, data }) {
13187
13195
  return data.toString("utf8");
13188
13196
  }
13189
13197
  if (type === "zstd") {
13190
- const zlib = require("node:zlib");
13191
- const out =
13192
- typeof zlib.zstdDecompressSync === "function"
13193
- ? zlib.zstdDecompressSync(data)
13194
- : Buffer.from(await require("@mongodb-js/zstd").decompress(data));
13198
+ const out = decompressBuiltinZstd(data);
13195
13199
  if (out.length > MAX_ZED_THREAD_JSON_BYTES) {
13196
13200
  throw new Error(`decoded zstd blob exceeds ${MAX_ZED_THREAD_JSON_BYTES} bytes`);
13197
13201
  }
@@ -23870,8 +23874,8 @@ async function parseTraeCnApiIncremental({
23870
23874
  // Zstd artifacts are a concatenated-frame container: a single decompress call
23871
23875
  // decodes only the first frame (silently dropping every event line), so we
23872
23876
  // identify the frame boundaries and decode each frame independently —
23873
- // built-in zlib zstd first, `@mongodb-js/zstd` as the Node 20 fallback —
23874
- // enforcing a cumulative plaintext bound as we go.
23877
+ // with the built-in Node zlib zstd decoder, enforcing a cumulative plaintext
23878
+ // bound as we go.
23875
23879
  // Dedup is a per-file `lastSeq` watermark — seq
23876
23880
  // is monotonic within an append-only session log, so a grow re-reads the file
23877
23881
  // and skips everything at or below the watermark. Torn tails never advance the
@@ -24119,18 +24123,11 @@ function inspectDshZstdFrames(data, maxOutputBytes = DSH_SESSION_TEXT_MAX_BYTES)
24119
24123
  };
24120
24124
  }
24121
24125
 
24122
- // Decode one already-isolated frame. Prefer Node's built-in zstd (22.15+):
24123
- // the desktop bundles install dependencies with --ignore-scripts, so
24124
- // @mongodb-js/zstd ships without its native binding there and the require()
24125
- // would throw — silently skipping every compressed session (issue #465). The
24126
- // MongoDB binding stays as the fallback for Node 20 CLI installs, where npm
24127
- // runs install scripts normally.
24126
+ // Decode one already-isolated frame with Node's built-in zstd. The previous
24127
+ // @mongodb-js/zstd fallback was only reachable on Node 20; the package engine
24128
+ // floor is now >=22.19, so a missing built-in decoder is a hard error.
24128
24129
  async function decodeDshZstdFrame(frameBytes) {
24129
- const zlib = require("node:zlib");
24130
- if (typeof zlib.zstdDecompressSync === "function") {
24131
- return zlib.zstdDecompressSync(frameBytes);
24132
- }
24133
- return Buffer.from(await require("@mongodb-js/zstd").decompress(frameBytes));
24130
+ return decompressBuiltinZstd(frameBytes);
24134
24131
  }
24135
24132
 
24136
24133
  // Decode one independent Harness append frame at a time; per-frame decoding
@@ -234,8 +234,8 @@ test("init's runtime copy works from the published layout (bin/tokenusage.js)",
234
234
  "declared dependency copied as a real directory",
235
235
  );
236
236
  assert.ok(
237
- fs.existsSync(path.join(appDir, "node_modules", "@sentry", "core")),
238
- "transitive dependency of @sentry/node copied too (flattened closure)",
237
+ fs.existsSync(path.join(appDir, "node_modules", "pend")),
238
+ "transitive dependency of yauzl copied too (flattened closure)",
239
239
  );
240
240
  // The marker is what every spawn point probes: without it hooks fall back
241
241
  // to the installed CLI instead of spawning a half copy.
@@ -0,0 +1,47 @@
1
+ "use strict";
2
+
3
+ // init summary rows: skipped call-sites already include a "Skipped (...)"
4
+ // detail, while formatSummaryLine added its own "Skipped - " prefix. The
5
+ // result was "Skipped - Skipped (--no-integrations)" (0.2.4 acceptance).
6
+ // Keep the double prefix out without changing the success rows.
7
+
8
+ const assert = require("node:assert/strict");
9
+ const { test } = require("node:test");
10
+
11
+ const { formatSummaryLine } = require("../src/lib/cli-ui");
12
+
13
+ test("a detail that already starts with Skipped is not prefixed twice", () => {
14
+ const line = formatSummaryLine({
15
+ label: "Integrations",
16
+ status: "skipped",
17
+ detail: "Skipped (--no-integrations); tool configs stay untouched",
18
+ });
19
+ assert.ok(line.includes("[Skipped (--no-integrations); tool configs stay untouched]"), line);
20
+ assert.ok(!line.includes("Skipped - Skipped"), line);
21
+ });
22
+
23
+ test("a plain skipped detail still gets the Skipped - prefix", () => {
24
+ const line = formatSummaryLine({
25
+ label: "Runtime copy",
26
+ status: "skipped",
27
+ detail: "not installed",
28
+ });
29
+ assert.ok(line.includes("[Skipped - not installed]"), line);
30
+ });
31
+
32
+ test("skipped rows without a detail keep the bare label", () => {
33
+ const line = formatSummaryLine({
34
+ label: "Scheduled task",
35
+ status: "skipped",
36
+ });
37
+ assert.ok(line.includes("[Skipped]"), line);
38
+ });
39
+
40
+ test("success rows keep their detail text untouched", () => {
41
+ const line = formatSummaryLine({
42
+ label: "Claude Code",
43
+ status: "installed",
44
+ detail: "hook -> ~/.claude/settings.json",
45
+ });
46
+ assert.ok(line.includes("[hook -> ~/.claude/settings.json]"), line);
47
+ });
@@ -10,6 +10,7 @@ const { test } = require("node:test");
10
10
 
11
11
  const PACKAGE_ROOT = path.join(__dirname, "..");
12
12
  const SRC = path.join(PACKAGE_ROOT, "src");
13
+ const BIN = path.join(PACKAGE_ROOT, "bin");
13
14
 
14
15
  function readSource(...segments) {
15
16
  return fs.readFileSync(path.join(PACKAGE_ROOT, ...segments), "utf8");
@@ -35,12 +36,35 @@ test("local-api spawns the package bin entry, not the missing bin/tracker.js", (
35
36
  );
36
37
  });
37
38
 
38
- test("telemetry opt-out flags still gate the optional Sentry reporting", () => {
39
- const { isTelemetryDisabled } = require("../src/lib/telemetry-pref");
40
- assert.equal(isTelemetryDisabled({ env: {}, config: {} }), false);
41
- assert.equal(isTelemetryDisabled({ env: { TOKENUSAGE_NO_TELEMETRY: "1" }, config: {} }), true);
42
- assert.equal(isTelemetryDisabled({ env: { DO_NOT_TRACK: "true" }, config: {} }), true);
43
- assert.equal(isTelemetryDisabled({ env: {}, config: { telemetry: false } }), true);
39
+ test("the unused Sentry error-reporting SDK stays out of the package", () => {
40
+ // 0.2.6 removed @sentry/node: it was only initialized (captureException had no
41
+ // call sites) and shipped 44 MB into the runtime copy plus a build-time
42
+ // bundler-plugin chain into every npm install. Keep it out unless the
43
+ // reporting path is actually wired and budgeted again.
44
+ const pkg = JSON.parse(readSource("package.json"));
45
+ const declared = { ...(pkg.dependencies || {}), ...(pkg.optionalDependencies || {}) };
46
+ assert.ok(
47
+ !Object.keys(declared).some((name) => name === "@sentry/node" || name.startsWith("@sentry/")),
48
+ "@sentry/* must not be a declared dependency",
49
+ );
50
+ assert.ok(
51
+ !fs.existsSync(path.join(SRC, "lib", "sentry.js")),
52
+ "the inert sentry bootstrap must stay deleted",
53
+ );
54
+ assert.ok(
55
+ !fs.existsSync(path.join(SRC, "lib", "telemetry-pref.js")),
56
+ "the opt-out helper only existed to gate Sentry",
57
+ );
58
+ const roots = [SRC, BIN];
59
+ for (const root of roots) {
60
+ const sourceFiles = fs
61
+ .readdirSync(root, { recursive: true })
62
+ .filter((rel) => String(rel).endsWith(".js"));
63
+ for (const rel of sourceFiles) {
64
+ const source = fs.readFileSync(path.join(root, String(rel)), "utf8");
65
+ assert.ok(!source.includes("@sentry/"), `${rel} must not require the removed SDK`);
66
+ }
67
+ }
44
68
  });
45
69
 
46
70
  test("the unported install heartbeat stays removed", () => {
@@ -0,0 +1,36 @@
1
+ "use strict";
2
+
3
+ // 0.2.5 dropped @mongodb-js/zstd: the package engine floor is >=22.19, so
4
+ // node:zlib can always decode zstd. These tests exercise the two real decode
5
+ // paths (Zed thread blobs and concatenated DeepSeek Harness frames) so a
6
+ // future dependency/engine change cannot silently remove zstd support.
7
+
8
+ const assert = require("node:assert/strict");
9
+ const zlib = require("node:zlib");
10
+ const { test } = require("node:test");
11
+
12
+ const { decodeDshZstd, decodeZedThreadBlob } = require("../src/lib/rollout");
13
+
14
+ test("the supported Node runtime has a built-in zstd decoder", () => {
15
+ assert.equal(typeof zlib.zstdDecompressSync, "function");
16
+ });
17
+
18
+ test("Zed zstd thread blobs decode through the built-in decoder", async () => {
19
+ const text = JSON.stringify({ tokens: { input: 10, output: 4 }, model: "zed-test" });
20
+ const decoded = await decodeZedThreadBlob({
21
+ dataType: "zstd",
22
+ data: zlib.zstdCompressSync(Buffer.from(text)),
23
+ });
24
+ assert.equal(decoded, text);
25
+ });
26
+
27
+ test("concatenated DeepSeek Harness zstd frames are decoded one by one", async () => {
28
+ const first = "{\"seq\":1}\n";
29
+ const second = "{\"seq\":2}\n";
30
+ const payload = Buffer.concat([
31
+ zlib.zstdCompressSync(Buffer.from(first)),
32
+ zlib.zstdCompressSync(Buffer.from(second)),
33
+ ]);
34
+ const decoded = await decodeDshZstd(payload, { maxOutputBytes: 4096 });
35
+ assert.equal(decoded.toString("utf8"), first + second);
36
+ });
package/src/lib/sentry.js DELETED
@@ -1,231 +0,0 @@
1
- "use strict";
2
-
3
- /**
4
- * Optional Sentry error reporting for the Node CLI / EmbeddedServer runtime.
5
- *
6
- * Privacy contract (keep in sync with README "隐私边界"):
7
- * - Off unless TOKENUSAGE_SENTRY_DSN (or SENTRY_DSN) is set.
8
- * - Honors the shared telemetry opt-out
9
- * (TOKENUSAGE_NO_TELEMETRY / DO_NOT_TRACK / config.json `"telemetry": false`).
10
- * - Fail-closed in the node test runner.
11
- * - Never intended to carry prompts, code, or provider secrets; beforeSend
12
- * scrubs common secret patterns from event payloads.
13
- */
14
-
15
- const path = require("node:path");
16
- const pkg = require("../../package.json");
17
- const { readJson } = require("./fs");
18
- const { isTelemetryDisabled } = require("./telemetry-pref");
19
-
20
- const SECRET_PATTERN =
21
- /(?:api[_-]?key|access[_-]?token|refresh[_-]?token|cookie|secret|password|dsn)[=:\s]+["']?[^\s"'\\]{8,}/gi;
22
- const BEARER_PATTERN = /\bBearer\s+[A-Za-z0-9._~+/=-]{8,}/gi;
23
- const JWT_PATTERN = /\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9._-]{10,}\.[A-Za-z0-9._-]{10,}\b/g;
24
-
25
- let initialized = false;
26
-
27
- function resolveDsn(env = process.env) {
28
- const raw = env?.TOKENUSAGE_SENTRY_DSN || env?.SENTRY_DSN || "";
29
- return typeof raw === "string" ? raw.trim() : "";
30
- }
31
-
32
- function resolveEnvironment(env = process.env) {
33
- const raw = typeof env?.TOKENUSAGE_SENTRY_ENVIRONMENT === "string"
34
- ? env.TOKENUSAGE_SENTRY_ENVIRONMENT.trim()
35
- : "";
36
- if (raw) return raw;
37
- if (env?.NODE_ENV === "production") return "production";
38
- return "development";
39
- }
40
-
41
- function resolveShell(env = process.env) {
42
- const raw = typeof env?.TOKENUSAGE_APP_SHELL === "string"
43
- ? env.TOKENUSAGE_APP_SHELL.trim().toLowerCase()
44
- : "";
45
- if (raw === "macos" || raw === "windows" || raw === "linux" || raw === "cli") return raw;
46
- return "cli";
47
- }
48
-
49
- function scrubString(value) {
50
- if (typeof value !== "string" || !value) return value;
51
- return value
52
- .replace(BEARER_PATTERN, "Bearer [Filtered]")
53
- .replace(JWT_PATTERN, "[Filtered]")
54
- .replace(SECRET_PATTERN, "[Filtered]");
55
- }
56
-
57
- function scrubValue(value, depth = 0) {
58
- if (depth > 6 || value == null) return value;
59
- if (typeof value === "string") return scrubString(value);
60
- if (Array.isArray(value)) return value.map((item) => scrubValue(item, depth + 1));
61
- if (typeof value === "object") {
62
- const out = {};
63
- for (const [key, nested] of Object.entries(value)) {
64
- const lower = key.toLowerCase();
65
- if (
66
- lower.includes("authorization") ||
67
- lower.includes("cookie") ||
68
- lower.includes("password") ||
69
- lower.includes("secret") ||
70
- lower.includes("token") ||
71
- lower.includes("api_key") ||
72
- lower.includes("apikey")
73
- ) {
74
- out[key] = "[Filtered]";
75
- } else {
76
- out[key] = scrubValue(nested, depth + 1);
77
- }
78
- }
79
- return out;
80
- }
81
- return value;
82
- }
83
-
84
- function truncateMaybe(value, max = 240) {
85
- if (typeof value !== "string") return value;
86
- if (value.length <= max) return value;
87
- return `${value.slice(0, max)}…[truncated]`;
88
- }
89
-
90
- function scrubEvent(event) {
91
- if (!event || typeof event !== "object") return event;
92
- const next = { ...event };
93
- if (next.message) next.message = truncateMaybe(scrubString(next.message));
94
- if (next.extra) next.extra = scrubValue(next.extra);
95
- if (next.contexts) next.contexts = scrubValue(next.contexts);
96
- if (next.request) next.request = scrubValue(next.request);
97
- // Console breadcrumbs can carry parser / JSON.parse excerpts of local logs
98
- // (prompt text). Drop them entirely; keep only non-console crumbs and scrub.
99
- if (Array.isArray(next.breadcrumbs)) {
100
- next.breadcrumbs = next.breadcrumbs
101
- .filter((crumb) => crumb && crumb.category !== "console")
102
- .map((crumb) => scrubValue(crumb));
103
- }
104
- if (next.exception?.values) {
105
- next.exception = {
106
- ...next.exception,
107
- values: next.exception.values.map((item) => ({
108
- ...item,
109
- value: truncateMaybe(scrubString(item?.value)),
110
- })),
111
- };
112
- }
113
- // Never attach IP / user PII by default.
114
- if (next.user) {
115
- next.user = { id: typeof next.user.id === "string" ? next.user.id : undefined };
116
- }
117
- return next;
118
- }
119
-
120
- /**
121
- * Initialise Sentry once for this process. Never throws.
122
- * @returns {Promise<{ enabled: boolean, reason?: string }>}
123
- */
124
- async function initSentry({
125
- env = process.env,
126
- trackerDir,
127
- version = pkg.version,
128
- } = {}) {
129
- try {
130
- if (initialized) return { enabled: true, reason: "already-initialized" };
131
- if (env?.NODE_TEST_CONTEXT !== undefined) {
132
- return { enabled: false, reason: "test-env" };
133
- }
134
-
135
- const dsn = resolveDsn(env);
136
- if (!dsn) return { enabled: false, reason: "no-dsn" };
137
-
138
- let config = {};
139
- if (trackerDir) {
140
- config = (await readJson(path.join(trackerDir, "config.json"))) || {};
141
- }
142
- if (isTelemetryDisabled({ env, config })) {
143
- return { enabled: false, reason: "opt-out" };
144
- }
145
-
146
- // Honor the same outbound proxy / fail-closed policy as the rest of the CLI.
147
- // Sentry's Node transport uses node:https (not undici), so pass proxy explicitly.
148
- const {
149
- readPersistedProxyConfig,
150
- resolveEffectiveProxySource,
151
- } = require("./proxy-env");
152
- const proxyConfig = readPersistedProxyConfig();
153
- const proxyResolved = resolveEffectiveProxySource({ env, proxyConfig });
154
- if (proxyResolved?.source === "blocked") {
155
- return { enabled: false, reason: "proxy-blocked" };
156
- }
157
-
158
- // Lazy require keeps unit tests that never call init from paying the
159
- // module graph cost at import time of unrelated modules.
160
- const Sentry = require("@sentry/node");
161
- Sentry.init({
162
- dsn,
163
- environment: resolveEnvironment(env),
164
- release: `tokenusage-cli@${String(version || "").trim() || "0.0.0"}`,
165
- sendDefaultPii: false,
166
- tracesSampleRate: 0,
167
- // Errors only: drop process-session + console; replace Http so request
168
- // sessions stay off (Http defaults sessions:true and bypasses beforeSend).
169
- integrations(defaults) {
170
- const filtered = defaults.filter((integration) => {
171
- const name = integration?.name || "";
172
- return name !== "ProcessSession" && name !== "Console" && name !== "Http";
173
- });
174
- filtered.push(Sentry.httpIntegration({ sessions: false }));
175
- return filtered;
176
- },
177
- transportOptions: proxyResolved?.proxyUrl
178
- ? { proxy: proxyResolved.proxyUrl }
179
- : undefined,
180
- beforeSend(event) {
181
- return scrubEvent(event);
182
- },
183
- initialScope: {
184
- tags: {
185
- shell: resolveShell(env),
186
- runtime: "node",
187
- },
188
- },
189
- });
190
- initialized = true;
191
- return { enabled: true };
192
- } catch {
193
- return { enabled: false, reason: "init-failed" };
194
- }
195
- }
196
-
197
- function captureException(error, context) {
198
- if (!initialized) return;
199
- try {
200
- const Sentry = require("@sentry/node");
201
- if (context) {
202
- Sentry.withScope((scope) => {
203
- if (context.tags) {
204
- for (const [key, value] of Object.entries(context.tags)) {
205
- scope.setTag(key, String(value));
206
- }
207
- }
208
- if (context.extra) scope.setExtras(scrubValue(context.extra));
209
- Sentry.captureException(error);
210
- });
211
- return;
212
- }
213
- Sentry.captureException(error);
214
- } catch {
215
- /* never break the CLI */
216
- }
217
- }
218
-
219
- module.exports = {
220
- initSentry,
221
- captureException,
222
- resolveDsn,
223
- resolveEnvironment,
224
- resolveShell,
225
- scrubEvent,
226
- scrubString,
227
- // test helpers
228
- _resetForTests() {
229
- initialized = false;
230
- },
231
- };
@@ -1,28 +0,0 @@
1
- "use strict";
2
-
3
- /**
4
- * Outbound telemetry opt-out, shared by the optional Sentry error reporting.
5
- *
6
- * 0.2.3 removed the upstream anonymous install heartbeat: this backend never
7
- * implemented `/api/functions/tokenusage-telemetry`, so every sync only earned
8
- * a 404. Sentry remains strictly opt-in (needs a DSN) and honors the same
9
- * opt-out switches:
10
- * - TOKENUSAGE_NO_TELEMETRY=1
11
- * - DO_NOT_TRACK (the standard)
12
- * - `"telemetry": false` in config.json
13
- */
14
-
15
- function isTruthyFlag(value) {
16
- if (typeof value !== "string") return false;
17
- const v = value.trim().toLowerCase();
18
- return v === "1" || v === "true";
19
- }
20
-
21
- function isTelemetryDisabled({ env = process.env, config = {} } = {}) {
22
- if (isTruthyFlag(env?.TOKENUSAGE_NO_TELEMETRY)) return true;
23
- if (isTruthyFlag(env?.DO_NOT_TRACK)) return true;
24
- if (config && config.telemetry === false) return true;
25
- return false;
26
- }
27
-
28
- module.exports = { isTelemetryDisabled };