dsh-cost-meter 1.9.56 → 1.9.58

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
 
7
7
  **Session cost tracking plugin for the DeepSeek Harness web GUI (bilingual UI)**
8
8
 
9
- [![version](https://img.shields.io/badge/version-1.9.56-4176E6)](https://github.com/Han-1413141/dsh-cost-meter)
9
+ [![version](https://img.shields.io/badge/version-1.9.58-4176E6)](https://github.com/Han-1413141/dsh-cost-meter)
10
10
  [![npm](https://img.shields.io/npm/v/dsh-cost-meter?label=npm)](https://www.npmjs.com/package/dsh-cost-meter)
11
11
  [![dshfind](https://dshfind.com/api/badge/Han-1413141/dsh-cost-meter?metric=downloads&lang=en)](https://dshfind.com/zh/plugins/Han-1413141/dsh-cost-meter?ref=badge)
12
12
  [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
@@ -16,11 +16,11 @@
16
16
 
17
17
  </div>
18
18
 
19
- **v1.9.56** keeps fee reads inside the two-second ledger save window, combining frequent updates into fewer background transactions while showing current usage.
19
+ **v1.9.58** fixes the session-cost entry disappearing when conversation usage arrives before plugin settings. The entry stays visible while loading and shows the amount once settings are ready.
20
20
 
21
21
  **Host floor: DSH 0.2.0** · Adaptation target: **0.2.2-alpha.3**. Expand the compatibility notes below for exact declarations, host records and troubleshooting.
22
22
 
23
- Release notes: [v1.9.56](docs/update-notes-1.9.56.md#english). Fully restart DSH after updating.
23
+ Release notes: [v1.9.58](docs/update-notes-1.9.58.md#english). Fully restart DSH after updating.
24
24
 
25
25
  ![Promo art](docs/promo.en.png)
26
26
 
@@ -352,22 +352,22 @@ On Node.js 20, use `npm install -g pnpm@10` instead. See [pnpm installation and
352
352
  dsh plugin --profile web add dsh-cost-meter
353
353
  ```
354
354
 
355
- **PowerShell one-click script** (copy the whole line, paste, press Enter; pnpm is provisioned automatically, git is auto-detected — no clone needed; the install chain is **pinned to the release tag `v1.9.56`** — review the script before running):
355
+ **PowerShell one-click script** (copy the whole line, paste, press Enter; pnpm is provisioned automatically, git is auto-detected — no clone needed; the install chain is **pinned to the release tag `v1.9.58`** — review the script before running):
356
356
 
357
357
  ```powershell
358
- irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.56/install.ps1 | iex
358
+ irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.58/install.ps1 | iex
359
359
  ```
360
360
 
361
361
  **Or a plain command line** (the machine must already have pnpm and git; also pinned to the tag):
362
362
 
363
363
  ```sh
364
- dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.56
364
+ dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.58
365
365
  ```
366
366
 
367
367
  Without git, use the GitHub tag archive:
368
368
 
369
369
  ```sh
370
- dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.56.tar.gz
370
+ dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.58.tar.gz
371
371
  ```
372
372
 
373
373
  After installing, **restart** `dsh web` (plugin rows, the Typert manifest and the client bundle are all scanned at startup):
@@ -432,7 +432,7 @@ Workarounds:
432
432
 
433
433
  ```yaml
434
434
  minimumReleaseAgeExclude:
435
- - dsh-cost-meter@1.9.56
435
+ - dsh-cost-meter@1.9.58
436
436
  ```
437
437
 
438
438
  3. For a one-off CLI operation, forward the pnpm flag: `dsh plugin --profile desktop remove dsh-cost-meter --config.minimum-release-age=0` (or add `--trust-lockfile` when only lockfile verification fails). Both flags were verified against pnpm 11.28.5 in a temporary profile.
package/README.zh-CN.md CHANGED
@@ -6,7 +6,7 @@
6
6
 
7
7
  **DeepSeek Harness 会话费用统计插件(界面中英双语)**
8
8
 
9
- [![version](https://img.shields.io/badge/version-1.9.56-4176E6)](https://github.com/Han-1413141/dsh-cost-meter)
9
+ [![version](https://img.shields.io/badge/version-1.9.58-4176E6)](https://github.com/Han-1413141/dsh-cost-meter)
10
10
  [![npm](https://img.shields.io/npm/v/dsh-cost-meter?label=npm)](https://www.npmjs.com/package/dsh-cost-meter)
11
11
  [![dshfind](https://dshfind.com/api/badge/Han-1413141/dsh-cost-meter?metric=downloads&lang=zh)](https://dshfind.com/zh/plugins/Han-1413141/dsh-cost-meter?ref=badge)
12
12
  [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
@@ -16,11 +16,11 @@
16
16
 
17
17
  </div>
18
18
 
19
- **v1.9.56** 修复费用查询提前保存账本的问题,在两秒窗口内合并频繁更新,减少重复后台事务,费用继续实时显示。
19
+ **v1.9.58** 修复输入框下方会话费用偶尔消失:会话用量先于插件设置到达时保留加载提示,设置加载后正常显示金额。
20
20
 
21
21
  **宿主要求:DSH 0.2.0 起支持** · 适配目标 **0.2.2-alpha.3**。完整声明、历史记录与安装排障见下方展开项。
22
22
 
23
- 更新说明:[v1.9.56](docs/update-notes-1.9.56.md#中文)。更新后完全重启 DSH。
23
+ 更新说明:[v1.9.58](docs/update-notes-1.9.58.md#中文)。更新后完全重启 DSH。
24
24
 
25
25
  ![宣传图](docs/promo.png)
26
26
 
@@ -352,22 +352,22 @@ Node.js 20 请改用 `npm install -g pnpm@10`。版本要求见 [pnpm 官方安
352
352
  dsh plugin --profile web add dsh-cost-meter
353
353
  ```
354
354
 
355
- **PowerShell 一键脚本**(复制整行粘贴回车;自动补齐 pnpm、自动探测 git,无需克隆仓库;安装链**固定到发布 tag `v1.9.56`**,建议先下载审阅再运行):
355
+ **PowerShell 一键脚本**(复制整行粘贴回车;自动补齐 pnpm、自动探测 git,无需克隆仓库;安装链**固定到发布 tag `v1.9.58`**,建议先下载审阅再运行):
356
356
 
357
357
  ```powershell
358
- irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.56/install.ps1 | iex
358
+ irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.58/install.ps1 | iex
359
359
  ```
360
360
 
361
361
  **或直接命令行**(机器上需已有 pnpm 与 git;同样固定到 tag):
362
362
 
363
363
  ```sh
364
- dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.56
364
+ dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.58
365
365
  ```
366
366
 
367
367
  没有 git 时可用 GitHub tag 打包直链:
368
368
 
369
369
  ```sh
370
- dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.56.tar.gz
370
+ dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.58.tar.gz
371
371
  ```
372
372
 
373
373
  安装后**重启** `dsh web`(插件行、Typert 清单与客户端 bundle 均在启动时扫描):
@@ -432,7 +432,7 @@ DSH **0.2.1-rc.1 / 0.2.1-rc.2** 宿主自带 pnpm **11.28.5**,默认启用内
432
432
 
433
433
  ```yaml
434
434
  minimumReleaseAgeExclude:
435
- - dsh-cost-meter@1.9.56
435
+ - dsh-cost-meter@1.9.58
436
436
  ```
437
437
 
438
438
  3. 仅本次操作用 CLI 透传 pnpm 参数:`dsh plugin --profile desktop remove dsh-cost-meter --config.minimum-release-age=0`;若只是 lockfile 校验阶段失败,可改用 `--trust-lockfile`。这两个参数已在本机 pnpm 11.28.5 临时 Profile 上实测可用。
@@ -0,0 +1,17 @@
1
+ # v1.9.57 — 桌面更新退出超时修复
2
+
3
+ ## 中文
4
+
5
+ - 修复历史回填已经开始后,Desktop Host 无法在退出期限内结束、导致客户端更新失败的问题。此前只清理启动计时器,已经开始的读取仍继续运行;现在退出时取消扫描,等待文件关闭,再保存账本。
6
+ - 取消信号覆盖 JSONL、zstd 压缩会话、SQLite 会话分页,以及搜索和 Billion Context 历史。取消后不继续读取其他会话,也不把取消当成文件损坏后继续尝试其他后端。
7
+ - 被中断的导入和 fork 修复保留重试状态,下次启动继续处理;正常读取和计费方式保持不变。
8
+
9
+ 将费用插件更新到 **1.9.57** 后,完全退出并重新启动 DSH,再执行客户端更新。若曾安装本地临时补丁,正式版本已包含该修复,无需重复修改文件。
10
+
11
+ ## English
12
+
13
+ - Fix Desktop update failures caused by startup history reads keeping the Host alive beyond its shutdown deadline. Shutdown now cancels active scans, waits for their readers to close, and then saves the ledger.
14
+ - Propagate cancellation through JSONL, zstd, SQLite pagination, search history and Billion Context history. Cancelled reads stop instead of falling back to another backend or continuing with the next session.
15
+ - Keep interrupted imports and fork repairs eligible for retry on the next startup, without changing normal billing behavior.
16
+
17
+ Update the cost plugin to **1.9.57**, fully quit and restart DSH, then retry the Desktop update. This release includes the local temporary fix; manual file patches are no longer needed.
@@ -0,0 +1,17 @@
1
+ # v1.9.58 — 会话费用入口消失修复
2
+
3
+ ## 中文
4
+
5
+ - 修复恢复已有会话或重新加载插件时,输入框下方会话金额偶尔消失的问题。会话用量或账本可能先于插件设置到达,旧版在设置尚未就绪时计算费用,触发渲染异常,使宿主停止显示费用入口。
6
+ - 现在设置未就绪时保留加载或错误提示,设置加载后正常显示已记录的金额。标题栏的费用入口也包含这一修复。
7
+ - 保留手动隐藏、显示位置、API/Plan 金额拆分和币种设置;切换会话时不会显示上一会话金额。
8
+
9
+ 将费用插件更新到 **1.9.58** 后,完全退出并重新启动 DSH,即可重新加载费用入口,无需修改显示设置。
10
+
11
+ ## English
12
+
13
+ - Fix the session-cost entry occasionally disappearing when restoring a conversation or reloading the plugin. Usage or ledger data could arrive before configuration, causing a render exception that made the host stop displaying the entry.
14
+ - Keep the loading or error message visible until configuration is available, then display the recorded amount. The header cost entry receives the same fix.
15
+ - Preserve explicit visibility preferences, display position, API/Plan separation and currency settings. Switching conversations does not display the previous conversation's amount.
16
+
17
+ Update the plugin to **1.9.58**, fully quit and restart DSH. No display-setting changes are required.
package/lib/backfill.js CHANGED
@@ -156,10 +156,11 @@ export function scanZstdFrames(buffer) {
156
156
  * 单文件累计解压字节数超过 maxDecompressed(默认 4GB,防解压炸弹)抛
157
157
  * RangeError,调用方按单文件损坏处理。
158
158
  * @param path - session.jsonl.zstd 或 session.jsonl 路径。
159
- * @param options - { maxDecompressed?: number, yieldEvery?: number }。
159
+ * @param options - { maxDecompressed?: number, yieldEvery?: number, signal?: AbortSignal }。
160
160
  * @yields {object} 逐行解析出的记录(空行/打包行/坏行已过滤)。
161
161
  */
162
162
  export async function* iterateSessionRecords(path, options = {}) {
163
+ options.signal?.throwIfAborted()
163
164
  const maxDecompressed = Number(options.maxDecompressed) > 0 ? Number(options.maxDecompressed) : MAX_DECOMPRESSED_PER_FILE
164
165
  const yieldEvery = Number(options.yieldEvery) > 0 ? Number(options.yieldEvery) : YIELD_EVERY_FRAMES
165
166
  const maxFrameBytes = Number(options.maxFrameBytes) > 0 ? Number(options.maxFrameBytes) : MAX_FRAME_BYTES
@@ -209,7 +210,9 @@ export async function* iterateSessionRecords(path, options = {}) {
209
210
  // Only invalid frame structure stops the scan; an incomplete frame keeps reading.
210
211
  let stopped = false
211
212
  for (;;) {
213
+ options.signal?.throwIfAborted()
212
214
  const chunk = await handle.read(Buffer.allocUnsafe(READ_CHUNK_BYTES), 0, READ_CHUNK_BYTES, null)
215
+ options.signal?.throwIfAborted()
213
216
  if (chunk.bytesRead === 0) break
214
217
  const data = tail.length > 0 ? Buffer.concat([tail, chunk.buffer.subarray(0, chunk.bytesRead)]) : chunk.buffer.subarray(0, chunk.bytesRead)
215
218
  tail = Buffer.alloc(0)
@@ -217,7 +220,7 @@ export async function* iterateSessionRecords(path, options = {}) {
217
220
  if (!isZstd) {
218
221
  consumeBytes(data, records)
219
222
  if (records.length >= YIELD_EVERY_EVENTS) {
220
- yield* drain(records)
223
+ yield* drain(records, options.signal)
221
224
  await new Promise(resolve => setImmediate(resolve))
222
225
  }
223
226
  continue
@@ -226,6 +229,7 @@ export async function* iterateSessionRecords(path, options = {}) {
226
229
  // 解压→逐行→释放引用;剩余不完整帧(块边界截断)留在 tail 等下一块。
227
230
  let offset = 0
228
231
  while (offset < data.length) {
232
+ options.signal?.throwIfAborted()
229
233
  const frame = scanZstdFrame(data, offset)
230
234
  if (frame.status !== 'complete') {
231
235
  if (frame.status === 'invalid') stopped = true
@@ -251,12 +255,12 @@ export async function* iterateSessionRecords(path, options = {}) {
251
255
  framesSinceYield += 1
252
256
  offset = frame.end
253
257
  if (framesSinceYield >= yieldEvery || records.length >= YIELD_EVERY_EVENTS) {
254
- yield* drain(records); framesSinceYield = 0
258
+ yield* drain(records, options.signal); framesSinceYield = 0
255
259
  await new Promise(resolve => setImmediate(resolve))
256
260
  }
257
261
  }
258
262
  if (framesSinceYield >= yieldEvery || records.length >= YIELD_EVERY_EVENTS) {
259
- yield* drain(records)
263
+ yield* drain(records, options.signal)
260
264
  framesSinceYield = 0
261
265
  await new Promise(resolve => setImmediate(resolve))
262
266
  }
@@ -270,15 +274,18 @@ export async function* iterateSessionRecords(path, options = {}) {
270
274
  records.push(rec)
271
275
  }
272
276
  }
273
- yield* drain(records)
277
+ yield* drain(records, options.signal)
274
278
  } finally {
275
279
  await handle.close()
276
280
  }
277
281
  }
278
282
 
279
283
  /** 把 records 数组逐条 yield 出去并清空(内部助手)。 */
280
- function* drain(records) {
281
- for (const rec of records) yield rec
284
+ function* drain(records, signal) {
285
+ for (const rec of records) {
286
+ signal?.throwIfAborted()
287
+ yield rec
288
+ }
282
289
  records.length = 0
283
290
  }
284
291
 
@@ -313,12 +320,20 @@ export function listSessionLogs(root, onlySessionIds = null) {
313
320
  * `listSessions(onlySessionIds)` 枚举 + `readRecords(id)` 读取(JSONL 文件或 SQLite
314
321
  * 由适配器自动选择);为空时保持原有 `listSessionLogs` 文件枚举路径不变。
315
322
  */
323
+ function checkBackfillActive(ledger, sessionSource) {
324
+ sessionSource?.signal?.throwIfAborted()
325
+ if (ledger.closed) throw new DOMException('Ledger closed', 'AbortError')
326
+ }
327
+
316
328
  async function* sessionBillingRecords(ledger, sessionsRoot, onlySessionIds = null, sessionSource = null) {
329
+ checkBackfillActive(ledger, sessionSource)
330
+ const signal = sessionSource?.signal
317
331
  const seen = new Set()
318
332
  const hasSource = sessionSource !== null && sessionSource !== undefined
319
333
  const sessions = hasSource ? await sessionSource.listSessions(onlySessionIds) : null
320
334
  const paths = hasSource ? null : listSessionLogs(sessionsRoot, onlySessionIds)
321
335
  for (const entry of sessions ?? paths) {
336
+ checkBackfillActive(ledger, sessionSource)
322
337
  let records
323
338
  let id
324
339
  try {
@@ -330,32 +345,46 @@ async function* sessionBillingRecords(ledger, sessionsRoot, onlySessionIds = nul
330
345
  records = await readSessionRecordsAsync(entry)
331
346
  id = records.find(record => record.type === 'session')?.id
332
347
  }
333
- } catch { yield null; continue }
348
+ } catch (error) {
349
+ checkBackfillActive(ledger, sessionSource)
350
+ yield null
351
+ continue
352
+ }
353
+ checkBackfillActive(ledger, sessionSource)
334
354
  if (!Array.isArray(records)) { yield null; continue }
335
355
  if (typeof id === 'string') {
336
356
  seen.add(id)
337
- try { for (const record of await nativeSearchRecordsFor(ledger.path, id)) records.push(record) } catch { /* Keep the readable host prefix. */ }
338
- try { records.push(...await billionRecordsFor(ledger.path, id)) } catch { /* Keep the readable host prefix. */ }
357
+ try { for (const record of await nativeSearchRecordsFor(ledger.path, id, signal)) records.push(record) } catch { checkBackfillActive(ledger, sessionSource) }
358
+ try { records.push(...await billionRecordsFor(ledger.path, id, signal)) } catch { checkBackfillActive(ledger, sessionSource) }
339
359
  }
360
+ checkBackfillActive(ledger, sessionSource)
340
361
  yield records
341
362
  }
342
363
  // A deleted/missing host log must not discard the plugin's surviving search history.
343
364
  for (const path of listNativeSearchHistory(ledger.path, seen, onlySessionIds)) {
365
+ checkBackfillActive(ledger, sessionSource)
344
366
  let history
345
- try { history = await readNativeSearchHistory(path) } catch { continue }
367
+ try { history = await readNativeSearchHistory(path, undefined, signal) } catch { checkBackfillActive(ledger, sessionSource); continue }
368
+ checkBackfillActive(ledger, sessionSource)
346
369
  const id = history.sessionId
347
370
  if (!id || seen.has(id) || (onlySessionIds !== null && !onlySessionIds.has(id))) continue
348
371
  seen.add(id)
349
- yield [{ type: 'session', id, createdAt: 0 }, ...history.records, ...await billionRecordsFor(ledger.path, id)]
372
+ const extra = await billionRecordsFor(ledger.path, id, signal)
373
+ checkBackfillActive(ledger, sessionSource)
374
+ yield [{ type: 'session', id, createdAt: 0 }, ...history.records, ...extra]
350
375
  }
351
- for (const file of listBillionFiles(ledger.path)) for await (const { row } of iterateBillionCalls(file)) {
376
+ for (const file of listBillionFiles(ledger.path)) for await (const { row } of iterateBillionCalls(file, 0, signal)) {
377
+ checkBackfillActive(ledger, sessionSource)
352
378
  const id = row.sessionId
353
379
  if (!seen.has(id) && (onlySessionIds === null || onlySessionIds.has(id))) {
354
380
  seen.add(id)
355
- yield [{ type: 'session', id, createdAt: 0 }, ...await billionRecordsFor(ledger.path, id)]
381
+ const extra = await billionRecordsFor(ledger.path, id, signal)
382
+ checkBackfillActive(ledger, sessionSource)
383
+ yield [{ type: 'session', id, createdAt: 0 }, ...extra]
356
384
  }
357
385
  break
358
386
  }
387
+ checkBackfillActive(ledger, sessionSource)
359
388
  }
360
389
 
361
390
  /**
@@ -1210,6 +1239,7 @@ function subtractBucketInto(target, source) {
1210
1239
  * @returns { sessions, days, scanned } 受影响的 fork 会话数、扣除的日期数与扫描文件数。
1211
1240
  */
1212
1241
  export async function repairForkSeed(ledger, sessionsRoot, sessionSource = null) {
1242
+ checkBackfillActive(ledger, sessionSource)
1213
1243
  const result = { sessions: 0, days: 0, scanned: 0 }
1214
1244
  const hasSource = sessionSource !== null && sessionSource !== undefined
1215
1245
  // 统一记录源(T3):RDB 下从数据库枚举会话;未传源时保持原 JSONL 文件枚举路径不变。
@@ -1220,12 +1250,16 @@ export async function repairForkSeed(ledger, sessionsRoot, sessionSource = null)
1220
1250
  // 全量同步扫描会长时间占住事件循环:每 8 份让出一次(与回填/导入同策略);
1221
1251
  // 单文件内部亦为流式读取(v1.7.7,issue #88)。
1222
1252
  if ((result.scanned += 1) % 8 === 0) await new Promise(resolve => setImmediate(resolve))
1253
+ checkBackfillActive(ledger, sessionSource)
1223
1254
  let records
1224
1255
  try {
1225
1256
  records = hasSource ? await sessionSource.readRecords(entry) : await readSessionRecordsAsync(entry)
1226
1257
  } catch {
1258
+ checkBackfillActive(ledger, sessionSource)
1227
1259
  continue // 单文件损坏(含解压预算超限)不阻断整体清洗。
1228
1260
  }
1261
+ checkBackfillActive(ledger, sessionSource)
1262
+ if (!Array.isArray(records)) continue
1229
1263
  // 只处理 fork 会话(header 带 parentSession);普通会话日志无种子段。
1230
1264
  const header = records.find(r => r?.type === 'session')
1231
1265
  if (header === undefined || header === null || typeof header.parentSession !== 'string' || header.parentSession.length === 0) continue
@@ -59,9 +59,9 @@ export function readBillionStreamFile(file) {
59
59
  && billionSessionKey(stream.sessionId) === file.key && file.path.endsWith(stream.streamId + '.json') ? stream : null
60
60
  } catch { return null }
61
61
  }
62
- export async function readBillionStreamFileAsync(file) {
62
+ export async function readBillionStreamFileAsync(file, signal) {
63
63
  try {
64
- const stream = await readSmallBillionJson(file.path)
64
+ const stream = await readSmallBillionJson(file.path, signal)
65
65
  return stream?.protocol === 1 && uuid(stream.streamId) && validSessionId(stream.sessionId)
66
66
  && billionSessionKey(stream.sessionId) === file.key && file.path.endsWith(stream.streamId + '.json') ? stream : null
67
67
  } catch (error) {
@@ -94,11 +94,12 @@ export function billionSourcePaths(ledgerPath, sessionId) {
94
94
  ...directories(streamRoot).filter(row => row.isFile() && row.name.endsWith('.json') && uuid(row.name.slice(0, -5))).map(row => join(streamRoot, row.name))]
95
95
  }
96
96
  /** Bounded UTF-8 rows; offsets only advance through complete newline-terminated writes. */
97
- export async function* iterateBillionCalls(file, start = 0) {
97
+ export async function* iterateBillionCalls(file, start = 0, signal) {
98
98
  let pending = Buffer.alloc(0), dropping = false, offset = start
99
- for await (const chunk of journalChunks(file.path, start)) {
99
+ for await (const chunk of journalChunks(file.path, start, undefined, signal)) {
100
100
  const bytes = Buffer.concat([pending, chunk]); let from = 0, end
101
101
  while ((end = bytes.indexOf(10, from)) >= 0) {
102
+ signal?.throwIfAborted()
102
103
  const line = bytes.subarray(from, end); offset += end - from + 1; from = end + 1
103
104
  if (dropping) { dropping = false; continue }
104
105
  let row
@@ -123,11 +124,11 @@ export function readBillionStream(ledgerPath, row) {
123
124
  return stream.protocol === 1 && stream.streamId === row.streamId && stream.sessionId === row.sessionId ? stream : null
124
125
  } catch { return null }
125
126
  }
126
- export async function readBillionStreamAsync(ledgerPath, row, archives = new Map()) {
127
+ export async function readBillionStreamAsync(ledgerPath, row, archives = new Map(), signal) {
127
128
  const file = { path: billionStreamPath(billionRoot(ledgerPath), row.sessionId, row.streamId), key: billionSessionKey(row.sessionId) }
128
- const stream = await readBillionStreamFileAsync(file)
129
+ const stream = await readBillionStreamFileAsync(file, signal)
129
130
  if (stream) return stream
130
- if (!archives.has(row.sessionId)) archives.set(row.sessionId, await archivedBillionStreams(ledgerPath, row.sessionId))
131
+ if (!archives.has(row.sessionId)) archives.set(row.sessionId, await archivedBillionStreams(ledgerPath, row.sessionId, signal))
131
132
  return archives.get(row.sessionId).get(row.streamId) ?? null
132
133
  }
133
134
  const coverageCounts = () => ({ known: 0, unknown: 0, matched: false })
@@ -159,10 +160,11 @@ function coverageResult(stream, { known, unknown, matched }) {
159
160
  return checkedBillionCoverage(stream, { version: 1, mode, known, unknown })
160
161
  }
161
162
  /** A persistence receipt says nothing about whether model usage was observed. */
162
- export async function billionCoverage(ledgerPath, stream, offsets) {
163
+ export async function billionCoverage(ledgerPath, stream, offsets, signal) {
164
+ signal?.throwIfAborted()
163
165
  if (stream.coverage?.version === 1) return checkedBillionCoverage(stream, stream.coverage)
164
166
  const counts = coverageCounts()
165
- for (const file of listBillionFiles(ledgerPath, stream.sessionId)) for await (const { row } of iterateBillionCalls(file, offsets?.get(file.path)?.offset ?? 0)) {
167
+ for (const file of listBillionFiles(ledgerPath, stream.sessionId)) for await (const { row } of iterateBillionCalls(file, offsets?.get(file.path)?.offset ?? 0, signal)) {
166
168
  if (row.streamId === stream.streamId) observeCoverage(counts, stream, row)
167
169
  }
168
170
  return coverageResult(stream, counts)
@@ -188,21 +190,22 @@ export function billionHostRecord(stream, coverage = stream.coverage ?? { mode:
188
190
  startedAtMs: stream.startedAtMs, requestId: stream.streamId, kind: 'model', turn: stream.turn ?? null, step: stream.step ?? null,
189
191
  metering: { source: 'billion-context', requestId: stream.streamId, attempt: 1, complete: false, status: 'complete', upstream: 'host-usage' } } }
190
192
  }
191
- export async function billionRecordsFor(ledgerPath, sessionId) {
193
+ export async function billionRecordsFor(ledgerPath, sessionId, signal) {
194
+ signal?.throwIfAborted()
192
195
  if (!ledgerPath) return []
193
196
  const records = [], streams = new Map(), counts = new Map(), seen = new Set(), archives = new Map()
194
197
  const key = billionSessionKey(sessionId), dir = join(billionRoot(ledgerPath), 'streams', key)
195
- for (const entry of await safeEntries(dir)) {
198
+ for (const entry of await safeEntries(dir, signal)) {
196
199
  if (!entry.isFile() || !entry.name.endsWith('.json') || !uuid(entry.name.slice(0, -5))) continue
197
- const stream = await readBillionStreamFileAsync({ path: join(dir, entry.name), key })
200
+ const stream = await readBillionStreamFileAsync({ path: join(dir, entry.name), key }, signal)
198
201
  if (stream?.finished) streams.set(stream.streamId, stream)
199
202
  }
200
203
  // Originals are removed only after archive publication. Read archives after
201
204
  // originals so a concurrent compaction cannot create a gap in history.
202
- for (const [id, stream] of await archivedBillionStreams(ledgerPath, sessionId)) if (!streams.has(id)) streams.set(id, stream)
203
- for (const file of listBillionFiles(ledgerPath, sessionId)) for await (const { row } of iterateBillionCalls(file)) {
205
+ for (const [id, stream] of await archivedBillionStreams(ledgerPath, sessionId, signal)) if (!streams.has(id)) streams.set(id, stream)
206
+ for (const file of listBillionFiles(ledgerPath, sessionId)) for await (const { row } of iterateBillionCalls(file, 0, signal)) {
204
207
  if (seen.has(row.callId)) continue
205
- const stream = streams.get(row.streamId) ?? await readBillionStreamAsync(ledgerPath, row, archives)
208
+ const stream = streams.get(row.streamId) ?? await readBillionStreamAsync(ledgerPath, row, archives, signal)
206
209
  if (!stream?.finished || !['full', 'partial'].includes(stream.policies?.[row.groupId])) continue
207
210
  seen.add(row.callId); streams.set(stream.streamId, stream)
208
211
  if (seen.size > 200000) throw Error('Proxy billing detail limit exceeded')
@@ -7,31 +7,40 @@ const id = '[a-f\\d]{8}(?:-[a-f\\d]{4}){3}-[a-f\\d]{12}'
7
7
  const streamName = new RegExp('^' + id + '\\.json$', 'i')
8
8
  const callName = new RegExp('^(' + id + ')(?:\\.sealed)?\\.jsonl(?:\\.gz)?$', 'i')
9
9
  const archiveName = new RegExp('^' + id + '\\.jsonl\\.gz$', 'i')
10
- export async function safeEntries(path) {
10
+ export async function safeEntries(path, signal) {
11
+ signal?.throwIfAborted()
11
12
  try {
12
13
  const info = await lstat(path)
14
+ signal?.throwIfAborted()
13
15
  if (!info.isDirectory() || info.isSymbolicLink()) return []
14
- return await readdir(path, { withFileTypes: true })
16
+ const entries = await readdir(path, { withFileTypes: true })
17
+ signal?.throwIfAborted()
18
+ return entries
15
19
  } catch (error) { if (error.code === 'ENOENT') return []; throw error }
16
20
  }
17
- export async function readSmallBillionJson(path) {
21
+ export async function readSmallBillionJson(path, signal) {
22
+ signal?.throwIfAborted()
18
23
  const info = await lstat(path)
19
24
  if (!info.isFile() || info.isSymbolicLink() || info.size > 8192) throw Object.assign(Error('Invalid proxy billing metadata'), { code: 'INVALID_BILLION_METADATA' })
20
- return JSON.parse(await readFile(path, 'utf8'))
25
+ const text = await readFile(path, { encoding: 'utf8', signal })
26
+ signal?.throwIfAborted()
27
+ return JSON.parse(text)
21
28
  }
22
29
  export function createBillionInventory(ledgerPath) {
23
30
  const root = ledgerPath + '.billion-context'
24
31
  let cache = new Map()
25
32
  return {
26
- async poll() {
33
+ async poll(signal) {
34
+ signal?.throwIfAborted()
27
35
  const streams = [], calls = [], archives = [], seen = new Set(), next = new Map(cache)
28
36
  let changed = false
29
37
  // Inspect directories, not every completed stream, on an idle tick. Calls
30
38
  // remain candidates because append-only writes do not change directory mtime.
31
- if (!(await safeEntries(root)).length) { cache.clear(); return { streams, calls, archives, retained: new Set() } }
39
+ if (!(await safeEntries(root, signal)).length) { cache.clear(); return { streams, calls, archives, retained: new Set() } }
32
40
  for (const kind of ['streams', 'calls', 'stream-archives']) {
33
41
  const parent = join(root, kind)
34
- for (const entry of await safeEntries(parent)) {
42
+ for (const entry of await safeEntries(parent, signal)) {
43
+ signal?.throwIfAborted()
35
44
  if (!entry.isDirectory() || !sessionKey.test(entry.name)) continue
36
45
  const directory = join(parent, entry.name)
37
46
  let info
@@ -42,7 +51,7 @@ export function createBillionInventory(ledgerPath) {
42
51
  let files = previous?.files
43
52
  if (previous?.revision !== revision) {
44
53
  changed = true
45
- files = (await safeEntries(directory)).flatMap(file => {
54
+ files = (await safeEntries(directory, signal)).flatMap(file => {
46
55
  if (!file.isFile()) return []
47
56
  const match = kind === 'calls' ? callName.exec(file.name) : (kind === 'streams' ? streamName : archiveName).exec(file.name)
48
57
  return match ? [{ path: join(directory, file.name), key: entry.name, ...(kind === 'calls' ? { source: match[1] } : {}) }] : []
@@ -55,6 +64,7 @@ export function createBillionInventory(ledgerPath) {
55
64
  }
56
65
  for (const directory of next.keys()) if (!seen.has(directory)) { next.delete(directory); changed = true }
57
66
  // A failed poll must not mark undispatched candidates as already scanned.
67
+ signal?.throwIfAborted()
58
68
  cache = next
59
69
  const retained = changed ? new Set([...cache.values()].flatMap(item => item.files.map(file => file.path))) : null
60
70
  return { streams, calls, archives, retained }
@@ -18,27 +18,29 @@ const contains = (row, delta) => row && fields.every(key => finite(amount(row, k
18
18
  && amount(row, key) + (['input', 'output', 'cacheRead', 'cacheWrite', 'reasoning', 'calls'].includes(key) ? 0 : 1e-8) >= delta[key])
19
19
  const subtract = (row, delta) => { for (const key of fields) row[key] = Math.max(0, amount(row, key) - delta[key]) }
20
20
 
21
- async function affectedStreams(path) {
22
- const inventory = await createBillionInventory(path).poll(), streams = new Map()
21
+ async function affectedStreams(path, signal) {
22
+ const inventory = await createBillionInventory(path).poll(signal), streams = new Map()
23
23
  for (const file of inventory.streams) {
24
- const stream = await readBillionStreamFileAsync(file)
24
+ const stream = await readBillionStreamFileAsync(file, signal)
25
25
  if (stream?.finished) streams.set(stream.streamId, stream)
26
26
  }
27
27
  // An archive can coexist with an active original during compaction. The active
28
28
  // original wins; for duplicate archives the newest published record wins.
29
29
  const active = new Set(streams.keys()), archives = []
30
30
  for (const file of inventory.archives) {
31
+ signal?.throwIfAborted()
31
32
  try { archives.push({ ...file, at: (await lstat(file.path)).mtimeMs }) }
32
33
  catch (error) { if (error.code !== 'ENOENT') throw error }
33
34
  }
34
35
  archives.sort((a, b) => a.at - b.at || a.path.localeCompare(b.path))
35
- for (const file of archives) for (const stream of await readBillionStreamArchive(file)) {
36
+ for (const file of archives) for (const stream of await readBillionStreamArchive(file, signal)) {
36
37
  if (!active.has(stream.streamId)) streams.set(stream.streamId, stream)
37
38
  if (streams.size > 200000) throw Error('Proxy billing repair stream limit exceeded')
38
39
  }
39
40
  const affected = []
40
41
  for (const stream of streams.values()) {
41
- const coverage = checkedBillionCoverage(stream, stream.coverage ?? await billionCoverage(path, stream))
42
+ signal?.throwIfAborted()
43
+ const coverage = checkedBillionCoverage(stream, stream.coverage ?? await billionCoverage(path, stream, undefined, signal))
42
44
  if (unverifiedBillionHost(coverage)) affected.push(stream)
43
45
  }
44
46
  return affected
@@ -49,10 +51,12 @@ async function affectedStreams(path) {
49
51
  * admitted. Sequence 2 retires that sequence-1 charge in the same transaction as
50
52
  * its subtraction. No global residual deletion or current-price recalculation.
51
53
  */
52
- export async function repairBillionHostLedger(ledger) {
54
+ export async function repairBillionHostLedger(ledger, signal) {
55
+ signal?.throwIfAborted()
53
56
  const stats = { repaired: 0, skipped: 0, input: 0, cost: 0, backup: '' }
54
57
  if (!ledger.path || ledger.closed) return stats
55
- const streams = await affectedStreams(ledger.path)
58
+ const streams = await affectedStreams(ledger.path, signal)
59
+ signal?.throwIfAborted()
56
60
  if (!streams.length || ledger.closed) return stats
57
61
  // Drain our worker before acquiring its lock; keep other hosts' new calls by
58
62
  // reading the latest disk snapshot inside that lock, then adopt the result.
@@ -42,10 +42,14 @@ export function validArchivedStream(stream, key) {
42
42
  return stream?.protocol === 1 && uuid.test(stream.streamId ?? '') && typeof stream.sessionId === 'string'
43
43
  && stream.sessionId.length > 0 && stream.sessionId.length <= 512 && keyOf(stream.sessionId) === key && stream.finished === true
44
44
  }
45
- export async function readBillionStreamArchive(file) {
45
+ export async function readBillionStreamArchive(file, signal) {
46
+ signal?.throwIfAborted()
46
47
  const info = await fileInfo(file.path)
47
48
  if (!info || info.size > MAX_ARCHIVE) throw Error('Invalid archived stream metadata')
48
- const bytes = await decompress(await readFile(file.path), { maxOutputLength: MAX_ARCHIVE })
49
+ const compressed = await readFile(file.path, { signal })
50
+ signal?.throwIfAborted()
51
+ const bytes = await decompress(compressed, { maxOutputLength: MAX_ARCHIVE })
52
+ signal?.throwIfAborted()
49
53
  const rows = []
50
54
  for (const line of bytes.toString('utf8').split('\n')) {
51
55
  if (!line) continue
@@ -56,18 +60,20 @@ export async function readBillionStreamArchive(file) {
56
60
  }
57
61
  return rows
58
62
  }
59
- export async function archivedBillionStreams(ledgerPath, sessionId) {
63
+ export async function archivedBillionStreams(ledgerPath, sessionId, signal) {
64
+ signal?.throwIfAborted()
60
65
  const key = keyOf(sessionId), root = resolve(ledgerPath + '.billion-context'), dir = join(root, 'stream-archives', key), streams = new Map()
61
66
  try { await confined(root, join(dir, 'record')) } catch (error) { if (error.code === 'ENOENT') return streams; throw error }
62
67
  const files = []
63
- for (const entry of await safeEntries(dir)) {
68
+ for (const entry of await safeEntries(dir, signal)) {
69
+ signal?.throwIfAborted()
64
70
  if (!entry.isFile() || !entry.name.endsWith('.jsonl.gz') || !uuid.test(entry.name.slice(0, -9))) continue
65
71
  const path = join(dir, entry.name), info = await fileInfo(path)
66
72
  if (info) files.push({ path, key, at: info.mtimeMs })
67
73
  }
68
74
  files.sort((a, b) => a.at - b.at || a.path.localeCompare(b.path))
69
75
  for (const file of files) {
70
- for (const row of await readBillionStreamArchive(file)) streams.set(row.streamId, row)
76
+ for (const row of await readBillionStreamArchive(file, signal)) streams.set(row.streamId, row)
71
77
  if (streams.size > 200000) throw Error('Proxy billing detail limit exceeded')
72
78
  }
73
79
  return streams