dsh-cost-meter 1.9.33 → 1.9.35
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -8
- package/README.zh-CN.md +8 -8
- package/docs/billing-storage.md +44 -0
- package/docs/update-notes-1.9.34.md +35 -0
- package/docs/update-notes-1.9.35.md +17 -0
- package/lib/billing-history.js +5 -3
- package/lib/billing-storage-cli.js +25 -0
- package/lib/billion-context-billing.js +35 -25
- package/lib/billion-context-history.js +30 -6
- package/lib/billion-context-inventory.js +60 -0
- package/lib/billion-context-storage.js +222 -0
- package/lib/client.billing-storage.js +1 -0
- package/lib/client.js +6 -6
- package/lib/client.peak.js +2 -2
- package/lib/index.js +18 -9
- package/lib/journal-storage.js +12 -1
- package/lib/ledger-writer-worker.js +1 -1
- package/lib/native-search-billing.js +2 -2
- package/lib/pricing.js +13 -8
- package/lib/store.js +5 -0
- package/lib/typert.host.js +13 -0
- package/package.json +6 -2
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
**Session cost tracking plugin for the DeepSeek Harness web GUI (bilingual UI)**
|
|
8
8
|
|
|
9
|
-
[](https://github.com/Han-1413141/dsh-cost-meter)
|
|
10
10
|
[](https://www.npmjs.com/package/dsh-cost-meter)
|
|
11
11
|
[](https://dshfind.com/zh/plugins/Han-1413141/dsh-cost-meter?ref=badge)
|
|
12
12
|
[](LICENSE)
|
|
@@ -16,11 +16,11 @@
|
|
|
16
16
|
|
|
17
17
|
</div>
|
|
18
18
|
|
|
19
|
-
**v1.9.
|
|
19
|
+
**v1.9.35** fixes billion-context idle scanning stalls and adds storage statistics, preview/confirm cleanup, 7/30-day retention and a CLI. Completed streams are compressed while billing history is retained. [Storage guide](docs/billing-storage.md).
|
|
20
20
|
|
|
21
21
|
**Host floor: DSH 0.2.0** · Adaptation target: **0.2.1-rc.2** (npm `next`). Expand the compatibility notes below for exact declarations, host records and troubleshooting.
|
|
22
22
|
|
|
23
|
-
Release notes: [v1.9.
|
|
23
|
+
Release notes: [v1.9.35](docs/update-notes-1.9.35.md#english). Fully restart DSH after updating.
|
|
24
24
|
|
|
25
25
|

|
|
26
26
|
|
|
@@ -352,22 +352,22 @@ On Node.js 20, use `npm install -g pnpm@10` instead. See [pnpm installation and
|
|
|
352
352
|
dsh plugin --profile web add dsh-cost-meter
|
|
353
353
|
```
|
|
354
354
|
|
|
355
|
-
**PowerShell one-click script** (copy the whole line, paste, press Enter; pnpm is provisioned automatically, git is auto-detected — no clone needed; the install chain is **pinned to the release tag `v1.9.
|
|
355
|
+
**PowerShell one-click script** (copy the whole line, paste, press Enter; pnpm is provisioned automatically, git is auto-detected — no clone needed; the install chain is **pinned to the release tag `v1.9.35`** — review the script before running):
|
|
356
356
|
|
|
357
357
|
```powershell
|
|
358
|
-
irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.
|
|
358
|
+
irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.35/install.ps1 | iex
|
|
359
359
|
```
|
|
360
360
|
|
|
361
361
|
**Or a plain command line** (the machine must already have pnpm and git; also pinned to the tag):
|
|
362
362
|
|
|
363
363
|
```sh
|
|
364
|
-
dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.
|
|
364
|
+
dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.35
|
|
365
365
|
```
|
|
366
366
|
|
|
367
367
|
Without git, use the GitHub tag archive:
|
|
368
368
|
|
|
369
369
|
```sh
|
|
370
|
-
dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.
|
|
370
|
+
dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.35.tar.gz
|
|
371
371
|
```
|
|
372
372
|
|
|
373
373
|
After installing, **restart** `dsh web` (plugin rows, the Typert manifest and the client bundle are all scanned at startup):
|
|
@@ -432,7 +432,7 @@ Workarounds:
|
|
|
432
432
|
|
|
433
433
|
```yaml
|
|
434
434
|
minimumReleaseAgeExclude:
|
|
435
|
-
- dsh-cost-meter@1.9.
|
|
435
|
+
- dsh-cost-meter@1.9.35
|
|
436
436
|
```
|
|
437
437
|
|
|
438
438
|
3. For a one-off CLI operation, forward the pnpm flag: `dsh plugin --profile desktop remove dsh-cost-meter --config.minimum-release-age=0` (or add `--trust-lockfile` when only lockfile verification fails). Both flags were verified against pnpm 11.28.5 in a temporary profile.
|
package/README.zh-CN.md
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
**DeepSeek Harness 会话费用统计插件(界面中英双语)**
|
|
8
8
|
|
|
9
|
-
[](https://github.com/Han-1413141/dsh-cost-meter)
|
|
10
10
|
[](https://www.npmjs.com/package/dsh-cost-meter)
|
|
11
11
|
[](https://dshfind.com/zh/plugins/Han-1413141/dsh-cost-meter?ref=badge)
|
|
12
12
|
[](LICENSE)
|
|
@@ -16,11 +16,11 @@
|
|
|
16
16
|
|
|
17
17
|
</div>
|
|
18
18
|
|
|
19
|
-
**v1.9.
|
|
19
|
+
**v1.9.35** 修复 billion-context 空闲扫描卡顿,新增计费历史存储统计、清理预览与确认、7/30 天保留和 CLI;压缩旧流并保留费用明细。[存储管理说明](docs/billing-storage.md)。
|
|
20
20
|
|
|
21
21
|
**宿主要求:DSH 0.2.0 起支持** · 适配目标 **0.2.1-rc.2**(npm `next`)。完整声明、历史记录与安装排障见下方展开项。
|
|
22
22
|
|
|
23
|
-
更新说明:[v1.9.
|
|
23
|
+
更新说明:[v1.9.35](docs/update-notes-1.9.35.md#中文)。更新后完全重启 DSH。
|
|
24
24
|
|
|
25
25
|

|
|
26
26
|
|
|
@@ -352,22 +352,22 @@ Node.js 20 请改用 `npm install -g pnpm@10`。版本要求见 [pnpm 官方安
|
|
|
352
352
|
dsh plugin --profile web add dsh-cost-meter
|
|
353
353
|
```
|
|
354
354
|
|
|
355
|
-
**PowerShell 一键脚本**(复制整行粘贴回车;自动补齐 pnpm、自动探测 git,无需克隆仓库;安装链**固定到发布 tag `v1.9.
|
|
355
|
+
**PowerShell 一键脚本**(复制整行粘贴回车;自动补齐 pnpm、自动探测 git,无需克隆仓库;安装链**固定到发布 tag `v1.9.35`**,建议先下载审阅再运行):
|
|
356
356
|
|
|
357
357
|
```powershell
|
|
358
|
-
irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.
|
|
358
|
+
irm https://raw.githubusercontent.com/Han-1413141/dsh-cost-meter/v1.9.35/install.ps1 | iex
|
|
359
359
|
```
|
|
360
360
|
|
|
361
361
|
**或直接命令行**(机器上需已有 pnpm 与 git;同样固定到 tag):
|
|
362
362
|
|
|
363
363
|
```sh
|
|
364
|
-
dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.
|
|
364
|
+
dsh plugin --profile web add github:Han-1413141/dsh-cost-meter#v1.9.35
|
|
365
365
|
```
|
|
366
366
|
|
|
367
367
|
没有 git 时可用 GitHub tag 打包直链:
|
|
368
368
|
|
|
369
369
|
```sh
|
|
370
|
-
dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.
|
|
370
|
+
dsh plugin --profile web add https://github.com/Han-1413141/dsh-cost-meter/archive/refs/tags/v1.9.35.tar.gz
|
|
371
371
|
```
|
|
372
372
|
|
|
373
373
|
安装后**重启** `dsh web`(插件行、Typert 清单与客户端 bundle 均在启动时扫描):
|
|
@@ -432,7 +432,7 @@ DSH **0.2.1-rc.1 / 0.2.1-rc.2** 宿主自带 pnpm **11.28.5**,默认启用内
|
|
|
432
432
|
|
|
433
433
|
```yaml
|
|
434
434
|
minimumReleaseAgeExclude:
|
|
435
|
-
- dsh-cost-meter@1.9.
|
|
435
|
+
- dsh-cost-meter@1.9.35
|
|
436
436
|
```
|
|
437
437
|
|
|
438
438
|
3. 仅本次操作用 CLI 透传 pnpm 参数:`dsh plugin --profile desktop remove dsh-cost-meter --config.minimum-release-age=0`;若只是 lockfile 校验阶段失败,可改用 `--trust-lockfile`。这两个参数已在本机 pnpm 11.28.5 临时 Profile 上实测可用。
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# 计费历史存储 / Billing history storage
|
|
2
|
+
|
|
3
|
+
## 中文
|
|
4
|
+
|
|
5
|
+
从 1.9.34 起,费用设置的「用量 → 计费历史存储」显示 billion-context 辅助文件的数量、大小,以及最早、最新修改时间。左下角设置和插件详情页共用此入口。
|
|
6
|
+
|
|
7
|
+
默认保留最近 **7 天**的原始辅助文件;可以选择 **30 天**,或「全部保留」关闭自动清理。后台每小时检查一次,每次最多处理 10,000 个符合条件的文件。正在使用、尚未完成、内容无效或缺少完成依据的记录会保留。
|
|
8
|
+
|
|
9
|
+
- `streams`:将超过保留时间、已完成且已有计费归属的记录合并为 gzip 归档。每批最多 256 条,写入并同步归档后才移除未被并发修改的原始文件。
|
|
10
|
+
- `groups`:删除超过保留时间且已完成、没有待处理请求的预留记录与状态回执。
|
|
11
|
+
- `stream-archives`:保留压缩后的计费流元数据,供历史查询、重启和重放使用。
|
|
12
|
+
- 账本、`calls` 调用记录、费用金额、调用次数和历史明细保留。
|
|
13
|
+
|
|
14
|
+
手动操作先点击「预览清理」,检查待归档条数、回执文件数、涉及会话数和预计释放空间,再点击「确认清理」。取消预览不会执行清理。后台或其他客户端可能在预览后处理文件,因此最终数量以执行结果为准。
|
|
15
|
+
|
|
16
|
+
### CLI
|
|
17
|
+
|
|
18
|
+
插件安装提供 `dsh-cost-meter` 命令;也可在安装目录使用 `node lib/billing-storage-cli.js`。默认只预览,只有 `--apply` 执行清理。
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
dsh-cost-meter clean --streams --older-than 7d --dry-run
|
|
22
|
+
dsh-cost-meter clean --older-than 30d --dry-run
|
|
23
|
+
dsh-cost-meter clean --older-than 30d --apply
|
|
24
|
+
dsh-cost-meter clean --groups --older-than 7d --apply
|
|
25
|
+
dsh-cost-meter clean --ledger /path/to/ledger.json --older-than 7d --dry-run
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
不指定 `--streams` / `--groups` 时同时处理两者。`--older-than all` 表示全部保留,不清理任何记录。CLI 参数只作用于本次命令,不修改设置中的自动保留时间。默认账本为 `$DSH_HOME/storages/cost-meter/ledger.json`,未设置 `DSH_HOME` 时使用用户目录下的 `.dsh`;使用其他账本时显式传入 `--ledger`。
|
|
29
|
+
|
|
30
|
+
### 归档与备份
|
|
31
|
+
|
|
32
|
+
备份时将账本和同级的 `<ledger>.billion-context` 整个目录一起保存。压缩归档由 1.9.34 及后续版本读取;降级到更早版本会缺少已归档流的历史明细。需要降级时使用清理前的完整备份。尚未归档且计划降级时,可先选「全部保留」关闭自动清理。
|
|
33
|
+
|
|
34
|
+
空闲计费扫描异步检查会话目录的修改时间。目录未变化时跳过已完成的 stream 文件;正在写入的调用日志和未完成流仍会继续跟进。存储统计仅在打开此面板、刷新或执行清理时加载,不加入每 3 秒的界面状态刷新。
|
|
35
|
+
|
|
36
|
+
## English
|
|
37
|
+
|
|
38
|
+
Open **Cost settings → Usage → Billing history storage** from either Settings or the plugin details page. The panel shows file counts, bytes, and oldest/newest modification times for streams, request receipts and compressed archives.
|
|
39
|
+
|
|
40
|
+
Original auxiliary files are retained for **7 days** by default. Choose **30 days** or **Keep all** to disable cleanup. Hourly maintenance processes up to 10,000 eligible files per run. Active, unfinished, invalid or unconfirmed records are retained. Completed streams with resolved billing coverage are compressed in batches of up to 256 records; the archive is written and synced before unchanged originals are removed. Expired, completed request receipts are removed. Ledger totals, calls, historical details and replay metadata remain available.
|
|
41
|
+
|
|
42
|
+
Manual cleanup requires **Preview cleanup → Confirm cleanup**. Canceling performs no cleanup. The CLI examples above use the same operation: preview is the default, `--apply` executes it, and `--streams` / `--groups` select categories (both by default). `--older-than all` retains everything. CLI options do not change the automatic-retention setting. Use `--ledger` for a custom ledger; the default is `$DSH_HOME/storages/cost-meter/ledger.json`, with `~/.dsh` as the fallback home.
|
|
43
|
+
|
|
44
|
+
Back up both the ledger and its adjacent `.billion-context` directory. Versions older than 1.9.34 cannot read these new stream archives; restore a complete pre-cleanup backup when downgrading. Idle billing scans use asynchronous directory revisions and skip completed streams in unchanged directories. The storage panel is loaded on demand.
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# v1.9.34 更新说明
|
|
2
|
+
|
|
3
|
+
## 中文
|
|
4
|
+
|
|
5
|
+
修复 billion-context 长时间运行后,空闲计费扫描周期性阻塞主线程的问题。原先每轮扫描都会同步查询每个 stream 文件的状态;现在异步检查会话目录变化,只重新读取变化目录中的计费流,同时继续跟进未完成流和追加写入的调用日志。
|
|
6
|
+
|
|
7
|
+
新增「费用设置 → 用量 → 计费历史存储」,显示文件数量、大小、最早与最新修改时间,支持清理预览、确认和取消。原始辅助文件默认保留 7 天,也可改为 30 天或全部保留;自动维护每小时检查一次。
|
|
8
|
+
|
|
9
|
+
旧的已完成流先合并压缩,再移除未被修改的原文件;过期且完成的请求回执可删除。保留账本、调用记录、费用、次数和历史明细,并覆盖清理中断、并发修改、重新导入和重复执行。新增等价 CLI,默认预览,使用 `--apply` 才执行。[操作与备份说明](billing-storage.md)。
|
|
10
|
+
|
|
11
|
+
### 性能复现
|
|
12
|
+
|
|
13
|
+
本地合成数据包含 **10,744 个 stream 文件、38 个会话目录**。为模拟 issue 中文件系统过滤驱动造成的开销,对旧实现的每次同步文件状态查询增加 **0.25 毫秒**,触发其 30 秒强制重扫。
|
|
14
|
+
|
|
15
|
+
| 指标 | 1.9.33 | 1.9.34 |
|
|
16
|
+
| --- | ---: | ---: |
|
|
17
|
+
| 空闲扫描耗时 | 2,918.61 ms | 3.21 ms |
|
|
18
|
+
| 扫描期间最大事件循环间隔 | 2,920.07 ms | 15.32 ms |
|
|
19
|
+
| 已完成 stream 文件同步状态查询 | 10,744 次 | 0 次 |
|
|
20
|
+
|
|
21
|
+
这是受控复现数据,不是报告者电脑的实测。回归测试同时确认:原子替换旧文件后仍可发现新用量,未变化目录不逐文件查询,损坏元数据不会阻塞其他流。浏览器验证使用合成数据,覆盖预览、确认、取消、保留时间保存、错误反馈、中英文和 360/560/960 像素宽度。
|
|
22
|
+
|
|
23
|
+
本地完整验证通过 314 组检查;另通过真实 DSH Registry/Gateway 的接口调用、参数校验和保留时间回显检查。前端主包为 259,018 字节,存储面板独立按需加载。
|
|
24
|
+
|
|
25
|
+
感谢 **[@ZiYuan258](https://github.com/ZiYuan258)** 在 [#397](https://github.com/Han-1413141/dsh-cost-meter/issues/397) 提供问题报告、文件规模与主线程耗时证据、同步扫描定位,以及异步处理、清理入口和保留时间建议。
|
|
26
|
+
|
|
27
|
+
## English
|
|
28
|
+
|
|
29
|
+
Fix periodic main-thread stalls during idle billion-context billing scans. Replace per-file synchronous scanning with asynchronous session-directory revisions, while continuing to follow unfinished streams and append-only call journals.
|
|
30
|
+
|
|
31
|
+
Add on-demand billing storage statistics and a preview/confirm cleanup flow. Retain original auxiliary files for 7 days by default, with 30-day and keep-all options. Hourly maintenance durably archives completed stream metadata before removing unchanged originals and removes expired completed request receipts. Preserve ledger totals, call counts, historical details and replay data. The new CLI previews by default and requires `--apply` to execute; see the [storage guide](billing-storage.md).
|
|
32
|
+
|
|
33
|
+
A controlled fixture with 10,744 streams in 38 sessions and a simulated 0.25 ms synchronous-stat cost reduced idle scan time from 2,918.61 ms to 3.21 ms. This is a synthetic benchmark, not a measurement on the reporter's machine. Regression checks cover replay, interruption, concurrent writes, repeated cleanup and invalid metadata; browser checks use synthetic data.
|
|
34
|
+
|
|
35
|
+
Credit **@ZiYuan258** for the report, reproduction evidence, synchronous-scan diagnosis and cleanup/retention proposals in [#397](https://github.com/Han-1413141/dsh-cost-meter/issues/397).
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# v1.9.35:火山方舟计费与余额提示
|
|
2
|
+
|
|
3
|
+
火山方舟的 `deepseek-v4-1-flash-260910`、`deepseek-v4-1-flash-261017` 等日期快照现在可以自动匹配主模型价格。精确模型报价优先;精确匹配模式和不同日期快照之间的隔离保持原样。
|
|
4
|
+
|
|
5
|
+
`volcengine`、`ark` 及其 `llm-` 包装渠道在使用 DeepSeek 峰谷价格时,按火山的工作日日历选档:北京时间周一至周五 09:00–12:00、14:00–18:00 为峰时,其余为谷时,不额外豁免中国法定节假日。DeepSeek 官方渠道继续使用自己的节假日规则。当前标准版价格沿用已有 DeepSeek 价格表;本版没有新增经济版、缓存存储时长计费或火山账户余额查询。
|
|
6
|
+
|
|
7
|
+
侧栏峰谷进度优先跟随当前会话的渠道;无法取得会话投影时,通过宿主公开的 `agentDefaultModel.currentSelection()` 读取默认渠道。旧宿主没有该服务时保留原有显示。切换提醒与进度条共用同一日历。
|
|
8
|
+
|
|
9
|
+
余额卡现在直接显示“DeepSeek 余额”,表示 DeepSeek 官方开放平台账户余额。它不代表火山方舟或其他渠道的余额。已有的官方端点白名单保持原样,对账和余额进度中的今日消费仍只累计官方渠道。
|
|
10
|
+
|
|
11
|
+
余额悬停提示改成短行,保留总额、赠送与充值、异常提醒和点击刷新。搜索提示例如“今日 2 次搜索未完整计入统计”,不再展示网络实现等技术说明。具体缺失原因和运行环境仍保存在本地搜索覆盖诊断文件中。
|
|
12
|
+
|
|
13
|
+
旧价格版本继续保留记录时的匹配和日历规则。更新不会自动改写历史金额;需要补算之前未匹配的快照或修正节假日金额时,在价格设置中手动重算对应模型。
|
|
14
|
+
|
|
15
|
+
来源与致谢:[@littleyellowbicycle](https://github.com/littleyellowbicycle) 在 [#399](https://github.com/Han-1413141/dsh-cost-meter/issues/399) 提供快照模型名的复现、定位及补丁建议,在 [#400](https://github.com/Han-1413141/dsh-cost-meter/issues/400) 提供火山日历差异和金额对照,在 [#401](https://github.com/Han-1413141/dsh-cost-meter/issues/401) 提出余额来源标识和多渠道对账建议。
|
|
16
|
+
|
|
17
|
+
官方依据(核对日期:2026-10-24):[火山模型价格](https://console.volcengine.com/ark/region:cn-beijing/docs/model-pricing)、[DeepSeek-V4.1-Flash 模型与日期版本](https://console.volcengine.com/ark/region:cn-beijing/docs/deepseek-v4-1-flash)。
|
package/lib/billing-history.js
CHANGED
|
@@ -32,14 +32,16 @@ function indexOf(history) {
|
|
|
32
32
|
}
|
|
33
33
|
|
|
34
34
|
/**
|
|
35
|
-
* @param stored -
|
|
36
|
-
*
|
|
35
|
+
* @param stored - Preserve each saved version's channel calendar and snapshot-matching policy.
|
|
36
|
+
* Missing flags keep the behavior of the version that recorded the call; live settings use
|
|
37
|
+
* current policies. Upgrades do not silently reprice already recorded history.
|
|
37
38
|
*/
|
|
38
39
|
export function pricingSnapshot(config, stored = false) {
|
|
39
40
|
// Credentials, prompts and other settings never enter a price version.
|
|
40
41
|
const snapshot = structuredClone(Object.fromEntries(fields.filter(key => key !== 'prices' && config?.[key] !== undefined).map(key => [key, config[key]])))
|
|
41
42
|
const prices = config?.prices ?? {}
|
|
42
43
|
snapshot.prices = { ...(prices.currency ? { currency: prices.currency } : {}), ollamaPeak: !stored || prices.ollamaPeak === true, ollamaTaggedPeak: !stored || prices.ollamaTaggedPeak === true, snapshotTags: !stored || prices.snapshotTags === true,
|
|
44
|
+
shortSnapshotTags: !stored || prices.shortSnapshotTags === true, volcenginePeak: !stored || prices.volcenginePeak === true,
|
|
43
45
|
default: ratesOnly(prices.default), models: modelsOnly(prices.models),
|
|
44
46
|
...(prices.custom ? { custom: modelsOnly(prices.custom) } : {}),
|
|
45
47
|
providers: Object.fromEntries(Object.entries(prices.providers ?? {}).filter(([key]) => safe(key)).map(([key, table]) => [key, {
|
|
@@ -70,7 +72,7 @@ export function normalizePricingHistory(raw, config, previous, stored = false) {
|
|
|
70
72
|
if (!Object.values(changes).some(row => row.at === 0)) {
|
|
71
73
|
// Preserve surviving activation boundaries when the initial row is missing.
|
|
72
74
|
const saved = Object.values(versions), prices = { ...config?.prices }
|
|
73
|
-
for (const policy of ['ollamaPeak', 'ollamaTaggedPeak', 'snapshotTags']) {
|
|
75
|
+
for (const policy of ['ollamaPeak', 'ollamaTaggedPeak', 'snapshotTags', 'shortSnapshotTags', 'volcenginePeak']) {
|
|
74
76
|
if (saved.length) prices[policy] = saved.every(version => version.prices?.[policy] === true)
|
|
75
77
|
}
|
|
76
78
|
const seed = createPricingHistory({ ...config, prices }, stored)
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { parseArgs } from 'node:util'
|
|
3
|
+
import { resolve, join } from 'node:path'
|
|
4
|
+
import { homedir } from 'node:os'
|
|
5
|
+
import { manageBillionStorage } from './billion-context-storage.js'
|
|
6
|
+
|
|
7
|
+
try {
|
|
8
|
+
const { values, positionals } = parseArgs({ allowPositionals: true, options: {
|
|
9
|
+
ledger: { type: 'string' }, 'older-than': { type: 'string', default: '7d' },
|
|
10
|
+
streams: { type: 'boolean' }, groups: { type: 'boolean' }, 'dry-run': { type: 'boolean' },
|
|
11
|
+
apply: { type: 'boolean', default: false }, help: { type: 'boolean', short: 'h' },
|
|
12
|
+
} })
|
|
13
|
+
if (values.help || !positionals.length) {
|
|
14
|
+
console.log('dsh-cost-meter clean [--ledger <ledger.json>] [--streams] [--groups] [--older-than 7d|30d|all] [--dry-run | --apply]\n默认预览;--apply 才执行。合并压缩已完成的旧计费流,清理过期请求回执,保留账本、调用记录和历史费用明细。\nPreview by default. --apply compacts completed streams and removes expired receipts; ledger, calls and historical cost details are preserved.')
|
|
15
|
+
} else {
|
|
16
|
+
if (positionals.length !== 1 || positionals[0] !== 'clean') throw Error('Expected command: clean')
|
|
17
|
+
const retentionDays = { '7d': 7, '30d': 30, all: 0 }[values['older-than']]
|
|
18
|
+
if (retentionDays === undefined) throw Error('--older-than must be 7d, 30d or all')
|
|
19
|
+
if (values.apply && values['dry-run']) throw Error('--dry-run and --apply cannot be combined')
|
|
20
|
+
const both = !values.streams && !values.groups
|
|
21
|
+
const ledger = values.ledger ? resolve(values.ledger) : join(resolve(process.env.DSH_HOME || join(homedir(), '.dsh')), 'storages', 'cost-meter', 'ledger.json')
|
|
22
|
+
console.log(JSON.stringify(await manageBillionStorage(ledger, { retentionDays, apply: values.apply,
|
|
23
|
+
streams: both || values.streams === true, groups: both || values.groups === true }), null, 2))
|
|
24
|
+
}
|
|
25
|
+
} catch (error) { console.error(error.message); process.exitCode = 1 }
|
|
@@ -1,13 +1,15 @@
|
|
|
1
1
|
/** DSH side of the local billion-context metering bridge. No inference or HTTP polling. */
|
|
2
2
|
import { AsyncLocalStorage } from 'node:async_hooks'
|
|
3
3
|
import { randomUUID } from 'node:crypto'
|
|
4
|
-
import {
|
|
4
|
+
import { lstat } from 'node:fs/promises'
|
|
5
5
|
import { dirname, join } from 'node:path'
|
|
6
6
|
import { billionVersionSupported } from './billion-context-compatibility.js'
|
|
7
7
|
import { installBillionAuto } from './billion-context-auto.js'
|
|
8
|
-
import {
|
|
9
|
-
import {
|
|
10
|
-
|
|
8
|
+
import { journalInfoAsync, archiveUsageJournals, notifyJournalChange } from './journal-storage.js'
|
|
9
|
+
import { createBillionInventory } from './billion-context-inventory.js'
|
|
10
|
+
import { maintainBillionStorage, readBillionStreamArchive } from './billion-context-storage.js'
|
|
11
|
+
import { billionRoot, billionSessionKey, billionStreamPath, writeBillionJson, readBillionJson, listBillionFiles, iterateBillionCalls, readBillionStreamAsync,
|
|
12
|
+
readBillionStreamFileAsync, billionProvider, usageOnly, usageBuckets, usageSignature, validUsage, validSessionId, billionCoverage, billionCoverageIndex, billionHostMeter } from './billion-context-history.js'
|
|
11
13
|
|
|
12
14
|
const bridgeKey = Symbol.for('dsh-cost-meter.billion-context.v1'), bridges = new WeakMap()
|
|
13
15
|
const plainId = value => typeof value === 'string' ? value.slice(0, 200) : ''
|
|
@@ -26,8 +28,7 @@ function stepIdentity(ctx, id) {
|
|
|
26
28
|
export function installBillionContextBilling(ctx, ledger) {
|
|
27
29
|
if (!ledger.path) return null
|
|
28
30
|
const root = billionRoot(ledger.path), scope = new AsyncLocalStorage(), streams = new Map(), groups = new Map(), files = new Map()
|
|
29
|
-
const dirtyStreams = new Map()
|
|
30
|
-
let inventoryAt = 0, inventoryRevision = '', callFiles = []
|
|
31
|
+
const dirtyStreams = new Map(), retryStreams = new Map(), inventory = createBillionInventory(ledger.path)
|
|
31
32
|
let stopped = false, pending = null, rerun = false, peerAt = 0, peerPending, releaseAuto
|
|
32
33
|
const status = { protocol: 1, requests: 0, unreported: 0, partial: 0, uncovered: 0, connected: false, failed: false,
|
|
33
34
|
detected: false, version: '', compatibility: 'unknown', noticeId: '', unresolved: 0 }
|
|
@@ -91,9 +92,7 @@ export function installBillionContextBilling(ctx, ledger) {
|
|
|
91
92
|
})
|
|
92
93
|
dirtyStreams.set(stream.streamId, { path: billionStreamPath(root, stream.sessionId, stream.streamId), key: billionSessionKey(stream.sessionId), sessionId: stream.sessionId })
|
|
93
94
|
if (stream.finishedAt) {
|
|
94
|
-
const previous = journalChangeRevision(root)
|
|
95
95
|
notifyJournalChange(root)
|
|
96
|
-
if (previous === inventoryRevision) inventoryRevision = journalChangeRevision(root)
|
|
97
96
|
}
|
|
98
97
|
}
|
|
99
98
|
// The proxy reads this fixed private file. No filesystem path is sent upstream.
|
|
@@ -107,27 +106,33 @@ export function installBillionContextBilling(ctx, ledger) {
|
|
|
107
106
|
rerun = false
|
|
108
107
|
ledger.refresh()
|
|
109
108
|
let work = 0
|
|
110
|
-
const legacyCoverage = new Map()
|
|
109
|
+
const legacyCoverage = new Map(), archives = new Map()
|
|
111
110
|
const dirty = [...dirtyStreams.values()]; dirtyStreams.clear()
|
|
112
|
-
const
|
|
113
|
-
|
|
114
|
-
const streamFiles =
|
|
115
|
-
if (
|
|
116
|
-
|
|
117
|
-
const
|
|
118
|
-
for (const path of files.keys()) if (!retainedPaths.has(path)) files.delete(path)
|
|
111
|
+
const found = await inventory.poll(), callFiles = found.calls
|
|
112
|
+
if (stopped) return
|
|
113
|
+
const streamFiles = [...new Map([...dirty, ...retryStreams.values(), ...found.streams].map(file => [file.path, file])).values()]
|
|
114
|
+
if (found.retained) {
|
|
115
|
+
for (const path of files.keys()) if (!found.retained.has(path)) files.delete(path)
|
|
116
|
+
for (const path of retryStreams.keys()) if (!found.retained.has(path)) retryStreams.delete(path)
|
|
119
117
|
}
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
118
|
+
for (const file of found.archives) {
|
|
119
|
+
if (files.has(file.path)) continue
|
|
120
|
+
for (const stream of await readBillionStreamArchive(file)) { applyCoverage(stream, stream.coverage); importHost(stream) }
|
|
121
|
+
files.set(file.path, { archived: true })
|
|
123
122
|
}
|
|
124
123
|
// A completed stream can legitimately contain zero proxy call rows.
|
|
125
124
|
// Discover its once-per-stream fallback independently of the call journal.
|
|
126
125
|
for (const file of streamFiles) {
|
|
127
|
-
|
|
126
|
+
if (stopped) break
|
|
127
|
+
let info
|
|
128
|
+
try { info = await lstat(file.path) } catch (error) { if (error.code === 'ENOENT') { retryStreams.delete(file.path); continue }; throw error }
|
|
129
|
+
if (!info.isFile() || info.isSymbolicLink()) continue
|
|
130
|
+
const cached = files.get(file.path)
|
|
128
131
|
if (cached?.size === info.size && cached?.mtime === info.mtimeMs) continue
|
|
129
|
-
const stream =
|
|
130
|
-
if (!stream
|
|
132
|
+
const stream = await readBillionStreamFileAsync(file)
|
|
133
|
+
if (!stream) { retryStreams.delete(file.path); continue }
|
|
134
|
+
if (!stream?.finished) { retryStreams.set(file.path, file); continue }
|
|
135
|
+
retryStreams.delete(file.path)
|
|
131
136
|
if (!stream.coverage) {
|
|
132
137
|
if (!legacyCoverage.has(stream.sessionId)) legacyCoverage.set(stream.sessionId, await billionCoverageIndex(ledger.path, stream.sessionId))
|
|
133
138
|
applyCoverage(stream, legacyCoverage.get(stream.sessionId).get(stream.streamId) ?? await billionCoverage(ledger.path, stream))
|
|
@@ -137,15 +142,17 @@ export function installBillionContextBilling(ctx, ledger) {
|
|
|
137
142
|
files.set(file.path, { size: info.size, mtime: info.mtimeMs })
|
|
138
143
|
}
|
|
139
144
|
for (const file of callFiles) {
|
|
145
|
+
if (stopped) break
|
|
140
146
|
let info
|
|
141
|
-
try { info =
|
|
147
|
+
try { info = await journalInfoAsync(file.path) } catch (error) { if (error.code === 'ENOENT') { rerun = true; continue }; throw error }
|
|
142
148
|
const cached = files.get(file.path)
|
|
143
149
|
if (cached?.size === info.size && cached?.mtime === info.mtimeMs) continue
|
|
144
150
|
let offset = cached?.offset <= info.size ? cached.offset : 0, blocked = false
|
|
145
151
|
const prepared = new Map()
|
|
146
152
|
for await (const item of iterateBillionCalls(file, offset)) {
|
|
153
|
+
if (stopped) break
|
|
147
154
|
const row = item.row
|
|
148
|
-
let stream = prepared.get(row.streamId) ??
|
|
155
|
+
let stream = prepared.get(row.streamId) ?? await readBillionStreamAsync(ledger.path, row, archives)
|
|
149
156
|
if (!stream || !stream.finished) { blocked = true; break }
|
|
150
157
|
if (!prepared.has(row.streamId)) {
|
|
151
158
|
const originalCoverage = stream.coverage
|
|
@@ -262,7 +269,10 @@ export function installBillionContextBilling(ctx, ledger) {
|
|
|
262
269
|
}
|
|
263
270
|
globalThis[bridgeKey] = api; bridges.set(ledger, api)
|
|
264
271
|
releaseAuto = installBillionAuto()
|
|
265
|
-
const timer = setInterval(() => {
|
|
272
|
+
const timer = setInterval(() => {
|
|
273
|
+
void ingest(); void refreshPeer(); void archiveUsageJournals(ledger.path)
|
|
274
|
+
void maintainBillionStorage(ledger.path, ledger.config?.billingStorageRetentionDays ?? 7)
|
|
275
|
+
}, 3000); timer.unref?.()
|
|
266
276
|
ctx.effect(() => () => { clearInterval(timer); api.dispose(); bridges.delete(ledger) }, 'cost-meter: billion-context billing')
|
|
267
277
|
ctx.on('session/event', (session, event) => {
|
|
268
278
|
if (!Number.isSafeInteger(event.seq) || event.seq < 0) return
|
|
@@ -3,6 +3,8 @@ import { createHash } from 'node:crypto'
|
|
|
3
3
|
import { mkdirSync, lstatSync, readFileSync, writeFileSync, renameSync, readdirSync } from 'node:fs'
|
|
4
4
|
import { join } from 'node:path'
|
|
5
5
|
import { journalChunks } from './journal-storage.js'
|
|
6
|
+
import { readSmallBillionJson, safeEntries } from './billion-context-inventory.js'
|
|
7
|
+
import { archivedBillionStreams } from './billion-context-storage.js'
|
|
6
8
|
|
|
7
9
|
export const BILLION_USAGE_EVENT = 'cost-meter/billion-context-usage'
|
|
8
10
|
export const BILLION_REPLACEMENT_EVENT = 'cost-meter/billion-context-replacement'
|
|
@@ -57,6 +59,13 @@ export function readBillionStreamFile(file) {
|
|
|
57
59
|
&& billionSessionKey(stream.sessionId) === file.key && file.path.endsWith(stream.streamId + '.json') ? stream : null
|
|
58
60
|
} catch { return null }
|
|
59
61
|
}
|
|
62
|
+
export async function readBillionStreamFileAsync(file) {
|
|
63
|
+
try {
|
|
64
|
+
const stream = await readSmallBillionJson(file.path)
|
|
65
|
+
return stream.protocol === 1 && uuid(stream.streamId) && validSessionId(stream.sessionId)
|
|
66
|
+
&& billionSessionKey(stream.sessionId) === file.key && file.path.endsWith(stream.streamId + '.json') ? stream : null
|
|
67
|
+
} catch { return null }
|
|
68
|
+
}
|
|
60
69
|
export function listBillionFiles(ledgerPath, sessionId) {
|
|
61
70
|
if (!ledgerPath) return []
|
|
62
71
|
const root = join(billionRoot(ledgerPath), 'calls')
|
|
@@ -76,7 +85,9 @@ export function listBillionFiles(ledgerPath, sessionId) {
|
|
|
76
85
|
export function billionSourcePaths(ledgerPath, sessionId) {
|
|
77
86
|
if (!ledgerPath) return []
|
|
78
87
|
const streamRoot = join(billionRoot(ledgerPath), 'streams', billionSessionKey(sessionId))
|
|
88
|
+
const archiveRoot = join(billionRoot(ledgerPath), 'stream-archives', billionSessionKey(sessionId))
|
|
79
89
|
return [...listBillionFiles(ledgerPath, sessionId).map(row => row.path),
|
|
90
|
+
...directories(archiveRoot).filter(row => row.isFile() && row.name.endsWith('.jsonl.gz') && uuid(row.name.slice(0, -9))).map(row => join(archiveRoot, row.name)),
|
|
80
91
|
...directories(streamRoot).filter(row => row.isFile() && row.name.endsWith('.json') && uuid(row.name.slice(0, -5))).map(row => join(streamRoot, row.name))]
|
|
81
92
|
}
|
|
82
93
|
/** Bounded UTF-8 rows; offsets only advance through complete newline-terminated writes. */
|
|
@@ -109,6 +120,13 @@ export function readBillionStream(ledgerPath, row) {
|
|
|
109
120
|
return stream.protocol === 1 && stream.streamId === row.streamId && stream.sessionId === row.sessionId ? stream : null
|
|
110
121
|
} catch { return null }
|
|
111
122
|
}
|
|
123
|
+
export async function readBillionStreamAsync(ledgerPath, row, archives = new Map()) {
|
|
124
|
+
const file = { path: billionStreamPath(billionRoot(ledgerPath), row.sessionId, row.streamId), key: billionSessionKey(row.sessionId) }
|
|
125
|
+
const stream = await readBillionStreamFileAsync(file)
|
|
126
|
+
if (stream) return stream
|
|
127
|
+
if (!archives.has(row.sessionId)) archives.set(row.sessionId, await archivedBillionStreams(ledgerPath, row.sessionId))
|
|
128
|
+
return archives.get(row.sessionId).get(row.streamId) ?? null
|
|
129
|
+
}
|
|
112
130
|
const coverageCounts = () => ({ known: 0, unknown: 0, matched: false })
|
|
113
131
|
function observeCoverage(counts, stream, row) {
|
|
114
132
|
if (!['full', 'partial'].includes(stream.policies?.[row.groupId]) || row.kind !== 'model') return
|
|
@@ -135,11 +153,11 @@ export async function billionCoverage(ledgerPath, stream, offsets) {
|
|
|
135
153
|
}
|
|
136
154
|
/** Legacy streams share one session scan instead of rescanning for every call. */
|
|
137
155
|
export async function billionCoverageIndex(ledgerPath, sessionId) {
|
|
138
|
-
const entries = new Map()
|
|
156
|
+
const entries = new Map(), archives = new Map()
|
|
139
157
|
for (const file of listBillionFiles(ledgerPath, sessionId)) for await (const { row } of iterateBillionCalls(file)) {
|
|
140
158
|
if (!entries.has(row.streamId)) {
|
|
141
159
|
if (entries.size >= 64000) throw Error('Proxy coverage stream limit exceeded')
|
|
142
|
-
entries.set(row.streamId, { stream:
|
|
160
|
+
entries.set(row.streamId, { stream: await readBillionStreamAsync(ledgerPath, row, archives), counts: coverageCounts() })
|
|
143
161
|
}
|
|
144
162
|
const { stream, counts } = entries.get(row.streamId)
|
|
145
163
|
if (stream?.finished && !stream.coverage) observeCoverage(counts, stream, row)
|
|
@@ -154,14 +172,20 @@ export function billionHostRecord(stream) {
|
|
|
154
172
|
metering: { source: 'billion-context', requestId: stream.streamId, attempt: 1, complete: false, status: 'complete', upstream: 'host-usage' } } }
|
|
155
173
|
}
|
|
156
174
|
export async function billionRecordsFor(ledgerPath, sessionId) {
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
175
|
+
if (!ledgerPath) return []
|
|
176
|
+
const records = [], streams = new Map(), counts = new Map(), seen = new Set(), archives = new Map()
|
|
177
|
+
const key = billionSessionKey(sessionId), dir = join(billionRoot(ledgerPath), 'streams', key)
|
|
178
|
+
for (const entry of await safeEntries(dir)) {
|
|
179
|
+
if (!entry.isFile() || !entry.name.endsWith('.json') || !uuid(entry.name.slice(0, -5))) continue
|
|
180
|
+
const stream = await readBillionStreamFileAsync({ path: join(dir, entry.name), key })
|
|
160
181
|
if (stream?.finished) streams.set(stream.streamId, stream)
|
|
161
182
|
}
|
|
183
|
+
// Originals are removed only after archive publication. Read archives after
|
|
184
|
+
// originals so a concurrent compaction cannot create a gap in history.
|
|
185
|
+
for (const [id, stream] of await archivedBillionStreams(ledgerPath, sessionId)) if (!streams.has(id)) streams.set(id, stream)
|
|
162
186
|
for (const file of listBillionFiles(ledgerPath, sessionId)) for await (const { row } of iterateBillionCalls(file)) {
|
|
163
187
|
if (seen.has(row.callId)) continue
|
|
164
|
-
const stream =
|
|
188
|
+
const stream = streams.get(row.streamId) ?? await readBillionStreamAsync(ledgerPath, row, archives)
|
|
165
189
|
if (!stream?.finished || !['full', 'partial'].includes(stream.policies?.[row.groupId])) continue
|
|
166
190
|
seen.add(row.callId); streams.set(stream.streamId, stream)
|
|
167
191
|
if (seen.size > 200000) throw Error('Proxy billing detail limit exceeded')
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
/** Incremental asynchronous inventory. Atomic metadata replacement changes its parent directory. */
|
|
2
|
+
import { lstat, readdir, readFile } from 'node:fs/promises'
|
|
3
|
+
import { join } from 'node:path'
|
|
4
|
+
|
|
5
|
+
const sessionKey = /^[a-f\d]{64}$/
|
|
6
|
+
const id = '[a-f\\d]{8}(?:-[a-f\\d]{4}){3}-[a-f\\d]{12}'
|
|
7
|
+
const streamName = new RegExp('^' + id + '\\.json$', 'i')
|
|
8
|
+
const callName = new RegExp('^(' + id + ')(?:\\.sealed)?\\.jsonl(?:\\.gz)?$', 'i')
|
|
9
|
+
const archiveName = new RegExp('^' + id + '\\.jsonl\\.gz$', 'i')
|
|
10
|
+
export async function safeEntries(path) {
|
|
11
|
+
try {
|
|
12
|
+
const info = await lstat(path)
|
|
13
|
+
if (!info.isDirectory() || info.isSymbolicLink()) return []
|
|
14
|
+
return await readdir(path, { withFileTypes: true })
|
|
15
|
+
} catch (error) { if (error.code === 'ENOENT') return []; throw error }
|
|
16
|
+
}
|
|
17
|
+
export async function readSmallBillionJson(path) {
|
|
18
|
+
const info = await lstat(path)
|
|
19
|
+
if (!info.isFile() || info.isSymbolicLink() || info.size > 8192) throw Error('Invalid proxy billing metadata')
|
|
20
|
+
return JSON.parse(await readFile(path, 'utf8'))
|
|
21
|
+
}
|
|
22
|
+
export function createBillionInventory(ledgerPath) {
|
|
23
|
+
const root = ledgerPath + '.billion-context', cache = new Map()
|
|
24
|
+
return {
|
|
25
|
+
async poll() {
|
|
26
|
+
const streams = [], calls = [], archives = [], seen = new Set()
|
|
27
|
+
let changed = false
|
|
28
|
+
// Inspect directories, not every completed stream, on an idle tick. Calls
|
|
29
|
+
// remain candidates because append-only writes do not change directory mtime.
|
|
30
|
+
if (!(await safeEntries(root)).length) { cache.clear(); return { streams, calls, archives, retained: new Set() } }
|
|
31
|
+
for (const kind of ['streams', 'calls', 'stream-archives']) {
|
|
32
|
+
const parent = join(root, kind)
|
|
33
|
+
for (const entry of await safeEntries(parent)) {
|
|
34
|
+
if (!entry.isDirectory() || !sessionKey.test(entry.name)) continue
|
|
35
|
+
const directory = join(parent, entry.name)
|
|
36
|
+
let info
|
|
37
|
+
try { info = await lstat(directory, { bigint: true }) } catch (error) { if (error.code === 'ENOENT') continue; throw error }
|
|
38
|
+
if (!info.isDirectory() || info.isSymbolicLink()) continue
|
|
39
|
+
seen.add(directory)
|
|
40
|
+
const revision = `${info.ino}:${info.mtimeNs}:${info.ctimeNs}`, previous = cache.get(directory)
|
|
41
|
+
let files = previous?.files
|
|
42
|
+
if (previous?.revision !== revision) {
|
|
43
|
+
changed = true
|
|
44
|
+
files = (await safeEntries(directory)).flatMap(file => {
|
|
45
|
+
if (!file.isFile()) return []
|
|
46
|
+
const match = kind === 'calls' ? callName.exec(file.name) : (kind === 'streams' ? streamName : archiveName).exec(file.name)
|
|
47
|
+
return match ? [{ path: join(directory, file.name), key: entry.name, ...(kind === 'calls' ? { source: match[1] } : {}) }] : []
|
|
48
|
+
})
|
|
49
|
+
cache.set(directory, { revision, files })
|
|
50
|
+
}
|
|
51
|
+
if (kind === 'calls') for (const file of files) calls.push(file)
|
|
52
|
+
else if (previous?.revision !== revision) for (const file of files) (kind === 'streams' ? streams : archives).push(file)
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
for (const directory of cache.keys()) if (!seen.has(directory)) { cache.delete(directory); changed = true }
|
|
56
|
+
const retained = changed ? new Set([...cache.values()].flatMap(item => item.files.map(file => file.path))) : null
|
|
57
|
+
return { streams, calls, archives, retained }
|
|
58
|
+
},
|
|
59
|
+
}
|
|
60
|
+
}
|