@ychris12138/dsh-usage-stats 0.3.1 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -5
- package/docs/release-checklist.md +15 -8
- package/docs/release-notes-v0.3.2.md +36 -0
- package/lib/index.js +190 -40
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# dsh-usage-stats
|
|
2
2
|
|
|
3
|
-
<!-- stable-version: 0.3.
|
|
3
|
+
<!-- stable-version: 0.3.2 -->
|
|
4
4
|
|
|
5
5
|
[](https://github.com/Ychris12138/dsh-usage-stats/releases/latest)
|
|
6
6
|
[](https://github.com/Ychris12138/dsh-usage-stats/actions/workflows/ci.yml)
|
|
@@ -39,7 +39,7 @@ Provider balances, subscription quotas, and token-usage analytics for the DeepSe
|
|
|
39
39
|
稳定版优先安装 npm 上的精确版本;这也是 DSH Desktop Market 使用的同一个包:
|
|
40
40
|
|
|
41
41
|
```bash
|
|
42
|
-
dsh plugin --profile web add "@ychris12138/dsh-usage-stats@0.3.
|
|
42
|
+
dsh plugin --profile web add "@ychris12138/dsh-usage-stats@0.3.2"
|
|
43
43
|
```
|
|
44
44
|
|
|
45
45
|
只有测试尚未发布的 source/RC 时才使用 `dsh plugin --profile web add "github:Ychris12138/dsh-usage-stats"`。GitHub `main` 可能领先 npm stable,不应把 source 安装当作市场安装验收。
|
|
@@ -53,7 +53,7 @@ dsh plugin --profile web add "@ychris12138/dsh-usage-stats@0.3.1"
|
|
|
53
53
|
- `catalog/catalog-source.json` — 来源 manifest(`catalog-source.schema.json` v1.0.0)
|
|
54
54
|
- `catalog/v1/plugins.json` — 标准 provider page(`catalog-provider-page.schema.json` v1.0.0)
|
|
55
55
|
|
|
56
|
-
**使用前提(重要)**:市场托管安装只接受 npm registry 的精确稳定版本,git 条目仅可浏览。`dsh-usage-stats` 这个 npm 名已被其他项目占用,因此目录条目身份使用 `@ychris12138/dsh-usage-stats`。当前 stable/catalog 版本是 `0.3.
|
|
56
|
+
**使用前提(重要)**:市场托管安装只接受 npm registry 的精确稳定版本,git 条目仅可浏览。`dsh-usage-stats` 这个 npm 名已被其他项目占用,因此目录条目身份使用 `@ychris12138/dsh-usage-stats`。当前 stable/catalog 版本是 `0.3.2`;每个新版本都按以下顺序发布:
|
|
57
57
|
|
|
58
58
|
1. 运行 `npm run release:sync -- <version>` 同步 `package.json` / `package-lock.json` / `catalog/v1/plugins.json`,再由 `npm run check:release` 阻止身份或版本漂移。
|
|
59
59
|
2. 发布 scoped 公共包:`npm publish --access public`。
|
|
@@ -349,7 +349,7 @@ Constraints:
|
|
|
349
349
|
|
|
350
350
|
Procedure:
|
|
351
351
|
1. Confirm node, npx, and dsh are available.
|
|
352
|
-
2. Prefer the exact npm stable used by Desktop Market: `dsh plugin --profile web add "@ychris12138/dsh-usage-stats@0.3.
|
|
352
|
+
2. Prefer the exact npm stable used by Desktop Market: `dsh plugin --profile web add "@ychris12138/dsh-usage-stats@0.3.2"` (or update the existing scoped package).
|
|
353
353
|
3. Use `github:Ychris12138/dsh-usage-stats` only when I explicitly ask to test unreleased source/RC code.
|
|
354
354
|
4. If dsh plugin is unavailable, use the compatible source installer only with my approval: `npx --yes github:Ychris12138/dsh-usage-stats`.
|
|
355
355
|
5. Do not combine bundle installation with an existing manual dsh-usage-stats Cordis entry.
|
|
@@ -444,7 +444,7 @@ node scripts/check-balance.mjs
|
|
|
444
444
|
|
|
445
445
|
## 兼容性与致谢 / Compatibility & credits
|
|
446
446
|
|
|
447
|
-
当前 npm stable 为 `0.3.
|
|
447
|
+
当前 npm stable 为 `0.3.2`;`v0.3.2` 的完整发布门禁见 [`docs/release-checklist.md`](docs/release-checklist.md),变更摘要见 [`docs/release-notes-v0.3.2.md`](docs/release-notes-v0.3.2.md)。插件依赖 Harness 客户端模块加载器、Cordis 服务与 session persistence;Harness 预发布接口变化时可能需要同步适配。
|
|
448
448
|
|
|
449
449
|
`display.currentSessionPill` 作为 v0.3.0 legacy boolean 配置键继续被接受,避免旧配置导致启动失败;当前客户端不再注册任何 composer UI,因此该键不再产生可见效果。`session-context` 服务端 API 暂时保留原有响应语义,供 v0.3.0 API compatibility 与后续集成使用。
|
|
450
450
|
|
|
@@ -46,6 +46,13 @@ npm pack --json
|
|
|
46
46
|
|
|
47
47
|
## 5. Real DSH release-candidate regression
|
|
48
48
|
|
|
49
|
+
### v0.3.2-specific gates
|
|
50
|
+
|
|
51
|
+
- [ ] Current DSH sessions use the public `seq` / `snapshotEvents()` API; no current-session path accesses the removed `session.events` property.
|
|
52
|
+
- [ ] Legacy sessions exposing `seq` plus an `events` array, but no `snapshotEvents()`, remain supported.
|
|
53
|
+
- [ ] UI steady state performs zero persisted `listSnapshots`, zero persisted `readFrom`, zero unchanged aggregate rebuilds, and zero unchanged cache writes.
|
|
54
|
+
- [ ] Background refresh skips `readFrom` and cache writes for unchanged revisions, uses incremental reads for changed revisions, and full-refolds truncation or rewrite cases.
|
|
55
|
+
|
|
49
56
|
- [ ] Test the latest supported `@deepseek-ai/dsh` Desktop/Web release candidate with an isolated profile.
|
|
50
57
|
- [ ] Confirm DSH starts without `Failed to load plugins` or loader identity errors.
|
|
51
58
|
- [ ] Confirm client bundle load and that `sidebar.footer.action` is the only extension point registered by dsh-usage-stats; verify panel open/close, provider switching, cost/budget states, and manual Retry.
|
|
@@ -61,8 +68,8 @@ npm pack --json
|
|
|
61
68
|
Do not run this section until every release-candidate gate above passes and the maintainer approves preparing the release commit.
|
|
62
69
|
|
|
63
70
|
- [ ] Create a release branch from the reviewed `main` at `RC_BASE_SHA`.
|
|
64
|
-
- [ ] Run `npm run release:sync -- 0.3.
|
|
65
|
-
- [ ] Confirm the v0.3.
|
|
71
|
+
- [ ] Run `npm run release:sync -- 0.3.2` once to update `package.json`, `package-lock.json`, the Community Market catalog, and documented stable-version references together.
|
|
72
|
+
- [ ] Confirm the v0.3.2 release notes use final stable wording, preserve the historical v0.3.0/v0.3.1 notes, and do not claim #84 fixed or include #88 behavior.
|
|
66
73
|
- [ ] Run the release gates again against the synchronized version:
|
|
67
74
|
|
|
68
75
|
```bash
|
|
@@ -71,20 +78,20 @@ npm test
|
|
|
71
78
|
npm pack --json
|
|
72
79
|
```
|
|
73
80
|
|
|
74
|
-
- [ ] Inspect the final pack manifest, then commit all version/release metadata changes with `chore: prepare v0.3.
|
|
81
|
+
- [ ] Inspect the final pack manifest, then commit all version/release metadata changes with `chore: prepare v0.3.2 release`.
|
|
75
82
|
- [ ] Record that commit as `RELEASE_SHA`; this replaces `RC_BASE_SHA` as the only publish/tag identity.
|
|
76
83
|
- [ ] Confirm the working tree is clean and `HEAD` equals `RELEASE_SHA`.
|
|
77
84
|
- [ ] Confirm the committed package version, not merely the working-tree version:
|
|
78
85
|
|
|
79
86
|
```bash
|
|
80
|
-
test "$(git show "$RELEASE_SHA:package.json" | jq -r .version)" = "0.3.
|
|
87
|
+
test "$(git show "$RELEASE_SHA:package.json" | jq -r .version)" = "0.3.2"
|
|
81
88
|
```
|
|
82
89
|
|
|
83
90
|
The release invariant is:
|
|
84
91
|
|
|
85
92
|
```text
|
|
86
93
|
npm published source commit
|
|
87
|
-
== v0.3.
|
|
94
|
+
== v0.3.2 tag commit
|
|
88
95
|
== GitHub Release commit
|
|
89
96
|
== package/catalog version commit
|
|
90
97
|
== RELEASE_SHA
|
|
@@ -98,14 +105,14 @@ Do not run this section until the immutable release commit exists and the mainta
|
|
|
98
105
|
- [ ] From a clean checkout/worktree at exactly `RELEASE_SHA`, run `npm publish --access public --registry=https://registry.npmjs.org/`.
|
|
99
106
|
- [ ] `npm view "@ychris12138/dsh-usage-stats" version --registry=https://registry.npmjs.org/` equals the target version.
|
|
100
107
|
- [ ] Merge or push the release commit to `main` according to the chosen branch workflow; verify `main` contains the exact `RELEASE_SHA` without recreating the release changes.
|
|
101
|
-
- [ ] Only after npm and `main` verification: create the signed/annotated `v0.3.
|
|
108
|
+
- [ ] Only after npm and `main` verification: create the signed/annotated `v0.3.2` tag pointing explicitly to `RELEASE_SHA`.
|
|
102
109
|
- [ ] Verify the tag resolves to the published source commit:
|
|
103
110
|
|
|
104
111
|
```bash
|
|
105
|
-
test "$(git rev-parse v0.3.
|
|
112
|
+
test "$(git rev-parse v0.3.2^{commit})" = "$RELEASE_SHA"
|
|
106
113
|
```
|
|
107
114
|
|
|
108
|
-
- [ ] Create the GitHub Release from `v0.3.
|
|
115
|
+
- [ ] Create the GitHub Release from `v0.3.2`; verify it resolves to `RELEASE_SHA`.
|
|
109
116
|
- [ ] Verify the public Pages `catalog-source.json` and `/v1/plugins` response content type, package name, and exact version.
|
|
110
117
|
- [ ] Install the exact npm version through DSH Desktop Community Market and restart the host.
|
|
111
118
|
- [ ] Close the npm/market release issue only after the Desktop Market installation succeeds.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# v0.3.2 release notes
|
|
2
|
+
|
|
3
|
+
`v0.3.2` is a focused maintenance release for current DSH session compatibility and lower steady-state usage aggregation overhead.
|
|
4
|
+
|
|
5
|
+
## Highlights
|
|
6
|
+
|
|
7
|
+
### Current DSH Session API compatibility
|
|
8
|
+
|
|
9
|
+
- Fixes #95 for DSH builds where the public `Session.events` array has been replaced by `seq` and `snapshotEvents()`.
|
|
10
|
+
- Current DSH sessions now use the formal `seq` / `snapshotEvents(fromSeq)` API and continue folding only the live event tail.
|
|
11
|
+
- Older DSH session objects without `snapshotEvents()` retain the legacy `events` fallback.
|
|
12
|
+
- Compatibility is capability-based rather than tied to a DSH version string.
|
|
13
|
+
- Preserves live-log shrink recovery and conservative live/persisted refolding semantics.
|
|
14
|
+
|
|
15
|
+
### Lower usage polling overhead
|
|
16
|
+
|
|
17
|
+
- Fixes #94 by separating high-frequency UI usage reads from persisted archive refreshes.
|
|
18
|
+
- Ordinary `/usage` requests fold live-session tails and reuse cached persisted aggregates instead of enumerating every archived session.
|
|
19
|
+
- Persisted archive scanning remains on the existing background refresh path and explicit export paths.
|
|
20
|
+
- Unchanged collections no longer rebuild global aggregates or rewrite `usage-stats-cache.json`.
|
|
21
|
+
- Persisted sessions with unchanged revision tokens perform no event reads.
|
|
22
|
+
- A changed persisted revision still uses the existing incremental cursor logic, with full refolding retained for truncation or rewrite recovery.
|
|
23
|
+
- Scan-mode single-flight guarantees that a required full persisted refresh cannot be satisfied accidentally by an in-flight live-only request.
|
|
24
|
+
|
|
25
|
+
## Compatibility and non-features
|
|
26
|
+
|
|
27
|
+
- No frontend polling interval change and no additional timer or polling loop.
|
|
28
|
+
- No pricing-rule or billing-semantics changes.
|
|
29
|
+
- Pricing/provider fingerprint changes still invalidate and persist derived billing state safely.
|
|
30
|
+
- No private JSONL/zstd reader or plugin-owned persistence index.
|
|
31
|
+
- No attempt to optimize live-to-persisted transitions by trusting the previous live fold.
|
|
32
|
+
- No DSH `sessionProjectionCache` migration or modification.
|
|
33
|
+
- No cold-session eviction or bounded-cache redesign.
|
|
34
|
+
- No provider or composer UI changes.
|
|
35
|
+
- No #84 quota-precision change and no #93 feature work is included in this release.
|
|
36
|
+
- JSONL suffix reads remain an upstream DSH concern; a changed large archive may still require the host persistence backend to decode more data than the logical suffix.
|
package/lib/index.js
CHANGED
|
@@ -176,6 +176,37 @@ let loadedCache = null;
|
|
|
176
176
|
let loadPromise = null;
|
|
177
177
|
let inflight = null;
|
|
178
178
|
|
|
179
|
+
// Runtime-only cache bookkeeping. Nothing in this WeakMap is serialized to
|
|
180
|
+
// disk: the on-disk schema remains CACHE_VERSION=5. Keeping the aggregate and
|
|
181
|
+
// dirty flags out of the persisted shape also means old cache files remain
|
|
182
|
+
// valid while hot UI reads can skip redundant work.
|
|
183
|
+
const cacheRuntime = new WeakMap();
|
|
184
|
+
|
|
185
|
+
function runtimeOf(cache) {
|
|
186
|
+
let runtime = cacheRuntime.get(cache);
|
|
187
|
+
if (runtime === void 0) {
|
|
188
|
+
runtime = {
|
|
189
|
+
dirty: false,
|
|
190
|
+
aggregate: null,
|
|
191
|
+
aggregateDirty: true
|
|
192
|
+
};
|
|
193
|
+
cacheRuntime.set(cache, runtime);
|
|
194
|
+
}
|
|
195
|
+
return runtime;
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
function markCacheChanged(cache) {
|
|
199
|
+
const runtime = runtimeOf(cache);
|
|
200
|
+
runtime.dirty = true;
|
|
201
|
+
runtime.aggregateDirty = true;
|
|
202
|
+
}
|
|
203
|
+
|
|
204
|
+
function diagnostic(ctx, name) {
|
|
205
|
+
const stats = ctx?.__usageStatsDiagnostics;
|
|
206
|
+
if (stats === null || typeof stats !== "object") return;
|
|
207
|
+
stats[name] = (stats[name] ?? 0) + 1;
|
|
208
|
+
}
|
|
209
|
+
|
|
179
210
|
/** Serialize one session's fold state (Maps → plain objects). */
|
|
180
211
|
function serializeSession(state) {
|
|
181
212
|
const days = {};
|
|
@@ -309,6 +340,7 @@ function parsePricingIdentityCutoffs(raw) {
|
|
|
309
340
|
}
|
|
310
341
|
|
|
311
342
|
function freshCache(pricingFingerprintValue, previous = null, transitionAt = Date.now()) {
|
|
343
|
+
const fingerprintChanged = previous !== null && previous.pricingFingerprint !== pricingFingerprintValue;
|
|
312
344
|
const pricingIdentityCutoffs = parsePricingIdentityCutoffs(previous?.pricingIdentityCutoffs);
|
|
313
345
|
let pricingIdentityCutoffAll = Number.isFinite(previous?.pricingIdentityCutoffAll) && previous.pricingIdentityCutoffAll >= 0
|
|
314
346
|
? previous.pricingIdentityCutoffAll
|
|
@@ -318,13 +350,19 @@ function freshCache(pricingFingerprintValue, previous = null, transitionAt = Dat
|
|
|
318
350
|
if (changedRoutes === null) pricingIdentityCutoffAll = Math.max(pricingIdentityCutoffAll ?? 0, transitionAt);
|
|
319
351
|
else for (const routeId of changedRoutes) pricingIdentityCutoffs[routeId] = Math.max(pricingIdentityCutoffs[routeId] ?? 0, transitionAt);
|
|
320
352
|
}
|
|
321
|
-
|
|
353
|
+
const cache = {
|
|
322
354
|
version: CACHE_VERSION,
|
|
323
355
|
pricingFingerprint: pricingFingerprintValue,
|
|
324
356
|
pricingIdentityCutoffAll,
|
|
325
357
|
pricingIdentityCutoffs,
|
|
326
358
|
sessions: {}
|
|
327
359
|
};
|
|
360
|
+
const runtime = runtimeOf(cache);
|
|
361
|
+
// A pricing identity/catalog change invalidates all derived billing. Mark
|
|
362
|
+
// the replacement dirty even when the next collection deliberately avoids
|
|
363
|
+
// a persisted scan, so the stale fingerprint cannot survive a restart.
|
|
364
|
+
if (fingerprintChanged) runtime.dirty = true;
|
|
365
|
+
return cache;
|
|
328
366
|
}
|
|
329
367
|
|
|
330
368
|
function restoredCache(parsed, pricingFingerprintValue) {
|
|
@@ -332,7 +370,7 @@ function restoredCache(parsed, pricingFingerprintValue) {
|
|
|
332
370
|
for (const [id, entry] of Object.entries(parsed.sessions)) {
|
|
333
371
|
if (typeof id === "string" && id.length > 0) sessions[id] = parseSession(entry);
|
|
334
372
|
}
|
|
335
|
-
|
|
373
|
+
const cache = {
|
|
336
374
|
version: CACHE_VERSION,
|
|
337
375
|
pricingFingerprint: pricingFingerprintValue,
|
|
338
376
|
pricingIdentityCutoffAll: Number.isFinite(parsed.pricingIdentityCutoffAll) && parsed.pricingIdentityCutoffAll >= 0
|
|
@@ -341,6 +379,8 @@ function restoredCache(parsed, pricingFingerprintValue) {
|
|
|
341
379
|
pricingIdentityCutoffs: parsePricingIdentityCutoffs(parsed.pricingIdentityCutoffs),
|
|
342
380
|
sessions
|
|
343
381
|
};
|
|
382
|
+
runtimeOf(cache);
|
|
383
|
+
return cache;
|
|
344
384
|
}
|
|
345
385
|
|
|
346
386
|
/** Load against the current runtime provider/pricing fingerprint. */
|
|
@@ -374,6 +414,8 @@ async function loadCache(pricingFingerprintValue) {
|
|
|
374
414
|
|
|
375
415
|
/** Persist the cache atomically (temp + rename); failures are logged, never fatal. */
|
|
376
416
|
async function saveCache(ctx, cache) {
|
|
417
|
+
const runtime = runtimeOf(cache);
|
|
418
|
+
if (!runtime.dirty) return false;
|
|
377
419
|
try {
|
|
378
420
|
const path = cachePath();
|
|
379
421
|
await mkdir(dirname(path), { recursive: true });
|
|
@@ -388,18 +430,32 @@ async function saveCache(ctx, cache) {
|
|
|
388
430
|
const tmp = `${path}.tmp`;
|
|
389
431
|
await writeFile(tmp, JSON.stringify(serialized), "utf8");
|
|
390
432
|
await rename(tmp, path);
|
|
433
|
+
runtime.dirty = false;
|
|
434
|
+
diagnostic(ctx, "cacheWrites");
|
|
435
|
+
return true;
|
|
391
436
|
} catch (error) {
|
|
392
437
|
ctx.logger.warn(`usage-stats: saving usage cache failed: ${String(error)}`);
|
|
438
|
+
return false;
|
|
393
439
|
}
|
|
394
440
|
}
|
|
395
441
|
|
|
396
|
-
/**
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
442
|
+
/**
|
|
443
|
+
* Single-flight guard: concurrent requests share compatible aggregation work.
|
|
444
|
+
* A full persisted scan cannot be satisfied by an already-running live-only
|
|
445
|
+
* collection; it waits for that run and then performs its own full pass.
|
|
446
|
+
*/
|
|
447
|
+
function withCollectionLock(scanPersisted, run) {
|
|
448
|
+
if (inflight !== null) {
|
|
449
|
+
if (!scanPersisted || inflight.scanPersisted) return inflight.promise;
|
|
450
|
+
const waitForLive = () => withCollectionLock(true, run);
|
|
451
|
+
return inflight.promise.then(waitForLive, waitForLive);
|
|
452
|
+
}
|
|
453
|
+
const entry = { scanPersisted, promise: null };
|
|
454
|
+
entry.promise = run().finally(() => {
|
|
455
|
+
if (inflight === entry) inflight = null;
|
|
401
456
|
});
|
|
402
|
-
|
|
457
|
+
inflight = entry;
|
|
458
|
+
return entry.promise;
|
|
403
459
|
}
|
|
404
460
|
//#endregion
|
|
405
461
|
|
|
@@ -416,10 +472,47 @@ function withLock(run) {
|
|
|
416
472
|
* to be contiguous with the last folded seq — a gap or an empty delta means
|
|
417
473
|
* the log was truncated/rewritten, so the session is refolded from scratch.
|
|
418
474
|
* Sessions that vanished are dropped, and a session switching between
|
|
419
|
-
* live/persisted is refolded from scratch to stay exact.
|
|
475
|
+
* live/persisted is refolded from scratch to stay exact. High-frequency UI
|
|
476
|
+
* reads pass `{ scanPersisted: false }`: cached persisted folds remain in the
|
|
477
|
+
* aggregate, but the persistence backend is not enumerated or read. The
|
|
478
|
+
* five-minute background fold and exports use the default full scan.
|
|
420
479
|
*/
|
|
421
|
-
|
|
422
|
-
|
|
480
|
+
function liveSessionAdapter(session) {
|
|
481
|
+
if (session === null || typeof session !== "object") throw new TypeError("live session must be an object");
|
|
482
|
+
if (typeof session.snapshotEvents === "function") {
|
|
483
|
+
if (!("seq" in session)) throw new TypeError("live session exposes an incomplete snapshot API");
|
|
484
|
+
return {
|
|
485
|
+
count() {
|
|
486
|
+
const count = session.seq;
|
|
487
|
+
if (!Number.isSafeInteger(count) || count < 0) throw new TypeError("live session seq must be a non-negative safe integer");
|
|
488
|
+
return count;
|
|
489
|
+
},
|
|
490
|
+
tail(fromSeq) {
|
|
491
|
+
const events = session.snapshotEvents(fromSeq);
|
|
492
|
+
if (!Array.isArray(events)) throw new TypeError("live session snapshotEvents() must return an array");
|
|
493
|
+
return events;
|
|
494
|
+
}
|
|
495
|
+
};
|
|
496
|
+
}
|
|
497
|
+
const legacyEvents = session.events;
|
|
498
|
+
if (!Array.isArray(legacyEvents)) throw new TypeError("live session does not expose a supported snapshot API");
|
|
499
|
+
return {
|
|
500
|
+
count() {
|
|
501
|
+
const events = session.events;
|
|
502
|
+
if (!Array.isArray(events)) throw new TypeError("legacy live session events must be an array");
|
|
503
|
+
return events.length;
|
|
504
|
+
},
|
|
505
|
+
tail(fromSeq) {
|
|
506
|
+
const events = session.events;
|
|
507
|
+
if (!Array.isArray(events)) throw new TypeError("legacy live session events must be an array");
|
|
508
|
+
return events.slice(fromSeq);
|
|
509
|
+
}
|
|
510
|
+
};
|
|
511
|
+
}
|
|
512
|
+
|
|
513
|
+
export async function collectUsage(ctx, config = { monitors: {}, budgets: validateBudgetConfig() }, options = {}) {
|
|
514
|
+
const scanPersisted = options?.scanPersisted !== false;
|
|
515
|
+
return withCollectionLock(scanPersisted, async () => {
|
|
423
516
|
const providers = await configuredProviders(ctx);
|
|
424
517
|
const runtimePricingFingerprint = pricingFingerprint({ providers, config });
|
|
425
518
|
const cache = await loadCache(runtimePricingFingerprint);
|
|
@@ -437,13 +530,23 @@ export async function collectUsage(ctx, config = { monitors: {}, budgets: valida
|
|
|
437
530
|
if (live !== void 0) {
|
|
438
531
|
for (const session of live.list()) {
|
|
439
532
|
attached.add(session.id);
|
|
440
|
-
const
|
|
441
|
-
|
|
533
|
+
const liveAdapter = liveSessionAdapter(session);
|
|
534
|
+
let state = cache.sessions[session.id];
|
|
535
|
+
let stateChanged = false;
|
|
536
|
+
if (state === void 0) {
|
|
537
|
+
state = createUsageState();
|
|
538
|
+
stateChanged = true;
|
|
539
|
+
}
|
|
540
|
+
if (typeof session.title === "string" && state.title !== session.title) {
|
|
541
|
+
state.title = session.title;
|
|
542
|
+
stateChanged = true;
|
|
543
|
+
}
|
|
442
544
|
if (state.kind !== "live") {
|
|
443
545
|
// Live/persisted transition: refold the whole in-memory log.
|
|
444
546
|
resetUsageState(state);
|
|
547
|
+
stateChanged = true;
|
|
445
548
|
}
|
|
446
|
-
const count =
|
|
549
|
+
const count = liveAdapter.count();
|
|
447
550
|
if (count < (state.consumed ?? 0)) {
|
|
448
551
|
// The in-memory log shrank below the folded cursor — DSH
|
|
449
552
|
// restores sessions from disk as compressed summaries after
|
|
@@ -452,45 +555,69 @@ export async function collectUsage(ctx, config = { monitors: {}, budgets: valida
|
|
|
452
555
|
// forever (#23). The cursor is meaningless against the
|
|
453
556
|
// rebuilt log: refold it from scratch.
|
|
454
557
|
resetUsageState(state);
|
|
558
|
+
stateChanged = true;
|
|
455
559
|
}
|
|
456
560
|
if ((state.consumed ?? 0) < count) {
|
|
457
|
-
applyUsageDelta(state,
|
|
561
|
+
applyUsageDelta(state, liveAdapter.tail(state.consumed ?? 0), { estimateCost });
|
|
458
562
|
state.consumed = count;
|
|
563
|
+
stateChanged = true;
|
|
564
|
+
}
|
|
565
|
+
if (state.kind !== "live") {
|
|
566
|
+
state.kind = "live";
|
|
567
|
+
stateChanged = true;
|
|
459
568
|
}
|
|
460
|
-
state.kind = "live";
|
|
461
569
|
cache.sessions[session.id] = state;
|
|
570
|
+
if (stateChanged) markCacheChanged(cache);
|
|
462
571
|
}
|
|
463
572
|
}
|
|
464
573
|
const persistence = ctx.get("sessionPersistence");
|
|
465
574
|
const persistedIds = new Set();
|
|
466
|
-
if (persistence !== void 0) {
|
|
575
|
+
if (scanPersisted && persistence !== void 0) {
|
|
467
576
|
// Prefer the backend's opaque per-log revisions (no file I/O in the
|
|
468
577
|
// plugin, works for any backend that exposes listSnapshots).
|
|
469
578
|
let snapshots = null;
|
|
470
579
|
if (typeof persistence.listSnapshots === "function") {
|
|
471
580
|
try {
|
|
581
|
+
diagnostic(ctx, "listSnapshots");
|
|
472
582
|
snapshots = await persistence.listSnapshots();
|
|
473
583
|
} catch (error) {
|
|
474
584
|
ctx.logger.warn(`usage-stats: listSnapshots failed, falling back to list(): ${String(error)}`);
|
|
475
585
|
}
|
|
476
586
|
}
|
|
477
|
-
|
|
587
|
+
let metas;
|
|
588
|
+
if (snapshots !== null) metas = snapshots.map((entry) => entry.header);
|
|
589
|
+
else {
|
|
590
|
+
diagnostic(ctx, "list");
|
|
591
|
+
metas = typeof persistence.list === "function" ? await persistence.list() : [];
|
|
592
|
+
}
|
|
478
593
|
const revisionOf = new Map();
|
|
479
594
|
if (snapshots !== null) for (const entry of snapshots) revisionOf.set(entry.header.id, entry.revision);
|
|
480
595
|
for (const meta of metas) {
|
|
481
596
|
persistedIds.add(meta.id);
|
|
482
597
|
if (attached.has(meta.id)) continue;
|
|
483
|
-
|
|
484
|
-
|
|
598
|
+
let state = cache.sessions[meta.id];
|
|
599
|
+
let stateChanged = false;
|
|
600
|
+
if (state === void 0) {
|
|
601
|
+
state = createUsageState();
|
|
602
|
+
stateChanged = true;
|
|
603
|
+
}
|
|
604
|
+
if (typeof meta.title === "string" && state.title !== meta.title) {
|
|
605
|
+
state.title = meta.title;
|
|
606
|
+
stateChanged = true;
|
|
607
|
+
}
|
|
485
608
|
const revision = revisionOf.get(meta.id);
|
|
486
|
-
const
|
|
609
|
+
const revisionChanged = revision !== void 0 && revision !== state.revision;
|
|
610
|
+
const changed = state.kind !== "persisted" || revisionChanged || revision === void 0;
|
|
611
|
+
if (revisionChanged) stateChanged = true;
|
|
487
612
|
if (changed) {
|
|
488
613
|
try {
|
|
489
614
|
const wasPersisted = state.kind === "persisted";
|
|
490
615
|
const fromSeq = wasPersisted ? state.consumed : 0;
|
|
616
|
+
diagnostic(ctx, "readFrom");
|
|
491
617
|
const { events } = await persistence.readFrom(meta.id, fromSeq);
|
|
492
618
|
if (!wasPersisted) {
|
|
493
619
|
resetUsageState(state);
|
|
620
|
+
stateChanged = true;
|
|
494
621
|
}
|
|
495
622
|
const fresh = wasPersisted ? events.filter((event) => event.seq > (state.consumed ?? 0)) : events;
|
|
496
623
|
// readFrom is inclusive: an unchanged log returns exactly the
|
|
@@ -504,31 +631,51 @@ export async function collectUsage(ctx, config = { monitors: {}, budgets: valida
|
|
|
504
631
|
if (!contiguous && state.consumed > 0) {
|
|
505
632
|
// Log truncated or rewritten: refold the whole log.
|
|
506
633
|
resetUsageState(state);
|
|
634
|
+
stateChanged = true;
|
|
635
|
+
diagnostic(ctx, "readFrom");
|
|
507
636
|
const { events: allEvents } = await persistence.readFrom(meta.id, 0);
|
|
508
637
|
applyUsageDelta(state, allEvents, { estimateCost });
|
|
509
638
|
state.consumed = allEvents.length > 0 ? allEvents[allEvents.length - 1].seq : 0;
|
|
510
639
|
} else if (fresh.length > 0) {
|
|
511
640
|
applyUsageDelta(state, fresh, { estimateCost });
|
|
512
641
|
state.consumed = fresh[fresh.length - 1].seq;
|
|
642
|
+
stateChanged = true;
|
|
643
|
+
}
|
|
644
|
+
if (state.kind !== "persisted") {
|
|
645
|
+
state.kind = "persisted";
|
|
646
|
+
stateChanged = true;
|
|
647
|
+
}
|
|
648
|
+
if (revision !== void 0 && state.revision !== revision) {
|
|
649
|
+
state.revision = revision;
|
|
650
|
+
stateChanged = true;
|
|
513
651
|
}
|
|
514
|
-
state.kind = "persisted";
|
|
515
|
-
if (revision !== void 0) state.revision = revision;
|
|
516
652
|
} catch (error) {
|
|
517
653
|
ctx.logger.warn(`usage-stats: reading persisted session "${meta.id}" failed: ${String(error)}`);
|
|
518
654
|
}
|
|
519
655
|
}
|
|
520
656
|
cache.sessions[meta.id] = state;
|
|
657
|
+
if (stateChanged) markCacheChanged(cache);
|
|
658
|
+
}
|
|
659
|
+
for (const id of Object.keys(cache.sessions)) {
|
|
660
|
+
if (!attached.has(id) && !persistedIds.has(id)) {
|
|
661
|
+
delete cache.sessions[id];
|
|
662
|
+
markCacheChanged(cache);
|
|
663
|
+
}
|
|
521
664
|
}
|
|
522
665
|
}
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
666
|
+
const runtime = runtimeOf(cache);
|
|
667
|
+
if (runtime.aggregate === null || runtime.aggregateDirty) {
|
|
668
|
+
diagnostic(ctx, "aggregateRebuilds");
|
|
669
|
+
const byDay = new Map();
|
|
670
|
+
const billingByDay = new Map();
|
|
671
|
+
for (const state of Object.values(cache.sessions)) {
|
|
672
|
+
mergeInto(byDay, state.days);
|
|
673
|
+
mergeBillingInto(billingByDay, state.billing.days);
|
|
674
|
+
}
|
|
675
|
+
runtime.aggregate = { byDay, billingByDay };
|
|
676
|
+
runtime.aggregateDirty = false;
|
|
531
677
|
}
|
|
678
|
+
const { byDay, billingByDay } = runtime.aggregate;
|
|
532
679
|
// Keep the atomic cache write inside the single-flight section. Otherwise
|
|
533
680
|
// overlapping saves can race on the same temporary file.
|
|
534
681
|
await saveCache(ctx, cache);
|
|
@@ -545,7 +692,7 @@ export async function collectUsage(ctx, config = { monitors: {}, budgets: valida
|
|
|
545
692
|
async function handleUsage(ctx, config, req, res) {
|
|
546
693
|
if (rejectForeignCaller(req, res)) return;
|
|
547
694
|
try {
|
|
548
|
-
const result = await collectUsage(ctx, config);
|
|
695
|
+
const result = await collectUsage(ctx, config, { scanPersisted: false });
|
|
549
696
|
json(res, 200, { ok: true, ...result });
|
|
550
697
|
} catch (error) {
|
|
551
698
|
ctx.logger.warn(`usage-stats: usage aggregation failed: ${String(error)}`);
|
|
@@ -562,7 +709,7 @@ function exportFailure(ctx, kind, error, res) {
|
|
|
562
709
|
async function handleDailyExport(ctx, config, req, res) {
|
|
563
710
|
if (rejectForeignCaller(req, res)) return;
|
|
564
711
|
try {
|
|
565
|
-
attachment(res, "text/csv; charset=utf-8", "dsh-usage-daily.csv", dailyCsv(await collectUsage(ctx, config)));
|
|
712
|
+
attachment(res, "text/csv; charset=utf-8", "dsh-usage-daily.csv", dailyCsv(await collectUsage(ctx, config, { scanPersisted: true })));
|
|
566
713
|
} catch (error) {
|
|
567
714
|
exportFailure(ctx, "daily CSV", error, res);
|
|
568
715
|
}
|
|
@@ -571,7 +718,7 @@ async function handleDailyExport(ctx, config, req, res) {
|
|
|
571
718
|
async function handleSessionsExport(ctx, config, req, res) {
|
|
572
719
|
if (rejectForeignCaller(req, res)) return;
|
|
573
720
|
try {
|
|
574
|
-
attachment(res, "text/csv; charset=utf-8", "dsh-usage-sessions.csv", sessionsCsv(await collectUsage(ctx, config)));
|
|
721
|
+
attachment(res, "text/csv; charset=utf-8", "dsh-usage-sessions.csv", sessionsCsv(await collectUsage(ctx, config, { scanPersisted: true })));
|
|
575
722
|
} catch (error) {
|
|
576
723
|
exportFailure(ctx, "session CSV", error, res);
|
|
577
724
|
}
|
|
@@ -580,7 +727,10 @@ async function handleSessionsExport(ctx, config, req, res) {
|
|
|
580
727
|
async function handleJsonExport(ctx, config, accounts, req, res) {
|
|
581
728
|
if (rejectForeignCaller(req, res)) return;
|
|
582
729
|
try {
|
|
583
|
-
const [usage, providerViews] = await Promise.all([
|
|
730
|
+
const [usage, providerViews] = await Promise.all([
|
|
731
|
+
collectUsage(ctx, config, { scanPersisted: true }),
|
|
732
|
+
accounts.providerViews()
|
|
733
|
+
]);
|
|
584
734
|
attachment(res, "application/json; charset=utf-8", "dsh-usage-stats.json", JSON.stringify(jsonExport(usage, providerViews)));
|
|
585
735
|
} catch (error) {
|
|
586
736
|
exportFailure(ctx, "JSON", error, res);
|
|
@@ -635,7 +785,7 @@ async function configuredProviders(ctx) {
|
|
|
635
785
|
* events), and add no stream listener, provider request, cache, or usage ledger.
|
|
636
786
|
*/
|
|
637
787
|
export async function collectSessionContext(ctx, sessionId, config = { monitors: {} }, selectedRoute = null) {
|
|
638
|
-
await collectUsage(ctx, config);
|
|
788
|
+
await collectUsage(ctx, config, { scanPersisted: false });
|
|
639
789
|
const sessions = ctx.get("sessions");
|
|
640
790
|
const live = sessions?.get?.(sessionId) ?? sessions?.list?.().find((session) => session.id === sessionId);
|
|
641
791
|
if (live === void 0) return null;
|
|
@@ -655,8 +805,8 @@ export async function collectSessionContext(ctx, sessionId, config = { monitors:
|
|
|
655
805
|
}
|
|
656
806
|
|
|
657
807
|
/** Resolve the current account ids for every live session without a new ledger or provider guess. */
|
|
658
|
-
export async function collectActiveAccountIds(ctx, config = { monitors: {} }) {
|
|
659
|
-
await collectUsage(ctx, config);
|
|
808
|
+
export async function collectActiveAccountIds(ctx, config = { monitors: {} }, options = { scanPersisted: false }) {
|
|
809
|
+
await collectUsage(ctx, config, options);
|
|
660
810
|
const sessions = ctx.get("sessions")?.list?.() ?? [];
|
|
661
811
|
const cache = loadedCache;
|
|
662
812
|
if (cache === null) return [];
|
|
@@ -917,8 +1067,8 @@ export function startBackgroundRefresh(ctx, accounts, deps = {}) {
|
|
|
917
1067
|
const at = now();
|
|
918
1068
|
if (force || at >= nextUsageAt) {
|
|
919
1069
|
try {
|
|
920
|
-
if (accountRefreshEnabled) accounts.setActiveProviders(await collectActiveAccountIds(ctx, config));
|
|
921
|
-
else await collectUsage(ctx, config);
|
|
1070
|
+
if (accountRefreshEnabled) accounts.setActiveProviders(await collectActiveAccountIds(ctx, config, { scanPersisted: true }));
|
|
1071
|
+
else await collectUsage(ctx, config, { scanPersisted: true });
|
|
922
1072
|
} catch (error) {
|
|
923
1073
|
ctx.logger.warn(`usage-stats: background usage refresh failed: ${String(error)}`);
|
|
924
1074
|
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ychris12138/dsh-usage-stats",
|
|
3
3
|
"description": "Token usage, provider accounts, session cost estimates, budgets, and exports for the dsh web GUI",
|
|
4
|
-
"version": "0.3.
|
|
4
|
+
"version": "0.3.2",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
7
7
|
"url": "git+https://github.com/Ychris12138/dsh-usage-stats.git"
|
|
@@ -24,6 +24,7 @@
|
|
|
24
24
|
"docs/release-checklist.md",
|
|
25
25
|
"docs/release-notes-v0.3.0.md",
|
|
26
26
|
"docs/release-notes-v0.3.1.md",
|
|
27
|
+
"docs/release-notes-v0.3.2.md",
|
|
27
28
|
"scripts/install.mjs",
|
|
28
29
|
"README.md",
|
|
29
30
|
"LICENSE",
|