usagemax 0.3.0 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -78,21 +78,31 @@ connector. UsageMax never invents usage that the source did not retain.
78
78
 
79
79
  ## Privacy, correctness, and load
80
80
 
81
- - The sync payload contains authoritative aggregate token counts,
81
+ Interrupted uploads resume with `bunx usagemax sync`. Server upload runs expire
82
+ after 30 idle days. If the server reports an expired run, use `sync --restart`:
83
+ this preserves the last committed checkpoint and performs a fresh full scan.
84
+ It never automatically replays an old authoritative deletion against newer data.
85
+
86
+ - The sync payload contains aggregate token counts,
82
87
  model/provider names, costs, source names, dates, coverage state, and opaque
83
88
  SHA-256 session identities.
84
89
  - It does not upload prompts, completions, source code, file contents, project
85
90
  paths, or provider credentials.
86
91
  - Sync is one-shot. There is no resident scanner or high-frequency polling loop.
87
- - A metadata inventory exits without parsing logs or using the network when
88
- nothing changed.
92
+ - A successful previous parse with a complete, unchanged metadata inventory skips
93
+ parsing and uploading again that day. Inventory stability is independent of
94
+ deletion authority; a no-change result retains the previous partial coverage
95
+ label. New sources, date rollover, explicit full/archive requests and weekly
96
+ reconciliation still trigger the appropriate scan.
89
97
  - Normal changed syncs parse today or today plus yesterday. A bounded weekly
90
98
  full reconciliation catches restored files, parser changes, and older logs.
91
- - Full history means all retained local history from 2024 onward. UsageMax
92
- publishes complete source/day partitions, so decreased or removed local rows
93
- correct the server-owned contribution instead of being silently retained.
94
- Deleted or
95
- never-persisted usage requires a provider export; no local tool can reconstruct it.
99
+ - Full history scans retained local history from 2024 onward. Inventory success
100
+ does not prove every file parsed. Until the parser certifies source/day coverage,
101
+ uploads are marked partial and any row with a decreasing counter retains its
102
+ entire previous vector. This includes explicit zeros and missing sources.
103
+ Older checkpoints survive incremental windows. Authoritative corrections remain
104
+ supported by the planner but are not claimed by this parser integration.
105
+ Deleted or never-persisted usage requires a provider export.
96
106
  - Source totals that cannot be assigned to a model are retained as
97
107
  `unattributed` rather than silently discarded.
98
108
  - The collector key is written with user-only permissions where the operating
@@ -103,7 +113,8 @@ connector. UsageMax never invents usage that the source did not retain.
103
113
  its hash and applies device binding, replay checks, payload caps, and quotas.
104
114
  - A private random installation ID survives collector rotation, relinking, and
105
115
  display-name changes. It is not a hardware fingerprint; the server stores only
106
- its SHA-256 hash. Concurrent and repeated syncs are idempotent.
116
+ its SHA-256 hash. A local config lock prevents overlapping commands from
117
+ overwriting pending runs; server receipts make repeated uploads idempotent.
107
118
  - The server, not the local checkpoint, owns the accounting baseline. A lost
108
119
  response or interrupted run can be retried without adding the same partition twice.
109
120
  - Do not point two different installations at the same copied or network-mounted
@@ -113,5 +124,36 @@ connector. UsageMax never invents usage that the source did not retain.
113
124
  Use `USAGEMAX_CONFIG_DIR` to select another config directory. Development and
114
125
  self-hosted installations may set `USAGEMAX_LINK_ENDPOINT` before linking.
115
126
 
127
+ ## Interrupted uploads and protocol 0.3.1
128
+
129
+ Before uploading, the CLI saves the exact run, ordered request payloads and next
130
+ checkpoint in the private config. `sync` resumes this journal before scanning new
131
+ data. Local snapshots advance only after completion is acknowledged. A resumed
132
+ command finishes the saved scan; run `sync` again to collect subsequent changes.
133
+ `sync --dry-run` reports pending work without uploading. Do not delete config.json
134
+ to retry a failed upload. A successful relink or unlink replaces/removes the local
135
+ journal along with its credential.
136
+
137
+ Network failures, malformed success responses, HTTP 429 and 5xx get at most five
138
+ attempts per request with exponential jitter. Retry-After seconds and HTTP dates
139
+ are honored up to 60 seconds; longer waits stop with an instruction to retry later.
140
+ Only completion's `snapshot_run_incomplete` HTTP 409 is retried, because server
141
+ cleanup can still be pending. Authentication, validation and other conflicts fail
142
+ with an actionable message and retain the journal. JSON `accepted` is null for
143
+ uploads because lost responses/replays cannot reliably reconstruct that count;
144
+ `changedRows` is the local planned count, not a server accounting receipt.
145
+
146
+ Each sorted source/day is sent as ordered chunks of at most 100 rows. Each has a
147
+ unique partitionId, zero-based chunkIndex and shared chunkCount. The payload hash
148
+ covers `{source, day, complete, pricingVersion, chunkIndex, chunkCount, rows}` in
149
+ that order. `partitionCount` counts transmitted chunks. The server must accept
150
+ this protocol, receipt replays, and finish omitted-row cleanup before completing
151
+ an authoritative run.
152
+
153
+ An abrupt process kill can leave `collector.lock` in the config directory. The
154
+ next command reports the owner PID and exact path. Confirm that process has exited
155
+ before removing only that lock file, then rerun sync. Never remove an active lock
156
+ or the saved config to recover. Normal completion and handled failures release it.
157
+
116
158
  See the [collector coverage audit](../../docs/collector-coverage-audit.md) for
117
159
  the full support matrix and known boundaries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "usagemax",
3
- "version": "0.3.0",
3
+ "version": "0.3.2",
4
4
  "description": "Link local coding-agent usage to your UsageMax profile",
5
5
  "license": "MIT",
6
6
  "author": "UsageMax",
@@ -23,6 +23,8 @@
23
23
  "src/core.js",
24
24
  "src/installation.js",
25
25
  "src/sources.js",
26
+ "src/transport.js",
27
+ "src/resume.js",
26
28
  "README.md",
27
29
  "LICENSE"
28
30
  ],
package/src/cli.js CHANGED
@@ -10,18 +10,19 @@ import process from "node:process";
10
10
  import { promisify } from "node:util";
11
11
 
12
12
  import { prepareArchiveRecovery } from "./archives.js";
13
- import { buildSessionPlan, buildSnapshotPlan, normalizeLinkCode, sourceSummary, validHttpsUrl } from "./core.js";
13
+ import { buildSessionPlan, buildSnapshotPlan, normalizeLinkCode, reportDateArgs, scanPolicy, sourceSummary, validHttpsUrl } from "./core.js";
14
14
  import { stableInstallationId } from "./installation.js";
15
+ import { requestSnapshot } from "./transport.js";
16
+ import { resumeUpload, restartExpiredUpload, withConfigLock } from "./resume.js";
15
17
  import { CCUSAGE_VERSION, ccusageEnvironment, ccusageHome, discoverProviderArchives, SOURCE_INVENTORY_VERSION, sourceInventory, SUPPORTED_SOURCES } from "./sources.js";
16
18
 
17
19
  const require = createRequire(import.meta.url);
18
20
  const executeFile = promisify(execFile);
19
- const VERSION = "0.3.0";
21
+ const VERSION = "0.3.2";
20
22
  const PUBLIC_API_ORIGIN = "https://usagemax.com/api";
21
23
  const DEFAULT_LINK_ENDPOINT = `${PUBLIC_API_ORIGIN}/v1/devices/link`;
22
24
  const CONFIG_FILE = "config.json";
23
25
  const MAX_REPORT_BYTES = 100 * 1024 * 1024;
24
- const FULL_RECONCILE_INTERVAL_MS = 7 * 24 * 60 * 60 * 1000;
25
26
 
26
27
  function configDirectory() {
27
28
  if (process.env.USAGEMAX_CONFIG_DIR) return process.env.USAGEMAX_CONFIG_DIR;
@@ -103,7 +104,7 @@ function help() {
103
104
  process.stdout.write(" usagemax Sync changed local usage\n");
104
105
  process.stdout.write(" usagemax link <one-use-code> Link and sync this computer\n");
105
106
  process.stdout.write(" [--no-sync] [--name <name>]\n");
106
- process.stdout.write(" usagemax sync [--full] [--archives] [--dry-run] [--explain] [--json]\n");
107
+ process.stdout.write(" usagemax sync [--full] [--archives] [--restart] [--dry-run] [--explain] [--json]\n");
107
108
  process.stdout.write(" Reconcile once; --archives performs one-time recovery\n");
108
109
  process.stdout.write(" usagemax status Show local link status\n");
109
110
  process.stdout.write(" usagemax doctor [--deep] [--json]\n");
@@ -122,12 +123,7 @@ async function ccusageJson(config, { full = false, env } = {}) {
122
123
  // Reconcile yesterday once after the UTC date changes. All other incremental
123
124
  // scans parse only today; a metadata fingerprint avoids invoking ccusage when
124
125
  // no supported local source changed at all.
125
- if (full) {
126
- args.push("--since", "2024-01-01", "--until", new Date().toISOString().slice(0, 10));
127
- } else if (config?.lastSyncAt) {
128
- const today = new Date().toISOString().slice(0, 10);
129
- args.push("--last", config.lastReconciledDay === today ? "1" : "2");
130
- }
126
+ args.push(...reportDateArgs(config, { full }));
131
127
  const { stdout } = await executeFile(process.execPath, args, {
132
128
  encoding: "utf8",
133
129
  maxBuffer: MAX_REPORT_BYTES,
@@ -142,22 +138,7 @@ function snapshotEndpoint(config) {
142
138
  }
143
139
 
144
140
  async function snapshotRequest(config, operation, payload, timeout = 30_000) {
145
- const response = await fetch(snapshotEndpoint(config), {
146
- method: "POST",
147
- headers: {
148
- authorization: `Bearer ${config.token}`,
149
- "content-type": "application/json",
150
- "x-usagemax-device-id": config.deviceId,
151
- },
152
- body: JSON.stringify({ operation, ...payload }),
153
- signal: AbortSignal.timeout(timeout),
154
- });
155
- const body = await response.json().catch(() => ({}));
156
- if (!response.ok) {
157
- if (body?.error === "unauthorized") throw new Error("This collector key is no longer valid. Link the computer again.");
158
- throw new Error(`UsageMax rejected ${operation} (${response.status}${body?.error ? `: ${body.error}` : ""}).`);
159
- }
160
- return body;
141
+ return requestSnapshot(snapshotEndpoint(config), config, operation, payload, { timeout });
161
142
  }
162
143
 
163
144
  function newerVersion(recommended) {
@@ -243,32 +224,44 @@ async function syncPrepared(args, suppliedConfig, recovery) {
243
224
  const dryRun = args.includes("--dry-run");
244
225
  const explain = args.includes("--explain");
245
226
  const json = args.includes("--json");
227
+ if (args.includes("--restart") && !dryRun) {
228
+ restartExpiredUpload(config);
229
+ await writeConfig(config);
230
+ }
231
+ if (config.pendingSync) {
232
+ if (dryRun) {
233
+ const result = { ...config.pendingSync.result, dryRun: true, pendingRunId: config.pendingSync.runId };
234
+ process.stdout.write(json ? `${JSON.stringify(result)}\n` : `Dry run: saved run ${config.pendingSync.runId} awaits resume; no upload.\n`);
235
+ return result;
236
+ }
237
+ const result = await resumeUpload(config, { save: writeConfig, request: snapshotRequest, warn: warnVersion });
238
+ process.stdout.write(json ? `${JSON.stringify(result)}\n` : "Resumed and completed the saved sync. Run sync again to scan newer local changes.\n");
239
+ return result;
240
+ }
246
241
  const inventory = await sourceInventory({ env: recovery.env, home: ccusageHome(recovery.env) });
247
242
  const today = new Date().toISOString().slice(0, 10);
248
243
  const knownSources = Array.isArray(config.knownSources) ? config.knownSources : [];
249
- const foundNewSource = inventory.sources.some((source) => !knownSources.includes(source));
250
- const lastFullSync = Date.parse(config.lastFullSyncAt || "");
251
- const fullDue = config.snapshotProtocolVersion !== 2
252
- || config.sourceInventoryVersion !== SOURCE_INVENTORY_VERSION
253
- || !Number.isFinite(lastFullSync)
254
- || Date.now() - lastFullSync >= FULL_RECONCILE_INTERVAL_MS
255
- || foundNewSource;
256
- const full = requestedFull || requestedArchives || fullDue;
257
- if (!full && inventory.complete && config.lastSyncComplete && config.lastReconciledDay === today && config.sourceFingerprint === inventory.fingerprint) {
258
- const result = { accepted: 0, changedRows: 0, sessions: 0, sources: inventory.sources, corrections: 0, scanned: false, full: false, coverage: inventory.complete ? "complete" : "partial" };
244
+ const { bootstrap, full, skip, inventoryStable } = scanPolicy(config, inventory, {
245
+ today, now: Date.now(), inventoryVersion: SOURCE_INVENTORY_VERSION, requestedFull, requestedArchives,
246
+ });
247
+ if (skip) {
248
+ const result = { accepted: 0, changedRows: 0, sessions: 0, sources: inventory.sources, corrections: 0, scanned: false, full: false, coverage: config.lastCoverage || "partial" };
259
249
  if (json) process.stdout.write(`${JSON.stringify(result)}\n`);
260
250
  else process.stdout.write("Already up to date. Local usage files have not changed; no logs were parsed or uploaded.\n");
261
251
  return;
262
252
  }
263
253
  const report = await ccusageJson(config, { env: recovery.env, full });
264
- const legacySnapshotBootstrap = config.snapshotProtocolVersion !== 2
254
+ // ccusage v20 exposes aggregates, not proof that every discovered file was
255
+ // parsed. Inventory success alone cannot authorize destructive corrections.
256
+ const authoritative = false;
257
+ const legacySnapshotBootstrap = bootstrap
265
258
  && Object.keys(config.snapshots || {}).length > 0;
266
259
  const runId = randomUUID();
267
260
  const revision = Date.now();
268
261
  const pricingVersion = `ccusage@${CCUSAGE_VERSION}`;
269
262
  const { partitions, nextSnapshots, regressions } = buildSnapshotPlan(report, config.snapshots, {
270
- bootstrap: config.snapshotProtocolVersion !== 2,
271
- complete: inventory.complete,
263
+ bootstrap,
264
+ complete: authoritative,
272
265
  full,
273
266
  pricingVersion,
274
267
  revision,
@@ -280,15 +273,17 @@ async function syncPrepared(args, suppliedConfig, recovery) {
280
273
  ? report.daily.map((row) => row?.period).filter((day) => /^\d{4}-\d{2}-\d{2}$/.test(day || "")).sort()
281
274
  : [];
282
275
  const result = {
283
- accepted: 0,
276
+ accepted: null,
284
277
  changedRows: partitions.reduce((sum, partition) => sum + partition.rows.filter((row) => JSON.stringify(row.previous) !== JSON.stringify(row.current)).length, 0),
285
278
  sessions: sessions.length,
286
279
  sources,
287
- corrections: regressions.length,
280
+ corrections: authoritative ? regressions.length : 0,
281
+ protectedRegressions: authoritative ? 0 : regressions.length,
288
282
  partitions: partitions.length,
289
283
  scanned: true,
290
284
  full,
291
- coverage: inventory.complete && !inventory.truncated && inventory.errors === 0 ? "complete" : "partial",
285
+ coverage: authoritative ? "complete" : "partial",
286
+ coverageReason: "Parser does not certify complete source/day coverage; decreases and deletions are protected.",
292
287
  range: { from: days[0], to: days.at(-1) },
293
288
  };
294
289
  if (requestedArchives) {
@@ -299,60 +294,52 @@ async function syncPrepared(args, suppliedConfig, recovery) {
299
294
  if (json) process.stdout.write(`${JSON.stringify({ ...result, dryRun: true })}\n`);
300
295
  else {
301
296
  process.stdout.write(`Dry run: ${partitions.length} partition(s), ${result.changedRows} changed row(s), ${sessions.length} private session identifiers, no upload.\n`);
302
- if (explain) process.stdout.write(`Coverage ${result.coverage}; ${sources.length} source(s); ${days[0] || "unknown"} to ${days.at(-1) || "unknown"}; ${regressions.length} downward correction(s).\n`);
297
+ if (explain) process.stdout.write(`Coverage ${result.coverage}; ${sources.length} source(s); ${days[0] || "unknown"} to ${days.at(-1) || "unknown"}; ${regressions.length} protected regression(s). ${result.coverageReason}\n`);
303
298
  }
304
299
  return result;
305
300
  }
306
- config.lastSyncComplete = false;
307
- await writeConfig(config);
308
- try {
309
- const begin = await snapshotRequest(config, "begin", {
301
+ const requests = [{ operation: "begin", payload: {
310
302
  runId,
311
303
  mode: requestedArchives ? "archives" : full ? "full" : "incremental",
312
304
  baselineMode: legacySnapshotBootstrap ? "adopt-current" : "apply",
313
305
  sourceCount: sources.length,
314
306
  partitionCount: partitions.length,
315
- inventoryComplete: inventory.complete,
307
+ inventoryComplete: authoritative,
316
308
  inventoryErrors: inventory.errors,
317
309
  inventoryTruncated: inventory.truncated,
318
310
  coverageStartDay: days[0],
319
311
  coverageEndDay: days.at(-1),
320
- });
321
- warnVersion(begin);
322
- for (let offset = 0; offset < sessions.length; offset += 100) {
323
- await snapshotRequest(config, "sessions", { runId, sessions: sessions.slice(offset, offset + 100) });
324
- }
325
- for (let offset = 0; offset < partitions.length; offset += 10) {
326
- const response = await snapshotRequest(config, "partitions", { runId, partitions: partitions.slice(offset, offset + 10) }, 60_000);
327
- result.accepted += Number(response.changedRows || 0);
328
- if (!json && process.stderr.isTTY && partitions.length > 10) {
329
- process.stderr.write(`UsageMax: uploaded ${Math.min(offset + 10, partitions.length)}/${partitions.length} history partitions\r`);
330
- }
331
- }
332
- if (!json && process.stderr.isTTY && partitions.length > 10) process.stderr.write("\n");
333
- const completed = await snapshotRequest(config, "complete", { runId });
334
- warnVersion(completed);
335
- config.snapshots = nextSnapshots;
336
- config.snapshotProtocolVersion = 2;
337
- config.lastSyncAt = new Date().toISOString();
338
- config.lastReconciledDay = today;
339
- config.lastSyncComplete = true;
340
- config.sourceInventoryVersion = SOURCE_INVENTORY_VERSION;
341
- config.knownSources = [...new Set([...knownSources, ...inventory.sources, ...sources])].sort();
342
- if (inventory.complete) config.sourceFingerprint = inventory.fingerprint;
343
- else delete config.sourceFingerprint;
344
- if (full) config.lastFullSyncAt = config.lastSyncAt;
345
- await writeConfig(config);
346
- } catch (error) {
347
- await snapshotRequest(config, "fail", { runId, failureCode: error instanceof Error ? error.message.slice(0, 80) : "sync_failed" }).catch(() => undefined);
348
- throw error;
312
+ } }];
313
+ for (let offset = 0; offset < sessions.length; offset += 100) {
314
+ requests.push({ operation: "sessions", payload: { runId, sessions: sessions.slice(offset, offset + 100) } });
349
315
  }
316
+ for (let offset = 0; offset < partitions.length; offset += 10) {
317
+ requests.push({ operation: "partitions", payload: { runId, partitions: partitions.slice(offset, offset + 10) } });
318
+ }
319
+ requests.push({ operation: "complete", payload: { runId } });
320
+ const syncedAt = new Date().toISOString();
321
+ config.pendingSync = {
322
+ version: 1, runId, cursor: 0, requests, result,
323
+ checkpoint: {
324
+ snapshots: nextSnapshots, snapshotProtocolVersion: 2,
325
+ lastSyncAt: syncedAt, lastReconciledDay: today, lastSyncComplete: authoritative,
326
+ lastScanSucceeded: true, lastCoverage: result.coverage,
327
+ sourceInventoryVersion: SOURCE_INVENTORY_VERSION,
328
+ knownSources: [...new Set([...knownSources, ...inventory.sources, ...sources])].sort(),
329
+ sourceFingerprint: inventoryStable ? inventory.fingerprint : null,
330
+ ...(full ? { lastFullSyncAt: syncedAt } : {}),
331
+ },
332
+ };
333
+ config.lastSyncComplete = false;
334
+ await writeConfig(config);
335
+ await resumeUpload(config, { save: writeConfig, request: snapshotRequest, warn: warnVersion });
350
336
  if (json) process.stdout.write(`${JSON.stringify(result)}\n`);
351
337
  else {
352
338
  process.stdout.write(partitions.length || sessions.length
353
- ? `Reconciled ${result.accepted} changed usage row(s) and ${sessions.length} session identifier(s) from ${sources.join(", ") || "local agents"}${full ? " across retained history" : ""}.\n`
339
+ ? `Completed ${partitions.length} usage chunk(s) and ${sessions.length} session identifier(s) from ${sources.join(", ") || "local agents"}${full ? " across retained history" : ""}.\n`
354
340
  : `Already up to date. No usage rows changed${full ? " after a full-history reconciliation" : ""}.\n`);
355
- if (regressions.length) process.stdout.write(`${regressions.length} local row(s) moved backward and were submitted as authoritative corrections.\n`);
341
+ if (regressions.length) process.stdout.write(`${regressions.length} local row(s) moved backward; prior counter dimensions were preserved because coverage is incomplete.\n`);
342
+ if (!authoritative) process.stdout.write(`${result.coverageReason}\n`);
356
343
  if (explain) process.stdout.write(`Coverage ${result.coverage}; ${sources.length} source(s); ${days[0] || "unknown"} to ${days.at(-1) || "unknown"}; ${partitions.length} atomic partition(s).\n`);
357
344
  }
358
345
  return result;
@@ -372,6 +359,7 @@ async function status() {
372
359
  process.stdout.write(`Linked: ${config.deviceName || deviceLabel()}${config.profileHandle ? ` → @${config.profileHandle}` : ""}\n`);
373
360
  process.stdout.write(`Last sync: ${config.lastSyncAt || "never"}\n`);
374
361
  process.stdout.write(`Last full reconciliation: ${config.lastFullSyncAt || "never"}\n`);
362
+ if (config.pendingSync) process.stdout.write(`Pending sync: ${config.pendingSync.runId}; rerun sync to resume\n`);
375
363
  process.stdout.write(`Profile: ${config.profileUrl || "https://usagemax.com/account"}\n`);
376
364
  }
377
365
 
@@ -458,12 +446,16 @@ async function main() {
458
446
  const command = args[0] || "sync";
459
447
  if (["--help", "-h", "help"].includes(command)) return help();
460
448
  if (["--version", "-v"].includes(command)) return process.stdout.write(`${VERSION}\n`);
461
- if (command === "link") return link(args.slice(1));
462
- if (command === "sync") return sync(args.slice(1));
463
- if (command === "status") return status();
464
- if (command === "doctor") return doctor(args.slice(1));
465
449
  if (command === "report") return report(args.slice(1));
466
- if (command === "unlink") return removeLink(args.slice(1));
450
+ if (["link", "sync", "status", "doctor", "unlink"].includes(command)) {
451
+ return withConfigLock(configDirectory(), async () => {
452
+ if (command === "link") return link(args.slice(1));
453
+ if (command === "sync") return sync(args.slice(1));
454
+ if (command === "status") return status();
455
+ if (command === "doctor") return doctor(args.slice(1));
456
+ return removeLink(args.slice(1));
457
+ });
458
+ }
467
459
  throw new Error(`Unknown command: ${command}. Run usagemax --help.`);
468
460
  }
469
461
 
package/src/core.js CHANGED
@@ -262,7 +262,7 @@ export function buildDeltaPlan(report, priorSnapshots, deviceId, pricingVersion
262
262
  pricingVersion,
263
263
  status: "ok",
264
264
  state: "synced",
265
- occurredAt: `${row.period}T12:00:00.000Z`,
265
+ occurredAt: `${row.period}T00:00:00.000Z`,
266
266
  completeness: "estimated",
267
267
  },
268
268
  });
@@ -300,7 +300,9 @@ export function buildSnapshotPlan(report, priorSnapshots, {
300
300
  }
301
301
 
302
302
  const grouped = new Map();
303
- const nextSnapshots = {};
303
+ // Carry forward history outside this scan's window. Only reconciled keys may
304
+ // replace or remove a checkpoint, including legacy three-part identities.
305
+ const nextSnapshots = Object.fromEntries([...priorRows].map(([key, row]) => [key, row.current]));
304
306
  const regressions = [];
305
307
  for (const key of allKeys) {
306
308
  const currentRow = rows.get(key);
@@ -308,9 +310,14 @@ export function buildSnapshotPlan(report, priorSnapshots, {
308
310
  const identity = currentRow || priorRow;
309
311
  if (!identity) continue;
310
312
  const previous = priorRow?.current ?? snapshotCounters();
311
- const current = currentRow?.current ?? snapshotCounters();
312
- if (snapshotCounterFields.some((field) => current[field] < previous[field])) regressions.push(key);
313
+ let current = currentRow?.current ?? snapshotCounters();
314
+ const regressed = snapshotCounterFields.some((field) => current[field] < previous[field]);
315
+ if (regressed) regressions.push(key);
316
+ // A larger total can conceal a missing dimension. Keep the coherent prior
317
+ // vector (including in the checkpoint) until coverage is authoritative.
318
+ if (!complete && regressed) current = { ...previous };
313
319
  if (!sameCounters(current, snapshotCounters())) nextSnapshots[key] = current;
320
+ else delete nextSnapshots[key];
314
321
  const partitionKey = `${identity.source}\u001f${identity.period}`;
315
322
  const rowsForPartition = grouped.get(partitionKey) ?? [];
316
323
  rowsForPartition.push({
@@ -321,7 +328,9 @@ export function buildSnapshotPlan(report, priorSnapshots, {
321
328
  current,
322
329
  costBasis: "estimated",
323
330
  contentHash: sha256(`${key}\u001f${JSON.stringify(current)}`),
324
- lastUsedAt: Date.parse(`${identity.period}T12:00:00.000Z`),
331
+ // A daily aggregate has no exact event time. Use the start of its UTC day
332
+ // so today's partition is valid even when the collector runs before noon.
333
+ lastUsedAt: Date.parse(`${identity.period}T00:00:00.000Z`),
325
334
  });
326
335
  grouped.set(partitionKey, rowsForPartition);
327
336
  }
@@ -341,17 +350,23 @@ export function buildSnapshotPlan(report, priorSnapshots, {
341
350
  contentHash,
342
351
  lastUsedAt,
343
352
  }));
344
- const payloadHash = sha256(JSON.stringify({ source, day, complete, pricingVersion, rows: wireRows }));
345
- partitions.push({
346
- partitionId: `${runId}:${sha256(partitionKey).slice(0, 24)}`,
347
- payloadHash,
348
- revision,
349
- source,
350
- day,
351
- complete,
352
- pricingVersion,
353
- rows: wireRows,
354
- });
353
+ const chunkCount = Math.ceil(wireRows.length / 100);
354
+ for (let chunkIndex = 0; chunkIndex < chunkCount; chunkIndex += 1) {
355
+ const rows = wireRows.slice(chunkIndex * 100, (chunkIndex + 1) * 100);
356
+ const payloadHash = sha256(JSON.stringify({ source, day, complete, pricingVersion, chunkIndex, chunkCount, rows }));
357
+ partitions.push({
358
+ partitionId: `${runId}:${sha256(partitionKey).slice(0, 24)}:${chunkIndex}`,
359
+ payloadHash,
360
+ revision,
361
+ source,
362
+ day,
363
+ complete,
364
+ pricingVersion,
365
+ chunkIndex,
366
+ chunkCount,
367
+ rows,
368
+ });
369
+ }
355
370
  }
356
371
  partitions.sort((left, right) => left.day === right.day ? left.source.localeCompare(right.source) : left.day.localeCompare(right.day));
357
372
  return { partitions, nextSnapshots, regressions };
@@ -361,6 +376,22 @@ function randomPlanId() {
361
376
  return `run-${Date.now()}-${Math.random().toString(36).slice(2, 10)}`;
362
377
  }
363
378
 
379
+ // Inventory stability certifies that repeating a successful parse is unnecessary;
380
+ // it does not certify authority to remove server history. Bootstrap is separate.
381
+ export function scanPolicy(config, inventory, { today, now, inventoryVersion, requestedFull = false, requestedArchives = false }) {
382
+ const knownSources = Array.isArray(config.knownSources) ? config.knownSources : [];
383
+ const lastFull = Date.parse(config.lastFullSyncAt || "");
384
+ const bootstrap = config.snapshotProtocolVersion !== 2;
385
+ const full = requestedFull || requestedArchives || bootstrap
386
+ || config.sourceInventoryVersion !== inventoryVersion
387
+ || !Number.isFinite(lastFull) || now - lastFull >= 7 * 24 * 60 * 60 * 1000
388
+ || inventory.sources.some((source) => !knownSources.includes(source));
389
+ const inventoryStable = inventory.complete && !inventory.truncated && inventory.errors === 0;
390
+ const skip = !config.pendingSync && !full && inventoryStable && config.lastScanSucceeded === true
391
+ && config.lastReconciledDay === today && config.sourceFingerprint === inventory.fingerprint;
392
+ return { bootstrap, full, skip, inventoryStable };
393
+ }
394
+
364
395
  export function buildSessionPlan(report, deviceId) {
365
396
  const sessionRows = Array.isArray(report?.session) ? report.session : [];
366
397
  const sessions = new Map();
@@ -396,3 +427,11 @@ export function sourceSummary(report) {
396
427
  });
397
428
  return [...new Set(sources)].sort();
398
429
  }
430
+ // Explicit UTC dates work with ccusage's combined daily/session sections.
431
+ export function reportDateArgs(config, { full = false, now = Date.now() } = {}) {
432
+ const today = new Date(now).toISOString().slice(0, 10);
433
+ const since = full || !config?.lastSyncAt ? "2024-01-01"
434
+ : config.lastReconciledDay === today ? today
435
+ : new Date(Date.parse(today + "T00:00:00Z") - 86_400_000).toISOString().slice(0, 10);
436
+ return ["--since", since, "--until", today];
437
+ }
package/src/resume.js ADDED
@@ -0,0 +1,58 @@
1
+ // A pending run is saved before network I/O. Persisting the acknowledged cursor
2
+ // after each request permits replays when the response or local write is lost.
3
+ // The server must receipt begin, chunks, sessions and complete idempotently.
4
+ export async function resumeUpload(config, { save, request, warn = () => {} }) {
5
+ const pending = config.pendingSync;
6
+ if (!pending || pending.version !== 1 || !Array.isArray(pending.requests)) {
7
+ throw new Error("Invalid saved sync; preserve the config for recovery.");
8
+ }
9
+ try {
10
+ for (let index = pending.cursor; index < pending.requests.length; index += 1) {
11
+ const { operation, payload } = pending.requests[index];
12
+ const response = await request(config, operation, payload, operation === "partitions" ? 60_000 : 30_000);
13
+ warn(response);
14
+ pending.cursor = index + 1;
15
+ await save(config);
16
+ }
17
+ // Commit local baseline and remove the journal in the same atomic write.
18
+ const next = { ...config, ...pending.checkpoint };
19
+ delete next.pendingSync;
20
+ await save(next);
21
+ Object.assign(config, next);
22
+ delete config.pendingSync;
23
+ return { ...pending.result, resumed: true };
24
+ } catch (error) {
25
+ if (error?.code === "snapshot_run_expired") {
26
+ pending.terminalError = "snapshot_run_expired";
27
+ await save(config);
28
+ throw new Error("Saved upload expired. Run usagemax sync --restart to rescan retained history while preserving the last committed checkpoint. Already accepted usage remains on the server.", { cause: error });
29
+ }
30
+ throw new Error(`${error instanceof Error ? error.message : "Upload failed."} Run usagemax sync again to resume saved run ${pending.runId}; its payload and checkpoints have been retained.`, { cause: error });
31
+ }
32
+ }
33
+
34
+ export function restartExpiredUpload(config) {
35
+ if (config.pendingSync?.terminalError !== "snapshot_run_expired") throw new Error("Only an expired upload can restart. Run sync normally to resume or verify its status first.");
36
+ delete config.pendingSync;
37
+ config.lastFullSyncAt = undefined;
38
+ config.sourceFingerprint = null;
39
+ }
40
+ import { mkdir, readFile, unlink, writeFile } from "node:fs/promises";
41
+ import { join } from "node:path";
42
+
43
+ export async function withConfigLock(directory, action) {
44
+ await mkdir(directory, { recursive: true, mode: 0o700 });
45
+ const path = join(directory, "collector.lock");
46
+ try {
47
+ await writeFile(path, `${process.pid}\n`, { flag: "wx", mode: 0o600 });
48
+ } catch (error) {
49
+ if (error.code !== "EEXIST") throw error;
50
+ const owner = (await readFile(path, "utf8").catch(() => "unknown")).trim();
51
+ throw new Error(`Collector config is locked by PID ${/^\d+$/.test(owner) ? owner : "unknown"}. If that process has exited, remove only ${path} and rerun sync; keep config.json for resume.`);
52
+ }
53
+ try {
54
+ return await action();
55
+ } finally {
56
+ await unlink(path);
57
+ }
58
+ }
@@ -0,0 +1,54 @@
1
+ import { setTimeout as delay } from "node:timers/promises";
2
+
3
+ export function retryAfterMs(value, now = Date.now()) {
4
+ if (value == null || value === "") return 0;
5
+ if (/^\d+(?:\.\d+)?$/.test(value)) return Number(value) * 1000;
6
+ const date = Date.parse(value);
7
+ return Number.isFinite(date) ? Math.max(0, date - now) : 0;
8
+ }
9
+
10
+ // Only snapshot operations have server receipts. Never automatically replay a
11
+ // one-use link request or apply this policy to arbitrary POST operations.
12
+ export async function requestSnapshot(endpoint, config, operation, payload, {
13
+ timeout = 30_000, attempts = 5, maxDelayMs = 60_000,
14
+ fetchImpl = fetch, sleep = delay, random = Math.random, now = Date.now,
15
+ } = {}) {
16
+ const body = JSON.stringify({ operation, ...payload });
17
+ for (let attempt = 0; attempt < attempts; attempt += 1) {
18
+ let response;
19
+ let result;
20
+ let failure;
21
+ let retryable = true;
22
+ try {
23
+ response = await fetchImpl(endpoint, {
24
+ method: "POST",
25
+ headers: { authorization: `Bearer ${config.token}`, "content-type": "application/json", "x-usagemax-device-id": config.deviceId },
26
+ body,
27
+ signal: AbortSignal.timeout(timeout),
28
+ });
29
+ result = await response.json().catch(() => null);
30
+ if (response.ok && result?.ok === true) return result;
31
+ retryable = response.ok || response.status === 429 || response.status >= 500
32
+ || (operation === "complete" && response.status === 409
33
+ && String(result?.error).toLowerCase() === "snapshot_run_incomplete");
34
+ const code = typeof result?.error === "string" ? result.error.replace(/[^a-zA-Z0-9_-]/g, "").slice(0, 80) : "invalid_response";
35
+ const advice = response.status === 401 ? " Link this computer again."
36
+ : response.status === 403 ? " Check workspace membership and collector permissions."
37
+ : response.status === 409 ? " The saved run needs reconciliation; do not delete its checkpoint or repeatedly start new runs."
38
+ : response.status === 400 ? " Check CLI/server protocol compatibility."
39
+ : "";
40
+ failure = new Error(`Snapshot ${operation} failed (HTTP ${response.status}: ${code}).${advice}`);
41
+ failure.code = code;
42
+ } catch {
43
+ failure = new Error(`Snapshot ${operation} could not reach UsageMax or timed out. Check your network connection.`);
44
+ }
45
+ if (!retryable || attempt + 1 >= attempts) throw failure;
46
+ const retryAfter = retryAfterMs(response?.headers?.get("retry-after"), now());
47
+ // Never retry earlier than the server's requested interval. Long backoffs
48
+ // remain durably resumable instead of blocking the command indefinitely.
49
+ if (retryAfter > maxDelayMs) throw new Error(`${failure.message} Server requested a longer wait; retry sync after ${Math.ceil(retryAfter / 1000)} seconds.`);
50
+ const jitter = Math.min(maxDelayMs, 1000 * 2 ** attempt) * (0.5 + random() * 0.5);
51
+ await sleep(Math.max(retryAfter, jitter));
52
+ }
53
+ throw new Error("Snapshot retry budget is empty.");
54
+ }