usagemax 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +51 -9
- package/package.json +3 -1
- package/src/cli.js +76 -84
- package/src/core.js +55 -16
- package/src/resume.js +58 -0
- package/src/transport.js +54 -0
package/README.md
CHANGED
|
@@ -78,21 +78,31 @@ connector. UsageMax never invents usage that the source did not retain.
|
|
|
78
78
|
|
|
79
79
|
## Privacy, correctness, and load
|
|
80
80
|
|
|
81
|
-
|
|
81
|
+
Interrupted uploads resume with `bunx usagemax sync`. Server upload runs expire
|
|
82
|
+
after 30 idle days. If the server reports an expired run, use `sync --restart`:
|
|
83
|
+
this preserves the last committed checkpoint and performs a fresh full scan.
|
|
84
|
+
It never automatically replays an old authoritative deletion against newer data.
|
|
85
|
+
|
|
86
|
+
- The sync payload contains aggregate token counts,
|
|
82
87
|
model/provider names, costs, source names, dates, coverage state, and opaque
|
|
83
88
|
SHA-256 session identities.
|
|
84
89
|
- It does not upload prompts, completions, source code, file contents, project
|
|
85
90
|
paths, or provider credentials.
|
|
86
91
|
- Sync is one-shot. There is no resident scanner or high-frequency polling loop.
|
|
87
|
-
- A
|
|
88
|
-
|
|
92
|
+
- A successful previous parse with a complete, unchanged metadata inventory skips
|
|
93
|
+
parsing and uploading again that day. Inventory stability is independent of
|
|
94
|
+
deletion authority; a no-change result retains the previous partial coverage
|
|
95
|
+
label. New sources, date rollover, explicit full/archive requests and weekly
|
|
96
|
+
reconciliation still trigger the appropriate scan.
|
|
89
97
|
- Normal changed syncs parse today or today plus yesterday. A bounded weekly
|
|
90
98
|
full reconciliation catches restored files, parser changes, and older logs.
|
|
91
|
-
- Full history
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
99
|
+
- Full history scans retained local history from 2024 onward. Inventory success
|
|
100
|
+
does not prove every file parsed. Until the parser certifies source/day coverage,
|
|
101
|
+
uploads are marked partial and any row with a decreasing counter retains its
|
|
102
|
+
entire previous vector. This includes explicit zeros and missing sources.
|
|
103
|
+
Older checkpoints survive incremental windows. Authoritative corrections remain
|
|
104
|
+
supported by the planner but are not claimed by this parser integration.
|
|
105
|
+
Deleted or never-persisted usage requires a provider export.
|
|
96
106
|
- Source totals that cannot be assigned to a model are retained as
|
|
97
107
|
`unattributed` rather than silently discarded.
|
|
98
108
|
- The collector key is written with user-only permissions where the operating
|
|
@@ -103,7 +113,8 @@ connector. UsageMax never invents usage that the source did not retain.
|
|
|
103
113
|
its hash and applies device binding, replay checks, payload caps, and quotas.
|
|
104
114
|
- A private random installation ID survives collector rotation, relinking, and
|
|
105
115
|
display-name changes. It is not a hardware fingerprint; the server stores only
|
|
106
|
-
its SHA-256 hash.
|
|
116
|
+
its SHA-256 hash. A local config lock prevents overlapping commands from
|
|
117
|
+
overwriting pending runs; server receipts make repeated uploads idempotent.
|
|
107
118
|
- The server, not the local checkpoint, owns the accounting baseline. A lost
|
|
108
119
|
response or interrupted run can be retried without adding the same partition twice.
|
|
109
120
|
- Do not point two different installations at the same copied or network-mounted
|
|
@@ -113,5 +124,36 @@ connector. UsageMax never invents usage that the source did not retain.
|
|
|
113
124
|
Use `USAGEMAX_CONFIG_DIR` to select another config directory. Development and
|
|
114
125
|
self-hosted installations may set `USAGEMAX_LINK_ENDPOINT` before linking.
|
|
115
126
|
|
|
127
|
+
## Interrupted uploads and protocol 0.3.1
|
|
128
|
+
|
|
129
|
+
Before uploading, the CLI saves the exact run, ordered request payloads and next
|
|
130
|
+
checkpoint in the private config. `sync` resumes this journal before scanning new
|
|
131
|
+
data. Local snapshots advance only after completion is acknowledged. A resumed
|
|
132
|
+
command finishes the saved scan; run `sync` again to collect subsequent changes.
|
|
133
|
+
`sync --dry-run` reports pending work without uploading. Do not delete config.json
|
|
134
|
+
to retry a failed upload. A successful relink or unlink replaces/removes the local
|
|
135
|
+
journal along with its credential.
|
|
136
|
+
|
|
137
|
+
Network failures, malformed success responses, HTTP 429 and 5xx get at most five
|
|
138
|
+
attempts per request with exponential jitter. Retry-After seconds and HTTP dates
|
|
139
|
+
are honored up to 60 seconds; longer waits stop with an instruction to retry later.
|
|
140
|
+
Only completion's `snapshot_run_incomplete` HTTP 409 is retried, because server
|
|
141
|
+
cleanup can still be pending. Authentication, validation and other conflicts fail
|
|
142
|
+
with an actionable message and retain the journal. JSON `accepted` is null for
|
|
143
|
+
uploads because lost responses/replays cannot reliably reconstruct that count;
|
|
144
|
+
`changedRows` is the local planned count, not a server accounting receipt.
|
|
145
|
+
|
|
146
|
+
Each sorted source/day is sent as ordered chunks of at most 100 rows. Each has a
|
|
147
|
+
unique partitionId, zero-based chunkIndex and shared chunkCount. The payload hash
|
|
148
|
+
covers `{source, day, complete, pricingVersion, chunkIndex, chunkCount, rows}` in
|
|
149
|
+
that order. `partitionCount` counts transmitted chunks. The server must accept
|
|
150
|
+
this protocol, receipt replays, and finish omitted-row cleanup before completing
|
|
151
|
+
an authoritative run.
|
|
152
|
+
|
|
153
|
+
An abrupt process kill can leave `collector.lock` in the config directory. The
|
|
154
|
+
next command reports the owner PID and exact path. Confirm that process has exited
|
|
155
|
+
before removing only that lock file, then rerun sync. Never remove an active lock
|
|
156
|
+
or the saved config to recover. Normal completion and handled failures release it.
|
|
157
|
+
|
|
116
158
|
See the [collector coverage audit](../../docs/collector-coverage-audit.md) for
|
|
117
159
|
the full support matrix and known boundaries.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "usagemax",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.2",
|
|
4
4
|
"description": "Link local coding-agent usage to your UsageMax profile",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "UsageMax",
|
|
@@ -23,6 +23,8 @@
|
|
|
23
23
|
"src/core.js",
|
|
24
24
|
"src/installation.js",
|
|
25
25
|
"src/sources.js",
|
|
26
|
+
"src/transport.js",
|
|
27
|
+
"src/resume.js",
|
|
26
28
|
"README.md",
|
|
27
29
|
"LICENSE"
|
|
28
30
|
],
|
package/src/cli.js
CHANGED
|
@@ -10,18 +10,19 @@ import process from "node:process";
|
|
|
10
10
|
import { promisify } from "node:util";
|
|
11
11
|
|
|
12
12
|
import { prepareArchiveRecovery } from "./archives.js";
|
|
13
|
-
import { buildSessionPlan, buildSnapshotPlan, normalizeLinkCode, sourceSummary, validHttpsUrl } from "./core.js";
|
|
13
|
+
import { buildSessionPlan, buildSnapshotPlan, normalizeLinkCode, reportDateArgs, scanPolicy, sourceSummary, validHttpsUrl } from "./core.js";
|
|
14
14
|
import { stableInstallationId } from "./installation.js";
|
|
15
|
+
import { requestSnapshot } from "./transport.js";
|
|
16
|
+
import { resumeUpload, restartExpiredUpload, withConfigLock } from "./resume.js";
|
|
15
17
|
import { CCUSAGE_VERSION, ccusageEnvironment, ccusageHome, discoverProviderArchives, SOURCE_INVENTORY_VERSION, sourceInventory, SUPPORTED_SOURCES } from "./sources.js";
|
|
16
18
|
|
|
17
19
|
const require = createRequire(import.meta.url);
|
|
18
20
|
const executeFile = promisify(execFile);
|
|
19
|
-
const VERSION = "0.3.
|
|
21
|
+
const VERSION = "0.3.2";
|
|
20
22
|
const PUBLIC_API_ORIGIN = "https://usagemax.com/api";
|
|
21
23
|
const DEFAULT_LINK_ENDPOINT = `${PUBLIC_API_ORIGIN}/v1/devices/link`;
|
|
22
24
|
const CONFIG_FILE = "config.json";
|
|
23
25
|
const MAX_REPORT_BYTES = 100 * 1024 * 1024;
|
|
24
|
-
const FULL_RECONCILE_INTERVAL_MS = 7 * 24 * 60 * 60 * 1000;
|
|
25
26
|
|
|
26
27
|
function configDirectory() {
|
|
27
28
|
if (process.env.USAGEMAX_CONFIG_DIR) return process.env.USAGEMAX_CONFIG_DIR;
|
|
@@ -103,7 +104,7 @@ function help() {
|
|
|
103
104
|
process.stdout.write(" usagemax Sync changed local usage\n");
|
|
104
105
|
process.stdout.write(" usagemax link <one-use-code> Link and sync this computer\n");
|
|
105
106
|
process.stdout.write(" [--no-sync] [--name <name>]\n");
|
|
106
|
-
process.stdout.write(" usagemax sync [--full] [--archives] [--dry-run] [--explain] [--json]\n");
|
|
107
|
+
process.stdout.write(" usagemax sync [--full] [--archives] [--restart] [--dry-run] [--explain] [--json]\n");
|
|
107
108
|
process.stdout.write(" Reconcile once; --archives performs one-time recovery\n");
|
|
108
109
|
process.stdout.write(" usagemax status Show local link status\n");
|
|
109
110
|
process.stdout.write(" usagemax doctor [--deep] [--json]\n");
|
|
@@ -122,12 +123,7 @@ async function ccusageJson(config, { full = false, env } = {}) {
|
|
|
122
123
|
// Reconcile yesterday once after the UTC date changes. All other incremental
|
|
123
124
|
// scans parse only today; a metadata fingerprint avoids invoking ccusage when
|
|
124
125
|
// no supported local source changed at all.
|
|
125
|
-
|
|
126
|
-
args.push("--since", "2024-01-01", "--until", new Date().toISOString().slice(0, 10));
|
|
127
|
-
} else if (config?.lastSyncAt) {
|
|
128
|
-
const today = new Date().toISOString().slice(0, 10);
|
|
129
|
-
args.push("--last", config.lastReconciledDay === today ? "1" : "2");
|
|
130
|
-
}
|
|
126
|
+
args.push(...reportDateArgs(config, { full }));
|
|
131
127
|
const { stdout } = await executeFile(process.execPath, args, {
|
|
132
128
|
encoding: "utf8",
|
|
133
129
|
maxBuffer: MAX_REPORT_BYTES,
|
|
@@ -142,22 +138,7 @@ function snapshotEndpoint(config) {
|
|
|
142
138
|
}
|
|
143
139
|
|
|
144
140
|
async function snapshotRequest(config, operation, payload, timeout = 30_000) {
|
|
145
|
-
|
|
146
|
-
method: "POST",
|
|
147
|
-
headers: {
|
|
148
|
-
authorization: `Bearer ${config.token}`,
|
|
149
|
-
"content-type": "application/json",
|
|
150
|
-
"x-usagemax-device-id": config.deviceId,
|
|
151
|
-
},
|
|
152
|
-
body: JSON.stringify({ operation, ...payload }),
|
|
153
|
-
signal: AbortSignal.timeout(timeout),
|
|
154
|
-
});
|
|
155
|
-
const body = await response.json().catch(() => ({}));
|
|
156
|
-
if (!response.ok) {
|
|
157
|
-
if (body?.error === "unauthorized") throw new Error("This collector key is no longer valid. Link the computer again.");
|
|
158
|
-
throw new Error(`UsageMax rejected ${operation} (${response.status}${body?.error ? `: ${body.error}` : ""}).`);
|
|
159
|
-
}
|
|
160
|
-
return body;
|
|
141
|
+
return requestSnapshot(snapshotEndpoint(config), config, operation, payload, { timeout });
|
|
161
142
|
}
|
|
162
143
|
|
|
163
144
|
function newerVersion(recommended) {
|
|
@@ -243,32 +224,44 @@ async function syncPrepared(args, suppliedConfig, recovery) {
|
|
|
243
224
|
const dryRun = args.includes("--dry-run");
|
|
244
225
|
const explain = args.includes("--explain");
|
|
245
226
|
const json = args.includes("--json");
|
|
227
|
+
if (args.includes("--restart") && !dryRun) {
|
|
228
|
+
restartExpiredUpload(config);
|
|
229
|
+
await writeConfig(config);
|
|
230
|
+
}
|
|
231
|
+
if (config.pendingSync) {
|
|
232
|
+
if (dryRun) {
|
|
233
|
+
const result = { ...config.pendingSync.result, dryRun: true, pendingRunId: config.pendingSync.runId };
|
|
234
|
+
process.stdout.write(json ? `${JSON.stringify(result)}\n` : `Dry run: saved run ${config.pendingSync.runId} awaits resume; no upload.\n`);
|
|
235
|
+
return result;
|
|
236
|
+
}
|
|
237
|
+
const result = await resumeUpload(config, { save: writeConfig, request: snapshotRequest, warn: warnVersion });
|
|
238
|
+
process.stdout.write(json ? `${JSON.stringify(result)}\n` : "Resumed and completed the saved sync. Run sync again to scan newer local changes.\n");
|
|
239
|
+
return result;
|
|
240
|
+
}
|
|
246
241
|
const inventory = await sourceInventory({ env: recovery.env, home: ccusageHome(recovery.env) });
|
|
247
242
|
const today = new Date().toISOString().slice(0, 10);
|
|
248
243
|
const knownSources = Array.isArray(config.knownSources) ? config.knownSources : [];
|
|
249
|
-
const
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
||
|
|
254
|
-
|| Date.now() - lastFullSync >= FULL_RECONCILE_INTERVAL_MS
|
|
255
|
-
|| foundNewSource;
|
|
256
|
-
const full = requestedFull || requestedArchives || fullDue;
|
|
257
|
-
if (!full && inventory.complete && config.lastSyncComplete && config.lastReconciledDay === today && config.sourceFingerprint === inventory.fingerprint) {
|
|
258
|
-
const result = { accepted: 0, changedRows: 0, sessions: 0, sources: inventory.sources, corrections: 0, scanned: false, full: false, coverage: inventory.complete ? "complete" : "partial" };
|
|
244
|
+
const { bootstrap, full, skip, inventoryStable } = scanPolicy(config, inventory, {
|
|
245
|
+
today, now: Date.now(), inventoryVersion: SOURCE_INVENTORY_VERSION, requestedFull, requestedArchives,
|
|
246
|
+
});
|
|
247
|
+
if (skip) {
|
|
248
|
+
const result = { accepted: 0, changedRows: 0, sessions: 0, sources: inventory.sources, corrections: 0, scanned: false, full: false, coverage: config.lastCoverage || "partial" };
|
|
259
249
|
if (json) process.stdout.write(`${JSON.stringify(result)}\n`);
|
|
260
250
|
else process.stdout.write("Already up to date. Local usage files have not changed; no logs were parsed or uploaded.\n");
|
|
261
251
|
return;
|
|
262
252
|
}
|
|
263
253
|
const report = await ccusageJson(config, { env: recovery.env, full });
|
|
264
|
-
|
|
254
|
+
// ccusage v20 exposes aggregates, not proof that every discovered file was
|
|
255
|
+
// parsed. Inventory success alone cannot authorize destructive corrections.
|
|
256
|
+
const authoritative = false;
|
|
257
|
+
const legacySnapshotBootstrap = bootstrap
|
|
265
258
|
&& Object.keys(config.snapshots || {}).length > 0;
|
|
266
259
|
const runId = randomUUID();
|
|
267
260
|
const revision = Date.now();
|
|
268
261
|
const pricingVersion = `ccusage@${CCUSAGE_VERSION}`;
|
|
269
262
|
const { partitions, nextSnapshots, regressions } = buildSnapshotPlan(report, config.snapshots, {
|
|
270
|
-
bootstrap
|
|
271
|
-
complete:
|
|
263
|
+
bootstrap,
|
|
264
|
+
complete: authoritative,
|
|
272
265
|
full,
|
|
273
266
|
pricingVersion,
|
|
274
267
|
revision,
|
|
@@ -280,15 +273,17 @@ async function syncPrepared(args, suppliedConfig, recovery) {
|
|
|
280
273
|
? report.daily.map((row) => row?.period).filter((day) => /^\d{4}-\d{2}-\d{2}$/.test(day || "")).sort()
|
|
281
274
|
: [];
|
|
282
275
|
const result = {
|
|
283
|
-
accepted:
|
|
276
|
+
accepted: null,
|
|
284
277
|
changedRows: partitions.reduce((sum, partition) => sum + partition.rows.filter((row) => JSON.stringify(row.previous) !== JSON.stringify(row.current)).length, 0),
|
|
285
278
|
sessions: sessions.length,
|
|
286
279
|
sources,
|
|
287
|
-
corrections: regressions.length,
|
|
280
|
+
corrections: authoritative ? regressions.length : 0,
|
|
281
|
+
protectedRegressions: authoritative ? 0 : regressions.length,
|
|
288
282
|
partitions: partitions.length,
|
|
289
283
|
scanned: true,
|
|
290
284
|
full,
|
|
291
|
-
coverage:
|
|
285
|
+
coverage: authoritative ? "complete" : "partial",
|
|
286
|
+
coverageReason: "Parser does not certify complete source/day coverage; decreases and deletions are protected.",
|
|
292
287
|
range: { from: days[0], to: days.at(-1) },
|
|
293
288
|
};
|
|
294
289
|
if (requestedArchives) {
|
|
@@ -299,60 +294,52 @@ async function syncPrepared(args, suppliedConfig, recovery) {
|
|
|
299
294
|
if (json) process.stdout.write(`${JSON.stringify({ ...result, dryRun: true })}\n`);
|
|
300
295
|
else {
|
|
301
296
|
process.stdout.write(`Dry run: ${partitions.length} partition(s), ${result.changedRows} changed row(s), ${sessions.length} private session identifiers, no upload.\n`);
|
|
302
|
-
if (explain) process.stdout.write(`Coverage ${result.coverage}; ${sources.length} source(s); ${days[0] || "unknown"} to ${days.at(-1) || "unknown"}; ${regressions.length}
|
|
297
|
+
if (explain) process.stdout.write(`Coverage ${result.coverage}; ${sources.length} source(s); ${days[0] || "unknown"} to ${days.at(-1) || "unknown"}; ${regressions.length} protected regression(s). ${result.coverageReason}\n`);
|
|
303
298
|
}
|
|
304
299
|
return result;
|
|
305
300
|
}
|
|
306
|
-
|
|
307
|
-
await writeConfig(config);
|
|
308
|
-
try {
|
|
309
|
-
const begin = await snapshotRequest(config, "begin", {
|
|
301
|
+
const requests = [{ operation: "begin", payload: {
|
|
310
302
|
runId,
|
|
311
303
|
mode: requestedArchives ? "archives" : full ? "full" : "incremental",
|
|
312
304
|
baselineMode: legacySnapshotBootstrap ? "adopt-current" : "apply",
|
|
313
305
|
sourceCount: sources.length,
|
|
314
306
|
partitionCount: partitions.length,
|
|
315
|
-
inventoryComplete:
|
|
307
|
+
inventoryComplete: authoritative,
|
|
316
308
|
inventoryErrors: inventory.errors,
|
|
317
309
|
inventoryTruncated: inventory.truncated,
|
|
318
310
|
coverageStartDay: days[0],
|
|
319
311
|
coverageEndDay: days.at(-1),
|
|
320
|
-
}
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
await snapshotRequest(config, "sessions", { runId, sessions: sessions.slice(offset, offset + 100) });
|
|
324
|
-
}
|
|
325
|
-
for (let offset = 0; offset < partitions.length; offset += 10) {
|
|
326
|
-
const response = await snapshotRequest(config, "partitions", { runId, partitions: partitions.slice(offset, offset + 10) }, 60_000);
|
|
327
|
-
result.accepted += Number(response.changedRows || 0);
|
|
328
|
-
if (!json && process.stderr.isTTY && partitions.length > 10) {
|
|
329
|
-
process.stderr.write(`UsageMax: uploaded ${Math.min(offset + 10, partitions.length)}/${partitions.length} history partitions\r`);
|
|
330
|
-
}
|
|
331
|
-
}
|
|
332
|
-
if (!json && process.stderr.isTTY && partitions.length > 10) process.stderr.write("\n");
|
|
333
|
-
const completed = await snapshotRequest(config, "complete", { runId });
|
|
334
|
-
warnVersion(completed);
|
|
335
|
-
config.snapshots = nextSnapshots;
|
|
336
|
-
config.snapshotProtocolVersion = 2;
|
|
337
|
-
config.lastSyncAt = new Date().toISOString();
|
|
338
|
-
config.lastReconciledDay = today;
|
|
339
|
-
config.lastSyncComplete = true;
|
|
340
|
-
config.sourceInventoryVersion = SOURCE_INVENTORY_VERSION;
|
|
341
|
-
config.knownSources = [...new Set([...knownSources, ...inventory.sources, ...sources])].sort();
|
|
342
|
-
if (inventory.complete) config.sourceFingerprint = inventory.fingerprint;
|
|
343
|
-
else delete config.sourceFingerprint;
|
|
344
|
-
if (full) config.lastFullSyncAt = config.lastSyncAt;
|
|
345
|
-
await writeConfig(config);
|
|
346
|
-
} catch (error) {
|
|
347
|
-
await snapshotRequest(config, "fail", { runId, failureCode: error instanceof Error ? error.message.slice(0, 80) : "sync_failed" }).catch(() => undefined);
|
|
348
|
-
throw error;
|
|
312
|
+
} }];
|
|
313
|
+
for (let offset = 0; offset < sessions.length; offset += 100) {
|
|
314
|
+
requests.push({ operation: "sessions", payload: { runId, sessions: sessions.slice(offset, offset + 100) } });
|
|
349
315
|
}
|
|
316
|
+
for (let offset = 0; offset < partitions.length; offset += 10) {
|
|
317
|
+
requests.push({ operation: "partitions", payload: { runId, partitions: partitions.slice(offset, offset + 10) } });
|
|
318
|
+
}
|
|
319
|
+
requests.push({ operation: "complete", payload: { runId } });
|
|
320
|
+
const syncedAt = new Date().toISOString();
|
|
321
|
+
config.pendingSync = {
|
|
322
|
+
version: 1, runId, cursor: 0, requests, result,
|
|
323
|
+
checkpoint: {
|
|
324
|
+
snapshots: nextSnapshots, snapshotProtocolVersion: 2,
|
|
325
|
+
lastSyncAt: syncedAt, lastReconciledDay: today, lastSyncComplete: authoritative,
|
|
326
|
+
lastScanSucceeded: true, lastCoverage: result.coverage,
|
|
327
|
+
sourceInventoryVersion: SOURCE_INVENTORY_VERSION,
|
|
328
|
+
knownSources: [...new Set([...knownSources, ...inventory.sources, ...sources])].sort(),
|
|
329
|
+
sourceFingerprint: inventoryStable ? inventory.fingerprint : null,
|
|
330
|
+
...(full ? { lastFullSyncAt: syncedAt } : {}),
|
|
331
|
+
},
|
|
332
|
+
};
|
|
333
|
+
config.lastSyncComplete = false;
|
|
334
|
+
await writeConfig(config);
|
|
335
|
+
await resumeUpload(config, { save: writeConfig, request: snapshotRequest, warn: warnVersion });
|
|
350
336
|
if (json) process.stdout.write(`${JSON.stringify(result)}\n`);
|
|
351
337
|
else {
|
|
352
338
|
process.stdout.write(partitions.length || sessions.length
|
|
353
|
-
? `
|
|
339
|
+
? `Completed ${partitions.length} usage chunk(s) and ${sessions.length} session identifier(s) from ${sources.join(", ") || "local agents"}${full ? " across retained history" : ""}.\n`
|
|
354
340
|
: `Already up to date. No usage rows changed${full ? " after a full-history reconciliation" : ""}.\n`);
|
|
355
|
-
if (regressions.length) process.stdout.write(`${regressions.length} local row(s) moved backward
|
|
341
|
+
if (regressions.length) process.stdout.write(`${regressions.length} local row(s) moved backward; prior counter dimensions were preserved because coverage is incomplete.\n`);
|
|
342
|
+
if (!authoritative) process.stdout.write(`${result.coverageReason}\n`);
|
|
356
343
|
if (explain) process.stdout.write(`Coverage ${result.coverage}; ${sources.length} source(s); ${days[0] || "unknown"} to ${days.at(-1) || "unknown"}; ${partitions.length} atomic partition(s).\n`);
|
|
357
344
|
}
|
|
358
345
|
return result;
|
|
@@ -372,6 +359,7 @@ async function status() {
|
|
|
372
359
|
process.stdout.write(`Linked: ${config.deviceName || deviceLabel()}${config.profileHandle ? ` → @${config.profileHandle}` : ""}\n`);
|
|
373
360
|
process.stdout.write(`Last sync: ${config.lastSyncAt || "never"}\n`);
|
|
374
361
|
process.stdout.write(`Last full reconciliation: ${config.lastFullSyncAt || "never"}\n`);
|
|
362
|
+
if (config.pendingSync) process.stdout.write(`Pending sync: ${config.pendingSync.runId}; rerun sync to resume\n`);
|
|
375
363
|
process.stdout.write(`Profile: ${config.profileUrl || "https://usagemax.com/account"}\n`);
|
|
376
364
|
}
|
|
377
365
|
|
|
@@ -458,12 +446,16 @@ async function main() {
|
|
|
458
446
|
const command = args[0] || "sync";
|
|
459
447
|
if (["--help", "-h", "help"].includes(command)) return help();
|
|
460
448
|
if (["--version", "-v"].includes(command)) return process.stdout.write(`${VERSION}\n`);
|
|
461
|
-
if (command === "link") return link(args.slice(1));
|
|
462
|
-
if (command === "sync") return sync(args.slice(1));
|
|
463
|
-
if (command === "status") return status();
|
|
464
|
-
if (command === "doctor") return doctor(args.slice(1));
|
|
465
449
|
if (command === "report") return report(args.slice(1));
|
|
466
|
-
if (
|
|
450
|
+
if (["link", "sync", "status", "doctor", "unlink"].includes(command)) {
|
|
451
|
+
return withConfigLock(configDirectory(), async () => {
|
|
452
|
+
if (command === "link") return link(args.slice(1));
|
|
453
|
+
if (command === "sync") return sync(args.slice(1));
|
|
454
|
+
if (command === "status") return status();
|
|
455
|
+
if (command === "doctor") return doctor(args.slice(1));
|
|
456
|
+
return removeLink(args.slice(1));
|
|
457
|
+
});
|
|
458
|
+
}
|
|
467
459
|
throw new Error(`Unknown command: ${command}. Run usagemax --help.`);
|
|
468
460
|
}
|
|
469
461
|
|
package/src/core.js
CHANGED
|
@@ -262,7 +262,7 @@ export function buildDeltaPlan(report, priorSnapshots, deviceId, pricingVersion
|
|
|
262
262
|
pricingVersion,
|
|
263
263
|
status: "ok",
|
|
264
264
|
state: "synced",
|
|
265
|
-
occurredAt: `${row.period}
|
|
265
|
+
occurredAt: `${row.period}T00:00:00.000Z`,
|
|
266
266
|
completeness: "estimated",
|
|
267
267
|
},
|
|
268
268
|
});
|
|
@@ -300,7 +300,9 @@ export function buildSnapshotPlan(report, priorSnapshots, {
|
|
|
300
300
|
}
|
|
301
301
|
|
|
302
302
|
const grouped = new Map();
|
|
303
|
-
|
|
303
|
+
// Carry forward history outside this scan's window. Only reconciled keys may
|
|
304
|
+
// replace or remove a checkpoint, including legacy three-part identities.
|
|
305
|
+
const nextSnapshots = Object.fromEntries([...priorRows].map(([key, row]) => [key, row.current]));
|
|
304
306
|
const regressions = [];
|
|
305
307
|
for (const key of allKeys) {
|
|
306
308
|
const currentRow = rows.get(key);
|
|
@@ -308,9 +310,14 @@ export function buildSnapshotPlan(report, priorSnapshots, {
|
|
|
308
310
|
const identity = currentRow || priorRow;
|
|
309
311
|
if (!identity) continue;
|
|
310
312
|
const previous = priorRow?.current ?? snapshotCounters();
|
|
311
|
-
|
|
312
|
-
|
|
313
|
+
let current = currentRow?.current ?? snapshotCounters();
|
|
314
|
+
const regressed = snapshotCounterFields.some((field) => current[field] < previous[field]);
|
|
315
|
+
if (regressed) regressions.push(key);
|
|
316
|
+
// A larger total can conceal a missing dimension. Keep the coherent prior
|
|
317
|
+
// vector (including in the checkpoint) until coverage is authoritative.
|
|
318
|
+
if (!complete && regressed) current = { ...previous };
|
|
313
319
|
if (!sameCounters(current, snapshotCounters())) nextSnapshots[key] = current;
|
|
320
|
+
else delete nextSnapshots[key];
|
|
314
321
|
const partitionKey = `${identity.source}\u001f${identity.period}`;
|
|
315
322
|
const rowsForPartition = grouped.get(partitionKey) ?? [];
|
|
316
323
|
rowsForPartition.push({
|
|
@@ -321,7 +328,9 @@ export function buildSnapshotPlan(report, priorSnapshots, {
|
|
|
321
328
|
current,
|
|
322
329
|
costBasis: "estimated",
|
|
323
330
|
contentHash: sha256(`${key}\u001f${JSON.stringify(current)}`),
|
|
324
|
-
|
|
331
|
+
// A daily aggregate has no exact event time. Use the start of its UTC day
|
|
332
|
+
// so today's partition is valid even when the collector runs before noon.
|
|
333
|
+
lastUsedAt: Date.parse(`${identity.period}T00:00:00.000Z`),
|
|
325
334
|
});
|
|
326
335
|
grouped.set(partitionKey, rowsForPartition);
|
|
327
336
|
}
|
|
@@ -341,17 +350,23 @@ export function buildSnapshotPlan(report, priorSnapshots, {
|
|
|
341
350
|
contentHash,
|
|
342
351
|
lastUsedAt,
|
|
343
352
|
}));
|
|
344
|
-
const
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
payloadHash,
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
353
|
+
const chunkCount = Math.ceil(wireRows.length / 100);
|
|
354
|
+
for (let chunkIndex = 0; chunkIndex < chunkCount; chunkIndex += 1) {
|
|
355
|
+
const rows = wireRows.slice(chunkIndex * 100, (chunkIndex + 1) * 100);
|
|
356
|
+
const payloadHash = sha256(JSON.stringify({ source, day, complete, pricingVersion, chunkIndex, chunkCount, rows }));
|
|
357
|
+
partitions.push({
|
|
358
|
+
partitionId: `${runId}:${sha256(partitionKey).slice(0, 24)}:${chunkIndex}`,
|
|
359
|
+
payloadHash,
|
|
360
|
+
revision,
|
|
361
|
+
source,
|
|
362
|
+
day,
|
|
363
|
+
complete,
|
|
364
|
+
pricingVersion,
|
|
365
|
+
chunkIndex,
|
|
366
|
+
chunkCount,
|
|
367
|
+
rows,
|
|
368
|
+
});
|
|
369
|
+
}
|
|
355
370
|
}
|
|
356
371
|
partitions.sort((left, right) => left.day === right.day ? left.source.localeCompare(right.source) : left.day.localeCompare(right.day));
|
|
357
372
|
return { partitions, nextSnapshots, regressions };
|
|
@@ -361,6 +376,22 @@ function randomPlanId() {
|
|
|
361
376
|
return `run-${Date.now()}-${Math.random().toString(36).slice(2, 10)}`;
|
|
362
377
|
}
|
|
363
378
|
|
|
379
|
+
// Inventory stability certifies that repeating a successful parse is unnecessary;
|
|
380
|
+
// it does not certify authority to remove server history. Bootstrap is separate.
|
|
381
|
+
export function scanPolicy(config, inventory, { today, now, inventoryVersion, requestedFull = false, requestedArchives = false }) {
|
|
382
|
+
const knownSources = Array.isArray(config.knownSources) ? config.knownSources : [];
|
|
383
|
+
const lastFull = Date.parse(config.lastFullSyncAt || "");
|
|
384
|
+
const bootstrap = config.snapshotProtocolVersion !== 2;
|
|
385
|
+
const full = requestedFull || requestedArchives || bootstrap
|
|
386
|
+
|| config.sourceInventoryVersion !== inventoryVersion
|
|
387
|
+
|| !Number.isFinite(lastFull) || now - lastFull >= 7 * 24 * 60 * 60 * 1000
|
|
388
|
+
|| inventory.sources.some((source) => !knownSources.includes(source));
|
|
389
|
+
const inventoryStable = inventory.complete && !inventory.truncated && inventory.errors === 0;
|
|
390
|
+
const skip = !config.pendingSync && !full && inventoryStable && config.lastScanSucceeded === true
|
|
391
|
+
&& config.lastReconciledDay === today && config.sourceFingerprint === inventory.fingerprint;
|
|
392
|
+
return { bootstrap, full, skip, inventoryStable };
|
|
393
|
+
}
|
|
394
|
+
|
|
364
395
|
export function buildSessionPlan(report, deviceId) {
|
|
365
396
|
const sessionRows = Array.isArray(report?.session) ? report.session : [];
|
|
366
397
|
const sessions = new Map();
|
|
@@ -396,3 +427,11 @@ export function sourceSummary(report) {
|
|
|
396
427
|
});
|
|
397
428
|
return [...new Set(sources)].sort();
|
|
398
429
|
}
|
|
430
|
+
// Explicit UTC dates work with ccusage's combined daily/session sections.
|
|
431
|
+
export function reportDateArgs(config, { full = false, now = Date.now() } = {}) {
|
|
432
|
+
const today = new Date(now).toISOString().slice(0, 10);
|
|
433
|
+
const since = full || !config?.lastSyncAt ? "2024-01-01"
|
|
434
|
+
: config.lastReconciledDay === today ? today
|
|
435
|
+
: new Date(Date.parse(today + "T00:00:00Z") - 86_400_000).toISOString().slice(0, 10);
|
|
436
|
+
return ["--since", since, "--until", today];
|
|
437
|
+
}
|
package/src/resume.js
ADDED
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
// A pending run is saved before network I/O. Persisting the acknowledged cursor
|
|
2
|
+
// after each request permits replays when the response or local write is lost.
|
|
3
|
+
// The server must receipt begin, chunks, sessions and complete idempotently.
|
|
4
|
+
export async function resumeUpload(config, { save, request, warn = () => {} }) {
|
|
5
|
+
const pending = config.pendingSync;
|
|
6
|
+
if (!pending || pending.version !== 1 || !Array.isArray(pending.requests)) {
|
|
7
|
+
throw new Error("Invalid saved sync; preserve the config for recovery.");
|
|
8
|
+
}
|
|
9
|
+
try {
|
|
10
|
+
for (let index = pending.cursor; index < pending.requests.length; index += 1) {
|
|
11
|
+
const { operation, payload } = pending.requests[index];
|
|
12
|
+
const response = await request(config, operation, payload, operation === "partitions" ? 60_000 : 30_000);
|
|
13
|
+
warn(response);
|
|
14
|
+
pending.cursor = index + 1;
|
|
15
|
+
await save(config);
|
|
16
|
+
}
|
|
17
|
+
// Commit local baseline and remove the journal in the same atomic write.
|
|
18
|
+
const next = { ...config, ...pending.checkpoint };
|
|
19
|
+
delete next.pendingSync;
|
|
20
|
+
await save(next);
|
|
21
|
+
Object.assign(config, next);
|
|
22
|
+
delete config.pendingSync;
|
|
23
|
+
return { ...pending.result, resumed: true };
|
|
24
|
+
} catch (error) {
|
|
25
|
+
if (error?.code === "snapshot_run_expired") {
|
|
26
|
+
pending.terminalError = "snapshot_run_expired";
|
|
27
|
+
await save(config);
|
|
28
|
+
throw new Error("Saved upload expired. Run usagemax sync --restart to rescan retained history while preserving the last committed checkpoint. Already accepted usage remains on the server.", { cause: error });
|
|
29
|
+
}
|
|
30
|
+
throw new Error(`${error instanceof Error ? error.message : "Upload failed."} Run usagemax sync again to resume saved run ${pending.runId}; its payload and checkpoints have been retained.`, { cause: error });
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
export function restartExpiredUpload(config) {
|
|
35
|
+
if (config.pendingSync?.terminalError !== "snapshot_run_expired") throw new Error("Only an expired upload can restart. Run sync normally to resume or verify its status first.");
|
|
36
|
+
delete config.pendingSync;
|
|
37
|
+
config.lastFullSyncAt = undefined;
|
|
38
|
+
config.sourceFingerprint = null;
|
|
39
|
+
}
|
|
40
|
+
import { mkdir, readFile, unlink, writeFile } from "node:fs/promises";
|
|
41
|
+
import { join } from "node:path";
|
|
42
|
+
|
|
43
|
+
export async function withConfigLock(directory, action) {
|
|
44
|
+
await mkdir(directory, { recursive: true, mode: 0o700 });
|
|
45
|
+
const path = join(directory, "collector.lock");
|
|
46
|
+
try {
|
|
47
|
+
await writeFile(path, `${process.pid}\n`, { flag: "wx", mode: 0o600 });
|
|
48
|
+
} catch (error) {
|
|
49
|
+
if (error.code !== "EEXIST") throw error;
|
|
50
|
+
const owner = (await readFile(path, "utf8").catch(() => "unknown")).trim();
|
|
51
|
+
throw new Error(`Collector config is locked by PID ${/^\d+$/.test(owner) ? owner : "unknown"}. If that process has exited, remove only ${path} and rerun sync; keep config.json for resume.`);
|
|
52
|
+
}
|
|
53
|
+
try {
|
|
54
|
+
return await action();
|
|
55
|
+
} finally {
|
|
56
|
+
await unlink(path);
|
|
57
|
+
}
|
|
58
|
+
}
|
package/src/transport.js
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
import { setTimeout as delay } from "node:timers/promises";
|
|
2
|
+
|
|
3
|
+
export function retryAfterMs(value, now = Date.now()) {
|
|
4
|
+
if (value == null || value === "") return 0;
|
|
5
|
+
if (/^\d+(?:\.\d+)?$/.test(value)) return Number(value) * 1000;
|
|
6
|
+
const date = Date.parse(value);
|
|
7
|
+
return Number.isFinite(date) ? Math.max(0, date - now) : 0;
|
|
8
|
+
}
|
|
9
|
+
|
|
10
|
+
// Only snapshot operations have server receipts. Never automatically replay a
|
|
11
|
+
// one-use link request or apply this policy to arbitrary POST operations.
|
|
12
|
+
export async function requestSnapshot(endpoint, config, operation, payload, {
|
|
13
|
+
timeout = 30_000, attempts = 5, maxDelayMs = 60_000,
|
|
14
|
+
fetchImpl = fetch, sleep = delay, random = Math.random, now = Date.now,
|
|
15
|
+
} = {}) {
|
|
16
|
+
const body = JSON.stringify({ operation, ...payload });
|
|
17
|
+
for (let attempt = 0; attempt < attempts; attempt += 1) {
|
|
18
|
+
let response;
|
|
19
|
+
let result;
|
|
20
|
+
let failure;
|
|
21
|
+
let retryable = true;
|
|
22
|
+
try {
|
|
23
|
+
response = await fetchImpl(endpoint, {
|
|
24
|
+
method: "POST",
|
|
25
|
+
headers: { authorization: `Bearer ${config.token}`, "content-type": "application/json", "x-usagemax-device-id": config.deviceId },
|
|
26
|
+
body,
|
|
27
|
+
signal: AbortSignal.timeout(timeout),
|
|
28
|
+
});
|
|
29
|
+
result = await response.json().catch(() => null);
|
|
30
|
+
if (response.ok && result?.ok === true) return result;
|
|
31
|
+
retryable = response.ok || response.status === 429 || response.status >= 500
|
|
32
|
+
|| (operation === "complete" && response.status === 409
|
|
33
|
+
&& String(result?.error).toLowerCase() === "snapshot_run_incomplete");
|
|
34
|
+
const code = typeof result?.error === "string" ? result.error.replace(/[^a-zA-Z0-9_-]/g, "").slice(0, 80) : "invalid_response";
|
|
35
|
+
const advice = response.status === 401 ? " Link this computer again."
|
|
36
|
+
: response.status === 403 ? " Check workspace membership and collector permissions."
|
|
37
|
+
: response.status === 409 ? " The saved run needs reconciliation; do not delete its checkpoint or repeatedly start new runs."
|
|
38
|
+
: response.status === 400 ? " Check CLI/server protocol compatibility."
|
|
39
|
+
: "";
|
|
40
|
+
failure = new Error(`Snapshot ${operation} failed (HTTP ${response.status}: ${code}).${advice}`);
|
|
41
|
+
failure.code = code;
|
|
42
|
+
} catch {
|
|
43
|
+
failure = new Error(`Snapshot ${operation} could not reach UsageMax or timed out. Check your network connection.`);
|
|
44
|
+
}
|
|
45
|
+
if (!retryable || attempt + 1 >= attempts) throw failure;
|
|
46
|
+
const retryAfter = retryAfterMs(response?.headers?.get("retry-after"), now());
|
|
47
|
+
// Never retry earlier than the server's requested interval. Long backoffs
|
|
48
|
+
// remain durably resumable instead of blocking the command indefinitely.
|
|
49
|
+
if (retryAfter > maxDelayMs) throw new Error(`${failure.message} Server requested a longer wait; retry sync after ${Math.ceil(retryAfter / 1000)} seconds.`);
|
|
50
|
+
const jitter = Math.min(maxDelayMs, 1000 * 2 ** attempt) * (0.5 + random() * 0.5);
|
|
51
|
+
await sleep(Math.max(retryAfter, jitter));
|
|
52
|
+
}
|
|
53
|
+
throw new Error("Snapshot retry budget is empty.");
|
|
54
|
+
}
|