copilot-metrics 0.1.4 → 0.1.5

package/CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## 0.1.5 - 2026-05-31
+
+### Fixed
+
+- VS Code Copilot token usage is now attributed to Jira labels by matching OTel `gen_ai.response.id` values to VS Code chat session `responseId` values.
+- Existing local stores with older VS Code usage rows are repaired by backfilling missing response IDs from already imported raw OTel records.
+- VS Code chat session files are parsed only in memory for label extraction; full chat content is not persisted in the metrics store.
+- Versioned model IDs such as dated Copilot telemetry model names now use the canonical per-token pricing row when one is available, so estimates show what the token usage would cost even during included or `0x` periods.
+
 ## 0.1.4 - 2026-05-31
 
 ### Changed

package/README.md

@@ -9,8 +9,8 @@ Costs are estimates, not official billing records. GitHub billing remains the so
 From npm:
 
 ```bash
-npx copilot-metrics@0.1.4 --help
-npx copilot-metrics@0.1.4 init
+npx copilot-metrics@0.1.5 --help
+npx copilot-metrics@0.1.5 init
 ```
 
 From this checkout:
@@ -38,8 +38,8 @@ export COPILOT_METRICS_HOME=/path/to/copilot-metrics-data
 Useful commands:
 
 ```bash
-npx copilot-metrics@0.1.4 init
-npx copilot-metrics@0.1.4 paths --json
+npx copilot-metrics@0.1.5 init
+npx copilot-metrics@0.1.5 paths --json
 ```
 
 `init` only creates the central data directory and local config. It does not modify editor or hook settings. `setup` performs integration setup for the current machine/workspace.
@@ -51,19 +51,19 @@ For Copilot CLI, `init` plus hooks are enough for local token reporting. Reports
 Install VS Code Copilot Chat OpenTelemetry settings:
 
 ```bash
-npx copilot-metrics@0.1.4 setup vscode
+npx copilot-metrics@0.1.5 setup vscode
 ```
 
 Install Copilot CLI hooks for the current workspace:
 
 ```bash
-npx copilot-metrics@0.1.4 setup copilot-cli
+npx copilot-metrics@0.1.5 setup copilot-cli
 ```
 
 Or set up both VS Code settings and workspace hooks in one command:
 
 ```bash
-npx copilot-metrics@0.1.4 setup
+npx copilot-metrics@0.1.5 setup
 ```
 
 Use `setup vscode --print` or `setup copilot-cli --print` to print the settings/optional environment exports without writing files. Copilot CLI OTel exports are optional because CLI token usage is read from local session-state files.
@@ -75,14 +75,14 @@ Content capture is disabled by default. Do not enable richer prompt capture unle
 Preview repo-local hook config. The default `--surface both` emits the Copilot CLI lower camel case hook format:
 
 ```bash
-npx copilot-metrics@0.1.4 hooks preview --scope local --surface both
+npx copilot-metrics@0.1.5 hooks preview --scope local --surface both
 ```
 
 Install repo-local or user-global hook config:
 
 ```bash
-npx copilot-metrics@0.1.4 hooks install --scope local --surface both
-npx copilot-metrics@0.1.4 hooks install --scope global --surface both
+npx copilot-metrics@0.1.5 hooks install --scope local --surface both
+npx copilot-metrics@0.1.5 hooks install --scope global --surface both
 ```
 
 Local install writes `.github/hooks/copilot-metrics.json`. Global install updates `~/.copilot/settings.json` idempotently, replacing prior `copilot-metrics` hook entries while preserving other settings and hooks. Use `--surface vscode` for VS Code-only PascalCase events or `--surface copilot-cli` for CLI-native lower camel case events. The hook logger writes redacted JSONL metadata to the central data directory. It extracts Jira-style labels such as `DEMO-12345` from safe metadata and does not store full prompt text by default.
@@ -92,39 +92,40 @@ Local install writes `.github/hooks/copilot-metrics.json`. Global install update
 Initialize the local SQLite store and import JSONL files manually:
 
 ```bash
-npx copilot-metrics@0.1.4 store init
-npx copilot-metrics@0.1.4 import --source vscode --file ~/.local/share/copilot-metrics/telemetry/vscode-copilot-otel.jsonl
-npx copilot-metrics@0.1.4 import --source copilot-cli --file ~/.local/share/copilot-metrics/telemetry/copilot-cli-otel.jsonl
-npx copilot-metrics@0.1.4 import --source copilot-session --file ~/.copilot/session-state/<session-id>/events.jsonl
-npx copilot-metrics@0.1.4 import --source hooks --file ~/.local/share/copilot-metrics/hooks/copilot-hooks.jsonl
+npx copilot-metrics@0.1.5 store init
+npx copilot-metrics@0.1.5 import --source vscode --file ~/.local/share/copilot-metrics/telemetry/vscode-copilot-otel.jsonl
+npx copilot-metrics@0.1.5 import --source copilot-cli --file ~/.local/share/copilot-metrics/telemetry/copilot-cli-otel.jsonl
+npx copilot-metrics@0.1.5 import --source copilot-session --file ~/.copilot/session-state/<session-id>/events.jsonl
+npx copilot-metrics@0.1.5 import --source vscode-chat --file ~/.config/Code\ -\ Insiders/User/workspaceStorage/<workspace-id>/chatSessions/<session-id>.jsonl
+npx copilot-metrics@0.1.5 import --source hooks --file ~/.local/share/copilot-metrics/hooks/copilot-hooks.jsonl
 ```
 
-Imports persist raw records, normalized LLM usage records, hook events, label evidence, and import warnings. Re-importing the same JSONL rows is idempotent. For Copilot session-state files, only shutdown usage rows are persisted; prompt-bearing session events are used in memory for label extraction and context and are not stored as raw records.
+Imports persist raw records, normalized LLM usage records, hook events, label evidence, and import warnings. Re-importing the same JSONL rows is idempotent. For Copilot session-state files, only shutdown usage rows are persisted; prompt-bearing session events are used in memory for label extraction and context and are not stored as raw records. VS Code chat session files are also parsed only in memory, then reduced to label evidence linked to VS Code OTel usage by exact response ID.
 
 ## Reports
 
 Run local reports from the SQLite store:
 
 ```bash
-npx copilot-metrics@0.1.4 report labels
-npx copilot-metrics@0.1.4 report label DEMO-12345
-npx copilot-metrics@0.1.4 report label DEMO-12345 --detail
-npx copilot-metrics@0.1.4 report models
-npx copilot-metrics@0.1.4 report repos
-npx copilot-metrics@0.1.4 report unattributed
+npx copilot-metrics@0.1.5 report labels
+npx copilot-metrics@0.1.5 report label DEMO-12345
+npx copilot-metrics@0.1.5 report label DEMO-12345 --detail
+npx copilot-metrics@0.1.5 report models
+npx copilot-metrics@0.1.5 report repos
+npx copilot-metrics@0.1.5 report unattributed
 ```
 
 Every report supports `--json`:
 
 ```bash
-npx copilot-metrics@0.1.4 report labels --json
+npx copilot-metrics@0.1.5 report labels --json
 ```
 
-Report commands automatically import newly appended configured VS Code OTel, optional Copilot CLI OTel, Copilot CLI session-state, and hook JSONL files before querying. Repeated reports skip already imported session-state files and already imported JSONL lines.
+Report commands automatically import newly appended configured VS Code OTel, VS Code chat session metadata, optional Copilot CLI OTel, Copilot CLI session-state, and hook JSONL files before querying. Repeated reports skip already imported session-state files and already imported JSONL lines.
 
 `report labels` shows accumulated totals per label. `report label <id>` shows the selected label summary plus a per-model breakdown by default. Label reports include input, output, cache read, cache creation, and reasoning token totals. Labels seen only in hooks remain visible as `evidence-only` with zero usage records, so attribution hints do not imply token-bearing usage.
 
-`AI Credits est.` is a local estimate derived from the pricing table. The project treats 1 AI Credit as $0.01 for estimates; GitHub billing remains the source of truth.
+`AI Credits est.` is a local what-would-this-cost estimate derived from the token pricing table, not a claim that the interaction was billed today. Some included or request-based models can appear as `0x` in Copilot while still having published per-token prices. The project treats 1 AI Credit as $0.01 for estimates; GitHub billing remains the source of truth.
 
 ## Attribution Model
 
@@ -132,6 +133,8 @@ The default extractor finds Jira-style labels such as `DEMO-12345` from safe met
 
 Attribution is stored as evidence with source, field, session, repo, branch, cwd, confidence, and related usage or hook record IDs. This makes the data useful for later analysis, such as deciding whether a label was the main task or a sidetrack.
 
+For VS Code Copilot Chat, token records from OTel are linked to chat labels by exact response ID. The OTel `gen_ai.response.id` value must match the VS Code chat session `responseId`; timestamp-only attribution is not used.
+
 Full prompt content is not stored by default. Prompt-like fields are only used to extract labels and the stored source value is reduced to the matched label.
 
 ## Custom Label Extractors
@@ -187,7 +190,7 @@ The manual prompt performs one harmless tool call so Copilot CLI hook execution
 ## Current Limits
 
 - Costs are estimates, not official billing records.
-- Official GitHub usage report reconciliation is not included in `0.1.4`.
-- Local OTLP collector mode is not included in `0.1.4`.
-- Richer prompt/content capture and redaction controls are not included in `0.1.4`.
+- Official GitHub usage report reconciliation is not included in `0.1.5`.
+- Local OTLP collector mode is not included in `0.1.5`.
+- Richer prompt/content capture and redaction controls are not included in `0.1.5`.
 - Dashboard views are deferred until the CLI/query model proves useful.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "copilot-metrics",
-  "version": "0.1.4",
+  "version": "0.1.5",
   "description": "Local-first Copilot usage telemetry setup and reporting tools.",
   "type": "commonjs",
   "homepage": "https://github.com/nnexai/copilot-metrics#readme",

package/src/cli.js

@@ -79,7 +79,7 @@ Usage:
   copilot-metrics hooks install [--scope local|global] [--surface both|vscode|copilot-cli] [--json]
   copilot-metrics hook-log --event <name>
   copilot-metrics store init [--json]
-  copilot-metrics import --source vscode|copilot-cli|copilot-session|hooks --file <path> [--json]
+  copilot-metrics import --source vscode|vscode-chat|copilot-cli|copilot-session|hooks --file <path> [--json]
   copilot-metrics report labels [--json]
   copilot-metrics report label <id> [--detail] [--json]
   copilot-metrics report models [--json]
@@ -305,8 +305,8 @@ async function main(args, io) {
         : source === 'hooks'
           ? paths.hookEventsJsonl
           : null);
-    if (!['vscode', 'copilot-cli', 'copilot-session', 'hooks'].includes(source)) {
-      throw new Error('import requires --source vscode|copilot-cli|copilot-session|hooks');
+    if (!['vscode', 'vscode-chat', 'copilot-cli', 'copilot-session', 'hooks'].includes(source)) {
+      throw new Error('import requires --source vscode|vscode-chat|copilot-cli|copilot-session|hooks');
     }
     if (!file) throw new Error('import requires --file <path>');
     ensureDataDirs(paths);

package/src/ingest.js

@@ -2,13 +2,23 @@
 
 const crypto = require('node:crypto');
 const fs = require('node:fs');
+const os = require('node:os');
 const path = require('node:path');
 const { readJsonl } = require('./jsonl');
 const { normalizePayload, normalizeHookEvent, normalizeCopilotSessionEvents } = require('./otel');
 const { estimateCost, PRICING_VERSION } = require('./pricing');
-const { existingRawFingerprints, importedLineHighWater, insertImport } = require('./sqlite-store');
+const {
+  attachVscodeChatLabelEvidence,
+  existingRawFingerprints,
+  importedLineHighWater,
+  insertImport,
+  queryRows,
+  updateUsageCostEstimates,
+  updateVscodeUsageResponseIds,
+  vscodeRawRecordsNeedingResponseBackfill,
+} = require('./sqlite-store');
 const { attachUsageLabelEvidence, attachHookLabelEvidence } = require('./labels');
-const { loadConfiguredExtractors } = require('./label-extractors');
+const { loadConfiguredExtractors, runLabelExtractors } = require('./label-extractors');
 
 function enrichCosts(records) {
   return records.map((record) => {
@@ -42,9 +52,221 @@ function isCopilotSessionUsageRecord(record) {
   return record.value && record.value.type === 'session.shutdown';
 }
 
+function pushText(values, value) {
+  if (typeof value === 'string' && value.trim()) values.push(value);
+}
+
+function pushPromptCandidates(values, value) {
+  if (!value || typeof value !== 'object') return;
+  if (Array.isArray(value)) {
+    for (const item of value) pushPromptCandidates(values, item);
+    return;
+  }
+  pushText(values, value.text);
+  pushText(values, value.value);
+  pushText(values, value.message);
+  pushText(values, value.prompt);
+  pushText(values, value.promptText);
+  pushText(values, value.renderedUserMessage);
+  pushText(values, value.userMessage);
+  if (value.renderedUserMessage && typeof value.renderedUserMessage === 'object') {
+    pushPromptCandidates(values, value.renderedUserMessage);
+  }
+  if (value.message && typeof value.message === 'object') pushPromptCandidates(values, value.message);
+  if (value.result && typeof value.result === 'object') pushPromptCandidates(values, value.result);
+  if (value.metadata && typeof value.metadata === 'object') pushPromptCandidates(values, value.metadata);
+}
+
+function responseId(value) {
+  if (!value || typeof value !== 'object') return null;
+  return value.responseId
+    || value.metadata?.responseId
+    || value.result?.responseId
+    || value.result?.metadata?.responseId
+    || value.modelMessageId
+    || value.metadata?.modelMessageId
+    || null;
+}
+
+function chatSessionId(value) {
+  if (!value || typeof value !== 'object') return null;
+  return value.sessionId
+    || value.sessionID
+    || value.metadata?.sessionId
+    || value.result?.sessionId
+    || value.result?.metadata?.sessionId
+    || null;
+}
+
+function chatRequestIndex(record) {
+  const key = Array.isArray(record.k) ? record.k : Array.isArray(record.key) ? record.key : [];
+  if (key[0] !== 'requests') return null;
+  const index = Number(key[1]);
+  return Number.isInteger(index) ? index : null;
+}
+
+function normalizeVscodeChatSession(records, extractors = []) {
+  const requests = new Map();
+  let defaultSessionId = null;
+
+  function entry(index) {
+    const key = String(index);
+    if (!requests.has(key)) requests.set(key, { texts: [] });
+    return requests.get(key);
+  }
+
+  function mergeRequest(index, request, sessionId) {
+    if (!request || typeof request !== 'object') return;
+    const current = entry(index);
+    current.sessionId = chatSessionId(request) || sessionId || current.sessionId;
+    current.responseId = responseId(request) || current.responseId;
+    pushPromptCandidates(current.texts, request);
+  }
+
+  for (const record of records) {
+    const value = record.value;
+    if (!value || typeof value !== 'object') continue;
+    const root = value.v && typeof value.v === 'object' ? value.v : value;
+    defaultSessionId = root.sessionId || root.sessionID || defaultSessionId;
+
+    if (Array.isArray(root.requests)) {
+      root.requests.forEach((request, index) => mergeRequest(index, request, defaultSessionId));
+    }
+
+    const key = Array.isArray(value.k) ? value.k : Array.isArray(value.key) ? value.key : [];
+    if (key.length === 1 && key[0] === 'requests' && Array.isArray(value.v)) {
+      const startIndex = requests.size;
+      value.v.forEach((request, offset) => mergeRequest(startIndex + offset, request, defaultSessionId));
+    }
+
+    const index = chatRequestIndex(value);
+    if (index !== null) {
+      const current = entry(index);
+      const patch = value.v;
+      if (patch && typeof patch === 'object') {
+        current.sessionId = chatSessionId(patch) || defaultSessionId || current.sessionId;
+        current.responseId = responseId(patch) || current.responseId;
+        pushPromptCandidates(current.texts, patch);
+      } else {
+        pushText(current.texts, patch);
+      }
+    }
+  }
+
+  return Array.from(requests.values())
+    .filter((request) => request.responseId)
+    .map((request) => {
+      const labelEvidence = runLabelExtractors('usage', { prompt: request.texts }, extractors)
+        .map((evidence) => ({
+          ...evidence,
+          source_type: 'usage',
+          source_field: 'vscode_chat_response',
+          source_value: request.responseId,
+          confidence: Math.max(Number(evidence.confidence || 0), 0.95),
+        }));
+      return {
+        responseId: request.responseId,
+        sessionId: request.sessionId || defaultSessionId || null,
+        label_evidence: labelEvidence,
+      };
+    })
+    .filter((request) => request.label_evidence.length > 0);
+}
+
+async function ingestVscodeChatSessionFile(options) {
+  const { dbPath, file } = options;
+  const sourceFile = path.resolve(file);
+  const parsed = readJsonl(sourceFile);
+  const mappings = normalizeVscodeChatSession(parsed.records, options.extractors || []);
+  const attached = await attachVscodeChatLabelEvidence(dbPath, mappings);
+  return {
+    source: 'vscode-chat',
+    file,
+    dbPath,
+    raw_records: 0,
+    new_raw_records: 0,
+    skipped_existing_records: 0,
+    usage_records: attached.matched_usage_records,
+    hook_events: 0,
+    label_evidence: attached.label_evidence,
+    warnings: parsed.warnings,
+    estimate_label: `estimate:${PRICING_VERSION}`,
+  };
+}
+
+async function backfillVscodeUsageResponseIds(dbPath, sourceFile) {
+  const rows = await vscodeRawRecordsNeedingResponseBackfill(dbPath, sourceFile);
+  const updates = [];
+  for (const row of rows) {
+    let payload;
+    try {
+      payload = JSON.parse(row.payload_json);
+    } catch {
+      continue;
+    }
+    for (const usage of normalizePayload(payload, 'vscode', row.line)) {
+      if (!usage.span_id) continue;
+      updates.push({
+        raw_line: usage.raw_line,
+        span_id: usage.span_id,
+        session_id: usage.session_id,
+        timestamp: usage.timestamp,
+        requested_model: usage.requested_model,
+        resolved_model: usage.resolved_model,
+        input_tokens: usage.input_tokens,
+        output_tokens: usage.output_tokens,
+        cache_read_tokens: usage.cache_read_tokens,
+        cache_creation_tokens: usage.cache_creation_tokens,
+        reasoning_tokens: usage.reasoning_tokens,
+      });
+    }
+  }
+  return updateVscodeUsageResponseIds(dbPath, updates);
+}
+
+function parseWarningsJson(value) {
+  try {
+    const parsed = JSON.parse(value || '[]');
+    return Array.isArray(parsed) ? parsed : [];
+  } catch {
+    return [];
+  }
+}
+
+async function repairUsageCostEstimates(dbPath) {
+  const rows = await queryRows(dbPath, `
+    SELECT id, requested_model, resolved_model, input_tokens, output_tokens,
+      cache_read_tokens, cache_creation_tokens, reasoning_tokens, warnings_json
+    FROM usage_records
+    WHERE estimated_ai_credits IS NULL
+      OR estimated_ai_credits = 0
+      OR warnings_json LIKE '%unknown_model:%'
+      OR warnings_json LIKE '%missing_model%'
+  `);
+  const updates = [];
+  for (const row of rows) {
+    const estimate = estimateCost(row);
+    if (estimate.warning) continue;
+    const warnings = parseWarningsJson(row.warnings_json)
+      .filter((warning) => !String(warning).startsWith('unknown_model:') && warning !== 'missing_model');
+    updates.push({
+      id: row.id,
+      estimated_usd: estimate.estimated_usd,
+      estimated_ai_credits: estimate.estimated_ai_credits,
+      warnings,
+    });
+  }
+  return updateUsageCostEstimates(dbPath, updates);
+}
+
 async function ingestFile(options) {
   const { dbPath, file, source } = options;
+  if (source === 'vscode-chat') return ingestVscodeChatSessionFile(options);
+
   const sourceFile = path.resolve(file);
+  const backfilledUsageRecords = source === 'vscode'
+    ? await backfillVscodeUsageResponseIds(dbPath, sourceFile)
+    : 0;
   const highWaterLine = await importedLineHighWater(dbPath, source, sourceFile);
   if (source === 'copilot-session' && highWaterLine > 0) {
     return {
@@ -57,6 +279,7 @@ async function ingestFile(options) {
       usage_records: 0,
       hook_events: 0,
       label_evidence: 0,
+      backfilled_usage_records: backfilledUsageRecords,
       warnings: [],
       estimate_label: `estimate:${PRICING_VERSION}`,
     };
@@ -108,6 +331,7 @@ async function ingestFile(options) {
   }
 
   await insertImport(dbPath, source, sourceFile, newRecords, enrichedUsage, enrichedHooks, warnings);
+  const repairedCostRecords = await repairUsageCostEstimates(dbPath);
 
   return {
     source,
@@ -118,6 +342,8 @@ async function ingestFile(options) {
     skipped_existing_records: highWaterLine,
     usage_records: enrichedUsage.length,
     hook_events: enrichedHooks.length,
+    backfilled_usage_records: backfilledUsageRecords,
+    repaired_cost_records: repairedCostRecords,
     label_evidence: enrichedUsage.reduce((sum, usage) => sum + (usage.label_evidence || []).length, 0)
       + enrichedHooks.reduce((sum, event) => sum + (event.label_evidence || []).length, 0),
     warnings,
@@ -130,6 +356,7 @@ function configuredSourceFiles(paths, config = {}) {
   const telemetryConfig = config.telemetry || {};
   const files = [
     { source: 'vscode', file: sourceConfig.vscode?.telemetry || telemetryConfig.vscode || paths.vscodeOtelJsonl },
+    ...discoverVscodeChatSessionFiles(sourceConfig.vscode?.chatSessions),
     { source: 'hooks', file: sourceConfig.vscode?.hooks || paths.hookEventsJsonl },
     { source: 'copilot-cli', file: sourceConfig.copilotCli?.telemetry || telemetryConfig.copilotCli || paths.copilotCliOtelJsonl },
     { source: 'hooks', file: sourceConfig.copilotCli?.hooks || paths.hookEventsJsonl },
@@ -147,6 +374,40 @@ function configuredSourceFiles(paths, config = {}) {
     });
 }
 
+function listJsonlFiles(dir) {
+  if (!dir || !fs.existsSync(dir)) return [];
+  return fs.readdirSync(dir, { withFileTypes: true })
+    .filter((entry) => entry.isFile() && entry.name.endsWith('.jsonl'))
+    .map((entry) => path.join(dir, entry.name));
+}
+
+function discoverWorkspaceChatSessions(workspaceStorageDir) {
+  if (!workspaceStorageDir || !fs.existsSync(workspaceStorageDir)) return [];
+  return fs.readdirSync(workspaceStorageDir, { withFileTypes: true })
+    .filter((entry) => entry.isDirectory())
+    .flatMap((entry) => listJsonlFiles(path.join(workspaceStorageDir, entry.name, 'chatSessions')));
+}
+
+function discoverVscodeChatSessionFiles(configured) {
+  const configuredEntries = Array.isArray(configured) ? configured : configured ? [configured] : [];
+  const files = configuredEntries.length > 0
+    ? configuredEntries.flatMap((entry) => {
+      const resolved = path.resolve(entry);
+      if (!fs.existsSync(resolved)) return [];
+      const stat = fs.statSync(resolved);
+      if (stat.isFile()) return [resolved];
+      return listJsonlFiles(resolved).concat(discoverWorkspaceChatSessions(resolved));
+    })
+    : [
+      path.join(os.homedir(), '.config', 'Code', 'User', 'workspaceStorage'),
+      path.join(os.homedir(), '.config', 'Code - Insiders', 'User', 'workspaceStorage'),
+    ].flatMap(discoverWorkspaceChatSessions);
+
+  return files
+    .sort()
+    .map((file) => ({ source: 'vscode-chat', file }));
+}
+
 function discoverCopilotSessionFiles(sessionStateDir) {
   if (!sessionStateDir || !fs.existsSync(sessionStateDir)) return [];
   return fs.readdirSync(sessionStateDir, { withFileTypes: true })
@@ -184,5 +445,9 @@ module.exports = {
   autoImportConfiguredSources,
   configuredSourceFiles,
   discoverCopilotSessionFiles,
+  discoverVscodeChatSessionFiles,
+  backfillVscodeUsageResponseIds,
   ingestFile,
+  normalizeVscodeChatSession,
+  repairUsageCostEstimates,
 };

package/src/otel.js

@@ -2,9 +2,16 @@
 
 function attrsToObject(attrs) {
   if (!attrs) return {};
+  if (attrs && typeof attrs === 'object' && Array.isArray(attrs._rawAttributes)) {
+    return Object.fromEntries(attrs._rawAttributes);
+  }
   if (!Array.isArray(attrs)) return attrs;
   const out = {};
   for (const attr of attrs) {
+    if (Array.isArray(attr) && attr.length >= 2) {
+      out[attr[0]] = attr[1];
+      continue;
+    }
     const value = attr.value;
     if (value && typeof value === 'object') {
       out[attr.key] = value.stringValue ?? value.intValue ?? value.doubleValue ?? value.boolValue ?? value.arrayValue;
@@ -61,8 +68,28 @@ function flattenSpans(payload) {
   return spans;
 }
 
+function timestampValue(value) {
+  if (!value) return null;
+  if (Array.isArray(value) && value.length >= 2) {
+    const millis = (Number(value[0]) * 1000) + (Number(value[1]) / 1e6);
+    return Number.isFinite(millis) ? new Date(millis).toISOString() : null;
+  }
+  if (typeof value === 'string' && /^\d+$/.test(value)) {
+    const numeric = Number(value);
+    if (!Number.isFinite(numeric)) return null;
+    const millis = numeric > 1e15 ? numeric / 1e6 : numeric;
+    return new Date(millis).toISOString();
+  }
+  if (typeof value === 'number') {
+    const millis = value > 1e15 ? value / 1e6 : value;
+    return new Date(millis).toISOString();
+  }
+  return value;
+}
+
 function classifySpan(span) {
   const attrs = attrsToObject(span.attributes);
+  const eventName = String(pick(attrs, ['event.name']) || '').toLowerCase();
   const operation = String(pick(attrs, ['gen_ai.operation.name', 'llm.operation']) || '').toLowerCase();
   const name = String(span.name || '').toLowerCase();
   const hasTokens = number(attrs, [
@@ -72,7 +99,14 @@ function classifySpan(span) {
     'llm.usage.completion_tokens',
   ]) > 0;
 
-  if (operation.includes('agent') || operation.includes('tool') || name.includes('agent') || name.includes('tool')) {
+  if (
+    eventName.includes('agent')
+    || eventName.includes('tool')
+    || operation.includes('agent')
+    || operation.includes('tool')
+    || name.includes('agent')
+    || name.includes('tool')
+  ) {
     return 'non_billable';
   }
   if (hasTokens || operation.includes('chat') || operation.includes('completion') || operation.includes('generate')) {
@@ -83,19 +117,19 @@ function classifySpan(span) {
 
 function normalizeSpan(span, source, rawLine) {
   const attrs = attrsToObject(span.attributes);
-  const resourceAttrs = attrsToObject(span.resourceAttributes);
+  const resourceAttrs = attrsToObject(span.resourceAttributes || span.resource);
   const type = classifySpan(span);
   if (type !== 'llm') return null;
 
   return {
     raw_line: rawLine,
-    span_id: span.spanId || span.span_id || null,
+    span_id: span.spanId || span.span_id || pick(attrs, ['gen_ai.response.id']) || null,
     trace_id: span.traceId || span.trace_id || null,
     parent_span_id: span.parentSpanId || span.parent_span_id || null,
-    timestamp: span.startTimeUnixNano || span.start_time || attrs['timestamp'] || null,
+    timestamp: timestampValue(span.startTimeUnixNano || span.start_time || span.hrTime || attrs.timestamp),
     surface: source,
     conversation_id: pick(attrs, ['gen_ai.conversation.id', 'conversation.id', 'copilot.conversation.id']),
-    session_id: pick(attrs, ['session.id', 'copilot.session.id']),
+    session_id: pick(attrs, ['session.id', 'copilot.session.id']) || pick(resourceAttrs, ['session.id', 'copilot.session.id']),
     requested_model: pick(attrs, ['gen_ai.request.model', 'llm.request.model', 'llm.model_name']),
     resolved_model: pick(attrs, ['gen_ai.response.model', 'llm.response.model', 'model']),
     repo: pick(attrs, ['vcs.repository.name', 'git.repository', 'repo']) || pick(resourceAttrs, ['vcs.repository.name', 'service.name']),
@@ -194,6 +228,7 @@ function normalizeHookEvent(payload, source, rawLine) {
   return {
     raw_line: rawLine,
     event: payload.event || null,
+    timestamp: payload.captured_at || payload.timestamp || null,
     session_id: payload.session_id || payload.sessionId || null,
     cwd: payload.cwd || null,
     repo: payload.repo || payload.repository || null,

package/src/pricing.js

@@ -2,7 +2,7 @@
 
 const PRICING_VERSION = 'github-copilot-2026-06-01';
 
-// USD per 1M tokens. Source: GitHub Copilot models and pricing docs, checked 2026-05-30.
+// USD per 1M tokens. Source: GitHub Copilot models and pricing docs, checked 2026-05-31.
 const MODEL_PRICES = {
   'gpt-4.1': { input: 2.00, cacheRead: 0.50, cacheWrite: 0, output: 8.00 },
   'gpt-5 mini': { input: 0.25, cacheRead: 0.025, cacheWrite: 0, output: 2.00 },
@@ -32,11 +32,19 @@ const MODEL_PRICES = {
 };
 
 function normalizeModelName(model) {
-  return String(model || '').trim().toLowerCase();
+  return String(model || '').trim().toLowerCase().replace(/^copilot\//, '');
+}
+
+function modelPriceKey(model) {
+  const normalized = normalizeModelName(model);
+  if (MODEL_PRICES[normalized]) return normalized;
+  const withoutDate = normalized.replace(/-\d{4}-\d{2}-\d{2}$/, '');
+  if (MODEL_PRICES[withoutDate]) return withoutDate;
+  return normalized;
 }
 
 function estimateCost(record) {
-  const model = normalizeModelName(record.resolved_model || record.requested_model);
+  const model = modelPriceKey(record.resolved_model || record.requested_model);
   const price = MODEL_PRICES[model];
   if (!model) {
     return { estimated_usd: null, estimated_ai_credits: null, warning: 'missing_model' };
@@ -63,4 +71,5 @@ module.exports = {
   PRICING_VERSION,
   MODEL_PRICES,
   estimateCost,
+  modelPriceKey,
 };

package/src/sqlite-store.js

@@ -141,6 +141,7 @@ function lastInsertId(db) {
 }
 
 function insertLabelEvidence(db, importedAt, evidenceRows) {
+  if (!evidenceRows.length) return;
   runPrepared(
     db,
     `INSERT INTO label_evidence (
@@ -283,6 +284,7 @@ async function insertImport(dbPath, source, sourceFile, rawRecords, usageRecords
             repo: event.repo,
             branch: event.branch,
             cwd: event.cwd,
+            timestamp: event.timestamp,
           });
         }
       }
@@ -307,6 +309,188 @@ async function insertImport(dbPath, source, sourceFile, rawRecords, usageRecords
   persistDatabase(dbPath, db);
 }
 
+async function attachVscodeChatLabelEvidence(dbPath, mappings) {
+  await initStore(dbPath);
+  if (!mappings.length) {
+    return { matched_usage_records: 0, label_evidence: 0 };
+  }
+
+  const db = await openDatabase(dbPath);
+  const importedAt = new Date().toISOString();
+  let matchedUsageRecords = 0;
+  let labelEvidence = 0;
+
+  const usageStatement = db.prepare(`
+    SELECT id, session_id, repo, branch, cwd, timestamp
+    FROM usage_records
+    WHERE source = 'vscode' AND span_id = ?
+  `);
+  const existingStatement = db.prepare(`
+    SELECT 1
+    FROM label_evidence
+    WHERE label = ?
+      AND source_type = 'usage'
+      AND source_field = 'vscode_chat_response'
+      AND source_value = ?
+      AND usage_record_id = ?
+    LIMIT 1
+  `);
+  const insertStatement = db.prepare(`
+    INSERT INTO label_evidence (
+      imported_at, label, source_type, source_field, source_value, confidence,
+      usage_record_id, hook_event_id, session_id, repo, branch, cwd, timestamp
+    ) VALUES (?, ?, 'usage', 'vscode_chat_response', ?, ?, ?, NULL, ?, ?, ?, ?, ?)
+  `);
+
+  db.run('BEGIN');
+  try {
+    for (const mapping of mappings) {
+      usageStatement.bind([mapping.responseId]);
+      const usageRows = [];
+      while (usageStatement.step()) usageRows.push(usageStatement.getAsObject());
+      usageStatement.reset();
+      matchedUsageRecords += usageRows.length;
+
+      for (const usage of usageRows) {
+        for (const evidence of mapping.label_evidence || []) {
+          existingStatement.bind([evidence.label, mapping.responseId, usage.id]);
+          const exists = existingStatement.step();
+          existingStatement.reset();
+          if (exists) continue;
+
+          insertStatement.run([
+            importedAt,
+            evidence.label,
+            mapping.responseId,
+            evidence.confidence || 0.95,
+            usage.id,
+            mapping.sessionId || usage.session_id || null,
+            usage.repo || null,
+            usage.branch || null,
+            usage.cwd || null,
+            usage.timestamp || null,
+          ]);
+          labelEvidence += 1;
+        }
+      }
+    }
+    db.run('COMMIT');
+  } catch (error) {
+    db.run('ROLLBACK');
+    throw error;
+  } finally {
+    usageStatement.free();
+    existingStatement.free();
+    insertStatement.free();
+  }
+
+  persistDatabase(dbPath, db);
+  return { matched_usage_records: matchedUsageRecords, label_evidence: labelEvidence };
+}
+
+async function vscodeRawRecordsNeedingResponseBackfill(dbPath, sourceFile) {
+  await initStore(dbPath);
+  return queryRows(dbPath, `
+    SELECT rr.line, rr.payload_json
+    FROM raw_records rr
+    WHERE rr.source = 'vscode'
+      AND rr.source_file = ?
+      AND EXISTS (
+        SELECT 1
+        FROM usage_records ur
+        WHERE ur.source = 'vscode'
+          AND ur.raw_line = rr.line
+          AND ur.span_id IS NULL
+      )
+  `, [sourceFile]);
+}
+
+async function updateVscodeUsageResponseIds(dbPath, updates) {
+  await initStore(dbPath);
+  if (!updates.length) return 0;
+
+  const db = await openDatabase(dbPath);
+  const statement = db.prepare(`
+    UPDATE usage_records
+    SET span_id = ?,
+        session_id = COALESCE(session_id, ?),
+        timestamp = COALESCE(timestamp, ?)
+    WHERE source = 'vscode'
+      AND raw_line = ?
+      AND span_id IS NULL
+      AND input_tokens = ?
+      AND output_tokens = ?
+      AND cache_read_tokens = ?
+      AND cache_creation_tokens = ?
+      AND reasoning_tokens = ?
+      AND COALESCE(resolved_model, requested_model, '') = COALESCE(?, '')
+  `);
+  let updated = 0;
+  db.run('BEGIN');
+  try {
+    for (const update of updates) {
+      statement.run([
+        update.span_id,
+        update.session_id || null,
+        update.timestamp || null,
+        update.raw_line,
+        update.input_tokens,
+        update.output_tokens,
+        update.cache_read_tokens,
+        update.cache_creation_tokens,
+        update.reasoning_tokens,
+        update.resolved_model || update.requested_model || '',
+      ]);
+      updated += typeof db.getRowsModified === 'function' ? db.getRowsModified() : 0;
+    }
+    db.run('COMMIT');
+  } catch (error) {
+    db.run('ROLLBACK');
+    throw error;
+  } finally {
+    statement.free();
+  }
+
+  persistDatabase(dbPath, db);
+  return updated;
+}
+
+async function updateUsageCostEstimates(dbPath, updates) {
+  await initStore(dbPath);
+  if (!updates.length) return 0;
+
+  const db = await openDatabase(dbPath);
+  const statement = db.prepare(`
+    UPDATE usage_records
+    SET estimated_usd = ?,
+        estimated_ai_credits = ?,
+        warnings_json = ?
+    WHERE id = ?
+  `);
+  let updated = 0;
+  db.run('BEGIN');
+  try {
+    for (const update of updates) {
+      statement.run([
+        update.estimated_usd,
+        update.estimated_ai_credits,
+        JSON.stringify(update.warnings || []),
+        update.id,
+      ]);
+      updated += typeof db.getRowsModified === 'function' ? db.getRowsModified() : 0;
+    }
+    db.run('COMMIT');
+  } catch (error) {
+    db.run('ROLLBACK');
+    throw error;
+  } finally {
+    statement.free();
+  }
+
+  persistDatabase(dbPath, db);
+  return updated;
+}
+
 async function queryOne(dbPath, sql) {
   const db = await openDatabase(dbPath);
   const result = db.exec(sql);
@@ -329,10 +513,14 @@ async function queryRows(dbPath, sql, params = []) {
 }
 
 module.exports = {
+  attachVscodeChatLabelEvidence,
   existingRawFingerprints,
   importedLineHighWater,
   initStore,
   insertImport,
   queryOne,
   queryRows,
+  updateUsageCostEstimates,
+  updateVscodeUsageResponseIds,
+  vscodeRawRecordsNeedingResponseBackfill,
 };