@vibescore/tracker 0.0.3 → 0.0.4

package/README.md

@@ -1,26 +1,97 @@
-# @vibescore/tracker
+<div align="center">
 
-Codex CLI token usage tracker (macOS-first, notify-driven).
+# 🟢 VIBESCORE
 
-## Quick Start
+**QUANTIFY YOUR AI OUTPUT**  
+_Real-time AI Analytics for Codex CLI_
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![Node.js Support](https://img.shields.io/badge/Node.js-%3E%3D18-brightgreen.svg)](https://nodejs.org/)
+[![Platform](https://img.shields.io/badge/Platform-macOS-lightgrey.svg)](https://www.apple.com/macos/)
+
+[**English**](README.md) • [**中文说明**](README.zh-CN.md)
+
+[**Documentation**](docs/) • [**Dashboard**](dashboard/) • [**Backend API**](BACKEND_API.md)
+
+</div>
+
+---
+
+## 🌌 Overview
+
+**VibeScore** is an intelligent token usage tracking system designed specifically for macOS developers. It monitors Codex CLI output in real-time, transforming your **AI Output** into quantifiable metrics via a high-fidelity, **Matrix-themed** dashboard.
+
+> [!TIP] > **Core Index**: Our signature metric that reflects your flow state by analyzing token consumption rates and patterns.
+
+## 🚀 Key Features
+
+- 📡 **Live Sniffer**: Real-time interception of Codex CLI pipes using low-level hooks to capture every completion event.
+- 📊 **Matrix Dashboard**: A high-performance React + Vite dashboard featuring heatmaps, trend charts, and live logs.
+- ⚡ **AI Analytics**: Deep analysis of Input/Output tokens, with dedicated tracking for Cached and Reasoning components.
+- 🔒 **Identity Core**: Robust authentication and permission management to secure your development data.
+
+## 🛠️ Quick Start
+
+### Installation
+
+Initialize your environment with a single command:
 
 ```bash
 npx --yes @vibescore/tracker init
+```
+
+### Sync & Status
+
+```bash
+# Sync latest local session data
 npx --yes @vibescore/tracker sync
+
+# Check current link status
 npx --yes @vibescore/tracker status
-npx --yes @vibescore/tracker uninstall
 ```
 
-## Requirements
+## 🏗️ Architecture
+
+```mermaid
+graph TD
+    A[Codex CLI] -->|Rollout Logs| B(Tracker CLI)
+    B -->|AI Tokens| C{Core Relay}
+    C --> D[VibeScore Dashboard]
+    C --> E[AI Analytics Engine]
+```
+
+## 💻 Developer Guide
+
+To run locally or contribute:
 
-- Node.js >= 18
-- macOS (current supported platform)
+### Dashboard Development
+
+```bash
+# Install dependencies
+cd dashboard
+npm install
+
+# Start dev server
+npm run dev
+```
+
+### Architecture Validation
+
+```bash
+# Validate Copy Registry
+npm run validate:copy
+
+# Run smoke tests
+npm run smoke
+```
 
-## Notes
+## 📜 License
 
-- `init` installs a Codex CLI notify hook and issues a device token.
-- `sync` parses `~/.codex/sessions/**/rollout-*.jsonl` and uploads token_count deltas.
+This project is licensed under the [MIT License](LICENSE).
 
-## License
+---
 
-MIT
+<div align="center">
+  <b>System_Ready // 2024 VibeScore OS</b><br/>
+  <i>"More Tokens. More Vibe."</i>
+</div>

package/README.zh-CN.md

@@ -0,0 +1,97 @@
+<div align="center">
+
+# 🟢 VIBESCORE
+
+**量化你的 AI 产出**  
+_Codex CLI 实时 AI 分析工具_
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![Node.js Support](https://img.shields.io/badge/Node.js-%3E%3D18-brightgreen.svg)](https://nodejs.org/)
+[![Platform](https://img.shields.io/badge/Platform-macOS-lightgrey.svg)](https://www.apple.com/macos/)
+
+[**English**](README.md) • [**中文说明**](README.zh-CN.md)
+
+[**文档**](docs/) • [**控制台**](dashboard/) • [**后端接口**](BACKEND_API.md)
+
+</div>
+
+---
+
+## 🌌 项目概述
+
+**VibeScore** 是一个专为 macOS 开发者设计的智能令牌（Token）使用追踪系统。它能够实时监控 Codex CLI 的输出，通过高度可视化的 **Matrix** 风格仪表盘，将你的 **AI 产出 (AI Output)** 转化为可量化的指标。
+
+> [!TIP] > **Core Index (核心指数)**: 我们的标志性指标，通过分析 Token 消耗速率与模式，反映你的开发心流状态。
+
+## 🚀 核心功能
+
+- 📡 **Live Sniffer (实时嗅探)**: 实时监听 Codex CLI 管道，通过底层 Hook 捕获每一次补全事件。
+- 📊 **Matrix Dashboard (矩阵控制台)**: 基于 React + Vite 的高性能仪表盘，具备热力图、趋势图与实时日志。
+- ⚡ **AI Analytics (AI 分析)**: 深度分析 Input/Output Token，支持缓存 (Cached) 与推理 (Reasoning) 部分的分离监控。
+- 🔒 **Identity Core (身份核心)**: 完备的身份验证与权限管理，保护你的开发数据资产。
+
+## 🛠️ 快速开始
+
+### 安装
+
+只需一行命令，即可初始化环境：
+
+```bash
+npx --yes @vibescore/tracker init
+```
+
+### 同步与状态查看
+
+```bash
+# 同步最新的本地会话数据
+npx --yes @vibescore/tracker sync
+
+# 查看当前连接状态
+npx --yes @vibescore/tracker status
+```
+
+## 🏗️ 系统架构
+
+```mermaid
+graph TD
+    A[Codex CLI] -->|Rollout Logs| B(Tracker CLI)
+    B -->|AI Tokens| C{Core Relay}
+    C --> D[VibeScore Dashboard]
+    C --> E[AI Analytics Engine]
+```
+
+## 💻 开发者指南
+
+如果你想在本地运行或贡献代码：
+
+### 仪表盘开发
+
+```bash
+# 安装依赖
+cd dashboard
+npm install
+
+# 启动开发服务器
+npm run dev
+```
+
+### 整体架构验证
+
+```bash
+# 验证 Copy 注册表
+npm run validate:copy
+
+# 执行烟雾测试
+npm run smoke
+```
+
+## 📜 开源协议
+
+本项目基于 [MIT](LICENSE) 协议开源。
+
+---
+
+<div align="center">
+  <b>System_Ready // 2014 VibeScore OS</b><br/>
+  <i>"More Tokens. More Vibe."</i>
+</div>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vibescore/tracker",
-  "version": "0.0.3",
+  "version": "0.0.4",
   "description": "Codex CLI token usage tracker (macOS-first, notify-driven).",
   "license": "MIT",
   "publishConfig": {

package/src/commands/init.js

@@ -7,6 +7,7 @@ const { prompt, promptHidden } = require('../lib/prompt');
 const { upsertCodexNotify, loadCodexNotifyOriginal } = require('../lib/codex-config');
 const { beginBrowserAuth } = require('../lib/browser-auth');
 const { issueDeviceTokenWithPassword, issueDeviceTokenWithAccessToken } = require('../lib/insforge');
+const { cmdSync } = require('./sync');
 
 async function cmdInit(argv) {
   const opts = parseArgs(argv);
@@ -104,6 +105,13 @@ async function cmdInit(argv) {
       ''
     ].join('\n')
   );
+
+  try {
+    await cmdSync([]);
+  } catch (err) {
+    const msg = err && err.message ? err.message : 'unknown error';
+    process.stderr.write(`Initial sync failed: ${msg}\n`);
+  }
 }
 
 function parseArgs(argv) {

package/src/commands/sync.js

@@ -45,7 +45,7 @@ async function cmdSync(argv) {
     const rolloutFiles = await listRolloutFiles(sessionsDir);
 
     if (progress?.enabled) {
-      progress.start(`Parsing ${renderBar(0)} 0/${formatNumber(rolloutFiles.length)} files | queued 0`);
+      progress.start(`Parsing ${renderBar(0)} 0/${formatNumber(rolloutFiles.length)} files | buckets 0`);
     }
 
     const parseResult = await parseRolloutIncremental({
@@ -56,7 +56,9 @@ async function cmdSync(argv) {
         if (!progress?.enabled) return;
         const pct = p.total > 0 ? p.index / p.total : 1;
         progress.update(
-          `Parsing ${renderBar(pct)} ${formatNumber(p.index)}/${formatNumber(p.total)} files | queued ${formatNumber(p.eventsQueued)}`
+          `Parsing ${renderBar(pct)} ${formatNumber(p.index)}/${formatNumber(p.total)} files | buckets ${formatNumber(
+            p.bucketsQueued
+          )}`
         );
       }
     });
@@ -148,7 +150,7 @@ async function cmdSync(argv) {
         [
           'Sync finished:',
           `- Parsed files: ${parseResult.filesProcessed}`,
-          `- New events queued: ${parseResult.eventsQueued}`,
+          `- New 30-min buckets queued: ${parseResult.bucketsQueued}`,
           deviceToken
             ? `- Uploaded: ${uploadResult.inserted} inserted, ${uploadResult.skipped} skipped`
             : '- Uploaded: skipped (no device token)',

package/src/lib/rollout.js

@@ -2,7 +2,6 @@ const fs = require('node:fs/promises');
 const fssync = require('node:fs');
 const path = require('node:path');
 const readline = require('node:readline');
-const crypto = require('node:crypto');
 
 const { ensureDir } = require('./fs');
 
@@ -37,10 +36,16 @@ async function listRolloutFiles(sessionsDir) {
 async function parseRolloutIncremental({ rolloutFiles, cursors, queuePath, onProgress }) {
   await ensureDir(path.dirname(queuePath));
   let filesProcessed = 0;
-  let eventsQueued = 0;
+  let eventsAggregated = 0;
 
   const cb = typeof onProgress === 'function' ? onProgress : null;
   const totalFiles = Array.isArray(rolloutFiles) ? rolloutFiles.length : 0;
+  const hourlyState = normalizeHourlyState(cursors?.hourly);
+  const touchedBuckets = new Set();
+
+  if (!cursors.files || typeof cursors.files !== 'object') {
+    cursors.files = {};
+  }
 
   for (let idx = 0; idx < rolloutFiles.length; idx++) {
     const filePath = rolloutFiles[idx];
@@ -59,7 +64,8 @@ async function parseRolloutIncremental({ rolloutFiles, cursors, queuePath, onPro
       startOffset,
       lastTotal,
       lastModel,
-      queuePath
+      hourlyState,
+      touchedBuckets
     });
 
     cursors.files[key] = {
@@ -71,7 +77,7 @@ async function parseRolloutIncremental({ rolloutFiles, cursors, queuePath, onPro
     };
 
     filesProcessed += 1;
-    eventsQueued += result.eventsQueued;
+    eventsAggregated += result.eventsAggregated;
 
     if (cb) {
       cb({
@@ -79,28 +85,32 @@ async function parseRolloutIncremental({ rolloutFiles, cursors, queuePath, onPro
         total: totalFiles,
         filePath,
         filesProcessed,
-        eventsQueued
+        eventsAggregated,
+        bucketsQueued: touchedBuckets.size
       });
     }
   }
 
-  return { filesProcessed, eventsQueued };
+  const bucketsQueued = await enqueueTouchedBuckets({ queuePath, hourlyState, touchedBuckets });
+  hourlyState.updatedAt = new Date().toISOString();
+  cursors.hourly = hourlyState;
+
+  return { filesProcessed, eventsAggregated, bucketsQueued };
 }
 
-async function parseRolloutFile({ filePath, startOffset, lastTotal, lastModel, queuePath }) {
+async function parseRolloutFile({ filePath, startOffset, lastTotal, lastModel, hourlyState, touchedBuckets }) {
   const st = await fs.stat(filePath);
   const endOffset = st.size;
   if (startOffset >= endOffset) {
-    return { endOffset, lastTotal, lastModel, eventsQueued: 0 };
+    return { endOffset, lastTotal, lastModel, eventsAggregated: 0 };
   }
 
   const stream = fssync.createReadStream(filePath, { encoding: 'utf8', start: startOffset });
   const rl = readline.createInterface({ input: stream, crlfDelay: Infinity });
 
-  const toAppend = [];
   let model = typeof lastModel === 'string' ? lastModel : null;
   let totals = lastTotal && typeof lastTotal === 'object' ? lastTotal : null;
-  let eventsQueued = 0;
+  let eventsAggregated = 0;
 
   for await (const line of rl) {
     if (!line) continue;
@@ -139,26 +149,122 @@ async function parseRolloutFile({ filePath, startOffset, lastTotal, lastModel, q
       totals = totalUsage;
     }
 
-    const event = {
-      event_id: sha256Hex(line),
-      token_timestamp: tokenTimestamp,
-      model: model || null,
-      input_tokens: delta.input_tokens || 0,
-      cached_input_tokens: delta.cached_input_tokens || 0,
-      output_tokens: delta.output_tokens || 0,
-      reasoning_output_tokens: delta.reasoning_output_tokens || 0,
-      total_tokens: delta.total_tokens || 0
-    };
+    const bucketStart = toUtcHalfHourStart(tokenTimestamp);
+    if (!bucketStart) continue;
+
+    const bucket = getHourlyBucket(hourlyState, bucketStart);
+    addTotals(bucket.totals, delta);
+    touchedBuckets.add(bucketStart);
+    eventsAggregated += 1;
+  }
+
+  return { endOffset, lastTotal: totals, lastModel: model, eventsAggregated };
+}
+
+async function enqueueTouchedBuckets({ queuePath, hourlyState, touchedBuckets }) {
+  if (!touchedBuckets || touchedBuckets.size === 0) return 0;
 
-    toAppend.push(JSON.stringify(event));
-    eventsQueued += 1;
+  const toAppend = [];
+  for (const bucketStart of touchedBuckets) {
+    const bucket = hourlyState.buckets[bucketStart];
+    if (!bucket || !bucket.totals) continue;
+    const key = totalsKey(bucket.totals);
+    if (bucket.queuedKey === key) continue;
+    toAppend.push(
+      JSON.stringify({
+        hour_start: bucketStart,
+        input_tokens: bucket.totals.input_tokens,
+        cached_input_tokens: bucket.totals.cached_input_tokens,
+        output_tokens: bucket.totals.output_tokens,
+        reasoning_output_tokens: bucket.totals.reasoning_output_tokens,
+        total_tokens: bucket.totals.total_tokens
+      })
+    );
+    bucket.queuedKey = key;
   }
 
   if (toAppend.length > 0) {
     await fs.appendFile(queuePath, toAppend.join('\n') + '\n', 'utf8');
   }
 
-  return { endOffset, lastTotal: totals, lastModel: model, eventsQueued };
+  return toAppend.length;
+}
+
+function normalizeHourlyState(raw) {
+  const state = raw && typeof raw === 'object' ? raw : {};
+  const buckets = state.buckets && typeof state.buckets === 'object' ? state.buckets : {};
+  return {
+    version: 1,
+    buckets,
+    updatedAt: typeof state.updatedAt === 'string' ? state.updatedAt : null
+  };
+}
+
+function getHourlyBucket(state, hourStart) {
+  const buckets = state.buckets;
+  let bucket = buckets[hourStart];
+  if (!bucket || typeof bucket !== 'object') {
+    bucket = { totals: initTotals(), queuedKey: null };
+    buckets[hourStart] = bucket;
+    return bucket;
+  }
+
+  if (!bucket.totals || typeof bucket.totals !== 'object') {
+    bucket.totals = initTotals();
+  }
+
+  if (bucket.queuedKey != null && typeof bucket.queuedKey !== 'string') {
+    bucket.queuedKey = null;
+  }
+
+  return bucket;
+}
+
+function initTotals() {
+  return {
+    input_tokens: 0,
+    cached_input_tokens: 0,
+    output_tokens: 0,
+    reasoning_output_tokens: 0,
+    total_tokens: 0
+  };
+}
+
+function addTotals(target, delta) {
+  target.input_tokens += delta.input_tokens || 0;
+  target.cached_input_tokens += delta.cached_input_tokens || 0;
+  target.output_tokens += delta.output_tokens || 0;
+  target.reasoning_output_tokens += delta.reasoning_output_tokens || 0;
+  target.total_tokens += delta.total_tokens || 0;
+}
+
+function totalsKey(totals) {
+  return [
+    totals.input_tokens || 0,
+    totals.cached_input_tokens || 0,
+    totals.output_tokens || 0,
+    totals.reasoning_output_tokens || 0,
+    totals.total_tokens || 0
+  ].join('|');
+}
+
+function toUtcHalfHourStart(ts) {
+  const dt = new Date(ts);
+  if (!Number.isFinite(dt.getTime())) return null;
+  const minutes = dt.getUTCMinutes();
+  const halfMinute = minutes >= 30 ? 30 : 0;
+  const bucketStart = new Date(
+    Date.UTC(
+      dt.getUTCFullYear(),
+      dt.getUTCMonth(),
+      dt.getUTCDate(),
+      dt.getUTCHours(),
+      halfMinute,
+      0,
+      0
+    )
+  );
+  return bucketStart.toISOString();
 }
 
 function pickDelta(lastUsage, totalUsage, prevTotals) {
@@ -209,10 +315,6 @@ function normalizeUsage(u) {
   return out;
 }
 
-function sha256Hex(s) {
-  return crypto.createHash('sha256').update(s, 'utf8').digest('hex');
-}
-
 function isNonEmptyObject(v) {
   return Boolean(v && typeof v === 'object' && !Array.isArray(v) && Object.keys(v).length > 0);
 }

package/src/lib/upload-throttle.js

@@ -1,6 +1,6 @@
 const DEFAULTS = {
-  intervalMs: 10 * 60_000,
-  jitterMsMax: 120_000,
+  intervalMs: 30 * 60_000,
+  jitterMsMax: 60_000,
   backlogBytes: 1_000_000,
   batchSize: 300,
   maxBatchesSmall: 2,

package/src/lib/uploader.js

@@ -3,7 +3,7 @@ const fssync = require('node:fs');
 const readline = require('node:readline');
 
 const { ensureDir, readJson, writeJson } = require('./fs');
-const { ingestEvents } = require('./vibescore-api');
+const { ingestHourly } = require('./vibescore-api');
 
 async function drainQueueToCloud({ baseUrl, deviceToken, queuePath, queueStatePath, maxBatches, batchSize, onProgress }) {
   await ensureDir(require('node:path').dirname(queueStatePath));
@@ -16,14 +16,14 @@ async function drainQueueToCloud({ baseUrl, deviceToken, queuePath, queueStatePa
 
   const cb = typeof onProgress === 'function' ? onProgress : null;
   const queueSize = await safeFileSize(queuePath);
-  const maxEvents = Math.max(1, Math.floor(Number(batchSize || 200)));
+  const maxBuckets = Math.max(1, Math.floor(Number(batchSize || 200)));
 
   for (let batch = 0; batch < maxBatches; batch++) {
-    const res = await readBatch(queuePath, offset, maxEvents);
-    if (res.events.length === 0) break;
+    const res = await readBatch(queuePath, offset, maxBuckets);
+    if (res.buckets.length === 0) break;
 
-    attempted += res.events.length;
-    const ingest = await ingestEvents({ baseUrl, deviceToken, events: res.events });
+    attempted += res.buckets.length;
+    const ingest = await ingestHourly({ baseUrl, deviceToken, hourly: res.buckets });
     inserted += ingest.inserted || 0;
     skipped += ingest.skipped || 0;
 
@@ -47,33 +47,37 @@ async function drainQueueToCloud({ baseUrl, deviceToken, queuePath, queueStatePa
   return { inserted, skipped, attempted };
 }
 
-async function readBatch(queuePath, startOffset, maxEvents) {
+async function readBatch(queuePath, startOffset, maxBuckets) {
   const st = await fs.stat(queuePath).catch(() => null);
-  if (!st || !st.isFile()) return { events: [], nextOffset: startOffset };
-  if (startOffset >= st.size) return { events: [], nextOffset: startOffset };
+  if (!st || !st.isFile()) return { buckets: [], nextOffset: startOffset };
+  if (startOffset >= st.size) return { buckets: [], nextOffset: startOffset };
 
   const stream = fssync.createReadStream(queuePath, { encoding: 'utf8', start: startOffset });
   const rl = readline.createInterface({ input: stream, crlfDelay: Infinity });
 
-  const events = [];
+  const bucketMap = new Map();
   let offset = startOffset;
+  let linesRead = 0;
   for await (const line of rl) {
     const bytes = Buffer.byteLength(line, 'utf8') + 1;
     offset += bytes;
     if (!line.trim()) continue;
-    let ev;
+    let bucket;
     try {
-      ev = JSON.parse(line);
+      bucket = JSON.parse(line);
     } catch (_e) {
       continue;
     }
-    events.push(ev);
-    if (events.length >= maxEvents) break;
+    const hourStart = typeof bucket?.hour_start === 'string' ? bucket.hour_start : null;
+    if (!hourStart) continue;
+    bucketMap.set(hourStart, bucket);
+    linesRead += 1;
+    if (linesRead >= maxBuckets) break;
   }
 
   rl.close();
   stream.close?.();
-  return { events, nextOffset: offset };
+  return { buckets: Array.from(bucketMap.values()), nextOffset: offset };
 }
 
 async function safeFileSize(p) {

package/src/lib/vibescore-api.js

@@ -35,13 +35,13 @@ async function issueDeviceToken({ baseUrl, accessToken, deviceName, platform = '
   return { token, deviceId };
 }
 
-async function ingestEvents({ baseUrl, deviceToken, events }) {
+async function ingestHourly({ baseUrl, deviceToken, hourly }) {
   const data = await invokeFunctionWithRetry({
     baseUrl,
     accessToken: deviceToken,
     slug: 'vibescore-ingest',
     method: 'POST',
-    body: { events },
+    body: { hourly },
     errorPrefix: 'Ingest failed',
     retry: { maxRetries: 3, baseDelayMs: 500, maxDelayMs: 5000 }
   });
@@ -72,7 +72,7 @@ async function syncHeartbeat({ baseUrl, deviceToken }) {
 module.exports = {
   signInWithPassword,
   issueDeviceToken,
-  ingestEvents,
+  ingestHourly,
   syncHeartbeat
 };