tokentrace 0.5.1 → 0.6.0

package/CHANGELOG.md

@@ -4,6 +4,37 @@ All notable changes to TokenTrace are documented here.
 
 ## Unreleased
 
+## [0.6.0] - 2026-05-10
+
+### Added
+
+- 0.6.0 Stable Daily Tool roadmap with explicit release cards and gates.
+- Support matrix for stable, best-effort, ignored, and unsupported TokenTrace surfaces.
+- Scan Doctor support matrix and scan freshness status.
+- Overview first-run checklist that explains missing roots, no scans, zero imports, and next actions.
+- `npm run smoke:cli` for clean-home CLI checks across scan, serve, doctor, status, Claude status-line, and watch-mode commands.
+- `npm run smoke:packed` for installing the packed tarball into a temp project and verifying the published CLI entrypoint.
+
+### Changed
+
+- Claude Code and Codex JSONL parsers now keep valid records when a transcript contains malformed lines.
+- Model alias suggestions now handle OpenAI/Codex provider-prefixed and dated snapshot names.
+- Overview period filter keeps the custom date fields and Apply button visible on desktop while presets absorb overflow.
+- Local development config declares localhost/127.0.0.1 dev origins and disables the standard Next.js dev indicator preference.
+- README documents the support matrix and unsupported product boundaries.
+- `npm run release:check` now includes CLI and packed-install smoke checks before package security inspection.
+- The npm package now excludes generated `.next` output and prepares the dashboard build in the user's TokenTrace app-data directory on first `tokentrace serve`.
+- Package trust inspection now verifies the publish tarball directly and fails if generated Next.js build output is included.
+- Packed-install smoke now starts `tokentrace serve` from the packed tarball and allows dependency install scripts needed by native SQLite bindings while keeping TokenTrace itself free of lifecycle scripts.
+- Dev security posture is cleaner: Vitest is updated to the Node 18-compatible patched line, and the unused `drizzle-kit` dev dependency has been removed.
+- `npm run security:package` now audits the full dependency graph at moderate-or-higher severity instead of checking only production high-severity advisories.
+
+### Fixed
+
+- Package inspection no longer depends on the user's global npm cache, which can contain root-owned files on some machines.
+- First-run dashboard preparation from a packed install no longer hits SQLite `database is locked` errors while Next.js imports API routes during build.
+- `tokentrace serve` now exits nonzero when the underlying Next.js server fails before readiness.
+
 ## [0.5.1] - 2026-05-10
 
 ### Added

package/README.md

@@ -235,11 +235,12 @@ Stop the server with `Ctrl+C` in the terminal where `tokentrace` is running.
 
 ## Package Trust
 
-- The npm package has no `preinstall`, `install`, or `postinstall` scripts.
-- Published Next.js server bundles are intentionally left readable instead of server-minified so Socket, npm, maintainers, and users can inspect generated runtime code.
-- `npm run package:inspect` fails if generated app route bundles look packed or unreadable.
+- The TokenTrace npm package has no `preinstall`, `install`, or `postinstall` scripts.
+- The published package ships readable application source and the compiled CLI runtime, not generated `.next/server` route bundles.
+- `tokentrace serve` prepares the local dashboard build in the user's TokenTrace app-data directory the first time it is needed.
+- `npm run package:inspect` fails if generated Next.js build output appears in the published tarball.
 - Public npm publishing is configured through GitHub Actions Trusted Publishing and provenance from version tags.
-- Socket GitHub checks and ProjScan are used as release guardrails, alongside `npm audit --omit=dev --audit-level=high`.
+- Socket GitHub checks and ProjScan are used as release guardrails, alongside `npm audit --audit-level=moderate`.
 - Release notes are published directly in GitHub Releases from the relevant changelog section, not as a link-only summary.
 
 See [SECURITY.md](SECURITY.md) for the full security and privacy model.
@@ -291,6 +292,21 @@ Adapters live under `src/ingestion/adapters/`:
 
 Formats for Claude Code and Codex CLI can vary across versions, so these adapters are defensive and best-effort. Unknown files fail safely and show warnings in the Raw Data page.
 
+## Support Matrix
+
+TokenTrace keeps a visible support contract so daily scans are easier to trust:
+
+| Surface | Support level | Notes |
+| --- | --- | --- |
+| Claude Code project transcripts | Stable | Primary local CLI ingestion source. |
+| Codex CLI session artifacts | Best-effort | Parsed defensively while CLI formats evolve. |
+| Generic JSONL, JSON, and text logs | Best-effort | Conservative usage-shaped records only. |
+| Claude/Codex cache, plugin, todo, config, and support files | Ignored | Tracked as non-usage files, not parser failures. |
+| Editable model pricing | Stable | Local pricing rows drive costs and unknown-cost repair queues. |
+| Claude Code status line | Stable | Uses Claude Code's documented statusLine stdin contract. |
+| Codex sticky status line | Best-effort fallback | Use `tokentrace watch --session --compact` in a split or tmux pane. |
+| Desktop app scraping, browser extensions, proxying, packet capture, telemetry | Unsupported | Outside TokenTrace's product boundary. |
+
 ## Extending Parsers
 
 Example generic JSONL fixtures are in `fixtures/generic-jsonl/`.

package/SECURITY.md

@@ -8,14 +8,17 @@ downloads.
 
 - No telemetry, cloud sync, traffic interception, proxying, packet sniffing, or
   browser extension is part of the product.
-- No `preinstall`, `install`, or `postinstall` npm lifecycle scripts are used.
+- The TokenTrace package has no `preinstall`, `install`, or `postinstall`
+  npm lifecycle scripts.
 - Runtime state is stored locally in the user's TokenTrace app-data directory.
 - Raw full prompts and responses are off by default.
 - Pricing refresh downloads a public model-pricing manifest only. It does not
   send usage logs, prompts, file paths, analytics, or identifiers. Set
   `TOKENTRACE_DISABLE_PRICE_REFRESH=1` to use bundled prices only.
-- Generated Next.js server bundles are intentionally published without server
-  minification so package scanners and maintainers can inspect them.
+- The published package ships readable application source and the compiled CLI
+  runtime, not generated `.next/server` route bundles. `tokentrace serve`
+  prepares the local dashboard build in the user's app-data directory when
+  needed.
 
 ## What TokenTrace Reads
 

package/app/diagnostics/page.tsx

@@ -176,6 +176,13 @@ function DoctorReportPanel({ report }: { report: DoctorReport }) {
           </div>
         </div>
 
+        <div className="rounded-md border bg-muted/30 p-3">
+          <div className="text-sm font-semibold">Scan freshness</div>
+          <div className="mt-1 text-xs leading-relaxed text-muted-foreground">
+            {report.scanFreshness.description}
+          </div>
+        </div>
+
         {report.latestScan.zeroImportExplanation ? (
           <div className="rounded-md border border-amber-300 bg-amber-50 p-3 text-sm text-amber-950">
             {report.latestScan.zeroImportExplanation}
@@ -224,6 +231,31 @@ function DoctorReportPanel({ report }: { report: DoctorReport }) {
             ))}
           </div>
         </div>
+
+        <div className="space-y-3">
+          <div>
+            <div className="text-sm font-semibold">Support matrix</div>
+            <div className="mt-1 text-xs text-muted-foreground">
+              {report.supportMatrix.summary.stable.toLocaleString()} stable,{" "}
+              {report.supportMatrix.summary.bestEffort.toLocaleString()} best-effort,{" "}
+              {report.supportMatrix.summary.ignored.toLocaleString()} ignored,{" "}
+              {report.supportMatrix.summary.unsupported.toLocaleString()} unsupported.
+            </div>
+          </div>
+          <div className="grid divide-y border-y lg:grid-cols-2 lg:divide-x lg:divide-y-0">
+            {report.supportMatrix.items.map((item) => (
+              <div key={item.id} className="p-3">
+                <div className="flex flex-wrap items-center gap-2">
+                  <div className="text-sm font-semibold">{item.label}</div>
+                  <Badge variant={item.level === "stable" ? "success" : item.level === "unsupported" ? "destructive" : item.level === "best-effort" ? "warning" : "secondary"}>
+                    {item.level}
+                  </Badge>
+                </div>
+                <div className="mt-1 text-xs leading-relaxed text-muted-foreground">{item.description}</div>
+              </div>
+            ))}
+          </div>
+        </div>
       </CardContent>
     </Card>
   );

package/app/page.tsx

@@ -1,6 +1,5 @@
 import Link from "next/link";
 import { ArrowRight, Coins, Database, MessageSquare, Minus, Sparkles, TrendingDown, TrendingUp } from "lucide-react";
-import { EmptyState } from "@/components/empty-state";
 import { RankBarChart } from "@/components/charts/rank-bar-chart";
 import { TrendChart } from "@/components/charts/trend-chart";
 import { PeriodFilter } from "@/components/period-filter";
@@ -10,7 +9,10 @@ import { Card, CardContent, CardDescription, CardHeader, CardTitle } from "@/com
 import { HelpTooltip } from "@/components/ui/help-tooltip";
 import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from "@/components/ui/table";
 import { DataValue, FieldLabel, MonoText, PageHeader } from "@/components/ui/typography";
-import { getAnalyticsData } from "@/src/lib/analytics";
+import { getAnalyticsData, getScanTrustData } from "@/src/lib/analytics";
+import { buildDoctorReport } from "@/src/lib/doctor";
+import { getDefaultSearchRoots } from "@/src/ingestion/discovery";
+import { buildFirstRunStatus, type FirstRunStatus } from "@/src/lib/first-run-status";
 import { resolveDateRange } from "@/src/lib/date-range";
 import { formatCurrency, formatTokens, percent } from "@/src/lib/format";
 import { cn } from "@/src/lib/utils";
@@ -125,6 +127,39 @@ function DeltaMetric({
   );
 }
 
+function FirstRunPanel({ status }: { status: FirstRunStatus }) {
+  return (
+    <Card className={status.tone === "warning" ? "border-amber-300 bg-amber-50/50" : undefined}>
+      <CardHeader className="flex flex-col gap-3 lg:flex-row lg:items-start lg:justify-between">
+        <div>
+          <CardTitle>{status.title}</CardTitle>
+          <CardDescription>{status.description}</CardDescription>
+        </div>
+        <Button asChild variant={status.tone === "warning" ? "outline" : "default"}>
+          <Link href={status.primaryAction.href}>
+            {status.primaryAction.label} <ArrowRight className="h-4 w-4" />
+          </Link>
+        </Button>
+      </CardHeader>
+      <CardContent>
+        <div className="grid divide-y border-y md:grid-cols-5 md:divide-x md:divide-y-0">
+          {status.checks.map((check) => (
+            <div key={check.id} className="p-3">
+              <div className="flex flex-wrap items-center gap-2">
+                <div className="text-sm font-semibold">{check.label}</div>
+                <Badge variant={check.state === "pass" ? "success" : check.state === "warn" ? "warning" : "secondary"}>
+                  {check.state}
+                </Badge>
+              </div>
+              <div className="mt-1 text-xs leading-relaxed text-muted-foreground">{check.detail}</div>
+            </div>
+          ))}
+        </div>
+      </CardContent>
+    </Card>
+  );
+}
+
 type OverviewPageProps = {
   searchParams?: Promise<Record<string, string | string[] | undefined>>;
 };
@@ -133,6 +168,22 @@ export default async function OverviewPage({ searchParams }: OverviewPageProps)
   const params = (await searchParams) ?? {};
   const range = resolveDateRange(params);
   const data = getAnalyticsData(range.filters);
+  const trust = getScanTrustData();
+  const roots = await getDefaultSearchRoots();
+  const doctorReport = buildDoctorReport({ ...trust, roots });
+  const firstRunStatus = buildFirstRunStatus({
+    rootCount: roots.length,
+    pricedModelCount: trust.pricedModelCount,
+    latestScan: doctorReport.latestScan.id
+      ? {
+          filesScanned: doctorReport.latestScan.filesScanned,
+          recordsImported: doctorReport.latestScan.recordsImported,
+          zeroImportExplanation: doctorReport.latestScan.zeroImportExplanation
+        }
+      : null,
+    interactions: trust.confidence.interactions,
+    unknownCostInteractions: trust.confidence.unknownCostInteractions
+  });
   const { summary } = data;
 
   return (
@@ -152,10 +203,7 @@ export default async function OverviewPage({ searchParams }: OverviewPageProps)
       <PeriodFilter range={range} />
 
       {summary.interactions === 0 ? (
-        <EmptyState
-          title="No usage imported yet"
-          description="Add custom folders if needed, run a scan from Settings, then return here for analytics."
-        />
+        <FirstRunPanel status={firstRunStatus} />
       ) : null}
 
       <Card>

package/bin/tokentrace.js

@@ -94,6 +94,96 @@ function nextBin() {
   }
 }
 
+function dashboardBuildId() {
+  return path.join(dashboardWorkdir(), ".next", "BUILD_ID");
+}
+
+function dashboardWorkdir() {
+  return path.join(appDataDir(), "dashboard-runtime");
+}
+
+function dashboardBuildMarker() {
+  return path.join(dashboardWorkdir(), ".tokentrace-dashboard-version");
+}
+
+function dependencyModulesDir() {
+  const localNodeModules = path.join(packageRoot, "node_modules");
+  if (fs.existsSync(localNodeModules)) return localNodeModules;
+
+  const parent = path.dirname(packageRoot);
+  if (path.basename(parent) === "node_modules") return parent;
+
+  return localNodeModules;
+}
+
+function copyDashboardSource(targetRoot) {
+  const directories = ["app", "components", "pricing", "public", "src"];
+  const files = [
+    "components.json",
+    "next.config.mjs",
+    "package.json",
+    "postcss.config.mjs",
+    "tailwind.config.ts",
+    "tsconfig.json"
+  ];
+
+  fs.rmSync(targetRoot, { recursive: true, force: true });
+  fs.mkdirSync(targetRoot, { recursive: true });
+
+  for (const directory of directories) {
+    const source = path.join(packageRoot, directory);
+    if (fs.existsSync(source)) {
+      fs.cpSync(source, path.join(targetRoot, directory), { recursive: true });
+    }
+  }
+
+  for (const file of files) {
+    const source = path.join(packageRoot, file);
+    if (fs.existsSync(source)) {
+      fs.copyFileSync(source, path.join(targetRoot, file));
+    }
+  }
+
+  const nodeModulesTarget = dependencyModulesDir();
+  const nodeModulesLink = path.join(targetRoot, "node_modules");
+  fs.symlinkSync(
+    nodeModulesTarget,
+    nodeModulesLink,
+    process.platform === "win32" ? "junction" : "dir"
+  );
+}
+
+function runNextBuild(cwd) {
+  return new Promise((resolve, reject) => {
+    const child = spawn(process.execPath, [nextBin(), "build"], {
+      cwd,
+      env: runtimeEnv(),
+      stdio: "inherit"
+    });
+    child.on("error", reject);
+    child.on("exit", (code) => {
+      if (code === 0) resolve();
+      else reject(new Error(`next build exited with code ${code}`));
+    });
+  });
+}
+
+async function ensureDashboardBuild() {
+  const workdir = dashboardWorkdir();
+  const marker = dashboardBuildMarker();
+  const builtVersion = fs.existsSync(marker) ? fs.readFileSync(marker, "utf8").trim() : null;
+  if (fs.existsSync(dashboardBuildId()) && builtVersion === packageJson.version) {
+    return workdir;
+  }
+
+  console.log("Preparing TokenTrace dashboard for this install...");
+  console.log("This runs locally and may take a moment the first time.");
+  copyDashboardSource(workdir);
+  await runNextBuild(workdir);
+  fs.writeFileSync(marker, `${packageJson.version}\n`);
+  return workdir;
+}
+
 function runtimeScriptPath(scriptName) {
   const compiled = path.join(packageRoot, "dist", "runtime", `${scriptName}.mjs`);
   if (fs.existsSync(compiled)) return compiled;
@@ -220,13 +310,8 @@ async function serve(args = []) {
     return;
   }
 
-  const buildId = path.join(packageRoot, ".next", "BUILD_ID");
-  if (!fs.existsSync(buildId)) {
-    console.error("TokenTrace is not built yet. Run `npm run build` before using the package CLI from a source checkout.");
-    process.exit(1);
-  }
-
   await initializeDatabase();
+  const dashboardRoot = await ensureDashboardBuild();
   const hostname = options.hostname;
   const port = options.port ?? (await getPort({ port: portNumbers(3030, 3999), host: hostname }));
   const url = `http://${hostname}:${port}`;
@@ -238,7 +323,7 @@ async function serve(args = []) {
     process.execPath,
     [nextBin(), "start", "--hostname", hostname, "--port", String(port)],
     {
-      cwd: packageRoot,
+      cwd: dashboardRoot,
       env: {
         ...runtimeEnv(),
         PORT: String(port),
@@ -263,6 +348,8 @@ async function serve(args = []) {
     }
   } catch (error) {
     console.error(error instanceof Error ? error.message : "Failed to start TokenTrace.");
+    stop();
+    process.exit(1);
   }
 
   child.on("exit", (code) => process.exit(code ?? 0));

package/components/period-filter.tsx

@@ -16,36 +16,38 @@ export function PeriodFilter({ range }: { range: ResolvedDateRange }) {
     <div className="rounded-lg bg-card p-3 outline outline-1 outline-border sm:p-4">
       <form className="overflow-x-auto" action="/">
         <input type="hidden" name="range" value="custom" />
-        <div className="flex min-w-max items-center gap-2">
-          <div className="flex items-center gap-2 pr-1 text-sm font-semibold">
+        <div className="flex min-w-[720px] items-center gap-2">
+          <div className="flex shrink-0 items-center gap-2 pr-1 text-sm font-semibold">
             <CalendarDays className="h-4 w-4 shrink-0 text-muted-foreground" />
             <span>Period</span>
-            <span className="whitespace-nowrap rounded-md bg-muted px-2 py-0.5 text-xs font-medium text-muted-foreground">
+            <span className="hidden whitespace-nowrap rounded-md bg-muted px-2 py-0.5 text-xs font-medium text-muted-foreground 2xl:inline-flex">
               {statusLabel}
             </span>
           </div>
           <div className="h-6 w-px shrink-0 bg-border" />
-          <div className="flex items-center gap-1.5">
-            {dateRangeOptions.map((option) => (
-              <Button
-                key={option.key}
-                asChild
-                size="sm"
-                variant={range.key === option.key ? "default" : "outline"}
-              >
-                <Link href={rangeHref(option.key)}>{option.label}</Link>
-              </Button>
-            ))}
+          <div className="min-w-0 flex-1 overflow-x-auto">
+            <div className="flex w-max items-center gap-1.5 pr-1">
+              {dateRangeOptions.map((option) => (
+                <Button
+                  key={option.key}
+                  asChild
+                  size="sm"
+                  variant={range.key === option.key ? "default" : "outline"}
+                >
+                  <Link href={rangeHref(option.key)}>{option.label}</Link>
+                </Button>
+              ))}
+            </div>
           </div>
           <div className="h-6 w-px shrink-0 bg-border" />
-          <div className="flex items-center gap-1.5">
+          <div className="flex shrink-0 items-center gap-1.5">
             <label className="flex items-center gap-1.5 text-xs text-muted-foreground">
               <span>From</span>
-              <Input type="date" name="from" defaultValue={range.fromInput} className="period-date-input h-8 w-36" />
+              <Input type="date" name="from" defaultValue={range.fromInput} className="period-date-input h-8 w-32" />
             </label>
             <label className="flex items-center gap-1.5 text-xs text-muted-foreground">
               <span>To</span>
-              <Input type="date" name="to" defaultValue={range.toInput} className="period-date-input h-8 w-36" />
+              <Input type="date" name="to" defaultValue={range.toInput} className="period-date-input h-8 w-32" />
             </label>
             <Button size="sm" type="submit" variant={range.key === "custom" ? "default" : "outline"}>
               Apply

package/components/settings-panel.tsx

@@ -164,7 +164,7 @@ export function SettingsPanel({
             {
               label: "Install scripts",
               value: "None",
-              detail: "No preinstall, install, or postinstall lifecycle scripts."
+              detail: "The TokenTrace package has no preinstall, install, or postinstall lifecycle scripts."
             },
             {
               label: "Network behavior",
@@ -174,7 +174,7 @@ export function SettingsPanel({
             {
               label: "Release proof",
               value: "Tag based",
-              detail: "Future npm releases publish through GitHub Trusted Publishing."
+              detail: "npm releases publish through GitHub Trusted Publishing."
             }
           ].map((item) => (
             <div key={item.label} className="p-3">

package/dist/runtime/db-migrate.mjs

@@ -410,6 +410,7 @@ function databaseUrlPath(value) {
 var dbPath = process.env.TOKENTRACE_DB ?? databaseUrlPath(process.env.DATABASE_URL) ?? defaultDbPath;
 fs.mkdirSync(path.dirname(dbPath), { recursive: true });
 var sqlite = new Database(dbPath);
+sqlite.pragma("busy_timeout = 10000");
 sqlite.pragma("foreign_keys = ON");
 applyMigrations(sqlite);
 var db = drizzle(sqlite, { schema: schema_exports });

package/dist/runtime/db-seed.mjs

@@ -410,6 +410,7 @@ function databaseUrlPath(value) {
 var dbPath = process.env.TOKENTRACE_DB ?? databaseUrlPath(process.env.DATABASE_URL) ?? defaultDbPath;
 fs.mkdirSync(path.dirname(dbPath), { recursive: true });
 var sqlite = new Database(dbPath);
+sqlite.pragma("busy_timeout = 10000");
 sqlite.pragma("foreign_keys = ON");
 applyMigrations(sqlite);
 var db = drizzle(sqlite, { schema: schema_exports });
@@ -1230,11 +1231,23 @@ function addClaudeCandidates(name, candidates) {
     candidates.push(`${prefix}-${family}-${major}.${minor}`);
   }
 }
+function addProviderPrefixCandidates(name, candidates) {
+  const withoutProvider = name.replace(/^[a-z0-9_.-]+\//i, "");
+  if (withoutProvider !== name) candidates.push(withoutProvider);
+}
+function addSnapshotDateCandidates(name, candidates) {
+  const withoutProvider = name.replace(/^[a-z0-9_.-]+\//i, "");
+  const lower = withoutProvider.toLowerCase();
+  const stripped = lower.replace(/[-_.]\d{4}[-_.]?\d{2}[-_.]?\d{2}$/, "");
+  if (stripped !== lower) candidates.push(stripped);
+}
 function modelNameCandidates(modelName) {
   const trimmed = modelName?.trim();
   if (!trimmed) return ["unknown"];
   const candidates = [trimmed];
+  addProviderPrefixCandidates(trimmed, candidates);
   addClaudeCandidates(trimmed, candidates);
+  addSnapshotDateCandidates(trimmed, candidates);
   return unique(candidates);
 }