@klars/agentobs 0.1.6 → 0.2.1

package/README.md

@@ -36,16 +36,31 @@ account, no cloud, no telemetry.
 ```bash
 npm install -g @klars/agentobs
 agentobs init
+agentobs import
 ```
 
-`init` prints a hook configuration block. Paste it into `~/.claude/settings.json`
-(or a project's `.claude/settings.json`), then:
+`import` reads Claude Code's own session transcripts from `~/.claude/projects/`
+and backfills everything you have already done — no configuration, no hooks.
+Then:
 
 ```bash
 agentobs dashboard
 ```
 
-Run Claude Code as usual. Tool calls appear in the dashboard within seconds.
+That is the fastest path to real data, and the one to try first.
+
+### Live capture (optional)
+
+`import` is after-the-fact. To record calls **as they happen** — and to let
+guardrails actually block them — add the hook configuration that
+`agentobs init` prints to `~/.claude/settings.json`, then restart Claude Code.
+
+> **If hooks record nothing:** this has been observed on at least one Windows
+> install, where Claude Code did not invoke the configured command at all — a
+> plain two-line `.cmd` file also never fired, so it is not specific to
+> AgentObs. Check by running any tool and then `agentobs stats --today`. If it
+> stays at zero, keep using `agentobs import`, which needs no hooks; only
+> guardrail *blocking* depends on them.
 
 ---
 
@@ -85,6 +100,7 @@ Everything lives in `~/.agentobs/`. Uninstalling is `rm -rf ~/.agentobs`.
 
 ```
 agentobs init                        Set up ~/.agentobs and print the hook config
+agentobs import [--days n] [--all]   Import Claude Code transcripts (no hooks needed)
 agentobs dashboard [--port] [--host] Serve the dashboard (default 127.0.0.1:4300)
 agentobs stats [--today] [--since]   Print totals in the terminal
 agentobs run -- <command...>         Observe any command (coarse detail)
@@ -148,22 +164,37 @@ Two deliberate behaviours worth knowing:
 
 ## Agent support
 
-| Agent           | How                        | Detail                                            |
-| --------------- | -------------------------- | ------------------------------------------------- |
-| **Claude Code** | Native hooks               | **Rich** — every tool call, plus policy enforcement |
-| Any CLI agent   | `agentobs run -- <cmd>`    | **Coarse** — duration and exit code only          |
-| Custom / in-house | `agentobs watch <file>`  | **Rich**, if it writes JSONL                      |
+| Agent | How | Detail | Needs setup? |
+| --- | --- | --- | --- |
+| **Claude Code** | `agentobs import` | **Rich** — every tool call, tokens, cost | **No** |
+| **Claude Code** | Native hooks | **Rich**, live, and can *block* calls | Yes — hook config |
+| Any CLI agent | `agentobs run -- <cmd>` | **Coarse** — duration and exit code only | No |
+| Custom / in-house | `agentobs watch <file>` | **Rich**, if it writes JSONL | No |
+
+`import` and hooks read the same underlying data. The difference is timing:
+hooks see a call *before* it runs, which is what makes blocking possible;
+`import` reads the transcript afterwards. If you only want observability,
+`import` is enough and needs no configuration.
 
 The dashboard labels coarse sessions as `coarse` rather than implying detail it
-does not have.
+does not have, and `agentobs stats` explains why a coarse-only range shows zero
+tool calls.
 
 ### A note on cost accuracy
 
-Claude Code's `PostToolUse` hook payload carries **no token or cost fields**.
-AgentObs therefore reads token usage from the session transcript at
-`SessionEnd`, which makes **session-level cost accurate** but leaves
-**per-tool-call cost blank** for hook-sourced data. It does not divide a total
-across calls to manufacture a number.
+Claude Code's `PostToolUse` hook payload carries **no token or cost fields**,
+so token usage comes from the session transcript — at `SessionEnd` for hooks,
+or directly via `agentobs import`. That makes **session-level cost accurate**
+while leaving **per-tool-call cost blank**: usage is reported per assistant
+message, not per tool call, and dividing a total across calls would be a
+manufactured number.
+
+**Cache tokens dominate a long session.** A cached conversation replays its
+whole context on every turn, so `cache_read` can reach hundreds of millions of
+tokens in a single session. AgentObs tracks cache reads (billed at 0.1x input)
+and cache writes (1.25x) separately from fresh tokens, and `agentobs import`
+prints the four lines separately — a single unexplained total looks like a bug
+when the cache line legitimately dwarfs everything else.
 
 Model prices live in `~/.agentobs/pricing.json` and are yours to edit. A model
 missing from that file shows cost as `—`, never `$0.00`.

package/dist/adapters/claude-transcript.js

@@ -0,0 +1,213 @@
+/**
+ * Claude Code transcript adapter - the hook-free path to rich data.
+ *
+ * Claude Code writes a JSONL transcript per session under
+ * `~/.claude/projects/<slug>/<session-id>.jsonl`, containing every assistant
+ * message with its `usage` block and every `tool_use` / `tool_result` pair.
+ * That is the same information the hooks would deliver, already on disk.
+ *
+ * Why this exists: hooks are the intended integration, but they depend on
+ * Claude Code actually invoking the configured command, which cannot be
+ * verified from inside AgentObs and has been observed silently not happening.
+ * Reading the transcript needs no configuration at all, so `agentobs import`
+ * works on a machine where hooks do not.
+ *
+ * Trade-off against hooks, stated plainly: this is after-the-fact. It cannot
+ * block a tool call, so guardrails still require the hook. It also attributes
+ * tokens per assistant message rather than per tool call, so per-call cost
+ * stays null rather than being invented by dividing a total.
+ */
+import { createReadStream, existsSync, readdirSync, statSync } from 'node:fs';
+import { createInterface } from 'node:readline';
+import { homedir } from 'node:os';
+import { join } from 'node:path';
+import { computeCost } from '../core/pricing.js';
+import { beginToolCall, completeToolCall, endSession, ensureSession, rollUpSession, startSession, } from '../core/repo.js';
+/** Root of Claude Code's per-project transcript directories. */
+export function transcriptRoot() {
+    return process.env.CLAUDE_CONFIG_DIR
+        ? join(process.env.CLAUDE_CONFIG_DIR, 'projects')
+        : join(homedir(), '.claude', 'projects');
+}
+/** Every transcript on this machine, newest first. */
+export function findTranscripts(root = transcriptRoot()) {
+    if (!existsSync(root))
+        return [];
+    const out = [];
+    for (const project of readdirSync(root)) {
+        const dir = join(root, project);
+        let entries;
+        try {
+            if (!statSync(dir).isDirectory())
+                continue;
+            entries = readdirSync(dir);
+        }
+        catch {
+            continue; // unreadable directory - skip rather than abort the scan
+        }
+        for (const name of entries) {
+            if (!name.endsWith('.jsonl'))
+                continue;
+            const path = join(dir, name);
+            try {
+                const st = statSync(path);
+                out.push({
+                    path,
+                    sessionId: name.replace(/\.jsonl$/, ''),
+                    project,
+                    modifiedAt: st.mtimeMs,
+                    sizeBytes: st.size,
+                });
+            }
+            catch {
+                /* vanished between readdir and stat - ignore */
+            }
+        }
+    }
+    return out.sort((a, b) => b.modifiedAt - a.modifiedAt);
+}
+/**
+ * Cache reads are billed at about a tenth of the normal input rate. Applying
+ * the full rate to a long session's replayed context overstates it by roughly
+ * an order of magnitude.
+ */
+const CACHE_READ_RATE = 0.1;
+/** A cache write bills at 1.25x the normal input rate. */
+const CACHE_WRITE_RATE = 1.25;
+/**
+ * Imports one transcript into the local database.
+ *
+ * Idempotent: session and tool-call ids come from the transcript itself, and
+ * the repository layer inserts with ON CONFLICT DO NOTHING, so re-importing a
+ * growing transcript adds only what is new. That matters because a session's
+ * file keeps being appended to while the session is live.
+ */
+export async function importTranscript(db, file) {
+    const rl = createInterface({
+        input: createReadStream(file.path, { encoding: 'utf8' }),
+        crlfDelay: Infinity,
+    });
+    const result = {
+        sessionId: file.sessionId,
+        toolCalls: 0,
+        tokensIn: 0,
+        tokensOut: 0,
+        cacheReadTokens: 0,
+        cacheWriteTokens: 0,
+        cost: null,
+        model: null,
+        startedAt: null,
+        endedAt: null,
+    };
+    // tool_use appears on an assistant message; its tool_result arrives later on
+    // a user message. Hold the open calls so the pair can be joined.
+    const pending = new Map();
+    let sessionStarted = false;
+    let cwd = null;
+    for await (const line of rl) {
+        if (!line.trim())
+            continue;
+        let row;
+        try {
+            row = JSON.parse(line);
+        }
+        catch {
+            continue; // a partially-flushed final line is normal on a live session
+        }
+        const timestamp = typeof row.timestamp === 'string' ? row.timestamp : null;
+        if (timestamp) {
+            if (!result.startedAt)
+                result.startedAt = timestamp;
+            result.endedAt = timestamp;
+        }
+        if (typeof row.cwd === 'string' && !cwd)
+            cwd = row.cwd;
+        if (!sessionStarted && timestamp) {
+            startSession(db, {
+                id: file.sessionId,
+                agentName: 'claude-code',
+                cwd,
+                fidelity: 'rich',
+                startedAt: timestamp,
+            });
+            sessionStarted = true;
+        }
+        const message = (row.message ?? {});
+        const usage = message.usage;
+        if (usage && typeof usage === 'object') {
+            // Only genuinely fresh input tokens go in tokensIn. Cache writes and
+            // reads are tracked separately because they bill at different rates,
+            // and because a cache read replays the whole context every turn - the
+            // headline token count must not include the same tokens hundreds of
+            // times.
+            result.tokensIn += usage.input_tokens ?? 0;
+            result.tokensOut += usage.output_tokens ?? 0;
+            result.cacheWriteTokens += usage.cache_creation_input_tokens ?? 0;
+            // cache_read is the whole conversation context replayed on every single
+            // message, so it re-counts the same tokens on each turn - summing it
+            // reported 413 million tokens for one session. It is tracked separately
+            // and priced at the cache-read rate, never added to tokens_in, which
+            // would make the headline token count meaningless.
+            result.cacheReadTokens += usage.cache_read_input_tokens ?? 0;
+            if (typeof message.model === 'string')
+                result.model = message.model;
+        }
+        const content = message.content;
+        if (!Array.isArray(content))
+            continue;
+        for (const block of content) {
+            if (block?.type === 'tool_use' && typeof block.id === 'string') {
+                const name = typeof block.name === 'string' ? block.name : 'unknown';
+                const at = timestamp ?? new Date().toISOString();
+                ensureSession(db, file.sessionId, 'claude-code', cwd);
+                beginToolCall(db, {
+                    id: block.id,
+                    sessionId: file.sessionId,
+                    toolName: name,
+                    input: block.input,
+                    startedAt: at,
+                });
+                pending.set(block.id, { name, startedAt: at });
+                result.toolCalls += 1;
+            }
+            if (block?.type === 'tool_result' && typeof block.tool_use_id === 'string') {
+                const isError = block.is_error === true;
+                completeToolCall(db, block.tool_use_id, {
+                    status: isError ? 'error' : 'success',
+                    output: block.content,
+                    errorMessage: isError ? String(block.content).slice(0, 300) : null,
+                    endedAt: timestamp ?? undefined,
+                    // No per-call tokens: usage is reported per assistant message, and
+                    // splitting it across calls would be a fabricated number.
+                });
+                pending.delete(block.tool_use_id);
+            }
+        }
+    }
+    if (!sessionStarted)
+        return result;
+    const baseCost = computeCost(result.model, result.tokensIn, result.tokensOut);
+    const readCost = computeCost(result.model, result.cacheReadTokens, 0);
+    const writeCost = computeCost(result.model, result.cacheWriteTokens, 0);
+    result.cost =
+        baseCost === null
+            ? null
+            : baseCost +
+                (readCost ?? 0) * CACHE_READ_RATE +
+                (writeCost ?? 0) * CACHE_WRITE_RATE;
+    // Session totals come from the transcript's own usage blocks, which are
+    // authoritative - the per-call rows have no tokens to sum.
+    db.prepare(`UPDATE sessions
+        SET total_tokens_in = ?, total_tokens_out = ?, total_cost_usd = ?,
+            ended_at = COALESCE(?, ended_at), updated_at = ?, synced_at = NULL
+      WHERE id = ?`).run(result.tokensIn, result.tokensOut, result.cost, result.endedAt, new Date().toISOString(), file.sessionId);
+    // Recompute counts from the rows just inserted, then restore the token
+    // totals, which rollUpSession would otherwise zero out (it sums per-call
+    // tokens, and those are deliberately null here).
+    rollUpSession(db, file.sessionId);
+    db.prepare(`UPDATE sessions SET total_tokens_in = ?, total_tokens_out = ?, total_cost_usd = ? WHERE id = ?`).run(result.tokensIn, result.tokensOut, result.cost, file.sessionId);
+    if (result.endedAt)
+        endSession(db, file.sessionId, { endedAt: result.endedAt });
+    return result;
+}
+//# sourceMappingURL=claude-transcript.js.map

package/dist/cli.js

@@ -46,7 +46,7 @@ export function buildProgram() {
     });
     program
         .command('watch')
-        .argument('<file>', 'JSONL file to tail')
+        .argument('[file]', 'JSONL file to tail')
         .description('Ingest a newline-delimited JSON agent log')
         .addHelpText('after', `
 Example:
@@ -78,6 +78,29 @@ On Windows, cmd.exe builtins (dir, echo, type) need: agentobs run -- cmd /c dir`
         const { run } = await import('./commands/run.js');
         await run(command ?? [], opts);
     });
+    program
+        .command('import')
+        .description("Import Claude Code's own transcripts (no hook setup needed)")
+        .option('--days <n>', 'only sessions modified in the last n days', '7')
+        .option('--all', 'import every transcript found, however old', false)
+        .option('--dry-run', 'list what would be imported, write nothing', false)
+        .option('--session <id>', 'import one specific session id')
+        .addHelpText('after', `
+Claude Code writes a JSONL transcript per session under
+~/.claude/projects/. This reads them directly, so it works even when
+hooks are not firing - and it backfills everything you have already done.
+
+Examples:
+  agentobs import                 last 7 days
+  agentobs import --all           everything
+  agentobs import --dry-run       show what would be imported
+
+Historical data cannot be blocked retroactively; guardrails still need
+the PreToolUse hook.`)
+        .action(async (opts) => {
+        const { importCommand } = await import('./commands/import.js');
+        await importCommand(opts);
+    });
     program
         .command('export')
         .description('Export recorded data')

package/dist/commands/import.js

@@ -0,0 +1,92 @@
+/**
+ * `agentobs import` - read Claude Code's own transcripts, no hooks required.
+ *
+ * This is the fallback when hooks are not firing, and the fastest way to get
+ * real data on a fresh install: everything Claude Code has already done is
+ * sitting on disk waiting to be read.
+ */
+import { openDb } from '../core/db.js';
+import { findTranscripts, importTranscript, transcriptRoot } from '../adapters/claude-transcript.js';
+const money = (v) => (v === null ? '—' : `$${v.toFixed(4)}`);
+export async function importCommand(opts = {}) {
+    const all = findTranscripts();
+    if (all.length === 0) {
+        console.error(`No Claude Code transcripts found under:
+  ${transcriptRoot()}
+
+Claude Code writes one JSONL file per session there. If you have used it on
+this machine, check the path above exists; set CLAUDE_CONFIG_DIR if your
+install keeps its config somewhere else.`);
+        process.exitCode = 1;
+        return;
+    }
+    const days = opts.all ? Infinity : Number(opts.days ?? 7);
+    const cutoff = Number.isFinite(days) ? Date.now() - days * 864e5 : 0;
+    const selected = opts.session
+        ? all.filter((t) => t.sessionId === opts.session)
+        : all.filter((t) => t.modifiedAt >= cutoff);
+    if (selected.length === 0) {
+        console.log(`Found ${all.length} transcript(s), but none in the last ${days} day(s).\n` +
+            `Use --all to import everything, or --days <n> for a wider window.`);
+        return;
+    }
+    if (opts.dryRun) {
+        console.log(`Would import ${selected.length} transcript(s):\n`);
+        for (const t of selected) {
+            const age = Math.round((Date.now() - t.modifiedAt) / 36e5);
+            console.log(`  ${t.sessionId.slice(0, 8)}  ${t.project.padEnd(24).slice(0, 24)}  ` +
+                `${String(Math.round(t.sizeBytes / 1024)).padStart(6)}KB  ${age}h ago`);
+        }
+        return;
+    }
+    const db = openDb();
+    console.log(`Importing ${selected.length} transcript(s) from ${transcriptRoot()}\n`);
+    let calls = 0;
+    let tokensIn = 0;
+    let tokensOut = 0;
+    let cacheRead = 0;
+    let cacheWrite = 0;
+    let cost = null;
+    for (const t of selected) {
+        try {
+            const r = await importTranscript(db, t);
+            calls += r.toolCalls;
+            tokensIn += r.tokensIn;
+            tokensOut += r.tokensOut;
+            cacheRead += r.cacheReadTokens;
+            cacheWrite += r.cacheWriteTokens;
+            if (r.cost !== null)
+                cost = (cost ?? 0) + r.cost;
+            console.log(`  ${t.sessionId.slice(0, 8)}  ${String(r.toolCalls).padStart(5)} calls  ` +
+                `${String(r.tokensIn + r.tokensOut).padStart(9)} tokens  ${money(r.cost).padStart(10)}`);
+        }
+        catch (err) {
+            // One unreadable transcript should not abandon the rest of the import.
+            console.log(`  ${t.sessionId.slice(0, 8)}  skipped: ${err.message}`);
+        }
+    }
+    // Break the cost down. On a long session the cache-read line dominates -
+    // the whole context is replayed on every turn, so it can reach billions of
+    // tokens. That is genuine billing, but a single unexplained total looks
+    // like a bug, so show where it comes from.
+    console.log(`
+  ${selected.length} session(s) · ${calls} tool calls
+
+  Fresh input     ${tokensIn.toLocaleString().padStart(15)} tokens
+  Output          ${tokensOut.toLocaleString().padStart(15)} tokens
+  Cache write     ${cacheWrite.toLocaleString().padStart(15)} tokens  (billed 1.25x input)
+  Cache read      ${cacheRead.toLocaleString().padStart(15)} tokens  (billed 0.10x input)
+  ${'-'.repeat(52)}
+  Estimated cost  ${money(cost).padStart(15)}
+
+Cache reads replay the conversation context on every turn, so a long
+session accumulates far more of them than fresh tokens - that line
+usually dominates the total. Prices come from ~/.agentobs/pricing.json;
+edit them if yours differ.
+
+Run "agentobs stats --today" or "agentobs dashboard" to see it.
+
+Note: imported data is historical, so guardrails cannot block anything
+retroactively. Blocking still requires the PreToolUse hook.`);
+}
+//# sourceMappingURL=import.js.map

package/dist/commands/watch.js

@@ -3,6 +3,30 @@
  */
 import { watchJsonl } from '../adapters/jsonl-watcher.js';
 export async function watch(file, opts = {}) {
+    if (!file) {
+        console.error(`No file given.
+
+  Usage:  agentobs watch <file.jsonl>
+
+  Tails a newline-delimited JSON log and records what it describes. Each
+  line is one JSON object with a "type" field:
+
+    {"type":"session_start","session_id":"s1","agent":"my-agent"}
+    {"type":"tool_call_start","session_id":"s1","id":"t1","tool":"Bash",
+     "input":{"command":"npm test"}}
+    {"type":"tool_call_end","id":"t1","status":"success",
+     "tokens_in":1200,"tokens_out":300,"model":"claude-sonnet-4"}
+    {"type":"session_end","session_id":"s1","exit_code":0}
+
+  Examples:
+    agentobs watch ./agent.jsonl
+    agentobs watch ./agent.jsonl --agent my-agent --no-follow
+
+  Unlike "agentobs run", this records full per-tool-call detail: tool
+  names, inputs, tokens and cost.`);
+        process.exitCode = 2;
+        return;
+    }
     let seen = 0;
     console.log(`Watching ${file} (agent: ${opts.agent ?? 'generic'}) — Ctrl-C to stop.`);
     await watchJsonl(file, {

package/dist/core/pricing.js

@@ -22,6 +22,9 @@ export const DEFAULT_PRICING = {
     _comment: 'Prices in USD per 1,000,000 tokens. Edit freely - AgentObs reads this file at runtime. A model missing here shows cost as blank rather than a guess.',
     updated: '2026-08-29',
     models: {
+        'claude-opus-5': { input_per_mtok: 15, output_per_mtok: 75 },
+        'claude-sonnet-5': { input_per_mtok: 3, output_per_mtok: 15 },
+        'claude-fable-5': { input_per_mtok: 3, output_per_mtok: 15 },
         'claude-opus-4': { input_per_mtok: 15, output_per_mtok: 75 },
         'claude-sonnet-4': { input_per_mtok: 3, output_per_mtok: 15 },
         'claude-haiku-4-5': { input_per_mtok: 1, output_per_mtok: 5 },

package/dist/core/queries.js

@@ -57,6 +57,15 @@ export function getSummary(db, range) {
        FROM tool_calls ${where}`)
         .get(...args);
     const sessions = db.prepare(`SELECT COUNT(*) AS n FROM sessions ${where}`).get(...args);
+    // Session-level totals. Transcript imports record tokens per session (usage
+    // is reported per assistant message, not per tool call), so summing the
+    // per-call columns alone would report a fully-imported session as zero.
+    const sessionTotals = db
+        .prepare(`SELECT COALESCE(SUM(total_tokens_in), 0) AS tokens_in,
+              COALESCE(SUM(total_tokens_out), 0) AS tokens_out,
+              SUM(total_cost_usd) AS cost
+         FROM sessions ${where}`)
+        .get(...args);
     const byFidelity = db
         .prepare(`SELECT
          COALESCE(SUM(CASE WHEN fidelity = 'coarse' THEN 1 ELSE 0 END), 0) AS coarse,
@@ -68,15 +77,19 @@ export function getSummary(db, range) {
     return {
         range,
         since,
-        total_cost_usd: calls.total_cost_usd === null ? null : Number(calls.total_cost_usd),
+        total_cost_usd: calls.total_cost_usd === null && sessionTotals.cost === null
+            ? null
+            : Math.max(Number(calls.total_cost_usd ?? 0), Number(sessionTotals.cost ?? 0)),
         uncosted_calls: Number(calls.uncosted_calls ?? 0),
         tool_calls: toolCalls,
         sessions: Number(sessions.n ?? 0),
         errors,
         blocked: Number(calls.blocked ?? 0),
         error_rate: toolCalls === 0 ? 0 : errors / toolCalls,
-        tokens_in: Number(calls.tokens_in ?? 0),
-        tokens_out: Number(calls.tokens_out ?? 0),
+        // Prefer whichever source actually has data: hook/JSONL data lands on the
+        // tool-call rows, transcript imports on the session rows.
+        tokens_in: Math.max(Number(calls.tokens_in ?? 0), Number(sessionTotals.tokens_in ?? 0)),
+        tokens_out: Math.max(Number(calls.tokens_out ?? 0), Number(sessionTotals.tokens_out ?? 0)),
         avg_duration_ms: calls.avg_duration_ms === null ? null : Number(calls.avg_duration_ms),
         coarse_sessions: Number(byFidelity.coarse ?? 0),
         rich_sessions: Number(byFidelity.rich ?? 0),

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@klars/agentobs",
-  "version": "0.1.6",
+  "version": "0.2.1",
   "description": "Observability and control layer for AI coding agents - see every tool call, token, and dollar your agents spend, and stop them before they do something risky.",
   "license": "MIT",
   "author": "Klars AI",