curt-hermes 0.0.0-stage → 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jordan Newell
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,218 @@
-# Temporary Holding Version
+# curt-hermes
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Two-direction bridge between the [Hermes](https://github.com/NousResearch/hermes)
+agent store and the [`.curt`](https://github.com/JordanNewell/dot_curt)
+single-file conversation format. Built on
+[`curt-format`](https://www.npmjs.com/package/curt-format) (reference
+implementation, v1 core). Part of the `curt-*` converter family (`curt-convert`
+for Claude Code, `curt-openclaw` for OpenClaw, `curt-wildchat` for
+WildChat-1M).
+
+**Hub-and-spoke positioning:** `.curt` is the archival hub. Direction 1
+exports Hermes sessions *out* into portable `.curt` archives. Direction 2
+emits the de-facto Claude Code session JSONL that `hermes sessions import`
+natively consumes — so `.curt` archives go back INTO Hermes (or any
+Claude-Code-transcript importer) without upstream changes.
+
+```sh
+npm install -g curt-hermes   # or: npx curt-hermes
+
+# Direction 1: Hermes state.db -> .curt files
+curt-hermes ~/.hermes/state.db -o ./out
+curt-hermes ~/.hermes/state.db --source cli|cron|discord|subagent
+curt-hermes ~/.hermes/state.db --session <sessionId>
+curt-hermes ~/.hermes/state.db --sample 5
+curt-hermes ~/.hermes/state.db --stdout > one.curt
+
+# Direction 2: .curt -> Claude Code session JSONL (for hermes sessions import)
+curt-hermes --emit-cc-jsonl ./out           # a directory of .curt files
+curt-hermes --emit-cc-jsonl one.curt -o ./jsonl
+```
+
+Direction 1 opens the SQLite store **read-only** (`mode=ro`; WAL sidecars are
+read, never written). Every `.curt` output is re-parsed with `parseCurt`
+before it is written; a file that fails round-trip validation is never
+written. Output names follow the Curtis bulk convention
+(`YYYY-MM-DD <slug> <suffix>.curt` via `curtFilename()`); a filename
+collision gets a `-2`/`-3` suffix instead of silently overwriting. Direction 2
+validates each emitted JSONL for line-parseability and `uuid`/`parentUuid`
+chain integrity before writing. A summary line with converted/skipped/
+round-trip-failure counts goes to stderr; per-session problems are counted,
+never fatal.
+
+**Database driver:** `node:sqlite` (`DatabaseSync`) — built into Node ≥ 22.5,
+zero native dependencies. A deliberate choice over `better-sqlite3`: no
+compile step, and the read-only URI mode matches the access posture.
+
+## Provenance
+
+Input schema verified read-only against a live Hermes gateway store
+(`~/.hermes/state.db`, **schema_version 30**, 8,602 sessions /
+157,870 messages) on **2026-11-01**. Real-corpus validation used a remote
+subset (all sessions with `source IN ('cli','discord')` and their messages +
+`session_model_usage` rows — 87 sessions, 1,810 messages) extracted with
+read-only SQLite ATTACH on the host, copied out, converted, and deleted from
+the host afterward. Real content stays out of this repo (gitignored `work/`
+by convention); `demos/` are fully synthetic.
+
+Schema findings that shaped the converter:
+
+- `sessions.started_at` / `ended_at` are REAL epoch **seconds** — normalized
+  to epoch milliseconds (`× 1000`) everywhere.
+- `messages.content` carries **no prefix sentinel** in schema_version 30: it
+  is either a plain string or a direct JSON document (tool payloads like
+  `{output, exit_code}`; assistant structured replies). JSON documents are
+  re-serialized canonically as the message text and counted. (The code-level
+  `_CONTENT_JSON_PREFIX` guard from the Hermes writer is honored
+  defensively: a sentinel-prefixed payload would fail `JSON.parse` and be
+  emitted verbatim, never dropped.)
+- `messages.tool_calls` (assistant rows) is a JSON array of OpenAI-style
+  calls **plus Hermes plumbing**:
+  `{id, call_id, response_item_id, type, function:{name, arguments},
+  extra_content?}`. Verified: `messages.tool_call_id` links on
+  `tool_calls[].id` (never `call_id`). Emitted .curt form is
+  `[{id, name, arguments}]` with `arguments` parsed to a value; plumbing
+  fields are counted as redacted.
+- Untrusted tool output is wrapped in
+  `<untrusted_tool_result source="...">` … `</untrusted_tool_result>` —
+  format metadata; the wrapper pair is stripped and counted, the inner body
+  is the content.
+- `role` distribution: `user` / `assistant` / `tool` / `session_meta`
+  (presence markers, NULL content — skipped+counted). No `system` rows exist
+  in the verified store; they would be skipped+counted if they appeared.
+- `reasoning` / `reasoning_content` (assistant) are plain text — joined to
+  the message `thinking` passthrough.
+- The documented `created_source` column does not exist in schema_version 30
+  (plan-vs-reality drift); the converter relies only on verified columns.
+
+## Mapping
+
+| Hermes (`state.db`) | .curt | Notes |
+|---|---|---|
+| `sessions.id` | `conversation.id` | |
+| `sessions.title` (when present; `title_source` `user`/`llm` are both titles, not raw content) | `conversation.title` | Falls back to first user text, single line, 60-char cap. Sessions without a title: derived (verified store: 63 of 87 subset sessions). |
+| `sessions.started_at` / `ended_at` (epoch s → ms) | `conversation.createdAt` / `updatedAt` | Falls back to min/max message timestamps. |
+| `sessions.model` | `conversation.model` | Messages carry no per-row model column in this schema. |
+| message `role:user` | message `role:"user"` | Content verbatim (or JSON re-serialized). |
+| message `role:assistant` | message `role:"assistant"` | Content plain or JSON-as-text. |
+| assistant `reasoning` + `reasoning_content` | message `thinking` (extra) | Joined `\n\n`. |
+| assistant `tool_calls` | message `toolCalls` (extra) | `[{id, name, arguments}]` — draft `tool-linkage` passthrough convention; plumbing (`call_id`, `response_item_id`, `type`, `extra_content`) dropped+counted. |
+| assistant `token_count` | message `tokens.totalTokens` (extra) | Single per-message number; maps to the stable `usage-metadata` `tokens` shape. |
+| assistant `finish_reason` | message `stopReason` (extra) | |
+| message `role:tool` | message `role:"tool"` | Extras: `toolCallId` (links `tool_calls[].id`), `toolName`. `<untrusted_tool_result>` wrapper stripped+counted. |
+| `role:session_meta` (and any unknown role) | — | Skipped and counted (`skippedRoles`). |
+| `sessions.source`, `chat_type`, `end_reason` | `conversation.hermes` (extra) | Generic surface metadata only. |
+| `sessions.input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `reasoning_tokens` | `conversation.hermesTokens` (extra) | Numbers only. |
+| `sessions.estimated_cost_usd` | `conversation.hermesCostUsd` (extra) | |
+| `session_model_usage` rows | `conversation.hermesModelUsage` (extra) | Only emitted when the session used >1 model (5 of 87 in the verified subset); `[{model, apiCallCount, inputTokens, outputTokens, cacheReadTokens, cacheWriteTokens, reasoningTokens, estimatedCostUsd}]`. |
+| row order (`messages.id`) | `messages` array order | |
+| — | message `id` | Synthesized `"<sessionId>-<n>"` over emitted messages. |
+
+### Direction 2 mapping (.curt → Claude Code JSONL)
+
+| .curt | Claude Code JSONL | Notes |
+|---|---|---|
+| conversation | one `.jsonl` file | Curtis bulk-convention name, `.jsonl` extension. |
+| `conversation.id` | `sessionId` on every line | |
+| message | one line | `type:"user"` / `type:"assistant"`; `uuid`/`parentUuid` chain (deterministic UUIDs — re-runs are byte-identical), ISO `timestamp`, `isSidechain:false`, `userType:"external"`. `cwd`/`gitBranch` emitted empty, `version` = generator. |
+| message `content` (user) | `message.content` string | |
+| message `thinking` | `message.content[].{type:"thinking", thinking}` block | |
+| message `content` (assistant) | `message.content[].{type:"text", text}` block | |
+| message `toolCalls` | `message.content[].{type:"tool_use", id, name, input}` blocks | `arguments` → `input`. |
+| message `role:"tool"` | `type:"user"` line with `message.content[].{type:"tool_result", tool_use_id, content}` | `isError` → `is_error`. |
+| message `model`, `stopReason`, `tokens` | `message.model`, `message.stop_reason`, `message.usage.output_tokens` | |
+| `role:"system"` | `type:"system"` line with `content` | None exist in the verified store; handled defensively. |
+
+Each emitted file is validated before writing: every line parses, the
+`uuid`/`parentUuid` chain is intact, duplicates rejected. Orphan
+`tool_result`s (aborted calls in the archive) are carried through and
+counted, not fatal. Emitting the de-facto interchange means `hermes sessions
+import` consumes `.curt` archives without any upstream Hermes changes.
+
+### Extensions
+
+The top-level `extensions` array declares `["usage-metadata"]` (stable in
+`EXTENSIONS.md`) because `tokens` is emitted. The **`tool-linkage` extension
+is draft**, so its field names (`toolCalls` on assistant messages,
+`toolCallId` on tool messages) ride as plain passthrough extras without
+declaring the extension — the same convention curt-openclaw uses. If/when
+`tool-linkage` reaches stable, declaring it is a one-line change.
+
+## Known losses
+
+Deliberate — identifiers, paths, prompts, and plumbing that must not leak
+into a portable file. Counted in `redactedFields` (aggregate only; never
+logged verbatim):
+
+- Never read as values at all (NULL-ness only, for the drop ledger):
+  `system_prompt`, `origin_json`, `display_name`, `user_id`, `session_key`,
+  `chat_id`, `thread_id`, `billing_provider`, `billing_base_url`,
+  `tool_names`. Also never touched: `cwd`, `git_branch`, `git_repo_root`,
+  `model_config`, `profile_name`, `parent_session_id`, `handoff_*`,
+  `transport_profile`, `last_activity_*`, `compression_*`.
+- `tool_calls` plumbing: `call_id`, `response_item_id`, `type`,
+  `extra_content`.
+- `<untrusted_tool_result source="...">` wrapper tag + source attribute
+  (the body is kept).
+- Structural, not redaction:
+  - `session_meta` rows (presence markers with NULL content) are skipped and
+    counted.
+  - Empty-content user/assistant rows (nothing to say) are dropped and
+    counted (`emptyDropped`) — 1 in the verified subset.
+  - A `toolCall` with no matching `tool` row (aborted call) still emits its
+    `toolCalls` entry; counted as `unmatchedToolCalls`. A `tool` row whose
+    `tool_call_id` was never seen is counted as `unmatchedToolResults` but
+    still emitted.
+  - JSON contents are re-serialized (key order canonicalized); text is never
+    altered.
+  - Per-message model attribution does not exist in this schema — only the
+    session-level `model` (and multi-model rollups via
+    `hermesModelUsage`).
+  - Malformed rows (unparseable `tool_calls` JSON, non-string content) are
+    counted, never fatal — 0 in the verified subset.
+
+## Privacy posture
+
+Direction 1 writes content-faithful files: conversation text, tool calls and
+outputs, and thinking blocks all land in `.curt` output — treat output as
+record-equivalent to the source store. The converter's own metadata layer
+never emits identifiers, keys, session/chat/thread ids, system prompts, or
+billing endpoints; credential columns are not even SELECTed as values
+(NULL-ness only). Direction 2 re-emits that same content into JSONL; nothing
+is added except synthetic chain UUIDs and neutral `cwd`/`gitBranch`/`version`
+fields. The verified-corpus run was spot-scanned: no banned identifier values
+appear in any output's metadata layer.
+
+## Development
+
+```sh
+npm i
+npm test       # vitest, 36 tests over a synthetic node:sqlite fixture DB
+npm run build  # tsc -> dist/
+```
+
+The fixture DB is built by `test/fixtures.ts` (schema mirrors the real
+store; all content synthetic). Regenerate the staged demos with:
+
+```sh
+node --experimental-strip-types test/make-demo.ts
+node dist/cli.js demos/synthetic-fixture/state.db -o demos/synthetic-flow
+node dist/cli.js --emit-cc-jsonl demos/synthetic-flow -o demos/synthetic-cc-jsonl
+rm -rf demos/synthetic-fixture   # regenerable; also keeps *.db out of the tree
+```
+
+Real-corpus runs write into an untracked, gitignored `work/` directory by
+convention.
+
+## Part of the .curt suite
+
+- [`curt-format`](https://github.com/JordanNewell/dot_curt) — the .curt format spec + reference implementation ([npm](https://www.npmjs.com/package/curt-format))
+- [`curt-convert`](https://github.com/JordanNewell/curt-convert) — Claude Code transcripts → .curt ([npm](https://www.npmjs.com/package/curt-convert))
+- [`curt-openclaw`](https://github.com/JordanNewell/curt-openclaw) — OpenClaw session transcripts → .curt ([npm](https://www.npmjs.com/package/curt-openclaw))
+- [`curt-wildchat`](https://github.com/JordanNewell/curt-wildchat) — WildChat-1M records → .curt ([npm](https://www.npmjs.com/package/curt-wildchat))
+- [`curt-vcon`](https://github.com/JordanNewell/curt-vcon) — .curt ⇄ vCon (IETF) bridge ([npm](https://www.npmjs.com/package/curt-vcon))
+- [`curt-openwith`](https://github.com/JordanNewell/curt-openwith) — double-click any .curt file: offline viewer, Windows + macOS + Linux
+
+MIT — see [LICENSE](LICENSE). See also
+[ACCESS-AND-LICENSE.md](ACCESS-AND-LICENSE.md) for the input-data access and
+content-licensing posture.

package/dist/cli.d.ts

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+/** curt-hermes CLI: Hermes state.db -> .curt files; .curt -> Claude Code JSONL. */
+export {};

package/dist/cli.js

@@ -0,0 +1,221 @@
+#!/usr/bin/env node
+/** curt-hermes CLI: Hermes state.db -> .curt files; .curt -> Claude Code JSONL. */
+import { existsSync, mkdirSync, readFileSync, readdirSync, statSync, writeFileSync } from "node:fs";
+import { extname, join, resolve } from "node:path";
+import { curtFilename, parseCurt } from "curt-format";
+import { GENERATOR, convertDb, emitClaudeCodeJsonl, resolveCurtWritePath, serializeValidated, validateCcJsonl, } from "./hermes.js";
+function usage() {
+    console.error(`curt-hermes — Hermes agent store <-> .curt conversations
+
+Direction 1 (export/archive): Hermes state.db -> .curt files
+  curt-hermes <state.db>
+  curt-hermes <state.db> -o ./out
+  curt-hermes <state.db> --source cli|cron|discord|subagent
+  curt-hermes <state.db> --session <sessionId>
+  curt-hermes <state.db> --sample 5
+  curt-hermes <state.db> --stdout > one.curt
+
+Input is the Hermes SQLite store (~/.hermes/state.db). The database is opened
+read-only. Credential and identifier columns (system_prompt, origin_json,
+display_name, user_id, session_key, chat_id, thread_id, billing_*, cwd, git
+fields) are never read as values — only their NULL-ness, for drop accounting.
+
+Direction 2 (bridge into Hermes): .curt -> Claude Code session JSONL
+  curt-hermes --emit-cc-jsonl <file.curt|dir> [-o out]
+
+Emits the de-facto Claude Code interchange format that 'hermes sessions
+import' natively consumes, so .curt archives go INTO Hermes without upstream
+changes. One .jsonl per conversation.
+
+Options:
+  -o, --out <dir>       Output directory (default: alongside the input)
+  --source <s>          Direction 1: only sessions with sessions.source = s
+  --session <id>        Direction 1: only the session with this id
+  --sample <n>          Direction 1: only the first n matching sessions
+  --stdout              Direction 1: print .curt bodies instead of writing
+  --emit-cc-jsonl       Direction 2: .curt file(s) -> Claude Code JSONL
+  -h, --help            Show this help`);
+    process.exit(2);
+}
+function parseArgs(argv) {
+    const options = { stdout: false };
+    let input;
+    for (let i = 0; i < argv.length; i++) {
+        const a = argv[i];
+        if (a === "-h" || a === "--help")
+            usage();
+        if (a === "--stdout") {
+            options.stdout = true;
+        }
+        else if (a === "--emit-cc-jsonl") {
+            options.emitCcJsonl = true;
+        }
+        else if (a === "-o" || a === "--out") {
+            const v = argv[++i];
+            if (!v)
+                usage();
+            options.outDir = v;
+        }
+        else if (a === "--source") {
+            const v = argv[++i];
+            if (!v)
+                usage();
+            options.source = v;
+        }
+        else if (a === "--session") {
+            const v = argv[++i];
+            if (!v)
+                usage();
+            options.session = v;
+        }
+        else if (a === "--sample") {
+            const v = argv[++i];
+            const n = v ? Number(v) : NaN;
+            if (!Number.isInteger(n) || n < 1)
+                usage();
+            options.sample = n;
+        }
+        else if (a.startsWith("-")) {
+            console.error(`Unknown option: ${a}`);
+            usage();
+        }
+        else if (input === undefined) {
+            input = a;
+        }
+        else {
+            console.error(`Unexpected extra argument: ${a}`);
+            usage();
+        }
+    }
+    return { input, options };
+}
+// ---- Direction 1 ----
+function runConvert(dbPath, options) {
+    const { items, converted, skipped, totals } = convertDb(dbPath, options);
+    let roundtripFailures = 0;
+    let collisions = 0;
+    for (const { result } of items) {
+        const body = serializeValidated(result.conversation);
+        if (!body) {
+            roundtripFailures++;
+            console.error(`round-trip validation failed for ${result.conversation.id}; not written`);
+            continue;
+        }
+        if (options.stdout) {
+            console.log(body);
+        }
+        else {
+            const outDir = resolve(options.outDir ?? ".");
+            mkdirSync(outDir, { recursive: true });
+            const file = join(outDir, curtFilename(result.conversation));
+            const target = resolveCurtWritePath(file, body);
+            if (target.collided) {
+                collisions++;
+                console.error(`filename collision for ${result.conversation.id}; wrote ${target.path} instead`);
+            }
+            writeFileSync(target.path, body);
+            console.log(target.path);
+        }
+    }
+    console.error(`${GENERATOR}: ${converted} converted, ${skipped} skipped, ${roundtripFailures} round-trip failures` +
+        (collisions > 0 ? `, ${collisions} filename collisions (suffixed)` : "") +
+        ` (messages: ${totals.roles.user} user / ${totals.roles.assistant} assistant / ${totals.roles.tool} tool; ` +
+        `${totals.skippedRoles} meta/system rows, ${totals.malformedRows} malformed rows, ` +
+        `${totals.jsonContents} JSON contents, ${totals.untrustedWrappers} untrusted wrappers stripped, ` +
+        `${totals.unmatchedToolCalls} unmatched toolCalls, ${totals.unmatchedToolResults} unmatched toolResults, ` +
+        `${totals.emptyDropped} empty dropped, ${totals.redactedFields} identifier fields dropped)`);
+    // Per-session problems are counted, never fatal.
+    return 0;
+}
+// ---- Direction 2 ----
+function jsonlName(conversationId, createdAt, title) {
+    // Curtis bulk-convention name, .curt -> .jsonl.
+    const fake = {
+        id: conversationId,
+        title,
+        createdAt: Number.isFinite(createdAt) ? createdAt : 0,
+    };
+    const name = curtFilename(fake);
+    return `${name.slice(0, -".curt".length)}.jsonl`;
+}
+function runEmitCcJsonl(input, options) {
+    const st = statSync(input);
+    const curtFiles = [];
+    if (st.isDirectory()) {
+        for (const f of readdirSync(input).sort()) {
+            if (extname(f) === ".curt")
+                curtFiles.push(join(input, f));
+        }
+    }
+    else if (extname(input) === ".curt") {
+        curtFiles.push(input);
+    }
+    else {
+        console.error(`Not a .curt file or directory of .curt files: ${input}`);
+        return 2;
+    }
+    if (curtFiles.length === 0) {
+        console.error(`No .curt files found in ${input}`);
+        return 2;
+    }
+    const outDir = options.outDir ? resolve(options.outDir) : st.isDirectory() ? resolve(input) : resolve(input, "..");
+    if (!options.stdout)
+        mkdirSync(outDir, { recursive: true });
+    let written = 0;
+    let rejected = 0;
+    for (const file of curtFiles) {
+        let body;
+        try {
+            body = readFileSync(file, "utf8");
+        }
+        catch {
+            console.error(`unreadable: ${file}`);
+            rejected++;
+            continue;
+        }
+        const parsed = parseCurt(body);
+        if (!parsed) {
+            console.error(`not a valid .curt file, skipped: ${file}`);
+            rejected++;
+            continue;
+        }
+        const jsonl = emitClaudeCodeJsonl(parsed);
+        const check = validateCcJsonl(jsonl);
+        if (check.errors.length > 0) {
+            console.error(`structural validation failed for ${parsed.id}; not written (${check.errors.length} errors)`);
+            rejected++;
+            continue;
+        }
+        if (check.orphanToolResults > 0) {
+            console.error(`${parsed.id}: ${check.orphanToolResults} orphan tool_result(s) carried through from the archive`);
+        }
+        if (options.stdout) {
+            process.stdout.write(jsonl);
+        }
+        else {
+            const target = join(outDir, jsonlName(parsed.id, parsed.createdAt, parsed.title));
+            writeFileSync(target, jsonl);
+            console.log(target);
+        }
+        written++;
+    }
+    console.error(`${GENERATOR}: ${written} JSONL emitted (${curtFiles.length - written - rejected} empty), ${rejected} rejected`);
+    return 0;
+}
+function main() {
+    const { input, options } = parseArgs(process.argv.slice(2));
+    if (options.emitCcJsonl) {
+        if (!input)
+            usage();
+        return runEmitCcJsonl(resolve(input), options);
+    }
+    if (!input)
+        usage();
+    const dbPath = resolve(input);
+    if (!existsSync(dbPath)) {
+        console.error(`No such file: ${dbPath}`);
+        return 2;
+    }
+    return runConvert(dbPath, options);
+}
+process.exitCode = main();