@tangle-network/harness-sessions 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Tangle Network
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,71 @@
-# Temporary Holding Version
+# @tangle-network/harness-sessions
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+One reader for every coding-agent harness's native session.
+
+Each harness persists its session in its own store: Claude Code and Pi append JSONL, Codex appends a rollout JSONL, and OpenCode writes SQLite.
+This package declares each store's layout once and parses each format into one normalized `HarnessSession`.
+The session holds the messages, the tool calls with their inputs and results, every model call with the model that served it and its usage, and the error the session ended with.
+
+```bash
+npm install @tangle-network/harness-sessions
+```
+
+## Where this fits
+
+- The sidecar's harness session contract (agent-dev-container `packages/cli-agent-registry`) imports `STORES` and `SessionFormatId` from here; it keeps no second copy of where a harness writes.
+- The capture path reads captures through `listCapture` / `readCapture`.
+- Discovery's trace gate, agent-eval, agent-record and the blog read sessions only through this package.
+- Its only dependencies are `@tangle-network/agent-trace-contract` and Node built-ins (`node:sqlite` for OpenCode).
+
+## Formats
+
+| Harness | Format | Store under HOME | Served model from | Usage |
+|---|---|---|---|---|
+| `claude-code` | `claude-code.projects-jsonl` | `.claude/projects/*/{id}.jsonl`, subagents at `*/{id}/subagents/**/*.jsonl` | provider response (`message.model`) | per response, counted once per `message.id` |
+| `codex` | `codex.rollout-jsonl` | `.codex/sessions/*/*/*/rollout-*-{id}.jsonl` | turn context (the configured model) | per response (`last_token_usage`); session total from `total_token_usage` |
+| `opencode` | `opencode.sqlite` | `.local/share/opencode/opencode.db` (shared by every session) | session record (`providerID/modelID`) | per step (`step-finish`) |
+| `pi` | `pi.session-jsonl` | `.pi/agent/sessions/**/*_{id}.jsonl` | session record (`provider/model`) | per assistant message |
+
+Readers for Kimi, Factory, Gemini, Amp, Qwen, Copilot, Forge, Hermes, Prime and OpenClaw are added with the same real-data proof.
+
+## Use
+
+```ts
+import { readerFor, listCapture, toChatMessages, toOtlpSpans, toTurns } from '@tangle-network/harness-sessions'
+
+// Sessions in a HOME, through the harness's declared store
+const reader = readerFor('pi')
+for (const ref of await reader.locate('/home/agent')) {
+  const summary = await reader.summarize(ref) // constant memory; a 537 MB rollout reads in ~200 MB RSS
+  const session = await reader.read(ref)      // full fidelity: nothing is truncated
+}
+
+// Sessions in a capture (provider retention layout today, raw-evidence archive v2 when given)
+const { sessions, missing } = await listCapture(captureDir)
+```
+
+Projections trim; the reader never does:
+
+- `toChatMessages(session)`: agent-eval's chat-with-tools messages.
+- `toOtlpSpans(session)`: contract spans (`@tangle-network/agent-trace-contract`), one LLM span per model call and one TOOL span per tool call.
+- `toTurns(session)`: the blog's turns, with the model, usage and error of each assistant turn.
+
+For callers that are not JavaScript:
+
+```bash
+harness-sessions read <file|home|capture> [--harness h] [--session id] [--summary] [--json]
+```
+
+## Rules the readers keep
+
+- The served model comes only from the provider response or the session record, never from the profile or the request. A failed request has no served model; `requestedModel` says which model it was sent to.
+- `usage` is `null` when the harness recorded none; it is never reported as zero. `input` excludes cache reads and cache writes, which have their own fields.
+- A session with zero model calls is not the trace of a turn.
+- An unparsable record is counted in `integrity.unparsedRecords` and never stops a read (`corruption: 'strict'` throws instead). A record cut off at the end of a live copy is reported as a torn tail, not as corruption.
+- `ending.status` is `completed`, `error`, `aborted` or `open`. `open` means the records stop mid-turn: the turn was still running or the process died.
+- An OpenCode store is read from a private copy, never in place.
+
+## Tests
+
+The fixtures are sessions that the real harness CLIs wrote while talking to the trace-proof scripted model; the tests assert that script.
+The proof against the readers this package replaces is a comparison over stored sessions, recorded with each release.

package/dist/builder.d.ts

@@ -0,0 +1,116 @@
+import type { HarnessSession, ModelCall, Part, SessionEnding, SessionError, SessionFormatId, SessionMessage, SessionSummary, TokenUsage } from './schema.js';
+import type { SourceStats } from './source.js';
+export type BuildMode = 'full' | 'summary';
+/** An ISO-8601 instant from an ISO string, epoch milliseconds or epoch seconds; null otherwise. */
+export declare function isoTime(value: unknown): string | null;
+export declare const isRecord: (value: unknown) => value is Record<string, unknown>;
+export declare const str: (value: unknown) => string | null;
+export declare function count(value: unknown): number | null;
+/** Parse tool arguments recorded as text; keep the text when it is not JSON. */
+export declare function argumentsOf(raw: unknown): {
+    input: unknown;
+    inputText: string | null;
+};
+/** The text of an output that is text, or of the text blocks in a content-block array. */
+export declare function textOf(value: unknown): string | null;
+/** The HTTP status in a provider error message such as `unexpected status 500 Internal Server Error`. */
+export declare function httpStatusIn(text: string | null | undefined): number | null;
+export declare function usageOrNull(usage: TokenUsage): TokenUsage | null;
+/** The key a served model is counted under: `provider/model`, or the bare model with no provider. */
+export declare function servedModelKey(call: Pick<ModelCall, 'provider' | 'servedModel'>): string | null;
+/**
+ * Folds one session's records into a {@link HarnessSession}. Every format reader drives this
+ * builder, and `read` and `summarize` share the same fold: in summary mode the builder keeps the
+ * counts, the model calls and the ending, and drops message and tool-call content, so a 700 MB
+ * rollout is summarized without holding its outputs.
+ */
+export declare class SessionBuilder {
+    readonly harness: string;
+    readonly format: SessionFormatId;
+    readonly modelSource: HarnessSession['modelSource'];
+    readonly full: boolean;
+    nativeSessionId: string;
+    parentNativeSessionId: string | null;
+    cwd: string | null;
+    private startedAt;
+    private endedAt;
+    private readonly messages;
+    private readonly messageById;
+    private readonly toolCalls;
+    private readonly toolCallIds;
+    private readonly openToolCalls;
+    private lastRole;
+    private inputsSinceLastCall;
+    private toolCallCount;
+    private replies;
+    readonly modelCalls: ModelCall[];
+    private readonly modelCallById;
+    private readonly children;
+    private readonly gaps;
+    private readonly sources;
+    private unparsed;
+    private truncated;
+    private sessionUsage;
+    private ending;
+    constructor(harness: string, format: SessionFormatId, modelSource: HarnessSession['modelSource'], mode: BuildMode, nativeSessionId: string, parentNativeSessionId?: string | null);
+    /** Record an instant the session was active. Returns the normalized instant. */
+    seen(value: unknown): string | null;
+    message(message: SessionMessage): SessionMessage;
+    /** Add parts to a message already recorded (one API response written over several records). */
+    appendParts(messageId: string, parts: Part[]): void;
+    /** Replace a tool call's arguments (a harness that streams them in parts). */
+    setToolInput(id: string, inputText: string): void;
+    hasMessage(messageId: string): boolean;
+    /** A tool call the agent made. Its part is added to `messageId` when the message is held. */
+    toolCall(call: {
+        id: string;
+        name: string;
+        input: unknown;
+        inputText: string | null;
+        at: string | null;
+        messageId: string;
+    }): void;
+    /** The result of a tool call. A result whose call was not recorded becomes a call of its own. */
+    toolResult(id: string, result: {
+        output: unknown;
+        text: string | null;
+        isError: boolean;
+        at: string | null;
+        name?: string;
+        messageId: string;
+        details?: unknown;
+    }): void;
+    modelCall(call: ModelCall): ModelCall;
+    getModelCall(id: string): ModelCall | undefined;
+    child(nativeSessionId: string, toolCallId: string | null): void;
+    gap(text: string): void;
+    source(stats: SourceStats | {
+        path: string;
+        sha256: string;
+        bytes: number;
+        unparsed?: number;
+        tornTail?: boolean;
+        unparsedLines?: number[];
+    }): void;
+    markTruncated(reason: string): void;
+    /** The harness's own session total, when it records one; it wins over the per-call sum. */
+    setSessionUsage(usage: TokenUsage | null): void;
+    end(ending: SessionEnding): void;
+    /**
+     * The ending every format shares, from what the session holds when the records run out:
+     * `aborted` when the harness recorded an abort last; `error` when the last model call failed
+     * and no new input followed it; `completed` when the last model call succeeded, every tool call
+     * has its result and the agent spoke last; `open` otherwise (the turn was still running, or the
+     * process died mid-turn).
+     */
+    inferEnding(aborted?: {
+        at: string | null;
+        error: SessionError | null;
+    }): void;
+    get endingStatus(): SessionEnding['status'];
+    private usage;
+    private servedModels;
+    build(): HarnessSession;
+    summary(): SessionSummary;
+}
+export declare function sessionError(kind: string, message: string | null | undefined, at: string | null, httpStatus?: number | null): SessionError;

package/dist/builder.js

@@ -0,0 +1,332 @@
+import { HARNESS_SESSION_SCHEMA } from './schema.js';
+/** An ISO-8601 instant from an ISO string, epoch milliseconds or epoch seconds; null otherwise. */
+export function isoTime(value) {
+    if (typeof value === 'string' && value.length > 0) {
+        const ms = Date.parse(value);
+        return Number.isNaN(ms) ? null : new Date(ms).toISOString();
+    }
+    if (typeof value === 'number' && Number.isFinite(value) && value > 0) {
+        // Seconds before 2001-09-09 in milliseconds are below 1e12; nothing here predates that.
+        return new Date(value < 1e12 ? value * 1000 : value).toISOString();
+    }
+    return null;
+}
+export const isRecord = (value) => typeof value === 'object' && value !== null && !Array.isArray(value);
+export const str = (value) => (typeof value === 'string' ? value : null);
+export function count(value) {
+    return typeof value === 'number' && Number.isFinite(value) && value >= 0 ? value : null;
+}
+/** Parse tool arguments recorded as text; keep the text when it is not JSON. */
+export function argumentsOf(raw) {
+    if (typeof raw !== 'string')
+        return { input: raw ?? null, inputText: null };
+    try {
+        return { input: JSON.parse(raw), inputText: raw };
+    }
+    catch {
+        return { input: raw, inputText: raw };
+    }
+}
+/** The text of an output that is text, or of the text blocks in a content-block array. */
+export function textOf(value) {
+    if (typeof value === 'string')
+        return value;
+    if (!Array.isArray(value))
+        return null;
+    const texts = [];
+    for (const block of value) {
+        if (isRecord(block) && typeof block.text === 'string')
+            texts.push(block.text);
+    }
+    return texts.length > 0 ? texts.join('\n') : null;
+}
+/** The HTTP status in a provider error message such as `unexpected status 500 Internal Server Error`. */
+export function httpStatusIn(text) {
+    if (!text)
+        return null;
+    const match = /\b(?:status(?: code)?|HTTP)[^\d]{0,4}([1-5]\d\d)\b/iu.exec(text) ?? /^([45]\d\d)\b/u.exec(text.trim());
+    return match ? Number(match[1]) : null;
+}
+function addUsage(total, usage) {
+    for (const key of ['input', 'output', 'reasoning', 'cacheRead', 'cacheWrite']) {
+        const value = usage[key];
+        if (value !== null)
+            total[key] = (total[key] ?? 0) + value;
+    }
+}
+export function usageOrNull(usage) {
+    return usage.input === null && usage.output === null && usage.reasoning === null
+        && usage.cacheRead === null && usage.cacheWrite === null
+        ? null
+        : usage;
+}
+/** The key a served model is counted under: `provider/model`, or the bare model with no provider. */
+export function servedModelKey(call) {
+    if (call.servedModel === null)
+        return null;
+    return call.provider === null ? call.servedModel : `${call.provider}/${call.servedModel}`;
+}
+/**
+ * Folds one session's records into a {@link HarnessSession}. Every format reader drives this
+ * builder, and `read` and `summarize` share the same fold: in summary mode the builder keeps the
+ * counts, the model calls and the ending, and drops message and tool-call content, so a 700 MB
+ * rollout is summarized without holding its outputs.
+ */
+export class SessionBuilder {
+    harness;
+    format;
+    modelSource;
+    full;
+    nativeSessionId;
+    parentNativeSessionId;
+    cwd = null;
+    startedAt = null;
+    endedAt = null;
+    messages = [];
+    messageById = new Map();
+    toolCalls = new Map();
+    toolCallIds = new Set();
+    openToolCalls = new Set();
+    lastRole = null;
+    inputsSinceLastCall = 0;
+    toolCallCount = 0;
+    replies = 0;
+    modelCalls = [];
+    modelCallById = new Map();
+    children = [];
+    gaps = [];
+    sources = [];
+    unparsed = 0;
+    truncated = false;
+    sessionUsage;
+    ending = { status: 'open', error: null, at: null };
+    constructor(harness, format, modelSource, mode, nativeSessionId, parentNativeSessionId = null) {
+        this.harness = harness;
+        this.format = format;
+        this.modelSource = modelSource;
+        this.full = mode === 'full';
+        this.nativeSessionId = nativeSessionId;
+        this.parentNativeSessionId = parentNativeSessionId;
+    }
+    /** Record an instant the session was active. Returns the normalized instant. */
+    seen(value) {
+        const at = isoTime(value);
+        if (at === null)
+            return null;
+        if (this.startedAt === null || at < this.startedAt)
+            this.startedAt = at;
+        if (this.endedAt === null || at > this.endedAt)
+            this.endedAt = at;
+        return at;
+    }
+    message(message) {
+        if (message.role === 'assistant' && message.actor === 'agent')
+            this.replies += 1;
+        // Only new input re-opens a turn: harness bookkeeping (a compaction summary, environment
+        // context) after a failed call does not.
+        if (message.role === 'user' && (message.actor === 'human' || message.actor === 'subagent-spawn'))
+            this.inputsSinceLastCall += 1;
+        this.lastRole = message.role;
+        if (this.full) {
+            this.messages.push(message);
+            this.messageById.set(message.id, message);
+        }
+        return message;
+    }
+    /** Add parts to a message already recorded (one API response written over several records). */
+    appendParts(messageId, parts) {
+        if (!this.full)
+            return;
+        const message = this.messageById.get(messageId);
+        if (message)
+            message.parts.push(...parts);
+    }
+    /** Replace a tool call's arguments (a harness that streams them in parts). */
+    setToolInput(id, inputText) {
+        const call = this.toolCalls.get(id);
+        if (!call)
+            return;
+        call.inputText = inputText;
+        call.input = argumentsOf(inputText).input;
+    }
+    hasMessage(messageId) {
+        return this.messageById.has(messageId);
+    }
+    /** A tool call the agent made. Its part is added to `messageId` when the message is held. */
+    toolCall(call) {
+        if (this.toolCallIds.has(call.id))
+            return;
+        this.toolCallIds.add(call.id);
+        this.openToolCalls.add(call.id);
+        this.toolCallCount += 1;
+        if (!this.full)
+            return;
+        this.toolCalls.set(call.id, {
+            id: call.id,
+            name: call.name,
+            input: call.input,
+            inputText: call.inputText,
+            result: null,
+            status: 'pending',
+            startedAt: call.at,
+            endedAt: null,
+            messageId: call.messageId,
+        });
+    }
+    /** The result of a tool call. A result whose call was not recorded becomes a call of its own. */
+    toolResult(id, result) {
+        if (!this.toolCallIds.has(id)) {
+            this.toolCall({ id, name: result.name ?? 'unknown', input: null, inputText: null, at: result.at, messageId: result.messageId });
+            this.gap(`tool result ${id} has no recorded call`);
+        }
+        this.openToolCalls.delete(id);
+        if (!this.full)
+            return;
+        const call = this.toolCalls.get(id);
+        if (!call)
+            return;
+        call.result = { output: result.output, text: result.text, isError: result.isError, at: result.at, details: result.details ?? null };
+        call.status = result.isError ? 'error' : 'completed';
+        call.endedAt = result.at;
+    }
+    modelCall(call) {
+        this.inputsSinceLastCall = 0;
+        this.modelCalls.push(call);
+        this.modelCallById.set(call.id, call);
+        return call;
+    }
+    getModelCall(id) {
+        return this.modelCallById.get(id);
+    }
+    child(nativeSessionId, toolCallId) {
+        if (this.children.some((c) => c.nativeSessionId === nativeSessionId))
+            return;
+        this.children.push({ nativeSessionId, toolCallId });
+    }
+    gap(text) {
+        // Keep the first occurrences; a damaged file can produce one gap per record.
+        if (this.gaps.length < 200)
+            this.gaps.push(text);
+    }
+    source(stats) {
+        this.sources.push({ path: stats.path, sha256: stats.sha256, bytes: stats.bytes });
+        this.unparsed += stats.unparsed ?? 0;
+        for (const line of stats.unparsedLines ?? [])
+            this.gap(`${stats.path}: line ${line} is not a JSON record`);
+        if (stats.tornTail) {
+            this.truncated = true;
+            this.gap(`${stats.path}: last record was still being written when the copy was taken`);
+        }
+    }
+    markTruncated(reason) {
+        this.truncated = true;
+        this.gap(reason);
+    }
+    /** The harness's own session total, when it records one; it wins over the per-call sum. */
+    setSessionUsage(usage) {
+        this.sessionUsage = usage;
+    }
+    end(ending) {
+        this.ending = ending;
+    }
+    /**
+     * The ending every format shares, from what the session holds when the records run out:
+     * `aborted` when the harness recorded an abort last; `error` when the last model call failed
+     * and no new input followed it; `completed` when the last model call succeeded, every tool call
+     * has its result and the agent spoke last; `open` otherwise (the turn was still running, or the
+     * process died mid-turn).
+     */
+    inferEnding(aborted) {
+        if (aborted) {
+            this.ending = { status: 'aborted', error: aborted.error, at: aborted.at };
+            return;
+        }
+        const last = this.modelCalls[this.modelCalls.length - 1];
+        if (last?.error && this.inputsSinceLastCall === 0) {
+            this.ending = { status: 'error', error: last.error, at: last.error.at ?? last.at };
+            return;
+        }
+        if (last && this.openToolCalls.size === 0 && this.lastRole === 'assistant') {
+            this.ending = { status: 'completed', error: null, at: this.endedAt };
+            return;
+        }
+        this.ending = { status: 'open', error: null, at: null };
+    }
+    get endingStatus() {
+        return this.ending.status;
+    }
+    usage() {
+        if (this.sessionUsage !== undefined)
+            return this.sessionUsage;
+        const total = { input: null, output: null, reasoning: null, cacheRead: null, cacheWrite: null };
+        for (const call of this.modelCalls)
+            if (call.usage)
+                addUsage(total, call.usage);
+        return usageOrNull(total);
+    }
+    servedModels() {
+        const models = {};
+        for (const call of this.modelCalls) {
+            const key = servedModelKey(call);
+            if (key !== null)
+                models[key] = (models[key] ?? 0) + 1;
+        }
+        return models;
+    }
+    build() {
+        const toolCalls = [...this.toolCalls.values()];
+        for (const call of toolCalls) {
+            if (call.status === 'pending' && this.ending.status !== 'open')
+                call.status = 'interrupted';
+        }
+        return {
+            schema: HARNESS_SESSION_SCHEMA,
+            harness: this.harness,
+            format: this.format,
+            nativeSessionId: this.nativeSessionId,
+            parentNativeSessionId: this.parentNativeSessionId,
+            cwd: this.cwd,
+            startedAt: this.startedAt,
+            endedAt: this.endedAt,
+            messages: this.messages,
+            toolCalls,
+            modelCalls: this.modelCalls,
+            servedModels: this.servedModels(),
+            modelSource: this.modelSource,
+            usage: this.usage(),
+            ending: this.ending,
+            children: this.children,
+            integrity: {
+                sourceFiles: this.sources,
+                unparsedRecords: this.unparsed,
+                truncated: this.truncated,
+                gaps: this.gaps,
+            },
+        };
+    }
+    summary() {
+        return {
+            harness: this.harness,
+            format: this.format,
+            nativeSessionId: this.nativeSessionId,
+            parentNativeSessionId: this.parentNativeSessionId,
+            modelCalls: this.modelCalls.length,
+            replies: this.replies,
+            toolCalls: this.toolCallCount,
+            servedModels: this.servedModels(),
+            lastServedModel: [...this.modelCalls].reverse().map(servedModelKey).find((key) => key !== null) ?? null,
+            modelSource: this.modelSource,
+            usage: this.usage(),
+            ending: this.ending,
+            unparsedRecords: this.unparsed,
+            bytes: this.sources.reduce((sum, s) => sum + s.bytes, 0),
+            truncated: this.truncated,
+            startedAt: this.startedAt,
+            endedAt: this.endedAt,
+        };
+    }
+}
+export function sessionError(kind, message, at, httpStatus) {
+    const text = message ?? '';
+    return { kind, message: text, httpStatus: httpStatus ?? httpStatusIn(text), at };
+}

package/dist/capture.d.ts

@@ -0,0 +1,44 @@
+import type { HarnessSession, NativeSessionCopy, RawEvidenceArchiveManifestV2, SessionRef } from './schema.js';
+export declare const RETENTION_SESSIONS: string;
+export declare const RETENTION_MANIFEST_KIND = "tangle-native-session-evidence.v1";
+/** One native session found in a capture, with the ids that join it to its execution. */
+export interface CapturedSessionRef {
+    ref: SessionRef;
+    harness: string;
+    /** The sandbox (provider) session the capture belongs to. */
+    sandboxSessionId: string | null;
+    executionId: string | null;
+    attempt: number | null;
+    copy: NativeSessionCopy['copy'] | 'retained';
+    /** True when the sidecar recorded this native session id for the execution. */
+    recordedId: boolean;
+    /** The capture's native root the session was found in, relative to the capture directory. */
+    root: string;
+    sourceId: string | null;
+    rootScope: string | null;
+}
+export interface CaptureListing {
+    sessions: CapturedSessionRef[];
+    /** Executions whose harness has no reader, and ids recorded but not found in the copy. */
+    missing: Array<{
+        sandboxSessionId: string | null;
+        harness: string | null;
+        reason: string;
+        nativeSessionId?: string;
+        /** The capture root (relative to the capture directory) and store file a reason refers to. */
+        root?: string;
+        path?: string;
+        detail?: string;
+    }>;
+}
+/**
+ * Every native session in a capture: from `manifest.nativeSessions` when a v2 manifest is given
+ * (files relative to `dir`), otherwise from the retention layout under `dir`.
+ */
+export declare function listCapture(dir: string, manifest?: RawEvidenceArchiveManifestV2 | null, options?: {
+    harness?: string;
+}): Promise<CaptureListing>;
+/** Every native session in a capture, read in full. */
+export declare function readCapture(dir: string, manifest?: RawEvidenceArchiveManifestV2 | null, options?: {
+    harness?: string;
+}): AsyncIterable<HarnessSession>;