cjm-harness-transcripts 0.0.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,51 @@
1
+ Metadata-Version: 2.4
2
+ Name: cjm-harness-transcripts
3
+ Version: 0.0.1
4
+ Summary: The harness-transcript extraction lib (DEC e8b2f397 increment ii; the d774d311 build item): parse agent-harness session transcripts (Claude Code JSONL first) into their append-only record DAG, walk the active path, extract user-facing prose both sides, and derive the transcript <-> on-graph session-key mapping. Pure-parse, zero graph dependencies — one extraction truth for the live scratchpad pull, the archive-absorb instrument, and the housekeeping miner. Born on-graph.
5
+ Author-email: "Christian J. Mills" <9126128+cj-mills@users.noreply.github.com>
6
+ License: Apache-2.0
7
+ Project-URL: Repository, https://github.com/cj-mills/cjm-harness-transcripts
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Programming Language :: Python :: 3 :: Only
10
+ Requires-Python: >=3.12
11
+ Description-Content-Type: text/markdown
12
+
13
+ # cjm-harness-transcripts
14
+
15
+ <!-- generated from the context graph by `cjm-context-graph readme` — do not edit by hand; edit the graph (the urge to hand-edit = move it on-graph) -->
16
+
17
+ Harness-transcript extraction library: reads Claude Code session transcripts (JSONL DAGs), walks the active path, and extracts the user-facing prose — assistant markdown bodies, cleaned user prompts, and curated tool-parameter prose (e.g. SendUserFile captions) — as ExtractedMessages ready to mint onto a context-graph session spine.
18
+
19
+ ## Modules
20
+
21
+ - **`cjm_harness_transcripts`** — cjm-harness-transcripts: parse agent-harness session transcripts (Claude Code
22
+ - **`cjm_harness_transcripts.extract`** — User-facing prose extraction — both sides, one truth.
23
+ - **`cjm_harness_transcripts.mapping`** — Transcript ↔ session-key mapping, derived at pull time.
24
+ - **`cjm_harness_transcripts.records`** — The transcript record layer: tolerant JSONL loading + the append-only DAG.
25
+
26
+ ## API
27
+
28
+ ### `cjm_harness_transcripts.extract`
29
+
30
+ - `ExtractedMessage` _class_ — One user-facing message off the active path, ready to become a Message node.
31
+ - `clean_user_text` _function_ — Strip harness wrapper blocks from a user prompt; collapse the scar tissue.
32
+ - `extract_messages` _function_ — The active path's user-facing messages, chronological.
33
+
34
+ ### `cjm_harness_transcripts.mapping`
35
+
36
+ - `BootInfo` _class_ — The first user prose of a transcript — the mapping's evidence.
37
+ - `TranscriptMatch` _class_ — One ranked candidate pairing for a session key.
38
+ - `find_transcripts_for_key` _function_ — All plausible transcripts for a session key, best first.
39
+ - `parse_record_timestamp` _function_ — A transcript record timestamp (ISO-8601, trailing Z) as an aware datetime.
40
+ - `parse_session_key` _function_ — A session key (local start timestamp, e.g. 2026-08-20_17-05-20) as an aware datetime.
41
+ - `read_boot_info` _function_ — The transcript's first user record bearing prose, or None (empty/tool-only file).
42
+
43
+ ### `cjm_harness_transcripts.records`
44
+
45
+ - `TranscriptDag` _class_ — The uuid-addressable view of a transcript: parent edges, forks, active path.
46
+ - `TranscriptRecord` _class_ — One JSONL record, uuid-addressable or not, with the raw dict preserved.
47
+ - `load_records` _function_ — Load a transcript JSONL tolerantly, in file order.
48
+
49
+ ## Dependencies
50
+
51
+ **Used by:** `cjm-context-graph-projection`
@@ -0,0 +1,39 @@
1
+ # cjm-harness-transcripts
2
+
3
+ <!-- generated from the context graph by `cjm-context-graph readme` — do not edit by hand; edit the graph (the urge to hand-edit = move it on-graph) -->
4
+
5
+ Harness-transcript extraction library: reads Claude Code session transcripts (JSONL DAGs), walks the active path, and extracts the user-facing prose — assistant markdown bodies, cleaned user prompts, and curated tool-parameter prose (e.g. SendUserFile captions) — as ExtractedMessages ready to mint onto a context-graph session spine.
6
+
7
+ ## Modules
8
+
9
+ - **`cjm_harness_transcripts`** — cjm-harness-transcripts: parse agent-harness session transcripts (Claude Code
10
+ - **`cjm_harness_transcripts.extract`** — User-facing prose extraction — both sides, one truth.
11
+ - **`cjm_harness_transcripts.mapping`** — Transcript ↔ session-key mapping, derived at pull time.
12
+ - **`cjm_harness_transcripts.records`** — The transcript record layer: tolerant JSONL loading + the append-only DAG.
13
+
14
+ ## API
15
+
16
+ ### `cjm_harness_transcripts.extract`
17
+
18
+ - `ExtractedMessage` _class_ — One user-facing message off the active path, ready to become a Message node.
19
+ - `clean_user_text` _function_ — Strip harness wrapper blocks from a user prompt; collapse the scar tissue.
20
+ - `extract_messages` _function_ — The active path's user-facing messages, chronological.
21
+
22
+ ### `cjm_harness_transcripts.mapping`
23
+
24
+ - `BootInfo` _class_ — The first user prose of a transcript — the mapping's evidence.
25
+ - `TranscriptMatch` _class_ — One ranked candidate pairing for a session key.
26
+ - `find_transcripts_for_key` _function_ — All plausible transcripts for a session key, best first.
27
+ - `parse_record_timestamp` _function_ — A transcript record timestamp (ISO-8601, trailing Z) as an aware datetime.
28
+ - `parse_session_key` _function_ — A session key (local start timestamp, e.g. 2026-08-20_17-05-20) as an aware datetime.
29
+ - `read_boot_info` _function_ — The transcript's first user record bearing prose, or None (empty/tool-only file).
30
+
31
+ ### `cjm_harness_transcripts.records`
32
+
33
+ - `TranscriptDag` _class_ — The uuid-addressable view of a transcript: parent edges, forks, active path.
34
+ - `TranscriptRecord` _class_ — One JSONL record, uuid-addressable or not, with the raw dict preserved.
35
+ - `load_records` _function_ — Load a transcript JSONL tolerantly, in file order.
36
+
37
+ ## Dependencies
38
+
39
+ **Used by:** `cjm-context-graph-projection`
@@ -0,0 +1,6 @@
1
+ """cjm-harness-transcripts: parse agent-harness session transcripts (Claude Code
2
+ JSONL first) — the record DAG, the active path, user-facing prose both sides,
3
+ and the transcript ↔ session-key mapping. Pure-parse; the graph-minting half
4
+ of the pull path lives with the projection verbs, which import this lib."""
5
+
6
+ __version__ = "0.0.1"
@@ -0,0 +1,156 @@
1
+ """User-facing prose extraction — both sides, one truth.
2
+
3
+ Filters a transcript's active path down to what the humans actually read:
4
+ assistant text blocks (no thinking, no tool calls/results), user prompts
5
+ (minus harness wrappers — caveat blocks, system reminders, command noise),
6
+ and the curated tool parameters the client renders as discourse
7
+ (TOOL_PARAM_PROSE — a SendUserFile caption is prose living in tool_use
8
+ input; finding 60d719fe).
9
+ The transcript stores assistant messages as RAW MARKDOWN SOURCE (rendering is
10
+ client-layered — verified DEC 671e9b11 point 9), so extracted text is the
11
+ authoritative body for a Message node, not a rendering of it.
12
+
13
+ This module is consumed identically by the live scratchpad pull, the
14
+ archive-absorb instrument, and the housekeeping miner (DEC e8b2f397 point 2).
15
+ """
16
+
17
+ import re
18
+ from dataclasses import dataclass
19
+
20
+ from .records import TranscriptDag, TranscriptRecord
21
+
22
+ # Harness-injected wrapper blocks that are not the user's own prose.
23
+ _WRAPPER_TAGS = (
24
+ "local-command-caveat",
25
+ "system-reminder",
26
+ "command-name",
27
+ "command-message",
28
+ "command-args",
29
+ "command-contents",
30
+ "local-command-stdout",
31
+ "local-command-stderr",
32
+ )
33
+ _WRAPPER_RE = re.compile(
34
+ "|".join(rf"<{t}>.*?</{t}>" for t in _WRAPPER_TAGS), re.DOTALL
35
+ )
36
+
37
+ # Tool parameters that carry user-facing prose (finding 60d719fe): the client
38
+ # renders these as discourse (a SendUserFile caption rides the file card), but
39
+ # the record holds only a tool_use block, so text-block extraction misses them.
40
+ # Curated allowlist: tool name -> dotted paths into the tool_use input.
41
+ TOOL_PARAM_PROSE: dict[str, tuple[str, ...]] = {
42
+ "SendUserFile": ("caption",),
43
+ }
44
+
45
+ # The birth-class facet stamped on tool-param messages (one label, many sources
46
+ # — DEC 91c47b4a pt 1; graph-side consumers mirror the literal).
47
+ TOOL_PARAM_SOURCE = "cc-tool-param"
48
+
49
+
50
+ @dataclass
51
+ class ExtractedMessage:
52
+ """One user-facing message off the active path, ready to become a Message node."""
53
+ role: str # "user" | "assistant"
54
+ text: str # Cleaned prose; raw markdown source on the assistant side
55
+ timestamp: str | None # ISO-8601 as recorded
56
+ uuid: str # Transcript identity (tool-param entries: the tool_use block id)
57
+ parent_uuid: str | None # DAG ancestry (NEXT edges mirror this succession)
58
+ source: str | None = None # Birth-class facet; None = plain transcript prose
59
+
60
+
61
+ def clean_user_text(text: str) -> str:
62
+ """Strip harness wrapper blocks from a user prompt; collapse the scar tissue."""
63
+ text = _WRAPPER_RE.sub("", text)
64
+ text = re.sub(r"\n{3,}", "\n\n", text)
65
+ return text.strip()
66
+
67
+
68
+ def _user_prose(rec: TranscriptRecord) -> str | None:
69
+ """A user record's own prose, or None when the record is tool/housekeeping traffic."""
70
+ content = rec.raw.get("message", {}).get("content")
71
+ if isinstance(content, str):
72
+ text = content
73
+ elif isinstance(content, list):
74
+ parts = [b.get("text", "") for b in content
75
+ if isinstance(b, dict) and b.get("type") == "text"]
76
+ if not parts:
77
+ return None # tool_result-only record
78
+ text = "\n\n".join(parts)
79
+ else:
80
+ return None
81
+ cleaned = clean_user_text(text)
82
+ return cleaned or None
83
+
84
+
85
+ def _assistant_prose(rec: TranscriptRecord) -> str | None:
86
+ """An assistant record's text blocks (raw markdown), or None on tool-only records."""
87
+ content = rec.raw.get("message", {}).get("content")
88
+ if not isinstance(content, list):
89
+ return None
90
+ parts = [b.get("text", "") for b in content
91
+ if isinstance(b, dict) and b.get("type") == "text" and b.get("text")]
92
+ text = "\n\n".join(parts).strip()
93
+ return text or None
94
+
95
+
96
+ def _tool_param_messages(
97
+ rec: TranscriptRecord,
98
+ table: dict[str, tuple[str, ...]], # tool name -> dotted paths into tool_use input
99
+ ) -> list[ExtractedMessage]:
100
+ """Messages for user-facing prose carried in tool parameters (finding 60d719fe).
101
+
102
+ One message per prose-bearing (tool_use block, path) hit, block order.
103
+ Identity is the block id (the carrier record may also bear its own text
104
+ message, so the record uuid is taken); ancestry points at the carrier."""
105
+ content = rec.raw.get("message", {}).get("content")
106
+ if not isinstance(content, list):
107
+ return []
108
+ out: list[ExtractedMessage] = []
109
+ for i, block in enumerate(content):
110
+ if not (isinstance(block, dict) and block.get("type") == "tool_use"):
111
+ continue
112
+ for path in table.get(block.get("name", ""), ()):
113
+ value = block.get("input")
114
+ for part in path.split("."):
115
+ value = value.get(part) if isinstance(value, dict) else None
116
+ if not (isinstance(value, str) and value.strip()):
117
+ continue
118
+ out.append(ExtractedMessage(
119
+ role="assistant", text=value.strip(), timestamp=rec.timestamp,
120
+ uuid=block.get("id") or f"{rec.uuid}#tp{i}", parent_uuid=rec.uuid,
121
+ source=TOOL_PARAM_SOURCE,
122
+ ))
123
+ return out
124
+
125
+
126
+ def extract_messages(
127
+ dag: TranscriptDag,
128
+ *,
129
+ tool_params: dict[str, tuple[str, ...]] | None = None, # None = the curated TOOL_PARAM_PROSE table; {} disables
130
+ ) -> list[ExtractedMessage]:
131
+ """The active path's user-facing messages, chronological.
132
+
133
+ One ExtractedMessage per prose-bearing record: an assistant turn that
134
+ interleaves status prose between tool calls yields one message per text
135
+ record, exactly as the client displayed them. Prose the client renders out
136
+ of tool_use parameters (the curated table — finding 60d719fe) extracts as
137
+ its own TOOL_PARAM_SOURCE-faceted message after the record's text."""
138
+ table = TOOL_PARAM_PROSE if tool_params is None else tool_params
139
+ messages: list[ExtractedMessage] = []
140
+ for rec in dag.active_path():
141
+ if rec.type == "user":
142
+ text, extras = _user_prose(rec), []
143
+ elif rec.type == "assistant":
144
+ text, extras = _assistant_prose(rec), _tool_param_messages(rec, table)
145
+ else:
146
+ continue
147
+ if text is not None:
148
+ messages.append(ExtractedMessage(
149
+ role=rec.type,
150
+ text=text,
151
+ timestamp=rec.timestamp,
152
+ uuid=rec.uuid,
153
+ parent_uuid=rec.parent_uuid,
154
+ ))
155
+ messages.extend(extras)
156
+ return messages
@@ -0,0 +1,108 @@
1
+ """Transcript ↔ session-key mapping, derived at pull time.
2
+
3
+ The workbench cannot record the pairing at mint: the Claude Code session
4
+ starts strictly AFTER the on-graph Shift+S mint, so the CC session UUID does
5
+ not exist yet (DEC fc6a0cdc point 1, superseding the record-at-mint variant).
6
+ What IS derivable: the minted session's boot prompt carries the
7
+ minted-in-workbench resume signal, session keys are local start timestamps,
8
+ and every transcript record is UTC-stamped. So: scan the project's transcript
9
+ dir, prefer boot records carrying the mint signal, and rank candidates by how
10
+ closely their boot time follows the key. The caller records the winning
11
+ pairing as a fact on the Session node — derived once, read thereafter.
12
+
13
+ Multiple transcripts may legitimately follow one key (a Claude Code restart
14
+ resumes the same on-graph session), so this module returns ALL candidates
15
+ ranked, never a single guess.
16
+ """
17
+
18
+ import re
19
+ from dataclasses import dataclass
20
+ from datetime import datetime
21
+ from pathlib import Path
22
+
23
+ from .records import load_records
24
+
25
+ # The boot-prompt signal the session-start ritual includes for minted sessions
26
+ # — a PREFIX, substring-matched, so every minting seat qualifies ("…minted
27
+ # in-workbench", "…minted in-scratchpad") and every legacy boot still carries it.
28
+ MINT_SIGNAL = "New session minted in-"
29
+
30
+ _KEY_RE = re.compile(r"^\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$")
31
+
32
+ # Boot-time slack: the boot record must not predate the mint by more than this
33
+ # (small negative tolerance absorbs sub-second clock jitter, nothing more).
34
+ _SLACK_SECONDS = 60.0
35
+
36
+
37
+ def parse_session_key(key: str) -> datetime:
38
+ """A session key (local start timestamp, e.g. 2026-08-20_17-05-20) as an aware datetime."""
39
+ if not _KEY_RE.match(key):
40
+ raise ValueError(f"not a session key: {key!r}")
41
+ naive = datetime.strptime(key, "%Y-%m-%d_%H-%M-%S")
42
+ return naive.astimezone() # interpret in the local timezone, the key's home
43
+
44
+
45
+ def parse_record_timestamp(ts: str) -> datetime:
46
+ """A transcript record timestamp (ISO-8601, trailing Z) as an aware datetime."""
47
+ return datetime.fromisoformat(ts.replace("Z", "+00:00"))
48
+
49
+
50
+ @dataclass
51
+ class BootInfo:
52
+ """The first user prose of a transcript — the mapping's evidence."""
53
+ text: str
54
+ timestamp: datetime
55
+ has_mint_signal: bool
56
+
57
+
58
+ def read_boot_info(path: str | Path) -> BootInfo | None:
59
+ """The transcript's first user record bearing prose, or None (empty/tool-only file)."""
60
+ for rec in load_records(path):
61
+ if rec.type != "user" or rec.is_sidechain or rec.timestamp is None:
62
+ continue
63
+ content = rec.raw.get("message", {}).get("content")
64
+ if not isinstance(content, str):
65
+ continue
66
+ return BootInfo(
67
+ text=content,
68
+ timestamp=parse_record_timestamp(rec.timestamp),
69
+ has_mint_signal=MINT_SIGNAL in content,
70
+ )
71
+ return None
72
+
73
+
74
+ @dataclass
75
+ class TranscriptMatch:
76
+ """One ranked candidate pairing for a session key."""
77
+ path: Path
78
+ cc_session_uuid: str # The transcript's filename stem IS the CC session UUID
79
+ boot: BootInfo
80
+ delay_seconds: float # Boot time minus mint time; small = started right after mint
81
+
82
+
83
+ def find_transcripts_for_key(
84
+ transcript_dir: str | Path, session_key: str, *, require_signal: bool = True,
85
+ ) -> list[TranscriptMatch]:
86
+ """All plausible transcripts for a session key, best first.
87
+
88
+ A transcript qualifies when its boot prose follows the mint time (within
89
+ slack) — and, by default, carries the mint signal; pass
90
+ ``require_signal=False`` to also rank signal-less candidates (resumed or
91
+ manually booted sessions). Ordering: signal-bearing first, then smallest
92
+ delay after the mint."""
93
+ key_dt = parse_session_key(session_key)
94
+ matches: list[TranscriptMatch] = []
95
+ for path in Path(transcript_dir).glob("*.jsonl"):
96
+ boot = read_boot_info(path)
97
+ if boot is None:
98
+ continue
99
+ delay = (boot.timestamp - key_dt).total_seconds()
100
+ if delay < -_SLACK_SECONDS:
101
+ continue # booted before the mint — a different session
102
+ if require_signal and not boot.has_mint_signal:
103
+ continue
104
+ matches.append(TranscriptMatch(
105
+ path=path, cc_session_uuid=path.stem, boot=boot, delay_seconds=delay,
106
+ ))
107
+ matches.sort(key=lambda m: (not m.boot.has_mint_signal, m.delay_seconds))
108
+ return matches
@@ -0,0 +1,109 @@
1
+ """The transcript record layer: tolerant JSONL loading + the append-only DAG.
2
+
3
+ A Claude Code transcript is an append-only DAG, never a linear log: every
4
+ addressable record carries a ``parentUuid``, a rewind never truncates the file
5
+ (post-rewind records append with an EARLIER parent, creating a fork), and
6
+ timestamps strictly increase — no regression signal exists. The ACTIVE path is
7
+ therefore a derived view: walk back from the tip (the newest main-chain
8
+ record is always on it). Empirical basis: the 2026-08-20 walkthrough sitting's
9
+ 59-transcript scan (793 fork points) — DEC 671e9b11 point 8.
10
+
11
+ Loading is deliberately tolerant: the live transcript is appended by the
12
+ harness while a session runs, so a partially flushed final line is expected
13
+ and skipped, and non-addressable housekeeping records (``file-history-snapshot``,
14
+ ``ai-title``, ``mode``, ...) are carried as raw records but excluded from the DAG.
15
+ """
16
+
17
+ import json
18
+ from dataclasses import dataclass, field
19
+ from pathlib import Path
20
+
21
+
22
+ @dataclass
23
+ class TranscriptRecord:
24
+ """One JSONL record, uuid-addressable or not, with the raw dict preserved."""
25
+ type: str # Record type: user / assistant / system / housekeeping kinds
26
+ uuid: str | None # Addressable identity (None on housekeeping records)
27
+ parent_uuid: str | None # The DAG edge; None on roots and housekeeping records
28
+ timestamp: str | None # ISO-8601 as recorded (UTC, trailing Z)
29
+ is_sidechain: bool # True on embedded sub-agent traffic — excluded from the main DAG
30
+ raw: dict # The full record, untouched
31
+
32
+
33
+ def load_records(path: str | Path) -> list[TranscriptRecord]:
34
+ """Load a transcript JSONL tolerantly, in file order.
35
+
36
+ Skips unparseable lines (the live file's partially flushed tail) and
37
+ non-dict payloads; never raises on malformed content."""
38
+ records: list[TranscriptRecord] = []
39
+ with open(path, encoding="utf-8") as f:
40
+ for line in f:
41
+ line = line.strip()
42
+ if not line:
43
+ continue
44
+ try:
45
+ rec = json.loads(line)
46
+ except json.JSONDecodeError:
47
+ continue # partially flushed tail line of a live transcript
48
+ if not isinstance(rec, dict):
49
+ continue
50
+ records.append(TranscriptRecord(
51
+ type=rec.get("type", ""),
52
+ uuid=rec.get("uuid"),
53
+ parent_uuid=rec.get("parentUuid"),
54
+ timestamp=rec.get("timestamp"),
55
+ is_sidechain=bool(rec.get("isSidechain")),
56
+ raw=rec,
57
+ ))
58
+ return records
59
+
60
+
61
+ @dataclass
62
+ class TranscriptDag:
63
+ """The uuid-addressable view of a transcript: parent edges, forks, active path."""
64
+ records: list[TranscriptRecord]
65
+ by_uuid: dict[str, TranscriptRecord] = field(init=False)
66
+ children: dict[str, list[TranscriptRecord]] = field(init=False)
67
+
68
+ def __post_init__(self):
69
+ self.by_uuid = {}
70
+ self.children = {}
71
+ for rec in self.records:
72
+ if rec.uuid is None or rec.is_sidechain:
73
+ continue
74
+ self.by_uuid[rec.uuid] = rec
75
+ if rec.parent_uuid is not None:
76
+ self.children.setdefault(rec.parent_uuid, []).append(rec)
77
+
78
+ @classmethod
79
+ def load(cls, path: str | Path) -> "TranscriptDag":
80
+ """Build the DAG straight from a transcript file."""
81
+ return cls(load_records(path))
82
+
83
+ def tip(self) -> TranscriptRecord | None:
84
+ """The newest main-chain record — by construction always on the active path."""
85
+ for rec in reversed(self.records):
86
+ if rec.uuid is not None and not rec.is_sidechain:
87
+ return rec
88
+ return None
89
+
90
+ def active_path(self) -> list[TranscriptRecord]:
91
+ """The live conversation line, chronological: tip ancestry, forks excluded.
92
+
93
+ Records down abandoned (rewound) branches are real discourse events and
94
+ stay loaded — they are simply not on this walk. A cycle or a dangling
95
+ parent ends the walk rather than raising (defensive: the file is
96
+ external input)."""
97
+ path: list[TranscriptRecord] = []
98
+ seen: set[str] = set()
99
+ rec = self.tip()
100
+ while rec is not None and rec.uuid not in seen:
101
+ seen.add(rec.uuid)
102
+ path.append(rec)
103
+ rec = self.by_uuid.get(rec.parent_uuid) if rec.parent_uuid else None
104
+ path.reverse()
105
+ return path
106
+
107
+ def fork_points(self) -> dict[str, list[TranscriptRecord]]:
108
+ """Parents with more than one child — rewind points and turn-internal forks."""
109
+ return {p: kids for p, kids in self.children.items() if len(kids) > 1}
@@ -0,0 +1,51 @@
1
+ Metadata-Version: 2.4
2
+ Name: cjm-harness-transcripts
3
+ Version: 0.0.1
4
+ Summary: The harness-transcript extraction lib (DEC e8b2f397 increment ii; the d774d311 build item): parse agent-harness session transcripts (Claude Code JSONL first) into their append-only record DAG, walk the active path, extract user-facing prose both sides, and derive the transcript <-> on-graph session-key mapping. Pure-parse, zero graph dependencies — one extraction truth for the live scratchpad pull, the archive-absorb instrument, and the housekeeping miner. Born on-graph.
5
+ Author-email: "Christian J. Mills" <9126128+cj-mills@users.noreply.github.com>
6
+ License: Apache-2.0
7
+ Project-URL: Repository, https://github.com/cj-mills/cjm-harness-transcripts
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Programming Language :: Python :: 3 :: Only
10
+ Requires-Python: >=3.12
11
+ Description-Content-Type: text/markdown
12
+
13
+ # cjm-harness-transcripts
14
+
15
+ <!-- generated from the context graph by `cjm-context-graph readme` — do not edit by hand; edit the graph (the urge to hand-edit = move it on-graph) -->
16
+
17
+ Harness-transcript extraction library: reads Claude Code session transcripts (JSONL DAGs), walks the active path, and extracts the user-facing prose — assistant markdown bodies, cleaned user prompts, and curated tool-parameter prose (e.g. SendUserFile captions) — as ExtractedMessages ready to mint onto a context-graph session spine.
18
+
19
+ ## Modules
20
+
21
+ - **`cjm_harness_transcripts`** — cjm-harness-transcripts: parse agent-harness session transcripts (Claude Code
22
+ - **`cjm_harness_transcripts.extract`** — User-facing prose extraction — both sides, one truth.
23
+ - **`cjm_harness_transcripts.mapping`** — Transcript ↔ session-key mapping, derived at pull time.
24
+ - **`cjm_harness_transcripts.records`** — The transcript record layer: tolerant JSONL loading + the append-only DAG.
25
+
26
+ ## API
27
+
28
+ ### `cjm_harness_transcripts.extract`
29
+
30
+ - `ExtractedMessage` _class_ — One user-facing message off the active path, ready to become a Message node.
31
+ - `clean_user_text` _function_ — Strip harness wrapper blocks from a user prompt; collapse the scar tissue.
32
+ - `extract_messages` _function_ — The active path's user-facing messages, chronological.
33
+
34
+ ### `cjm_harness_transcripts.mapping`
35
+
36
+ - `BootInfo` _class_ — The first user prose of a transcript — the mapping's evidence.
37
+ - `TranscriptMatch` _class_ — One ranked candidate pairing for a session key.
38
+ - `find_transcripts_for_key` _function_ — All plausible transcripts for a session key, best first.
39
+ - `parse_record_timestamp` _function_ — A transcript record timestamp (ISO-8601, trailing Z) as an aware datetime.
40
+ - `parse_session_key` _function_ — A session key (local start timestamp, e.g. 2026-08-20_17-05-20) as an aware datetime.
41
+ - `read_boot_info` _function_ — The transcript's first user record bearing prose, or None (empty/tool-only file).
42
+
43
+ ### `cjm_harness_transcripts.records`
44
+
45
+ - `TranscriptDag` _class_ — The uuid-addressable view of a transcript: parent edges, forks, active path.
46
+ - `TranscriptRecord` _class_ — One JSONL record, uuid-addressable or not, with the raw dict preserved.
47
+ - `load_records` _function_ — Load a transcript JSONL tolerantly, in file order.
48
+
49
+ ## Dependencies
50
+
51
+ **Used by:** `cjm-context-graph-projection`
@@ -0,0 +1,13 @@
1
+ README.md
2
+ pyproject.toml
3
+ cjm_harness_transcripts/__init__.py
4
+ cjm_harness_transcripts/extract.py
5
+ cjm_harness_transcripts/mapping.py
6
+ cjm_harness_transcripts/records.py
7
+ cjm_harness_transcripts.egg-info/PKG-INFO
8
+ cjm_harness_transcripts.egg-info/SOURCES.txt
9
+ cjm_harness_transcripts.egg-info/dependency_links.txt
10
+ cjm_harness_transcripts.egg-info/top_level.txt
11
+ tests/test_extract.py
12
+ tests/test_mapping.py
13
+ tests/test_records.py
@@ -0,0 +1 @@
1
+ cjm_harness_transcripts
@@ -0,0 +1,20 @@
1
+ [build-system]
2
+ requires = ["setuptools>=64"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "cjm-harness-transcripts"
7
+ version = "0.0.1"
8
+ description = "The harness-transcript extraction lib (DEC e8b2f397 increment ii; the d774d311 build item): parse agent-harness session transcripts (Claude Code JSONL first) into their append-only record DAG, walk the active path, extract user-facing prose both sides, and derive the transcript <-> on-graph session-key mapping. Pure-parse, zero graph dependencies — one extraction truth for the live scratchpad pull, the archive-absorb instrument, and the housekeeping miner. Born on-graph."
9
+ readme = "README.md"
10
+ requires-python = ">=3.12"
11
+ license = {text = "Apache-2.0"}
12
+ authors = [{name = "Christian J. Mills", email = "9126128+cj-mills@users.noreply.github.com"}]
13
+ classifiers = ["Programming Language :: Python :: 3", "Programming Language :: Python :: 3 :: Only"]
14
+ dependencies = []
15
+
16
+ [project.urls]
17
+ Repository = "https://github.com/cj-mills/cjm-harness-transcripts"
18
+
19
+ [tool.setuptools.packages.find]
20
+ include = ["cjm_harness_transcripts*"]
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+
@@ -0,0 +1,109 @@
1
+ """Extraction: user-facing prose both sides, wrappers stripped, tool traffic dropped."""
2
+
3
+ from cjm_harness_transcripts.extract import (
4
+ TOOL_PARAM_SOURCE, clean_user_text, extract_messages)
5
+ from cjm_harness_transcripts.records import TranscriptDag
6
+
7
+ from conftest import rec
8
+
9
+
10
+ def test_clean_user_text_strips_harness_wrappers():
11
+ raw = (
12
+ "<local-command-caveat>Caveat: local commands.</local-command-caveat>\n"
13
+ "Real question here.\n\n"
14
+ "<system-reminder>\nbackground noise\n</system-reminder>\n"
15
+ )
16
+ assert clean_user_text(raw) == "Real question here."
17
+
18
+
19
+ def test_clean_user_text_strips_command_wrappers():
20
+ raw = "<command-name>/foo</command-name>\n<command-args>bar</command-args>\n"
21
+ assert clean_user_text(raw) == ""
22
+
23
+
24
+ def test_extract_keeps_both_sides_and_drops_tool_traffic(make_transcript):
25
+ path = make_transcript([
26
+ rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z",
27
+ content="<system-reminder>ctx</system-reminder>\nHello there"),
28
+ rec("assistant", "a1", "u1", ts="2026-08-20T22:00:03.000Z",
29
+ content=[{"type": "text", "text": "Looking now."},
30
+ {"type": "tool_use", "name": "Bash", "input": {}}]),
31
+ rec("user", "t1", "a1", ts="2026-08-20T22:00:04.000Z",
32
+ content=[{"type": "tool_result", "content": "output"}]),
33
+ rec("assistant", "a2", "t1", ts="2026-08-20T22:00:08.000Z",
34
+ content=[{"type": "thinking", "thinking": "private"},
35
+ {"type": "text", "text": "**Found it.**"}]),
36
+ ])
37
+ messages = extract_messages(TranscriptDag.load(path))
38
+ assert [(m.role, m.text) for m in messages] == [
39
+ ("user", "Hello there"),
40
+ ("assistant", "Looking now."),
41
+ ("assistant", "**Found it.**"), # raw markdown source, verbatim
42
+ ]
43
+ assert messages[0].uuid == "u1" and messages[0].parent_uuid is None
44
+
45
+
46
+ def test_extract_skips_abandoned_branch_messages(make_transcript):
47
+ path = make_transcript([
48
+ rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z", content="q"),
49
+ rec("user", "u2a", "u1", ts="2026-08-20T22:01:00.000Z", content="retracted"),
50
+ rec("user", "u2b", "u1", ts="2026-08-20T22:02:00.000Z", content="final form"),
51
+ ])
52
+ messages = extract_messages(TranscriptDag.load(path))
53
+ assert [m.text for m in messages] == ["q", "final form"]
54
+
55
+
56
+ def test_tool_only_and_empty_records_yield_nothing(make_transcript):
57
+ path = make_transcript([
58
+ rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z",
59
+ content="<local-command-caveat>only wrapper</local-command-caveat>"),
60
+ rec("assistant", "a1", "u1", ts="2026-08-20T22:00:03.000Z",
61
+ content=[{"type": "tool_use", "name": "Bash", "input": {}}]),
62
+ ])
63
+ assert extract_messages(TranscriptDag.load(path)) == []
64
+
65
+
66
+ def test_tool_param_prose_extracts_as_faceted_message(make_transcript):
67
+ # The finding-60d719fe class: a SendUserFile caption is user-facing prose
68
+ # living in tool_use input — it becomes its own message, block-id identity,
69
+ # cc-tool-param facet, after the carrier record's own text.
70
+ path = make_transcript([
71
+ rec("user", "u1", None, ts="2026-08-21T21:00:00.000Z", content="ship it"),
72
+ rec("assistant", "a1", "u1", ts="2026-08-21T21:00:05.000Z",
73
+ content=[{"type": "text", "text": "Exporting now."},
74
+ {"type": "tool_use", "id": "toolu_1", "name": "SendUserFile",
75
+ "input": {"files": ["report.md"],
76
+ "caption": "Here's the exported report."}}]),
77
+ ])
78
+ messages = extract_messages(TranscriptDag.load(path))
79
+ assert [(m.role, m.text) for m in messages] == [
80
+ ("user", "ship it"),
81
+ ("assistant", "Exporting now."),
82
+ ("assistant", "Here's the exported report."),
83
+ ]
84
+ caption = messages[-1]
85
+ assert caption.uuid == "toolu_1" and caption.parent_uuid == "a1"
86
+ assert caption.source == TOOL_PARAM_SOURCE
87
+ assert messages[0].source is None and messages[1].source is None
88
+
89
+
90
+ def test_tool_param_table_is_curated_and_optional(make_transcript):
91
+ # Blank/absent params and unlisted tools yield nothing; tool_params={}
92
+ # disables the lane outright.
93
+ quiet = make_transcript([
94
+ rec("assistant", "a1", None, ts="2026-08-21T21:00:00.000Z",
95
+ content=[{"type": "tool_use", "id": "t1", "name": "SendUserFile",
96
+ "input": {"files": ["x"], "caption": " "}},
97
+ {"type": "tool_use", "id": "t2", "name": "Bash",
98
+ "input": {"command": "ls"}},
99
+ {"type": "tool_use", "id": "t3", "name": "SendUserFile",
100
+ "input": {"files": ["y"]}}]),
101
+ ])
102
+ assert extract_messages(TranscriptDag.load(quiet)) == []
103
+ captioned = make_transcript([
104
+ rec("assistant", "a1", None, ts="2026-08-21T21:00:09.000Z",
105
+ content=[{"type": "tool_use", "id": "t4", "name": "SendUserFile",
106
+ "input": {"caption": "real"}}]),
107
+ ], name="bbbb1111-2222-3333-4444-555566667777.jsonl")
108
+ assert [m.text for m in extract_messages(TranscriptDag.load(captioned))] == ["real"]
109
+ assert extract_messages(TranscriptDag.load(captioned), tool_params={}) == []
@@ -0,0 +1,67 @@
1
+ """Mapping: session keys, boot info, and ranked transcript candidates."""
2
+
3
+ from datetime import timedelta, timezone
4
+
5
+ import pytest
6
+
7
+ from cjm_harness_transcripts.mapping import (
8
+ MINT_SIGNAL,
9
+ find_transcripts_for_key,
10
+ parse_session_key,
11
+ read_boot_info,
12
+ )
13
+
14
+ from conftest import rec, write_jsonl
15
+
16
+ KEY = "2026-08-20_17-05-20"
17
+
18
+
19
+ def iso_after(key, seconds):
20
+ """A UTC record timestamp `seconds` after the (local) session key."""
21
+ dt = parse_session_key(key) + timedelta(seconds=seconds)
22
+ return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.000Z")
23
+
24
+
25
+ def boot_transcript(tmp_path, name, key, delay, signal=True, text="Resume the project."):
26
+ prose = f"{text} {MINT_SIGNAL}." if signal else text
27
+ return write_jsonl(tmp_path / name, [
28
+ {"type": "file-history-snapshot", "snapshot": {}},
29
+ rec("user", "u1", None, ts=iso_after(key, delay), content=prose),
30
+ ])
31
+
32
+
33
+ def test_parse_session_key_is_local_and_strict():
34
+ dt = parse_session_key(KEY)
35
+ assert dt.tzinfo is not None
36
+ assert (dt.year, dt.hour, dt.second) == (2026, 17, 20)
37
+ with pytest.raises(ValueError):
38
+ parse_session_key("2026-08-20 17:05:20")
39
+
40
+
41
+ def test_read_boot_info_finds_first_user_prose(tmp_path):
42
+ path = boot_transcript(tmp_path, "aaaa.jsonl", KEY, 90)
43
+ boot = read_boot_info(path)
44
+ assert boot.has_mint_signal and MINT_SIGNAL in boot.text
45
+
46
+
47
+ def test_candidates_ranked_by_delay_signal_first(tmp_path):
48
+ boot_transcript(tmp_path, "11111111-aaaa-aaaa-aaaa-aaaaaaaaaaaa.jsonl", KEY, 3600)
49
+ boot_transcript(tmp_path, "22222222-bbbb-bbbb-bbbb-bbbbbbbbbbbb.jsonl", KEY, 90)
50
+ boot_transcript(tmp_path, "33333333-cccc-cccc-cccc-cccccccccccc.jsonl", KEY, -7200) # before mint
51
+ boot_transcript(tmp_path, "44444444-dddd-dddd-dddd-dddddddddddd.jsonl", KEY, 30, signal=False)
52
+
53
+ matches = find_transcripts_for_key(tmp_path, KEY)
54
+ assert [m.cc_session_uuid[:8] for m in matches] == ["22222222", "11111111"]
55
+ assert matches[0].delay_seconds == pytest.approx(90, abs=1)
56
+
57
+ loose = find_transcripts_for_key(tmp_path, KEY, require_signal=False)
58
+ assert [m.cc_session_uuid[:8] for m in loose] == ["22222222", "11111111", "44444444"]
59
+
60
+
61
+ def test_mint_signal_prefix_matches_every_minting_seat():
62
+ # has_mint_signal is a substring check, so the seat-agnostic PREFIX must
63
+ # sit inside both minting seats' boot phrases — legacy workbench boots
64
+ # keep matching, scratchpad-minted sessions start matching.
65
+ from cjm_harness_transcripts.mapping import MINT_SIGNAL
66
+ assert MINT_SIGNAL in "New session minted in-workbench"
67
+ assert MINT_SIGNAL in "New session minted in-scratchpad"
@@ -0,0 +1,62 @@
1
+ """The record layer: tolerant loading + DAG walks over rewind forks."""
2
+
3
+ from cjm_harness_transcripts.records import TranscriptDag, load_records
4
+
5
+ from conftest import rec
6
+
7
+
8
+ def test_load_tolerates_garbage_and_partial_tail(make_transcript):
9
+ path = make_transcript(
10
+ [rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z", content="hi")],
11
+ tail='{"type": "assistant", "uuid": "trunc', # live-file partially flushed line
12
+ )
13
+ path.write_text(path.read_text() + "\nnot json at all\n[1, 2, 3]\n", encoding="utf-8")
14
+ records = load_records(path)
15
+ assert [r.uuid for r in records] == ["u1"]
16
+
17
+
18
+ def test_housekeeping_records_load_but_stay_off_the_dag(make_transcript):
19
+ path = make_transcript([
20
+ {"type": "file-history-snapshot", "snapshot": {}},
21
+ rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z", content="hi"),
22
+ {"type": "ai-title", "title": "x"},
23
+ ])
24
+ dag = TranscriptDag.load(path)
25
+ assert len(dag.records) == 3
26
+ assert set(dag.by_uuid) == {"u1"}
27
+
28
+
29
+ def test_active_path_follows_tip_ancestry_past_a_rewind(make_transcript):
30
+ # u1 -> a1 -> u2a (abandoned) ; rewind: u2b appends later with parent a1
31
+ path = make_transcript([
32
+ rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z", content="q"),
33
+ rec("assistant", "a1", "u1", ts="2026-08-20T22:00:05.000Z", content=[{"type": "text", "text": "r"}]),
34
+ rec("user", "u2a", "a1", ts="2026-08-20T22:01:00.000Z", content="first try"),
35
+ rec("assistant", "a2a", "u2a", ts="2026-08-20T22:01:05.000Z", content=[{"type": "text", "text": "dead"}]),
36
+ rec("user", "u2b", "a1", ts="2026-08-20T22:02:00.000Z", content="second try"),
37
+ rec("assistant", "a2b", "u2b", ts="2026-08-20T22:02:05.000Z", content=[{"type": "text", "text": "live"}]),
38
+ ])
39
+ dag = TranscriptDag.load(path)
40
+ assert dag.tip().uuid == "a2b"
41
+ assert [r.uuid for r in dag.active_path()] == ["u1", "a1", "u2b", "a2b"]
42
+ assert set(dag.fork_points()) == {"a1"}
43
+
44
+
45
+ def test_sidechain_records_never_become_tip_or_path(make_transcript):
46
+ path = make_transcript([
47
+ rec("user", "u1", None, ts="2026-08-20T22:00:00.000Z", content="q"),
48
+ rec("assistant", "a1", "u1", ts="2026-08-20T22:00:05.000Z", content=[{"type": "text", "text": "r"}]),
49
+ rec("user", "s1", "a1", ts="2026-08-20T22:00:06.000Z", content="agent brief", sidechain=True),
50
+ rec("assistant", "s2", "s1", ts="2026-08-20T22:00:07.000Z", content=[{"type": "text", "text": "agent reply"}], sidechain=True),
51
+ ])
52
+ dag = TranscriptDag.load(path)
53
+ assert dag.tip().uuid == "a1"
54
+ assert [r.uuid for r in dag.active_path()] == ["u1", "a1"]
55
+
56
+
57
+ def test_dangling_parent_ends_walk_instead_of_raising(make_transcript):
58
+ path = make_transcript([
59
+ rec("assistant", "a9", "missing-parent", ts="2026-08-20T22:00:00.000Z",
60
+ content=[{"type": "text", "text": "orphan"}]),
61
+ ])
62
+ assert [r.uuid for r in TranscriptDag.load(path).active_path()] == ["a9"]