cc-transcript 2.0.0__tar.gz → 3.0.1__tar.gz

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cc-transcript
-Version: 2.0.0
+Version: 3.0.1
 Classifier: Development Status :: 3 - Alpha
 Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
@@ -16,16 +16,14 @@ Requires-Dist: orjson>=3.10
 Requires-Dist: pytest>=8.0 ; extra == 'dev'
 Requires-Dist: ty>=0.0.44 ; extra == 'dev'
 Requires-Dist: ruff>=0.8 ; extra == 'dev'
-Requires-Dist: cc-transcript[sentiment] ; extra == 'lexicon'
 Requires-Dist: spawnllm>=0.1.3 ; extra == 'llm'
 Requires-Dist: spacy>=3.8 ; extra == 'sentiment'
 Requires-Dist: afinn>=0.1 ; extra == 'sentiment'
 Provides-Extra: dev
-Provides-Extra: lexicon
 Provides-Extra: llm
 Provides-Extra: sentiment
 License-File: LICENSE
-Summary: The Claude Code session-activity platform: a superset JSONL parser (Rust fast path, Python reference), typed tool calls with a cross-language content digest, turns/edits/evidence over one spine, context windows with labeled fidelity, a unified decision ledger, LLM judging, and a transcript-investigation CLI.
+Summary: The Claude Code session-activity platform: a superset JSONL parser (Rust fast path, Python reference), typed tool calls with a cross-language content digest, turns/edits/evidence over one spine, context windows with labeled fidelity, unified decision and correction ledgers, LLM judging, and a transcript-investigation CLI.
 Keywords: claude-code,claude,anthropic,transcripts,jsonl,parser,cli,agents,sentiment,feedback-mining
 Author-email: Yasyf Mohamedali <yasyfm@gmail.com>
 License-Expression: PolyForm-Noncommercial-1.0.0
@@ -88,8 +86,7 @@ clean = list(apply_spec(events, spec))
 ```
 
 `NOISE_SPEC` is a ready-made spec for the universal structural noise (system reminders,
-local-command output, skill banners). For flag-style filtering, `FilterConfig` is also
-available — every rule is off by default, so a bare `FilterConfig()` passes everything through.
+local-command output, skill banners).
 
 ## The CLI
 
@@ -161,7 +158,7 @@ The plugin's skill teaches Claude to answer questions about its own history —
 ## What problems does this solve?
 
 - **One faithful parse.** Anything reading Claude Code transcripts re-implements the same JSONL quirks (str-or-list content, tool results nested two ways, envelope-less mode markers). This is that parser, written once and typed strictly.
-- **Non-lossy by design.** The event model is a superset: sidechains, `<synthetic>` turns, thinking blocks, and unrecognized entry types all survive parsing. You decide what to drop, via composable filter specs (`build_spec`) or `FilterConfig`.
+- **Non-lossy by design.** The event model is a superset: sidechains, `<synthetic>` turns, thinking blocks, and unrecognized entry types all survive parsing. You decide what to drop, via composable filter specs (`build_spec`).
 - **Incremental ingestion.** `FileStateStore` tracks per-file mtimes in SQLite (WAL, safe across concurrent tasks) so re-runs only reparse changed files, and you compose your own writes in the same transaction.
 - **Two engines, one contract.** A single `Backend` protocol with two implementations: `RustBackend` (PyO3 + rayon) is the default fast path, and `PythonBackend` is the readable reference — parity-asserted against each other. Filter specs are portable, so a spec built in Python runs Rust-side without giving up the fast path.
 - **One activity spine.** `SessionActivity` lifts parsed events into turns, typed tool calls, and first-class edits; `ContextWindow` persists `EventRef`s plus labeled previews and hydrates back to full fidelity while the transcript lives; rendering is `Budget`-bounded in exactly one place.

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/README.md

@@ -49,8 +49,7 @@ clean = list(apply_spec(events, spec))
 ```
 
 `NOISE_SPEC` is a ready-made spec for the universal structural noise (system reminders,
-local-command output, skill banners). For flag-style filtering, `FilterConfig` is also
-available — every rule is off by default, so a bare `FilterConfig()` passes everything through.
+local-command output, skill banners).
 
 ## The CLI
 
@@ -122,7 +121,7 @@ The plugin's skill teaches Claude to answer questions about its own history —
 ## What problems does this solve?
 
 - **One faithful parse.** Anything reading Claude Code transcripts re-implements the same JSONL quirks (str-or-list content, tool results nested two ways, envelope-less mode markers). This is that parser, written once and typed strictly.
-- **Non-lossy by design.** The event model is a superset: sidechains, `<synthetic>` turns, thinking blocks, and unrecognized entry types all survive parsing. You decide what to drop, via composable filter specs (`build_spec`) or `FilterConfig`.
+- **Non-lossy by design.** The event model is a superset: sidechains, `<synthetic>` turns, thinking blocks, and unrecognized entry types all survive parsing. You decide what to drop, via composable filter specs (`build_spec`).
 - **Incremental ingestion.** `FileStateStore` tracks per-file mtimes in SQLite (WAL, safe across concurrent tasks) so re-runs only reparse changed files, and you compose your own writes in the same transaction.
 - **Two engines, one contract.** A single `Backend` protocol with two implementations: `RustBackend` (PyO3 + rayon) is the default fast path, and `PythonBackend` is the readable reference — parity-asserted against each other. Filter specs are portable, so a spec built in Python runs Rust-side without giving up the fast path.
 - **One activity spine.** `SessionActivity` lifts parsed events into turns, typed tool calls, and first-class edits; `ContextWindow` persists `EventRef`s plus labeled previews and hydrates back to full fidelity while the transcript lives; rendering is `Budget`-bounded in exactly one place.

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/__init__.py

@@ -101,6 +101,13 @@ EXPORTS: dict[str, str] = {
             "git_corrections",
             "harvest_pairs",
             "match_corrections",
+            "record_harvest",
+        ),
+        "cc_transcript.corrections": (
+            "CORRECTIONS_DDL",
+            "Correction",
+            "CorrectionLog",
+            "Origin",
         ),
         "cc_transcript.context": (
             "ContextWindow",
@@ -121,10 +128,10 @@ EXPORTS: dict[str, str] = {
             "export_activity",
             "load_export",
         ),
-        "cc_transcript.filters": ("JUNK_USER_MESSAGE_RE", "FilterConfig", "apply_filters"),
         "cc_transcript.filterspec": (
             "ASSISTANTS",
             "INTERRUPT_MARKER_RE",
+            "JUNK_USER_MESSAGE_RE",
             "RESUME_PHRASE_SET",
             "STOP_HOOK_RE",
             "STRUCTURAL_NOISE_RE",
@@ -251,6 +258,18 @@ if TYPE_CHECKING:
     from cc_transcript.context import (
         capture_window as capture_window,
     )
+    from cc_transcript.corrections import (
+        CORRECTIONS_DDL as CORRECTIONS_DDL,
+    )
+    from cc_transcript.corrections import (
+        Correction as Correction,
+    )
+    from cc_transcript.corrections import (
+        CorrectionLog as CorrectionLog,
+    )
+    from cc_transcript.corrections import (
+        Origin as Origin,
+    )
     from cc_transcript.decisions import (
         DECISIONS_DDL as DECISIONS_DDL,
     )
@@ -317,14 +336,8 @@ if TYPE_CHECKING:
     from cc_transcript.evidence import (
         match_corrections as match_corrections,
     )
-    from cc_transcript.filters import (
-        JUNK_USER_MESSAGE_RE as JUNK_USER_MESSAGE_RE,
-    )
-    from cc_transcript.filters import (
-        FilterConfig as FilterConfig,
-    )
-    from cc_transcript.filters import (
-        apply_filters as apply_filters,
+    from cc_transcript.evidence import (
+        record_harvest as record_harvest,
     )
     from cc_transcript.filterspec import (
         ASSISTANTS as ASSISTANTS,
@@ -332,6 +345,9 @@ if TYPE_CHECKING:
     from cc_transcript.filterspec import (
         INTERRUPT_MARKER_RE as INTERRUPT_MARKER_RE,
     )
+    from cc_transcript.filterspec import (
+        JUNK_USER_MESSAGE_RE as JUNK_USER_MESSAGE_RE,
+    )
     from cc_transcript.filterspec import (
         RESUME_PHRASE_SET as RESUME_PHRASE_SET,
     )

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/activity.py

@@ -20,7 +20,7 @@ from cc_transcript.discovery import TranscriptDiscovery, TranscriptExpiredError,
 from cc_transcript.ids import EventRef
 from cc_transcript.models import AssistantEvent, SystemEvent, ToolResultBlock, ToolUseBlock, UserEvent
 from cc_transcript.parser import TranscriptParser
-from cc_transcript.tools import file_path_of, hunks_of
+from cc_transcript.tools import file_path_of, hunks_of, parse_tool_call
 
 if TYPE_CHECKING:
     from collections.abc import Mapping, Sequence
@@ -299,7 +299,7 @@ def lift_turn(
         tool_uses=tuple(
             ToolUse(
                 ref=EventRef(session_id, event.meta.uuid, block.id),
-                call=block.call,
+                call=parse_tool_call(block.name, block.input, on_error="other"),
                 result=results.get(block.id),
                 turn_index=index,
                 ts=event.meta.timestamp,

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/context.py

@@ -24,11 +24,10 @@ if TYPE_CHECKING:
 
     from cc_transcript.activity import Turn
 
-SCHEMA = "cc-transcript.context/1"
+SCHEMA = "cc-transcript.context/2"
 SUMMARY_LABEL = "[summary fidelity — transcript unavailable]"
 
 type Fidelity = Literal["full", "summary"]
-type Origin = Literal["live", "migrated"]
 type Role = Literal["user", "assistant"]
 
 
@@ -43,8 +42,7 @@ class TurnRef:
     Attributes:
         role: Whether a user prompt opened the turn or it is assistant-only
             preamble.
-        refs: References to the turn's events; may be ``()`` for migrated
-            legacy rows.
+        refs: References to the turn's events.
         preview: The turn rendered at capture time, at the preview budget.
         tool_digests: Content digests of the turn's tool calls, in order.
     """
@@ -60,16 +58,14 @@ class ContextWindow:
     """The turns around an anchor event, persisted as refs plus previews.
 
     Attributes:
-        anchor: The event the window centers on; None for migrated legacy
-            rows, which can never hydrate.
+        anchor: The event the window centers on.
         before: Turns preceding the anchor's turn, oldest first.
-        trigger: The anchor's own turn.
+        trigger: The anchor's own turn, or None when the window carries no
+            trigger preview.
         after: Turns following the anchor's turn, oldest first.
         fidelity: ``'full'`` while the transcript backs the window;
             ``'summary'`` once only the previews remain.
         preview_chars: The preview budget the window was captured at.
-        origin: ``'live'`` for captured windows, ``'migrated'`` for rows
-            converted from legacy stores.
 
     Example:
         >>> window = capture_window(activity, anchor)
@@ -77,13 +73,12 @@ class ContextWindow:
         >>> text = hydrated.render(budget=Budget()) if hydrated else window.render_preview(budget=Budget())
     """
 
-    anchor: EventRef | None
+    anchor: EventRef
     before: tuple[TurnRef, ...]
     trigger: TurnRef | None
     after: tuple[TurnRef, ...]
     fidelity: Fidelity
     preview_chars: int
-    origin: Origin
 
     def render_preview(self, *, budget: Budget) -> str:
         """Render the persisted previews, never touching the transcript.
@@ -102,12 +97,10 @@ class ContextWindow:
         """Resolve every ref back to real turns for full-fidelity rendering.
 
         Returns:
-            None when the anchor is absent, the transcript has expired, or
-            any ref was compacted away — callers fall back to
-            :meth:`render_preview`, never hydrate-or-fail.
+            None when the transcript has expired or any ref was compacted
+            away — callers fall back to :meth:`render_preview`, never
+            hydrate-or-fail.
         """
-        if self.anchor is None:
-            return None
         try:
             activity = await SessionActivity.from_session(self.anchor.session_id)
         except TranscriptExpiredError:
@@ -121,17 +114,16 @@ class ContextWindow:
         return HydratedWindow(window=self, turns=tuple(turns))
 
     def to_json(self) -> str:
-        """Serialize to the ``cc-transcript.context/1`` wire schema, byte-stably."""
+        """Serialize to the ``cc-transcript.context/2`` wire schema, byte-stably."""
         return json.dumps(
             {
                 "schema": SCHEMA,
-                "anchor": None if self.anchor is None else ref_payload(self.anchor),
+                "anchor": ref_payload(self.anchor),
                 "before": [turn_ref_payload(ref) for ref in self.before],
                 "trigger": None if self.trigger is None else turn_ref_payload(self.trigger),
                 "after": [turn_ref_payload(ref) for ref in self.after],
                 "fidelity": self.fidelity,
                 "preview_chars": self.preview_chars,
-                "origin": self.origin,
             },
             ensure_ascii=False,
             separators=(",", ":"),
@@ -144,19 +136,18 @@ class ContextWindow:
 
         Raises:
             SchemaError: When ``data`` does not carry the literal
-                ``cc-transcript.context/1`` schema.
+                ``cc-transcript.context/2`` schema.
         """
         payload = json.loads(data)
         if not isinstance(payload, dict) or payload.get("schema") != SCHEMA:
             raise SchemaError(f"expected schema {SCHEMA!r}, got: {data[:120]}")
         return cls(
-            anchor=None if payload["anchor"] is None else ref_from(payload["anchor"]),
+            anchor=ref_from(payload["anchor"]),
             before=tuple(turn_ref_from(item) for item in payload["before"]),
             trigger=None if payload["trigger"] is None else turn_ref_from(payload["trigger"]),
             after=tuple(turn_ref_from(item) for item in payload["after"]),
             fidelity=payload["fidelity"],
             preview_chars=payload["preview_chars"],
-            origin=payload["origin"],
         )
 
 
@@ -213,7 +204,6 @@ def capture_window(
         after=tuple(turn_ref(turn, budget) for turn in activity.turns[trigger.index + 1 : trigger.index + 1 + after]),
         fidelity="full",
         preview_chars=preview_chars,
-        origin="live",
     )
 
 

cc_transcript-3.0.1/cc_transcript/corrections.py

@@ -0,0 +1,231 @@
+"""The shared code-correction ledger every consumer reads and writes.
+
+One SQLite table (``corrections_v1``) records each incorrect edit a developer
+pushed back on paired with the correction that later overwrote it — from the
+same session or from git history. cc-pushback writes it from its enrich pass;
+captain-hook and cc-review join it to their own records by ``incorrect_digest``,
+the same cross-language tool digest the decision ledger keys on. The DDL below
+is vendored byte-identical into the Go repo and byte-compared in CI.
+
+Import-light by contract, like :mod:`cc_transcript.ids` and
+:mod:`cc_transcript.decisions`: the standard library plus identity primitives
+only, so a hook reading corrections pays nothing for the parser. The harvest
+that produces rows lives in :func:`cc_transcript.evidence.record_harvest`,
+which is allowed the heavier activity imports.
+"""
+
+from __future__ import annotations
+
+import json
+import sqlite3
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import TYPE_CHECKING, Literal, Self
+
+if TYPE_CHECKING:
+    from collections.abc import Mapping
+    from typing import Any
+
+    from cc_transcript.ids import EventUuid, SessionId, ToolDigest
+
+CORRECTIONS_DDL = """\
+CREATE TABLE IF NOT EXISTS corrections_v1 (
+    id INTEGER PRIMARY KEY,
+    ts_ms INTEGER NOT NULL,
+    session_id TEXT NOT NULL,
+    source TEXT NOT NULL,
+    anchor_uuid TEXT NOT NULL,
+    incorrect_digest TEXT NOT NULL,
+    incorrect_file TEXT NOT NULL,
+    incorrect_old TEXT NOT NULL,
+    incorrect_new TEXT NOT NULL,
+    correction_origin TEXT CHECK (correction_origin IN ('session', 'git')),
+    correction_file TEXT,
+    correction_old TEXT,
+    correction_new TEXT,
+    correction_commit TEXT,
+    overlap REAL NOT NULL DEFAULT 0,
+    extractor_version INTEGER NOT NULL,
+    detail_json TEXT NOT NULL DEFAULT '{}',
+    UNIQUE (session_id, anchor_uuid, incorrect_digest, extractor_version)
+);
+
+CREATE INDEX IF NOT EXISTS idx_corrections_v1_session_ts ON corrections_v1 (session_id, ts_ms);
+
+CREATE INDEX IF NOT EXISTS idx_corrections_v1_incorrect_digest ON corrections_v1 (incorrect_digest);
+"""
+
+Origin = Literal["session", "git"]
+
+CORRECTION_COLUMNS = (
+    "ts_ms, session_id, source, anchor_uuid, incorrect_digest, incorrect_file, incorrect_old, "
+    "incorrect_new, correction_origin, correction_file, correction_old, correction_new, "
+    "correction_commit, overlap, extractor_version, detail_json"
+)
+
+
+@dataclass(frozen=True, slots=True)
+class Correction:
+    """One incorrect edit and the correction that later overwrote it.
+
+    Attributes:
+        ts_ms: Integer-millisecond timestamp of the incorrect edit.
+        session_id: The Claude session UUID the edit fired in.
+        source: The writing system, e.g. ``cc-pushback``.
+        anchor_uuid: The transcript uuid of the feedback the harvest anchored on.
+        incorrect_digest: The cross-language content digest of the incorrect
+            edit's tool call — the join key shared with ``decisions_v1``.
+        incorrect_file: The file the incorrect edit targeted.
+        incorrect_old: The content the incorrect edit replaced (hunks joined).
+        incorrect_new: The content the incorrect edit wrote (hunks joined).
+        extractor_version: The deterministic-extraction version that derived the
+            pair; part of the UNIQUE key.
+        correction_origin: ``'session'`` or ``'git'``, or None when no
+            correction was found.
+        correction_file: The file the correction touched, when one exists.
+        correction_old: The content the correction replaced, when one exists.
+        correction_new: The content the correction wrote, when one exists.
+        correction_commit: The full commit hash, when the correction is a git fix.
+        overlap: The hunk-overlap score linking incorrect and correction; 0.0
+            when there is no correction.
+        detail: Structured extras, serialized to ``detail_json``.
+    """
+
+    ts_ms: int
+    session_id: SessionId
+    source: str
+    anchor_uuid: EventUuid
+    incorrect_digest: ToolDigest
+    incorrect_file: str
+    incorrect_old: str
+    incorrect_new: str
+    extractor_version: int
+    correction_origin: Origin | None = None
+    correction_file: str | None = None
+    correction_old: str | None = None
+    correction_new: str | None = None
+    correction_commit: str | None = None
+    overlap: float = 0.0
+    detail: Mapping[str, Any] = field(default_factory=dict)
+
+
+class CorrectionLog:
+    """The ``corrections_v1`` ledger at ``~/.cc-transcript/corrections.db``.
+
+    Opened in WAL mode with a busy timeout because writers across the family
+    (and cc-review's Go port) touch the same file concurrently. Durable by
+    convention: rows are never auto-dropped. Requires a local disk — WAL does
+    not work over NFS.
+
+    Example:
+        >>> log = CorrectionLog.open()
+        >>> log.append(correction)
+        >>> log.by_digest(session_id, incorrect_digest=digest)
+    """
+
+    def __init__(self, conn: sqlite3.Connection) -> None:
+        self.conn = conn
+
+    @classmethod
+    def open(cls, path: Path | None = None) -> Self:
+        """Opens (creating if needed) the ledger at ``path``.
+
+        Args:
+            path: The database file path; its parents are created if absent.
+                Defaults to ``~/.cc-transcript/corrections.db``.
+
+        Returns:
+            The opened log.
+        """
+        path = path or Path.home() / ".cc-transcript" / "corrections.db"
+        path.parent.mkdir(parents=True, exist_ok=True)
+        conn = sqlite3.connect(path, autocommit=True)
+        conn.row_factory = sqlite3.Row
+        conn.execute("PRAGMA journal_mode = WAL")
+        conn.execute("PRAGMA busy_timeout = 2000")
+        conn.executescript(CORRECTIONS_DDL)
+        return cls(conn)
+
+    def append(self, correction: Correction) -> None:
+        """Appends ``correction`` as a single ``INSERT OR IGNORE``.
+
+        Idempotent on the UNIQUE key ``(session_id, anchor_uuid,
+        incorrect_digest, extractor_version)`` — re-harvesting the same edit,
+        across reruns or across the pairs of one event, writes one row.
+        """
+        self.conn.execute(
+            f"INSERT OR IGNORE INTO corrections_v1 ({CORRECTION_COLUMNS}) "
+            "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
+            (
+                correction.ts_ms,
+                correction.session_id,
+                correction.source,
+                correction.anchor_uuid,
+                correction.incorrect_digest,
+                correction.incorrect_file,
+                correction.incorrect_old,
+                correction.incorrect_new,
+                correction.correction_origin,
+                correction.correction_file,
+                correction.correction_old,
+                correction.correction_new,
+                correction.correction_commit,
+                correction.overlap,
+                correction.extractor_version,
+                json.dumps(dict(correction.detail)),
+            ),
+        )
+
+    def for_session(self, session_id: SessionId) -> tuple[Correction, ...]:
+        """All corrections for ``session_id``, ordered by timestamp."""
+        return tuple(
+            correction_of(row)
+            for row in self.conn.execute(
+                "SELECT * FROM corrections_v1 WHERE session_id = ? ORDER BY ts_ms, id", (session_id,)
+            )
+        )
+
+    def for_anchor(self, session_id: SessionId, anchor_uuid: EventUuid) -> tuple[Correction, ...]:
+        """The corrections harvested around one feedback ``anchor_uuid``."""
+        return tuple(
+            correction_of(row)
+            for row in self.conn.execute(
+                "SELECT * FROM corrections_v1 WHERE session_id = ? AND anchor_uuid = ? ORDER BY ts_ms, id",
+                (session_id, anchor_uuid),
+            )
+        )
+
+    def by_digest(self, session_id: SessionId, *, incorrect_digest: ToolDigest) -> tuple[Correction, ...]:
+        """Corrections of the tool call with ``incorrect_digest`` in ``session_id``.
+
+        The cross-consumer join: pass the ``tool_digest`` a hook recorded in
+        ``decisions_v1`` to learn whether that exact edit was later corrected.
+        """
+        return tuple(
+            correction_of(row)
+            for row in self.conn.execute(
+                "SELECT * FROM corrections_v1 WHERE session_id = ? AND incorrect_digest = ? ORDER BY ts_ms, id",
+                (session_id, incorrect_digest),
+            )
+        )
+
+
+def correction_of(row: sqlite3.Row) -> Correction:
+    return Correction(
+        ts_ms=row["ts_ms"],
+        session_id=row["session_id"],
+        source=row["source"],
+        anchor_uuid=row["anchor_uuid"],
+        incorrect_digest=row["incorrect_digest"],
+        incorrect_file=row["incorrect_file"],
+        incorrect_old=row["incorrect_old"],
+        incorrect_new=row["incorrect_new"],
+        extractor_version=row["extractor_version"],
+        correction_origin=row["correction_origin"],
+        correction_file=row["correction_file"],
+        correction_old=row["correction_old"],
+        correction_new=row["correction_new"],
+        correction_commit=row["correction_commit"],
+        overlap=row["overlap"],
+        detail=json.loads(row["detail_json"]),
+    )

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/evidence.py

@@ -15,12 +15,17 @@ from datetime import UTC, datetime
 from typing import TYPE_CHECKING
 
 from cc_transcript.activity import hunk_overlap
+from cc_transcript.corrections import Correction
+from cc_transcript.ids import tool_digest
+from cc_transcript.models import AssistantEvent, ToolUseBlock
 from cc_transcript.tools import Hunk
 
 if TYPE_CHECKING:
+    from collections.abc import Sequence
     from pathlib import Path
 
     from cc_transcript.activity import Edit, SessionActivity
+    from cc_transcript.corrections import CorrectionLog, Origin
     from cc_transcript.ids import EventRef
 
 EXTRACTOR_VERSION = 1
@@ -196,6 +201,94 @@ def run_git(repo: Path, *args: str) -> str | None:
     return proc.stdout if proc.returncode == 0 else None
 
 
+def record_harvest(
+    log: CorrectionLog,
+    activity: SessionActivity,
+    anchor: EventRef,
+    pairs: Sequence[CandidatePair],
+    *,
+    source: str,
+    extractor_version: int = EXTRACTOR_VERSION,
+) -> int:
+    """Records ``pairs`` harvested around ``anchor`` into the shared ledger.
+
+    Lowers each :class:`CandidatePair` to a :class:`~cc_transcript.corrections.Correction`
+    — resolving the incorrect edit's raw tool call to the cross-language
+    ``incorrect_digest`` — and appends it. Idempotent: re-recording the same
+    harvest writes nothing new.
+
+    Returns:
+        The number of corrections appended this call.
+    """
+    rows = [
+        correction
+        for pair in pairs
+        if (correction := lower_pair(activity, anchor, pair, source=source, extractor_version=extractor_version))
+    ]
+    for row in rows:
+        log.append(row)
+    return len(rows)
+
+
+def lower_pair(
+    activity: SessionActivity, anchor: EventRef, pair: CandidatePair, *, source: str, extractor_version: int
+) -> Correction | None:
+    if (block := incorrect_block(activity, pair.incorrect)) is None:
+        return None
+    incorrect_old, incorrect_new = joined_hunks(pair.incorrect.hunks)
+    origin, correction_file, correction_old, correction_new, commit = correction_columns(pair.correction)
+    return Correction(
+        ts_ms=int(pair.incorrect.ts.timestamp() * 1000),
+        session_id=anchor.session_id,
+        source=source,
+        anchor_uuid=anchor.event_uuid,
+        incorrect_digest=tool_digest(block.name, block.input),
+        incorrect_file=pair.incorrect.file_path,
+        incorrect_old=incorrect_old,
+        incorrect_new=incorrect_new,
+        extractor_version=extractor_version,
+        correction_origin=origin,
+        correction_file=correction_file,
+        correction_old=correction_old,
+        correction_new=correction_new,
+        correction_commit=commit,
+        overlap=pair.overlap,
+    )
+
+
+def incorrect_block(activity: SessionActivity, edit: Edit) -> ToolUseBlock | None:
+    if (turn := activity.turn_of(edit.ref)) is None:
+        return None
+    return next(
+        (
+            block
+            for event in turn.events
+            if isinstance(event, AssistantEvent) and event.meta.uuid == edit.ref.event_uuid
+            for block in event.blocks
+            if isinstance(block, ToolUseBlock) and block.id == edit.ref.tool_use_id
+        ),
+        None,
+    )
+
+
+def joined_hunks(hunks: Sequence[Hunk]) -> tuple[str, str]:
+    return "\n".join(hunk.old for hunk in hunks), "\n".join(hunk.new for hunk in hunks)
+
+
+def correction_columns(
+    correction: Edit | GitFix | None,
+) -> tuple[Origin | None, str | None, str | None, str | None, str | None]:
+    match correction:
+        case None:
+            return None, None, None, None, None
+        case GitFix(file_path=file_path, hunks=hunks, commit=commit):
+            old, new = joined_hunks(hunks)
+            return "git", file_path, old, new, commit
+        case edit:
+            old, new = joined_hunks(edit.hunks)
+            return "session", edit.file_path, old, new, None
+
+
 def parse_show_hunks(diff: str) -> tuple[Hunk, ...]:
     sections: list[tuple[list[str], list[str]]] = []
     for line in diff.splitlines():

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/filterspec.py

@@ -512,3 +512,4 @@ def predicate_to_dict(predicate: Predicate) -> dict[str, Any]:
 STRUCTURAL_NOISE_RE = compile_groups(STRUCTURAL_NOISE_GROUPS, True)
 INTERRUPT_MARKER_RE = compile_groups(INTERRUPT_MARKER_GROUPS, True)
 STOP_HOOK_RE = compile_groups(STOP_HOOK_GROUPS, True)
+JUNK_USER_MESSAGE_RE = compile_groups(SENTIMENT_JUNK_GROUPS, True)

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/mining/__init__.py

@@ -25,7 +25,6 @@ from cc_transcript.mining.confidence import (
     VERY_HIGH,
     CandidateSignal,
     Confidence,
-    effective_confidence,
     firm,
     noise,
     strong,

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/mining/candidates.py

@@ -31,11 +31,10 @@ class FeedbackCandidate:
         text: The verbatim pushback text.
         window: The durable context window around the feedback, captured via
             :func:`~cc_transcript.context.capture_window`.
-        ref: The resolvable reference to the originating event; None for
-            migrated legacy rows.
+        ref: The resolvable reference to the originating event.
+        signal: The de-noising confidence signal.
         session_id: The transcript session the feedback came from.
         cc_version: The Claude Code version recorded for the origin.
-        signal: The de-noising confidence signal, when computed.
         payload: Detector-specific metadata preserved verbatim.
     """
 
@@ -44,10 +43,10 @@ class FeedbackCandidate:
     occurred_at: datetime
     text: str
     window: ContextWindow
-    ref: EventRef | None = None
+    ref: EventRef
+    signal: CandidateSignal
     session_id: SessionId | None = None
     cc_version: str | None = None
-    signal: CandidateSignal | None = None
     payload: Mapping[str, Any] | None = None
 
 

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/mining/confidence.py

@@ -55,18 +55,11 @@ def noise(*reasons: str, durable: bool = True) -> CandidateSignal:
     return CandidateSignal(NONE, reasons, durable)
 
 
-def effective_confidence(signal: CandidateSignal | None) -> Confidence:
-    """Returns ``signal``'s confidence, or :data:`MEDIUM` when no signal is set."""
-    return signal.confidence if signal else MEDIUM
-
-
 def to_payload(signal: CandidateSignal) -> dict[str, Any]:
     return {"confidence": signal.confidence, "reasons": list(signal.reasons), "durable": signal.durable}
 
 
-def from_payload(data: Mapping[str, Any] | None) -> CandidateSignal | None:
-    if data is None:
-        return None
+def from_payload(data: Mapping[str, Any]) -> CandidateSignal:
     return CandidateSignal(
         confidence=Confidence(data["confidence"]),
         reasons=tuple(data["reasons"]),

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/mining/filterspec.py

@@ -18,8 +18,6 @@ from __future__ import annotations
 from dataclasses import dataclass
 from typing import TYPE_CHECKING
 
-from cc_transcript.mining.confidence import effective_confidence
-
 if TYPE_CHECKING:
     from collections.abc import Iterable, Iterator
 
@@ -30,13 +28,7 @@ if TYPE_CHECKING:
 
 @dataclass(frozen=True, slots=True)
 class ConfidenceAtLeast:
-    """Matches candidates whose effective confidence is at least ``floor``.
-
-    A candidate without a stored signal scores
-    :data:`~cc_transcript.mining.confidence.MEDIUM` via
-    :func:`~cc_transcript.mining.confidence.effective_confidence`, so
-    legacy rows pass any floor at or below it.
-    """
+    """Matches candidates whose signal confidence is at least ``floor``."""
 
     floor: Confidence
 
@@ -57,11 +49,7 @@ class HasReason:
 
 @dataclass(frozen=True, slots=True)
 class IsDurable:
-    """Matches candidates whose signal durability equals ``want``.
-
-    A candidate without a stored signal counts as durable, mirroring
-    :class:`~cc_transcript.mining.confidence.CandidateSignal`'s default.
-    """
+    """Matches candidates whose signal durability equals ``want``."""
 
     want: bool
 
@@ -92,13 +80,13 @@ class CandidateFilterSpec:
 def predicate_matches(predicate: CandidatePredicate, candidate: FeedbackCandidate) -> bool:
     match predicate:
         case ConfidenceAtLeast(floor):
-            return effective_confidence(candidate.signal) >= floor
+            return candidate.signal.confidence >= floor
         case SourceKindIn(kinds):
             return candidate.source_kind in kinds
         case HasReason(reason):
-            return candidate.signal is not None and reason in candidate.signal.reasons
+            return reason in candidate.signal.reasons
         case IsDurable(want):
-            return (candidate.signal is None or candidate.signal.durable) is want
+            return candidate.signal.durable is want
 
 
 def keep_candidate(candidate: FeedbackCandidate, spec: CandidateFilterSpec) -> bool:

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/mining/signals.py

@@ -12,8 +12,7 @@ Every signal carries a calibrated :class:`CandidateSignal` spanning the full
 confidence band: arithmetic bumps and demotions over the anchors, with named
 reason codes (``trigger_proximate``, ``short_followup``, ``substantive``,
 ``hedged``, ``embedded_text``, ``bare_marker``, ``structural_only``) so apps can
-filter on :func:`~cc_transcript.mining.confidence.effective_confidence`
-and reasons instead of re-deriving them.
+filter on the signal's confidence and reasons instead of re-deriving them.
 """
 
 from __future__ import annotations
@@ -71,9 +70,9 @@ class MiningSignal:
         cc_version: The Claude Code version recorded for the origin.
         trigger_index: The nearest preceding assistant index, or None — a hint the
             app may use; absence never disqualifies the fact here.
+        signal: The de-noising confidence signal; apps re-derive as needed.
         lower_bound: A context anchor, such as the plan-reentry edit index.
         evidence: Detector-specific metadata preserved verbatim.
-        signal: The de-noising confidence signal; apps re-derive as needed.
     """
 
     kind: SourceKind
@@ -85,9 +84,9 @@ class MiningSignal:
     text: str
     cc_version: CcVersion | None
     trigger_index: int | None
+    signal: CandidateSignal
     lower_bound: int | None = None
     evidence: Mapping[str, Any] = field(default_factory=dict)
-    signal: CandidateSignal | None = None
 
 
 class ScoredText(NamedTuple):

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/mining/store.py

@@ -50,16 +50,15 @@ def now() -> str:
 
 def event_row(candidate: FeedbackCandidate, ingested_at: str) -> tuple[object, ...]:
     payload = dict(candidate.payload or {})
-    if candidate.signal is not None:
-        payload["signal"] = to_payload(candidate.signal)
+    payload["signal"] = to_payload(candidate.signal)
     return (
         candidate.dedup_key,
         candidate.source_kind,
         candidate.session_id,
-        candidate.ref.event_uuid if candidate.ref is not None else None,
+        candidate.ref.event_uuid,
         candidate.occurred_at.isoformat(),
         candidate.text,
-        json.dumps(payload) if payload else None,
+        json.dumps(payload),
         candidate.window.to_json(),
         candidate.cc_version,
         ingested_at,

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/models.py

@@ -207,3 +207,4 @@ class OtherEvent:
 
 
 TranscriptEvent = UserEvent | AssistantEvent | SystemEvent | ModeEvent | OtherEvent
+"""The union of every typed event a parsed transcript can yield."""

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/cc_transcript/tools.py

@@ -226,10 +226,11 @@ def parse_tool_call(
     """Parse a tool's name and raw input into the typed hierarchy.
 
     Strict by default: a known tool whose input is malformed raises
-    :class:`ToolInputError` (leniency lives in tests). The hook runtime alone
-    passes ``on_error='other'`` so a Claude Code shape change degrades the
-    call to :class:`OtherCall` — with a still-correct digest, since the raw
-    mapping is the substrate — instead of crashing every hook fire.
+    :class:`ToolInputError` (leniency lives in tests). The wild-data boundaries
+    — the hook runtime and the activity lift — pass ``on_error='other'`` so a
+    Claude Code shape change or a model-emitted invalid call degrades to
+    :class:`OtherCall` — with a still-correct digest, since the raw mapping is
+    the substrate — instead of crashing every hook fire or session lift.
 
     Example:
         >>> call = parse_tool_call("Edit", {"file_path": "a.py", "old_string": "x", "new_string": "y"})

{cc_transcript-2.0.0 → cc_transcript-3.0.1}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "cc-transcript"
-version = "2.0.0"
-description = "The Claude Code session-activity platform: a superset JSONL parser (Rust fast path, Python reference), typed tool calls with a cross-language content digest, turns/edits/evidence over one spine, context windows with labeled fidelity, a unified decision ledger, LLM judging, and a transcript-investigation CLI."
+version = "3.0.1"
+description = "The Claude Code session-activity platform: a superset JSONL parser (Rust fast path, Python reference), typed tool calls with a cross-language content digest, turns/edits/evidence over one spine, context windows with labeled fidelity, unified decision and correction ledgers, LLM judging, and a transcript-investigation CLI."
 readme = "README.md"
 license = "PolyForm-Noncommercial-1.0.0"
 license-files = ["LICENSE"]
@@ -46,8 +46,6 @@ sentiment = [
     "spacy>=3.8",
     "afinn>=0.1",
 ]
-# Back-compat alias for the pre-0.6 extra name; prefer [sentiment].
-lexicon = ["cc-transcript[sentiment]"]
 llm = ["spawnllm>=0.1.3"]
 
 [project.scripts]

cc_transcript-2.0.0/cc_transcript/filters.py

@@ -1,109 +0,0 @@
-from __future__ import annotations
-
-import re
-from dataclasses import dataclass, field
-from typing import TYPE_CHECKING
-
-from cc_transcript.filterspec import (
-    ASSISTANTS,
-    SENTIMENT_JUNK_GROUPS,
-    USERS,
-    Clause,
-    EntrypointIn,
-    FilterSpec,
-    KindIs,
-    MetaFlag,
-    ModelIs,
-    TextEmpty,
-    TextMatchesAny,
-    apply_spec,
-    compile_groups,
-)
-from cc_transcript.models import AssistantEvent, ModeEvent, OtherEvent, SystemEvent, UserEvent
-
-if TYPE_CHECKING:
-    from collections.abc import Iterable, Iterator
-
-    from cc_transcript.filterspec import EventKind
-    from cc_transcript.models import TranscriptEvent
-
-JUNK_USER_MESSAGE_RE = compile_groups(SENTIMENT_JUNK_GROUPS, True)
-
-KIND_BY_TYPE: dict[type[TranscriptEvent], EventKind] = {
-    UserEvent: "user",
-    AssistantEvent: "assistant",
-    SystemEvent: "system",
-    ModeEvent: "mode",
-    OtherEvent: "other",
-}
-
-
-@dataclass(frozen=True, slots=True)
-class FilterConfig:
-    """Opt-in, consumer-side filtering of a transcript event stream.
-
-    A back-compatible flag-bag that lowers to a :class:`~cc_transcript.FilterSpec`
-    via :meth:`to_spec`. Every flag defaults off, so a bare ``FilterConfig()``
-    passes events through untouched.
-
-    Attributes:
-        keep_types: When set, drop every event not an instance of one of these
-            types; a type-level allowlist applied before the per-event rules.
-        drop_sidechain: Drop events whose envelope marks a sidechain.
-        drop_synthetic: Drop assistant events with model ``<synthetic>``.
-        drop_compacted: Drop compaction-summary and transcript-only entries
-            (envelope flags on :class:`~cc_transcript.models.EntryMeta`).
-        drop_empty: Drop user events with no text and assistant events with
-            neither text nor a tool use.
-        drop_ephemeral_entrypoints: Drop events from these entrypoints.
-        junk_pattern: Drop user events whose text matches this pattern.
-    """
-
-    keep_types: tuple[type[TranscriptEvent], ...] | None = None
-    drop_sidechain: bool = False
-    drop_synthetic: bool = False
-    drop_compacted: bool = False
-    drop_empty: bool = False
-    drop_ephemeral_entrypoints: frozenset[str] = frozenset()
-    junk_pattern: re.Pattern[str] | None = field(default=None)
-
-    def to_spec(self) -> FilterSpec:
-        """Lowers this flag-bag into an equivalent ordered :class:`FilterSpec`."""
-        return FilterSpec(clauses=tuple(self.clauses()))
-
-    def clauses(self) -> Iterator[Clause]:
-        if self.keep_types is not None:
-            yield Clause(KindIs(frozenset(KIND_BY_TYPE[kind] for kind in self.keep_types)), negate=True)
-        if self.drop_synthetic:
-            yield Clause(ModelIs(frozenset({"<synthetic>"})), applies_to=ASSISTANTS)
-        if self.drop_empty:
-            yield Clause(TextEmpty(consider_tool_use=True), applies_to=ASSISTANTS)
-            yield Clause(TextEmpty(consider_tool_use=False), applies_to=USERS)
-        if self.junk_pattern is not None:
-            yield Clause(
-                TextMatchesAny(
-                    (("junk", self.junk_pattern.pattern),),
-                    ignore_case=bool(self.junk_pattern.flags & re.IGNORECASE),
-                ),
-                applies_to=USERS,
-            )
-        if self.drop_sidechain:
-            yield Clause(MetaFlag("is_sidechain"))
-        if self.drop_compacted:
-            yield Clause(MetaFlag("is_compact_summary"))
-            yield Clause(MetaFlag("is_visible_in_transcript_only"))
-        if self.drop_ephemeral_entrypoints:
-            yield Clause(EntrypointIn(self.drop_ephemeral_entrypoints))
-
-
-def apply_filters(events: Iterable[TranscriptEvent], config: FilterConfig) -> Iterator[TranscriptEvent]:
-    """Yields the events that survive ``config``.
-
-    Args:
-        events: The events to filter.
-        config: The filtering rules to apply.
-
-    Yields:
-        The events for which every enabled rule holds.
-    """
-    return apply_spec(events, config.to_spec())