cjm-transcript-decomp-core 0.0.2__tar.gz → 0.0.5__tar.gz

{cjm_transcript_decomp_core-0.0.2/cjm_transcript_decomp_core.egg-info → cjm_transcript_decomp_core-0.0.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cjm-transcript-decomp-core
-Version: 0.0.2
+Version: 0.0.5
 Summary: A frontend-agnostic core for the transcript decomposition workflow — composes isolated capability workers (forced alignment, VAD, graph storage) into a headless pipeline that decomposes transcription run manifests into a VAD-aligned context-graph spine with traceable provenance, and a CLI as its first driver.
 Author-email: "Christian J. Mills" <9126128+cj-mills@users.noreply.github.com>
 License: Apache-2.0
@@ -10,10 +10,10 @@ Classifier: Programming Language :: Python :: 3 :: Only
 Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: cjm_substrate>=0.0.51
-Requires-Dist: cjm_context_graph_primitives>=0.0.12
-Requires-Dist: cjm_capability_primitives>=0.0.10
-Requires-Dist: cjm-context-graph-layer>=0.0.10
+Requires-Dist: cjm_substrate>=0.0.57
+Requires-Dist: cjm_context_graph_primitives>=0.0.15
+Requires-Dist: cjm_capability_primitives>=0.0.11
+Requires-Dist: cjm-context-graph-layer>=0.0.11
 Requires-Dist: cjm-transcript-graph-schema>=0.0.7
 Dynamic: license-file
 
@@ -38,6 +38,8 @@ A frontend-agnostic core for the transcript decomposition workflow — composes
 - `assign_words_to_chunks` _function_ — Assign each FA word to a VAD chunk by timestamp overlap.
 - `build_segments_from_alignment` _function_ — Build a TextSegment per VAD chunk by grouping words by chunk assignment.
 - `map_fa_words_to_text` _function_ — Map forced-alignment words back to character spans in the original text.
+- `sentence_end_word_indices` _function_ — Map capability-delivered sentence boundaries onto FA words (B.5: the
+- `split_chunks_at_sentence_gaps` _function_ — The sentence-split stage (SENTENCE_SPLIT_POLICY, DEC f1024568): refine the
 - `tier1_alignment_checks` _function_ — Tier-1 deterministic pre-filters for the alignment-review seam (no AI).
 
 ### `cjm_transcript_decomp_core.cli`
@@ -45,7 +47,7 @@ A frontend-agnostic core for the transcript decomposition workflow — composes
 - `build_parser` _function_ — Build the CLI parser (subcommands: run).
 - `load_capabilities` _function_ — Discover manifests + load each requested capability (default instance).
 - `main` _function_ — CLI entry point (console script: `cjm-transcript-decomp-core`).
-- `run_command` _function_ — Execute the `run` subcommand: extend a transcription-emitted root with the fine spine.
+- `run_command` _function_ — Execute the `run` subcommand: extend transcription-run manifest(s) with the fine spine.
 
 ### `cjm_transcript_decomp_core.graph`
 
@@ -68,16 +70,20 @@ A frontend-agnostic core for the transcript decomposition workflow — composes
 
 ### `cjm_transcript_decomp_core.pipeline`
 
-- `build_alignment_composition` _function_ — Build the whole-source M×(VAD ∥ T×FA) composition (D8 fan-in, stage-5 variants).
+- `build_alignment_composition` _function_ — Build the whole-source M×(VAD ∥ T×FA ∥ SEG) composition (D8 fan-in, stage-5 variants).
 - `collect_capability_info` _function_ — Record capability identity + data-DB pointers for the run manifest (provenance).
+- `compute_skeleton_hash` _function_ — Skeleton identity, pure (DEC f1024568 + 9241564f).
 - `confirm_seam` _function_ — HITL approval seam in its cheapest viable form (log + optional CLI prompt).
+- `decomp_replay_handlers` _function_ — The decomp core's replay vocabulary (DEC 426658f1, replay stays DOMAIN-OWNED).
 - `decompose_source` _function_ — Decompose one source into aligned fine segments with per-transcriber variants.
 - `fa_words_from_result` _function_ — Normalize a typed forced-alignment result into FA words (pure; stage 3).
 - `load_source_manifest` _function_ — Load + lightly validate a transcription-core run manifest.
 - `run_decomp` _function_ — Extend every source in a transcription run manifest with its fine spine.
+- `sentence_spans_from_result` _function_ — Normalize a typed sentence-segmentation result into char-span tuples (pure; B.5).
 - `submit_and_wait` _function_ — Submit one capability job, wait for it, and return its result (raise on failure).
 - `vad_chunks_from_result` _function_ — Normalize a typed VAD result into segment-local VAD chunks.
 
 ## Dependencies
 
 **Depends on:** `cjm-capability-primitives`, `cjm-context-graph-layer`, `cjm-context-graph-primitives`, `cjm-substrate`, `cjm-transcript-graph-schema`
+**Used by:** `cjm-transcript-decomp-tui`

{cjm_transcript_decomp_core-0.0.2 → cjm_transcript_decomp_core-0.0.5}/README.md

@@ -19,6 +19,8 @@ A frontend-agnostic core for the transcript decomposition workflow — composes
 - `assign_words_to_chunks` _function_ — Assign each FA word to a VAD chunk by timestamp overlap.
 - `build_segments_from_alignment` _function_ — Build a TextSegment per VAD chunk by grouping words by chunk assignment.
 - `map_fa_words_to_text` _function_ — Map forced-alignment words back to character spans in the original text.
+- `sentence_end_word_indices` _function_ — Map capability-delivered sentence boundaries onto FA words (B.5: the
+- `split_chunks_at_sentence_gaps` _function_ — The sentence-split stage (SENTENCE_SPLIT_POLICY, DEC f1024568): refine the
 - `tier1_alignment_checks` _function_ — Tier-1 deterministic pre-filters for the alignment-review seam (no AI).
 
 ### `cjm_transcript_decomp_core.cli`
@@ -26,7 +28,7 @@ A frontend-agnostic core for the transcript decomposition workflow — composes
 - `build_parser` _function_ — Build the CLI parser (subcommands: run).
 - `load_capabilities` _function_ — Discover manifests + load each requested capability (default instance).
 - `main` _function_ — CLI entry point (console script: `cjm-transcript-decomp-core`).
-- `run_command` _function_ — Execute the `run` subcommand: extend a transcription-emitted root with the fine spine.
+- `run_command` _function_ — Execute the `run` subcommand: extend transcription-run manifest(s) with the fine spine.
 
 ### `cjm_transcript_decomp_core.graph`
 
@@ -49,16 +51,20 @@ A frontend-agnostic core for the transcript decomposition workflow — composes
 
 ### `cjm_transcript_decomp_core.pipeline`
 
-- `build_alignment_composition` _function_ — Build the whole-source M×(VAD ∥ T×FA) composition (D8 fan-in, stage-5 variants).
+- `build_alignment_composition` _function_ — Build the whole-source M×(VAD ∥ T×FA ∥ SEG) composition (D8 fan-in, stage-5 variants).
 - `collect_capability_info` _function_ — Record capability identity + data-DB pointers for the run manifest (provenance).
+- `compute_skeleton_hash` _function_ — Skeleton identity, pure (DEC f1024568 + 9241564f).
 - `confirm_seam` _function_ — HITL approval seam in its cheapest viable form (log + optional CLI prompt).
+- `decomp_replay_handlers` _function_ — The decomp core's replay vocabulary (DEC 426658f1, replay stays DOMAIN-OWNED).
 - `decompose_source` _function_ — Decompose one source into aligned fine segments with per-transcriber variants.
 - `fa_words_from_result` _function_ — Normalize a typed forced-alignment result into FA words (pure; stage 3).
 - `load_source_manifest` _function_ — Load + lightly validate a transcription-core run manifest.
 - `run_decomp` _function_ — Extend every source in a transcription run manifest with its fine spine.
+- `sentence_spans_from_result` _function_ — Normalize a typed sentence-segmentation result into char-span tuples (pure; B.5).
 - `submit_and_wait` _function_ — Submit one capability job, wait for it, and return its result (raise on failure).
 - `vad_chunks_from_result` _function_ — Normalize a typed VAD result into segment-local VAD chunks.
 
 ## Dependencies
 
 **Depends on:** `cjm-capability-primitives`, `cjm-context-graph-layer`, `cjm-context-graph-primitives`, `cjm-substrate`, `cjm-transcript-graph-schema`
+**Used by:** `cjm-transcript-decomp-tui`

cjm_transcript_decomp_core-0.0.5/cjm_transcript_decomp_core/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.0.5"

{cjm_transcript_decomp_core-0.0.2 → cjm_transcript_decomp_core-0.0.5}/cjm_transcript_decomp_core/alignment.py

@@ -1,13 +1,23 @@
 """Pure forced-alignment logic (no capability calls): map FA words back to character spans in the original text, assign words to VAD chunks by timestamp, and build one text segment per VAD chunk. Extracted from the page-centric ForcedAlignmentService (Tier-1 logic)."""
 
 import re
-from typing import Dict, List, Optional, Tuple
+from typing import Dict, List, Optional, Set, Tuple
 
 from cjm_transcript_decomp_core.models import FAWord, TextSegment, VADChunk
 
 # Strip punctuation for comparison (matches what FA models strip).
 _PUNCT_RE = re.compile(r"[^\w\s]", re.UNICODE)
 
+# Sentence-split policy tag (DEC f1024568): versioned because the policy is a
+# SKELETON IDENTITY input — any change to the split rule must bump it, or
+# re-runs would silently collide with spines a different rule produced.
+# 'capability' (B.5, DEC cc904eee): sentence boundaries now come from a
+# sentence-segmentation CAPABILITY (pySBD first) as char spans over the
+# authoritative text — the v1-v3 token heuristic (closer/abbreviation/stub
+# lists) is RETIRED. The segmenter's identity (capability name + config hash)
+# joins the skeleton-identity composite beside this tag.
+SENTENCE_SPLIT_POLICY = "sentence-split/capability"
+
 
 def _strip_punct(
     text: str,  # Text to normalize
@@ -89,8 +99,13 @@ def assign_words_to_chunks(
 ) -> List[int]:  # Chunk index for each FA word
     """Assign each FA word to a VAD chunk by timestamp overlap.
 
-    Words whose start_time falls within a chunk's [start, end] are assigned to
-    that chunk; words in silence gaps go to the nearest chunk by time proximity.
+    Words whose start_time falls within a chunk's [start, end) are assigned to
+    that chunk — HALF-OPEN, so a word starting exactly on a shared boundary
+    belongs to the chunk that STARTS there (sentence-split cuts land exactly on
+    the next word's start when FA words are contiguous; the old inclusive end
+    pulled that word one chunk LEFT — the off-by-one the 2026-07-22 probe drive
+    caught). Words in silence gaps (incl. exactly at the last chunk's end) go
+    to the nearest chunk by time proximity.
     """
     if not vad_chunks:
         return [0] * len(fa_items)
@@ -101,7 +116,7 @@ def assign_words_to_chunks(
         best_idx = 0
         best_dist = float("inf")
         for i, chunk in enumerate(vad_chunks):
-            if chunk.start_time <= t <= chunk.end_time:
+            if chunk.start_time <= t < chunk.end_time:
                 best_idx = i
                 break
             dist = min(abs(t - chunk.start_time), abs(t - chunk.end_time))
@@ -164,3 +179,79 @@ def tier1_alignment_checks(
             f"{empty}/{len(segments)} segment(s) have EMPTY text (VAD chunk with no aligned words)"
         )
     return warnings
+
+
+def sentence_end_word_indices(
+    spans: List[Tuple[int, int]],           # FA-word char spans from map_fa_words_to_text (ordered)
+    sentence_spans: List[Tuple[int, int]],  # Sentence char spans over the SAME text (ordered, non-overlapping)
+) -> Set[int]:  # Indices of FA words that END a sentence
+    """Map capability-delivered sentence boundaries onto FA words (B.5: the
+    successor of the v1-v3 `_ends_sentence` token heuristic).
+
+    A word ends a sentence when it is the LAST word whose span STARTS before
+    that sentence's end_char. Both lists are ordered over the same text, so a
+    single forward merge walk suffices; a sentence no word starts in
+    contributes nothing, and the trailing word of the text marks its sentence
+    but can never produce a cut (cuts live BETWEEN consecutive words)."""
+    out: Set[int] = set()
+    wi = 0
+    for ss, se in sentence_spans:
+        last = None
+        while wi < len(spans) and spans[wi][0] < se:
+            if spans[wi][0] >= ss:  # the word must START inside the sentence
+                last = wi
+            wi += 1
+        if last is not None:
+            out.add(last)
+    return out
+
+
+def split_chunks_at_sentence_gaps(
+    vad_chunks: List[VADChunk],  # The VAD skeleton (segment-local times)
+    fa_items: List[FAWord],      # The AUTHORITATIVE transcriber's FA words (segment-local times)
+    end_words: Set[int],         # FA-word indices that END a sentence (sentence_end_word_indices)
+    min_chunk_s: float = 0.5,    # Min sub-chunk duration — a split never mints a sliver
+) -> List[VADChunk]:  # The refined skeleton, re-indexed (identical content when nothing splits)
+    """The sentence-split stage (SENTENCE_SPLIT_POLICY, DEC f1024568): refine the
+    VAD skeleton by cutting any chunk whose assigned text crosses a sentence end.
+
+    Runs POST-FA, PRE-fold: a chunk holding a sentence-ending word that is not
+    its last word splits at the corresponding FA word gap (midpoint between the
+    ending word's end and the next word's start — the pause the VAD's min-sil
+    threshold failed to cut, finding bc69e3e6). Sentence ends arrive
+    PRECOMPUTED (B.5): the segmentation capability's char spans over the
+    authoritative text, mapped onto FA words by `sentence_end_word_indices` —
+    this function no longer inspects text. The split decision reads ONLY the
+    authoritative transcriber (montage/textless chunks have no words here and
+    pass through untouched); every transcriber then re-folds over the refined
+    skeleton, so variants stay per-chunk consistent by construction. Both
+    sides of an accepted cut must be >= `min_chunk_s` at accept time (greedy
+    left-to-right), so FA jitter cannot mint unplayable slivers.
+    """
+    assignments = assign_words_to_chunks(fa_items, vad_chunks)
+    by_chunk: Dict[int, List[int]] = {}
+    for wi, ci in enumerate(assignments):
+        by_chunk.setdefault(ci, []).append(wi)
+
+    refined: List[VADChunk] = []
+    for chunk in vad_chunks:
+        words = by_chunk.get(chunk.index, [])
+        cuts: List[float] = []
+        cur_start = chunk.start_time
+        for p in range(len(words) - 1):
+            wi, wj = words[p], words[p + 1]
+            if wi not in end_words:
+                continue
+            cut = (fa_items[wi].end_time + fa_items[wj].start_time) / 2.0
+            if not (chunk.start_time < cut < chunk.end_time):
+                continue
+            if cut - cur_start < min_chunk_s or chunk.end_time - cut < min_chunk_s:
+                continue
+            cuts.append(cut)
+            cur_start = cut
+        bounds = [chunk.start_time] + cuts + [chunk.end_time]
+        for k in range(len(bounds) - 1):
+            refined.append(VADChunk(index=0, start_time=bounds[k], end_time=bounds[k + 1]))
+    for i, c in enumerate(refined):
+        c.index = i
+    return refined

{cjm_transcript_decomp_core-0.0.2 → cjm_transcript_decomp_core-0.0.5}/cjm_transcript_decomp_core/cli.py

@@ -10,9 +10,11 @@ Prerequisite runtime (once, from the repo root):
     cjm-ctl --cjm-config cjm.yaml setup-runtime
     cjm-ctl --cjm-config cjm.yaml install-all --capabilities capabilities_test.yaml --force
 
-Then decompose a transcription-core run manifest:
+Then decompose transcription-core run manifest(s) — a multi-manifest batch
+shares ONE loaded capability stack (models load once, per-manifest runs):
 
     cjm-transcript-decomp-core run path/to/transcription-run.json --yes
+    cjm-transcript-decomp-core run runs/run_a.json runs/run_b.json --yes
     # GPU runs: opt into CR-7 GPU subtree attribution
     cjm-transcript-decomp-core run run.json --yes --sysmon-capability cjm-capability-monitor-nvidia
 """
@@ -21,11 +23,13 @@ import argparse
 import asyncio
 import getpass
 import logging
+import os
 from pathlib import Path
 from typing import Any, Dict, List, Optional
 
 from cjm_substrate.core.manager import CapabilityManager
 from cjm_substrate.core.queue import JobQueue
+from cjm_substrate.core.workspace import resolve_workspace
 from cjm_transcript_decomp_core.models import DecompConfig
 from cjm_transcript_decomp_core.pipeline import load_source_manifest, run_decomp
 
@@ -41,7 +45,8 @@ def build_parser() -> argparse.ArgumentParser:  # Configured CLI parser
     sub = parser.add_subparsers(dest="command", required=True)
 
     run = sub.add_parser("run", help="Extend a transcription-emitted graph root with the fine spine")
-    run.add_argument("manifest", help="Transcription-core run manifest JSON (the source to decompose)")
+    run.add_argument("manifests", nargs="+",
+                     help="Transcription-core run manifest JSON(s) — a batch shares ONE loaded capability stack")
     run.add_argument("--manifests-dir", default=".cjm/manifests", help="Capability manifests directory")
     run.add_argument("--vad-capability", default="cjm-capability-silero-vad", help="VAD capability name")
     run.add_argument("--fa-capability", default="cjm-capability-qwen3-forced-aligner", help="Forced-alignment capability name")
@@ -54,9 +59,32 @@ def build_parser() -> argparse.ArgumentParser:  # Configured CLI parser
                           "default: the manifest's sole transcriber)")
     run.add_argument("--sysmon-capability", default=None, help="monitor capability for GPU subtree attribution (CR-7); loaded first; default: no monitor")
     run.add_argument("--language", default="English", help="Forced-alignment language")
+    run.add_argument("--sentence-split", action=argparse.BooleanOptionalAction, default=True,
+                     help="Run the post-FA sentence-split stage (DEC f1024568): chunks whose "
+                          "text crosses a sentence end split at the FA word gap; commits a "
+                          "PARALLEL spine (new skeleton hash) — existing spines are untouched. "
+                          "DEFAULT-ON (DEC 552bde8d); --no-sentence-split opts out")
+    run.add_argument("--seg-capability", default="cjm-capability-pysbd",
+                     help="Sentence-segmentation capability name (B.5; skipped with --no-sentence-split)")
+    run.add_argument("--split-min-chunk-s", type=float, default=0.5,
+                     help="Sentence-split min sub-chunk duration guard, seconds (identity input)")
+    run.add_argument("--respine", action="store_true",
+                     help="Mint a FRESH skeleton spine under the SAME config (DEC 9241564f): "
+                          "the run id joins the Segment identity input, so the new spine "
+                          "coexists with — never collides into — the config-identical one. "
+                          "The recovery for post-upgrade re-runs the verify-collide refuses "
+                          "(finding e8458f6e); provenance = this run's manifest")
     run.add_argument("--force", action="store_true", help="Bypass capability-side caches (VAD + FA)")
     run.add_argument("-y", "--yes", action="store_true", help="Auto-accept HITL seams (headless mode)")
-    run.add_argument("--output", default=None, help="Decomp-manifest output path (default: runs/<run_id>.json)")
+    run.add_argument("--output", default=None, help="Decomp-manifest output path (single-manifest runs only; default: <output-dir>/<run_id>.json)")
+    run.add_argument("--output-dir", default=None,
+                     help="Decomp-manifest output directory (callers with a shared runs dir "
+                          "pass it so manifests land beside their sources; default: the "
+                          "workspace's runs/ when one is active, else runs/ under the cwd)")
+    run.add_argument("--workspace", default=None,
+                     help="Workspace root (5daadfc4; default: CJM_WORKSPACE env, else upward walk "
+                          "from cwd). Supplies the --output-dir default and is exported so "
+                          "substrate config + capability workers resolve workspace-scoped paths")
     run.add_argument("--actor", default=None,
                      help="Journal attribution for who/what initiated this run (default: cli:<username>)")
     run.add_argument("-v", "--verbose", action="store_true", help="DEBUG-level logging")
@@ -85,11 +113,31 @@ def load_capabilities(
 
 async def run_command(
     args: argparse.Namespace,  # Parsed CLI arguments for the `run` subcommand
-) -> int:  # Process exit code (0 = all sources extended + verified)
-    """Execute the `run` subcommand: extend a transcription-emitted root with the fine spine."""
-    manifest_path = str(Path(args.manifest).resolve())
-    if not Path(manifest_path).exists():
-        raise SystemExit(f"source manifest not found: {manifest_path}")
+) -> int:  # Process exit code (0 = every manifest fully extended + verified)
+    """Execute the `run` subcommand: extend transcription-run manifest(s) with the fine spine.
+
+    Batch shape (decomp-TUI demand, work item 0ff6bf0f): N manifests ride ONE
+    capability stack — the models load once and every manifest reuses them,
+    where N CLI invocations would pay N model loads. Each manifest keeps its
+    own run_id, decomp manifest, and RUN_STARTED/RUN_FINISHED journal bracket,
+    so a batch member is indistinguishable from a solo run downstream. A failed
+    member records and the batch continues (unattended queueing must not sink
+    the rest); the exit code aggregates across members.
+    """
+    # 5daadfc4 workspace: resolve BEFORE any substrate config loads; export so
+    # the process tree (substrate config, capability workers) is workspace-scoped.
+    ws = resolve_workspace(explicit=getattr(args, "workspace", None))
+    if ws is not None:
+        os.environ["CJM_WORKSPACE"] = str(ws.root)
+    output_dir = (Path(args.output_dir) if args.output_dir
+                  else (ws.runs_dir if ws is not None else Path("runs")))
+    manifest_paths = [str(Path(m).resolve()) for m in args.manifests]
+    missing = [m for m in manifest_paths if not Path(m).exists()]
+    if missing:
+        raise SystemExit(f"source manifest(s) not found: {', '.join(missing)}")
+    if args.output and len(manifest_paths) > 1:
+        raise SystemExit("--output names ONE decomp manifest — omit it for batch runs "
+                         "(each lands at runs/<run_id>.json)")
 
     cfg = DecompConfig(
         vad_capability=args.vad_capability,
@@ -99,6 +147,10 @@ async def run_command(
         language=args.language,
         force=args.force,
         assume_yes=args.yes,
+        sentence_split=args.sentence_split,
+        split_min_chunk_s=args.split_min_chunk_s,
+        seg_capability=args.seg_capability,
+        respine=args.respine,
     )
 
     # CR-7 GPU subtree attribution is opt-in: --sysmon-capability threads the monitor
@@ -109,20 +161,50 @@ async def run_command(
         sysmon_capability_name=args.sysmon_capability,
     )
     instance_ids = [cfg.vad_capability, cfg.fa_capability, cfg.graph_capability]
+    if cfg.sentence_split:
+        # B.5: the segmentation capability loads only when the split stage runs.
+        instance_ids.insert(2, cfg.seg_capability)
     load_order = ([args.sysmon_capability] if args.sysmon_capability else []) + instance_ids
     # F10: --graph-db-path threads a caller-wins config into the graph load
     # (the C8 pattern correction-core already used; scratch-graph loop-backs).
     configs = ({cfg.graph_capability: {"db_path": args.graph_db_path}}
                if args.graph_db_path else None)
-    load_capabilities(manager, load_order, configs=configs)
+    try:
+        load_capabilities(manager, load_order, configs=configs)
+    except Exception as e:
+        # Sentence-split is DEFAULT-ON (DEC 552bde8d): an env without the
+        # segmentation capability must refuse with both outs, not crash.
+        if cfg.sentence_split and cfg.seg_capability in str(e):
+            raise SystemExit(
+                f"sentence-split is default-on and needs {cfg.seg_capability}: {e}\n"
+                f"either install it (cjm-ctl install-all with the capabilities yaml "
+                f"adds the env + manifest additively) or pass --no-sentence-split")
+        raise
 
     queue = JobQueue(deps=manager, sysmon_capability_name=args.sysmon_capability)
     await queue.start()
+    all_ok = True
     try:
         # CR-14 follow-up: actor attribution (operator identity by default;
         # agents/services pass --actor explicitly).
         actor = args.actor or f"cli:{getpass.getuser()}"
-        manifest = await run_decomp(manager, queue, cfg, manifest_path, actor=actor)
+        for mp in manifest_paths:
+            try:
+                manifest = await run_decomp(manager, queue, cfg, mp, actor=actor)
+            except Exception as e:  # One member's failure must not sink the batch
+                logger.error(f"decomp failed for {mp}: {e}")
+                print(f"FAILED {mp}: {e}")
+                all_ok = False
+                continue
+            out = (Path(args.output) if args.output
+                   else output_dir / f"{manifest.run_id}.json")
+            manifest.save(out, workspace=ws)
+            n_manifest_sources = len(load_source_manifest(mp).get("sources", []) or [])
+            n_sources = len(manifest.sources)
+            n_segs = sum(s.segment_count for s in manifest.sources)
+            print(f"decomp manifest: {out}")
+            print(f"sources extended: {n_sources}/{n_manifest_sources}  segment nodes: {n_segs}")
+            all_ok = all_ok and (n_sources == n_manifest_sources)
     finally:
         await queue.stop()
         for iid in reversed(load_order):  # Reverse load order; the monitor unloads last
@@ -130,15 +212,7 @@ async def run_command(
                 manager.unload_capability(iid)
             except Exception as e:  # Best-effort teardown; never mask the run's outcome
                 logger.warning(f"unload {iid} failed: {e}")
-
-    out = Path(args.output) if args.output else Path("runs") / f"{manifest.run_id}.json"
-    manifest.save(out)
-    n_manifest_sources = len(load_source_manifest(manifest_path).get("sources", []) or [])
-    n_sources = len(manifest.sources)
-    n_segs = sum(s.segment_count for s in manifest.sources)
-    print(f"decomp manifest: {out}")
-    print(f"sources extended: {n_sources}/{n_manifest_sources}  segment nodes: {n_segs}")
-    return 0 if n_sources == n_manifest_sources else 1
+    return 0 if all_ok else 1
 
 
 def main(

{cjm_transcript_decomp_core-0.0.2 → cjm_transcript_decomp_core-0.0.5}/cjm_transcript_decomp_core/graph.py

@@ -66,9 +66,10 @@ def resolve_root_ids(
 def build_extension_payload(
     source_entry: Dict[str, Any],             # One source entry from the transcription manifest
     capabilities_info: Dict[str, Dict[str, Any]],  # The transcription manifest's capabilities block
-    vad_config_hash: str,                     # THIS run's VAD capability config hash (skeleton identity input)
+    skeleton_config_hash: str,                # THIS run's skeleton-config hash (Segment identity input; = the VAD config hash when no split stage ran)
     text_from: str,                           # Authoritative transcriber (layer-0 text designation)
     segments: List[DecompSegment],            # Ordered aligned segments (per-transcriber variants attached)
+    split_policy: Optional[str] = None,       # Split policy+version that refined the skeleton (node metadata, never identity)
 ) -> Tuple[List[Dict[str, Any]], List[Dict[str, Any]], Dict[str, Any]]:  # (nodes, edges, ids)
     """Build the fine-spine EXTENSION payload (pure; no capability calls).
 
@@ -100,12 +101,12 @@ def build_extension_payload(
                                              end_char=var.end_char, text=var.text))
             used_transcripts.add(tid)
         node = SegmentNode(
-            rendition=a["rendition"], vad_config_hash=vad_config_hash,
+            rendition=a["rendition"], vad_config_hash=skeleton_config_hash,
             chunk_start=seg.chunk_start, chunk_end=seg.chunk_end,
             index=seg.index, start_time=seg.start_time, end_time=seg.end_time,
             text=seg.text, audio_hash=a["model_input_hash"], source=source_id,
             text_from=(a["transcripts"].get(text_from) if seg.text.strip() else None),
-            text_slices=slices,
+            split_policy=split_policy, text_slices=slices,
         )
         nodes.append(node.to_graph_node())
         seg_ids.append(node.id)
@@ -153,6 +154,7 @@ async def verify_source(
     graph_id: str,            # Graph-storage capability id
     source_id: str,           # Source node id to verify
     rendition_ids: List[str], # The AudioRendition ids the run extended (the fine spine is scoped to these)
+    segment_ids: Optional[List[str]] = None,  # THIS run's committed Segment ids — id-scopes the fine-layer checks (parallel spines); None = all segments under the renditions
 ) -> Optional[SourceVerification]:  # Result, or None if the Source is not found
     """Verify a Source's committed extension via server-side AGGREGATES (D13/D19).
 
@@ -204,21 +206,49 @@ async def verify_source(
         res = await graph_task(queue, graph_id, "query_nodes", query=q.to_dict())
         return int(res.count or 0)
 
-    n_segs = await _ncount()
-    seg_next = await _ecount(relation_type=SpineRelations.NEXT, source_related=part_of_rend)
-    seg_part_of = await _ecount(relation_type=SpineRelations.PART_OF, target_ids=rendition_ids)
-    rend_starts = await _ecount(relation_type=SpineRelations.STARTS_WITH, source_ids=rendition_ids)
-    missing_start = await _ncount(where=[PropertyPredicate("start_time", "is_null")])
-    missing_end = await _ncount(where=[PropertyPredicate("end_time", "is_null")])
-
-    # Bounded projection: sources + owning rendition in ONE pass.
-    sq = NodeQuery(label=TranscriptGraphLabels.SEGMENT, related=part_of_rend,
-                   project=["sources", "rendition_id"])
-    res = await graph_task(queue, graph_id, "query_nodes", query=sq.to_dict())
+    proj_rows: List[Dict[str, Any]] = []
+    if segment_ids is None:
+        n_segs = await _ncount()
+        seg_next = await _ecount(relation_type=SpineRelations.NEXT, source_related=part_of_rend)
+        seg_part_of = await _ecount(relation_type=SpineRelations.PART_OF, target_ids=rendition_ids)
+        rend_starts = await _ecount(relation_type=SpineRelations.STARTS_WITH, source_ids=rendition_ids)
+        missing_start = await _ncount(where=[PropertyPredicate("start_time", "is_null")])
+        missing_end = await _ncount(where=[PropertyPredicate("end_time", "is_null")])
+        # Bounded projection: sources + owning rendition in ONE pass.
+        sq = NodeQuery(label=TranscriptGraphLabels.SEGMENT, related=part_of_rend,
+                       project=["sources", "rendition_id"])
+        res = await graph_task(queue, graph_id, "query_nodes", query=sq.to_dict())
+        proj_rows = list(res.rows or [])
+    else:
+        # Id-scoped mode (parallel spines, DEC f1024568): every fine-layer
+        # aggregate counts ONLY this run's committed segments — a coexisting
+        # sibling skeleton under the same renditions (its own NEXT chain +
+        # STARTS_WITH anchor) must not fail the structural checks. Batched
+        # (bounded reads; the 500-id convention).
+        async def _icount(ids, **kw):
+            q = NodeQuery(ids=list(ids), label=TranscriptGraphLabels.SEGMENT,
+                          count=True, **kw)
+            res = await graph_task(queue, graph_id, "query_nodes", query=q.to_dict())
+            return int(res.count or 0)
+
+        n_segs = seg_next = seg_part_of = rend_starts = missing_start = missing_end = 0
+        for i in range(0, len(segment_ids), 500):
+            b = list(segment_ids[i:i + 500])
+            n_segs += await _icount(b)
+            seg_next += await _ecount(relation_type=SpineRelations.NEXT, source_ids=b)
+            seg_part_of += await _ecount(relation_type=SpineRelations.PART_OF, source_ids=b)
+            rend_starts += await _ecount(relation_type=SpineRelations.STARTS_WITH, target_ids=b)
+            missing_start += await _icount(b, where=[PropertyPredicate("start_time", "is_null")])
+            missing_end += await _icount(b, where=[PropertyPredicate("end_time", "is_null")])
+            pq = NodeQuery(ids=b, label=TranscriptGraphLabels.SEGMENT,
+                           project=["sources", "rendition_id"])
+            res = await graph_task(queue, graph_id, "query_nodes", query=pq.to_dict())
+            proj_rows.extend(res.rows or [])
+
     missing_sources = 0
     locators = set()
     owning_renditions = set()
-    for r in (res.rows or []):
+    for r in proj_rows:
         sources = r.get("sources") or []
         if not sources:
             missing_sources += 1

{cjm_transcript_decomp_core-0.0.2 → cjm_transcript_decomp_core-0.0.5}/cjm_transcript_decomp_core/models.py

@@ -7,6 +7,8 @@ from dataclasses import asdict, dataclass, field
 from pathlib import Path
 from typing import Any, Dict, List, Optional, Union
 
+from cjm_substrate.core.workspace import relativize_recorded
+
 
 @dataclass
 class FAWord:
@@ -115,6 +117,10 @@ class DecompConfig:
     media_type: str = "audio"  # Source media type
     force: bool = False        # Bypass capability-side caches (VAD + FA)
     assume_yes: bool = False   # Auto-accept HITL seams (headless mode)
+    sentence_split: bool = True     # Run the post-FA sentence-split stage (DEC f1024568; parallel spine) — DEFAULT-ON per DEC 552bde8d (hiding splits just hides model failures)
+    split_min_chunk_s: float = 0.5  # Sentence-split min sub-chunk duration guard (seconds)
+    seg_capability: str = "cjm-capability-pysbd"  # Sentence-segmentation capability id (B.5; loaded only when sentence_split)
+    respine: bool = False           # Mint a FRESH skeleton under the SAME config (DEC 9241564f): the run id joins the identity input, so the new spine never collides with the config-identical one (post-upgrade re-runs, finding e8458f6e)
 
     def to_dict(self) -> Dict[str, Any]:  # Plain-dict snapshot for the manifest
         """Serialize to a plain dict."""
@@ -151,9 +157,11 @@ class DecompManifest:
     source_version: str = ""  # Upstream manifest schema version
     capabilities: Dict[str, Dict[str, Any]] = field(default_factory=dict)  # instance_id -> identity/db_path/config_hash
     sources: List[DecompSourceRecord] = field(default_factory=list)   # Extended sources, input order
+    skeleton_config_hash: str = ""      # Segment identity input (0.2.2): VAD config hash, or the composite when a split stage ran
+    split_policy: Optional[str] = None  # Split policy+version that refined this run's skeleton (None = raw VAD)
 
     FORMAT: str = field(default="cjm-transcript-decomp-core/run-manifest", repr=False)  # Format tag
-    VERSION: str = field(default="0.2.1", repr=False)                                   # Schema version
+    VERSION: str = field(default="0.2.3", repr=False)                                   # Schema version (0.2.3: capability-driven sentence split — seg capability in `capabilities` + segmenter identity in the skeleton composite)
 
     def to_dict(self) -> Dict[str, Any]:  # Plain-dict form for JSON serialization
         """Serialize to a plain dict with nested sources."""
@@ -167,17 +175,24 @@ class DecompManifest:
             "source_format": self.source_format,
             "source_version": self.source_version,
             "capabilities": self.capabilities,
+            "skeleton_config_hash": self.skeleton_config_hash,
+            "split_policy": self.split_policy,
             "sources": [s.to_dict() for s in self.sources],
         }
 
     def save(
         self,
         path: Union[str, Path],  # Destination JSON file (parent dirs created)
+        workspace=None,  # Active Workspace; owned paths record as ${WS}/<rel> (5daadfc4 rung f)
     ) -> Path:  # The written path
-        """Write the manifest as pretty-printed JSON."""
+        """Write the manifest as pretty-printed JSON.
+
+        With `workspace`, recorded paths under its root take the ${WS}/ token
+        form (relativize_recorded), so the manifest relocates with the
+        workspace; readers resolve via resolve_recorded_tree at load."""
         out = Path(path)
         out.parent.mkdir(parents=True, exist_ok=True)
-        out.write_text(json.dumps(self.to_dict(), indent=2))
+        out.write_text(json.dumps(relativize_recorded(self.to_dict(), workspace), indent=2))
         return out