utterplan 0.5.0__tar.gz → 0.5.1__tar.gz

{utterplan-0.5.0 → utterplan-0.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: utterplan
-Version: 0.5.0
+Version: 0.5.1
 Summary: Engine-independent text-to-speech planning compiler and interchange format
 Author-email: Holger Nahrstaedt <nahrstaedt@gmail.com>
 License:                                  Apache License

{utterplan-0.5.0 → utterplan-0.5.1}/docs/architecture.md

@@ -28,6 +28,12 @@ The dependency direction is from SSMD, spokenform, and phrasplit into UtterPlan,
 
 `FlowPlan` is the current v5 executable contract: ordered `FlowUnit` objects each own ordered `FlowSegment` records. Segments include prepared text, effective language, local tokens, semantic pause intents, directives, markers, and optional heading metadata. `DocumentInfo` retains compact portable document information; `LinguisticProvenance` identifies providers without retaining their documents. UtterPlan ends before G2P. Engine/model selection, synthesis, and audio composition happen downstream.
 
+## Compiler-owned segmentation topology
+
+Segmentation has four distinct layers: parser-owned paragraph events define paragraph regions; PhraseSplit supplies advisory sentence-boundary proposals; UtterPlan validates those proposals and owns canonical paragraph, sentence, and private sentence-part spans in prepared spoken-text coordinates; renderer-required language, directive, pause, marker, and media cuts subdivide a part into final segments. Positive-width atomic media spans protect their interiors from automatic sentence/part boundaries; zero-width media events remain renderer operations, not parts. The public v5 `FlowPlan` contains only the resulting compact ordered flow—there is no serialized sentence-parts table.
+
+PhraseSplit failure or an invalid proposal batch falls back to deterministic, conservative terminal scanning. Unsafe automatic cuts inside provider-aware lexical content are merged or dropped before projection; the final flow-projection token-edge check remains defense in depth. Exact surface and non-whitespace coverage checks run before and after applicable repairs. This normalization is independent of `renderability_mode`: strict and repair compilation use the same topology, while renderability mode governs only the existing punctuation-repair policy. See the [coordinate-space contract](coordinate-spaces) for offset ownership.
+
 ## SSMD and preparation
 
 The document parser calls SSMD with `dialect="0.9"` for every SSMD source path, including unversioned fragments selected with `document_format="ssmd"`. UtterPlan does not parse SSMD 0.8 or migrate source. Older documents must first be converted with `ssmd migrate FILE --to 0.9`. The `utterplan migrate` command imports serialized legacy plans and applies schema migrations only.

{utterplan-0.5.0 → utterplan-0.5.1}/docs/changelog.md

@@ -4,6 +4,18 @@
 
 ## [Unreleased]
 
+## [0.5.1] - 2026-10-10
+
+### Changed
+
+- Improved sentence topology with validated fallback recovery and token-edge safeguards
+
+### Fixed
+
+- Fixed fallback token projection and sentence-boundary repair for punctuation-separated text
+
+<!-- generated by releaseledger -->
+
 ## [0.5.0] - 2026-10-09
 
 ### Added

{utterplan-0.5.0 → utterplan-0.5.1}/docs/coordinate-spaces.md

@@ -19,4 +19,8 @@ Trace coordinates use Python string character boundaries (Unicode code points),
 - `structural_to_spoken` maps structural character boundaries to spoken boundaries; `spoken_to_structural` maps spoken boundaries back to structural boundaries.
 - `PreparationTraceUnit.source_start` / `source_end` address structural text. `PreparationChange.source_start` / `source_end` use structural coordinates; `output_start` / `output_end` use prepared spoken coordinates.
 
+## Compiler-only segmentation coordinates
+
+Paragraph, sentence, and sentence-part spans use half-open Python-character offsets into the prepared global `spoken_text`. PhraseSplit offsets are run-local and are translated to global spoken coordinates exactly once before validation. Renderer segments retain the exact `spoken_text[start:end]` slice; language-run edges and other required renderer cuts do not change their owning paragraph, sentence, or part index. These spans are compiler topology, not fields in the v5 flow. Segmentation diagnostics map their spoken ranges back to exact source coordinates through the compiler's source map.
+
 Diagnostic source offsets use the exact original-source coordinate space. Diagnostic `line` and `column` values are 1-based; columns count Python characters from the start of a line. The trace retains all these maps and source evidence. The executable FlowPlan intentionally does not.

{utterplan-0.5.0 → utterplan-0.5.1}/docs/debugging.md

@@ -17,6 +17,8 @@ utterplan inspect-trace chapter.trace.toml --preparation --boundaries
 
 Safe punctuation-only repair is the default during compilation. If a semantic blocker prevents repair, the error includes source location, prepared fragment, spoken context, blocked repair reason, and a concrete next action; UtterPlan never guesses symbol pronunciation. Use `--renderability strict` to see safe repair opportunities without applying them. Batch compilation reports each repair and continues with later sources.
 
+Sentence segmentation is a compiler-integrity step, not a renderability repair. PhraseSplit is advisory: dependency failure or a malformed/partial proposal batch produces a `segmentation.phrasplit_*` diagnostic and triggers conservative fallback. Unsafe automatic token-edge candidates are dropped or merged before projection; exact text and non-whitespace coverage are checked in both strict and repair modes. Inspect compiler diagnostics with `compile --trace FILE` and `inspect-trace FILE --boundaries`; sentence-part topology itself is intentionally not persisted in v5.
+
 Use `inspect --tokens` to inspect local `TokenView` values. Fallback analysis may leave POS, tag, and morphology unavailable. `explain --details` shows token spans relative to the owning segment and its flow hash. Token spans always slice `segment.text`; they are not offsets into the original source or another segment.
 
 If spoken wording is wrong, inspect `FlowSegment.text`. If effective language is wrong, inspect the segment and `FlowPlan.linguistics`. If a pause seems missing, distinguish the semantic `PauseIntent` on the segment edge from the consumer's engine-specific timing policy. Exact authored timed breaks remain explicit; semantic strengths do not prescribe a duration.

{utterplan-0.5.0 → utterplan-0.5.1}/docs/format.md

@@ -24,6 +24,12 @@ text = "Hello world."
 token.span = [[0, 5], [6, 12]]
 ```
 
+## Compiler-only segmentation topology
+
+The compiler validates PhraseSplit proposals and owns paragraph, sentence, and sentence-part spans in spoken-text coordinates. Semicolon/context-safe colon boundaries and optional clause/parenthetical proposals may create renderer cuts; language, directive, pause, marker, and media subdivisions retain the owning sentence-part identity internally. If proposals are malformed or incomplete, deterministic fallback is used and non-whitespace text remains covered exactly.
+
+Sentence parts and their indices are deliberately absent from this v5 format. The `flow` contains only executable renderer segments and remains the compact public contract; no schema field, FlowPlan/FlowUnit/FlowSegment shape, or migration rule records private topology. Schema v5 is unchanged.
+
 ## Executable flow
 
 Each ordered `flow` entry is a `FlowUnit` with a stable local `hash` and one or more `segment` entries. A segment contains already-prepared speech `text`, an optional language override, local token columns, optional semantic `pause_before` / `pause_after` intents, typed renderer-neutral directives, markers, and an optional heading level. Language defaults to the root `language`; `unit` identifies paragraph or sentence grouping. Consumers render segments in the stored order and do not need compiler lookup tables or segment-ID joins.

{utterplan-0.5.0 → utterplan-0.5.1}/docs/python-api.md

@@ -12,6 +12,12 @@ The Python defaults are deliberately spaCy-free and repair-first: `PlannerConfig
 For an explicit contextual-G2P configuration, use `LinguisticsConfig(use_spacy=True, spacy_model="en_core_web_sm", require_spacy=True)`. The resulting plan records final pass-B token provenance in `linguistic_runs`; no provider document is retained.
 spaCy enrichment is opt-in through the CLI's `--spacy auto` policy or an explicit `LinguisticsConfig` with a compatible local model. It may provide richer tokenization, POS tags, lemmas, and tags, but UtterPlan never downloads a model implicitly.
 
+## Sentence segmentation and private topology
+
+PhraseSplit is an advisory sentence-boundary provider, not an owner of compiler topology. UtterPlan validates each whole proposal batch, maps accepted run-local offsets once into prepared spoken-text coordinates, then recomputes paragraph and sentence indices from parser paragraph events and accepted boundaries. If PhraseSplit fails or returns malformed/incomplete coverage, deterministic conservative fallback preserves the text and emits diagnostics. Automatic cuts that divide provider-aware lexical content are discarded or merged; the final flow-projection token-edge check remains active.
+
+Semicolon and context-safe colon separators, plus optional trusted clause/parenthetical candidates, form private sentence parts. Renderer-required language, directive, pause, marker, and media cuts subdivide those parts without changing their indices. Exact surface and non-whitespace coverage validation runs in strict and repair renderability modes alike. Parts are not a public field: only the compact ordered v5 `flow` is serialized, and `unit="sentence"` groups by the recomputed sentence topology.
+
 ## Canonical single-document compiler
 
 `compile_document` is the shared semantic interpretation boundary for independent

{utterplan-0.5.0 → utterplan-0.5.1}/tests/migration/goldens/multilingual_preparation.json

@@ -63,8 +63,8 @@
       "annotation_ids": []
     },
     {
-      "text": "Bonjour.",
-      "spoken_start": 6,
+      "text": " Bonjour.",
+      "spoken_start": 5,
       "spoken_end": 14,
       "language": "fr",
       "paragraph": 0,

{utterplan-0.5.0 → utterplan-0.5.1}/tests/migration/goldens/ssmd_break.json

@@ -60,7 +60,7 @@
       "language": "en-us",
       "paragraph": 0,
       "sentence": 0,
-      "clause": 1,
+      "clause": 0,
       "pause_before": {
         "seconds": 0.0,
         "events": []

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_cli.py

@@ -220,7 +220,7 @@ def test_compile_strict_renderability_reports_source_and_writes_no_plan(
     assert "Automatic repair: safe;" in captured.err
 
 
-def test_compile_repair_cli_reports_guaranteed_renderability(
+def test_compile_repair_cli_blocks_unattachable_punctuation_without_loss(
     tmp_path: Path, capsys: pytest.CaptureFixture[str]
 ) -> None:
     source = tmp_path / "repair.txt"
@@ -244,9 +244,11 @@ def test_compile_repair_cli_reports_guaranteed_renderability(
     )
 
     captured = capsys.readouterr()
-    assert result == 0
-    assert FlowPlan.load(output).flow
-    assert "renderability: guaranteed; 1 punctuation segment repaired" in captured.err
+    assert result == 1
+    assert not output.exists()
+    assert "isolated punctuation became its own speech segment" in captured.err
+    assert "Prepared fragment: '.'" in captured.err
+    assert "Automatic repair: safe; remove the isolated punctuation-only segment." in captured.err
 
 
 def test_top_level_version(capsys: pytest.CaptureFixture[str]) -> None:

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_cli_batch.py

@@ -237,7 +237,7 @@ def test_compile_many_protects_existing_output_and_force_replaces_it(
     )
 
 
-def test_compile_many_default_repair_and_strict_mode_are_both_observable(
+def test_compile_many_repair_and_strict_modes_preserve_unattachable_punctuation(
     tmp_path: Path, capsys: pytest.CaptureFixture[str]
 ) -> None:
     source = _write(tmp_path / "chapter.txt", "Hello.\n\n.\n\nWorld.")
@@ -257,9 +257,13 @@ def test_compile_many_default_repair_and_strict_mode_are_both_observable(
         ]
     )
     repair_output = capsys.readouterr()
-    assert repair_result == 0
-    assert "repaired at 3:1" in repair_output.err
-    assert "removing the punctuation-only segment" in repair_output.err
+    assert repair_result == 1
+    assert "isolated punctuation became its own speech segment" in repair_output.err
+    assert "Prepared fragment: '.'" in repair_output.err
+    assert (
+        "Automatic repair: safe; remove the isolated punctuation-only segment." in repair_output.err
+    )
+    assert not (repair_dir / "chapter.utterplan.toml").exists()
 
     strict_dir = tmp_path / "strict"
     strict_result = main(

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_cli_ssmdbook.py

@@ -429,3 +429,42 @@ def test_report_path_collision_with_plan_output_is_preflighted(
     assert result == 2
     assert "report path collides with plan output" in captured.err
     assert not output_dir.exists()
+
+
+def test_compile_book_localizes_fallback_boundaries_with_default_settings(
+    tmp_path: Path, capsys: pytest.CaptureFixture[str]
+) -> None:
+    root = tmp_path / "fallback-boundaries.ssmdbook"
+    _write_book(
+        root,
+        (5, 6),
+        contents={
+            5: _ssmd('[alpha]{emphasis="strong"}—beta'),
+            6: _ssmd('alpha—[beta]{emphasis="strong"}'),
+        },
+    )
+
+    result = main(["compile-book", str(root), "--chapters", "5-6"])
+    capsys.readouterr()
+
+    assert result == 0
+    output_dir = root / "utterplan"
+    report = _report(output_dir / "compile-report.toml")
+    assert report["requested"] == 2
+    assert report["written"] == 2
+    assert report["failed"] == 0
+    assert report["skipped"] == 0
+    assert report["complete"] is True
+
+    for number, expected_surface in ((5, "alpha"), (6, "beta")):
+        plan_path = output_dir / f"chapter-{number:04d}.utterplan.toml"
+        assert plan_path.exists()
+        plan = FlowPlan.load(plan_path)
+        assert plan.linguistics[0].provider == "fallback"
+        surfaces = {
+            segment.text[token.start : token.end]
+            for unit in plan.flow
+            for segment in unit.segments
+            for token in segment.tokens
+        }
+        assert expected_surface in surfaces

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_consumer_contract.py

@@ -128,8 +128,11 @@ def test_multilingual_and_sentence_units_remain_in_render_order() -> None:
 
     rendered = _fake_renderer(plan)
     assert [item[1] for item in rendered] == ["en-us", "fr"]
-    assert len(plan.flow) == 1
-    assert len(plan.flow[0].segments) == 2
+    assert len(plan.flow) == 2
+    assert [[segment.text for segment in unit.segments] for unit in plan.flow] == [
+        ["One."],
+        ["Bonjour."],
+    ]
     assert_public_consumer_contract(plan)
 
 

utterplan-0.5.1/tests/test_flow_compilation.py

@@ -0,0 +1,220 @@
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+
+from utterplan import (
+    FlowPlan,
+    LinguisticsConfig,
+    PauseConfig,
+    PlannerConfig,
+    UtterancePlanner,
+    compile_document,
+)
+from utterplan.exceptions import PlanValidationError
+from utterplan.flow_projection import _token_providers, _tokens_for_segment
+from utterplan.migrations.v4_to_v5 import migrate_v4_to_v5
+from utterplan.model import LinguisticRun, TokenAnnotation
+
+
+def _segments(plan: FlowPlan):
+    return tuple(segment for unit in plan.flow for segment in unit.segments)
+
+
+def test_fresh_compile_returns_current_v5_local_flow() -> None:
+    text = 'Hello [world]{emphasis="strong"}.'
+    config = PlannerConfig(language="en-us", document_format="ssmd", text_preparation="identity")
+    result = compile_document(text, input_format="ssmd", config=config)
+    planner = UtterancePlanner(config)
+    try:
+        planned = planner.plan(text)
+    finally:
+        planner.close()
+
+    assert isinstance(result.plan, FlowPlan)
+    assert result.plan.schema_version == 5
+    assert planned == result.plan
+    payload = result.plan.to_toml()
+    assert payload == result.plan.to_toml()
+    assert FlowPlan.from_toml(payload) == result.plan
+    assert not hasattr(result.plan, "tokens")
+    assert not hasattr(result.plan, "segments")
+    assert any(segment.directives.emphasis is not None for segment in _segments(result.plan))
+    for segment in _segments(result.plan):
+        assert all(segment.text[token.start : token.end] for token in segment.tokens)
+
+
+@pytest.mark.parametrize(
+    "fixture_name",
+    ("basic_en", "ssmd_breaks", "markers", "multilingual", "parenthetical", "spokenform_offsets"),
+)
+def test_fresh_compilation_converges_with_v4_migration_on_unambiguous_input(
+    fixture_name: str,
+) -> None:
+    fixture = Path(__file__).parent / f"migration/fixtures/v4/{fixture_name}.utterplan.json"
+    v4_plan = json.loads(fixture.read_text(encoding="utf-8"))
+    old_config = v4_plan["config"]
+    old_pauses = old_config.get("pauses", {})
+    config = PlannerConfig(
+        language=old_config["language"],
+        document_format=old_config["document_format"],
+        unit=old_config.get("unit", "paragraph"),
+        text_preparation=old_config.get("text_preparation", "spokenform"),
+        pauses=PauseConfig(
+            mode=old_pauses.get("mode", "tts"),
+            enabled=old_pauses.get("enabled", True),
+        ),
+    )
+    fresh = compile_document(
+        v4_plan["source"]["text"],
+        input_format=old_config["document_format"],
+        config=config,
+    ).plan
+    migrated = FlowPlan.from_dict(migrate_v4_to_v5(v4_plan))
+
+    if fixture_name == "multilingual":
+        # The compiler now preserves the inter-language whitespace on the right-hand
+        # renderer segment instead of dropping it at the run boundary.
+        assert fresh.flow != migrated.flow
+        assert [[segment.text for segment in unit.segments] for unit in fresh.flow] == [
+            ["Hello", " Bonjour."]
+        ]
+        assert fresh.document == migrated.document
+        assert fresh.linguistics == migrated.linguistics
+        assert fresh.language == migrated.language
+        assert fresh.unit == migrated.unit
+    else:
+        assert fresh.plan_id == migrated.plan_id
+        assert fresh.flow == migrated.flow
+        assert fresh.document == migrated.document
+        assert fresh.linguistics == migrated.linguistics
+        assert fresh.language == migrated.language
+        assert fresh.unit == migrated.unit
+
+
+def test_fresh_compile_localizes_a_token_cut_by_ssmd_semantics() -> None:
+    fixture = Path(__file__).parent / "migration/fixtures/v4/ssmd_09_comprehensive.utterplan.json"
+    source = json.loads(fixture.read_text(encoding="utf-8"))["source"]["text"]
+    result = compile_document(
+        source,
+        input_format="ssmd",
+        config=PlannerConfig(
+            language="sr-Latn",
+            document_format="ssmd",
+            text_preparation="identity",
+        ),
+    )
+
+    segments = _segments(result.plan)
+    water_segment = next(segment for segment in segments if segment.text == "H2O")
+    assert any(segment.text == ", and" for segment in segments)
+    assert any(
+        token.start == 0 and token.end == 3 and water_segment.text[token.start : token.end] == "H2O"
+        for token in water_segment.tokens
+    )
+    for segment in segments:
+        for token in segment.tokens:
+            assert segment.text[token.start : token.end]
+
+
+def _project_partial_token(
+    text: str,
+    start: int,
+    end: int,
+    *,
+    provider: str = "fallback",
+    lemma: str | None = None,
+):
+    token = TokenAnnotation(0, len(text), text, lemma=lemma)
+    plan = SimpleNamespace(
+        texts=SimpleNamespace(spoken=text),
+        tokens=(token,),
+        linguistic_runs=(LinguisticRun("language-0001", provider, 0, 1),),
+    )
+    segment = SimpleNamespace(
+        id="seg-000123",
+        language="en-US",
+        spoken_start=start,
+        spoken_end=end,
+        token_indices=(0,),
+    )
+    return _tokens_for_segment(
+        plan,
+        segment,
+        _token_providers(plan),
+        "$.flow[0].segments[0].tokens",
+    )
+
+
+@pytest.mark.parametrize(
+    ("body", "expected_surface"),
+    (
+        ('[alpha]{emphasis="strong"}—beta', "alpha"),
+        ('alpha—[beta]{emphasis="strong"}', "beta"),
+    ),
+)
+def test_fresh_compile_localizes_fallback_tokens_at_internal_punctuation(
+    body: str, expected_surface: str
+) -> None:
+    source = f'---\nssmd_version: "0.9"\nlanguage: en-US\n---\n{body}\n'
+    result = compile_document(
+        source,
+        input_format="ssmd",
+        config=PlannerConfig(
+            language="en-US",
+            document_format="ssmd",
+            text_preparation="identity",
+            linguistics=LinguisticsConfig(use_spacy=False),
+        ),
+    )
+
+    assert result.plan.linguistics[0].provider == "fallback"
+    segments = _segments(result.plan)
+    assert any(
+        segment.text[token.start : token.end] == expected_surface
+        for segment in segments
+        for token in segment.tokens
+    )
+    for segment in segments:
+        assert all(segment.text[token.start : token.end] for token in segment.tokens)
+
+
+@pytest.mark.parametrize(
+    ("start", "end", "split_side"),
+    ((0, 3, "end"), (2, 5, "start")),
+)
+def test_fallback_provider_rejects_true_alphanumeric_splits_with_coordinates(
+    start: int, end: int, split_side: str
+) -> None:
+    with pytest.raises(PlanValidationError) as exc_info:
+        _project_partial_token("hello", start, end, lemma="hello")
+
+    error = exc_info.value
+    assert error.code == "token.segment_split"
+    assert error.path == "$.flow[0].segments[0].tokens"
+    assert f"segment seg-000123 [{start}:{end}]" in str(error)
+    assert f"token 0 [0:5] at its {split_side}" in str(error)
+
+
+@pytest.mark.parametrize("provider", ("spacy", "unknown"))
+def test_nonfallback_provider_keeps_punctuation_connected_partial_tokens_strict(
+    provider: str,
+) -> None:
+    with pytest.raises(PlanValidationError) as exc_info:
+        _project_partial_token("alpha—beta", 0, 5, provider=provider, lemma="alpha—beta")
+
+    assert exc_info.value.code == "token.segment_split"
+
+
+def test_fallback_partial_default_lemmas_are_casefolded_per_fragment() -> None:
+    text = "Straße—Bahn"
+    default_lemma = text.lower()
+
+    left = _project_partial_token(text, 0, 6, lemma=default_lemma)
+    right = _project_partial_token(text, 7, 11, lemma=default_lemma)
+
+    assert [token.lemma for token in left] == ["strasse"]
+    assert [token.lemma for token in right] == ["bahn"]

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_linguistics.py

@@ -13,11 +13,9 @@ from utterplan import (
     PlannerConfig,
     UtterancePlan,
 )
-from utterplan.exceptions import PlanningError
 from utterplan.explain import format_explanation
 from utterplan.language import LanguageRun
 from utterplan.linguistics import LinguisticResourcePool, analyze_run_analyses
-from utterplan.planner import _split_run
 
 
 class FakeProviderDoc(list):
@@ -209,7 +207,7 @@ def test_clausal_boundaries_reuse_provider_document(monkeypatch):
     assert len(docs) == 1
     assert detected == [(source, docs[-1])]
     boundary = next(item for item in plan.boundaries if item.kind == "clausal_comma")
-    assert boundary.position == source.index(",")
+    assert boundary.position == source.index(",") + 2
     semantic = next(item for item in plan.semantic_boundaries if item.kind == "clause")
     assert semantic.position == source.index(",") + 2
     assert semantic.attrs["detector_start"] == source.index(",")
@@ -274,8 +272,8 @@ def test_semantic_clause_is_independent_of_pause_activation(monkeypatch):
     assert [(item.position, item.kind, item.id) for item in clauses] == [
         (source.index(",") + 2, "clause", "semantic-boundary-000000")
     ] * 3
-    assert len(plans[0].segments) == 1
-    assert plans[0].segments[0].spoken_start < clauses[0].position < plans[0].segments[0].spoken_end
+    assert len(plans[0].segments) == 2
+    assert plans[0].segments[0].spoken_end == clauses[0].position
     assert not any(
         segment.pause_before is not None or segment.pause_after is not None
         for segment in plans[0].segments
@@ -331,17 +329,6 @@ def test_fallback_does_not_attempt_clausal_boundary_detection(monkeypatch):
     assert not any(item.kind == "clause" for item in plan.semantic_boundaries)
 
 
-def test_segmentation_type_error_is_not_a_whole_document_fallback(monkeypatch):
-    import phrasplit
-
-    def broken_split(*args, **kwargs):
-        raise TypeError("invalid nlp integration")
-
-    monkeypatch.setattr(phrasplit, "split_with_offsets", broken_split)
-    with pytest.raises(PlanningError, match="sentence segmentation integration failed"):
-        _split_run("One sentence. Two sentences.", "en-us")
-
-
 def _fake_spacy_plan(
     monkeypatch, *, tag: str = "VBP", morph: str = "Tense=Pres|VerbForm=Fin"
 ) -> UtterancePlan:

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_planner.py

@@ -2,6 +2,7 @@ import pytest
 
 from tests.compiler_helpers import CompilerTestPlanner as UtterancePlanner
 from utterplan import PauseConfig, PauseIntent, PlannerConfig, PlanningError
+from utterplan.exceptions import PlanValidationError
 
 
 def test_plain_sentences_paragraphs_and_pauses():
@@ -16,6 +17,28 @@ def test_plain_sentences_paragraphs_and_pauses():
     assert plan.segments[1].pause_after == PauseIntent("paragraph")
 
 
+def test_sentence_part_edges_split_renderer_segments_and_auto_pause_at_part_boundary():
+    source = "Before; after."
+    plans = [
+        UtterancePlanner(
+            PlannerConfig(
+                language="en-us",
+                text_preparation="identity",
+                pauses=PauseConfig(mode=mode, enabled=enabled),
+            )
+        ).plan(source)
+        for mode, enabled in (("tts", True), ("auto", True), ("auto", False))
+    ]
+    expected_texts = ["Before; ", "after."]
+    for plan in plans:
+        assert [segment.text for segment in plan.segments] == expected_texts
+        assert [segment.clause for segment in plan.segments] == [0, 1]
+        assert plan.semantic_boundaries[0].position == len("Before; ")
+    assert plans[0].segments[0].pause_after is None
+    assert plans[1].segments[0].pause_after == PauseIntent("clause")
+    assert plans[2].segments[0].pause_after is None
+
+
 def test_ssmd_preparation_and_explicit_directives():
     plan = UtterancePlanner(PlannerConfig(language="en-us", document_format="ssmd")).plan(
         '[tomato]{ph="təˈmeɪtoʊ"}'
@@ -97,7 +120,7 @@ def test_unrepresentable_audio_topology_is_rejected(source: str, message: str) -
         planner.plan(source)
 
 
-def test_adjacent_audio_annotations_are_distinct_and_supported() -> None:
+def test_adjacent_audio_annotations_report_required_token_conflict() -> None:
     planner = UtterancePlanner(
         PlannerConfig(
             language="en-us",
@@ -105,18 +128,11 @@ def test_adjacent_audio_annotations_are_distinct_and_supported() -> None:
             text_preparation="identity",
         )
     )
-    plan = planner.plan('[first]{src="same.wav"}[second]{src="same.wav"}')
-    audio_segments = [segment for segment in plan.segments if segment.directives.audio is not None]
 
-    assert len(audio_segments) == 2
-    first_audio = audio_segments[0].directives.audio
-    second_audio = audio_segments[1].directives.audio
+    with pytest.raises(PlanValidationError) as exc_info:
+        planner.plan('[first]{src="same.wav"}[second]{src="same.wav"}')
 
-    assert first_audio is not None
-    assert second_audio is not None
-    assert first_audio.src == second_audio.src
-    assert audio_segments[0].id != audio_segments[1].id
-    assert audio_segments[0].spoken_start < audio_segments[1].spoken_start
+    assert exc_info.value.code == "segmentation.required_cut_conflict"
 
 
 def test_audio_fallback_is_atomic_across_sentence_segmentation() -> None:

{utterplan-0.5.0 → utterplan-0.5.1}/tests/test_renderability_repair.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 from dataclasses import replace
+from typing import Literal
 
 import pytest
 
@@ -72,26 +73,20 @@ def _assess(
     return replace(issue, repair_assessment=assessment)
 
 
-def test_repair_is_the_default_and_records_a_source_located_guarantee() -> None:
+def test_repair_mode_preserves_isolated_punctuation_instead_of_dropping_content() -> None:
     config = PlannerConfig(language="en-US", text_preparation="identity")
-    plan = UtterancePlanner(config).plan("Hello.\n\n.\n\nWorld.")
-    repeated = UtterancePlanner(config).plan("Hello.\n\n.\n\nWorld.")
+    planner = UtterancePlanner(config)
+    attempt = planner.compile_attempt("Hello.\n\n.\n\nWorld.")
+    repeated = UtterancePlanner(config).compile_attempt("Hello.\n\n.\n\nWorld.")
 
     assert config.renderability_mode == "repair"
-    assert repeated.plan_id == plan.plan_id
-    assert repeated.to_dict() == plan.to_dict()
-    assert [segment.text for segment in plan.segments] == ["Hello.", "World."]
-    assert plan.document_metadata["planning"]["renderability"] == {
-        "mode": "repair",
-        "checked_segments": 3,
-        "repair_count": 1,
-        "guaranteed": True,
-    }
-    repairs = [item for item in plan.diagnostics if item.code == "planning.renderability.repaired"]
-    assert len(repairs) == 1
-    assert "by removing the punctuation-only segment" in repairs[0].message
-    assert repairs[0].line == 3
-    assert plan.texts.spoken == "Hello.\n\n.\n\nWorld."
+    assert attempt.status == "blocked"
+    assert repeated.attempt_id == attempt.attempt_id
+    assert [segment.text for segment in attempt.candidate.segments] == ["Hello.", ".", "World."]
+    assert attempt.candidate.document_metadata["planning"]["renderability"]["guaranteed"] is False
+    assert not any(item.code == "planning.renderability.repaired" for item in attempt.diagnostics)
+    assert attempt.renderability.issues[0].line == 3
+    assert attempt.candidate.texts.spoken == "Hello.\n\n.\n\nWorld."
 
 
 def test_default_repairs_exact_comma_space_with_isolated_planner_topology(
@@ -362,18 +357,18 @@ def test_strict_attempt_retains_blocked_candidate_and_compile_still_raises() ->
     assert error.value.issues == attempt.renderability.issues
 
 
-def test_safe_repair_attempt_returns_a_canonical_plan() -> None:
+def test_unattachable_punctuation_remains_in_a_blocked_repair_attempt() -> None:
     config = PlannerConfig(language="en-US", text_preparation="identity")
     attempt = UtterancePlanner(config).compile_attempt("Hello.\n\n.\n\nWorld.")
 
-    assert attempt.status == "repaired"
-    assert attempt.ok
-    assert len(attempt.repairs) == 1
-    assert attempt.repairs[0].repair_assessment is not None
-    assert attempt.repairs[0].repair_assessment.safe
-    plan = attempt.candidate.to_plan()
-    plan.validate()
-    assert [segment.text for segment in plan.segments] == ["Hello.", "World."]
+    assert attempt.status == "blocked"
+    assert not attempt.ok
+    assert attempt.repairs == ()
+    assert attempt.renderability.issues[0].repair_assessment is not None
+    assert attempt.renderability.issues[0].repair_assessment.safe
+    assert [segment.text for segment in attempt.candidate.segments] == ["Hello.", ".", "World."]
+    with pytest.raises(ValueError, match="blocked planning-attempt draft"):
+        attempt.candidate.to_plan()
 
 
 def test_unsafe_repair_attempt_is_blocked_and_draft_structure_is_checked() -> None:
@@ -405,12 +400,12 @@ def test_unsafe_repair_attempt_is_blocked_and_draft_structure_is_checked() -> No
     ("source", "mode", "expected_status"),
     [
         ("Ready.", "strict", "renderable"),
-        ("Hello.\n\n.\n\nWorld.", "repair", "repaired"),
+        ("Hello.\n\n.\n\nWorld.", "repair", "blocked"),
         ("€", "strict", "blocked"),
     ],
 )
 def test_planning_attempt_toml_roundtrips_all_outcomes_and_stays_noncanonical(
-    source: str, mode: str, expected_status: str
+    source: str, mode: Literal["strict", "repair"], expected_status: str
 ) -> None:
     config = PlannerConfig(
         language="en-US",