openkos 0.2.4__tar.gz → 0.2.6__tar.gz

{openkos-0.2.4 → openkos-0.2.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: openkos
-Version: 0.2.4
+Version: 0.2.6
 Summary: Local-first engine that turns your scattered text into a living, portable knowledge base in the Open Knowledge Format (OKF)
 Keywords: okf,open-knowledge-format,knowledge-base,knowledge-graph,local-first,rag,llm,second-brain,cli
 Author: Jason
@@ -21,7 +21,7 @@ Classifier: Typing :: Typed
 Requires-Dist: networkx>=3.4
 Requires-Dist: python-frontmatter>=1.3.0
 Requires-Dist: pyyaml>=6.0.3
-Requires-Dist: scipy>=1.14
+Requires-Dist: rich>=13.8.0
 Requires-Dist: sqlite-vec>=0.1.6
 Requires-Dist: typer>=0.27.0
 Requires-Python: >=3.12
@@ -59,15 +59,23 @@ ollama pull qwen3:8b   # chat model — extraction and answers (~5 GB)
 ollama pull bge-m3     # embedding model — semantic search (~1.2 GB)
 ```
 
-> There is a third, **optional** model for naming relations during curation (`ollama pull gemma2:27b`, **15.6 GB**). Skip it for now — if a command wants it, the error message tells you exactly what to pull.
+> These two are all curation needs out of the box. For naming relations during curation there is an **optional measured upgrade** (`ollama pull gemma2:27b`, **15.6 GB**) that nearly doubles relation-type accuracy — opt in later with `models: {edge_typing: gemma2:27b}` in `openkos.yaml`; `openkos doctor` and `curate` both point at it.
 
 **3 · Install the engine** (needs Python 3.12+ and [git](https://git-scm.com)):
 
 ```bash
 uv tool install openkos   # or: pipx install openkos — or: pip install openkos
+openkos --version         # check what you actually got
 ```
 
-To track the latest unreleased `main` instead: `uv tool install git+https://github.com/jasonssdev/openkos`.
+**Check that version.** If it does not match the badge at the top of this page, your package index is serving an older release. Install from the repository instead — same engine, current code:
+
+```bash
+uv tool install --force git+https://github.com/jasonssdev/openkos
+openkos --version
+```
+
+That command is also how you track unreleased `main` at any time.
 
 **4 · Check the setup before anything can fail:**
 
@@ -75,7 +83,7 @@ To track the latest unreleased `main` instead: `uv tool install git+https://gith
 openkos doctor
 ```
 
-Every line should read `[PASS]` except *Workspace initialized* — you haven't created one yet. `doctor` is also the first thing to run whenever something misbehaves later.
+Outside a workspace, expect one `[FAIL]` — *Workspace initialized*, since you haven't created one yet — and six `[SKIP]` lines for the checks that need a workspace to inspect. Everything else should `[PASS]`. Anything that fails prints the command that fixes it. `doctor` is also the first thing to run whenever something misbehaves later.
 
 **5 · Create a knowledge base and run the loop:**
 
@@ -140,7 +148,7 @@ The wedge, in one line: **the local-first, personal producer-consumer-runtime fo
 OpenKOS ships in three MVP arcs, each usable on its own. Full detail in [`docs/roadmap.md`](https://github.com/jasonssdev/openkos/blob/main/docs/roadmap.md).
 
 - **MVP 1 — The Compiler. (Complete.)** The Karpathy loop, locally, over text: ingest → OKF concepts with provenance → cited query → freshness lint. Useful in an afternoon.
-- **MVP 2 — The Graph and Memory. (Complete.)** Entity/relationship extraction and reversible merge, a typed knowledge graph (an OpenKOS layer over OKF's untyped links — other tools still read the bundle fine), hybrid retrieval (lexical and semantic, rank-fused), a fail-closed sensitivity filter (confidential concepts never leave the machine — held back from any backend that is not verifiably local), a guided curation loop, reference-aware `forget` plus an irreversible `purge` (right-to-be-forgotten), and answers that file back into the base (the two-output rule).
+- **MVP 2 — The Graph and Memory. (Complete.)** Entity/relationship extraction and reversible merge, a typed knowledge graph (an OpenKOS layer over OKF's untyped links — other tools still read the bundle fine), hybrid retrieval (lexical and semantic, rank-fused), contradiction detection with durable verdicts (findings persist, so a repeat check costs no model calls, and `reconcile` records how you settled each one), a fail-closed sensitivity filter (confidential concepts never leave the machine — held back from any backend that is not verifiably local), a guided curation loop, reference-aware `forget` plus an irreversible `purge` (right-to-be-forgotten), and answers that file back into the base (the two-output rule).
 - **MVP 3 — The Runtime and Interoperability.** An MCP server and APIs so agents use OpenKOS as durable memory; full OKF import/export with the wider ecosystem.
 
 Beyond that: a desktop app, graph visualization, richer memory, and federation — explored only after the MVPs prove out with real users.

{openkos-0.2.4 → openkos-0.2.6}/README.md

@@ -26,15 +26,23 @@ ollama pull qwen3:8b   # chat model — extraction and answers (~5 GB)
 ollama pull bge-m3     # embedding model — semantic search (~1.2 GB)
 ```
 
-> There is a third, **optional** model for naming relations during curation (`ollama pull gemma2:27b`, **15.6 GB**). Skip it for now — if a command wants it, the error message tells you exactly what to pull.
+> These two are all curation needs out of the box. For naming relations during curation there is an **optional measured upgrade** (`ollama pull gemma2:27b`, **15.6 GB**) that nearly doubles relation-type accuracy — opt in later with `models: {edge_typing: gemma2:27b}` in `openkos.yaml`; `openkos doctor` and `curate` both point at it.
 
 **3 · Install the engine** (needs Python 3.12+ and [git](https://git-scm.com)):
 
 ```bash
 uv tool install openkos   # or: pipx install openkos — or: pip install openkos
+openkos --version         # check what you actually got
 ```
 
-To track the latest unreleased `main` instead: `uv tool install git+https://github.com/jasonssdev/openkos`.
+**Check that version.** If it does not match the badge at the top of this page, your package index is serving an older release. Install from the repository instead — same engine, current code:
+
+```bash
+uv tool install --force git+https://github.com/jasonssdev/openkos
+openkos --version
+```
+
+That command is also how you track unreleased `main` at any time.
 
 **4 · Check the setup before anything can fail:**
 
@@ -42,7 +50,7 @@ To track the latest unreleased `main` instead: `uv tool install git+https://gith
 openkos doctor
 ```
 
-Every line should read `[PASS]` except *Workspace initialized* — you haven't created one yet. `doctor` is also the first thing to run whenever something misbehaves later.
+Outside a workspace, expect one `[FAIL]` — *Workspace initialized*, since you haven't created one yet — and six `[SKIP]` lines for the checks that need a workspace to inspect. Everything else should `[PASS]`. Anything that fails prints the command that fixes it. `doctor` is also the first thing to run whenever something misbehaves later.
 
 **5 · Create a knowledge base and run the loop:**
 
@@ -107,7 +115,7 @@ The wedge, in one line: **the local-first, personal producer-consumer-runtime fo
 OpenKOS ships in three MVP arcs, each usable on its own. Full detail in [`docs/roadmap.md`](https://github.com/jasonssdev/openkos/blob/main/docs/roadmap.md).
 
 - **MVP 1 — The Compiler. (Complete.)** The Karpathy loop, locally, over text: ingest → OKF concepts with provenance → cited query → freshness lint. Useful in an afternoon.
-- **MVP 2 — The Graph and Memory. (Complete.)** Entity/relationship extraction and reversible merge, a typed knowledge graph (an OpenKOS layer over OKF's untyped links — other tools still read the bundle fine), hybrid retrieval (lexical and semantic, rank-fused), a fail-closed sensitivity filter (confidential concepts never leave the machine — held back from any backend that is not verifiably local), a guided curation loop, reference-aware `forget` plus an irreversible `purge` (right-to-be-forgotten), and answers that file back into the base (the two-output rule).
+- **MVP 2 — The Graph and Memory. (Complete.)** Entity/relationship extraction and reversible merge, a typed knowledge graph (an OpenKOS layer over OKF's untyped links — other tools still read the bundle fine), hybrid retrieval (lexical and semantic, rank-fused), contradiction detection with durable verdicts (findings persist, so a repeat check costs no model calls, and `reconcile` records how you settled each one), a fail-closed sensitivity filter (confidential concepts never leave the machine — held back from any backend that is not verifiably local), a guided curation loop, reference-aware `forget` plus an irreversible `purge` (right-to-be-forgotten), and answers that file back into the base (the two-output rule).
 - **MVP 3 — The Runtime and Interoperability.** An MCP server and APIs so agents use OpenKOS as durable memory; full OKF import/export with the wider ecosystem.
 
 Beyond that: a desktop app, graph visualization, richer memory, and federation — explored only after the MVPs prove out with real users.

{openkos-0.2.4 → openkos-0.2.6}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "openkos"
-version = "0.2.4"
+version = "0.2.6"
 description = "Local-first engine that turns your scattered text into a living, portable knowledge base in the Open Knowledge Format (OKF)"
 readme = "README.md"
 license = "Apache-2.0"
@@ -34,7 +34,7 @@ dependencies = [
     "networkx>=3.4",
     "python-frontmatter>=1.3.0",
     "pyyaml>=6.0.3",
-    "scipy>=1.14",
+    "rich>=13.8.0",
     "sqlite-vec>=0.1.6",
     "typer>=0.27.0",
 ]
@@ -61,10 +61,8 @@ dev = [
     "git-filter-repo>=2.47.0",
     "mypy>=2.3.0",
     "pre-commit>=4.6.0",
-    "pydantic>=2.13.4",
     "pytest>=9.1.1",
     "pytest-cov>=7.1.0",
-    "ruamel-yaml>=0.19.1",
     "ruff>=0.15.21",
     "types-networkx>=3.6.1.20260624",
     "types-pyyaml>=6.0.12.20260518",

{openkos-0.2.4 → openkos-0.2.6}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "openkos"
-version = "0.2.4"
+version = "0.2.6"
 description = "Local-first engine that turns your scattered text into a living, portable knowledge base in the Open Knowledge Format (OKF)"
 readme = "README.md"
 license = "Apache-2.0"
@@ -40,7 +40,12 @@ dependencies = [
     "networkx>=3.4",
     "python-frontmatter>=1.3.0",
     "pyyaml>=6.0.3",
-    "scipy>=1.14",
+    # `cli/main.py` imports `rich.console.Console` directly. Typer happens to
+    # depend on rich too, but an undeclared transitive import is a dependency
+    # we do not control: the day Typer drops or vendors it, openkos breaks at
+    # import time on an install that resolved cleanly. Declare what we import.
+    # The floor matches Typer's own so the resolver has no extra constraint.
+    "rich>=13.8.0",
     "sqlite-vec>=0.1.6",
     "typer>=0.27.0",
 ]
@@ -59,14 +64,15 @@ requires = ["uv_build>=0.11.29,<0.12.0"]
 build-backend = "uv_build"
 
 [dependency-groups]
+# `git-filter-repo` is a real test requirement, not just `purge`'s runtime
+# system tool: `test_filter_repo_available_true_when_installed` asserts the
+# PATH probe finds it, so it fails on a machine without the binary.
 dev = [
     "git-filter-repo>=2.47.0",
     "mypy>=2.3.0",
     "pre-commit>=4.6.0",
-    "pydantic>=2.13.4",
     "pytest>=9.1.1",
     "pytest-cov>=7.1.0",
-    "ruamel-yaml>=0.19.1",
     "ruff>=0.15.21",
     "types-networkx>=3.6.1.20260624",
     "types-pyyaml>=6.0.12.20260518",

{openkos-0.2.4 → openkos-0.2.6}/src/openkos/bundle/decisions.py

@@ -151,7 +151,16 @@ def read_decisions(concept_id: str, bundle_dir: Path) -> list[DecisionRecord]:
     sidecar on disk (no decision ever declined/reopened under this concept
     id) returns `[]` -- mirrors `bundle.ledger.read_entries`'s own
     "absent file" contract."""
-    path = decisions_path_for(concept_id, bundle_dir)
+    return read_decisions_at(decisions_path_for(concept_id, bundle_dir))
+
+
+def read_decisions_at(path: Path) -> list[DecisionRecord]:
+    """Read every `DecisionRecord` from the sidecar AT `path` -- the walked-
+    path reader the privacy sweep and the declined-listing views use, so a
+    record is read from the file actually on disk, never from a path rebuilt
+    from a (possibly drifted or hostile) `concept_id` frontmatter field.
+    `read_decisions` is the id-addressed wrapper for callers that legitimately
+    own the id. Absent file returns `[]`."""
     if not path.is_file():
         return []
     metadata, _ = okf.load_frontmatter(path.read_text(encoding="utf-8"))
@@ -172,7 +181,26 @@ def write_decisions(
 
     Written via `fsio.write_atomic`, over `okf.dump_frontmatter`'s output
     with an empty body (ADR-0002 invariant 3, preserved literally)."""
-    path = decisions_path_for(concept_id, bundle_dir)
+    return rewrite_decisions_at(
+        decisions_path_for(concept_id, bundle_dir),
+        concept_id=concept_id,
+        records=records,
+    )
+
+
+def rewrite_decisions_at(
+    path: Path, *, concept_id: str, records: list[DecisionRecord]
+) -> Path:
+    """(Re)write the sidecar AT `path` to hold EXACTLY `records`, using
+    `concept_id` only as container CONTENT -- never to derive the path.
+
+    The walked-path twin of `bundle.ledger.rewrite_entries_at`: the privacy
+    sweep must write each rewrite back to the path it WALKED, not to a path
+    rebuilt from the (possibly drifted or hostile) `concept_id` frontmatter
+    field, which would let a traversal id escape the bundle and silently
+    scrub the wrong file. `write_decisions` is the id-addressed wrapper. An
+    empty `records` list removes the file; removing an absent file is a
+    no-op."""
     if not records:
         if path.is_file():
             path.unlink()

{openkos-0.2.4 → openkos-0.2.6}/src/openkos/bundle/index.py

@@ -52,6 +52,70 @@ def _reject_newline(field: str, value: str) -> None:
         raise ValueError(f"index.md: {field!r} must not contain a newline")
 
 
+_TITLE_LABEL_DELIMITERS_RE = re.compile(r"[\[\]]")
+_SLUG_TARGET_DELIMITERS_RE = re.compile(r"[()]")
+
+
+def reject_title_link_delimiters(field: str, value: str) -> None:
+    r"""Raise `ValueError` if `value` contains `[` or `]` (RISK-1, insert path).
+
+    A `title` is interpolated verbatim into the LABEL span of the bullet
+    (`* [{title}](...)`). A `[`/`]` there forges a second link or closes the
+    label early, so `_LINK_RE`/`_LABELLED_LINK_RE` then match the FORGED link
+    instead of the real one -- the entry becomes invisible to
+    `remove_index_entry`/`relabel_index_entry`, and a forged id can delete a
+    DIFFERENT entry. This is the untrusted, LLM-derived counterpart of
+    `_reject_markdown_link_delimiters` (which guards the operator-supplied
+    relabel path): the STRICT insert-time backstop that fails closed for any
+    caller. Callers that want to PRESERVE a title carrying brackets (a benign
+    filename like `notes[1].txt`, as well as a hostile one) sanitize it first
+    with `sanitize_link_label`, which neutralizes the delimiters instead of
+    rejecting the whole value.
+
+    Parentheses and backticks are deliberately NOT rejected here: inside the
+    label they sit within `[^\]]*` and never reach the target, so a common
+    title like ``map() function`` or ``` `useState` `` stays valid --
+    rejecting them would be a false positive with no security benefit.
+    """
+    if _TITLE_LABEL_DELIMITERS_RE.search(value):
+        raise ValueError(
+            f"index.md: {field!r} must not contain a markdown link label "
+            "delimiter ('[' or ']')"
+        )
+
+
+def _reject_slug_link_delimiters(field: str, value: str) -> None:
+    """Raise `ValueError` if `value` contains `(` or `)` (RISK-1, insert path).
+
+    A `slug` is interpolated into the link TARGET span
+    (`](/{link_dir}/{slug}.md)`). A `(`/`)` there closes or forges the
+    target, changing the bullet's resolved identity. Slugs produced by
+    `_slugify` can never contain these, so this is defense-in-depth for the
+    generic public inserter.
+    """
+    if _SLUG_TARGET_DELIMITERS_RE.search(value):
+        raise ValueError(
+            f"index.md: {field!r} must not contain a markdown link target "
+            "delimiter ('(' or ')')"
+        )
+
+
+def sanitize_link_label(title: str) -> str:
+    r"""Return `title` with markdown link LABEL delimiters (`[`, `]`) replaced
+    by their round-paren equivalents, so the value is safe to interpolate
+    into a bullet's `[label](...)` span in `index.md`/`log.md`.
+
+    Brackets close or forge the label for the regex-based `_LINK_RE`/
+    `_LABELLED_LINK_RE` matchers -- maliciously, via an injected LLM-extracted
+    title, or benignly, via a filename like `notes[1].txt`. Parentheses sit
+    inside `[^\]]*` and are inert there, so the substitution keeps the entry
+    matchable/removable by its id while preserving the visual grouping. This
+    is DISPLAY text only: identity is the slug, which `_slugify` derives
+    independently, so the substitution never changes which object exists.
+    """
+    return title.replace("[", "(").replace("]", ")")
+
+
 # `_CANONICAL_SECTION_ORDER` is now derived from
 # `openkos.model.types.REGISTRY` -- see that module for the single source of
 # truth.
@@ -83,10 +147,14 @@ def insert_index_entry(
     eight currently exist -- preserving the historical Sources-last behavior
     byte-identically.
     `title`/`slug`/`description` are each rejected (`ValueError`) if they
-    contain a newline (RISK-1) -- see `_reject_newline`. This guard applies
-    to every section, including untrusted, LLM-derived object fields.
-    `section` MUST be one of the canonical sections, else `ValueError` --
-    there is no defined rank for an unknown section.
+    contain a newline (RISK-1) -- see `_reject_newline`. `title` is
+    additionally rejected if it carries a markdown link LABEL delimiter
+    (`[`/`]`) and `slug` if it carries a link TARGET delimiter (`(`/`)`) --
+    see `reject_title_link_delimiters`/`_reject_slug_link_delimiters` -- so an
+    untrusted, LLM-derived title can never forge or break the bullet's first
+    link. This guard applies to every section, including untrusted object
+    fields. `section` MUST be one of the canonical sections, else
+    `ValueError` -- there is no defined rank for an unknown section.
     """
     if section not in _CANONICAL_SECTION_ORDER:
         raise ValueError(
@@ -95,6 +163,8 @@ def insert_index_entry(
     _reject_newline("title", title)
     _reject_newline("slug", slug)
     _reject_newline("description", description)
+    reject_title_link_delimiters("title", title)
+    _reject_slug_link_delimiters("slug", slug)
     frontmatter_block, body = _split_frontmatter_verbatim(index_text)
     chunks = _SECTION_SPLIT_RE.split(body)
     preamble, section_chunks = chunks[0], chunks[1:]

{openkos-0.2.4 → openkos-0.2.6}/src/openkos/bundle/ledger.py

@@ -212,7 +212,28 @@ def write_entries(
     `entries` list removes the sidecar file entirely (a survivor with no
     remaining merges keeps no empty ledger on disk); removing an
     already-absent sidecar is a no-op, not an error."""
-    path = ledger_path_for(concept_id, bundle_dir)
+    return rewrite_entries_at(
+        ledger_path_for(concept_id, bundle_dir),
+        survivor_id=survivor_id,
+        entries=entries,
+    )
+
+
+def rewrite_entries_at(
+    path: Path, *, survivor_id: str, entries: list[okf.MergeLedgerEntry]
+) -> Path:
+    """(Re)write the sidecar AT `path` to hold EXACTLY `entries`, using
+    `survivor_id` only as container CONTENT -- never to derive the path.
+
+    The privacy sweep walks real sidecar paths and MUST write each rewrite
+    back to the path it WALKED, not to a path rebuilt from the (possibly
+    drifted or hostile) `survivor_id` frontmatter field. Rebuilding it both
+    lets a traversal id escape the bundle (arbitrary-file create/delete with
+    the `.ledger.okf` suffix) AND silently writes a legit NFC/NFD-drifted
+    sidecar to the wrong place while reporting the walked one as cleaned --
+    for a privacy sweep, a silent scrub miss. `write_entries` is the
+    id-addressed wrapper for callers that legitimately own the id. An empty
+    `entries` list removes the file; removing an absent file is a no-op."""
     if not entries:
         if path.is_file():
             path.unlink()

{openkos-0.2.4 → openkos-0.2.6}/src/openkos/bundle/merge.py

@@ -72,6 +72,21 @@ def _reject_same_or_blank(survivor_id: str, absorbed_id: str) -> None:
         )
 
 
+def _absorbed_snapshot_has_content(absorbed_snapshot: str) -> bool:
+    """`True` unless `absorbed_snapshot`'s BODY is provably empty or
+    whitespace-only (#685): only a body with actual content can have been
+    woven undelimited into a reconciled survivor, so only it earns the
+    carried-content annotation. Fails CLOSED to `True` (annotate) on a
+    snapshot this planner cannot parse -- under-redacting a body that might
+    exist is the failure mode the annotation exists to prevent (#602's
+    privacy-over-reversibility rule)."""
+    try:
+        _, body = okf.load_frontmatter(absorbed_snapshot)
+    except Exception:  # broad: any parse failure means "cannot prove empty"
+        return True
+    return body.strip() != ""
+
+
 def _reject_already_merged(
     absorbed_id: str, existing_entries: list[okf.MergeLedgerEntry]
 ) -> None:
@@ -115,9 +130,10 @@ def plan_merge(
     rewrite-provenance-on-merge) likewise defaults to `[]`; the actual
     third-party inbound scan (`bundle/provenance.py`) and CLI wiring are
     PR2's concern -- this layer only carries whatever the caller injects
-    into the new ledger entry, and ALWAYS writes `MERGE_LEDGER_SCHEMA_V3`
-    (the reader still accepts v1 and v2 entries already on disk from before
-    this merge).
+    into the new ledger entry, and ALWAYS writes `MERGE_LEDGER_SCHEMA_V4`
+    (#667: v4 adds `carried_content_ids`, computed HERE from
+    `existing_entries` vs `survivor_text`; the reader still accepts v1-v3
+    entries already on disk from before this merge).
 
     `existing_entries` (durable-derived-state slice 1a) is the survivor's
     CURRENT sidecar content, read by the caller via `bundle.ledger.
@@ -148,9 +164,25 @@ def plan_merge(
         survivor_id,
     )
 
+    # #667: a prior absorbed id whose delimited section is ABSENT from the
+    # survivor text being snapshotted has been woven in undelimited (a #645
+    # reconciliation, or an operator edit -- either way the #602 structural
+    # excision cannot reach it inside THIS entry's `survivor_before`).
+    # Recorded at snapshot time, when the fact is still decidable.
+    # #685: a prior whose absorbed BODY was empty/whitespace-only
+    # contributed no content, so its missing delimiter must not annotate --
+    # the annotation would buy no privacy and cost a wholesale
+    # `survivor_before` redaction on a later forget.
+    carried_content_ids = [
+        prior.absorbed_id
+        for prior in existing
+        if okf.merged_content_heading(prior.absorbed_id) not in survivor_text
+        and _absorbed_snapshot_has_content(prior.absorbed_snapshot)
+    ]
+
     sensitivity_before = survivor_metadata.get("sensitivity")
     entry = okf.MergeLedgerEntry(
-        schema=okf.MERGE_LEDGER_SCHEMA_V3,
+        schema=okf.MERGE_LEDGER_SCHEMA_V4,
         merged_at=merged_at,
         absorbed_id=absorbed_id,
         absorbed_snapshot=absorbed_text,
@@ -168,6 +200,7 @@ def plan_merge(
         provenance_rewrites=list(provenance_rewrites)
         if provenance_rewrites is not None
         else [],
+        carried_content_ids=carried_content_ids,
     )
 
     merged_survivor = okf.dump_frontmatter(merged_metadata, merged_body)
@@ -242,6 +275,18 @@ def plan_unmerge(
             f"one command, or unmerge each pair in that order"
         )
 
+    # #667: forget's sweep replaces a carried-content snapshot with the
+    # redaction sentinel (privacy over reversibility). Restoring it would
+    # overwrite the LIVE survivor body with the notice string -- refuse
+    # loudly instead; this merge is no longer reversible by design.
+    if tail.survivor_before == okf.REDACTED_SNAPSHOT_SENTINEL:
+        raise ValueError(
+            f"{survivor_id!r}'s ledger entry for {absorbed_id!r} was redacted "
+            "by `openkos forget` (it carried a forgotten concept's "
+            "reconciled content); this merge is no longer reversible -- "
+            "unmerge refused"
+        )
+
     return UnmergePlan(
         restored_survivor=tail.survivor_before,
         restored_absorbed=tail.absorbed_snapshot,

{openkos-0.2.4 → openkos-0.2.6}/src/openkos/bundle/provenance.py

@@ -370,6 +370,170 @@ def _source_levels(files: Mapping[str, str]) -> dict[str, str]:
     return source_levels
 
 
+def _levels_by_id(files: Mapping[str, str]) -> dict[str, object]:
+    """Parse every file once into `id -> raw sensitivity value`, keeping the
+    value RAW (`object`, not `str`).
+
+    Ranking is unaffected by the choice: `okf.combine_sensitivity` ranks
+    through the private fail-closed `okf._rank` (ADR-0003), where a missing
+    value floors at `private` and a dirty non-string floors at `confidential`
+    -- and an unrecognized STRING floors at `confidential` too, so `42` and
+    `"42"` rank identically. This docstring used to claim the opposite (that
+    coercing to `str`, as `_source_levels` must to satisfy
+    `resolve_source_raises`'s `level: str` signature, "would turn a dirty
+    `int` into a string that no longer fails closed the same way"); measured
+    against every value YAML frontmatter can produce -- `int`, `bool`, `list`,
+    `dict`, `float`, `date`, blank and unrecognized strings -- that claim is
+    FALSE, and `_source_levels`'s own docstring already said so correctly
+    (issue #736). Nothing in this module's behavior rested on it.
+
+    WHAT RAW IS ACTUALLY LOAD-BEARING FOR is reporting, not ranking:
+    `resolve_cited_high_water_raises` hands this value straight to
+    `okf.DescendantRaise.current`, which a preview shows the operator as what
+    the file HOLDS. Coercing would print `'42'` where the frontmatter holds
+    `42` -- a preview claiming a string the operator never wrote. Of everything
+    stated here, that reporting property is the ONLY one a `str` coercion
+    breaks, and it is what
+    `test_a_present_but_dirty_sensitivity_fails_closed_and_propagates` pins.
+
+    A file whose frontmatter fails to parse is SKIPPED, mirroring
+    `_parse_provenance_by_id`.
+    """
+    levels: dict[str, object] = {}
+    for path, text in files.items():
+        metadata: dict[str, object] | None
+        try:
+            metadata, _ = okf.load_frontmatter(text)
+        except Exception:  # broad: malformed frontmatter is skipped rather
+            # than surfaced, mirroring `_parse_provenance_by_id`
+            metadata = None
+        if metadata is None:
+            continue
+        levels[_normalize_id(path)] = metadata.get("sensitivity")
+    return levels
+
+
+def resolve_cited_high_water_raises(
+    files: Mapping[str, str],
+) -> list[okf.DescendantRaise]:
+    """Pure fixpoint maintaining ADR-0003's high-water mark for a concept
+    whose provenance spans MORE THAN ONE Source (issue #697).
+
+    ADR-0012 deferred exactly this case, and named the deferral: a concept in
+    no single Source's closure "stays reported, not resolved, until a human
+    acts". `resolve_source_raises` cannot reach it because
+    `provenance_closure`'s conservative NON-EMPTY SUBSET rule keeps out any
+    concept still citing a Source outside the root's closure -- correct for
+    propagating ONE Source's level down its exclusive descendants, and exactly
+    wrong for a high-water mark, where citing a confidential source and a
+    public one means confidential. The subset rule under-classifies here.
+
+    THE RULE MIRRORS BIRTH, not the closure walk. `cli/main.py`'s
+    `_stage_filed_answer` folds `okf.combine_sensitivity` over each CITED
+    concept's own level to compute a filing's sensitivity; this recomputes the
+    same fold over the same direct `provenance` entries, so an object's
+    maintained level is the one its own birth rule would produce today. Reading
+    Source ANCESTORS instead (`provenance_source_ancestors`, #628) would miss a
+    cited intermediate concept raised by hand without its Source.
+
+    WHY A FIXPOINT rather than one pass: raising `mid` changes the input for
+    everything citing `mid`. A single pass would repair one level and leave the
+    same gap one level up. Termination is the same argument
+    `provenance_closure` makes -- `combine_sensitivity` is monotone and
+    `okf.SENSITIVITY_ORDER` is finite, so levels only ever rise and the loop
+    halts, including on a provenance cycle (two concepts citing each other
+    converge on their shared maximum) and on self-referential provenance.
+
+    THE RE-SCAN IS NOT WHERE THE TIME GOES, so do not "fix" it first. The walk
+    order is arbitrary, and on a linear chain inserted in reverse the loop
+    needs one pass per link -- genuinely quadratic. Measured once, on synthetic
+    bundles, 2026-08-16 (issue #736); the ratios are what to trust, not the
+    millisecond figures, which are one machine on one day. A real bundle shape,
+    where every insight cites Sources DIRECTLY as `query --save` files them,
+    converged in TWO passes and spent 2.3% of the call in this loop at 1k docs
+    and at 10k alike; the other ~98% was the three whole-bundle frontmatter
+    parses above. Even the adversarial 400-deep reverse chain -- a shape
+    nothing in this product builds -- cost 36ms.
+    Reordering the walk topologically was therefore REJECTED: it optimizes 2%
+    of a maintenance verb an operator runs explicitly. If this call ever does
+    need to be faster, collapse the three parses into one.
+
+    A concept is staged only when EVERY cited id resolves to a file in `files`.
+    A dangling citation leaves it unstaged, matching `lint`'s own
+    `multi-source-uncovered` rule and ADR-0012 design D8: that signal belongs
+    to `check_dangling_provenance`. The alternative -- folding an unresolvable
+    citation to `confidential`, as `_stage_filed_answer` does at birth -- is
+    deliberately NOT copied here. At birth it guards ONE document the operator
+    is creating; in a bundle-wide sweep it would raise every descendant of one
+    dangling ref, a blast radius no preview makes safe, and it would contradict
+    the `lint` finding that sent the operator to this verb.
+
+    A `type: Source` is NEVER staged: a Source's level is operator-set truth,
+    never derived from what it cites. In practice a Source cites its raw
+    `resource`, which never resolves to a bundle id and would leave it unstaged
+    anyway; the explicit type guard states the invariant rather than resting on
+    that coincidence.
+
+    Raise-only by construction (`combine_sensitivity` never lowers), so a
+    concept deliberately classified ABOVE everything it cites keeps its level.
+    The returned list is `sorted()` by `concept_id`, each entry's `content` a
+    full `okf.dump_frontmatter` re-render of the ORIGINAL metadata with only
+    `sensitivity` replaced.
+    """
+    provenance_by_id = _parse_provenance_by_id(files)
+    levels = _levels_by_id(files)
+    source_ids = set(_source_levels(files))
+
+    # Only concepts whose every citation resolves are eligible; the ineligible
+    # are excluded once, before the loop, so the fixpoint cannot admit one
+    # through a later iteration.
+    eligible = {
+        concept_id: parents
+        for concept_id, parents in provenance_by_id.items()
+        if parents
+        and concept_id not in source_ids
+        and all(parent in levels for parent in parents)
+    }
+
+    resolved = dict(levels)
+    changed = True
+    while changed:
+        changed = False
+        for concept_id, parents in eligible.items():
+            current = resolved[concept_id]
+            combined = current
+            for parent in parents:
+                combined = okf.combine_sensitivity(combined, resolved[parent])
+            if combined != current:
+                resolved[concept_id] = combined
+                changed = True
+
+    raises: list[okf.DescendantRaise] = []
+    for concept_id in sorted(eligible):
+        current = levels[concept_id]
+        # Re-combined rather than read straight out of `resolved`, for a typed
+        # `str`: the fold seeds from RAW frontmatter values (`object`), which
+        # is what keeps `okf._rank`'s fail-closed ranking intact. The fold only
+        # ever raises, so this is the canonical form of `resolved[concept_id]`.
+        # The comparison stays against the RAW `current`, matching
+        # `resolve_source_raises`: a document with no `sensitivity` key at all
+        # differs from any canonical level and is written explicitly.
+        new_level = okf.combine_sensitivity(current, resolved[concept_id])
+        if new_level == current:
+            continue
+        metadata, body = okf.load_frontmatter(files[f"{concept_id}.md"])
+        metadata["sensitivity"] = new_level
+        raises.append(
+            okf.DescendantRaise(
+                concept_id=concept_id,
+                current=current,
+                new_level=new_level,
+                content=okf.dump_frontmatter(metadata, body),
+            )
+        )
+    return raises
+
+
 def resolve_backfill_raises(files: Mapping[str, str]) -> list[okf.DescendantRaise]:
     """Pure, bundle-wide sweep core `backfill-sensitivity` needs (design
     D4/D5/D6) -- everything the verb requires that is NOT the Typer command:
@@ -418,15 +582,27 @@ def resolve_backfill_raises(files: Mapping[str, str]) -> list[okf.DescendantRais
 
     best: dict[str, okf.DescendantRaise] = {}
     best_rank: dict[str, int] = {}
+
+    def _offer(descendant_raise: okf.DescendantRaise) -> None:
+        concept_id = descendant_raise.concept_id
+        new_rank = okf.SENSITIVITY_ORDER.index(descendant_raise.new_level)
+        if concept_id not in best or new_rank > best_rank[concept_id]:
+            best[concept_id] = descendant_raise
+            best_rank[concept_id] = new_rank
+
     for source_id in sorted(source_levels):
         for descendant_raise in resolve_source_raises(
             files, source_id=source_id, level=source_levels[source_id]
         ):
-            concept_id = descendant_raise.concept_id
-            new_rank = okf.SENSITIVITY_ORDER.index(descendant_raise.new_level)
-            if concept_id not in best or new_rank > best_rank[concept_id]:
-                best[concept_id] = descendant_raise
-                best_rank[concept_id] = new_rank
+            _offer(descendant_raise)
+
+    # Second producer (issue #697), merged by the SAME max rule and offered
+    # LAST so a strictly higher cited high-water mark wins over a per-Source
+    # raise, while an equal one leaves the closure sweep's record in place --
+    # the existing "ties resolve to the first" contract, extended by one
+    # producer rather than reinterpreted.
+    for descendant_raise in resolve_cited_high_water_raises(files):
+        _offer(descendant_raise)
 
     return [best[concept_id] for concept_id in sorted(best)]