clm-kernel 0.0.16__tar.gz → 0.0.18__tar.gz

{clm_kernel-0.0.16 → clm_kernel-0.0.18}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: clm-kernel
-Version: 0.0.16
+Version: 0.0.18
 Summary: Card Lifecycle Manager (CLM) Kernel — Sovereign Functional Agentic Operating System
 Author: GovTech Digital Identity & Autonomous Systems
 License: MIT

{clm_kernel-0.0.16 → clm_kernel-0.0.18}/README.md

@@ -10,7 +10,7 @@
 
 > **The Cubical Logic Model (CLM)** is a polyglot computational substrate and conversational operating system implementing the Kenotic Layer 0–5 strata. Using **Cordis Calculus** (spatiotemporal composability & reversible `Fiber` effects), **Type Lattice Theory** (stratified universes, Bounded Place Type Lattice $\mathcal{L}_N$, & MISU Meet gates), and **Petri Net Algebra** (All-`MCard` Triad, linear incidence conservation $x^\top D = 0^\top$, cyclic T-invariants $D y = 0$, Siphons & Traps) as foundational primitives, CLM articulates **Storage Resource**, **Data Transport Activities**, **Learning State Changes**, and **Security Perimeter** across Python, Rust, and TypeScript runtimes. Cross-runtime equivalence is established per declared feature and shared test vector; it is not a blanket claim that every API or behavior is identical.
 
-> **MCard sync status (`EPIC-KX` / `SyncContract v1`):** the published package line is `0.0.14`; the workspace line is `0.0.15`. Under `EPIC-KX` (`KX-01`..`KX-07`), CLM implements the additive 4-module **`SyncContract v1`** (`types`, `codec`, `petri`, `service`) across JavaScript/TypeScript (`kernel/clm_js_core/src/layer2/sync/`, exported at `clm-kernel/sync`), Python (`kernel/clm_py_core/clm/layer2/sync/`), and Rust (`kernel/clm_rust_core/src/layer2/sync/`), verified against shared golden fixtures (`tests/fixtures/sync-contract-v1/`), Petri P-invariant checks (`tools/guards/check_sync_net_invariants.py`), and 14 cross-runtime parity invariants (`tools/guards/check_kx_cross_runtime.py`) while preserving 100% of existing `0.0.10`/`0.0.11`/`0.0.12`/`0.0.13`/`0.0.14` public APIs and export subpaths. Downstream npm consumers (`mcard-studio`, `LandingPage`) are exact-pinned to published `0.0.14` across npm, PyPI, and crates.io.
+> **MCard sync status (`EPIC-KX` / `SyncContract v1`):** the published package line is `0.0.17`; the workspace line is `0.0.18`. Under `EPIC-KX` (`KX-01`..`KX-07`), CLM implements the additive 4-module **`SyncContract v1`** (`types`, `codec`, `petri`, `service`) across JavaScript/TypeScript (`kernel/clm_js_core/src/layer2/sync/`, exported at `clm-kernel/sync`), Python (`kernel/clm_py_core/clm/layer2/sync/`), and Rust (`kernel/clm_rust_core/src/layer2/sync/`), verified against shared golden fixtures (`tests/fixtures/sync-contract-v1/`), Petri P-invariant checks (`tools/guards/check_sync_net_invariants.py`), and 14 cross-runtime parity invariants (`tools/guards/check_kx_cross_runtime.py`) while preserving 100% of existing `0.0.10`–`0.0.17` public APIs and export subpaths. Downstream npm consumers (`mcard-studio`, `LandingPage`) are exact-pinned to published `clm-kernel` across npm, PyPI, and crates.io.
 >
 > **Active sprint program (2026-10-29):** **Sprints 566..571 (`INV-566`..`INV-571`, `24/24` DoDs verified, `Revision 4`)** were implemented, verified, and **graduated on `2026-10-30`** into [`02_petri-net/`](docs/sprints/02_petri-net/README.md) (`567`), [`04_storage-network/`](docs/sprints/04_storage-network/README.md) (`571`), and [`05_runtime-interfaces/`](docs/sprints/05_runtime-interfaces/README.md) (`566`, `568`, `569`, `570`, and the master runbook); [`docs/sprints/_active/`](docs/sprints/_active/README.md) now holds no open suite targeting `0.0.15` correctness defects, browser sovereignty, and capability extensions across three waves: pure FIPS 180-4 browser sync SHA-256 fallback, `ContentHash` `mcard:` prefix parsing, tri-runtime `parseMcardUri` hash classification (`Gate 30`), and `clm-kernel/browser` URI exports (`INV-566`); LIFO disposable invocation on `PetriProcessPCard` across TypeScript and Python, `executeAsync()`, `PtrCordisEngine` savepoint commit plus direct `FiberLifecycle` return, and `ProcessFibration` `disposeSync` and `disposeFiber` teardown (`INV-567`); ISO 14496-12 and JPEG 2000 offset-12 `ftyp` brand inspection across TS, Python, and Rust (`Gate 23`), `subDialect` and `truncatedSignature` diagnostics, and immutable `TypeInterpreter.extend(defs)` (`INV-568`); zero-Node `clm-kernel/browser/process` subpath (`INV-569`); typed `checkCoeffects` SPI, `CodecRegistryService`, `SatoriSurfaceEnvelope` projector, and `useSyncExternalStore` membrane store in `clm-kernel/membrane` (`INV-570`); and deterministic JSON-safe `ColoredPetriVM.toProjection()`, stratum-local `TypedValue` codecs at `U0`, `U1`, and `U4`, and bounded byte-range framing (`INV-571`). All six landed on `2026-10-29`: `566` (`fd92498e`), `567` (`e5d63966`), `570` (`660ef635`), `568` (`e6a28f75`), `571` (`3aff6851`), and `569` (`f4beb908`), verified by `make test` (`36/36` invariants, `27/27` stages, `382` tests) and `make check-boundaries` (`36/36` gates). **Sprints 557..561 (`15/15` DoDs)** and **Sprints 562..565 (`12/12` DoDs)** shipped in `clm-kernel@0.0.14` and **graduated on `2026-10-29`** across [`04_storage-network/`](docs/sprints/04_storage-network/README.md) and [`07_test-infrastructure/`](docs/sprints/07_test-infrastructure/README.md).
 
@@ -46,6 +46,34 @@
 
 ---
 
+## ♻️ Card Lifecycle & Durability Guarantees
+
+Every tape enforces the same three-table Triad and the same retirement contract, in every runtime.
+
+| Guarantee | Where it is enforced |
+|:---|:---|
+| A handle head always names a real card | `handle_registry.current_hash` foreign key + `NOT NULL` |
+| A handle cannot vanish while its history references it | `handle_history.handle` foreign key; `removeHandle` retires instead of deleting |
+| Retirement is a name-plane event, never a deletion | `retire()` appends a `TOMBSTONE` record; content stays readable by hash |
+| A tape that cannot enforce its constraints is refused | schema shape checked at open against `PRAGMA foreign_key_list` |
+| The schema declarations cannot drift silently | `npm run check:ddl` — one signed digest over all nine declarations |
+
+```python
+from clm.layer2 import MCardFileSystem
+
+fs = MCardFileSystem()
+fs.write_file("workspace/report.md", "quarterly figures", "did:key:z6MkuAuthor")
+fs.retire("workspace/report.md", reason="superseded")
+
+assert not fs.exists("workspace/report.md")   # the name is retired
+# The content card and the handle's history stay on the tape, addressed by hash.
+```
+
+> **Upgrading from 0.0.16:** a tape created by an earlier TypeScript kernel has no foreign keys on
+> `handle_registry`/`handle_history` and will now be refused at open with a message naming each missing
+> constraint. Rebuild the tape, or pass `allowLegacySchema: true` to `NodeSqliteBackend` for read-only
+> hydration. `retire()`/`deleteFile()` also bind the head to a tombstone record instead of `NULL`.
+
 ## 🛠️ Installation & Setup
 
 Before running the `clm` CLI or switching runtimes, install and activate the environment. Python execution throughout the repository is hermetically managed via **[`uv`](https://docs.astral.sh/uv/)**.
@@ -110,9 +138,9 @@ npm install clm-kernel          # Node.js / TypeScript (npm)
 cargo install clm-kernel        # Rust (crates.io)
 ```
 
-> **Cross-Platform Verification (Windows, Linux, macOS)**: Windows, Linux, and Mac have all proven to pass 100% of tests for CLM's `make test`, `make test-parallel-fast`, and all 36 boundary gates and 27 layer-by-stage verification suites. On Windows, execution is fully supported natively (PowerShell / Git Bash) and via WSL without POSIX-specific failures. Python toolchains are unified through `uv run python` and `uv run pytest`, eliminating path space issues. Native PyO3 builds automatically produce `kernel/clm_py_core/clm_rust_core.pyd` alongside `clm_rust_core.exe`. See [`docs/guides/windows-setup-and-practices.md`](docs/guides/windows-setup-and-practices.md) for automated bootstrap scripts, path normalization, and MSVC settings.
+> **Cross-Platform Verification (Windows, Linux, macOS — `x86_64` & `aarch64`)**: Windows, Linux (`x86_64` and `aarch64`), and macOS have all proven to pass 100% of tests for CLM's `make test`, `make test-parallel-fast`, and all 36 boundary gates and 27 layer-by-stage verification suites. On Windows, execution is fully supported natively (PowerShell / Git Bash) and via WSL without POSIX-specific failures. Python toolchains are unified through `uv run python` and `uv run pytest`, eliminating path space issues. Native PyO3 builds automatically produce `kernel/clm_py_core/clm_rust_core.pyd` alongside `clm_rust_core.exe`, and C-ABI FFI buffers in [`kernel/clm_rust_core/src/c_abi.rs`](kernel/clm_rust_core/src/c_abi.rs) are typed against `std::os::raw::c_char` so `cargo test` and Gate 34 compile cleanly on both signed-`char` (`x86_64`) and unsigned-`char` (`aarch64-unknown-linux-gnu`) targets. See [`docs/guides/windows-setup-and-practices.md`](docs/guides/windows-setup-and-practices.md) for automated bootstrap scripts, path normalization, and MSVC settings.
 >
-> Windows-specific hardening (see [`CHANGELOG.md`](CHANGELOG.md), `0.0.11`–`0.0.15`):
+> Platform-specific hardening (see [`CHANGELOG.md`](CHANGELOG.md), `0.0.11`–`0.0.17`):
 > - Test subprocesses decode output as UTF-8, so the default cp1252 console codec no longer crashes them.
 > - Latency benchmarks warm up first and use `TCP_NODELAY`.
 > - `msvcrt` lock contention and read-only SQLite `-wal`/`-shm` files are handled.
@@ -252,13 +280,13 @@ CLM enforces strict 100% SemVer consensus across all three runtime package manif
 make release-check
 
 # 2. Synchronize and bump version deterministically across all runtimes
-make bump VERSION=0.0.15
+make bump VERSION=0.0.18
 
 # 3. Query public package registries (PyPI, npm, crates.io) for live publication status
-make verify-published VERSION=0.0.14
+make verify-published VERSION=0.0.17
 
 # 4. Atomically sync all in-tree manifests with published registry artifacts
-make sync-published VERSION=0.0.14
+make sync-published VERSION=0.0.17
 ```
 
 For extended guides, see [`docs/guides/polyglot-quick-start.md`](docs/guides/polyglot-quick-start.md) and [`docs/guides/web-ui-quick-start.md`](docs/guides/web-ui-quick-start.md).

{clm_kernel-0.0.16 → clm_kernel-0.0.18}/kernel/clm_py_core/clm/__init__.py

@@ -237,4 +237,4 @@ __all__ = [
     "MagicSignature",
 ]
 
-__version__ = "0.0.16"
+__version__ = "0.0.18"

{clm_kernel-0.0.16 → clm_kernel-0.0.18}/kernel/clm_py_core/clm/layer0/type_interpreter_magic.py

@@ -99,6 +99,10 @@ def _isobmff_mime(major: str, compatible: tuple[str, ...]) -> str:
 
 def inspect_container_brand(data: bytes) -> BrandMatch | None:
     """Resolve ISO BMFF and JPEG 2000 `ftyp` brands without mutating the input."""
+    # Raw JPEG 2000 codestream: SOC (FF4F) then SIZ (FF51), no JP2 signature or ftyp.
+    if len(data) >= 4 and data[0] == 0xFF and data[1] == 0x4F and data[2] == 0xFF and data[3] == 0x51:
+        return BrandMatch("image/j2k", "raw-codestream", None, ())
+
     if data.startswith(JP2_SIGNATURE):
         if len(data) < 24 or data[16:20] != b"ftyp":
             return BrandMatch("image/jpx")

{clm_kernel-0.0.16 → clm_kernel-0.0.18}/kernel/clm_py_core/clm/layer2/mcard_fs.py

@@ -12,10 +12,11 @@ Enforces:
 from __future__ import annotations
 
 import datetime
+import itertools
 import json
 import re
 import sqlite3
-from typing import Any
+from typing import Any, Callable
 
 from pydantic import BaseModel, ConfigDict
 
@@ -26,6 +27,28 @@ from clm.layer0.schema.ddl import CANONICAL_TRIAD_DDL
 
 _ENVELOPE_HASH_RE = re.compile(rb'"hash"\s*:\s*"([^"\\]+)"')
 
+_TOMBSTONE_KIND = "TOMBSTONE"
+_SAVEPOINT_IDS = itertools.count()
+
+
+def _tombstone_payload(canonical_uri: str, previous_hash: str, author: str, reason: str | None, retired_at: str) -> bytes:
+    """Canonical bytes for the record that retires a handle.
+
+    Retirement is a name-plane event, not a deletion: the record is appended to the tape and the
+    handle head is bound to it, so `current_hash` always names a real card and the history survives.
+    """
+    payload: dict[str, Any] = {
+        "kind": _TOMBSTONE_KIND,
+        "handle": canonical_uri,
+        "from": previous_hash,
+        "to": None,
+        "author": author,
+        "retiredAt": retired_at,
+    }
+    if reason is not None:
+        payload["reason"] = reason
+    return json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
+
 
 class MCardStat(BaseModel):
     """Metadata stat for a virtual MCard file."""
@@ -104,6 +127,51 @@ class MCardFileSystem:
             return f"mcard:{clean}"
         return f"mcard:{self.context_prefix}/{clean}"
 
+    def _atomic(self, action: Callable[[], Any]) -> Any:
+        """Runs `action` atomically: a savepoint inside the caller's transaction, else its own.
+
+        Entering `with self.conn` while the caller already holds a transaction commits the caller's
+        work, and skipping the boundary entirely leaves a failed write half-applied. Neither is
+        acceptable, so a nested savepoint is used instead.
+        """
+        if not self.conn.in_transaction:
+            with self.conn:
+                return action()
+        name = f"mcard_fs_{next(_SAVEPOINT_IDS)}"
+        self.conn.execute(f"SAVEPOINT {name}")
+        try:
+            result = action()
+        except BaseException:
+            self.conn.execute(f"ROLLBACK TO SAVEPOINT {name}")
+            self.conn.execute(f"RELEASE SAVEPOINT {name}")
+            raise
+        self.conn.execute(f"RELEASE SAVEPOINT {name}")
+        return result
+
+    def _head_is_tombstone(self, hash_str: str) -> bool:
+        """True when a handle head names a retirement record rather than live content."""
+        row = self.conn.execute("SELECT content FROM card WHERE hash = ?", (hash_str,)).fetchone()
+        if not row or row[0] is None:
+            return False
+        raw = row[0].encode("utf-8") if isinstance(row[0], str) else bytes(row[0])
+        if len(raw) > 4096:
+            return False
+        try:
+            payload = json.loads(raw)
+        except (json.JSONDecodeError, UnicodeDecodeError, ValueError):
+            return False
+        return isinstance(payload, dict) and payload.get("kind") == _TOMBSTONE_KIND
+
+    def _qualified_ref(self, hash_str: str) -> str:
+        """Names a stored row the way a transition record does: `<algorithm>:<hex>`."""
+        row = self.conn.execute("SELECT content, g_time FROM card WHERE hash = ?", (hash_str,)).fetchone()
+        if not row:
+            return f"blake3:{hash_str}"
+        content = row[0].encode("utf-8") if isinstance(row[0], str) else bytes(row[0])
+        verdict = verify_record(MCardRecord(hash=hash_str, content=content, g_time=row[1]))
+        algorithm = verdict.algorithm.value if verdict.algorithm is not None else "blake3"
+        return f"{algorithm}:{hash_str}"
+
     def _init_schema(self) -> None:
         has_idx = self.conn.execute(
             "SELECT 1 FROM sqlite_master WHERE type='index' AND name='idx_handle_history_lookup'"
@@ -151,10 +219,7 @@ class MCardFileSystem:
             )
             return mcard
 
-        if self.conn.in_transaction:
-            return _do_write()
-        with self.conn:
-            return _do_write()
+        return self._atomic(_do_write)
 
     def read_file(self, uri: str) -> MCard:
         """Reads and cryptographically verifies an MCard by handle URI."""
@@ -166,6 +231,8 @@ class MCardFileSystem:
             raise KeyError(f"Handle '{canonical_uri}' not found in MCardFs")
 
         current_hash = row[0]
+        if self._head_is_tombstone(current_hash):
+            raise KeyError(f"Handle '{canonical_uri}' is retired")
         card_row = self.conn.execute("SELECT content, g_time FROM card WHERE hash = ?", (current_hash,)).fetchone()
         if not card_row:
             raise ValueError(f"Card content for hash '{current_hash}' corrupted or missing")
@@ -178,9 +245,12 @@ class MCardFileSystem:
         return MCard(**json.loads(raw_bytes))
 
     def exists(self, uri: str) -> bool:
-        """Checks if a handle exists in the filesystem."""
+        """Checks if a handle is bound to live content (a retired handle does not exist)."""
         canonical_uri = self.canonicalize_uri(uri)
-        return self.conn.execute("SELECT 1 FROM handle_registry WHERE handle = ? LIMIT 1", (canonical_uri,)).fetchone() is not None
+        row = self.conn.execute("SELECT current_hash FROM handle_registry WHERE handle = ? LIMIT 1", (canonical_uri,)).fetchone()
+        if row is None:
+            return False
+        return not self._head_is_tombstone(row[0])
 
     def stat_file(self, uri: str) -> MCardStat:
         """Returns stat metadata for a handle URI without transferring multi-megabyte card BLOBs."""
@@ -192,6 +262,8 @@ class MCardFileSystem:
             raise KeyError(f"Handle '{canonical_uri}' not found")
 
         current_hash, updated_at = row
+        if self._head_is_tombstone(current_hash):
+            raise KeyError(f"Handle '{canonical_uri}' is retired")
         card_row = self.conn.execute(
             "SELECT length(content), CASE WHEN length(content) <= 4096 THEN content ELSE substr(content, 1, 2048) END "
             "FROM card WHERE hash = ?",
@@ -210,22 +282,51 @@ class MCardFileSystem:
             pattern = "mcard:%"
         else:
             pattern = f"{canonical_prefix}/%"
-        cur = self.conn.execute("SELECT handle FROM handle_registry WHERE handle LIKE ? ORDER BY handle ASC", (pattern,))
-        return [r[0] for r in cur.fetchall()]
+        cur = self.conn.execute(
+            "SELECT handle, current_hash FROM handle_registry WHERE handle LIKE ? ORDER BY handle ASC", (pattern,)
+        )
+        return [handle for handle, head in cur.fetchall() if not self._head_is_tombstone(head)]
 
-    def delete_file(self, uri: str) -> None:
-        """Deregisters a handle URI while preserving the CAS content blob."""
+    def retire(self, uri: str, reason: str | None = None, author: AgentDid | str = "did:clm:agent:default") -> None:
+        """Retires a handle by appending a TOMBSTONE record and binding the head to it.
+
+        Retirement is a name-plane event: the content card stays readable by hash, the handle row
+        and its history stay on the tape, and `current_hash` names the record that retired it.
+        """
         canonical_uri = self.canonicalize_uri(uri)
+        author_str = author if isinstance(author, str) else str(author)
         now = datetime.datetime.now(datetime.timezone.utc).isoformat()
-        with self.conn:
-            row = self.conn.execute("SELECT current_hash FROM handle_registry WHERE handle = ?", (canonical_uri,)).fetchone()
+
+        def _do_retire() -> None:
+            row = self.conn.execute(
+                "SELECT current_hash FROM handle_registry WHERE handle = ?", (canonical_uri,)
+            ).fetchone()
             if not row:
-                raise KeyError(f"Handle '{canonical_uri}' not found for deletion")
+                raise KeyError(f"Handle '{canonical_uri}' not found for retirement")
+            previous_hash = row[0]
+            qualified_from = self._qualified_ref(previous_hash)
+            record = create_record(_tombstone_payload(canonical_uri, qualified_from, author_str, reason, now))
+            raw_hash = record.hash.split(":", 1)[1] if ":" in record.hash else record.hash
+            self.conn.execute(
+                "INSERT OR IGNORE INTO card (hash, content, g_time) VALUES (?, ?, ?)",
+                (raw_hash, record.content, record.g_time),
+            )
             self.conn.execute(
                 "INSERT INTO handle_history (handle, previous_hash, changed_at) VALUES (?, ?, ?)",
-                (canonical_uri, row[0], now),
+                (canonical_uri, previous_hash, now),
+            )
+            # UPDATE, not INSERT OR REPLACE: replacing the row would delete it, and the history row
+            # just written references it.
+            self.conn.execute(
+                "UPDATE handle_registry SET current_hash = ?, updated_at = ? WHERE handle = ?",
+                (raw_hash, now, canonical_uri),
             )
-            self.conn.execute("DELETE FROM handle_registry WHERE handle = ?", (canonical_uri,))
+
+        self._atomic(_do_retire)
+
+    def delete_file(self, uri: str) -> None:
+        """Retires a handle, preserving the CAS content blob and its history."""
+        self.retire(uri)
 
     def write_raw_bytes(self, uri: str | None, content: bytes | str, algorithm: str = "blake3") -> MCardRecord:
         """Writes raw content bytes into `card` and binds `uri` in `handle_registry` when provided (INV-510-01, INV-510-02)."""

{clm_kernel-0.0.16 → clm_kernel-0.0.18}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "clm-kernel"
-version = "0.0.16"
+version = "0.0.18"
 description = "Card Lifecycle Manager (CLM) Kernel — Sovereign Functional Agentic Operating System"
 readme = "kernel/clm_py_core/README.md"
 requires-python = ">=3.11"