clm-kernel 0.0.16__tar.gz → 0.0.17__tar.gz

{clm_kernel-0.0.16 → clm_kernel-0.0.17}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: clm-kernel
-Version: 0.0.16
+Version: 0.0.17
 Summary: Card Lifecycle Manager (CLM) Kernel — Sovereign Functional Agentic Operating System
 Author: GovTech Digital Identity & Autonomous Systems
 License: MIT

{clm_kernel-0.0.16 → clm_kernel-0.0.17}/README.md

@@ -46,6 +46,34 @@
 
 ---
 
+## ♻️ Card Lifecycle & Durability Guarantees
+
+Every tape enforces the same three-table Triad and the same retirement contract, in every runtime.
+
+| Guarantee | Where it is enforced |
+|:---|:---|
+| A handle head always names a real card | `handle_registry.current_hash` foreign key + `NOT NULL` |
+| A handle cannot vanish while its history references it | `handle_history.handle` foreign key; `removeHandle` retires instead of deleting |
+| Retirement is a name-plane event, never a deletion | `retire()` appends a `TOMBSTONE` record; content stays readable by hash |
+| A tape that cannot enforce its constraints is refused | schema shape checked at open against `PRAGMA foreign_key_list` |
+| The schema declarations cannot drift silently | `npm run check:ddl` — one signed digest over all nine declarations |
+
+```python
+from clm.layer2 import MCardFileSystem
+
+fs = MCardFileSystem()
+fs.write_file("workspace/report.md", "quarterly figures", "did:key:z6MkuAuthor")
+fs.retire("workspace/report.md", reason="superseded")
+
+assert not fs.exists("workspace/report.md")   # the name is retired
+# The content card and the handle's history stay on the tape, addressed by hash.
+```
+
+> **Upgrading from 0.0.16:** a tape created by an earlier TypeScript kernel has no foreign keys on
+> `handle_registry`/`handle_history` and will now be refused at open with a message naming each missing
+> constraint. Rebuild the tape, or pass `allowLegacySchema: true` to `NodeSqliteBackend` for read-only
+> hydration. `retire()`/`deleteFile()` also bind the head to a tombstone record instead of `NULL`.
+
 ## 🛠️ Installation & Setup
 
 Before running the `clm` CLI or switching runtimes, install and activate the environment. Python execution throughout the repository is hermetically managed via **[`uv`](https://docs.astral.sh/uv/)**.
@@ -110,9 +138,9 @@ npm install clm-kernel          # Node.js / TypeScript (npm)
 cargo install clm-kernel        # Rust (crates.io)
 ```
 
-> **Cross-Platform Verification (Windows, Linux, macOS)**: Windows, Linux, and Mac have all proven to pass 100% of tests for CLM's `make test`, `make test-parallel-fast`, and all 36 boundary gates and 27 layer-by-stage verification suites. On Windows, execution is fully supported natively (PowerShell / Git Bash) and via WSL without POSIX-specific failures. Python toolchains are unified through `uv run python` and `uv run pytest`, eliminating path space issues. Native PyO3 builds automatically produce `kernel/clm_py_core/clm_rust_core.pyd` alongside `clm_rust_core.exe`. See [`docs/guides/windows-setup-and-practices.md`](docs/guides/windows-setup-and-practices.md) for automated bootstrap scripts, path normalization, and MSVC settings.
+> **Cross-Platform Verification (Windows, Linux, macOS — `x86_64` & `aarch64`)**: Windows, Linux (`x86_64` and `aarch64`), and macOS have all proven to pass 100% of tests for CLM's `make test`, `make test-parallel-fast`, and all 36 boundary gates and 27 layer-by-stage verification suites. On Windows, execution is fully supported natively (PowerShell / Git Bash) and via WSL without POSIX-specific failures. Python toolchains are unified through `uv run python` and `uv run pytest`, eliminating path space issues. Native PyO3 builds automatically produce `kernel/clm_py_core/clm_rust_core.pyd` alongside `clm_rust_core.exe`, and C-ABI FFI buffers in [`kernel/clm_rust_core/src/c_abi.rs`](kernel/clm_rust_core/src/c_abi.rs) are typed against `std::os::raw::c_char` so `cargo test` and Gate 34 compile cleanly on both signed-`char` (`x86_64`) and unsigned-`char` (`aarch64-unknown-linux-gnu`) targets. See [`docs/guides/windows-setup-and-practices.md`](docs/guides/windows-setup-and-practices.md) for automated bootstrap scripts, path normalization, and MSVC settings.
 >
-> Windows-specific hardening (see [`CHANGELOG.md`](CHANGELOG.md), `0.0.11`–`0.0.15`):
+> Platform-specific hardening (see [`CHANGELOG.md`](CHANGELOG.md), `0.0.11`–`0.0.17`):
 > - Test subprocesses decode output as UTF-8, so the default cp1252 console codec no longer crashes them.
 > - Latency benchmarks warm up first and use `TCP_NODELAY`.
 > - `msvcrt` lock contention and read-only SQLite `-wal`/`-shm` files are handled.

{clm_kernel-0.0.16 → clm_kernel-0.0.17}/kernel/clm_py_core/clm/__init__.py

@@ -237,4 +237,4 @@ __all__ = [
     "MagicSignature",
 ]
 
-__version__ = "0.0.16"
+__version__ = "0.0.17"

{clm_kernel-0.0.16 → clm_kernel-0.0.17}/kernel/clm_py_core/clm/layer2/mcard_fs.py

@@ -12,10 +12,11 @@ Enforces:
 from __future__ import annotations
 
 import datetime
+import itertools
 import json
 import re
 import sqlite3
-from typing import Any
+from typing import Any, Callable
 
 from pydantic import BaseModel, ConfigDict
 
@@ -26,6 +27,28 @@ from clm.layer0.schema.ddl import CANONICAL_TRIAD_DDL
 
 _ENVELOPE_HASH_RE = re.compile(rb'"hash"\s*:\s*"([^"\\]+)"')
 
+_TOMBSTONE_KIND = "TOMBSTONE"
+_SAVEPOINT_IDS = itertools.count()
+
+
+def _tombstone_payload(canonical_uri: str, previous_hash: str, author: str, reason: str | None, retired_at: str) -> bytes:
+    """Canonical bytes for the record that retires a handle.
+
+    Retirement is a name-plane event, not a deletion: the record is appended to the tape and the
+    handle head is bound to it, so `current_hash` always names a real card and the history survives.
+    """
+    payload: dict[str, Any] = {
+        "kind": _TOMBSTONE_KIND,
+        "handle": canonical_uri,
+        "from": previous_hash,
+        "to": None,
+        "author": author,
+        "retiredAt": retired_at,
+    }
+    if reason is not None:
+        payload["reason"] = reason
+    return json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
+
 
 class MCardStat(BaseModel):
     """Metadata stat for a virtual MCard file."""
@@ -104,6 +127,51 @@ class MCardFileSystem:
             return f"mcard:{clean}"
         return f"mcard:{self.context_prefix}/{clean}"
 
+    def _atomic(self, action: Callable[[], Any]) -> Any:
+        """Runs `action` atomically: a savepoint inside the caller's transaction, else its own.
+
+        Entering `with self.conn` while the caller already holds a transaction commits the caller's
+        work, and skipping the boundary entirely leaves a failed write half-applied. Neither is
+        acceptable, so a nested savepoint is used instead.
+        """
+        if not self.conn.in_transaction:
+            with self.conn:
+                return action()
+        name = f"mcard_fs_{next(_SAVEPOINT_IDS)}"
+        self.conn.execute(f"SAVEPOINT {name}")
+        try:
+            result = action()
+        except BaseException:
+            self.conn.execute(f"ROLLBACK TO SAVEPOINT {name}")
+            self.conn.execute(f"RELEASE SAVEPOINT {name}")
+            raise
+        self.conn.execute(f"RELEASE SAVEPOINT {name}")
+        return result
+
+    def _head_is_tombstone(self, hash_str: str) -> bool:
+        """True when a handle head names a retirement record rather than live content."""
+        row = self.conn.execute("SELECT content FROM card WHERE hash = ?", (hash_str,)).fetchone()
+        if not row or row[0] is None:
+            return False
+        raw = row[0].encode("utf-8") if isinstance(row[0], str) else bytes(row[0])
+        if len(raw) > 4096:
+            return False
+        try:
+            payload = json.loads(raw)
+        except (json.JSONDecodeError, UnicodeDecodeError, ValueError):
+            return False
+        return isinstance(payload, dict) and payload.get("kind") == _TOMBSTONE_KIND
+
+    def _qualified_ref(self, hash_str: str) -> str:
+        """Names a stored row the way a transition record does: `<algorithm>:<hex>`."""
+        row = self.conn.execute("SELECT content, g_time FROM card WHERE hash = ?", (hash_str,)).fetchone()
+        if not row:
+            return f"blake3:{hash_str}"
+        content = row[0].encode("utf-8") if isinstance(row[0], str) else bytes(row[0])
+        verdict = verify_record(MCardRecord(hash=hash_str, content=content, g_time=row[1]))
+        algorithm = verdict.algorithm.value if verdict.algorithm is not None else "blake3"
+        return f"{algorithm}:{hash_str}"
+
     def _init_schema(self) -> None:
         has_idx = self.conn.execute(
             "SELECT 1 FROM sqlite_master WHERE type='index' AND name='idx_handle_history_lookup'"
@@ -151,10 +219,7 @@ class MCardFileSystem:
             )
             return mcard
 
-        if self.conn.in_transaction:
-            return _do_write()
-        with self.conn:
-            return _do_write()
+        return self._atomic(_do_write)
 
     def read_file(self, uri: str) -> MCard:
         """Reads and cryptographically verifies an MCard by handle URI."""
@@ -166,6 +231,8 @@ class MCardFileSystem:
             raise KeyError(f"Handle '{canonical_uri}' not found in MCardFs")
 
         current_hash = row[0]
+        if self._head_is_tombstone(current_hash):
+            raise KeyError(f"Handle '{canonical_uri}' is retired")
         card_row = self.conn.execute("SELECT content, g_time FROM card WHERE hash = ?", (current_hash,)).fetchone()
         if not card_row:
             raise ValueError(f"Card content for hash '{current_hash}' corrupted or missing")
@@ -178,9 +245,12 @@ class MCardFileSystem:
         return MCard(**json.loads(raw_bytes))
 
     def exists(self, uri: str) -> bool:
-        """Checks if a handle exists in the filesystem."""
+        """Checks if a handle is bound to live content (a retired handle does not exist)."""
         canonical_uri = self.canonicalize_uri(uri)
-        return self.conn.execute("SELECT 1 FROM handle_registry WHERE handle = ? LIMIT 1", (canonical_uri,)).fetchone() is not None
+        row = self.conn.execute("SELECT current_hash FROM handle_registry WHERE handle = ? LIMIT 1", (canonical_uri,)).fetchone()
+        if row is None:
+            return False
+        return not self._head_is_tombstone(row[0])
 
     def stat_file(self, uri: str) -> MCardStat:
         """Returns stat metadata for a handle URI without transferring multi-megabyte card BLOBs."""
@@ -192,6 +262,8 @@ class MCardFileSystem:
             raise KeyError(f"Handle '{canonical_uri}' not found")
 
         current_hash, updated_at = row
+        if self._head_is_tombstone(current_hash):
+            raise KeyError(f"Handle '{canonical_uri}' is retired")
         card_row = self.conn.execute(
             "SELECT length(content), CASE WHEN length(content) <= 4096 THEN content ELSE substr(content, 1, 2048) END "
             "FROM card WHERE hash = ?",
@@ -210,22 +282,51 @@ class MCardFileSystem:
             pattern = "mcard:%"
         else:
             pattern = f"{canonical_prefix}/%"
-        cur = self.conn.execute("SELECT handle FROM handle_registry WHERE handle LIKE ? ORDER BY handle ASC", (pattern,))
-        return [r[0] for r in cur.fetchall()]
+        cur = self.conn.execute(
+            "SELECT handle, current_hash FROM handle_registry WHERE handle LIKE ? ORDER BY handle ASC", (pattern,)
+        )
+        return [handle for handle, head in cur.fetchall() if not self._head_is_tombstone(head)]
 
-    def delete_file(self, uri: str) -> None:
-        """Deregisters a handle URI while preserving the CAS content blob."""
+    def retire(self, uri: str, reason: str | None = None, author: AgentDid | str = "did:clm:agent:default") -> None:
+        """Retires a handle by appending a TOMBSTONE record and binding the head to it.
+
+        Retirement is a name-plane event: the content card stays readable by hash, the handle row
+        and its history stay on the tape, and `current_hash` names the record that retired it.
+        """
         canonical_uri = self.canonicalize_uri(uri)
+        author_str = author if isinstance(author, str) else str(author)
         now = datetime.datetime.now(datetime.timezone.utc).isoformat()
-        with self.conn:
-            row = self.conn.execute("SELECT current_hash FROM handle_registry WHERE handle = ?", (canonical_uri,)).fetchone()
+
+        def _do_retire() -> None:
+            row = self.conn.execute(
+                "SELECT current_hash FROM handle_registry WHERE handle = ?", (canonical_uri,)
+            ).fetchone()
             if not row:
-                raise KeyError(f"Handle '{canonical_uri}' not found for deletion")
+                raise KeyError(f"Handle '{canonical_uri}' not found for retirement")
+            previous_hash = row[0]
+            qualified_from = self._qualified_ref(previous_hash)
+            record = create_record(_tombstone_payload(canonical_uri, qualified_from, author_str, reason, now))
+            raw_hash = record.hash.split(":", 1)[1] if ":" in record.hash else record.hash
+            self.conn.execute(
+                "INSERT OR IGNORE INTO card (hash, content, g_time) VALUES (?, ?, ?)",
+                (raw_hash, record.content, record.g_time),
+            )
             self.conn.execute(
                 "INSERT INTO handle_history (handle, previous_hash, changed_at) VALUES (?, ?, ?)",
-                (canonical_uri, row[0], now),
+                (canonical_uri, previous_hash, now),
+            )
+            # UPDATE, not INSERT OR REPLACE: replacing the row would delete it, and the history row
+            # just written references it.
+            self.conn.execute(
+                "UPDATE handle_registry SET current_hash = ?, updated_at = ? WHERE handle = ?",
+                (raw_hash, now, canonical_uri),
             )
-            self.conn.execute("DELETE FROM handle_registry WHERE handle = ?", (canonical_uri,))
+
+        self._atomic(_do_retire)
+
+    def delete_file(self, uri: str) -> None:
+        """Retires a handle, preserving the CAS content blob and its history."""
+        self.retire(uri)
 
     def write_raw_bytes(self, uri: str | None, content: bytes | str, algorithm: str = "blake3") -> MCardRecord:
         """Writes raw content bytes into `card` and binds `uri` in `handle_registry` when provided (INV-510-01, INV-510-02)."""

{clm_kernel-0.0.16 → clm_kernel-0.0.17}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "clm-kernel"
-version = "0.0.16"
+version = "0.0.17"
 description = "Card Lifecycle Manager (CLM) Kernel — Sovereign Functional Agentic Operating System"
 readme = "kernel/clm_py_core/README.md"
 requires-python = ">=3.11"