MetaList 0.4.0__tar.gz → 0.4.2__tar.gz

{metalist-0.4.0/metalist.egg-info → metalist-0.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: metalist
-Version: 0.4.0
+Version: 0.4.2
 Summary: A minimalist single-user note-taking app with SSR, fast in-memory tree operations, and efficient sync.
 Project-URL: Homepage, https://github.com/evolvingstuff/metalist
 Project-URL: Repository, https://github.com/evolvingstuff/metalist
@@ -27,6 +27,7 @@ Requires-Dist: starlette==1.3.1
 Requires-Dist: tqdm==4.70.0
 Requires-Dist: tree-sitter==0.25.2
 Requires-Dist: tree-sitter-javascript==0.25.0
+Requires-Dist: typing_extensions==4.16.0
 Requires-Dist: uvicorn==0.52.0
 Requires-Dist: zxcvbn==4.5.0
 Provides-Extra: dev
@@ -101,7 +102,11 @@ After the first installation, update and restart MetaList with one cross-platfor
 ```bash
 metalist update
 ```
-The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it stops running namespaces, hands off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock, installs the exact version reported by PyPI with a forced cache refresh, launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.4.0.`).
+The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it checks the installer prerequisites, stops running namespaces, and creates and verifies a new backup of every namespace before installation can begin. Each archive includes the complete notes/settings database, the attachments/sounds database when present, and any legacy search-history database. Locked namespaces are backed up with their encrypted data and key metadata intact, without requiring a password.
+
+Backups are saved to `~/MetaList/namespaces/<namespace>/backups/<namespace>-<timestamp>.metalist-backup.tar.gz`; the updater prints each verified path. Existing backups remain unchanged and are not pruned. If any backup fails, the update aborts with the current installation intact; run `metalist` to restart the stopped servers after resolving the failure.
+
+Only after all backups pass does the updater hand off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock. It installs the exact version reported by PyPI with a forced cache refresh, launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.4.2.`). This protection requires an installed version containing the backup safeguard and applies to `metalist update`; direct pip/uv install commands do not run it.
 
 For pip, users can run `pip install metalist`. For a non-editable local install from this checkout, use `uv pip install .` or `pip install .` instead of the editable command below.
 
@@ -114,6 +119,8 @@ npm install
 ```
 
 ### Run
+
+Persistent data defaults to `~/MetaList`. Set `METALIST_DATA_DIRECTORY` to an absolute directory path before launch to use a separate data root, including namespaces, backups, certificates, logs, and runtime jobs. This is used by release tests to keep their disposable data separate from real notes.
 The installed entrypoint starts or restarts every known namespace, prints their URLs, and exits:
 ```bash
 metalist
@@ -269,8 +276,9 @@ This repo now packages itself under the PyPI distribution name `metalist`. Curre
 
 Recommended release path:
 1. In the existing PyPI project `metalist`, configure GitHub Trusted Publishing for `evolvingstuff/metalist` and the workflow file `.github/workflows/publish-pypi.yml`.
-2. Push a tag such as `v0.3.5`.
-3. After the GitHub Actions workflow completes, users can run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
+2. Push the candidate commit to a branch and wait for the `Publish to PyPI` validation workflow to pass for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on Python 3.10 through 3.13. You do not need those operating systems locally. Archive verification also checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
+3. Only after every required check passes, create and push the release tag for that same commit. Any intervening change requires a new validation run before tagging. Branch and pull-request runs validate without publishing.
+4. The tag workflow repeats validation and publishes the same artifacts tested by its full matrix. Manual publication also requires all checks to pass. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
 
 If `--input` is omitted, a file picker opens (when `tkinter` is available).
 Notes tagged with `@implies` are converted into ontology rules and are not imported as notes. Legacy rules that are invalid under the current ontology grammar are reported and skipped while valid rules continue importing.

{metalist-0.4.0 → metalist-0.4.2}/README.md

@@ -65,7 +65,11 @@ After the first installation, update and restart MetaList with one cross-platfor
 ```bash
 metalist update
 ```
-The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it stops running namespaces, hands off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock, installs the exact version reported by PyPI with a forced cache refresh, launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.4.0.`).
+The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it checks the installer prerequisites, stops running namespaces, and creates and verifies a new backup of every namespace before installation can begin. Each archive includes the complete notes/settings database, the attachments/sounds database when present, and any legacy search-history database. Locked namespaces are backed up with their encrypted data and key metadata intact, without requiring a password.
+
+Backups are saved to `~/MetaList/namespaces/<namespace>/backups/<namespace>-<timestamp>.metalist-backup.tar.gz`; the updater prints each verified path. Existing backups remain unchanged and are not pruned. If any backup fails, the update aborts with the current installation intact; run `metalist` to restart the stopped servers after resolving the failure.
+
+Only after all backups pass does the updater hand off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock. It installs the exact version reported by PyPI with a forced cache refresh, launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.4.2.`). This protection requires an installed version containing the backup safeguard and applies to `metalist update`; direct pip/uv install commands do not run it.
 
 For pip, users can run `pip install metalist`. For a non-editable local install from this checkout, use `uv pip install .` or `pip install .` instead of the editable command below.
 
@@ -78,6 +82,8 @@ npm install
 ```
 
 ### Run
+
+Persistent data defaults to `~/MetaList`. Set `METALIST_DATA_DIRECTORY` to an absolute directory path before launch to use a separate data root, including namespaces, backups, certificates, logs, and runtime jobs. This is used by release tests to keep their disposable data separate from real notes.
 The installed entrypoint starts or restarts every known namespace, prints their URLs, and exits:
 ```bash
 metalist
@@ -233,8 +239,9 @@ This repo now packages itself under the PyPI distribution name `metalist`. Curre
 
 Recommended release path:
 1. In the existing PyPI project `metalist`, configure GitHub Trusted Publishing for `evolvingstuff/metalist` and the workflow file `.github/workflows/publish-pypi.yml`.
-2. Push a tag such as `v0.3.5`.
-3. After the GitHub Actions workflow completes, users can run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
+2. Push the candidate commit to a branch and wait for the `Publish to PyPI` validation workflow to pass for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on Python 3.10 through 3.13. You do not need those operating systems locally. Archive verification also checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
+3. Only after every required check passes, create and push the release tag for that same commit. Any intervening change requires a new validation run before tagging. Branch and pull-request runs validate without publishing.
+4. The tag workflow repeats validation and publishes the same artifacts tested by its full matrix. Manual publication also requires all checks to pass. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
 
 If `--input` is omitted, a file picker opens (when `tkinter` is available).
 Notes tagged with `@implies` are converted into ontology rules and are not imported as notes. Legacy rules that are invalid under the current ontology grammar are reported and skipped while valid rules continue importing.

{metalist-0.4.0 → metalist-0.4.2}/app/api/routes/ai.py

@@ -7,12 +7,13 @@ import json
 import math
 import time
 from collections.abc import AsyncIterator
-from typing import Annotated, Any, Literal, Self
+from typing import Annotated, Any, Literal
 from uuid import uuid4
 
 from fastapi import APIRouter, Depends, HTTPException, Response
 from fastapi.responses import StreamingResponse
 from pydantic import BaseModel, ConfigDict, Field, SecretStr, field_validator, model_validator
+from typing_extensions import Self
 
 from app.api.request_auth import require_request_auth_token
 from app.api.transactions import transactional_route

{metalist-0.4.0 → metalist-0.4.2}/app/api/routes/auth.py

@@ -60,7 +60,6 @@ from app.services import auth_cache_state
 from app.services.ontology_rules_store import ensure_rules_decrypted_and_compiled
 from app.services.link_titles import link_title_store
 from app.services.reminders import reminder_store
-from app.services.embedded_documents import document_store
 from app.services.search_history import search_history_store
 from app.services.sound_storage import sound_store
 from app.services.runtime_lock import purge_decrypted_runtime_state
@@ -347,7 +346,6 @@ def _reset_runtime_state_after_restore() -> bool:
     tab_state_store.reset()
     link_title_store.reset()
     reminder_store.reset()
-    document_store.reset()
     search_history_store.reset()
     sound_store.reset()
     clear_all_locks()
@@ -378,7 +376,6 @@ def _reset_runtime_state_after_restore() -> bool:
             tab_state_store.bootstrap(connection=connection)
             link_title_store.bootstrap(connection=connection)
             reminder_store.bootstrap(connection=connection)
-            document_store.bootstrap(connection=connection)
             if not password_required or database_version == CURRENT_DATABASE_VERSION:
                 search_history_store.bootstrap(connection=connection)
             else:
@@ -520,7 +517,6 @@ def login(
     tab_state_store.ensure_decrypted(token="")
     link_title_store.ensure_decrypted(token="")
     reminder_store.ensure_decrypted(token="")
-    document_store.ensure_decrypted(token="")
     search_history_store.ensure_decrypted(token="")
 
     ai_chat_store.reset()

metalist-0.4.2/app/data_directory.py

@@ -0,0 +1,22 @@
+"""Resolve the persistent data root without importing application runtime state.
+
+METALIST_DATA_DIRECTORY accepts an absolute directory path. When omitted,
+MetaList uses the current user's ~/MetaList directory.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from pathlib import Path
+
+
+def resolve_data_directory(*, environ: Mapping[str, str]) -> Path:
+    if "METALIST_DATA_DIRECTORY" not in environ:
+        return Path.home() / "MetaList"
+    configured_directory = environ["METALIST_DATA_DIRECTORY"]
+    if not isinstance(configured_directory, str) or configured_directory.strip() == "":
+        raise ValueError("METALIST_DATA_DIRECTORY must be a nonempty absolute path")
+    directory = Path(configured_directory)
+    if not directory.is_absolute():
+        raise ValueError("METALIST_DATA_DIRECTORY must be an absolute path")
+    return directory

{metalist-0.4.0 → metalist-0.4.2}/app/db/migrations.py

@@ -5,10 +5,10 @@ from __future__ import annotations
 from collections.abc import Callable
 from dataclasses import dataclass
 import sqlite3
+import re
 
 from app.db.version import CURRENT_DATABASE_VERSION
 from app.db.schema import create_namespace_content_migrations_table
-from app.db.schema import create_embedded_documents_table
 from app.services.encryption import EncryptionService
 
 
@@ -309,10 +309,65 @@ def _migration_6_to_7(
     encryption_enabled: bool,
     encryption_service: EncryptionService | None,
 ) -> int:
-    create_embedded_documents_table(connection)
+    # Retain the historical step so upgrades remain ordered and transactional.
+    connection.execute(
+        "CREATE TABLE IF NOT EXISTS embedded_documents ("
+        "id TEXT PRIMARY KEY, payload TEXT NOT NULL, nonce BLOB, tag BLOB)"
+    )
     return 0
 
 
+def _remove_retired_document_references(
+    *, connection: sqlite3.Connection, document_ids: set[str],
+    encryption_enabled: bool, encryption_service: EncryptionService | None,
+) -> int:
+    if encryption_enabled and (encryption_service is None or encryption_service.dek is None):
+        raise RuntimeError("Database migration 7→8 requires the namespace DEK")
+    token_pattern = re.compile(r"!?\[\[([0-9a-fA-F-]{36})\]\]")
+    rewritten_count = 0
+    rows = connection.execute("SELECT id, content, encryption_nonce, encryption_tag FROM notes").fetchall()
+    for note_id, content, nonce, tag in rows:
+        if (nonce is None) != (tag is None):
+            raise RuntimeError("Note has incomplete encryption metadata during migration 7→8")
+        plaintext = content
+        if nonce is not None:
+            if encryption_service is None or encryption_service.dek is None:
+                raise RuntimeError("Encrypted note requires the namespace DEK during migration 7→8")
+            plaintext = encryption_service.decrypt_from_storage(content, nonce, tag)
+        cleaned = token_pattern.sub(
+            lambda match: "" if match.group(1).lower() in document_ids else match.group(0),
+            plaintext,
+        )
+        if cleaned == plaintext:
+            continue
+        fields = (cleaned, None, None)
+        if encryption_enabled:
+            assert encryption_service is not None
+            fields = encryption_service.encrypt_for_storage(cleaned)
+        connection.execute(
+            "UPDATE notes SET content = ?, encryption_nonce = ?, encryption_tag = ? WHERE id = ?",
+            (*fields, note_id),
+        )
+        rewritten_count += 1
+    return rewritten_count
+
+
+def _migration_7_to_8(
+    *, connection: sqlite3.Connection, encryption_enabled: bool,
+    encryption_service: EncryptionService | None,
+) -> int:
+    """Remove retired diagrams and their placements from the installed live DB only."""
+    document_ids = {row[0] for row in connection.execute("SELECT id FROM embedded_documents")}
+    rewritten_count = 0
+    if document_ids:
+        rewritten_count = _remove_retired_document_references(
+            connection=connection, document_ids=document_ids,
+            encryption_enabled=encryption_enabled, encryption_service=encryption_service,
+        )
+    connection.execute("DROP TABLE embedded_documents")
+    return rewritten_count
+
+
 _MIGRATIONS: dict[
     int,
     Callable[..., int],
@@ -324,6 +379,7 @@ _MIGRATIONS: dict[
     4: _migration_4_to_5,
     5: _migration_5_to_6,
     6: _migration_6_to_7,
+    7: _migration_7_to_8,
 }
 
 

{metalist-0.4.0 → metalist-0.4.2}/app/db/schema.py

@@ -14,15 +14,6 @@ SEARCH_HISTORY_TABLE = "search_interaction_history"
 NAMESPACE_LAUNCH_PROFILE_TABLE = "namespace_launch_profile"
 NAMESPACE_CONTENT_MIGRATIONS_TABLE = "namespace_content_migrations"
 
-_CREATE_EMBEDDED_DOCUMENTS_TABLE = """
-CREATE TABLE IF NOT EXISTS embedded_documents (
-    id TEXT PRIMARY KEY,
-    payload TEXT NOT NULL,
-    nonce BLOB,
-    tag BLOB
-);
-"""
-
 _CREATE_NOTES_TABLE = f"""
 CREATE TABLE IF NOT EXISTS {NOTES_TABLE} (
     id TEXT PRIMARY KEY,
@@ -190,11 +181,6 @@ def create_namespace_content_migrations_table(connection: Connection) -> None:
     connection.execute(_CREATE_NAMESPACE_CONTENT_MIGRATIONS_TABLE)
 
 
-def create_embedded_documents_table(connection: Connection) -> None:
-    """Create the v7 editable-document table in the live namespace database."""
-    connection.execute(_CREATE_EMBEDDED_DOCUMENTS_TABLE)
-
-
 def _ensure_columns(connection: Connection, table: str, columns: dict[str, str]) -> None:
     existing = {
         row[1]
@@ -217,7 +203,6 @@ def initialize_schema(connection: Connection) -> None:
     connection.execute(_CREATE_TAB_STATE_TABLE)
     connection.execute(_CREATE_LINK_TITLES_TABLE)
     connection.execute(_CREATE_REMINDERS_TABLE)
-    create_embedded_documents_table(connection)
     connection.execute(_CREATE_SEARCH_HISTORY_TABLE)
     connection.execute(_CREATE_NAMESPACE_LAUNCH_PROFILE_TABLE)
     connection.execute(_CREATE_NAMESPACE_CONTENT_MIGRATIONS_TABLE)

{metalist-0.4.0 → metalist-0.4.2}/app/db/version.py

@@ -1,3 +1,3 @@
 """Database schema version constants without runtime configuration imports."""
 
-CURRENT_DATABASE_VERSION = 7
+CURRENT_DATABASE_VERSION = 8

{metalist-0.4.0 → metalist-0.4.2}/app/encryption_audit.py

@@ -285,7 +285,6 @@ _MAIN_SCHEMA_BEFORE_CONTENT_MIGRATIONS = {
             "next_check_after", "failure_count", "created_at", "updated_at",
         }
     ),
-    "embedded_documents": frozenset({"id", "payload", "nonce", "tag"}),
     "reminders": frozenset(
         {
             "id", "payload_json", "payload_encryption_nonce", "payload_encryption_tag",
@@ -424,6 +423,7 @@ _MIGRATION_DEFERRED_PLAINTEXT_FIELDS_BY_DATABASE_VERSION = {
     4: frozenset({("notes", "proposed_tags")}),
     5: frozenset({("notes", "proposed_tags")}),
     6: frozenset(),
+    7: frozenset(),
 }
 
 
@@ -453,12 +453,11 @@ def _main_schema_for_database_version(
     if database_version < 0 or database_version > CURRENT_DATABASE_VERSION:
         raise ValueError("database_version is outside the supported audit range")
     resolved_schema = expected_schema
-    # Audit precedes live schema initialization. Only pre-v7 namespaces may
-    # legitimately lack this table; if present, all columns/payloads are audited.
-    if database_version < 7 and "embedded_documents" not in actual_tables:
+    # Read-only audits must still validate retired payloads before the live 7→8 migration.
+    if database_version < 8 and (database_version == 7 or "embedded_documents" in actual_tables):
         resolved_schema = {
-            table: columns for table, columns in resolved_schema.items()
-            if table != "embedded_documents"
+            **resolved_schema,
+            "embedded_documents": frozenset({"id", "payload", "nonce", "tag"}),
         }
     if database_version < 6 and _PROPOSED_TAG_COLUMNS.isdisjoint(actual_notes_columns):
         note_columns = resolved_schema["notes"]

{metalist-0.4.0 → metalist-0.4.2}/app/main.py

@@ -28,7 +28,6 @@ from app.services.ontology_rules_store import bootstrap_ontology_rules_store
 from app.services.tab_state import tab_state_store
 from app.services.link_titles import link_title_store
 from app.services.reminders import reminder_store
-from app.services.embedded_documents import document_store
 from app.services.search_history import search_history_store
 from app.services.sound_storage import sound_store
 from app.services.runtime_hardening import apply_runtime_hardening
@@ -58,7 +57,6 @@ from app.api.routes.sounds import router as api2_sounds_router
 from app.api.routes.ontology import router as api2_ontology_router
 from app.api.routes.backups import router as api2_backup_router
 from app.api.routes.reminders import router as api2_reminders_router
-from app.api.routes.embedded_documents import router as api2_documents_router
 from app.api.routes.remote_images import router as api2_remote_images_router
 from app.api.routes.ai import router as api2_ai_router
 from app.api.routes.test import router as api2_test_router
@@ -198,7 +196,6 @@ with begin_writer() as connection:
     tab_state_store.bootstrap(connection=connection)
     link_title_store.bootstrap(connection=connection)
     reminder_store.bootstrap(connection=connection)
-    document_store.bootstrap(connection=connection)
     if not bool(settings["encryption_enabled"]) or database_version >= CURRENT_DATABASE_VERSION:
         search_history_store.bootstrap(connection=connection)
     else:
@@ -323,7 +320,6 @@ app.include_router(api2_files_router, prefix=API_PREFIX)
 app.include_router(api2_sounds_router, prefix=API_PREFIX)
 app.include_router(api2_ontology_router, prefix=API_PREFIX)
 app.include_router(api2_reminders_router, prefix=API_PREFIX)
-app.include_router(api2_documents_router, prefix=API_PREFIX)
 app.include_router(api2_remote_images_router, prefix=API_PREFIX)
 app.include_router(api2_ai_router, prefix=API_PREFIX)
 if TEST_MODE:

{metalist-0.4.0 → metalist-0.4.2}/app/presentation/render/note_renderer.py

@@ -8,7 +8,6 @@ Acts as the single source of truth for note rendering across the application.
 import re
 import logging
 
-from app.services.document_references import render_editable_documents
 from app.services.content_formatting import format_note_content_for_view
 from app.services.content_formatting import has_scoped_footnotes
 from app.services.content_formatting import extract_plain_text_from_note_html
@@ -150,7 +149,7 @@ def render_collapsed_read_only_mode(note) -> str:
 
 
 def render_editing_mode(note) -> str:
-    return render_editable_documents(note.content)
+    return note.content
 
 
 def render_redacted_mode(note) -> str:

{metalist-0.4.0 → metalist-0.4.2}/app/server_runtime.py

@@ -3,7 +3,7 @@ from __future__ import annotations
 import argparse
 from collections.abc import Mapping, MutableMapping, Sequence
 from dataclasses import dataclass
-from datetime import UTC, datetime, timedelta
+from datetime import datetime, timedelta, timezone
 import ipaddress
 import os
 from pathlib import Path
@@ -17,6 +17,7 @@ from cryptography import x509
 from cryptography.hazmat.primitives import hashes, serialization
 from cryptography.hazmat.primitives.asymmetric import rsa
 from cryptography.x509.oid import NameOID
+from app.data_directory import resolve_data_directory
 from app.db.schema import NAMESPACE_LAUNCH_PROFILE_TABLE
 from app.db.schema import initialize_schema
 from app.db.settings_sql import insert_default_settings
@@ -28,10 +29,12 @@ from app.security.shell_execution import SHELL_EXECUTION_ENV_NAME
 _LOOPBACK_BIND_HOSTS = frozenset({"127.0.0.1", "localhost", "0.0.0.0", "::1"})
 _DEFAULT_API_PREFIX = "/api2"
 _DEFAULT_V1_API_PREFIX = "/api"
-_DEFAULT_DATABASE_DIRECTORY = Path.home() / "MetaList"
+_DEFAULT_DATABASE_DIRECTORY = resolve_data_directory(environ=os.environ)
 _DEFAULT_CERT_PATH = _DEFAULT_DATABASE_DIRECTORY / "certs" / "metalist-cert.pem"
 _DEFAULT_KEY_PATH = _DEFAULT_DATABASE_DIRECTORY / "certs" / "metalist-key.pem"
 _DEFAULT_RUNTIME_DIRECTORY = Path(tempfile.gettempdir()) / "metalist-runtime"
+if "METALIST_DATA_DIRECTORY" in os.environ:
+    _DEFAULT_RUNTIME_DIRECTORY = _DEFAULT_DATABASE_DIRECTORY / "runtime"
 _DEFAULT_NAMESPACES_DIRECTORY_NAME = "namespaces"
 _DEFAULT_NAMESPACE_DELETE_JOBS_DIRECTORY_NAME = "namespace-delete-jobs"
 _DEFAULT_NAMESPACE_RENAME_JOBS_DIRECTORY_NAME = "namespace-rename-jobs"
@@ -180,7 +183,7 @@ def _coerce_optional_db_port(*, value: object) -> int | None:
 
 
 def _utc_timestamp() -> str:
-    return datetime.now(UTC).isoformat()
+    return datetime.now(timezone.utc).isoformat()
 
 
 def _connect_namespace_database(
@@ -305,7 +308,34 @@ def load_namespace_launch_profile(*, namespace: str) -> NamespaceLaunchProfile |
     return profile
 
 
-def load_all_namespace_launch_profiles() -> list[NamespaceLaunchProfile]:
+def _read_namespace_launch_profile(
+    *,
+    database_path: Path,
+    namespace: str,
+) -> NamespaceLaunchProfile | None:
+    database_uri = f"{database_path.resolve().as_uri()}?mode=ro"
+    connection = sqlite3.connect(database_uri, uri=True, check_same_thread=False)
+    try:
+        connection.execute("PRAGMA query_only = ON")
+        profile_table = connection.execute(
+            "SELECT name FROM sqlite_master WHERE type = 'table' AND name = ?",
+            (NAMESPACE_LAUNCH_PROFILE_TABLE,),
+        ).fetchone()
+        if profile_table is None:
+            return None
+        _assert_launch_profile_rows_belong_to_namespace(
+            connection=connection,
+            namespace=namespace,
+        )
+        return _fetch_launch_profile_from_connection(
+            connection=connection,
+            namespace=namespace,
+        )
+    finally:
+        connection.close()
+
+
+def _load_all_namespace_launch_profiles(*, read_only: bool) -> list[NamespaceLaunchProfile]:
     namespaces_directory = resolve_namespaces_directory()
     if not namespaces_directory.exists():
         return []
@@ -330,13 +360,27 @@ def load_all_namespace_launch_profiles() -> list[NamespaceLaunchProfile]:
                 f"Namespace {normalized_namespace} directory exists but database is missing: "
                 f"{database_path}"
             )
-        profile = load_namespace_launch_profile(namespace=normalized_namespace)
+        if read_only:
+            profile = _read_namespace_launch_profile(
+                database_path=database_path,
+                namespace=normalized_namespace,
+            )
+        else:
+            profile = load_namespace_launch_profile(namespace=normalized_namespace)
         if profile is None:
             continue
         profiles.append(profile)
     return profiles
 
 
+def load_all_namespace_launch_profiles() -> list[NamespaceLaunchProfile]:
+    return _load_all_namespace_launch_profiles(read_only=False)
+
+
+def load_all_namespace_launch_profiles_read_only() -> list[NamespaceLaunchProfile]:
+    return _load_all_namespace_launch_profiles(read_only=True)
+
+
 def save_namespace_launch_profile(
     *,
     namespace: str,
@@ -896,7 +940,7 @@ def ensure_default_tls_pair(*, environ: Mapping[str, str]) -> tuple[str, str] |
     subject = issuer = x509.Name(
         [x509.NameAttribute(NameOID.COMMON_NAME, common_name)]
     )
-    now = datetime.now(UTC)
+    now = datetime.now(timezone.utc)
     certificate = (
         x509.CertificateBuilder()
         .subject_name(subject)

{metalist-0.4.0 → metalist-0.4.2}/app/services/agent/actions.py

@@ -7,9 +7,10 @@ from collections.abc import Iterator
 from contextlib import contextmanager
 from contextvars import ContextVar
 from dataclasses import dataclass
-from typing import Annotated, Literal, Self, Union
+from typing import Annotated, Literal, Union
 
 from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, field_validator, model_validator
+from typing_extensions import Self
 
 from app.services.search_query import parse_search_query
 

{metalist-0.4.0 → metalist-0.4.2}/app/services/agent/scope.py

@@ -6,9 +6,10 @@ from collections.abc import Callable
 from dataclasses import dataclass
 from datetime import datetime, timezone
 from types import MappingProxyType
-from typing import Literal, Mapping, Self
+from typing import Literal, Mapping
 
 from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+from typing_extensions import Self
 
 from app.services.agent.cloud_privacy import CloudPrivacyBoundary
 from app.services.agent.cloud_privacy import CloudPrivacyEvaluator

{metalist-0.4.0 → metalist-0.4.2}/app/services/agent/skills/__init__.py

@@ -12,5 +12,4 @@ def load_skill(name: str) -> str:
     return skill.rstrip("\n")
 
 
-SEARCH_NOTES_SKILL = load_skill("search-notes.md")
 SCOPED_INVESTIGATION_SKILL = load_skill("scoped-investigation.md")

metalist-0.4.2/app/services/agent/skills/scoped-investigation.md

@@ -0,0 +1,27 @@
+# Investigate Current Scope
+
+This skill is active because the user explicitly asked for evidence from saved
+notes in the frozen current MetaList scope.
+
+The runtime supplies one authoritative evidence payload. It contains complete
+root-note trees in the same order visible to the user. Every included note has
+its full agent-visible content; there is no per-note truncation. If the complete
+scope exceeds the configured token limit, the runtime keeps the longest leading
+prefix of complete result trees that fits and reports the included and omitted
+counts. Never imply that a truncated scope is exhaustive.
+
+Treat each nested object as a note. Child notes inherit their ancestors' context,
+but only explicit tags are serialized on each note. Privacy-redacted content is
+not present. Do not invent omitted content.
+
+Answer the current user request directly from relevant nodes in the payload.
+Shared scope terms do not make every node relevant. Exclude material that does
+not directly help answer the exact request.
+
+For every factual claim derived from a note, cite the exact supporting note as
+`[[note_id]]`. Copy the ID from the same object whose `content_text` supports
+the claim. Do not cite an enclosing root merely because the supporting child is
+inside it. Do not emit duplicate adjacent citations.
+
+Write the final response in Markdown. LaTeX and Mermaid are available when they
+materially help.

{metalist-0.4.0 → metalist-0.4.2}/app/services/auth_service.py

@@ -54,7 +54,6 @@ from app.services.sound_storage import decrypt_all_sounds_for_plaintext
 from app.services.sound_storage import encrypt_all_sounds_for_active_dek
 from app.services.link_titles import rewrite_persisted_link_titles
 from app.services.reminders import reminder_store
-from app.services.embedded_documents import document_store
 from app.utils.text_utils import strip_html
 from app.services.encryption import EncryptionService
 from app.services.maintenance_mode import maintenance_service
@@ -529,11 +528,6 @@ class AuthService:
                     encryption_service=self.encryption,
                     force_plaintext=False,
                 )
-                document_store.rewrite_storage(
-                    connection=connection,
-                    encryption_service=self.encryption,
-                    force_plaintext=False,
-                )
         finally:
             maintenance_service.exit_maintenance()
 
@@ -848,11 +842,6 @@ class AuthService:
                     encryption_service=self.encryption,
                     force_plaintext=True,
                 )
-                document_store.rewrite_storage(
-                    connection=connection,
-                    encryption_service=self.encryption,
-                    force_plaintext=True,
-                )
                 clear_password_settings(connection)
             self.encryption.clear_keys()
         finally:

{metalist-0.4.0 → metalist-0.4.2}/app/services/embedded_references.py

@@ -6,7 +6,6 @@ from dataclasses import dataclass
 from html.parser import HTMLParser
 from typing import Callable, FrozenSet, List, Optional, Tuple
 
-from app.services.embedded_documents import document_store, render_document
 from app.services.content_formatting import find_consumed_content_wrapper_keys
 from app.services.content_formatting import find_global_credential_tag
 from app.services.content_formatting import format_note_content_for_view
@@ -342,11 +341,7 @@ def collapsed_preview_source_has_media(content_html: str) -> bool:
     preview_source_html = extract_collapsed_preview_source_html(content_html)
     if preview_source_html == "":
         return False
-    if _fragment_has_media_tag(preview_source_html):
-        return True
-    # Editable documents store UUID tokens, but render as visual previews.
-    return any(document_store.has(token.note_id)
-               for token in collect_reference_tokens_from_html(preview_source_html))
+    return _fragment_has_media_tag(preview_source_html)
 
 
 def collapsed_preview_source_has_image_file_embed(
@@ -551,11 +546,6 @@ def _render_reference_block(
     redact_passwords: bool,
     render_note_embeds_as_links: bool,
 ) -> str:
-    if document_store.has(reference_note_id):
-        prefix = ""
-        if is_embed:
-            prefix = "!"
-        return render_document(reference_note_id, token=prefix + "[[" + reference_note_id + "]]", static_export=static_export)
     mode = "link"
     if is_embed:
         mode = "embed"

{metalist-0.4.0 → metalist-0.4.2}/app/services/managed_ollama_runtime.py

@@ -4,7 +4,7 @@ from __future__ import annotations
 
 from collections.abc import Mapping
 from dataclasses import asdict, dataclass
-from datetime import UTC, datetime
+from datetime import datetime, timezone
 import json
 import os
 from pathlib import Path
@@ -283,7 +283,7 @@ class ManagedOllamaRuntime:
                     executable=executable,
                     pid=process.pid,
                     port=self.config.port,
-                    started_at=datetime.now(UTC).isoformat(),
+                    started_at=datetime.now(timezone.utc).isoformat(),
                     version=version,
                 )
                 self._write_state(state=state)