MetaList 0.3.12__tar.gz → 0.3.14__tar.gz

{metalist-0.3.12/metalist.egg-info → metalist-0.3.14}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: metalist
-Version: 0.3.12
+Version: 0.3.14
 Summary: A minimalist single-user note-taking app with SSR, fast in-memory tree operations, and efficient sync.
 Project-URL: Homepage, https://github.com/evolvingstuff/metalist
 Project-URL: Repository, https://github.com/evolvingstuff/metalist
@@ -19,6 +19,7 @@ Requires-Dist: mutagen==1.48.1
 Requires-Dist: nh3==0.3.6
 Requires-Dist: pillow==12.3.0
 Requires-Dist: pydantic==2.13.4
+Requires-Dist: python-dotenv==1.2.2
 Requires-Dist: python-multipart==0.0.32
 Requires-Dist: starlette==1.3.1
 Requires-Dist: tqdm==4.70.0
@@ -66,6 +67,20 @@ A minimalist single-user note-taking app focused on server-side rendering (SSR),
 - The browser client drives interaction via `/api2` JSON endpoints.
 - Notes are loaded/decrypted into an in-memory store at startup; a post-startup DB read guard prevents accidental runtime SELECTs.
 
+## Security Boundary
+
+Password protection encrypts namespace data at rest. While a namespace is
+unlocked, the server must hold its data-encryption key and decrypted working
+data in process memory, and the browser holds the decrypted content currently
+rendered in the page. A sufficiently privileged local process, debugger,
+administrator, browser extension, or process-memory dump can therefore expose
+an unlocked namespace.
+
+Explicit logout removes the live key and purges decrypted runtime stores, but
+it cannot guarantee forensic overwriting of memory previously allocated by the
+Python or browser runtimes. Protect the host account and operating system, and
+see the detailed [security architecture and threat model](docs/security/README.md).
+
 ## Development
 
 ### Setup
@@ -84,7 +99,7 @@ After the first installation, update and restart MetaList with one cross-platfor
 ```bash
 metalist update
 ```
-The updater stops running namespaces, hands off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock, runs the forced cache-refreshed uv update, and launches MetaList again.
+The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it stops running namespaces, hands off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock, installs the exact version reported by PyPI with a forced cache refresh, launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.3.14.`).
 
 For pip, users can run `pip install metalist`. For a non-editable local install from this checkout, use `uv pip install .` or `pip install .` instead of the editable command below.
 
@@ -268,6 +283,18 @@ node --test tests/unit/*.mjs
 .venv/bin/python -c "from pathlib import Path; import main; main._run_startup_sanity_gates(repo_root=Path.cwd())"
 ```
 
+Source-checkout development launches can enable the mandatory Python and
+JavaScript sanity gates with an ignored repository-root `.env` file:
+
+```dotenv
+METALIST_ENVIRONMENT=development
+```
+
+`main.py` prints the resolved environment at startup. An explicit process
+environment value takes precedence over `.env`; only `development` and
+`production` are accepted. When the variable and `.env` file are absent,
+MetaList runs in production mode and skips the source sanity gates.
+
 `TEST_MODE=1` and `POST /api2/test/reset` still exist for deterministic browser automation if we decide to add a new harness later, but Cypress is not part of the current workflow.
 
 ### Diagrams

{metalist-0.3.12 → metalist-0.3.14}/README.md

@@ -33,6 +33,20 @@ A minimalist single-user note-taking app focused on server-side rendering (SSR),
 - The browser client drives interaction via `/api2` JSON endpoints.
 - Notes are loaded/decrypted into an in-memory store at startup; a post-startup DB read guard prevents accidental runtime SELECTs.
 
+## Security Boundary
+
+Password protection encrypts namespace data at rest. While a namespace is
+unlocked, the server must hold its data-encryption key and decrypted working
+data in process memory, and the browser holds the decrypted content currently
+rendered in the page. A sufficiently privileged local process, debugger,
+administrator, browser extension, or process-memory dump can therefore expose
+an unlocked namespace.
+
+Explicit logout removes the live key and purges decrypted runtime stores, but
+it cannot guarantee forensic overwriting of memory previously allocated by the
+Python or browser runtimes. Protect the host account and operating system, and
+see the detailed [security architecture and threat model](docs/security/README.md).
+
 ## Development
 
 ### Setup
@@ -51,7 +65,7 @@ After the first installation, update and restart MetaList with one cross-platfor
 ```bash
 metalist update
 ```
-The updater stops running namespaces, hands off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock, runs the forced cache-refreshed uv update, and launches MetaList again.
+The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it stops running namespaces, hands off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock, installs the exact version reported by PyPI with a forced cache refresh, launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.3.14.`).
 
 For pip, users can run `pip install metalist`. For a non-editable local install from this checkout, use `uv pip install .` or `pip install .` instead of the editable command below.
 
@@ -235,6 +249,18 @@ node --test tests/unit/*.mjs
 .venv/bin/python -c "from pathlib import Path; import main; main._run_startup_sanity_gates(repo_root=Path.cwd())"
 ```
 
+Source-checkout development launches can enable the mandatory Python and
+JavaScript sanity gates with an ignored repository-root `.env` file:
+
+```dotenv
+METALIST_ENVIRONMENT=development
+```
+
+`main.py` prints the resolved environment at startup. An explicit process
+environment value takes precedence over `.env`; only `development` and
+`production` are accepted. When the variable and `.env` file are absent,
+MetaList runs in production mode and skips the source sanity gates.
+
 `TEST_MODE=1` and `POST /api2/test/reset` still exist for deterministic browser automation if we decide to add a new harness later, but Cypress is not part of the current workflow.
 
 ### Diagrams

{metalist-0.3.12 → metalist-0.3.14}/app/api/middleware/auth.py

@@ -7,7 +7,6 @@ from app.api.request_auth import read_request_auth_token
 from app.services.tokens import token_service
 from app.services.auth_service import AuthService
 from app.services.maintenance_mode import maintenance_service
-from app.services.runtime_lock import purge_decrypted_runtime_state
 from app.models.database import SafeSession
 from app.config import API_PREFIX, TEST_MODE
 
@@ -62,9 +61,6 @@ class AuthMiddleware(BaseHTTPMiddleware):
         """Check authentication for protected routes."""
         path = request.url.path
 
-        if not token_service.has_active_tokens():
-            purge_decrypted_runtime_state()
-
         # Block any v1 API usage with an explicit 410 Gone (no DB access)
         if path.startswith('/api') and not (path.startswith(API_PREFIX) or path == API_PREFIX):
             ref = request.headers.get('referer', '-')

{metalist-0.3.12 → metalist-0.3.14}/app/api/routes/auth.py

@@ -1,10 +1,8 @@
 from __future__ import annotations
 
 import logging
-import secrets
-import sqlite3
 from fastapi import APIRouter, Depends, Header, HTTPException, Request, Response
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, SecretStr
 from typing import Annotated, Optional
 from concurrent.futures import Future, ThreadPoolExecutor
 from threading import Lock
@@ -16,20 +14,13 @@ import time
 from app.api.transactions import transactional_route
 from app.api.deps import get_db
 from app.config import ACTIVE_NAMESPACE
-from app.config import KDF_ALGORITHM
-from app.config import KDF_MAX_MEMORY_COST_KIB
-from app.config import KDF_MAX_PARALLELISM
-from app.config import KDF_MAX_TIME_COST
-from app.config import KDF_MIN_MEMORY_COST_KIB
-from app.config import KDF_MIN_PARALLELISM
-from app.config import KDF_MIN_TIME_COST
 from app.config import KDF_TIME_COST
-from app.config import VAULT_VERSION
 from app.config import VERSION
-from app.db.schema import APP_SETTINGS_TABLE
+from app.db.migrations import CURRENT_DATABASE_VERSION
 from app.db.settings_sql import fetch_settings
 from app.models.database import SafeSession
 from app.services.auth_service import AuthService
+from app.services.auth_service import migrate_restored_database_if_unlocked
 from app.services.backup_service import (
     BackupFileInfo,
     create_timestamped_backup,
@@ -70,6 +61,8 @@ from app.services.reminders import reminder_store
 from app.services.search_history import search_history_store
 from app.services.sound_storage import sound_store
 from app.services.runtime_lock import purge_decrypted_runtime_state
+from app.services.diagnostics import activate_authenticated_logging
+from app.services.diagnostics import deactivate_authenticated_logging
 from app.services.hydration_state import hydration_state
 from app.services.file_registry import file_registry
 from app.services.file_storage import bootstrap_file_registry
@@ -98,7 +91,7 @@ logger = logging.getLogger(__name__)
 
 
 class LoginRequest(BaseModel):
-    password: str
+    password: SecretStr
 
 
 class LoginResponse(BaseModel):
@@ -143,17 +136,17 @@ class HydrationStatusResponse(BaseModel):
 
 
 class PasswordCreateRequest(BaseModel):
-    password: str
+    password: SecretStr
 
 
 class PasswordChangeRequest(BaseModel):
-    current_password: str
-    new_password: str
+    current_password: SecretStr
+    new_password: SecretStr
     iterations: Optional[int] = None
 
 
 class PasswordRemoveRequest(BaseModel):
-    current_password: str
+    current_password: SecretStr
 
 
 class SessionTimeoutResponse(BaseModel):
@@ -226,6 +219,14 @@ _server_restart_lock = Lock()
 _server_restart_scheduled = False
 
 
+def _active_diagnostics_namespace() -> str:
+    if ACTIVE_NAMESPACE is None:
+        return "default"
+    if ACTIVE_NAMESPACE == "":
+        raise RuntimeError("Active namespace must not be empty")
+    return ACTIVE_NAMESPACE
+
+
 def _reexec_server_process() -> None:
     argv = [sys.executable, *sys.argv]
     os.execv(sys.executable, argv)
@@ -265,156 +266,6 @@ def _namespace_target_exists(*, namespace: str) -> bool:
     return False
 
 
-def _read_namespace_auth_settings(*, namespace: str) -> dict[str, object] | None:
-    database_path = resolve_namespaced_database_path(namespace=namespace)
-    if not database_path.exists():
-        return None
-    if not database_path.is_file():
-        raise RuntimeError(f"Namespace database path is not a file: {database_path}")
-
-    connection = sqlite3.connect(database_path)
-    connection.row_factory = sqlite3.Row
-    try:
-        table_row = connection.execute(
-            "SELECT 1 FROM sqlite_master WHERE type = ? AND name = ?",
-            ("table", APP_SETTINGS_TABLE),
-        ).fetchone()
-        if table_row is None:
-            raise RuntimeError(f"Namespace database has no {APP_SETTINGS_TABLE} table: {namespace}")
-        row = connection.execute(
-            f"""
-            SELECT
-                auth_verifier,
-                auth_salt,
-                auth_iterations,
-                kek_iterations,
-                vault_version,
-                kdf_algorithm,
-                kdf_memory_cost_kib,
-                kdf_parallelism,
-                encryption_enabled
-            FROM {APP_SETTINGS_TABLE}
-            WHERE id = 1
-            """
-        ).fetchone()
-        if row is None:
-            raise RuntimeError(f"Namespace database has no app_settings row: {namespace}")
-        return {
-            "auth_verifier": row["auth_verifier"],
-            "auth_salt": row["auth_salt"],
-            "auth_iterations": row["auth_iterations"],
-            "kek_iterations": row["kek_iterations"],
-            "vault_version": row["vault_version"],
-            "kdf_algorithm": row["kdf_algorithm"],
-            "kdf_memory_cost_kib": row["kdf_memory_cost_kib"],
-            "kdf_parallelism": row["kdf_parallelism"],
-            "encryption_enabled": row["encryption_enabled"],
-        }
-    finally:
-        connection.close()
-
-
-def _namespace_auth_settings_have_password(*, settings: dict[str, object] | None) -> bool:
-    if settings is None:
-        return False
-    encryption_enabled = settings["encryption_enabled"]
-    if encryption_enabled not in (0, 1, False, True):
-        raise RuntimeError(f"Invalid encryption_enabled value in namespace DB: {encryption_enabled!r}")
-    if not bool(encryption_enabled):
-        return False
-    _assert_supported_namespace_vault_profile(settings=settings)
-    if settings["auth_verifier"] is None:
-        raise RuntimeError("Namespace encryption is enabled but auth_verifier is missing")
-    if settings["auth_salt"] is None:
-        raise RuntimeError("Namespace auth_verifier is set but auth_salt is NULL")
-    if settings["auth_iterations"] is None:
-        raise RuntimeError("Namespace auth_verifier is set but auth_iterations is NULL")
-    return True
-
-
-def _namespace_requires_password(*, namespace: str) -> bool:
-    settings = _read_namespace_auth_settings(namespace=namespace)
-    return _namespace_auth_settings_have_password(settings=settings)
-
-
-def _assert_supported_namespace_vault_profile(*, settings: dict[str, object]) -> None:
-    vault_version = settings["vault_version"]
-    if vault_version is None:
-        raise RuntimeError("Namespace encryption is enabled but vault_version is NULL")
-    if vault_version != VAULT_VERSION:
-        raise RuntimeError(f"Unsupported namespace vault version: {vault_version}")
-    kdf_algorithm = settings["kdf_algorithm"]
-    if kdf_algorithm is None:
-        raise RuntimeError("Namespace encryption is enabled but kdf_algorithm is NULL")
-    if kdf_algorithm != KDF_ALGORITHM:
-        raise RuntimeError(f"Unsupported namespace kdf_algorithm: {kdf_algorithm}")
-    auth_iterations = settings["auth_iterations"]
-    if auth_iterations is None:
-        raise RuntimeError("Namespace encryption is enabled but auth_iterations is NULL")
-    if not isinstance(auth_iterations, int):
-        raise RuntimeError(f"Namespace auth_iterations is not an integer: {auth_iterations!r}")
-    if not (KDF_MIN_TIME_COST <= auth_iterations <= KDF_MAX_TIME_COST):
-        raise RuntimeError(f"Namespace auth_iterations out of range: {auth_iterations}")
-    kek_iterations = settings["kek_iterations"]
-    if kek_iterations is None:
-        raise RuntimeError("Namespace encryption is enabled but kek_iterations is NULL")
-    if not isinstance(kek_iterations, int):
-        raise RuntimeError(f"Namespace kek_iterations is not an integer: {kek_iterations!r}")
-    if not (KDF_MIN_TIME_COST <= kek_iterations <= KDF_MAX_TIME_COST):
-        raise RuntimeError(f"Namespace kek_iterations out of range: {kek_iterations}")
-    memory_cost_kib = settings["kdf_memory_cost_kib"]
-    if memory_cost_kib is None:
-        raise RuntimeError("Namespace encryption is enabled but kdf_memory_cost_kib is NULL")
-    if not isinstance(memory_cost_kib, int):
-        raise RuntimeError(f"Namespace kdf_memory_cost_kib is not an integer: {memory_cost_kib!r}")
-    if not (KDF_MIN_MEMORY_COST_KIB <= memory_cost_kib <= KDF_MAX_MEMORY_COST_KIB):
-        raise RuntimeError(f"Namespace kdf_memory_cost_kib out of range: {memory_cost_kib}")
-    parallelism = settings["kdf_parallelism"]
-    if parallelism is None:
-        raise RuntimeError("Namespace encryption is enabled but kdf_parallelism is NULL")
-    if not isinstance(parallelism, int):
-        raise RuntimeError(f"Namespace kdf_parallelism is not an integer: {parallelism!r}")
-    if not (KDF_MIN_PARALLELISM <= parallelism <= KDF_MAX_PARALLELISM):
-        raise RuntimeError(f"Namespace kdf_parallelism out of range: {parallelism}")
-
-
-def _verify_namespace_password(*, namespace: str, password: str) -> None:
-    if not isinstance(password, str):
-        raise TypeError("namespace password must be a string")
-    settings = _read_namespace_auth_settings(namespace=namespace)
-    if not _namespace_auth_settings_have_password(settings=settings):
-        return
-    if password == "":
-        raise HTTPException(status_code=400, detail="Namespace password is required")
-
-    assert settings is not None
-    auth_salt = settings["auth_salt"]
-    auth_iterations = settings["auth_iterations"]
-    memory_cost_kib = settings["kdf_memory_cost_kib"]
-    parallelism = settings["kdf_parallelism"]
-    auth_verifier = settings["auth_verifier"]
-    if not isinstance(auth_salt, bytes):
-        raise RuntimeError("Namespace auth_salt is not bytes")
-    if not isinstance(auth_iterations, int):
-        raise RuntimeError("Namespace auth_iterations is not an integer")
-    if not isinstance(memory_cost_kib, int):
-        raise RuntimeError("Namespace kdf_memory_cost_kib is not an integer")
-    if not isinstance(parallelism, int):
-        raise RuntimeError("Namespace kdf_parallelism is not an integer")
-    if not isinstance(auth_verifier, str):
-        raise RuntimeError("Namespace auth_verifier is not a string")
-
-    candidate = EncryptionService().derive_master_key(
-        password,
-        auth_salt,
-        auth_iterations,
-        memory_cost_kib,
-        parallelism,
-    ).hex()
-    if not secrets.compare_digest(candidate, auth_verifier):
-        raise HTTPException(status_code=401, detail="Invalid password")
-
-
 def _run_hydration(*, rebuild_required: bool) -> None:
     hydration_state.set_phase(
         phase="database_check",
@@ -505,19 +356,27 @@ def _reset_runtime_state_after_restore() -> bool:
 
     session = SafeSession()
     try:
+        connection = session.connection()
         with SafeSession.allow_reads("auth:backup_restore:settings"):
-            settings = fetch_settings(session.connection())
+            settings = fetch_settings(connection)
         if settings is None:
             raise RuntimeError("App settings missing after backup restore")
 
         password_required = bool(settings["encryption_enabled"])
         set_encryption_required(password_required)
+        database_version = migrate_restored_database_if_unlocked(
+            connection=connection,
+            password_required=password_required,
+        )
 
         with SafeSession.allow_reads("auth:backup_restore:runtime_stores"):
-            tab_state_store.bootstrap(connection=session.connection())
-            link_title_store.bootstrap(connection=session.connection())
-            reminder_store.bootstrap(connection=session.connection())
-            search_history_store.bootstrap(connection=session.connection())
+            tab_state_store.bootstrap(connection=connection)
+            link_title_store.bootstrap(connection=connection)
+            reminder_store.bootstrap(connection=connection)
+            if not password_required or database_version == CURRENT_DATABASE_VERSION:
+                search_history_store.bootstrap(connection=connection)
+            else:
+                search_history_store.reset()
 
         if password_required:
             return True
@@ -587,15 +446,6 @@ def _require_string_field(payload: dict[str, object], field_name: str) -> str:
     return value
 
 
-def _require_string_field_allow_empty(payload: dict[str, object], field_name: str) -> str:
-    if field_name not in payload:
-        raise HTTPException(status_code=400, detail=f"{field_name} is required")
-    value = payload[field_name]
-    if not isinstance(value, str):
-        raise HTTPException(status_code=400, detail=f"{field_name} must be a string")
-    return value
-
-
 def _require_int_field(payload: dict[str, object], field_name: str) -> int:
     if field_name not in payload:
         raise HTTPException(status_code=400, detail=f"{field_name} is required")
@@ -634,6 +484,7 @@ def login(
     tab_id: Annotated[str, Depends(_require_tab_id)],
     db: Annotated[SafeSession, Depends(get_db)],
 ):
+    submitted_password = payload.password.get_secret_value()
     rate_limit_key = _login_rate_limit_key(request)
     allowed, retry_after_seconds = login_rate_limiter.check_allowed(rate_limit_key)
     if not allowed:
@@ -645,14 +496,20 @@ def login(
     auth = AuthService(db)
     if not auth.has_password():
         raise HTTPException(status_code=400, detail="No password is set. Please set a password first.")
-    if not auth.verify_password(payload.password):
+    if not auth.verify_password(submitted_password):
         login_rate_limiter.record_failure(rate_limit_key)
         raise HTTPException(status_code=401, detail="Invalid password")
 
-    dek = auth.unwrap_dek_for_password(payload.password)
+    dek = auth.unwrap_dek_for_password(submitted_password)
     auth.run_authenticated_database_migrations(dek=dek)
 
     set_session_dek(dek)
+    with SafeSession.allow_reads("auth:search_history_post_migration"):
+        search_history_store.bootstrap(connection=db.connection())
+    activate_authenticated_logging(
+        namespace=_active_diagnostics_namespace(),
+        dek=dek,
+    )
     ensure_rules_decrypted_and_compiled(token="")
     tab_state_store.ensure_decrypted(token="")
     link_title_store.ensure_decrypted(token="")
@@ -679,6 +536,7 @@ def logout(
     if not purge_decrypted_runtime_state():
         clear_all_locks()
         clear_encryption_key()
+    deactivate_authenticated_logging()
     clear_auth_cookie(response=response)
     return {"message": "Logout successful"}
 
@@ -726,6 +584,7 @@ def restore_from_backup(
     try:
         backup_file = restore_backup(payload.filename)
         password_required = _reset_runtime_state_after_restore()
+        deactivate_authenticated_logging()
     finally:
         maintenance_service.exit_maintenance()
 
@@ -978,15 +837,8 @@ def save_namespace_ports(
 def _delete_namespace_from_body(*, body: dict[str, object], target_namespace: str) -> dict[str, object]:
     normalized_target_namespace = validate_namespace(namespace=target_namespace)
     confirmed_namespace = _require_string_field(body, "confirmed_namespace")
-    current_password = _require_string_field_allow_empty(body, "current_password")
     redirect_namespace = _require_string_field(body, "redirect_namespace")
 
-    if _namespace_requires_password(namespace=normalized_target_namespace):
-        _verify_namespace_password(
-            namespace=normalized_target_namespace,
-            password=current_password,
-        )
-
     delete_capture = CapturedExceptionContext(
         RuntimeError,
         ValueError,
@@ -1028,9 +880,6 @@ def namespace_delete_preflight(
         namespace=_require_string_field(body, "target_namespace"),
     )
     target_exists = _namespace_target_exists(namespace=target_namespace)
-    target_requires_password = False
-    if target_exists:
-        target_requires_password = _namespace_requires_password(namespace=target_namespace)
     catalog = build_namespace_catalog(
         environ=os.environ,
         current_namespace=ACTIVE_NAMESPACE,
@@ -1054,7 +903,6 @@ def namespace_delete_preflight(
     return {
         "target_namespace": target_namespace,
         "target_exists": target_exists,
-        "target_requires_password": target_requires_password,
         "is_current_namespace": target_namespace == ACTIVE_NAMESPACE,
         "redirect_namespaces": redirect_namespaces,
         "recreates_default": recreates_default,
@@ -1199,7 +1047,7 @@ def create_password(
     auth = AuthService(db)
     if auth.has_password():
         raise HTTPException(status_code=400, detail="Password already exists. Use change endpoint instead.")
-    success, message = auth.set_password(payload.password, KDF_TIME_COST)
+    success, message = auth.set_password(payload.password.get_secret_value(), KDF_TIME_COST)
     if not success:
         raise HTTPException(status_code=400, detail=message)
     token_service.revoke_all_tokens()
@@ -1220,8 +1068,8 @@ def change_password(
     else:
         time_cost = KDF_TIME_COST
     success, message = auth.change_password(
-        payload.current_password,
-        payload.new_password,
+        payload.current_password.get_secret_value(),
+        payload.new_password.get_secret_value(),
         time_cost,
     )
     if not success:
@@ -1239,12 +1087,13 @@ def remove_password(
     token: Annotated[str, Depends(_require_auth)],
 ):
     auth = AuthService(db)
-    success, message = auth.remove_password(payload.current_password)
+    success, message = auth.remove_password(payload.current_password.get_secret_value())
     if not success:
         raise HTTPException(status_code=400, detail=message)
     token_service.revoke_all_tokens()
     clear_all_locks()
     clear_encryption_key()
+    deactivate_authenticated_logging()
     return {"message": message}