MetaList 0.25.0__tar.gz → 0.26.0__tar.gz

metalist-0.26.0/PKG-INFO

@@ -0,0 +1,368 @@
+Metadata-Version: 2.4
+Name: metalist
+Version: 0.26.0
+Summary: A minimalist single-user note-taking app with SSR, fast in-memory tree operations, and efficient sync.
+Project-URL: Homepage, https://metalist.app
+Project-URL: Repository, https://github.com/metalist-app/metalist
+Project-URL: Issues, https://github.com/metalist-app/metalist/issues
+Project-URL: Changelog, https://github.com/metalist-app/metalist/blob/main/CHANGELOG.md
+Requires-Python: <3.15,>=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: anthropic==1.12.1
+Requires-Dist: argon2-cffi==25.1.0
+Requires-Dist: cryptography==50.0.0
+Requires-Dist: fastapi==0.141.1
+Requires-Dist: httpcore==1.0.9
+Requires-Dist: httpx==0.28.1
+Requires-Dist: instructor==1.15.4
+Requires-Dist: latex2mathml==3.81.0
+Requires-Dist: loguru==0.7.3
+Requires-Dist: Mako==1.4.2
+Requires-Dist: mammoth==1.13.0
+Requires-Dist: nh3==0.3.6
+Requires-Dist: olefile==0.47
+Requires-Dist: openai==2.54.0
+Requires-Dist: pillow==13.0.0
+Requires-Dist: pydantic==2.13.4
+Requires-Dist: pygments==2.20.0
+Requires-Dist: pypdf==6.21.4
+Requires-Dist: pypdfium2==5.14.0
+Requires-Dist: python-dotenv==1.2.2
+Requires-Dist: python-multipart==0.0.32
+Requires-Dist: PyYAML==6.0.3
+Requires-Dist: resvg_py==0.5.0
+Requires-Dist: rich==14.3.4
+Requires-Dist: starlette==1.3.1
+Requires-Dist: tqdm==4.70.0
+Requires-Dist: tree-sitter==0.25.2
+Requires-Dist: tree-sitter-javascript==0.25.0
+Requires-Dist: typing_extensions==4.16.0
+Requires-Dist: uvicorn==0.52.0
+Requires-Dist: zxcvbn==4.5.0
+Provides-Extra: dev
+Requires-Dist: hypothesis==6.164.0; extra == "dev"
+Requires-Dist: pytest==9.1.1; extra == "dev"
+Requires-Dist: pytest-cov==7.1.0; extra == "dev"
+Requires-Dist: shockharness==0.26.0; extra == "dev"
+Requires-Dist: mypy==2.4.0; extra == "dev"
+
+# MetaList
+
+A minimalist single-user note-taking app focused on server-side rendering (SSR), fast in-memory tree operations, and efficient sync/diff updates.
+
+## Features
+- Rich text editing (ContentEditable) with image support
+- Drag-and-drop note reordering
+- Real-time content saving
+- Keyboard shortcuts / cheatsheet (press `?` in the app)
+- Linked-list ordering model for efficient reorders
+- Optional password protection + encryption at rest (AES-GCM)
+- Multi-tab search contexts with server-persisted scroll/search state (survives browser restarts)
+- Manual namespace backups/restores to a user-selected backup folder with retention controls
+
+## Changes
+
+What changed in each release: [CHANGELOG.md](https://github.com/metalist-app/metalist/blob/main/CHANGELOG.md).
+
+## Technology Stack
+
+### Backend
+- FastAPI
+- SQLite (via stdlib `sqlite3`) with a guard-aware wrapper (`SafeSession`)
+- Mako templates for SSR
+
+### Frontend
+- Vanilla JavaScript (no framework)
+- HTML5 Drag and Drop API
+- ContentEditable for rich text editing
+- CSS custom properties for theming
+
+### Testing
+- Python/unit tests plus manual regression passes
+
+## Architecture (High Level)
+- Server renders the base page via Mako templates.
+- The browser client drives interaction via `/api2` JSON endpoints.
+- Passwordless notes load into memory at startup; encrypted notes load after login. Rendering/search and ordinary authentication use memory; a read guard restricts intentional SQLite access to explicit windows.
+
+## Security Boundary
+
+Password protection encrypts namespace data at rest. While a namespace is
+unlocked, the server must hold its data-encryption key and decrypted working
+data in process memory, and the browser holds the decrypted content currently
+rendered in the page. A sufficiently privileged local process, debugger,
+administrator, browser extension, or process-memory dump can therefore expose
+an unlocked namespace.
+
+Explicit logout removes the live key and purges decrypted runtime stores, but
+it cannot guarantee forensic overwriting of memory previously allocated by the
+Python or browser runtimes. Protect the host account and operating system, and
+see the detailed [security architecture and threat model](docs/security/README.md).
+
+## Development
+
+### Setup
+For a published one-off run with uv:
+```bash
+uvx metalist
+```
+
+For a persistent uv tool install:
+```bash
+uv tool install metalist
+metalist
+```
+
+For a persistent uv installation, you can update from inside MetaList: type **update** in the menu, open **Version Info**, and choose **Update to …** when a newer release is available. Each browser load or refresh asks the server to check PyPI asynchronously. A reminder-style notice links to Version Info the first time a newer version is discovered; the announced version is remembered in that namespace's client preferences. A slow or failed check does not block the app. Version Info can retry a failed check.
+
+The in-app updater checks the selected release, creates verified backups, and restarts every namespace. Keep Version Info open to follow progress and reload when ready; closing and reopening it in the same page resumes progress. Signing in again may be necessary. Source checkouts and unmanaged installations still check for releases, but show why in-app installation is unavailable. Update job records and logs live under `~/MetaList/update-jobs/` (or the configured data directory). If installation/restart fails after servers stop, inspect the updater output and run `metalist` on the server to restart it; there is no automatic rollback.
+
+You can also update and restart MetaList with one cross-platform command:
+```bash
+metalist update
+```
+The updater checks the installed version against the latest PyPI release first. If MetaList is already current, it reports the installed version and leaves all running namespaces untouched. When an update is available, it checks installer prerequisites and uses uv to install the candidate into a disposable tool environment with the current base Python interpreter. It checks dependencies and starts a temporary namespace to verify HTTP readiness, version, and runtime assets. A failed preflight leaves the current installation and running namespaces unchanged. After preflight succeeds, it stops running namespaces, and creates and verifies a new backup of every namespace before installation can begin. Each archive includes the complete notes/settings database, the attachments database when present, and any legacy search-history database. Locked namespaces are backed up with their encrypted data and key metadata intact, without requiring a password.
+
+MetaList downloads its own checksum-verified uv 0.12.17 archive for the current operating system, architecture, and Linux C library. The verified archive and extracted executable are stored in the operating system's user cache; every update verifies the archive against its pinned upstream SHA-256 and the executable against the verified archive before use. That same executable performs both preflight and final installation. MetaList does not execute or replace the user's global uv. Download, verification, platform-selection, or version failures occur before any namespace is stopped. Already-installed releases retain their own updater code; the standalone repair launcher described in [recovery](docs/security/recovery.md#windows-updater-pe-resource-failure) supplies this managed installer to an older Windows updater.
+
+Backups are saved to `~/MetaList/namespaces/<namespace>/backups/<namespace>-<timestamp>.metalist-backup.tar.gz`; the updater prints each verified path. Existing backups remain unchanged and are not pruned. If any backup fails, the update aborts with the current installation intact; run `metalist` to restart the stopped servers after resolving the failure.
+
+Only after all backups pass does the updater hand off to an external PowerShell process on Windows or `/bin/sh` on macOS/Linux so the installed environment can unlock. It delegates installation to uv, pinning the current base interpreter and the exact tested package/dependency versions, using the cache populated during preflight with network access disabled. It then launches MetaList again, and reports the installed version (for example, `MetaList updated to v0.5.0.`). This protection requires an installed version containing the backup safeguard and applies to `metalist update` and the in-app update action; direct pip/uv install commands do not run it.
+
+For pip, users can run `pip install metalist`. For a non-editable local install from this checkout, use `uv pip install .` or `pip install .` instead of the editable command below.
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install --require-hashes -r requirements/ci.txt
+python -m pip install --no-deps --no-build-isolation -e ".[dev]"
+
+npm install
+```
+
+On Windows, create the same `.venv` with `python -m venv .venv` and activate it with `.venv\Scripts\Activate.ps1` in PowerShell. Run the same `python -m pip` commands after activation. Node dependencies are developer tooling; the application itself does not require Node.
+
+### Run
+
+Persistent data defaults to `~/MetaList`. Set `METALIST_DATA_DIRECTORY` to an absolute directory path before launch to use a separate data root, including namespaces, backups, certificates, logs, and runtime jobs. This is used by release tests to keep their disposable data separate from real notes.
+The installed entrypoint starts or restarts every known namespace, prints their URLs, and exits:
+```bash
+metalist
+```
+For source-checkout compatibility, `python main.py` performs the same orchestration. Use `metalist work`, `python main.py --namespace work`, or `python main.py work` when you want one foreground namespace process.
+
+Shell execution is disabled by default. To enable `@shell` for every namespace
+started by the top-level orchestrator, use either:
+
+```bash
+metalist --enable-shell
+python main.py --enable-shell
+```
+
+This flag is propagated to every namespace child process. The top-level process
+prints a conspicuous shell-enabled banner before its namespace launch results.
+Use `python main.py work --enable-shell` for one foreground namespace. Shell
+routes remain restricted to loopback clients using a loopback request host,
+even when the rest of MetaList is intentionally exposed to a LAN.
+
+`metalist` and explicit single-namespace source runs bind to loopback at `127.0.0.1:8000` by default. This keeps a normal laptop launch off LAN and public interfaces unless remote access is explicitly configured.
+On first startup, MetaList also auto-generates a self-signed TLS pair at `~/MetaList/certs/metalist-cert.pem` and `~/MetaList/certs/metalist-key.pem`, then enables HTTPS on the same bind host at port `8443`. If you already have real PEM files, point `METALIST_TLS_CERT` and `METALIST_TLS_KEY` at them instead. Set `METALIST_AUTO_GENERATE_TLS=0` only if you explicitly want HTTP-only startup.
+
+Database selection:
+- No explicit namespace on a single-namespace launch: `~/MetaList/namespaces/default/default.metalist.db`
+- `--namespace work` or `METALIST_NAMESPACE=work`: `~/MetaList/namespaces/work/work.metalist.db`
+- The related files DB is derived automatically, so `namespaces/work/work.metalist.db` uses `namespaces/work/work.metalist.files.db`
+- Remembered launch ports are stored as plaintext metadata inside each namespace's main `*.metalist.db`
+- Launch precedence is: explicit CLI flags > env vars > saved namespace profile; if a namespace has no saved profile, launch it once with explicit ports or configure ports from the UI
+- Backups stay beside the namespace data under `~/MetaList/namespaces/work/backups/` and use one archive per snapshot with filenames like `work-<timestamp>.metalist-backup.tar.gz`
+- The Backup Settings modal targets one user-selected backup folder and can include multiple namespaces in a single run
+- Restoring `work` into `work` is the normal overwrite path; importing a backup under a different namespace name can create a new target namespace with automatically selected conflict-free ports.
+
+Useful env flags:
+- `CRASH_SERVER_ON_FAIL=1` (default): fail-fast on validation errors
+- `API_PREFIX=/api2`: override API prefix (client assumes `/api2` by default)
+- `METALIST_NAMESPACE=work`: select `~/MetaList/namespaces/work/work.metalist.db`
+- `METALIST_HOST=127.0.0.1` (default): bind the main app to a specific interface; use a LAN IP or `0.0.0.0` only for intentional remote access
+- `METALIST_ALLOWED_HOSTS=notes.example.com,192.168.1.20`: comma-separated public/LAN hostnames accepted in HTTP `Host` headers; loopback aliases and a specific non-wildcard `METALIST_HOST` are accepted automatically
+- `METALIST_PORT=8000` (default): bind the main app to a different port
+- `METALIST_HTTPS_PORT=8443`: override the HTTPS port when TLS is enabled
+- `METALIST_TLS_CERT=/path/to/fullchain.pem` + `METALIST_TLS_KEY=/path/to/privkey.pem`: override TLS paths
+- `METALIST_AUTO_GENERATE_TLS=0`: disable automatic creation of the default self-signed TLS pair
+- default TLS paths: `~/MetaList/certs/metalist-cert.pem` and `~/MetaList/certs/metalist-key.pem`
+- `METALIST_FORWARDED_ALLOW_IPS=127.0.0.1,::1` (default): trust proxy headers only from those reverse-proxy IPs
+- `--enable-shell`: opt in to local `@shell` execution for this launch; shell routes require a loopback client and loopback request host, and the capability is never persisted in namespace data
+
+### Remote Access / HTTPS
+LAN or VPN access must be enabled explicitly. Prefer binding the machine's specific LAN address, which also adds that address to the accepted-host set:
+```bash
+METALIST_HOST=192.168.1.20 metalist
+```
+On a fresh machine, that first launch also creates the default TLS cert pair automatically. Then open either `http://192.168.1.20:8000` or `https://192.168.1.20:8443` from the other machine.
+
+Starting with 0.7.1, an explicit `METALIST_HOST` or `METALIST_ALLOWED_HOSTS`
+configuration is saved in `~/MetaList/network-settings.json` (under
+`METALIST_DATA_DIRECTORY` when overridden). Subsequent source launches, plain
+`metalist` launches, and updates reuse it. Environment values override and update
+the corresponding saved settings. This file stores only the bind address and
+allowed hosts; namespace ports remain in their launch profiles, and shell access
+still requires `--enable-shell` on each launch. Invalid settings stop startup
+with an error. Fresh installations continue to bind to localhost by default.
+To switch back to local-only access, run `METALIST_HOST=127.0.0.1 metalist` once.
+Older installed versions do not read this file; keep supplying their LAN settings
+until upgrading to 0.7.1 or later.
+
+To keep `@shell` available on the host laptop while allowing another laptop to
+use normal MetaList features, bind both interfaces and explicitly allow the LAN
+address:
+
+```bash
+METALIST_HOST=0.0.0.0 \
+METALIST_ALLOWED_HOSTS=10.0.0.31 \
+metalist --enable-shell
+```
+
+Use `http://127.0.0.1:<namespace-http-port>` on the host laptop when running
+`@shell`. Other devices may use `https://10.0.0.31:<namespace-https-port>`, but
+their shell start/status requests receive `403`.
+
+Namespaced launch example:
+```bash
+metalist --namespace work --port 8001
+```
+This starts a separate process backed by `~/MetaList/namespaces/work/work.metalist.db` on `http://127.0.0.1:8001`.
+Its backup snapshots live under `~/MetaList/namespaces/work/backups/` with filenames like `work-<timestamp>.metalist-backup.tar.gz`. New backups are versioned `.tar.gz` workspace archives; legacy `.bak` backups remain restorable.
+
+After you launch a namespace once with explicit ports, MetaList remembers them in that namespace's main DB, so later you can use the shorthand:
+```bash
+metalist work
+```
+and MetaList will reuse the saved HTTP / HTTPS ports for `work`. The same applies to the default namespace: `metalist` will reuse the saved default-namespace profile.
+
+Equivalent explicit launch, if you want it:
+```bash
+METALIST_HOST=0.0.0.0 \
+METALIST_ALLOWED_HOSTS=192.168.1.20 \
+METALIST_PORT=8000 \
+METALIST_HTTPS_PORT=8443 \
+metalist
+```
+From the other machine, open `https://<laptop-ip>:8443`.
+
+If you already have a real certificate and key, use the same dual-listener flow:
+```bash
+METALIST_HOST=0.0.0.0 \
+METALIST_ALLOWED_HOSTS=192.168.1.20 \
+METALIST_PORT=8000 \
+METALIST_HTTPS_PORT=8443 \
+METALIST_TLS_CERT=/path/to/fullchain.pem \
+METALIST_TLS_KEY=/path/to/privkey.pem \
+metalist
+```
+
+If you want to rotate or regenerate the default self-signed pair manually, the helper script is still available:
+```bash
+generate-lan-cert.sh
+```
+
+When HTTPS is enabled:
+- remote HTTP requests to `http://<laptop-ip>:8000` are redirected to HTTPS
+- localhost HTTP requests still stay on plain `http://127.0.0.1:8000` so the laptop can keep using the non-TLS port
+
+If TLS is terminated by a reverse proxy on the same machine instead, keep MetaList on loopback and let the proxy forward to it:
+```bash
+METALIST_HOST=127.0.0.1 \
+METALIST_ALLOWED_HOSTS=notes.example.com \
+METALIST_PORT=8000 \
+METALIST_FORWARDED_ALLOW_IPS=127.0.0.1,::1 \
+metalist
+```
+
+The reverse proxy must preserve the browser-facing `Host` header and set
+`X-Forwarded-Proto`. MetaList never trusts `X-Forwarded-Host`; forwarded
+scheme/client metadata is accepted only from `METALIST_FORWARDED_ALLOW_IPS`.
+Do not widen that list beyond the actual proxy addresses.
+
+### Legacy Import
+`convert-from-legacy.py` replaces the SQLite database referenced by `app.config.DATABASE_URL`, clears its files sidecar, and imports notes from a legacy JSON export.
+
+This is destructive. It deletes the existing notes DB and related files DB before rebuilding the namespace.
+
+An export the old MetaList encrypted is read too: the importer asks for the old password, then offers it for the converted namespace. Delete old exports once the conversion works; the old encryption is weak.
+
+It also reads the old MetaList's own file instead of an export, and never changes it:
+```bash
+convert-from-legacy.py --namespace work --legacy-database ~/MetaList/metalist.db
+```
+
+Most people never need the command: a newly installed MetaList that finds the old MetaList's file in its data folder offers its notes when it opens (`docs/ui/legacy-import.md`).
+
+Example usage:
+```bash
+convert-from-legacy.py --input /path/to/legacy-export.json
+```
+
+Target a namespaced database during import:
+```bash
+convert-from-legacy.py --namespace work --input /path/to/legacy-export.json
+```
+
+If `--namespace`, `--port`, or `--https-port` are omitted, the import script prompts for them and saves the resulting launch profile inside the target namespace DB. That means a one-time import into `work` can immediately seed later shorthand launches like `metalist work`.
+
+Namespace startup allows 120 seconds per child, reports progress every five seconds, and stops/reaps a failed child instead of leaving it running after reporting failure. Set `METALIST_STARTUP_TIMEOUT_SECONDS` to a positive number of seconds to allow more time on a heavily loaded machine. A crashed child still fails immediately. This is a startup allowance, not a guarantee that any machine will finish within two minutes.
+
+The preflight is a disposable empty namespace; it cannot guarantee startup against every real database or prevent a later disk/process failure. If installation succeeds but real namespace startup fails, the new package remains installed: resolve the reported startup issue and run `metalist` again. No automatic code/database rollback is attempted, and backups remain unchanged.
+
+### Publishing
+For the real user-facing install flow:
+```bash
+uvx metalist
+# or:
+uv tool install metalist
+metalist
+```
+
+This repo now packages itself under the PyPI distribution name `metalist`. Current releases support Python 3.11 through 3.14.
+
+Recommended release path:
+1. In the existing PyPI project `metalist`, configure GitHub Trusted Publishing for `metalist-app/metalist` and the workflow file `.github/workflows/publish-pypi.yml`.
+2. Develop and review the candidate on a feature branch with focused checks. Merge it into `main` and push `main` once. This runs the full `Publish to PyPI` validation matrix for that exact commit. GitHub-hosted Windows, macOS, and Linux runners install the built wheel and test application startup outside the source checkout on the supported Python endpoints, 3.11 and 3.14. Every installed-package matrix job also exercises the real `metalist update` path with uv, backups, interpreter preservation, and two-namespace restart against a local release index. The old-version fixture carries the candidate updater so this validates the mechanism being shipped; it does not retroactively fix older installed updaters. You do not need those operating systems locally. Archive verification checks runtime files, including agent Markdown resources, against both the wheel and source distribution.
+3. Wait for the exact-commit main release matrix to pass, then run `.venv/bin/python scripts/release.py X.Y.Z` to create and push the tag. Any intervening change requires a new main validation run before tagging.
+4. The tag job independently verifies every main-job conclusion and reuses the wheel and source distribution from that run for GitHub Trusted Publishing; it does not repeat the matrix. Manual dispatch cannot publish. The release driver compares PyPI hashes with the tested artifact and checks a clean public install. Users can then run it with `uvx metalist`, install it persistently with `uv tool install metalist`, or install it with `pip install metalist`.
+
+If `--input` is omitted, a file picker opens (when `tkinter` and its native `_tkinter` component are available).
+Notes tagged with `@implies` are converted into ontology rules and are not imported as notes. Legacy rules that are invalid under the current ontology grammar are reported and skipped while valid rules continue importing.
+
+### Run Tests
+
+Python/unit test examples:
+```bash
+source .venv/bin/activate
+.venv/bin/pytest
+node --test tests/unit/*.mjs
+.venv/bin/python -c "from pathlib import Path; import main; main._run_startup_sanity_gates(repo_root=Path.cwd())"
+```
+
+Source-checkout development launches can enable the mandatory Python and
+JavaScript sanity gates with an ignored repository-root `.env` file:
+
+```dotenv
+METALIST_ENVIRONMENT=development
+```
+
+`main.py` prints the resolved environment at startup. An explicit process
+environment value takes precedence over `.env`; only `development` and
+`production` are accepted. When the variable and `.env` file are absent,
+MetaList runs in production mode and skips the source sanity gates.
+
+`TEST_MODE=1` and `POST /api2/test/reset` still exist for deterministic browser automation if we decide to add a new harness later, but Cypress is not part of the current workflow.
+
+### Diagrams
+Render Mermaid diagrams to PNGs:
+```bash
+npm run render-diagrams
+```
+
+
+## Maintenance and recovery
+
+See the [dependency and release procedure](docs/security/supply-chain.md), [recovery runbook](docs/security/recovery.md), and [current test/implementation map](docs/testing/coverage-map.md). The dated [August code review](CODE_REVIEW.md) is historical.