declaro_persistum 0.1.0__tar.gz

declaro_persistum-0.1.0/.gitignore

@@ -0,0 +1,56 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+ENV/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+.nox/
+
+# Mypy
+.mypy_cache/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Project specific
+*.db
+*.sqlite
+*-client_wal_index
+
+# Claude
+.claude/
+.hypothesis/
+.env

declaro_persistum-0.1.0/BUG_REPORT_autoindex_stale.md

@@ -0,0 +1,76 @@
+# Bug: Stale `sqlite_autoindex_*_new_1` indexes block table reconstruction
+
+## Summary
+
+Table reconstruction fails with `Parse error: index "sqlite_autoindex_{table}_new_1" already exists` when a second `alter_column` op targets the same table in a single migration batch. The error also persists across migration runs if a previous run left stale autoindexes.
+
+## Reproduction
+
+Schema diff emits two `alter_column` ops for the same table (e.g., `database_routes`). The applier runs them sequentially:
+
+1. First `alter_column` → reconstruction succeeds → `sqlite_autoindex_database_routes_new_1` gets created by SQLite for the temp table, then survives the `ALTER TABLE ... RENAME TO` as a stale artifact
+2. Second `alter_column` → reconstruction fails because `sqlite_autoindex_database_routes_new_1` already exists
+
+## Observed logs (Render production)
+
+```
+INFO:declaro_persistum.abstractions.reconstruction:Successfully reconstructed table 'database_routes'
+INFO:declaro_persistum.applier.turso:Applied: alter_column on database_routes
+ERROR:declaro_persistum.abstractions.reconstruction:Table reconstruction failed for 'database_routes': Parse error: index "sqlite_autoindex_database_routes_new_1" already exists
+WARNING:declaro_persistum.applier.turso:Skipped unsupported operation: alter_column on database_routes: Parse error: index "sqlite_autoindex_database_routes_new_1" already exists
+```
+
+Same pattern for `pricing_invites`, `users`, `subscriptions`, `billing_events`.
+
+## Analysis
+
+The reconstruction code in `reconstruction.py` now uses UUID-based temp names (`_declaro_tmp_{table}_{uuid}`), which should prevent this. However:
+
+1. **Legacy stale indexes**: Previous versions may have used `{table}_new` naming. If those indexes survive in the database, any new reconstruction attempt that creates a temp table will collide if SQLite internally reuses the `_new` suffix pattern.
+
+2. **SQLite autoindex naming**: When SQLite creates a table with UNIQUE or PK constraints, it auto-creates indexes named `sqlite_autoindex_{table}_N`. After `ALTER TABLE temp RENAME TO real`, these autoindexes keep their original name. So a table created as `_declaro_tmp_database_routes_abc123` would get `sqlite_autoindex__declaro_tmp_database_routes_abc123_1`, which is fine. But if there's a stale `sqlite_autoindex_database_routes_new_1` from a legacy `database_routes_new` temp table, it collides.
+
+3. **`_recover_orphaned_tmp_tables`** (migrations.py line 304) cleans up `_declaro_tmp_*` tables but may not clean up legacy `*_new` tables or their stale autoindexes.
+
+## Suggested fixes
+
+### Fix A: Drop stale autoindexes before reconstruction
+
+In `execute_reconstruction_async()`, before creating the temp table, query `sqlite_master` for any `sqlite_autoindex_*_new_*` indexes referencing the target table and drop them:
+
+```python
+# Before creating temp table, clean up stale autoindexes from legacy naming
+cursor = await connection.execute(
+    "SELECT name FROM sqlite_master WHERE type='index' "
+    "AND name LIKE 'sqlite_autoindex_%_new_%'"
+)
+stale = await cursor.fetchall()
+for (idx_name,) in stale:
+    await connection.execute(f'DROP INDEX IF EXISTS "{idx_name}"')
+```
+
+### Fix B: Coalesce same-table operations in the applier
+
+If the differ emits N `alter_column` ops for the same table, the applier should merge them into a single reconstruction pass. This is both safer (one reconstruction instead of N) and faster.
+
+In `TursoApplier.apply()`, group operations by table before the execution loop:
+
+```python
+# Group reconstruction ops by table, apply as single reconstruction
+from itertools import groupby
+# ... merge alter_column ops targeting same table into one combined op
+```
+
+### Fix C: Expand orphan recovery to handle legacy `_new` tables
+
+In `_recover_orphaned_tmp_tables`, also look for `{table}_new` pattern tables and drop them along with their autoindexes.
+
+## Affected versions
+
+Observed in production with declaro_persistum installed from pip (version in multicardz .venv). The UUID temp naming was added to prevent this, but legacy databases still have stale artifacts.
+
+## Impact
+
+- Non-fatal: the second `alter_column` is skipped, and the migration reports partial success
+- But the skipped operations mean schema drift accumulates — the same ops re-run on every deploy
+- Combined with the migration hash not being pushed to cloud (separate issue, fixed in multicardz), this caused 31 migration ops on every single deploy

declaro_persistum-0.1.0/BUG_REPORT_cdc_mvcc_conflict.md

@@ -0,0 +1,54 @@
+# Bug Report: CDC/MVCC conflict breaks cloud sync
+
+## Summary
+
+`TursoPool._initialize()` unconditionally sets `PRAGMA journal_mode = 'mvcc'`, which crashes when cloud sync (CDC replication) is active.
+
+## Error
+
+```
+sync engine operation failed: database error: Parse error: CDC is not supported in MVCC mode
+```
+
+## Reproduction
+
+```python
+pool = await ConnectionPool.turso(
+    "./data/central.db",
+    remote_url="libsql://mc-central-xxx.turso.io",
+    auth_token="eyJ...",
+)
+# Crashes during _initialize() because:
+# 1. connect_async() opens with turso.aio.sync.connect() (CDC mode)
+# 2. _initialize() runs PRAGMA journal_mode = 'mvcc'
+# 3. MVCC conflicts with CDC → exception
+# 4. The exception handler catches it, but the commit() on line 601
+#    triggers the sync engine which fails
+```
+
+## Root Cause
+
+In `pool.py` `TursoPool._initialize()` (line ~584):
+
+```python
+cur = await self._write_holder.conn.execute("PRAGMA journal_mode = 'mvcc'")
+```
+
+This runs regardless of whether `remote_url` is set. When cloud sync is active, the connection uses CDC replication, which is incompatible with MVCC journal mode.
+
+## Suggested Fix
+
+Skip the MVCC pragma when `self._remote_url` is set. CDC replication has its own journaling; MVCC is only useful for local-only connections.
+
+Additionally, the `PRAGMA cache_size` and subsequent `commit()` may also interact badly with CDC — the maintainer should verify.
+
+## Environment
+
+- pyturso 0.5.1 (PyPI, pre-built wheels)
+- Also reproduced with pyturso 0.6.0rc1 (git source)
+- Turso Cloud URL format: `libsql://...turso.io`
+- Deployed on Render (Linux x86_64)
+
+## Impact
+
+Blocks all cloud sync usage. Admin, public, and prod services cannot share a central Turso database.

declaro_persistum-0.1.0/BUG_REPORT_deep_link_migration.md

@@ -0,0 +1,58 @@
+---
+type: bug
+priority: 1
+reporter: downstream consumer
+date: 2026-03-15
+---
+
+# Migration batch aborts on unsupported operation, preventing valid ADD COLUMN
+
+## Problem
+
+`apply_migrations_async()` correctly detects a new column but fails because it batches all 47 detected schema differences into a single transaction. One unsupported operation (e.g., `add_foreign_key` on pyturso/SQLite) causes `MigrationError`, which aborts the entire batch — including the `ADD COLUMN` that would have succeeded independently.
+
+## Observed Behavior
+
+```
+INFO:declaro_persistum.migrations:Loading schema from .../project_tables.py
+INFO:declaro_persistum.migrations:Found 47 schema differences
+INFO:declaro_persistum.migrations:  - add_index on table import_batches
+INFO:declaro_persistum.migrations:  - add_foreign_key on table comments
+  ... (43 more index/FK/alter_column operations) ...
+INFO:declaro_persistum.migrations:  - add_column on table cards        <-- needed
+INFO:declaro_persistum.migrations:  - add_index on table cards
+  ...
+ERROR: Failed to execute operation
+declaro_persistum.exceptions.MigrationError: Failed to execute operation
+```
+
+The pool factory catches the error and continues with the old schema. The `add_column` is never applied.
+
+On subsequent pool creations, the same 47 differences are detected again, the same operation fails again, and the column is never added. The migration is stuck.
+
+## Root Cause
+
+1. The schema defines foreign keys and indexes that pyturso/SQLite cannot apply (e.g., `ADD FOREIGN KEY` is not supported in SQLite `ALTER TABLE`)
+2. These unsupported operations accumulate as permanent "phantom" differences
+3. Every migration attempt re-detects them and fails on them
+4. Valid operations (like `ADD COLUMN`) are bundled in the same batch and never execute
+
+## Expected Behavior
+
+Any of these would fix it:
+1. **Best**: Apply each operation independently — skip failures, continue with the rest, report which succeeded/failed
+2. **Good**: Skip operations known to be unsupported on the current dialect before attempting them
+3. **Acceptable**: Order operations so safe ones (`ADD COLUMN`, `CREATE TABLE`) run first, then attempt riskier ones (`ADD FOREIGN KEY`, `ALTER COLUMN`)
+
+## Steps to Reproduce
+
+1. Create a schema with tables that define foreign keys and indexes
+2. Create a database using pyturso backend — some FK/index operations will silently fail or be ignored
+3. Add a new nullable field to one of the models
+4. Call `apply_migrations_async(pool, "sqlite", schema_path, expand_enums=True)`
+5. Observe: 47 differences detected, batch fails on an FK operation, `ADD COLUMN` never applied
+
+## Environment
+
+- Database backend: pyturso (SQLite-compatible, no `ALTER TABLE ADD FOREIGN KEY` support)
+- Migration call: `apply_migrations_async(pool, "sqlite", schema_path, expand_enums=True)`

declaro_persistum-0.1.0/BUG_REPORT_migrate_remote_connect.md

@@ -0,0 +1,49 @@
+# Bug: migrate-remote uses turso.aio.connect() which doesn't support remote URLs
+
+## Problem
+
+`cmd_migrate_remote` at line 344 calls:
+```python
+conn = await turso.aio.connect(remote_url, auth_token=auth_token)
+```
+
+But `turso.aio.connect()` only accepts a local file path — it has no `auth_token` parameter. The call fails with `Error: open: NotFound`.
+
+## pyturso API
+
+- `turso.aio.connect(database)` — local file only, no auth_token param
+- `turso.aio.sync.connect(path, remote_url, auth_token=...)` — requires local path + remote URL
+
+There is no direct-to-cloud-only connection in pyturso.
+
+## Fix
+
+Use `turso.aio.sync.connect()` with a temp local file:
+
+```python
+import tempfile, os
+
+with tempfile.TemporaryDirectory() as tmpdir:
+    local_path = os.path.join(tmpdir, "migrate_remote.db")
+    conn = await turso.aio.sync.connect(
+        local_path,
+        remote_url=remote_url,
+        auth_token=auth_token,
+    )
+    # pull from cloud to get current state
+    await conn.sync()
+    # ... introspect, diff, apply DDL ...
+    # push changes back to cloud
+    await conn.sync()
+```
+
+## Reproduction
+
+```
+uv run declaro migrate-remote \
+  --remote "libsql://mc-central-adamzwasserman.aws-us-west-2.turso.io" \
+  --token "$CENTRAL_DB_TOKEN" \
+  --schema apps/shared/schema/central_tables.py
+```
+
+Output: `Error: open: NotFound`

declaro_persistum-0.1.0/BUG_REPORT_migrate_remote_data_loss.md

@@ -0,0 +1,83 @@
+# Bug: migrate-remote drops and recreates tables instead of altering them (DATA LOSS)
+
+## Severity: CRITICAL — Production data loss
+
+## Summary
+
+`declaro migrate-remote` emits `create_table` operations for tables that already exist on the cloud DB, instead of `add_column` or `alter_column` for the actual schema diff. This drops all rows in the affected tables.
+
+## Reproduction
+
+1. Run `migrate-remote` to create initial schema on cloud (6 tables created — correct)
+2. Add one column (`invite_token`) to the `users` model in the schema file
+3. Run `migrate-remote` again
+
+### Expected
+
+```
+Found 1 schema difference:
+  - add_column on users (invite_token)
+Applied 1 operation to cloud DB
+```
+
+### Actual
+
+```
+Found 6 schema differences:
+  - create_table on pricing_invites
+  - create_table on subscriptions
+  - create_table on users
+  - create_table on subscription_plans
+  - create_table on database_routes
+  - create_table on billing_events
+
+Applied 6 operations to cloud DB
+```
+
+All 6 tables were dropped and recreated. All rows in all tables were lost.
+
+## Impact
+
+- All user records deleted
+- All database_routes deleted (workspace provisioning broken)
+- All pricing_invites deleted (invite links broken)
+- All subscriptions and billing_events deleted
+- Production login loop caused by missing user records
+
+## Root Cause
+
+The `cmd_migrate_remote` function connects via `turso.aio.sync.connect()` with a temporary local file. It pulls from cloud into the temp file, introspects, diffs, and applies. But the introspection found 0 tables in the temp file — meaning the pull from cloud did not bring the existing tables into the local replica.
+
+Possible causes:
+1. The pull into a fresh temp file doesn't actually sync the cloud state — the temp DB starts empty regardless of pull
+2. The sync connection's pull is a no-op for a brand new local file (no replication frame to start from)
+3. The introspection runs before the pull completes
+
+Because the introspect sees 0 tables locally, the differ compares 0 existing tables vs 6 schema tables and emits 6 `create_table` operations. When these are pushed to cloud, the cloud's existing tables are replaced with empty ones.
+
+## Suggested Fix
+
+Before diffing, verify the introspected table count matches expectations. If the cloud DB should have tables but introspection finds 0, abort with an error rather than proceeding with `create_table` operations that will cause data loss.
+
+Additionally, `create_table` on a table that already exists in the cloud should be rejected or converted to an `alter_table` diff. The applier should never silently drop a table that has rows.
+
+## Workaround
+
+None. Data is lost. Users, routes, invites, and subscriptions must be re-created manually or restored from backups.
+
+## Command Used
+
+```bash
+uv run declaro migrate-remote \
+  --remote "libsql://mc-central-adamzwasserman.aws-us-west-2.turso.io" \
+  --token "$CENTRAL_DB_TOKEN" \
+  --schema apps/shared/schema/central_tables.py
+```
+
+## Timeline
+
+1. 00:27 UTC — First `migrate-remote` run: created 6 tables on cloud (correct, cloud was empty)
+2. 00:32 UTC — App verified working: invite query returned 404 (table exists, token not found)
+3. ~03:20 UTC — Added `invite_token` column to User schema
+4. ~03:20 UTC — Second `migrate-remote` run: recreated all 6 tables (DATA LOSS)
+5. ~03:25 UTC — Login loop: users table empty, database_routes empty, all provisioning fails

declaro_persistum-0.1.0/BUG_REPORT_reconstruction_sync_corruption.md

@@ -0,0 +1,67 @@
+# Bug: Table reconstruction via embedded replica corrupts both local and cloud DBs
+
+## Severity: CRITICAL — Tables silently destroyed, migration reports success
+
+## Summary
+
+When `apply_migrations_async` detects `alter_column` differences on a Turso embedded replica, it reconstructs the table (create temp, copy data, drop original, rename temp). The DROP syncs to cloud but the CREATE/RENAME does not. After the push failure, the connection is "refreshed" from cloud, which now has the table dropped. Result: table is gone from both local and cloud, but the migration reports success.
+
+## Reproduction
+
+1. Create tables on Turso Cloud via `migrate-remote --init` (creates basic tables without NOT NULL constraints, indexes, or foreign keys)
+2. Start an app that uses `ConnectionPool.turso()` with auto-migration enabled
+3. Auto-migration detects `alter_column` differences (e.g., NOT NULL constraints missing from `--init` tables)
+4. Migration reconstructs tables locally (drop + recreate with correct schema)
+5. Push to cloud fails: `sync engine operation failed: database sync engine error: failed to execute sql: Error { message: "SQLite error: no such table: main.<table_name>", code: "SQLITE_UNKNOWN" }`
+
+### Expected
+
+Either:
+- Migration detects that push will fail and skips destructive reconstruction, OR
+- Reconstruction is atomic (drop + create sync together or not at all), OR
+- Migration reports failure when push fails after reconstruction
+
+### Actual
+
+```
+WARNING:declaro_persistum.pool:push after acquire_write commit failed
+  turso.lib.DatabaseError: sync engine operation failed: ...no such table: subscription_plans
+INFO:declaro_persistum.pool:Write holder connection refreshed after migration
+INFO:declaro_persistum.migrations:Successfully applied 21 migrations
+```
+
+Migration reports "Successfully applied 21 migrations" but the tables no longer exist. Subsequent queries fail with `Parse error: no such table: users`.
+
+The cloud DB also loses the tables — the DROP was synced but the CREATE was not.
+
+## Root cause
+
+Table reconstruction involves these steps:
+1. `CREATE TABLE _declaro_tmp_<name>_<hash>` (new schema)
+2. `INSERT INTO _declaro_tmp ... SELECT FROM <original>`
+3. `DROP TABLE <original>`
+4. `ALTER TABLE _declaro_tmp ... RENAME TO <original>`
+5. `CREATE INDEX ...`
+
+The Turso sync engine pushes step 3 (DROP) to cloud but fails on subsequent steps. The `push after acquire_write commit failed` error triggers a connection refresh, which pulls the now-dropped state from cloud, destroying the local table too.
+
+## Compounding factor
+
+This creates a vicious cycle:
+1. `migrate-remote --init` creates basic tables in cloud
+2. App migration sees 28 differences, reconstructs tables, push fails → tables destroyed
+3. Next `migrate-remote --init` recreates them → app destroys them again on startup
+
+## Environment
+
+- declaro-persistum: installed via uv from GitHub
+- pyturso (turso SDK)
+- Turso Cloud embedded replica
+
+## Suggested fixes (pick any)
+
+1. **Wrap reconstruction in a savepoint** and roll back if push fails
+2. **Don't push DDL through sync engine** — use direct HTTP for schema changes
+3. **Detect embedded replica mode** and skip alter_column operations (log warning instead)
+4. **Report failure accurately** — if push fails after DROP, don't report "Successfully applied"
+5. **Make `migrate-remote --init` create full schema** (with NOT NULL, FKs, indexes) so embedded replicas see 0 differences

declaro_persistum-0.1.0/FEATURE_REQUEST_sync_reads.md

@@ -0,0 +1,34 @@
+# Feature Request: Read connections should use sync driver without pull
+
+## Problem
+
+When `TursoPool` has cloud sync enabled (`remote_url` set), the write holder connects via `turso.aio.sync.connect(path, remote_url, auth_token)`. Read connections from `acquire()` currently use `turso.aio.connect(path)` (plain local driver, no remote_url — changed in 106a4ae).
+
+These two pyturso driver types don't share WAL state on the same file. Tables created by the sync write holder during migration are invisible to plain local reads. Result: `no such table` errors for tables that exist.
+
+## Observed behavior
+
+```
+# Write holder (sync driver) creates tables during migration — succeeds
+# Read connection (plain driver) queries those tables — fails
+turso.lib.DatabaseError: Parse error: no such table: pricing_invites
+```
+
+## Requested change
+
+In `TursoPool.acquire()`, create the read connection holder with the same driver type as the write holder — `turso.aio.sync.connect(path, remote_url, auth_token)` — but **skip the pull**. This ensures both sides use the same driver and see the same local state.
+
+```python
+# Current (106a4ae):
+holder = _TursoConnectionHolder(self._database_path)  # plain driver
+
+# Requested:
+holder = _TursoConnectionHolder(self._database_path, self._remote_url, self._auth_token)
+# But do NOT call holder.pull() — just connect and read local state
+```
+
+This may require a flag on `_TursoConnectionHolder` or `connect_async()` to skip the automatic pull that `turso.aio.sync.connect()` may do on connection open. If `turso.aio.sync.connect()` doesn't pull automatically (only on explicit `pull()` call), then simply passing `remote_url` without calling `pull()` should be sufficient.
+
+## Impact
+
+Blocking: public app's invite page returns 503 on every request because `pricing_invites` table is invisible to reads.