activerecord-beagle-turso 0.1.5 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +28 -0
- data/lib/active_record/connection_adapters/beagle_turso/errors.rb +18 -0
- data/lib/active_record/connection_adapters/beagle_turso/migration_push.rb +207 -0
- data/lib/active_record/connection_adapters/beagle_turso_adapter.rb +121 -6
- data/lib/activerecord/beagle_turso/version.rb +1 -1
- data/lib/activerecord-beagle-turso.rb +2 -0
- metadata +4 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 51cf88616ac49583385f9d993560cc4f5fb53169707dec16c68b3c37f6d60884
|
|
4
|
+
data.tar.gz: 21732cc614964877aeca0da6e34d066cfcfb2061747ac68f9eb2803abff60d1a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: fe0832ec4ae1acc466934cb0ed1a0c47f2de5e5d5f86b47091ffbe1c5e8f77c86ee50534808d2cf7dbef318aa526cd0026146ac367d1c3f56275723d8630ccc3
|
|
7
|
+
data.tar.gz: bf86e7af654df142cd722228c9245bd5456b6cfd9b2de5a43ae9067e018fee5491e17005896eec7f693ef0fc9e8d9bd9ac1d5791033bfff579879562fd69d1b9
|
data/README.md
CHANGED
|
@@ -67,6 +67,8 @@ An optional `bootstrap_if_empty:` (default `true`) controls whether a brand
|
|
|
67
67
|
new/empty local file is populated from the remote on first open — see
|
|
68
68
|
`Beagle::Turso::Database.open`.
|
|
69
69
|
|
|
70
|
+
The sync engine's own tables (`turso_cdc`, `turso_cdc_version`, `turso_sync_last_change_id`) are hidden from `connection.tables`/`data_sources` and therefore from schema dumps and table-truncating tooling.
|
|
71
|
+
|
|
70
72
|
### Durability note: writes commit on-sync, not synchronously
|
|
71
73
|
|
|
72
74
|
Writes go to the **local** file immediately, as part of the normal
|
|
@@ -123,6 +125,18 @@ A recurring `SyncJob` against a local-only connection will fail every run
|
|
|
123
125
|
with `NotSyncedError` — only add the recurring entry for environments whose
|
|
124
126
|
`database.yml` connection actually carries `remote_url:`/`auth_token:`.
|
|
125
127
|
|
|
128
|
+
## Synced replicas: what's guarded
|
|
129
|
+
|
|
130
|
+
Three bugs in the Turso 0.8.3 sync engine affect a synced replica (`remote_url:` + `auth_token:`). Bug B is [tursodatabase/turso#9551](https://github.com/tursodatabase/turso/issues/9551); bugs A and C are being reported upstream. This adapter guards against each. Local-only connections are unchanged.
|
|
131
|
+
|
|
132
|
+
- **`rename_table` is refused (bug C).** The engine does not capture `ALTER TABLE ... RENAME TO` or `RENAME COLUMN` for sync, so the remote keeps the old name and every later write to the table is lost on push. `rename_table`, and raw `execute("ALTER TABLE ... RENAME ...")`, raise `ActiveRecord::ConnectionAdapters::BeagleTurso::UnsupportedOnSyncedReplica`. Instead, create a new table, copy the data, and drop the old one; or apply the rename directly on the remote and pull, with writes to that table paused. `rename_column` is allowed: Rails 8.1's SQLite adapter implements it as a table copy, not a native `RENAME COLUMN`.
|
|
133
|
+
- **Insert ids come from `RETURNING` (bug B).** On an MVCC replica with change capture (every synced replica of a `--tursodb` remote) the engine's `last_insert_rowid` returns an internal change id. Model creates already use `INSERT ... RETURNING "id"`. Plain `INSERT`s (including `WITH ... INSERT`, and `connection.insert` / `exec_insert` on a table without a named primary key) now run as `INSERT ... RETURNING rowid` and the id is read from the result. Override with `insert_returning_rowid: true|false` in `database.yml`.
|
|
134
|
+
- **Migrations push first and last (bug A).** Unpushed row writes to a table that is altered or dropped before the next push are skipped by the remote and then lost on pull. When a migration is pending (or the database is new), `db:migrate`, `db:prepare` and their `:<primary>`-namespaced forms push before and after. `db:migrate:up/down/redo`, `db:rollback` and `db:forward` (also with the `:<primary>` suffix) always push, since they always run a migration. With nothing pending, `db:prepare` (which Rails 8 runs on every container boot) does no push and no network I/O.
|
|
135
|
+
- If the push before fails, the task aborts without migrating.
|
|
136
|
+
- A migration that copies a table (`change_column*`, `remove_column`, `add_foreign_key`, `add_timestamps`, ...) makes the first push after it fail once, although the remote ends up with everything. The push after is therefore retried once: if the retry works you only get a warning. If it fails twice the task fails, telling you the schema change exists locally and the push failed.
|
|
137
|
+
- Set `BEAGLE_TURSO_SKIP_MIGRATION_PUSH=1` to skip both pushes, with a warning (unpushed writes to altered tables may be lost).
|
|
138
|
+
- Only the primary database is guarded. `db:schema:load`, `db:setup` and `db:reset` do not work with this adapter (Rails raises `DatabaseNotSupported`); use migrations.
|
|
139
|
+
|
|
126
140
|
## Test coverage
|
|
127
141
|
|
|
128
142
|
This gem's own spec suite (`spec/`, run via `bundle exec rspec`) is 35
|
|
@@ -226,6 +240,20 @@ this gem.
|
|
|
226
240
|
each sync — pick a sync cadence accordingly, or run Solid Queue in a separate
|
|
227
241
|
process.
|
|
228
242
|
|
|
243
|
+
## Known engine issue: self-referencing foreign keys
|
|
244
|
+
|
|
245
|
+
Turso's engine has open bugs with foreign keys that reference their own table
|
|
246
|
+
(such as `parent_id`): a multi-row `delete_all`, `update_all` or `DROP TABLE` that SQLite accepts can fail with
|
|
247
|
+
`FOREIGN KEY constraint failed`, and `ON DELETE SET NULL` / `CASCADE` can leave
|
|
248
|
+
rows behind or orphaned
|
|
249
|
+
([tursodatabase/turso#8713](https://github.com/tursodatabase/turso/issues/8713),
|
|
250
|
+
[#9279](https://github.com/tursodatabase/turso/issues/9279),
|
|
251
|
+
[#9293](https://github.com/tursodatabase/turso/issues/9293),
|
|
252
|
+
[#9294](https://github.com/tursodatabase/turso/issues/9294)). Foreign keys
|
|
253
|
+
between two different tables are unaffected. Workaround: avoid `ON DELETE`
|
|
254
|
+
actions on self-referencing foreign keys, and delete children before parents,
|
|
255
|
+
one level at a time.
|
|
256
|
+
|
|
229
257
|
## License
|
|
230
258
|
|
|
231
259
|
[MIT](https://opensource.org/licenses/MIT).
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ActiveRecord
|
|
4
|
+
module ConnectionAdapters
|
|
5
|
+
module BeagleTurso
|
|
6
|
+
# Raised instead of running a schema change that the Turso sync engine
|
|
7
|
+
# cannot replicate. On a synced replica, `ALTER TABLE ... RENAME TO` and
|
|
8
|
+
# `RENAME COLUMN` are not captured for sync (engine 0.8.3), so the remote
|
|
9
|
+
# never sees them and every later write to the renamed table or column is
|
|
10
|
+
# lost on push.
|
|
11
|
+
class UnsupportedOnSyncedReplica < ::ActiveRecord::ActiveRecordError; end
|
|
12
|
+
|
|
13
|
+
# Raised by the db:migrate family of rake tasks when pending writes could
|
|
14
|
+
# not be pushed to Turso before (or after) a migration.
|
|
15
|
+
class MigrationPushError < ::ActiveRecord::ActiveRecordError; end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_record/connection_adapters/beagle_turso/errors"
|
|
4
|
+
require "active_record/connection_adapters/beagle_turso/sync_manager"
|
|
5
|
+
|
|
6
|
+
module ActiveRecord
|
|
7
|
+
module ConnectionAdapters
|
|
8
|
+
module BeagleTurso
|
|
9
|
+
# Pushes a synced replica to Turso before and after schema-changing rake
|
|
10
|
+
# tasks, when (and only when) a migration is actually going to run.
|
|
11
|
+
#
|
|
12
|
+
# Why: the sync engine replays each unpushed row change against the
|
|
13
|
+
# CURRENT local schema when it pushes. If the table was dropped, or lost
|
|
14
|
+
# columns, since the write, the remote skips the statement, commits the
|
|
15
|
+
# rest of the batch anyway, and the next pull resets the replica to the
|
|
16
|
+
# remote -- the write is gone on both sides. Pushing immediately before
|
|
17
|
+
# the migration leaves nothing pending to be mis-replayed; pushing again
|
|
18
|
+
# afterwards publishes the new schema. (A copy-table migration on a table
|
|
19
|
+
# with rows makes that second push fail once, spuriously, for the same
|
|
20
|
+
# reason; the data is all on the remote and a retry succeeds.)
|
|
21
|
+
#
|
|
22
|
+
# Local-only connections are never touched.
|
|
23
|
+
module MigrationPush
|
|
24
|
+
SKIP_ENV = "BEAGLE_TURSO_SKIP_MIGRATION_PUSH"
|
|
25
|
+
|
|
26
|
+
# Pushes only when a migration is pending (or the database is not
|
|
27
|
+
# initialised): db:prepare runs on every container boot in Rails 8 and
|
|
28
|
+
# must not need the network when there is nothing to do.
|
|
29
|
+
CHECKED_TASKS = %w[db:migrate db:prepare].freeze
|
|
30
|
+
|
|
31
|
+
# Tasks that always run a specific migration, so always push.
|
|
32
|
+
ALWAYS_TASKS = %w[
|
|
33
|
+
db:migrate:up db:migrate:down db:migrate:redo db:rollback db:forward
|
|
34
|
+
].freeze
|
|
35
|
+
|
|
36
|
+
# db:schema:load has no effect with this adapter (Rails raises
|
|
37
|
+
# DatabaseNotSupported for it) but is listed so that it is guarded if
|
|
38
|
+
# that ever changes.
|
|
39
|
+
SCHEMA_TASKS = %w[db:schema:load].freeze
|
|
40
|
+
|
|
41
|
+
# Prepended onto Rake::Task so it works whatever order the Railtie and
|
|
42
|
+
# ActiveRecord's own rake tasks load in.
|
|
43
|
+
module TaskHook
|
|
44
|
+
def execute(args = nil)
|
|
45
|
+
if name.start_with?("db:") && MigrationPush.guarded?(name)
|
|
46
|
+
MigrationPush.around(name) { super }
|
|
47
|
+
else
|
|
48
|
+
super
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
class << self
|
|
54
|
+
def install!
|
|
55
|
+
return if ::Rake::Task.ancestors.include?(TaskHook)
|
|
56
|
+
::Rake::Task.prepend(TaskHook)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# "db:migrate", "db:migrate:primary", "db:rollback:primary", ...
|
|
60
|
+
# The suffix is the PRIMARY config's own name, never hard-coded.
|
|
61
|
+
def guarded?(task_name)
|
|
62
|
+
base = base_task(task_name)
|
|
63
|
+
base && (CHECKED_TASKS + ALWAYS_TASKS + SCHEMA_TASKS).include?(base)
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def base_task(task_name)
|
|
67
|
+
all = CHECKED_TASKS + ALWAYS_TASKS + SCHEMA_TASKS
|
|
68
|
+
return task_name if all.include?(task_name)
|
|
69
|
+
|
|
70
|
+
name = primary_config&.name
|
|
71
|
+
return unless name && task_name.end_with?(":#{name}")
|
|
72
|
+
|
|
73
|
+
base = task_name.delete_suffix(":#{name}")
|
|
74
|
+
base if all.include?(base)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def around(task_name)
|
|
78
|
+
return yield if depth.positive? # nested task: the outermost one guards
|
|
79
|
+
|
|
80
|
+
config = primary_config
|
|
81
|
+
return yield unless config && synced_config?(config.configuration_hash)
|
|
82
|
+
|
|
83
|
+
if skip?
|
|
84
|
+
warn "beagle_turso: WARNING: #{SKIP_ENV}=1 -- NOT pushing pending writes to Turso around " \
|
|
85
|
+
"#{task_name}. Unpushed writes to tables altered by this task may be lost."
|
|
86
|
+
return yield
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
existing = local_database_exists?(config.configuration_hash)
|
|
90
|
+
if CHECKED_TASKS.include?(base_task(task_name)) && existing
|
|
91
|
+
return yield unless migration_pending?
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
# A replica that does not exist yet has nothing local to push.
|
|
95
|
+
push_before! if existing
|
|
96
|
+
self.depth = 1
|
|
97
|
+
begin
|
|
98
|
+
result = yield
|
|
99
|
+
ensure
|
|
100
|
+
self.depth = 0
|
|
101
|
+
end
|
|
102
|
+
push_after!
|
|
103
|
+
result
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def primary_config
|
|
107
|
+
::ActiveRecord::Base.configurations.find_db_config(rails_env)
|
|
108
|
+
rescue StandardError
|
|
109
|
+
nil
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
def synced_config?(hash)
|
|
113
|
+
hash[:adapter].to_s == "beagle_turso" && !hash[:remote_url].to_s.empty? && !hash[:auth_token].to_s.empty?
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def skip?
|
|
117
|
+
ENV[SKIP_ENV] == "1"
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
private
|
|
121
|
+
|
|
122
|
+
def rails_env
|
|
123
|
+
defined?(::Rails) && ::Rails.respond_to?(:env) ? ::Rails.env.to_s : (ENV["RAILS_ENV"] || "development")
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
def local_database_exists?(hash)
|
|
127
|
+
path = hash[:database].to_s
|
|
128
|
+
return false if path.empty? || path == ":memory:"
|
|
129
|
+
|
|
130
|
+
root = defined?(::Rails) && ::Rails.respond_to?(:root) ? ::Rails.root : nil
|
|
131
|
+
File.exist?(root ? File.expand_path(path, root.to_s) : path)
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
# True when a migration is pending, or the schema is not initialised
|
|
135
|
+
# (no schema_migrations table reads as everything pending). Any
|
|
136
|
+
# trouble answering counts as pending: the guard then runs.
|
|
137
|
+
def migration_pending?
|
|
138
|
+
connection.pool.migration_context.needs_migration?
|
|
139
|
+
rescue StandardError
|
|
140
|
+
true
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
# Re-resolved on every use: multi-database tasks swap connection pools
|
|
144
|
+
# (with_temporary_pool), so a connection held across the task could
|
|
145
|
+
# belong to a pool that is gone. lease_connection does not raise under
|
|
146
|
+
# permanent_connection_checkout = :disallowed, as .connection does.
|
|
147
|
+
def connection
|
|
148
|
+
::ActiveRecord::Base.lease_connection
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
def push_before!
|
|
152
|
+
::Beagle::Turso::SyncManager.push!(connection)
|
|
153
|
+
rescue StandardError => e
|
|
154
|
+
raise MigrationPushError.new(
|
|
155
|
+
"beagle_turso: could not push pending writes to Turso before migrating (#{describe(e)}). " \
|
|
156
|
+
"Fix sync first, or set #{SKIP_ENV}=1 to migrate anyway (unpushed writes to altered tables " \
|
|
157
|
+
"may be lost)."
|
|
158
|
+
), cause: nil
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# A copy-table migration on a table with rows (change_column*,
|
|
162
|
+
# remove_column, add_foreign_key, add_timestamps, ...) makes the first
|
|
163
|
+
# push after it fail with a parse error even though the remote ends up
|
|
164
|
+
# with everything (sync engine bug A); the second push succeeds. So
|
|
165
|
+
# retry once before reporting a failure.
|
|
166
|
+
def push_after!
|
|
167
|
+
::Beagle::Turso::SyncManager.push!(connection)
|
|
168
|
+
rescue StandardError => first
|
|
169
|
+
begin
|
|
170
|
+
::Beagle::Turso::SyncManager.push!(connection)
|
|
171
|
+
warn "beagle_turso: WARNING: the push after the migration failed once (#{describe(first)}) and " \
|
|
172
|
+
"succeeded on retry. This is expected after a migration that copies a table (sync engine " \
|
|
173
|
+
"quirk); no data was lost."
|
|
174
|
+
rescue StandardError => second
|
|
175
|
+
raise MigrationPushError.new(
|
|
176
|
+
"beagle_turso: the schema change was applied to the local replica, but pushing it to Turso " \
|
|
177
|
+
"failed twice (#{describe(first)}; then #{describe(second)}). The remote may not have the " \
|
|
178
|
+
"new schema yet: check sync, then run Beagle::Turso::SyncManager.push!."
|
|
179
|
+
), cause: nil
|
|
180
|
+
end
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
def describe(error)
|
|
184
|
+
message = error.message
|
|
185
|
+
message = ::Beagle::Turso::Redaction.message(message) if defined?(::Beagle::Turso::Redaction)
|
|
186
|
+
prefix = error.class.name.to_s.end_with?("OpenError") ? "could not open the synced replica: " : ""
|
|
187
|
+
"#{error.class}: #{prefix}#{message}"
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
def depth = Thread.current[:beagle_turso_migration_push_depth].to_i
|
|
191
|
+
def depth=(value)
|
|
192
|
+
Thread.current[:beagle_turso_migration_push_depth] = value
|
|
193
|
+
end
|
|
194
|
+
end
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
if defined?(::Rails::Railtie)
|
|
198
|
+
class Railtie < ::Rails::Railtie
|
|
199
|
+
rake_tasks do
|
|
200
|
+
require "rake"
|
|
201
|
+
MigrationPush.install!
|
|
202
|
+
end
|
|
203
|
+
end
|
|
204
|
+
end
|
|
205
|
+
end
|
|
206
|
+
end
|
|
207
|
+
end
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "active_record"
|
|
4
4
|
require "active_record/connection_adapters/sqlite3_adapter"
|
|
5
5
|
require "beagle/turso"
|
|
6
|
+
require "active_record/connection_adapters/beagle_turso/errors"
|
|
6
7
|
|
|
7
8
|
module ActiveRecord
|
|
8
9
|
module ConnectionAdapters
|
|
@@ -59,6 +60,20 @@ module ActiveRecord
|
|
|
59
60
|
INSERT_REGEX = /\bINSERT\b/i
|
|
60
61
|
private_constant :INSERT_REGEX
|
|
61
62
|
|
|
63
|
+
# String literals, quoted identifiers and comments: where words like
|
|
64
|
+
# RETURNING or RENAME are data, not syntax. Masked before any keyword
|
|
65
|
+
# matching (same approach as sequel-beagle-turso).
|
|
66
|
+
NON_CODE = %r{'(?:[^']|'')*'|"(?:[^"]|"")*"|`(?:[^`]|``)*`|\[[^\]]*\]|--[^\n]*|/\*.*?\*/}m
|
|
67
|
+
private_constant :NON_CODE
|
|
68
|
+
|
|
69
|
+
# `ALTER TABLE <t> RENAME ...`, tested on masked SQL.
|
|
70
|
+
RENAME_REGEX = /\A\s*ALTER\s+TABLE\s+\S+\s+RENAME\b/i
|
|
71
|
+
private_constant :RENAME_REGEX
|
|
72
|
+
|
|
73
|
+
# INSERT / REPLACE, optionally behind a WITH clause, tested on masked SQL.
|
|
74
|
+
INSERT_STATEMENT_REGEX = /\A\s*(?:(?:INSERT|REPLACE)\b|WITH\b[\s\S]*\)\s*(?:INSERT|REPLACE)\b)/i
|
|
75
|
+
private_constant :INSERT_STATEMENT_REGEX
|
|
76
|
+
|
|
62
77
|
class << self
|
|
63
78
|
# Open a beagle-turso Database (synced remote when +remote_url+ is
|
|
64
79
|
# present, otherwise local/in-memory) and wrap it so the SQLite3Adapter
|
|
@@ -77,6 +92,10 @@ module ActiveRecord
|
|
|
77
92
|
end
|
|
78
93
|
Client.new(database)
|
|
79
94
|
end
|
|
95
|
+
|
|
96
|
+
def synced_config?(config)
|
|
97
|
+
!config[:remote_url].to_s.empty? && !config[:auth_token].to_s.empty?
|
|
98
|
+
end
|
|
80
99
|
end
|
|
81
100
|
|
|
82
101
|
# Adapts a Beagle::Turso::Database + its connected Connection to the subset
|
|
@@ -205,7 +224,36 @@ module ActiveRecord
|
|
|
205
224
|
end
|
|
206
225
|
end
|
|
207
226
|
|
|
227
|
+
# Refused on a synced replica: the sync engine (turso 0.8.3) does not
|
|
228
|
+
# capture `ALTER TABLE ... RENAME TO`, so the remote never gets the new
|
|
229
|
+
# name and every later write to the table is lost on push.
|
|
230
|
+
# (rename_column is allowed: Rails' SQLite adapter implements it as a
|
|
231
|
+
# copy-table, not a native RENAME COLUMN.)
|
|
232
|
+
def rename_table(table_name, new_name, **options)
|
|
233
|
+
refuse_rename!("rename_table(#{table_name.inspect}, #{new_name.inspect})")
|
|
234
|
+
super
|
|
235
|
+
end
|
|
236
|
+
|
|
237
|
+
# True when this connection is a synced replica, decided from the config
|
|
238
|
+
# exactly as new_client decides it. Deliberately does NOT touch
|
|
239
|
+
# raw_connection: the public accessor disables lazy transactions and marks
|
|
240
|
+
# the connection dirty, which would make every local-only app pay for it.
|
|
241
|
+
def synced_replica?
|
|
242
|
+
self.class.synced_config?(@config)
|
|
243
|
+
end
|
|
244
|
+
|
|
208
245
|
private
|
|
246
|
+
def refuse_rename!(what)
|
|
247
|
+
return unless synced_replica?
|
|
248
|
+
|
|
249
|
+
raise BeagleTurso::UnsupportedOnSyncedReplica,
|
|
250
|
+
"beagle_turso: #{what} is not supported on a synced replica. The Turso sync engine does not " \
|
|
251
|
+
"replicate ALTER TABLE ... RENAME, so the remote would keep the old name and every later " \
|
|
252
|
+
"write to it would be lost on push. Instead: create a new table, copy the data across, then " \
|
|
253
|
+
"drop the old one; or apply the rename directly on the remote database, then pull (with " \
|
|
254
|
+
"writes to that table paused)."
|
|
255
|
+
end
|
|
256
|
+
|
|
209
257
|
# Every table-listing question Rails asks (`tables`, `views`,
|
|
210
258
|
# `data_sources`, `table_exists?`, and through them the schema cache and
|
|
211
259
|
# the schema dumper) is built from this one string in
|
|
@@ -219,8 +267,21 @@ module ActiveRecord
|
|
|
219
267
|
# `create_table "__turso_internal_seq_..."` block per table into
|
|
220
268
|
# db/schema.rb, and the next `db:schema:load` tries to create them for
|
|
221
269
|
# real.
|
|
270
|
+
#
|
|
271
|
+
# On a synced replica the sync engine also keeps its change log in
|
|
272
|
+
# three tables (`turso_cdc`, `turso_cdc_version`,
|
|
273
|
+
# `turso_sync_last_change_id`). They are matched by EXACT name so a user
|
|
274
|
+
# table such as `turso_sync_log` stays visible. Left unfiltered,
|
|
275
|
+
# `db:schema:dump` would dump them and `db:truncate_all`, fixtures or
|
|
276
|
+
# DatabaseCleaner truncation would wipe the change log (losing unpushed
|
|
277
|
+
# sync data).
|
|
278
|
+
SYNC_ENGINE_TABLES = %w[turso_cdc turso_cdc_version turso_sync_last_change_id].freeze
|
|
279
|
+
private_constant :SYNC_ENGINE_TABLES
|
|
280
|
+
|
|
222
281
|
def data_source_sql(name = nil, type: nil)
|
|
223
|
-
|
|
282
|
+
hidden = SYNC_ENGINE_TABLES.map { |t| quote(t) }.join(", ")
|
|
283
|
+
"#{super} AND name NOT LIKE '\\_\\_turso\\_internal\\_%' ESCAPE '\\'" \
|
|
284
|
+
" AND name NOT IN (#{hidden})"
|
|
224
285
|
end
|
|
225
286
|
|
|
226
287
|
# A column definition in the DDL that carries a parenthesised type
|
|
@@ -289,22 +350,31 @@ module ActiveRecord
|
|
|
289
350
|
# Result#affected_rows, so building the Result correctly is all we need.
|
|
290
351
|
def perform_query(raw_connection, sql, binds, type_casted_binds, prepare:, notification_payload:, batch: false)
|
|
291
352
|
params = type_casted_binds || []
|
|
353
|
+
refuse_rename!(sql.strip[0, 60]) if synced_replica? && RENAME_REGEX.match?(mask_non_code(sql))
|
|
292
354
|
|
|
293
355
|
result =
|
|
294
356
|
if batch
|
|
295
357
|
raw_connection.execute_batch(sql)
|
|
296
358
|
::ActiveRecord::Result.empty
|
|
297
359
|
elsif write_statement?(sql)
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
360
|
+
if (rewritten = sql_with_returning_rowid(sql))
|
|
361
|
+
insert_returning_rowid(raw_connection, rewritten, params)
|
|
362
|
+
else
|
|
363
|
+
affected = raw_connection.execute(sql, params)
|
|
364
|
+
@last_inserted_rowid = raw_connection.last_insert_rowid
|
|
365
|
+
::ActiveRecord::Result.empty(affected_rows: affected)
|
|
366
|
+
end
|
|
301
367
|
else
|
|
302
368
|
columns, rows = raw_connection.query_result(sql, params)
|
|
303
369
|
# Belt-and-suspenders: a RETURNING insert lands here (id read from
|
|
304
370
|
# rows by last_inserted_id), and a data-modifying CTE that slips past
|
|
305
371
|
# write_statement? would too -- keep the rowid so an inserted id is
|
|
306
372
|
# never silently lost even on a heuristic miss.
|
|
307
|
-
|
|
373
|
+
if INSERT_REGEX.match?(sql)
|
|
374
|
+
# On a replica where last_insert_rowid is untrustworthy, never
|
|
375
|
+
# offer it as a fallback id (a RETURNING insert's id is in rows).
|
|
376
|
+
@last_inserted_rowid = insert_returning_rowid? ? nil : raw_connection.last_insert_rowid
|
|
377
|
+
end
|
|
308
378
|
::ActiveRecord::Result.new(columns, rows)
|
|
309
379
|
end
|
|
310
380
|
|
|
@@ -322,10 +392,55 @@ module ActiveRecord
|
|
|
322
392
|
# though it mutates. write_query? is a pure predicate here, so calling it a
|
|
323
393
|
# second time is side-effect free.
|
|
324
394
|
def write_statement?(sql)
|
|
325
|
-
return false if
|
|
395
|
+
return false if returning_clause?(sql)
|
|
326
396
|
write_query?(sql) || data_modifying_cte?(sql)
|
|
327
397
|
end
|
|
328
398
|
|
|
399
|
+
# On an MVCC replica with change capture (every synced replica of a
|
|
400
|
+
# `--tursodb` remote) the engine's last_insert_rowid returns the CDC
|
|
401
|
+
# change id, not the inserted row's id (tursodatabase/turso#9551).
|
|
402
|
+
# INSERT ... RETURNING is unaffected, so plain INSERTs are run as
|
|
403
|
+
# `INSERT ... RETURNING rowid` and the id is read from the result.
|
|
404
|
+
# Default: on for synced replicas; `insert_returning_rowid: true|false`
|
|
405
|
+
# in database.yml overrides (e.g. a local MVCC+CDC file).
|
|
406
|
+
def insert_returning_rowid?
|
|
407
|
+
setting = @config[:insert_returning_rowid]
|
|
408
|
+
cast = setting.nil? || setting == "" ? nil : ::ActiveModel::Type::Boolean.new.cast(setting)
|
|
409
|
+
cast.nil? ? synced_replica? : cast
|
|
410
|
+
end
|
|
411
|
+
|
|
412
|
+
# Comments become spaces (they are not part of the statement); literals
|
|
413
|
+
# and quoted identifiers become "x" (they are).
|
|
414
|
+
def mask_non_code(sql)
|
|
415
|
+
sql.gsub(NON_CODE) { |m| (m.start_with?("--", "/*") ? " " : "x") * m.length }
|
|
416
|
+
end
|
|
417
|
+
|
|
418
|
+
# A real RETURNING clause, not the word inside a literal, identifier or
|
|
419
|
+
# comment.
|
|
420
|
+
def returning_clause?(sql)
|
|
421
|
+
RETURNING_REGEX.match?(sql) && RETURNING_REGEX.match?(mask_non_code(sql))
|
|
422
|
+
end
|
|
423
|
+
|
|
424
|
+
# [rewritten_sql] when this plain INSERT should run as RETURNING rowid.
|
|
425
|
+
def sql_with_returning_rowid(sql)
|
|
426
|
+
return unless insert_returning_rowid?
|
|
427
|
+
|
|
428
|
+
masked = mask_non_code(sql)
|
|
429
|
+
return unless INSERT_STATEMENT_REGEX.match?(masked)
|
|
430
|
+
return if RETURNING_REGEX.match?(masked)
|
|
431
|
+
|
|
432
|
+
# Before any trailing ";" and trailing comment (a "--" comment would
|
|
433
|
+
# otherwise swallow the clause).
|
|
434
|
+
at = masked.sub(/;\s*\z/, "").rstrip.length
|
|
435
|
+
"#{sql[0, at]}\nRETURNING rowid#{sql[at..]}"
|
|
436
|
+
end
|
|
437
|
+
|
|
438
|
+
def insert_returning_rowid(raw_connection, rewritten, params)
|
|
439
|
+
_columns, rows = raw_connection.query_result(rewritten, params)
|
|
440
|
+
@last_inserted_rowid = rows.last&.first
|
|
441
|
+
::ActiveRecord::Result.empty(affected_rows: rows.length)
|
|
442
|
+
end
|
|
443
|
+
|
|
329
444
|
def data_modifying_cte?(sql)
|
|
330
445
|
DATA_MODIFYING_CTE_REGEX.match?(sql)
|
|
331
446
|
end
|
|
@@ -4,3 +4,5 @@ require "activerecord/beagle_turso/version"
|
|
|
4
4
|
require "active_record/connection_adapters/beagle_turso_adapter"
|
|
5
5
|
require "active_record/connection_adapters/beagle_turso/sync_manager"
|
|
6
6
|
require "active_record/connection_adapters/beagle_turso/sync_job"
|
|
7
|
+
require "active_record/connection_adapters/beagle_turso/errors"
|
|
8
|
+
require "active_record/connection_adapters/beagle_turso/migration_push"
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: activerecord-beagle-turso
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.7
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- BeagleSoftwareUK
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-18 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -93,6 +93,8 @@ extra_rdoc_files: []
|
|
|
93
93
|
files:
|
|
94
94
|
- README.md
|
|
95
95
|
- config/recurring.yml.example
|
|
96
|
+
- lib/active_record/connection_adapters/beagle_turso/errors.rb
|
|
97
|
+
- lib/active_record/connection_adapters/beagle_turso/migration_push.rb
|
|
96
98
|
- lib/active_record/connection_adapters/beagle_turso/sync_job.rb
|
|
97
99
|
- lib/active_record/connection_adapters/beagle_turso/sync_manager.rb
|
|
98
100
|
- lib/active_record/connection_adapters/beagle_turso_adapter.rb
|