activerecord-beagle-turso 0.1.6 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +33 -0
- data/lib/active_record/connection_adapters/beagle_turso/errors.rb +18 -0
- data/lib/active_record/connection_adapters/beagle_turso/migration_push.rb +207 -0
- data/lib/active_record/connection_adapters/beagle_turso_adapter.rb +171 -19
- data/lib/activerecord/beagle_turso/version.rb +1 -1
- data/lib/activerecord-beagle-turso.rb +2 -0
- metadata +6 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: cd2ebde88afce8658ba27c6a8ae41ffb3228e6ec7bb6c65be6ff0a5a1c56d6ec
|
|
4
|
+
data.tar.gz: 2c6ac050108dd08d75808fb5aa28854164f91835c6529ff400ff97351d81c449
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 0cc05e0873fd0a0d143c2bae7758320768b06bdca6f6cd676de504cc09bc61360877aa81fa7d4b88ca7836758103135d3056d15cc5587721481b826fd9d0855c
|
|
7
|
+
data.tar.gz: d0be294e7c35be6e5275cbbbcb54e80546ddb1b4c30a6f6ea32b72ec88c8554eded57b6537e128581ce599c3d1c173e49c443f7ca4e88ba7366641d351b9916d
|
data/README.md
CHANGED
|
@@ -63,6 +63,13 @@ some diagnostics, but these are custom `database.yml` keys, so that masking
|
|
|
63
63
|
does not cover them automatically. Source them from encrypted credentials or
|
|
64
64
|
`ENV`, as above, and avoid logging/inspecting the raw connection config.
|
|
65
65
|
|
|
66
|
+
`timeout:` (milliseconds, as in the stock SQLite adapter; Rails' generated
|
|
67
|
+
`database.yml` sets 5000) is how long a write waits for another connection's
|
|
68
|
+
transaction to finish before raising `Beagle::Turso::BusyError`. Transactions
|
|
69
|
+
begin with `BEGIN IMMEDIATE`, so a writer waits at `BEGIN`, never part-way
|
|
70
|
+
through. The wait sleeps in Ruby, so other threads and fibers keep running.
|
|
71
|
+
Without `timeout:` (or with 0), a busy write raises at once.
|
|
72
|
+
|
|
66
73
|
An optional `bootstrap_if_empty:` (default `true`) controls whether a brand
|
|
67
74
|
new/empty local file is populated from the remote on first open — see
|
|
68
75
|
`Beagle::Turso::Database.open`.
|
|
@@ -125,6 +132,18 @@ A recurring `SyncJob` against a local-only connection will fail every run
|
|
|
125
132
|
with `NotSyncedError` — only add the recurring entry for environments whose
|
|
126
133
|
`database.yml` connection actually carries `remote_url:`/`auth_token:`.
|
|
127
134
|
|
|
135
|
+
## Synced replicas: what's guarded
|
|
136
|
+
|
|
137
|
+
Three bugs in the Turso 0.8.3 sync engine affect a synced replica (`remote_url:` + `auth_token:`). Bug B is [tursodatabase/turso#9551](https://github.com/tursodatabase/turso/issues/9551); bugs A and C are being reported upstream. This adapter guards against each. Local-only connections are unchanged.
|
|
138
|
+
|
|
139
|
+
- **`rename_table` is refused (bug C).** The engine does not capture `ALTER TABLE ... RENAME TO` or `RENAME COLUMN` for sync, so the remote keeps the old name and every later write to the table is lost on push. `rename_table`, and raw `execute("ALTER TABLE ... RENAME ...")`, raise `ActiveRecord::ConnectionAdapters::BeagleTurso::UnsupportedOnSyncedReplica`. Instead, create a new table, copy the data, and drop the old one; or apply the rename directly on the remote and pull, with writes to that table paused. `rename_column` is allowed: Rails 8.1's SQLite adapter implements it as a table copy, not a native `RENAME COLUMN`.
|
|
140
|
+
- **Insert ids come from `RETURNING` (bug B).** On an MVCC replica with change capture (every synced replica of a `--tursodb` remote) the engine's `last_insert_rowid` returns an internal change id. Model creates already use `INSERT ... RETURNING "id"`. Plain `INSERT`s (including `WITH ... INSERT`, and `connection.insert` / `exec_insert` on a table without a named primary key) now run as `INSERT ... RETURNING rowid` and the id is read from the result. Override with `insert_returning_rowid: true|false` in `database.yml`.
|
|
141
|
+
- **Migrations push first and last (bug A).** Unpushed row writes to a table that is altered or dropped before the next push are skipped by the remote and then lost on pull. When a migration is pending (or the database is new), `db:migrate`, `db:prepare` and their `:<primary>`-namespaced forms push before and after. `db:migrate:up/down/redo`, `db:rollback` and `db:forward` (also with the `:<primary>` suffix) always push, since they always run a migration. With nothing pending, `db:prepare` (which Rails 8 runs on every container boot) does no push and no network I/O.
|
|
142
|
+
- If the push before fails, the task aborts without migrating.
|
|
143
|
+
- A migration that copies a table (`change_column*`, `remove_column`, `add_foreign_key`, `add_timestamps`, ...) makes the first push after it fail once, although the remote ends up with everything. The push after is therefore retried once: if the retry works you only get a warning. If it fails twice the task fails, telling you the schema change exists locally and the push failed.
|
|
144
|
+
- Set `BEAGLE_TURSO_SKIP_MIGRATION_PUSH=1` to skip both pushes, with a warning (unpushed writes to altered tables may be lost).
|
|
145
|
+
- Only the primary database is guarded. `db:schema:load`, `db:setup` and `db:reset` do not work with this adapter (Rails raises `DatabaseNotSupported`); use migrations.
|
|
146
|
+
|
|
128
147
|
## Test coverage
|
|
129
148
|
|
|
130
149
|
This gem's own spec suite (`spec/`, run via `bundle exec rspec`) is 35
|
|
@@ -228,6 +247,20 @@ this gem.
|
|
|
228
247
|
each sync — pick a sync cadence accordingly, or run Solid Queue in a separate
|
|
229
248
|
process.
|
|
230
249
|
|
|
250
|
+
## Known engine issue: self-referencing foreign keys
|
|
251
|
+
|
|
252
|
+
Turso's engine has open bugs with foreign keys that reference their own table
|
|
253
|
+
(such as `parent_id`): a multi-row `delete_all`, `update_all` or `DROP TABLE` that SQLite accepts can fail with
|
|
254
|
+
`FOREIGN KEY constraint failed`, and `ON DELETE SET NULL` / `CASCADE` can leave
|
|
255
|
+
rows behind or orphaned
|
|
256
|
+
([tursodatabase/turso#8713](https://github.com/tursodatabase/turso/issues/8713),
|
|
257
|
+
[#9279](https://github.com/tursodatabase/turso/issues/9279),
|
|
258
|
+
[#9293](https://github.com/tursodatabase/turso/issues/9293),
|
|
259
|
+
[#9294](https://github.com/tursodatabase/turso/issues/9294)). Foreign keys
|
|
260
|
+
between two different tables are unaffected. Workaround: avoid `ON DELETE`
|
|
261
|
+
actions on self-referencing foreign keys, and delete children before parents,
|
|
262
|
+
one level at a time.
|
|
263
|
+
|
|
231
264
|
## License
|
|
232
265
|
|
|
233
266
|
[MIT](https://opensource.org/licenses/MIT).
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ActiveRecord
|
|
4
|
+
module ConnectionAdapters
|
|
5
|
+
module BeagleTurso
|
|
6
|
+
# Raised instead of running a schema change that the Turso sync engine
|
|
7
|
+
# cannot replicate. On a synced replica, `ALTER TABLE ... RENAME TO` and
|
|
8
|
+
# `RENAME COLUMN` are not captured for sync (engine 0.8.3), so the remote
|
|
9
|
+
# never sees them and every later write to the renamed table or column is
|
|
10
|
+
# lost on push.
|
|
11
|
+
class UnsupportedOnSyncedReplica < ::ActiveRecord::ActiveRecordError; end
|
|
12
|
+
|
|
13
|
+
# Raised by the db:migrate family of rake tasks when pending writes could
|
|
14
|
+
# not be pushed to Turso before (or after) a migration.
|
|
15
|
+
class MigrationPushError < ::ActiveRecord::ActiveRecordError; end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_record/connection_adapters/beagle_turso/errors"
|
|
4
|
+
require "active_record/connection_adapters/beagle_turso/sync_manager"
|
|
5
|
+
|
|
6
|
+
module ActiveRecord
|
|
7
|
+
module ConnectionAdapters
|
|
8
|
+
module BeagleTurso
|
|
9
|
+
# Pushes a synced replica to Turso before and after schema-changing rake
|
|
10
|
+
# tasks, when (and only when) a migration is actually going to run.
|
|
11
|
+
#
|
|
12
|
+
# Why: the sync engine replays each unpushed row change against the
|
|
13
|
+
# CURRENT local schema when it pushes. If the table was dropped, or lost
|
|
14
|
+
# columns, since the write, the remote skips the statement, commits the
|
|
15
|
+
# rest of the batch anyway, and the next pull resets the replica to the
|
|
16
|
+
# remote -- the write is gone on both sides. Pushing immediately before
|
|
17
|
+
# the migration leaves nothing pending to be mis-replayed; pushing again
|
|
18
|
+
# afterwards publishes the new schema. (A copy-table migration on a table
|
|
19
|
+
# with rows makes that second push fail once, spuriously, for the same
|
|
20
|
+
# reason; the data is all on the remote and a retry succeeds.)
|
|
21
|
+
#
|
|
22
|
+
# Local-only connections are never touched.
|
|
23
|
+
module MigrationPush
|
|
24
|
+
SKIP_ENV = "BEAGLE_TURSO_SKIP_MIGRATION_PUSH"
|
|
25
|
+
|
|
26
|
+
# Pushes only when a migration is pending (or the database is not
|
|
27
|
+
# initialised): db:prepare runs on every container boot in Rails 8 and
|
|
28
|
+
# must not need the network when there is nothing to do.
|
|
29
|
+
CHECKED_TASKS = %w[db:migrate db:prepare].freeze
|
|
30
|
+
|
|
31
|
+
# Tasks that always run a specific migration, so always push.
|
|
32
|
+
ALWAYS_TASKS = %w[
|
|
33
|
+
db:migrate:up db:migrate:down db:migrate:redo db:rollback db:forward
|
|
34
|
+
].freeze
|
|
35
|
+
|
|
36
|
+
# db:schema:load has no effect with this adapter (Rails raises
|
|
37
|
+
# DatabaseNotSupported for it) but is listed so that it is guarded if
|
|
38
|
+
# that ever changes.
|
|
39
|
+
SCHEMA_TASKS = %w[db:schema:load].freeze
|
|
40
|
+
|
|
41
|
+
# Prepended onto Rake::Task so it works whatever order the Railtie and
|
|
42
|
+
# ActiveRecord's own rake tasks load in.
|
|
43
|
+
module TaskHook
|
|
44
|
+
def execute(args = nil)
|
|
45
|
+
if name.start_with?("db:") && MigrationPush.guarded?(name)
|
|
46
|
+
MigrationPush.around(name) { super }
|
|
47
|
+
else
|
|
48
|
+
super
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
class << self
|
|
54
|
+
def install!
|
|
55
|
+
return if ::Rake::Task.ancestors.include?(TaskHook)
|
|
56
|
+
::Rake::Task.prepend(TaskHook)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# "db:migrate", "db:migrate:primary", "db:rollback:primary", ...
|
|
60
|
+
# The suffix is the PRIMARY config's own name, never hard-coded.
|
|
61
|
+
def guarded?(task_name)
|
|
62
|
+
base = base_task(task_name)
|
|
63
|
+
base && (CHECKED_TASKS + ALWAYS_TASKS + SCHEMA_TASKS).include?(base)
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def base_task(task_name)
|
|
67
|
+
all = CHECKED_TASKS + ALWAYS_TASKS + SCHEMA_TASKS
|
|
68
|
+
return task_name if all.include?(task_name)
|
|
69
|
+
|
|
70
|
+
name = primary_config&.name
|
|
71
|
+
return unless name && task_name.end_with?(":#{name}")
|
|
72
|
+
|
|
73
|
+
base = task_name.delete_suffix(":#{name}")
|
|
74
|
+
base if all.include?(base)
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def around(task_name)
|
|
78
|
+
return yield if depth.positive? # nested task: the outermost one guards
|
|
79
|
+
|
|
80
|
+
config = primary_config
|
|
81
|
+
return yield unless config && synced_config?(config.configuration_hash)
|
|
82
|
+
|
|
83
|
+
if skip?
|
|
84
|
+
warn "beagle_turso: WARNING: #{SKIP_ENV}=1 -- NOT pushing pending writes to Turso around " \
|
|
85
|
+
"#{task_name}. Unpushed writes to tables altered by this task may be lost."
|
|
86
|
+
return yield
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
existing = local_database_exists?(config.configuration_hash)
|
|
90
|
+
if CHECKED_TASKS.include?(base_task(task_name)) && existing
|
|
91
|
+
return yield unless migration_pending?
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
# A replica that does not exist yet has nothing local to push.
|
|
95
|
+
push_before! if existing
|
|
96
|
+
self.depth = 1
|
|
97
|
+
begin
|
|
98
|
+
result = yield
|
|
99
|
+
ensure
|
|
100
|
+
self.depth = 0
|
|
101
|
+
end
|
|
102
|
+
push_after!
|
|
103
|
+
result
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def primary_config
|
|
107
|
+
::ActiveRecord::Base.configurations.find_db_config(rails_env)
|
|
108
|
+
rescue StandardError
|
|
109
|
+
nil
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
def synced_config?(hash)
|
|
113
|
+
hash[:adapter].to_s == "beagle_turso" && !hash[:remote_url].to_s.empty? && !hash[:auth_token].to_s.empty?
|
|
114
|
+
end
|
|
115
|
+
|
|
116
|
+
def skip?
|
|
117
|
+
ENV[SKIP_ENV] == "1"
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
private
|
|
121
|
+
|
|
122
|
+
def rails_env
|
|
123
|
+
defined?(::Rails) && ::Rails.respond_to?(:env) ? ::Rails.env.to_s : (ENV["RAILS_ENV"] || "development")
|
|
124
|
+
end
|
|
125
|
+
|
|
126
|
+
def local_database_exists?(hash)
|
|
127
|
+
path = hash[:database].to_s
|
|
128
|
+
return false if path.empty? || path == ":memory:"
|
|
129
|
+
|
|
130
|
+
root = defined?(::Rails) && ::Rails.respond_to?(:root) ? ::Rails.root : nil
|
|
131
|
+
File.exist?(root ? File.expand_path(path, root.to_s) : path)
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
# True when a migration is pending, or the schema is not initialised
|
|
135
|
+
# (no schema_migrations table reads as everything pending). Any
|
|
136
|
+
# trouble answering counts as pending: the guard then runs.
|
|
137
|
+
def migration_pending?
|
|
138
|
+
connection.pool.migration_context.needs_migration?
|
|
139
|
+
rescue StandardError
|
|
140
|
+
true
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
# Re-resolved on every use: multi-database tasks swap connection pools
|
|
144
|
+
# (with_temporary_pool), so a connection held across the task could
|
|
145
|
+
# belong to a pool that is gone. lease_connection does not raise under
|
|
146
|
+
# permanent_connection_checkout = :disallowed, as .connection does.
|
|
147
|
+
def connection
|
|
148
|
+
::ActiveRecord::Base.lease_connection
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
def push_before!
|
|
152
|
+
::Beagle::Turso::SyncManager.push!(connection)
|
|
153
|
+
rescue StandardError => e
|
|
154
|
+
raise MigrationPushError.new(
|
|
155
|
+
"beagle_turso: could not push pending writes to Turso before migrating (#{describe(e)}). " \
|
|
156
|
+
"Fix sync first, or set #{SKIP_ENV}=1 to migrate anyway (unpushed writes to altered tables " \
|
|
157
|
+
"may be lost)."
|
|
158
|
+
), cause: nil
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# A copy-table migration on a table with rows (change_column*,
|
|
162
|
+
# remove_column, add_foreign_key, add_timestamps, ...) makes the first
|
|
163
|
+
# push after it fail with a parse error even though the remote ends up
|
|
164
|
+
# with everything (sync engine bug A); the second push succeeds. So
|
|
165
|
+
# retry once before reporting a failure.
|
|
166
|
+
def push_after!
|
|
167
|
+
::Beagle::Turso::SyncManager.push!(connection)
|
|
168
|
+
rescue StandardError => first
|
|
169
|
+
begin
|
|
170
|
+
::Beagle::Turso::SyncManager.push!(connection)
|
|
171
|
+
warn "beagle_turso: WARNING: the push after the migration failed once (#{describe(first)}) and " \
|
|
172
|
+
"succeeded on retry. This is expected after a migration that copies a table (sync engine " \
|
|
173
|
+
"quirk); no data was lost."
|
|
174
|
+
rescue StandardError => second
|
|
175
|
+
raise MigrationPushError.new(
|
|
176
|
+
"beagle_turso: the schema change was applied to the local replica, but pushing it to Turso " \
|
|
177
|
+
"failed twice (#{describe(first)}; then #{describe(second)}). The remote may not have the " \
|
|
178
|
+
"new schema yet: check sync, then run Beagle::Turso::SyncManager.push!."
|
|
179
|
+
), cause: nil
|
|
180
|
+
end
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
def describe(error)
|
|
184
|
+
message = error.message
|
|
185
|
+
message = ::Beagle::Turso::Redaction.message(message) if defined?(::Beagle::Turso::Redaction)
|
|
186
|
+
prefix = error.class.name.to_s.end_with?("OpenError") ? "could not open the synced replica: " : ""
|
|
187
|
+
"#{error.class}: #{prefix}#{message}"
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
def depth = Thread.current[:beagle_turso_migration_push_depth].to_i
|
|
191
|
+
def depth=(value)
|
|
192
|
+
Thread.current[:beagle_turso_migration_push_depth] = value
|
|
193
|
+
end
|
|
194
|
+
end
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
if defined?(::Rails::Railtie)
|
|
198
|
+
class Railtie < ::Rails::Railtie
|
|
199
|
+
rake_tasks do
|
|
200
|
+
require "rake"
|
|
201
|
+
MigrationPush.install!
|
|
202
|
+
end
|
|
203
|
+
end
|
|
204
|
+
end
|
|
205
|
+
end
|
|
206
|
+
end
|
|
207
|
+
end
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "active_record"
|
|
4
4
|
require "active_record/connection_adapters/sqlite3_adapter"
|
|
5
5
|
require "beagle/turso"
|
|
6
|
+
require "active_record/connection_adapters/beagle_turso/errors"
|
|
6
7
|
|
|
7
8
|
module ActiveRecord
|
|
8
9
|
module ConnectionAdapters
|
|
@@ -59,6 +60,20 @@ module ActiveRecord
|
|
|
59
60
|
INSERT_REGEX = /\bINSERT\b/i
|
|
60
61
|
private_constant :INSERT_REGEX
|
|
61
62
|
|
|
63
|
+
# String literals, quoted identifiers and comments: where words like
|
|
64
|
+
# RETURNING or RENAME are data, not syntax. Masked before any keyword
|
|
65
|
+
# matching (same approach as sequel-beagle-turso).
|
|
66
|
+
NON_CODE = %r{'(?:[^']|'')*'|"(?:[^"]|"")*"|`(?:[^`]|``)*`|\[[^\]]*\]|--[^\n]*|/\*.*?\*/}m
|
|
67
|
+
private_constant :NON_CODE
|
|
68
|
+
|
|
69
|
+
# `ALTER TABLE <t> RENAME ...`, tested on masked SQL.
|
|
70
|
+
RENAME_REGEX = /\A\s*ALTER\s+TABLE\s+\S+\s+RENAME\b/i
|
|
71
|
+
private_constant :RENAME_REGEX
|
|
72
|
+
|
|
73
|
+
# INSERT / REPLACE, optionally behind a WITH clause, tested on masked SQL.
|
|
74
|
+
INSERT_STATEMENT_REGEX = /\A\s*(?:(?:INSERT|REPLACE)\b|WITH\b[\s\S]*\)\s*(?:INSERT|REPLACE)\b)/i
|
|
75
|
+
private_constant :INSERT_STATEMENT_REGEX
|
|
76
|
+
|
|
62
77
|
class << self
|
|
63
78
|
# Open a beagle-turso Database (synced remote when +remote_url+ is
|
|
64
79
|
# present, otherwise local/in-memory) and wrap it so the SQLite3Adapter
|
|
@@ -77,6 +92,10 @@ module ActiveRecord
|
|
|
77
92
|
end
|
|
78
93
|
Client.new(database)
|
|
79
94
|
end
|
|
95
|
+
|
|
96
|
+
def synced_config?(config)
|
|
97
|
+
!config[:remote_url].to_s.empty? && !config[:auth_token].to_s.empty?
|
|
98
|
+
end
|
|
80
99
|
end
|
|
81
100
|
|
|
82
101
|
# Adapts a Beagle::Turso::Database + its connected Connection to the subset
|
|
@@ -92,9 +111,10 @@ module ActiveRecord
|
|
|
92
111
|
attr_reader :database, :connection
|
|
93
112
|
|
|
94
113
|
def initialize(database)
|
|
95
|
-
@database
|
|
96
|
-
@connection
|
|
97
|
-
@closed
|
|
114
|
+
@database = database
|
|
115
|
+
@connection = database.connect
|
|
116
|
+
@closed = false
|
|
117
|
+
@busy_timeout = nil
|
|
98
118
|
end
|
|
99
119
|
|
|
100
120
|
# A bare read of exactly +defer_foreign_keys+ or +read_uncommitted+
|
|
@@ -162,11 +182,39 @@ module ActiveRecord
|
|
|
162
182
|
def push = database.push
|
|
163
183
|
def pull = database.pull
|
|
164
184
|
|
|
165
|
-
# ---
|
|
166
|
-
# configure_connection
|
|
167
|
-
|
|
168
|
-
|
|
185
|
+
# --- busy handling ---
|
|
186
|
+
# configure_connection passes database.yml's `timeout:` (ms) here, as it
|
|
187
|
+
# does to the sqlite3 gem's busy handler. Unset or 0 means no waiting:
|
|
188
|
+
# a BusyError surfaces at once, as with the sqlite3 gem.
|
|
189
|
+
def busy_handler_timeout=(milliseconds)
|
|
190
|
+
@busy_timeout = milliseconds.to_i.positive? ? milliseconds.to_i / 1000.0 : nil
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
# Retries a statement that lost the single-writer race, with capped
|
|
194
|
+
# exponential backoff, for up to the busy timeout. The engine never
|
|
195
|
+
# waits for a lock itself (from beagle-turso 0.4.0 it raises at once),
|
|
196
|
+
# so this is where the wait happens. sleep releases the GVL and yields
|
|
197
|
+
# to a fiber scheduler, so the writer holding the lock can finish. The
|
|
198
|
+
# caller decides what is safe to retry (see BeagleTursoAdapter#perform_query).
|
|
199
|
+
def retry_busy
|
|
200
|
+
deadline = nil
|
|
201
|
+
delay = 0.002
|
|
202
|
+
begin
|
|
203
|
+
yield
|
|
204
|
+
rescue ::Beagle::Turso::BusyError => e
|
|
205
|
+
raise if @busy_timeout.nil? || e.message.include?("snapshot is stale")
|
|
206
|
+
|
|
207
|
+
now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
208
|
+
deadline ||= now + @busy_timeout
|
|
209
|
+
raise if now + delay > deadline
|
|
210
|
+
|
|
211
|
+
sleep delay
|
|
212
|
+
delay = [delay * 2, 0.1].min
|
|
213
|
+
retry
|
|
214
|
+
end
|
|
169
215
|
end
|
|
216
|
+
|
|
217
|
+
# --- sqlite3-gem lifecycle shims the SQLite3Adapter drives ---
|
|
170
218
|
# connected? => !(@raw_connection.nil? || @raw_connection.closed?)
|
|
171
219
|
def closed? = @closed
|
|
172
220
|
# Release the beagle-turso session eagerly rather than waiting for GC.
|
|
@@ -205,7 +253,36 @@ module ActiveRecord
|
|
|
205
253
|
end
|
|
206
254
|
end
|
|
207
255
|
|
|
256
|
+
# Refused on a synced replica: the sync engine (turso 0.8.3) does not
|
|
257
|
+
# capture `ALTER TABLE ... RENAME TO`, so the remote never gets the new
|
|
258
|
+
# name and every later write to the table is lost on push.
|
|
259
|
+
# (rename_column is allowed: Rails' SQLite adapter implements it as a
|
|
260
|
+
# copy-table, not a native RENAME COLUMN.)
|
|
261
|
+
def rename_table(table_name, new_name, **options)
|
|
262
|
+
refuse_rename!("rename_table(#{table_name.inspect}, #{new_name.inspect})")
|
|
263
|
+
super
|
|
264
|
+
end
|
|
265
|
+
|
|
266
|
+
# True when this connection is a synced replica, decided from the config
|
|
267
|
+
# exactly as new_client decides it. Deliberately does NOT touch
|
|
268
|
+
# raw_connection: the public accessor disables lazy transactions and marks
|
|
269
|
+
# the connection dirty, which would make every local-only app pay for it.
|
|
270
|
+
def synced_replica?
|
|
271
|
+
self.class.synced_config?(@config)
|
|
272
|
+
end
|
|
273
|
+
|
|
208
274
|
private
|
|
275
|
+
def refuse_rename!(what)
|
|
276
|
+
return unless synced_replica?
|
|
277
|
+
|
|
278
|
+
raise BeagleTurso::UnsupportedOnSyncedReplica,
|
|
279
|
+
"beagle_turso: #{what} is not supported on a synced replica. The Turso sync engine does not " \
|
|
280
|
+
"replicate ALTER TABLE ... RENAME, so the remote would keep the old name and every later " \
|
|
281
|
+
"write to it would be lost on push. Instead: create a new table, copy the data across, then " \
|
|
282
|
+
"drop the old one; or apply the rename directly on the remote database, then pull (with " \
|
|
283
|
+
"writes to that table paused)."
|
|
284
|
+
end
|
|
285
|
+
|
|
209
286
|
# Every table-listing question Rails asks (`tables`, `views`,
|
|
210
287
|
# `data_sources`, `table_exists?`, and through them the schema cache and
|
|
211
288
|
# the schema dumper) is built from this one string in
|
|
@@ -302,23 +379,16 @@ module ActiveRecord
|
|
|
302
379
|
# Result#affected_rows, so building the Result correctly is all we need.
|
|
303
380
|
def perform_query(raw_connection, sql, binds, type_casted_binds, prepare:, notification_payload:, batch: false)
|
|
304
381
|
params = type_casted_binds || []
|
|
382
|
+
refuse_rename!(sql.strip[0, 60]) if synced_replica? && RENAME_REGEX.match?(mask_non_code(sql))
|
|
305
383
|
|
|
306
384
|
result =
|
|
307
385
|
if batch
|
|
308
386
|
raw_connection.execute_batch(sql)
|
|
309
387
|
::ActiveRecord::Result.empty
|
|
310
|
-
elsif
|
|
311
|
-
|
|
312
|
-
@last_inserted_rowid = raw_connection.last_insert_rowid
|
|
313
|
-
::ActiveRecord::Result.empty(affected_rows: affected)
|
|
388
|
+
elsif busy_retry?(sql)
|
|
389
|
+
raw_connection.retry_busy { run_statement(raw_connection, sql, params) }
|
|
314
390
|
else
|
|
315
|
-
|
|
316
|
-
# Belt-and-suspenders: a RETURNING insert lands here (id read from
|
|
317
|
-
# rows by last_inserted_id), and a data-modifying CTE that slips past
|
|
318
|
-
# write_statement? would too -- keep the rowid so an inserted id is
|
|
319
|
-
# never silently lost even on a heuristic miss.
|
|
320
|
-
@last_inserted_rowid = raw_connection.last_insert_rowid if INSERT_REGEX.match?(sql)
|
|
321
|
-
::ActiveRecord::Result.new(columns, rows)
|
|
391
|
+
run_statement(raw_connection, sql, params)
|
|
322
392
|
end
|
|
323
393
|
|
|
324
394
|
verified!
|
|
@@ -327,6 +397,43 @@ module ActiveRecord
|
|
|
327
397
|
result
|
|
328
398
|
end
|
|
329
399
|
|
|
400
|
+
# A busy statement is retried only when that is safe: the BEGIN that
|
|
401
|
+
# opens a transaction (Rails sends BEGIN IMMEDIATE, so a writer waits
|
|
402
|
+
# for the lock there rather than failing mid-way), or a statement
|
|
403
|
+
# outside any transaction. Inside one, a BusyError means the
|
|
404
|
+
# transaction is dead; it must surface and roll back. A multi-statement
|
|
405
|
+
# batch is never retried (it may have run in part).
|
|
406
|
+
BEGIN_REGEX = /\A\s*BEGIN\b/i
|
|
407
|
+
private_constant :BEGIN_REGEX
|
|
408
|
+
|
|
409
|
+
def busy_retry?(sql)
|
|
410
|
+
!transaction_open? || BEGIN_REGEX.match?(mask_non_code(sql))
|
|
411
|
+
end
|
|
412
|
+
|
|
413
|
+
def run_statement(raw_connection, sql, params)
|
|
414
|
+
if write_statement?(sql)
|
|
415
|
+
if (rewritten = sql_with_returning_rowid(sql))
|
|
416
|
+
insert_returning_rowid(raw_connection, rewritten, params)
|
|
417
|
+
else
|
|
418
|
+
affected = raw_connection.execute(sql, params)
|
|
419
|
+
@last_inserted_rowid = raw_connection.last_insert_rowid
|
|
420
|
+
::ActiveRecord::Result.empty(affected_rows: affected)
|
|
421
|
+
end
|
|
422
|
+
else
|
|
423
|
+
columns, rows = raw_connection.query_result(sql, params)
|
|
424
|
+
# Belt-and-suspenders: a RETURNING insert lands here (id read from
|
|
425
|
+
# rows by last_inserted_id), and a data-modifying CTE that slips past
|
|
426
|
+
# write_statement? would too -- keep the rowid so an inserted id is
|
|
427
|
+
# never silently lost even on a heuristic miss.
|
|
428
|
+
if INSERT_REGEX.match?(sql)
|
|
429
|
+
# On a replica where last_insert_rowid is untrustworthy, never
|
|
430
|
+
# offer it as a fallback id (a RETURNING insert's id is in rows).
|
|
431
|
+
@last_inserted_rowid = insert_returning_rowid? ? nil : raw_connection.last_insert_rowid
|
|
432
|
+
end
|
|
433
|
+
::ActiveRecord::Result.new(columns, rows)
|
|
434
|
+
end
|
|
435
|
+
end
|
|
436
|
+
|
|
330
437
|
# Route to the WRITE branch (execute -> affected count + last_insert_rowid)
|
|
331
438
|
# rather than the read branch. A statement is a write when SQLite3's own
|
|
332
439
|
# read/write classifier says so, OR when it is a data-modifying CTE that
|
|
@@ -335,10 +442,55 @@ module ActiveRecord
|
|
|
335
442
|
# though it mutates. write_query? is a pure predicate here, so calling it a
|
|
336
443
|
# second time is side-effect free.
|
|
337
444
|
def write_statement?(sql)
|
|
338
|
-
return false if
|
|
445
|
+
return false if returning_clause?(sql)
|
|
339
446
|
write_query?(sql) || data_modifying_cte?(sql)
|
|
340
447
|
end
|
|
341
448
|
|
|
449
|
+
# On an MVCC replica with change capture (every synced replica of a
|
|
450
|
+
# `--tursodb` remote) the engine's last_insert_rowid returns the CDC
|
|
451
|
+
# change id, not the inserted row's id (tursodatabase/turso#9551).
|
|
452
|
+
# INSERT ... RETURNING is unaffected, so plain INSERTs are run as
|
|
453
|
+
# `INSERT ... RETURNING rowid` and the id is read from the result.
|
|
454
|
+
# Default: on for synced replicas; `insert_returning_rowid: true|false`
|
|
455
|
+
# in database.yml overrides (e.g. a local MVCC+CDC file).
|
|
456
|
+
def insert_returning_rowid?
|
|
457
|
+
setting = @config[:insert_returning_rowid]
|
|
458
|
+
cast = setting.nil? || setting == "" ? nil : ::ActiveModel::Type::Boolean.new.cast(setting)
|
|
459
|
+
cast.nil? ? synced_replica? : cast
|
|
460
|
+
end
|
|
461
|
+
|
|
462
|
+
# Comments become spaces (they are not part of the statement); literals
|
|
463
|
+
# and quoted identifiers become "x" (they are).
|
|
464
|
+
def mask_non_code(sql)
|
|
465
|
+
sql.gsub(NON_CODE) { |m| (m.start_with?("--", "/*") ? " " : "x") * m.length }
|
|
466
|
+
end
|
|
467
|
+
|
|
468
|
+
# A real RETURNING clause, not the word inside a literal, identifier or
|
|
469
|
+
# comment.
|
|
470
|
+
def returning_clause?(sql)
|
|
471
|
+
RETURNING_REGEX.match?(sql) && RETURNING_REGEX.match?(mask_non_code(sql))
|
|
472
|
+
end
|
|
473
|
+
|
|
474
|
+
# [rewritten_sql] when this plain INSERT should run as RETURNING rowid.
|
|
475
|
+
def sql_with_returning_rowid(sql)
|
|
476
|
+
return unless insert_returning_rowid?
|
|
477
|
+
|
|
478
|
+
masked = mask_non_code(sql)
|
|
479
|
+
return unless INSERT_STATEMENT_REGEX.match?(masked)
|
|
480
|
+
return if RETURNING_REGEX.match?(masked)
|
|
481
|
+
|
|
482
|
+
# Before any trailing ";" and trailing comment (a "--" comment would
|
|
483
|
+
# otherwise swallow the clause).
|
|
484
|
+
at = masked.sub(/;\s*\z/, "").rstrip.length
|
|
485
|
+
"#{sql[0, at]}\nRETURNING rowid#{sql[at..]}"
|
|
486
|
+
end
|
|
487
|
+
|
|
488
|
+
def insert_returning_rowid(raw_connection, rewritten, params)
|
|
489
|
+
_columns, rows = raw_connection.query_result(rewritten, params)
|
|
490
|
+
@last_inserted_rowid = rows.last&.first
|
|
491
|
+
::ActiveRecord::Result.empty(affected_rows: rows.length)
|
|
492
|
+
end
|
|
493
|
+
|
|
342
494
|
def data_modifying_cte?(sql)
|
|
343
495
|
DATA_MODIFYING_CTE_REGEX.match?(sql)
|
|
344
496
|
end
|
|
@@ -4,3 +4,5 @@ require "activerecord/beagle_turso/version"
|
|
|
4
4
|
require "active_record/connection_adapters/beagle_turso_adapter"
|
|
5
5
|
require "active_record/connection_adapters/beagle_turso/sync_manager"
|
|
6
6
|
require "active_record/connection_adapters/beagle_turso/sync_job"
|
|
7
|
+
require "active_record/connection_adapters/beagle_turso/errors"
|
|
8
|
+
require "active_record/connection_adapters/beagle_turso/migration_push"
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: activerecord-beagle-turso
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.8
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- BeagleSoftwareUK
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-18 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: activerecord
|
|
@@ -30,14 +30,14 @@ dependencies:
|
|
|
30
30
|
requirements:
|
|
31
31
|
- - "~>"
|
|
32
32
|
- !ruby/object:Gem::Version
|
|
33
|
-
version: '0.
|
|
33
|
+
version: '0.3'
|
|
34
34
|
type: :runtime
|
|
35
35
|
prerelease: false
|
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
|
37
37
|
requirements:
|
|
38
38
|
- - "~>"
|
|
39
39
|
- !ruby/object:Gem::Version
|
|
40
|
-
version: '0.
|
|
40
|
+
version: '0.3'
|
|
41
41
|
- !ruby/object:Gem::Dependency
|
|
42
42
|
name: sqlite3
|
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -93,6 +93,8 @@ extra_rdoc_files: []
|
|
|
93
93
|
files:
|
|
94
94
|
- README.md
|
|
95
95
|
- config/recurring.yml.example
|
|
96
|
+
- lib/active_record/connection_adapters/beagle_turso/errors.rb
|
|
97
|
+
- lib/active_record/connection_adapters/beagle_turso/migration_push.rb
|
|
96
98
|
- lib/active_record/connection_adapters/beagle_turso/sync_job.rb
|
|
97
99
|
- lib/active_record/connection_adapters/beagle_turso/sync_manager.rb
|
|
98
100
|
- lib/active_record/connection_adapters/beagle_turso_adapter.rb
|