beagle-turso 0.1.3-aarch64-linux → 0.3.0-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +116 -1
- data/lib/beagle/turso/errors.rb +19 -0
- data/lib/beagle/turso/redaction.rb +41 -0
- data/lib/beagle/turso/syncer.rb +293 -0
- data/lib/beagle/turso/version.rb +1 -1
- data/lib/beagle/turso.rb +21 -2
- data/lib/beagle_turso/3.3/beagle_turso.so +2 -2
- data/lib/beagle_turso/3.4/beagle_turso.so +2 -2
- data/lib/beagle_turso/4.0/beagle_turso.so +2 -2
- metadata +5 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ee35fc3653fd80e413a40da233110ce76385da3a4bdbc3bdf0202a7b034910a7
|
|
4
|
+
data.tar.gz: 5e79213a92598f43e4f3ad808714502814a62088e85c4a605e792bf2611880f5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '0929cc02e5f67962ff76c267dc4ec6a957637a59e4db4456d4d7485fd4561741c7adaa4d76f90d8e85c4a4ee5ccd138f41e83364c011063901a2e6d93767e4ea'
|
|
7
|
+
data.tar.gz: d3ea50bd3271f04970124569779176f1d21bfbdb585d3d5a21bbc30cb5d4aa85b52ffa0431c3613c0775df6016a893e11fe06479a531f055843261c2357beae9
|
data/README.md
CHANGED
|
@@ -68,7 +68,8 @@ db.pull # pull remote changes down to the local file (returns true/false)
|
|
|
68
68
|
`Database.open` also accepts `bootstrap_if_empty:` (defaults to `true`),
|
|
69
69
|
which pulls all remote data down on first open if the local file is new/empty.
|
|
70
70
|
Calling `push`/`pull` on a database opened via `open_local` (or via `open`
|
|
71
|
-
without both `remote_url:` and `auth_token:`) raises
|
|
71
|
+
without both `remote_url:` and `auth_token:`) raises
|
|
72
|
+
`Beagle::Turso::NotSyncedError`.
|
|
72
73
|
|
|
73
74
|
### Durability note: writes commit on-sync, not synchronously
|
|
74
75
|
|
|
@@ -103,6 +104,120 @@ Two things to avoid:
|
|
|
103
104
|
A single remote database, dedicated to this engine and written by one process
|
|
104
105
|
at a time, is the model it's built for.
|
|
105
106
|
|
|
107
|
+
## Errors
|
|
108
|
+
|
|
109
|
+
Every error the driver raises is a `Beagle::Turso::Error`, a `RuntimeError`
|
|
110
|
+
subclass: `OpenError`, `ConnectError`, `QueryError`, `SyncError`,
|
|
111
|
+
`NotSyncedError` (push/pull on a local-only database), `BusyError` (another
|
|
112
|
+
session or process holds the database), `ClosedError` and `ForkedError`. An
|
|
113
|
+
unsupported bind parameter type raises Ruby's `TypeError`. Auth, network and
|
|
114
|
+
protocol failures during sync all arrive as `SyncError` (or `OpenError` at
|
|
115
|
+
open): the engine does not distinguish them. Messages never contain the auth
|
|
116
|
+
token.
|
|
117
|
+
|
|
118
|
+
Busy conditions now raise `BusyError` with a message beginning
|
|
119
|
+
`database is busy:` (0.1.x said `query failed:` / `sync failed:`); it is
|
|
120
|
+
still a `RuntimeError`. A replica file already held by another process raises
|
|
121
|
+
`OpenError` ("... File is locked by another process").
|
|
122
|
+
|
|
123
|
+
## Threads, fibers and fork
|
|
124
|
+
|
|
125
|
+
- One `Database` may be shared by threads. Give each thread its own
|
|
126
|
+
`Connection`; a shared `Connection` works but serializes its callers.
|
|
127
|
+
- `open`, `connect`, `push` and `pull` release the GVL, so other threads keep
|
|
128
|
+
running while they wait on the network. Queries keep it; they are local and
|
|
129
|
+
fast.
|
|
130
|
+
- While one thread pushes, `closed?` returns immediately on other threads, and
|
|
131
|
+
queries on other connections proceed.
|
|
132
|
+
- On a synced database, `connect` called while another thread is pushing waits
|
|
133
|
+
inside the engine until the push finishes (about the push's duration); the
|
|
134
|
+
GVL is released while it waits, so other threads and fibers keep running.
|
|
135
|
+
Open connections up front (e.g. at boot / pool warm-up) rather than per
|
|
136
|
+
request.
|
|
137
|
+
- `close` during an in-flight `push` is safe: the push finishes or raises, and
|
|
138
|
+
later calls raise `ClosedError`.
|
|
139
|
+
- A blocked native call cannot be interrupted: `Timeout` and `Thread#kill`
|
|
140
|
+
take effect when it returns.
|
|
141
|
+
- **Falcon / Async:** on Ruby 4.0+ with `async` 2.25+ (enable its worker pool
|
|
142
|
+
with `ASYNC_SCHEDULER_WORKER_POOL=true` or
|
|
143
|
+
`Async::Scheduler.new(worker_pool: true)`), a fiber scheduler runs these
|
|
144
|
+
calls on a worker thread, so the reactor keeps serving other requests during
|
|
145
|
+
a sync. On Ruby 3.3/3.4 the GVL is still released, but the scheduler's own
|
|
146
|
+
thread blocks. Run sync from a dedicated thread there. Run one Falcon
|
|
147
|
+
process per replica (`--count 1`): the replica file is locked to one
|
|
148
|
+
process.
|
|
149
|
+
- **Fork:** handles are bound to the process that opened them. In a forked
|
|
150
|
+
child, `open`, `connect`, `push`, `pull` and queries on an inherited handle
|
|
151
|
+
raise `ForkedError`; open a new database there. `close` is a no-op and
|
|
152
|
+
`closed?` is safe (it reports the handle as closed if it can't be inspected),
|
|
153
|
+
so inherited handles need no cleanup in the child.
|
|
154
|
+
|
|
155
|
+
## Background sync
|
|
156
|
+
|
|
157
|
+
`Beagle::Turso::Syncer` keeps the remote copy current without a job system:
|
|
158
|
+
|
|
159
|
+
```ruby
|
|
160
|
+
db = Beagle::Turso::Database.open(local_path: "app.db", remote_url: url, auth_token: token)
|
|
161
|
+
|
|
162
|
+
syncer = Beagle::Turso::Syncer.new(
|
|
163
|
+
db,
|
|
164
|
+
interval: 30, # seconds between cycles
|
|
165
|
+
pull: true, # pull after each push (false = push only)
|
|
166
|
+
max_backoff: 300, # retry delay cap, seconds
|
|
167
|
+
on_error: ->(error, status) { ErrorTracker.notify(error) },
|
|
168
|
+
logger: Logger.new($stderr),
|
|
169
|
+
).start
|
|
170
|
+
|
|
171
|
+
syncer.status # => #<data Status running=true, syncing=false, last_push_at=..., ...>
|
|
172
|
+
syncer.sync_now # one cycle now, on this thread; :ok, or :busy if one is running
|
|
173
|
+
# (a failure is recorded, passed to on_error, then re-raised)
|
|
174
|
+
|
|
175
|
+
# on shutdown, before closing the database:
|
|
176
|
+
syncer.stop(timeout: 10) # waits for an in-flight sync, then pushes once more
|
|
177
|
+
# syncer.stop(timeout: 10, final_push: false) # waits, but doesn't push again
|
|
178
|
+
db.close
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
- **Opt-in.** Requiring the gem starts nothing. `start` runs one background
|
|
182
|
+
thread. `push` and `pull` release the GVL, so that thread doesn't hold up
|
|
183
|
+
requests on other threads or a Falcon reactor.
|
|
184
|
+
- **Failures don't stop it.** Each failure is logged (at `warn`) and passed to
|
|
185
|
+
`on_error`. The next attempt waits `interval × 2ⁿ`, capped at
|
|
186
|
+
`max_backoff`, with ±20% jitter. A success resets the count. A failed push
|
|
187
|
+
skips that cycle's pull. A logger or `on_error` that raises a
|
|
188
|
+
`StandardError` is ignored (an `on_error` error is logged).
|
|
189
|
+
- **`sync_now` re-raises.** A failed `sync_now` records the failure in
|
|
190
|
+
`status`, logs it, calls `on_error`, then raises the error to you. Since
|
|
191
|
+
`on_error` has already seen it, don't report it again from a `rescue`. If
|
|
192
|
+
the error message carried a URL or credential, both `on_error` and the
|
|
193
|
+
caller get a copy of the error (same class and backtrace) with the redacted
|
|
194
|
+
message.
|
|
195
|
+
- **Status is for monitoring, not lag.** `last_push_at` says when a push last
|
|
196
|
+
succeeded, not how much is still unpushed. Alert when `consecutive_failures`
|
|
197
|
+
grows, or when `last_push_at` gets older than you can tolerate. The site keeps
|
|
198
|
+
serving from the local replica either way. `destination` is the remote host
|
|
199
|
+
only, and error messages are stripped of URLs and tokens.
|
|
200
|
+
- **Shutdown is best-effort.** `stop(timeout:, final_push: true)` stops the
|
|
201
|
+
loop, waits for a cycle in flight on any thread (the loop or a concurrent
|
|
202
|
+
`sync_now`), then pushes once more unless `final_push: false`. It waits up
|
|
203
|
+
to `timeout` seconds in total. A push or pull already in flight can't be
|
|
204
|
+
interrupted, so if the deadline passes, `stop` returns with
|
|
205
|
+
`status.syncing == true` and the work finishes in the background; process
|
|
206
|
+
exit then waits for that native call to return (until your platform's kill
|
|
207
|
+
timeout). `stop(timeout: 0)` skips the final push. Called from inside
|
|
208
|
+
`on_error`, `stop` can't wait for the cycle it is part of, so it returns
|
|
209
|
+
without a final push. If the network is down, the deadline passes, or the
|
|
210
|
+
process is killed, writes stay local until the next process pushes them.
|
|
211
|
+
`stop` never closes the database; do that after it returns.
|
|
212
|
+
- **Don't call `stop` from a signal handler.** Inside `Signal.trap` it raises
|
|
213
|
+
`ThreadError` (it takes locks). Call it from your server's shutdown hook or
|
|
214
|
+
`at_exit`.
|
|
215
|
+
- **One syncer per process, created after fork.** A stopped syncer can't be
|
|
216
|
+
restarted, so create a new one. With Puma in single mode, create it in an
|
|
217
|
+
initializer or `on_booted`; in cluster mode, create it in `on_worker_boot`
|
|
218
|
+
(each worker gets its own) and call `stop` from `on_worker_shutdown` or
|
|
219
|
+
`at_exit`.
|
|
220
|
+
|
|
106
221
|
## Credential safety
|
|
107
222
|
|
|
108
223
|
`auth_token` is never included in `inspect` output or in error messages —
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
module Beagle
|
|
2
|
+
module Turso
|
|
3
|
+
# Base class for every error the native driver raises. A RuntimeError so
|
|
4
|
+
# code written against 0.1.x, which raised bare RuntimeErrors, still rescues.
|
|
5
|
+
class Error < RuntimeError; end
|
|
6
|
+
|
|
7
|
+
class OpenError < Error; end
|
|
8
|
+
class ConnectError < Error; end
|
|
9
|
+
class QueryError < Error; end
|
|
10
|
+
class SyncError < Error; end
|
|
11
|
+
# push/pull on a database opened without remote_url:/auth_token:.
|
|
12
|
+
class NotSyncedError < Error; end
|
|
13
|
+
# The engine reported the database busy (another session or process holds it).
|
|
14
|
+
class BusyError < Error; end
|
|
15
|
+
class ClosedError < Error; end
|
|
16
|
+
# A handle opened in a parent process was used after fork; open a new one.
|
|
17
|
+
class ForkedError < Error; end
|
|
18
|
+
end
|
|
19
|
+
end
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
require "uri"
|
|
2
|
+
|
|
3
|
+
module Beagle
|
|
4
|
+
module Turso
|
|
5
|
+
# Strips credentials from text before it reaches a log, a status or a
|
|
6
|
+
# callback. Engine error messages can embed the remote URL; the auth token
|
|
7
|
+
# itself is sent as a header, but a URL can still carry userinfo or a
|
|
8
|
+
# token query parameter, so reduce every URL to scheme://host.
|
|
9
|
+
module Redaction
|
|
10
|
+
# scheme://[userinfo@]host[...]: keep scheme and host (a bracketed IPv6
|
|
11
|
+
# literal included), drop userinfo, port, path, query and fragment. The
|
|
12
|
+
# scheme length is bounded so a long run of "a.a.a." stays linear.
|
|
13
|
+
URL = %r{\b([a-z][a-z0-9+.-]{0,31})://(?:[^/\s?#"'@\[]+@)?(\[[0-9a-f:.]*\]|[^/\s?#"':\[]+)[^\s"']*}i
|
|
14
|
+
# Authorization schemes: keep the scheme word, blank the credential.
|
|
15
|
+
AUTH_SCHEME = /\b(Bearer|Basic)\s+[^\s"',;]+/i
|
|
16
|
+
# key=value / key: value / "key": "value" / key%3Dvalue, where the key is
|
|
17
|
+
# a whole word ending in a secret-ish name (turso_auth_token, x-api-key,
|
|
18
|
+
# access_token, ...). The lookbehind pins the key to a word start, which
|
|
19
|
+
# keeps the unbounded prefix linear.
|
|
20
|
+
SECRET_PARAM = /(?<![\w-])([\w-]*?(?:auth_?token|token|jwt|api[_-]?key|password|passwd|secret|authorization))(["']?\s*(?:[=:]|%3D)\s*["']?)(?!(?:Bearer|Basic)\s)[^\s"'&,;]+/i
|
|
21
|
+
# A bare JWT (header.payload.signature, base64url).
|
|
22
|
+
JWT = /\beyJ[\w-]+\.[\w-]+\.[\w-]+/
|
|
23
|
+
|
|
24
|
+
module_function
|
|
25
|
+
|
|
26
|
+
def message(text)
|
|
27
|
+
text.to_s
|
|
28
|
+
.gsub(URL) { "#{Regexp.last_match(1)}://#{Regexp.last_match(2)}" }
|
|
29
|
+
.gsub(JWT, "[redacted]")
|
|
30
|
+
.gsub(AUTH_SCHEME, '\1 [redacted]')
|
|
31
|
+
.gsub(SECRET_PARAM) { "#{Regexp.last_match(1)}#{Regexp.last_match(2)}[redacted]" }
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def host(url)
|
|
35
|
+
URI.parse(url.to_s).host
|
|
36
|
+
rescue URI::InvalidURIError
|
|
37
|
+
nil
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
end
|
|
@@ -0,0 +1,293 @@
|
|
|
1
|
+
module Beagle
|
|
2
|
+
module Turso
|
|
3
|
+
# Keeps a synced Database's remote copy current: runs push (then pull) on
|
|
4
|
+
# a background thread, backs off on failure, reports status, and pushes
|
|
5
|
+
# one last time on stop. Framework-neutral and opt-in: nothing runs until
|
|
6
|
+
# #start, and it never closes the database (its owner does, after #stop).
|
|
7
|
+
#
|
|
8
|
+
# push/pull release the GVL (see the README), so the syncer's thread does
|
|
9
|
+
# not stall other Ruby threads or a fiber scheduler's reactor.
|
|
10
|
+
class Syncer
|
|
11
|
+
Status = Data.define(
|
|
12
|
+
:running, :syncing,
|
|
13
|
+
:last_push_at, :last_pull_at,
|
|
14
|
+
:last_error_class, :last_error_message, :last_error_at,
|
|
15
|
+
:consecutive_failures, :destination
|
|
16
|
+
)
|
|
17
|
+
|
|
18
|
+
# Retry delays are spread by ±JITTER so restarted processes don't retry
|
|
19
|
+
# in lockstep.
|
|
20
|
+
JITTER = 0.2
|
|
21
|
+
|
|
22
|
+
# Backoff stops growing past interval * 2**MAX_EXPONENT (max_backoff
|
|
23
|
+
# caps it long before), so a long outage never builds a bignum.
|
|
24
|
+
MAX_EXPONENT = 32
|
|
25
|
+
|
|
26
|
+
# How often a final push polls for an in-flight cycle to finish.
|
|
27
|
+
LOCK_POLL = 0.005
|
|
28
|
+
|
|
29
|
+
def initialize(database, interval: 30, pull: true, max_backoff: 300, on_error: nil, logger: nil, random: Random.new)
|
|
30
|
+
unless database.synced?
|
|
31
|
+
raise NotSyncedError, "Syncer needs a synced database (opened with remote_url: and auth_token:)"
|
|
32
|
+
end
|
|
33
|
+
raise ArgumentError, "interval must be positive" unless interval.positive?
|
|
34
|
+
raise ArgumentError, "max_backoff must be at least interval" if max_backoff < interval
|
|
35
|
+
|
|
36
|
+
@database = database
|
|
37
|
+
@interval = interval
|
|
38
|
+
@pull = pull
|
|
39
|
+
@max_backoff = max_backoff
|
|
40
|
+
@on_error = on_error
|
|
41
|
+
@logger = logger
|
|
42
|
+
@random = random
|
|
43
|
+
|
|
44
|
+
@cycle_lock = Mutex.new # held for the whole of one push/pull cycle
|
|
45
|
+
@state_lock = Mutex.new # guards the fields below and the wake-up signal
|
|
46
|
+
@wake = ConditionVariable.new
|
|
47
|
+
@thread = nil
|
|
48
|
+
@stopping = false
|
|
49
|
+
@stopped = false
|
|
50
|
+
@syncing = false
|
|
51
|
+
@last_push_at = nil
|
|
52
|
+
@last_pull_at = nil
|
|
53
|
+
@last_error_class = nil
|
|
54
|
+
@last_error_message = nil
|
|
55
|
+
@last_error_at = nil
|
|
56
|
+
@consecutive_failures = 0
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# Runs one cycle on the calling thread. Returns :ok, or :busy (without
|
|
60
|
+
# waiting) when another cycle is in progress. A failure is recorded in
|
|
61
|
+
# #status, reported to on_error/logger, then re-raised. If the message
|
|
62
|
+
# carried credentials, both on_error and the caller get a copy of the
|
|
63
|
+
# error (same class and backtrace) with the redacted message instead.
|
|
64
|
+
def sync_now
|
|
65
|
+
cycle(pull: @pull)
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
def status
|
|
69
|
+
@state_lock.synchronize do
|
|
70
|
+
Status.new(
|
|
71
|
+
running: loop_running?,
|
|
72
|
+
syncing: @syncing,
|
|
73
|
+
last_push_at: @last_push_at,
|
|
74
|
+
last_pull_at: @last_pull_at,
|
|
75
|
+
last_error_class: @last_error_class,
|
|
76
|
+
last_error_message: @last_error_message,
|
|
77
|
+
last_error_at: @last_error_at,
|
|
78
|
+
consecutive_failures: @consecutive_failures,
|
|
79
|
+
destination: @database.remote_host
|
|
80
|
+
)
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# Starts the background thread. Idempotent while running. A stopped
|
|
85
|
+
# syncer cannot be restarted; create a new one.
|
|
86
|
+
def start
|
|
87
|
+
@state_lock.synchronize do
|
|
88
|
+
raise Error, "syncer has been stopped; create a new one" if @stopped
|
|
89
|
+
return self if @thread
|
|
90
|
+
|
|
91
|
+
@thread = Thread.new { run_loop }
|
|
92
|
+
@thread.name = "beagle-turso-syncer"
|
|
93
|
+
end
|
|
94
|
+
self
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
# True while the background thread is alive and not stopping. A loop
|
|
98
|
+
# thread that died (e.g. on_error raised a non-StandardError) is not
|
|
99
|
+
# running.
|
|
100
|
+
def running?
|
|
101
|
+
@state_lock.synchronize { loop_running? }
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
# Stops the loop, waits for a cycle in flight on any other thread
|
|
105
|
+
# (the loop or a concurrent sync_now), then, if final_push and the
|
|
106
|
+
# deadline allows, pushes once more. Waits at most `timeout` seconds in
|
|
107
|
+
# total: an in-flight push or pull cannot be interrupted, so if the
|
|
108
|
+
# deadline passes first this returns anyway with status.syncing true and
|
|
109
|
+
# the work finishes in the background. Called from inside a cycle (from
|
|
110
|
+
# on_error), it cannot wait for that cycle and skips the final push.
|
|
111
|
+
# Never closes the database. Idempotent.
|
|
112
|
+
def stop(timeout: 10, final_push: true)
|
|
113
|
+
unless timeout.is_a?(Numeric) && timeout >= 0
|
|
114
|
+
raise ArgumentError, "timeout must be a non-negative number of seconds"
|
|
115
|
+
end
|
|
116
|
+
|
|
117
|
+
first = @state_lock.synchronize do
|
|
118
|
+
next false if @stopped
|
|
119
|
+
|
|
120
|
+
@stopped = true
|
|
121
|
+
@stopping = true
|
|
122
|
+
@wake.broadcast
|
|
123
|
+
true
|
|
124
|
+
end
|
|
125
|
+
return status unless first
|
|
126
|
+
|
|
127
|
+
deadline = monotonic_now + timeout
|
|
128
|
+
# Joining our own thread (stop called from on_error) would raise; that
|
|
129
|
+
# thread exits by itself once the callback returns.
|
|
130
|
+
join_quietly(@thread, deadline) if @thread && @thread != Thread.current
|
|
131
|
+
# Inside a cycle (stop from on_error) this thread holds the cycle lock,
|
|
132
|
+
# so waiting for it would only burn the deadline.
|
|
133
|
+
unless @cycle_lock.owned? || loop_alive?
|
|
134
|
+
if final_push && remaining(deadline).positive?
|
|
135
|
+
pusher = Thread.new do
|
|
136
|
+
cycle(pull: false, wait_until: deadline)
|
|
137
|
+
rescue StandardError
|
|
138
|
+
nil # already recorded, logged and reported by cycle
|
|
139
|
+
end
|
|
140
|
+
join_quietly(pusher, deadline)
|
|
141
|
+
elsif lock_by(deadline)
|
|
142
|
+
@cycle_lock.unlock # only waiting for the in-flight cycle
|
|
143
|
+
end
|
|
144
|
+
end
|
|
145
|
+
status
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
private
|
|
149
|
+
|
|
150
|
+
def run_loop
|
|
151
|
+
loop do
|
|
152
|
+
return if @state_lock.synchronize { @stopping }
|
|
153
|
+
|
|
154
|
+
delay =
|
|
155
|
+
begin
|
|
156
|
+
cycle(pull: @pull)
|
|
157
|
+
@interval
|
|
158
|
+
rescue StandardError
|
|
159
|
+
retry_delay(@state_lock.synchronize { @consecutive_failures })
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
@state_lock.synchronize do
|
|
163
|
+
return if @stopping
|
|
164
|
+
|
|
165
|
+
@wake.wait(@state_lock, delay)
|
|
166
|
+
end
|
|
167
|
+
end
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
def loop_alive?
|
|
171
|
+
@thread&.alive? && @thread != Thread.current
|
|
172
|
+
end
|
|
173
|
+
|
|
174
|
+
# Caller holds @state_lock.
|
|
175
|
+
def loop_running?
|
|
176
|
+
!!@thread&.alive? && !@stopping
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
# Joins a helper thread until the deadline. A thread that died from an
|
|
180
|
+
# exception re-raises it here; stop must still finish, so log it.
|
|
181
|
+
def join_quietly(thread, deadline)
|
|
182
|
+
thread.join(remaining(deadline))
|
|
183
|
+
rescue Exception => e # rubocop:disable Lint/RescueException
|
|
184
|
+
raise if thread.alive? # not the thread's death: e.g. an interrupt aimed at us
|
|
185
|
+
|
|
186
|
+
safe_warn("beagle-turso syncer thread died (#{e.class}): #{Redaction.message(e.message)}")
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
# Takes the cycle lock, waiting for an in-flight cycle until the
|
|
190
|
+
# deadline. False if the deadline passed first.
|
|
191
|
+
def lock_by(deadline)
|
|
192
|
+
until @cycle_lock.try_lock
|
|
193
|
+
left = remaining(deadline)
|
|
194
|
+
return false unless left.positive?
|
|
195
|
+
|
|
196
|
+
sleep([LOCK_POLL, left].min)
|
|
197
|
+
end
|
|
198
|
+
true
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
def remaining(deadline)
|
|
202
|
+
[deadline - monotonic_now, 0].max
|
|
203
|
+
end
|
|
204
|
+
|
|
205
|
+
def monotonic_now
|
|
206
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
207
|
+
end
|
|
208
|
+
|
|
209
|
+
# One push (then pull) under the cycle lock. Without wait_until it never
|
|
210
|
+
# waits (returns :busy); with it, it waits for the lock until then.
|
|
211
|
+
def cycle(pull:, wait_until: nil)
|
|
212
|
+
locked = wait_until ? lock_by(wait_until) : @cycle_lock.try_lock
|
|
213
|
+
return :busy unless locked
|
|
214
|
+
|
|
215
|
+
begin
|
|
216
|
+
set_syncing(true)
|
|
217
|
+
@database.push
|
|
218
|
+
@state_lock.synchronize { @last_push_at = Time.now }
|
|
219
|
+
if pull
|
|
220
|
+
@database.pull
|
|
221
|
+
@state_lock.synchronize { @last_pull_at = Time.now }
|
|
222
|
+
end
|
|
223
|
+
@state_lock.synchronize { @consecutive_failures = 0 }
|
|
224
|
+
:ok
|
|
225
|
+
rescue StandardError => e
|
|
226
|
+
reported = record_failure(e)
|
|
227
|
+
raise e if reported.equal?(e)
|
|
228
|
+
|
|
229
|
+
raise reported, cause: nil # the cause would carry the unredacted original
|
|
230
|
+
ensure
|
|
231
|
+
begin
|
|
232
|
+
set_syncing(false)
|
|
233
|
+
ensure
|
|
234
|
+
@cycle_lock.unlock
|
|
235
|
+
end
|
|
236
|
+
end
|
|
237
|
+
end
|
|
238
|
+
|
|
239
|
+
def set_syncing(value)
|
|
240
|
+
@state_lock.synchronize { @syncing = value }
|
|
241
|
+
end
|
|
242
|
+
|
|
243
|
+
# Records, logs and reports a failure. Returns the error to hand on:
|
|
244
|
+
# the original, or a redacted copy if its message carried credentials.
|
|
245
|
+
def record_failure(error)
|
|
246
|
+
message = Redaction.message(error.message).freeze
|
|
247
|
+
@state_lock.synchronize do
|
|
248
|
+
@consecutive_failures += 1
|
|
249
|
+
@last_error_class = error.class.to_s
|
|
250
|
+
@last_error_message = message
|
|
251
|
+
@last_error_at = Time.now
|
|
252
|
+
end
|
|
253
|
+
safe_warn("beagle-turso sync failed (#{error.class}): #{message}")
|
|
254
|
+
reported = redacted_error(error, message)
|
|
255
|
+
notify(reported)
|
|
256
|
+
reported
|
|
257
|
+
end
|
|
258
|
+
|
|
259
|
+
def redacted_error(error, message)
|
|
260
|
+
return error if message == error.message
|
|
261
|
+
|
|
262
|
+
copy = begin
|
|
263
|
+
error.exception(message)
|
|
264
|
+
rescue StandardError
|
|
265
|
+
nil
|
|
266
|
+
end
|
|
267
|
+
copy = Error.new(message) unless copy.is_a?(Exception) && copy.message == message
|
|
268
|
+
copy.set_backtrace(error.backtrace) if copy.backtrace.nil? && error.backtrace
|
|
269
|
+
copy
|
|
270
|
+
end
|
|
271
|
+
|
|
272
|
+
def notify(error)
|
|
273
|
+
@on_error&.call(error, status)
|
|
274
|
+
rescue StandardError => e
|
|
275
|
+
safe_warn("beagle-turso syncer on_error raised (#{e.class}): #{Redaction.message(e.message)}")
|
|
276
|
+
end
|
|
277
|
+
|
|
278
|
+
# A broken logger must not skip on_error or replace the sync error.
|
|
279
|
+
def safe_warn(message)
|
|
280
|
+
@logger&.warn(message)
|
|
281
|
+
rescue StandardError
|
|
282
|
+
nil
|
|
283
|
+
end
|
|
284
|
+
|
|
285
|
+
# Delay before the next attempt after `failures` consecutive failures:
|
|
286
|
+
# interval * 2**failures, capped at max_backoff, then spread by ±JITTER.
|
|
287
|
+
def retry_delay(failures)
|
|
288
|
+
base = [@interval * (2**[failures, MAX_EXPONENT].min), @max_backoff].min
|
|
289
|
+
base * (1 + @random.rand(-JITTER..JITTER))
|
|
290
|
+
end
|
|
291
|
+
end
|
|
292
|
+
end
|
|
293
|
+
end
|
data/lib/beagle/turso/version.rb
CHANGED
data/lib/beagle/turso.rb
CHANGED
|
@@ -1,4 +1,7 @@
|
|
|
1
1
|
require_relative "turso/version"
|
|
2
|
+
require_relative "turso/errors"
|
|
3
|
+
require_relative "turso/redaction"
|
|
4
|
+
require_relative "turso/syncer"
|
|
2
5
|
|
|
3
6
|
# Load the native extension. Precompiled ("fat") gems ship the .so under a
|
|
4
7
|
# per-Ruby-ABI subdir (lib/beagle_turso/<major.minor>/beagle_turso.so); a
|
|
@@ -16,12 +19,28 @@ module Beagle
|
|
|
16
19
|
# Opens a local-only database, or one synced with a remote Turso
|
|
17
20
|
# database when both `remote_url` and `auth_token` are given (see
|
|
18
21
|
# `#push`/`#pull`). Delegates to the native `_open` primitive, which
|
|
19
|
-
# takes its arguments positionally.
|
|
22
|
+
# takes its arguments positionally. Only the remote host is kept on
|
|
23
|
+
# the Ruby object; the token is never stored.
|
|
20
24
|
def self.open(local_path:, remote_url: nil, auth_token: nil, bootstrap_if_empty: true)
|
|
21
|
-
_open(local_path, remote_url, auth_token, bootstrap_if_empty)
|
|
25
|
+
db = _open(local_path, remote_url, auth_token, bootstrap_if_empty)
|
|
26
|
+
if remote_url && auth_token
|
|
27
|
+
db.instance_variable_set(:@synced, true)
|
|
28
|
+
db.instance_variable_set(:@remote_host, Redaction.host(remote_url))
|
|
29
|
+
end
|
|
30
|
+
db
|
|
22
31
|
end
|
|
23
32
|
|
|
24
33
|
private_class_method :_open
|
|
34
|
+
|
|
35
|
+
# True when opened with both remote_url: and auth_token:, i.e. when
|
|
36
|
+
# #push/#pull talk to a remote.
|
|
37
|
+
def synced?
|
|
38
|
+
@synced == true
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# The remote database's host, without scheme, path, query or
|
|
42
|
+
# credentials; nil for a local-only database.
|
|
43
|
+
attr_reader :remote_host
|
|
25
44
|
end
|
|
26
45
|
end
|
|
27
46
|
end
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
[diffend] Oversized file quarantined before diffing.
|
|
2
2
|
name: data/lib/beagle_turso/3.3/beagle_turso.so
|
|
3
|
-
size:
|
|
4
|
-
sha256:
|
|
3
|
+
size: 36921488 bytes
|
|
4
|
+
sha256: 044a5218fd95dba89cb3fe0b6f8be54f15eaee467f8ab8e5569b86174668804e
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
[diffend] Oversized file quarantined before diffing.
|
|
2
2
|
name: data/lib/beagle_turso/3.4/beagle_turso.so
|
|
3
|
-
size:
|
|
4
|
-
sha256:
|
|
3
|
+
size: 36936656 bytes
|
|
4
|
+
sha256: 6caa37e3b69c317e3efb32e4d42747e2d4efbf1cac3bbbfd73196b420116aa02
|
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
[diffend] Oversized file quarantined before diffing.
|
|
2
2
|
name: data/lib/beagle_turso/4.0/beagle_turso.so
|
|
3
|
-
size:
|
|
4
|
-
sha256:
|
|
3
|
+
size: 36936656 bytes
|
|
4
|
+
sha256: ff8624044f732e37f46f4d65a2230412dfd0f738d2831da40c94cf751d62eed2
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: beagle-turso
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: aarch64-linux
|
|
6
6
|
authors:
|
|
7
7
|
- BeagleSoftwareUK
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-10-17 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: |
|
|
14
14
|
A Ruby driver for Turso's database engine, backed by a native Rust
|
|
@@ -24,6 +24,9 @@ extra_rdoc_files: []
|
|
|
24
24
|
files:
|
|
25
25
|
- README.md
|
|
26
26
|
- lib/beagle/turso.rb
|
|
27
|
+
- lib/beagle/turso/errors.rb
|
|
28
|
+
- lib/beagle/turso/redaction.rb
|
|
29
|
+
- lib/beagle/turso/syncer.rb
|
|
27
30
|
- lib/beagle/turso/version.rb
|
|
28
31
|
- lib/beagle_turso/3.3/beagle_turso.so
|
|
29
32
|
- lib/beagle_turso/3.4/beagle_turso.so
|