slatedb 0.6.1-aarch64-linux → 0.7.0-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +114 -44
- data/lib/slatedb/3.3/slatedb.so +0 -0
- data/lib/slatedb/3.4/slatedb.so +0 -0
- data/lib/slatedb/4.0/slatedb.so +0 -0
- data/lib/slatedb/database.rb +40 -26
- data/lib/slatedb/reader.rb +25 -19
- data/lib/slatedb/snapshot.rb +17 -9
- data/lib/slatedb/transaction.rb +17 -9
- data/lib/slatedb/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: af0687022f9a14510d013da646aefbd6fedfa2b770235460a1e9d416b05d444a
|
|
4
|
+
data.tar.gz: ea04403866e4d528bb85e07a5a08392a6e2e73b073eea7c7685d2324fae72cf2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5dd9197393629666a17468ae03966a34cad89313a8aeac54bcec427f40baf91ae1f7964fdcd1a06d0a4d2847170a013b6da0c6aafa49a8cb0256466cde133427
|
|
7
|
+
data.tar.gz: 2c01206156a4ff9c9a037df96b08c936ca055d1888dd52adc4817b34d0a99a81e174d7ab9078ac01fc2a6293885758ec8ba63924d85ac38f05fa0477a8760aa5
|
data/README.md
CHANGED
|
@@ -30,6 +30,38 @@ gem install slatedb
|
|
|
30
30
|
|
|
31
31
|
Precompiled native gems are published for common Linux and Darwin platforms. Building from source still requires a Rust toolchain.
|
|
32
32
|
|
|
33
|
+
## Upgrading to SlateDB 0.18.1
|
|
34
|
+
|
|
35
|
+
This update moves the Rust engine from 0.17.0 to 0.18.1. Existing database files
|
|
36
|
+
remain readable. New Ruby options keep the existing defaults unless enabled.
|
|
37
|
+
|
|
38
|
+
- Two-phase checkpoints: `begin_checkpoint` returns a handle with `id` and `wait`.
|
|
39
|
+
Both checkpoint methods accept `scope: :all` to capture pending writes, or
|
|
40
|
+
`scope: :durable` (the existing default) for writes already durable.
|
|
41
|
+
- Point reads accept `max_concurrent_sources:` on databases, readers, snapshots,
|
|
42
|
+
and transactions; database metadata reads accept it too.
|
|
43
|
+
- `Database.open` accepts `experimental_incremental_compaction:` and
|
|
44
|
+
`sst_block_alignment:`. Both are disabled by default; see the warnings below.
|
|
45
|
+
- Snapshot and transaction range/prefix scans now accept `order:`, like database
|
|
46
|
+
and reader scans. Reader `merge_operator:` is now applied rather than ignored.
|
|
47
|
+
- **Breaking:** Ruby Proc/lambda merge operators are no longer accepted. They
|
|
48
|
+
silently used concatenation on other threads, which could change stored values.
|
|
49
|
+
Keep `:concat` / `:string_concat` for concatenation. For custom updates, use a
|
|
50
|
+
serializable transaction that reads the value and writes the computed result;
|
|
51
|
+
handle `SlateDb::TransactionError` conflicts in the application. Databases with
|
|
52
|
+
unresolved custom merge operands need an application-specific migration before
|
|
53
|
+
upgrading. Do not reopen them with `:concat` unless concatenation was their
|
|
54
|
+
intended merge rule.
|
|
55
|
+
- Reader disk-cache keywords remain supported, but upstream deprecated the
|
|
56
|
+
underlying `CachedObjectStore` in 0.18.1. The new mirror and Rust trait-based
|
|
57
|
+
extension APIs are not exposed by this gem yet.
|
|
58
|
+
|
|
59
|
+
Upstream also fixes clone-compaction retention/data-loss cases, writer ownership
|
|
60
|
+
validation after WAL replay, write stalls, upload retry/concurrency behavior, and
|
|
61
|
+
iterator fetch cancellation. It improves memtable seeks and point-read/scan work.
|
|
62
|
+
See the full [0.18.0 release notes](https://github.com/slatedb/slatedb/releases/tag/v0.18.0)
|
|
63
|
+
and [0.18.1 release notes](https://github.com/slatedb/slatedb/releases/tag/v0.18.1).
|
|
64
|
+
|
|
33
65
|
## Usage
|
|
34
66
|
|
|
35
67
|
### Basic Operations
|
|
@@ -64,6 +96,19 @@ SlateDb::Database.open("/tmp/mydb") do |db|
|
|
|
64
96
|
end # automatically closed
|
|
65
97
|
```
|
|
66
98
|
|
|
99
|
+
### Point-Read Concurrency
|
|
100
|
+
|
|
101
|
+
```ruby
|
|
102
|
+
db.get("key", max_concurrent_sources: 4, cache_blocks: false)
|
|
103
|
+
db.get_entry("key", max_concurrent_sources: 4) # value plus sequence/timestamps
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
`max_concurrent_sources` defaults to 1 (serial fetching); zero also means 1.
|
|
107
|
+
SlateDB checks the newest source alone, then fetches up to this many sources
|
|
108
|
+
concurrently after a miss. Higher values can reduce latency but increase object
|
|
109
|
+
store requests and bandwidth. The same keyword works with `get` on readers,
|
|
110
|
+
snapshots, and transactions. It does not change isolation or durability.
|
|
111
|
+
|
|
67
112
|
### Compaction Controls
|
|
68
113
|
|
|
69
114
|
SlateDB 0.17 adds controls for how long compaction inputs remain protected from
|
|
@@ -87,6 +132,32 @@ projected sorted-run count exceeds the threshold. It is not a hard limit.
|
|
|
87
132
|
Zero disables it (the default); use at least two to enable it. SlateDB warns and
|
|
88
133
|
disables the trigger for one. Low thresholds can increase storage reads and writes.
|
|
89
134
|
|
|
135
|
+
SlateDB 0.18 adds an opt-in experimental mode:
|
|
136
|
+
|
|
137
|
+
```ruby
|
|
138
|
+
SlateDb::Database.open("mydb", url: "s3://mybucket/path",
|
|
139
|
+
experimental_incremental_compaction: true) do |db|
|
|
140
|
+
db.put("key", "value")
|
|
141
|
+
end
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
This publishes output from sorted-run compactions at most once a minute before
|
|
145
|
+
the full compaction finishes. It can reduce read amplification and allow earlier
|
|
146
|
+
storage reclamation after the compactor checkpoint expires. Compactions with L0
|
|
147
|
+
inputs still publish only when complete. It is disabled by default, experimental,
|
|
148
|
+
and may change or disappear upstream.
|
|
149
|
+
|
|
150
|
+
### SST Block Alignment
|
|
151
|
+
|
|
152
|
+
`Database.open(..., sst_block_alignment: true)` pads SST data blocks to 4 KiB
|
|
153
|
+
boundaries, including oversized entries. It is disabled by default, uses more
|
|
154
|
+
storage, and cannot be combined with compression.
|
|
155
|
+
|
|
156
|
+
**Upgrade every reader before enabling alignment on a writer.** Older engines
|
|
157
|
+
that do not understand padded SSTs cannot read them. Once padded files exist,
|
|
158
|
+
rolling those readers back is not safe; disabling the option does not rewrite
|
|
159
|
+
existing files.
|
|
160
|
+
|
|
90
161
|
### Key Size
|
|
91
162
|
|
|
92
163
|
Keys can now exceed the previous 65,535-byte limit, up to SlateDB's new
|
|
@@ -341,53 +412,19 @@ db.batch do |b|
|
|
|
341
412
|
end
|
|
342
413
|
```
|
|
343
414
|
|
|
344
|
-
####
|
|
345
|
-
|
|
346
|
-
You can provide a Ruby Proc/lambda as a custom merge operator:
|
|
347
|
-
|
|
348
|
-
```ruby
|
|
349
|
-
# Counter merge operator (adds numbers)
|
|
350
|
-
counter_merge = ->(key, existing, new_value) {
|
|
351
|
-
existing_num = existing ? existing.to_i : 0
|
|
352
|
-
(existing_num + new_value.to_i).to_s
|
|
353
|
-
}
|
|
354
|
-
|
|
355
|
-
SlateDb::Database.open("/tmp/mydb", merge_operator: counter_merge) do |db|
|
|
356
|
-
db.merge("visits", "1")
|
|
357
|
-
db.merge("visits", "1")
|
|
358
|
-
db.merge("visits", "1")
|
|
359
|
-
|
|
360
|
-
db.get("visits") # => "3"
|
|
361
|
-
end
|
|
415
|
+
#### Available Merge Operators
|
|
362
416
|
|
|
363
|
-
|
|
364
|
-
max_merge = ->(key, existing, new_value) {
|
|
365
|
-
existing_num = existing ? existing.to_i : 0
|
|
366
|
-
new_num = new_value.to_i
|
|
367
|
-
[existing_num, new_num].max.to_s
|
|
368
|
-
}
|
|
417
|
+
- `:string_concat` (or `:concat`) - Concatenates byte values (built-in)
|
|
369
418
|
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
db.merge("high_score", "250")
|
|
373
|
-
db.merge("high_score", "150")
|
|
419
|
+
Ruby Proc/lambda operators are rejected. Use a serializable read/modify/write
|
|
420
|
+
transaction for custom logic, with application-level conflict handling:
|
|
374
421
|
|
|
375
|
-
|
|
422
|
+
```ruby
|
|
423
|
+
db.transaction(isolation: :serializable) do |txn|
|
|
424
|
+
txn.put("visits", (txn.get("visits").to_i + 1).to_s)
|
|
376
425
|
end
|
|
377
426
|
```
|
|
378
427
|
|
|
379
|
-
The proc receives three arguments:
|
|
380
|
-
- `key` - The key being merged
|
|
381
|
-
- `existing` - The existing value (nil if no value exists)
|
|
382
|
-
- `new_value` - The new merge operand
|
|
383
|
-
|
|
384
|
-
**Note:** Custom Proc merge operators work best with direct `db.merge()` calls. When used with transactions or batches, some merge operations may be processed on background threads and fall back to string concatenation.
|
|
385
|
-
|
|
386
|
-
#### Available Merge Operators
|
|
387
|
-
|
|
388
|
-
- `:string_concat` (or `:concat`) - Concatenates byte values (built-in)
|
|
389
|
-
- Any `Proc` or `lambda` - Custom merge logic
|
|
390
|
-
|
|
391
428
|
### Write Batches
|
|
392
429
|
|
|
393
430
|
Perform multiple writes atomically:
|
|
@@ -495,6 +532,29 @@ SlateDb::Database.open("/tmp/mydb", url: "file:///tmp/mydb") do |db|
|
|
|
495
532
|
end
|
|
496
533
|
```
|
|
497
534
|
|
|
535
|
+
`create_checkpoint` waits until the checkpoint is stored. Its default
|
|
536
|
+
`scope: :durable` preserves the existing behavior. To include writes made with
|
|
537
|
+
`await_durable: false`, use `scope: :all`; no separate `flush` is needed.
|
|
538
|
+
|
|
539
|
+
To separate capturing the write boundary from waiting for storage:
|
|
540
|
+
|
|
541
|
+
```ruby
|
|
542
|
+
db.put("key", "before", await_durable: false)
|
|
543
|
+
handle = db.begin_checkpoint(scope: :all, name: "backup", lifetime: 3_600_000)
|
|
544
|
+
id = handle.id # UUID is available immediately
|
|
545
|
+
db.put("key", "after") # not part of this checkpoint
|
|
546
|
+
checkpoint = handle.wait # => { id: id, manifest_id: ... }
|
|
547
|
+
```
|
|
548
|
+
|
|
549
|
+
With `scope: :all`, writes completed before `begin_checkpoint` starts are included;
|
|
550
|
+
writes started after it returns are excluded. Concurrent writes during the call
|
|
551
|
+
may be included. Starting the checkpoint can still wait for internal flush work;
|
|
552
|
+
it is not a fully nonblocking Ruby call. `wait` releases the Ruby GVL while
|
|
553
|
+
waiting, consumes the handle, and raises `SlateDb::ClosedError` if called again.
|
|
554
|
+
`id` remains readable. Dropping the handle does not cancel or delete the checkpoint.
|
|
555
|
+
If waiting fails, use the UUID with `Admin#list_checkpoints` to check its outcome
|
|
556
|
+
before starting another checkpoint.
|
|
557
|
+
|
|
498
558
|
### Snapshots
|
|
499
559
|
|
|
500
560
|
Point-in-time consistent reads:
|
|
@@ -517,6 +577,11 @@ value = snap.get("key")
|
|
|
517
577
|
snap.close
|
|
518
578
|
```
|
|
519
579
|
|
|
580
|
+
Range and prefix scans on snapshots and transactions accept the same `order:`
|
|
581
|
+
keyword as databases and readers, including `:asc`/`:ascending` and
|
|
582
|
+
`:desc`/`:descending`. Bounds remain lower-inclusive and upper-exclusive in
|
|
583
|
+
either direction.
|
|
584
|
+
|
|
520
585
|
### Reader (Read-Only Access)
|
|
521
586
|
|
|
522
587
|
Open a database in read-only mode, useful for replicas:
|
|
@@ -568,6 +633,14 @@ SlateDb::Reader.open("/tmp/mydb",
|
|
|
568
633
|
end
|
|
569
634
|
```
|
|
570
635
|
|
|
636
|
+
When the writer uses `merge_operator: :concat`, pass the same operator to
|
|
637
|
+
`Reader.open` so WAL replay and reads apply the correct merge rule. Unknown
|
|
638
|
+
operator names now raise `SlateDb::InvalidArgumentError` instead of being ignored.
|
|
639
|
+
|
|
640
|
+
The `cache_root` and `max_open_file_handles` keywords still work, but SlateDB
|
|
641
|
+
0.18.1 deprecates this legacy disk cache in favor of its separate mirror crate or
|
|
642
|
+
tiered block cache. Those replacement APIs are not yet exposed here.
|
|
643
|
+
|
|
571
644
|
### Admin Operations
|
|
572
645
|
|
|
573
646
|
Administrative operations for database management:
|
|
@@ -594,9 +667,6 @@ admin.delete_checkpoint("uuid")
|
|
|
594
667
|
# Garbage Collection
|
|
595
668
|
admin.run_gc # Run with default settings
|
|
596
669
|
admin.run_gc(min_age: 3600_000) # Set min age for all directories (1 hour)
|
|
597
|
-
admin.run_gc(manifest_min_age: 86400_000) # Custom age for manifest (1 day)
|
|
598
|
-
admin.run_gc(wal_min_age: 60_000) # Custom age for WAL (1 minute)
|
|
599
|
-
admin.run_gc(compacted_min_age: 60_000) # Custom age for compacted (1 minute)
|
|
600
670
|
|
|
601
671
|
# Delete a database (SlateDB >= 0.16.0)
|
|
602
672
|
would_delete = admin.delete_db # Dry run: returns the object paths
|
data/lib/slatedb/3.3/slatedb.so
CHANGED
|
Binary file
|
data/lib/slatedb/3.4/slatedb.so
CHANGED
|
Binary file
|
data/lib/slatedb/4.0/slatedb.so
CHANGED
|
Binary file
|
data/lib/slatedb/database.rb
CHANGED
|
@@ -9,15 +9,18 @@ module SlateDb
|
|
|
9
9
|
#
|
|
10
10
|
# @param path [String] The path identifier for the database
|
|
11
11
|
# @param url [String, nil] Optional object store URL (e.g., "s3://bucket/path")
|
|
12
|
-
# @param merge_operator [Symbol, String,
|
|
13
|
-
#
|
|
14
|
-
# that takes (key, existing_value, new_value) and returns the merged value.
|
|
12
|
+
# @param merge_operator [Symbol, String, nil] Optional built-in merge operator
|
|
13
|
+
# (:string_concat or :concat). Ruby callbacks are not supported.
|
|
15
14
|
# @param compactor_checkpoint_lifetime [Integer, nil] How long compaction inputs
|
|
16
15
|
# are protected from garbage collection, in milliseconds (default: 15 minutes).
|
|
17
16
|
# Reads using those inputs must finish within this time.
|
|
18
17
|
# @param sorted_run_consolidation_threshold [Integer, nil] Sorted-run count above
|
|
19
18
|
# which fallback consolidation starts. Zero disables it (default); use at least
|
|
20
19
|
# two to enable it. One is disabled by SlateDB with a warning. This is not a hard limit.
|
|
20
|
+
# @param experimental_incremental_compaction [Boolean, nil] Publish sorted-run
|
|
21
|
+
# compaction output before completion (experimental, disabled by default).
|
|
22
|
+
# @param sst_block_alignment [Boolean, nil] Align SST data blocks to 4 KiB
|
|
23
|
+
# boundaries (disabled by default; incompatible with compression).
|
|
21
24
|
# @yield [db] If a block is given, yields the database and ensures it's closed
|
|
22
25
|
# @return [Database] The opened database (or block result if block given)
|
|
23
26
|
#
|
|
@@ -34,32 +37,22 @@ module SlateDb
|
|
|
34
37
|
# @example Open with S3 backend
|
|
35
38
|
# db = SlateDb::Database.open("/tmp/mydb", url: "s3://mybucket/path")
|
|
36
39
|
#
|
|
37
|
-
# @example Open with a custom merge operator (Proc)
|
|
38
|
-
# # Custom merge that adds numbers
|
|
39
|
-
# db = SlateDb::Database.open("/tmp/mydb", merge_operator: ->(key, existing, new_val) {
|
|
40
|
-
# existing_num = existing ? existing.to_i : 0
|
|
41
|
-
# (existing_num + new_val.to_i).to_s
|
|
42
|
-
# })
|
|
43
|
-
# db.merge("counter", "5")
|
|
44
|
-
# db.merge("counter", "3")
|
|
45
|
-
# db.get("counter") # => "8"
|
|
46
|
-
#
|
|
47
40
|
def open(path, url: nil, merge_operator: nil,
|
|
48
|
-
compactor_checkpoint_lifetime: nil, sorted_run_consolidation_threshold: nil
|
|
41
|
+
compactor_checkpoint_lifetime: nil, sorted_run_consolidation_threshold: nil,
|
|
42
|
+
experimental_incremental_compaction: nil, sst_block_alignment: nil)
|
|
49
43
|
opts = {
|
|
50
44
|
compactor_checkpoint_lifetime: compactor_checkpoint_lifetime,
|
|
51
|
-
sorted_run_consolidation_threshold: sorted_run_consolidation_threshold
|
|
45
|
+
sorted_run_consolidation_threshold: sorted_run_consolidation_threshold,
|
|
46
|
+
experimental_incremental_compaction: experimental_incremental_compaction,
|
|
47
|
+
sst_block_alignment: sst_block_alignment
|
|
52
48
|
}.compact
|
|
53
49
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
opts[:merge_operator] = merge_operator.to_s
|
|
57
|
-
when Proc
|
|
58
|
-
# Store the proc to prevent GC and pass to Rust
|
|
59
|
-
@_merge_operator_proc = merge_operator
|
|
60
|
-
opts[:merge_operator_proc] = merge_operator
|
|
50
|
+
unless merge_operator.nil? || merge_operator.is_a?(Symbol) || merge_operator.is_a?(String)
|
|
51
|
+
raise ArgumentError, "merge_operator must be :string_concat or :concat; Ruby callbacks are not supported"
|
|
61
52
|
end
|
|
62
53
|
|
|
54
|
+
opts[:merge_operator] = merge_operator.to_s if merge_operator
|
|
55
|
+
|
|
63
56
|
db = _open(path, url, opts)
|
|
64
57
|
|
|
65
58
|
if block_given?
|
|
@@ -84,6 +77,8 @@ module SlateDb
|
|
|
84
77
|
# @param durability_filter [String, nil] Filter by durability level ("remote" or "memory")
|
|
85
78
|
# @param dirty [Boolean, nil] Whether to include uncommitted data
|
|
86
79
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
80
|
+
# @param max_concurrent_sources [Integer, nil] Concurrent sources to fetch after
|
|
81
|
+
# the first source misses (default: 1; zero also means serial fetching).
|
|
87
82
|
# @return [String, nil] The value, or nil if not found
|
|
88
83
|
#
|
|
89
84
|
# @example Basic get
|
|
@@ -92,11 +87,12 @@ module SlateDb
|
|
|
92
87
|
# @example Get with options
|
|
93
88
|
# value = db.get("mykey", durability_filter: "memory", dirty: true)
|
|
94
89
|
#
|
|
95
|
-
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
|
|
90
|
+
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
|
|
96
91
|
opts = {}
|
|
97
92
|
opts[:durability_filter] = durability_filter.to_s if durability_filter
|
|
98
93
|
opts[:dirty] = dirty unless dirty.nil?
|
|
99
94
|
opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
|
|
95
|
+
opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
|
|
100
96
|
|
|
101
97
|
if opts.empty?
|
|
102
98
|
_get(key)
|
|
@@ -112,17 +108,20 @@ module SlateDb
|
|
|
112
108
|
# @param dirty [Boolean, nil] Whether to include uncommitted data
|
|
113
109
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
114
110
|
# @return [Hash, nil] A hash with :key, :value, :seq, :create_ts, and :expire_ts, or nil if not found
|
|
111
|
+
# @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
|
|
112
|
+
# miss. See {#get}.
|
|
115
113
|
#
|
|
116
114
|
# @example Inspect metadata
|
|
117
115
|
# entry = db.get_key_value("mykey")
|
|
118
116
|
# entry[:value] # => "myvalue"
|
|
119
117
|
# entry[:seq] # => sequence number
|
|
120
118
|
#
|
|
121
|
-
def get_key_value(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
|
|
119
|
+
def get_key_value(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
|
|
122
120
|
opts = {}
|
|
123
121
|
opts[:durability_filter] = durability_filter.to_s if durability_filter
|
|
124
122
|
opts[:dirty] = dirty unless dirty.nil?
|
|
125
123
|
opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
|
|
124
|
+
opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
|
|
126
125
|
|
|
127
126
|
if opts.empty?
|
|
128
127
|
_get_key_value(key)
|
|
@@ -494,6 +493,8 @@ module SlateDb
|
|
|
494
493
|
#
|
|
495
494
|
# @param lifetime [Integer, nil] Checkpoint lifetime in milliseconds
|
|
496
495
|
# @param name [String, nil] Optional name for the checkpoint
|
|
496
|
+
# @param scope [Symbol, String] :durable (default) captures durable writes;
|
|
497
|
+
# :all also flushes pending writes before storing the checkpoint.
|
|
497
498
|
# @return [Hash] Hash with :id (UUID string) and :manifest_id (integer)
|
|
498
499
|
#
|
|
499
500
|
# @example Create a named checkpoint
|
|
@@ -503,13 +504,26 @@ module SlateDb
|
|
|
503
504
|
# @example Create a checkpoint with lifetime
|
|
504
505
|
# checkpoint = db.create_checkpoint(lifetime: 3600_000) # 1 hour
|
|
505
506
|
#
|
|
506
|
-
def create_checkpoint(lifetime: nil, name: nil)
|
|
507
|
-
opts = {}
|
|
507
|
+
def create_checkpoint(lifetime: nil, name: nil, scope: :durable)
|
|
508
|
+
opts = { scope: scope.to_s }
|
|
508
509
|
opts[:lifetime] = lifetime if lifetime
|
|
509
510
|
opts[:name] = name if name
|
|
510
511
|
_create_checkpoint(opts)
|
|
511
512
|
end
|
|
512
513
|
|
|
514
|
+
# Start a checkpoint without waiting for it to be stored.
|
|
515
|
+
#
|
|
516
|
+
# With scope: :all, captures writes completed before this call and excludes
|
|
517
|
+
# writes started after it returns. Writes during the call may be included.
|
|
518
|
+
# Options match {#create_checkpoint}. Dropping the handle does not cancel it.
|
|
519
|
+
# @return [CheckpointHandle] #id is available immediately; #wait returns the
|
|
520
|
+
# :id/:manifest_id hash once stored and consumes the handle. A second wait
|
|
521
|
+
# raises ClosedError. #id remains available after waiting.
|
|
522
|
+
def begin_checkpoint(lifetime: nil, name: nil, scope: :durable)
|
|
523
|
+
opts = { scope: scope.to_s, lifetime: lifetime, name: name }.compact
|
|
524
|
+
_begin_checkpoint(opts)
|
|
525
|
+
end
|
|
526
|
+
|
|
513
527
|
# Get database metrics registry.
|
|
514
528
|
#
|
|
515
529
|
# @return [Metrics] Metrics registry
|
data/lib/slatedb/reader.rb
CHANGED
|
@@ -21,6 +21,7 @@ module SlateDb
|
|
|
21
21
|
# @param cache_root [String, nil] Root folder for the reader's local on-disk
|
|
22
22
|
# object-store cache. Setting this enables the cached object store; when it is
|
|
23
23
|
# not set the cache (and `max_open_file_handles`) has no effect.
|
|
24
|
+
# This legacy cache is deprecated upstream in SlateDB 0.18.1.
|
|
24
25
|
# @param max_open_file_handles [Integer, nil] Maximum number of file handles to keep
|
|
25
26
|
# open in the reader's file-handle cache. When the limit is reached, the least
|
|
26
27
|
# recently used handle is closed (default: 1000). Only takes effect when
|
|
@@ -54,29 +55,31 @@ module SlateDb
|
|
|
54
55
|
manifest_poll_interval: nil, checkpoint_lifetime: nil,
|
|
55
56
|
max_memtable_bytes: nil, cache_root: nil, max_open_file_handles: nil,
|
|
56
57
|
merge_operator: nil)
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
opts
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
58
|
+
unless merge_operator.nil? || merge_operator.is_a?(Symbol) || merge_operator.is_a?(String)
|
|
59
|
+
raise ArgumentError, "merge_operator must be :string_concat or :concat; Ruby callbacks are not supported"
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
opts = {
|
|
63
|
+
follow_latest: follow_latest,
|
|
64
|
+
manifest_poll_interval: manifest_poll_interval,
|
|
65
|
+
checkpoint_lifetime: checkpoint_lifetime,
|
|
66
|
+
max_memtable_bytes: max_memtable_bytes,
|
|
67
|
+
cache_root: cache_root,
|
|
68
|
+
max_open_file_handles: max_open_file_handles,
|
|
69
|
+
merge_operator: merge_operator&.to_s
|
|
70
|
+
}.compact
|
|
65
71
|
|
|
66
72
|
reader = _open(path, url, checkpoint_id, opts)
|
|
73
|
+
return reader unless block_given?
|
|
67
74
|
|
|
68
|
-
|
|
75
|
+
begin
|
|
76
|
+
yield reader
|
|
77
|
+
ensure
|
|
69
78
|
begin
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
reader.close
|
|
74
|
-
rescue StandardError
|
|
75
|
-
nil
|
|
76
|
-
end
|
|
79
|
+
reader.close
|
|
80
|
+
rescue StandardError
|
|
81
|
+
nil
|
|
77
82
|
end
|
|
78
|
-
else
|
|
79
|
-
reader
|
|
80
83
|
end
|
|
81
84
|
end
|
|
82
85
|
end
|
|
@@ -87,13 +90,16 @@ module SlateDb
|
|
|
87
90
|
# @param durability_filter [String, nil] Filter by durability level ("remote" or "memory")
|
|
88
91
|
# @param dirty [Boolean, nil] Whether to include uncommitted data
|
|
89
92
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
93
|
+
# @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
|
|
94
|
+
# miss (default: 1; zero also means serial fetching).
|
|
90
95
|
# @return [String, nil] The value, or nil if not found
|
|
91
96
|
#
|
|
92
|
-
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
|
|
97
|
+
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
|
|
93
98
|
opts = {}
|
|
94
99
|
opts[:durability_filter] = durability_filter.to_s if durability_filter
|
|
95
100
|
opts[:dirty] = dirty unless dirty.nil?
|
|
96
101
|
opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
|
|
102
|
+
opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
|
|
97
103
|
|
|
98
104
|
if opts.empty?
|
|
99
105
|
_get(key)
|
data/lib/slatedb/snapshot.rb
CHANGED
|
@@ -8,13 +8,16 @@ module SlateDb
|
|
|
8
8
|
# @param durability_filter [String, nil] Filter by durability level
|
|
9
9
|
# @param dirty [Boolean, nil] Whether to include uncommitted data
|
|
10
10
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
11
|
+
# @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
|
|
12
|
+
# miss (default: 1; zero also means serial fetching).
|
|
11
13
|
# @return [String, nil] The value, or nil if not found
|
|
12
14
|
#
|
|
13
|
-
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
|
|
15
|
+
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
|
|
14
16
|
opts = {}
|
|
15
17
|
opts[:durability_filter] = durability_filter.to_s if durability_filter
|
|
16
18
|
opts[:dirty] = dirty unless dirty.nil?
|
|
17
19
|
opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
|
|
20
|
+
opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
|
|
18
21
|
|
|
19
22
|
if opts.empty?
|
|
20
23
|
_get(key)
|
|
@@ -27,16 +30,19 @@ module SlateDb
|
|
|
27
30
|
#
|
|
28
31
|
# @param start_key [String] The start key (inclusive)
|
|
29
32
|
# @param end_key [String, nil] The end key (exclusive)
|
|
33
|
+
# @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
|
|
30
34
|
# @return [Iterator] An iterator over key-value pairs
|
|
31
35
|
#
|
|
32
36
|
def scan(start_key, end_key = nil, durability_filter: nil, dirty: nil,
|
|
33
|
-
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, &)
|
|
34
|
-
opts = {
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
37
|
+
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil, &)
|
|
38
|
+
opts = {
|
|
39
|
+
durability_filter: durability_filter&.to_s,
|
|
40
|
+
dirty: dirty,
|
|
41
|
+
read_ahead_bytes: read_ahead_bytes,
|
|
42
|
+
cache_blocks: cache_blocks,
|
|
43
|
+
max_fetch_tasks: max_fetch_tasks,
|
|
44
|
+
order: order&.to_s
|
|
45
|
+
}.compact
|
|
40
46
|
|
|
41
47
|
iter = if opts.empty?
|
|
42
48
|
_scan(start_key, end_key)
|
|
@@ -59,6 +65,7 @@ module SlateDb
|
|
|
59
65
|
# @param read_ahead_bytes [Integer, nil] Number of bytes to read ahead
|
|
60
66
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
61
67
|
# @param max_fetch_tasks [Integer, nil] Maximum number of fetch tasks
|
|
68
|
+
# @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
|
|
62
69
|
# @param from [String, nil] Inclusive lower bound suffix, appended to the
|
|
63
70
|
# prefix, to start scanning from. Defaults to the start of the prefix.
|
|
64
71
|
# @param to [String, nil] Exclusive upper bound suffix, appended to the
|
|
@@ -66,7 +73,7 @@ module SlateDb
|
|
|
66
73
|
# @return [Iterator] An iterator over key-value pairs
|
|
67
74
|
#
|
|
68
75
|
def scan_prefix(prefix, durability_filter: nil, dirty: nil,
|
|
69
|
-
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil,
|
|
76
|
+
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil,
|
|
70
77
|
from: nil, to: nil, &)
|
|
71
78
|
opts = {
|
|
72
79
|
durability_filter: durability_filter&.to_s,
|
|
@@ -74,6 +81,7 @@ module SlateDb
|
|
|
74
81
|
read_ahead_bytes: read_ahead_bytes,
|
|
75
82
|
cache_blocks: cache_blocks,
|
|
76
83
|
max_fetch_tasks: max_fetch_tasks,
|
|
84
|
+
order: order&.to_s,
|
|
77
85
|
subrange_from: from,
|
|
78
86
|
subrange_to: to
|
|
79
87
|
}.compact
|
data/lib/slatedb/transaction.rb
CHANGED
|
@@ -8,13 +8,16 @@ module SlateDb
|
|
|
8
8
|
# @param durability_filter [String, nil] Filter by durability level
|
|
9
9
|
# @param dirty [Boolean, nil] Whether to include uncommitted data
|
|
10
10
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
11
|
+
# @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
|
|
12
|
+
# miss (default: 1; zero also means serial fetching).
|
|
11
13
|
# @return [String, nil] The value, or nil if not found
|
|
12
14
|
#
|
|
13
|
-
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
|
|
15
|
+
def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
|
|
14
16
|
opts = {}
|
|
15
17
|
opts[:durability_filter] = durability_filter.to_s if durability_filter
|
|
16
18
|
opts[:dirty] = dirty unless dirty.nil?
|
|
17
19
|
opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
|
|
20
|
+
opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
|
|
18
21
|
|
|
19
22
|
if opts.empty?
|
|
20
23
|
_get(key)
|
|
@@ -66,16 +69,19 @@ module SlateDb
|
|
|
66
69
|
#
|
|
67
70
|
# @param start_key [String] The start key (inclusive)
|
|
68
71
|
# @param end_key [String, nil] The end key (exclusive)
|
|
72
|
+
# @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
|
|
69
73
|
# @return [Iterator] An iterator over key-value pairs
|
|
70
74
|
#
|
|
71
75
|
def scan(start_key, end_key = nil, durability_filter: nil, dirty: nil,
|
|
72
|
-
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, &)
|
|
73
|
-
opts = {
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
76
|
+
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil, &)
|
|
77
|
+
opts = {
|
|
78
|
+
durability_filter: durability_filter&.to_s,
|
|
79
|
+
dirty: dirty,
|
|
80
|
+
read_ahead_bytes: read_ahead_bytes,
|
|
81
|
+
cache_blocks: cache_blocks,
|
|
82
|
+
max_fetch_tasks: max_fetch_tasks,
|
|
83
|
+
order: order&.to_s
|
|
84
|
+
}.compact
|
|
79
85
|
|
|
80
86
|
iter = if opts.empty?
|
|
81
87
|
_scan(start_key, end_key)
|
|
@@ -98,6 +104,7 @@ module SlateDb
|
|
|
98
104
|
# @param read_ahead_bytes [Integer, nil] Number of bytes to read ahead
|
|
99
105
|
# @param cache_blocks [Boolean, nil] Whether to cache blocks
|
|
100
106
|
# @param max_fetch_tasks [Integer, nil] Maximum number of fetch tasks
|
|
107
|
+
# @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
|
|
101
108
|
# @param from [String, nil] Inclusive lower bound suffix, appended to the
|
|
102
109
|
# prefix, to start scanning from. Defaults to the start of the prefix.
|
|
103
110
|
# @param to [String, nil] Exclusive upper bound suffix, appended to the
|
|
@@ -105,7 +112,7 @@ module SlateDb
|
|
|
105
112
|
# @return [Iterator] An iterator over key-value pairs
|
|
106
113
|
#
|
|
107
114
|
def scan_prefix(prefix, durability_filter: nil, dirty: nil,
|
|
108
|
-
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil,
|
|
115
|
+
read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil,
|
|
109
116
|
from: nil, to: nil, &)
|
|
110
117
|
opts = {
|
|
111
118
|
durability_filter: durability_filter&.to_s,
|
|
@@ -113,6 +120,7 @@ module SlateDb
|
|
|
113
120
|
read_ahead_bytes: read_ahead_bytes,
|
|
114
121
|
cache_blocks: cache_blocks,
|
|
115
122
|
max_fetch_tasks: max_fetch_tasks,
|
|
123
|
+
order: order&.to_s,
|
|
116
124
|
subrange_from: from,
|
|
117
125
|
subrange_to: to
|
|
118
126
|
}.compact
|
data/lib/slatedb/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: slatedb
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.7.0
|
|
5
5
|
platform: aarch64-linux
|
|
6
6
|
authors:
|
|
7
7
|
- SlateDB Contributors
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-11-11 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: rake
|