slatedb 0.5.0-x86_64-linux → 0.7.0-x86_64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d6dce4c0cc80ba8c3a20bd8eaa7ced30f93f5c39f1eb8e91e7d3059fc1331679
4
- data.tar.gz: 939115b5cfac3925f1cbc80e4d9fcf2e0bdcac8387c3b5b2f9a62f5ed4e843b7
3
+ metadata.gz: d5e77022db179bfe7b7d53db10212a9ca62645596d22e039b99eb6df637a58bf
4
+ data.tar.gz: bacdcb0fe2b02fc010e4ae159e36dd76242b09850c5bede4a98014b506fd07e5
5
5
  SHA512:
6
- metadata.gz: a2229a26ed0be19cf80c02ba3827cccd64f65768f877cb4b2710de212f4d1385d68443a39742334b981dc6a363a230dbf8cbadc6ef097d91b30de773fbe0a96a
7
- data.tar.gz: a1e79440a3ac1fe779e5eea40fd91cda72be3e10dc35023698648a9e55737c092e09cd9eb23476d64a07624acb475c4bc23022d199db646d1e5d145523ed2e71
6
+ metadata.gz: a702d764db4e94d6b5337dd03c866f09f9ffe4ec69556fe13737bc343a112840a807228ce071cb3f0ab28ba7f034100b8fe1d6cdad014981a64c1948d094fcf8
7
+ data.tar.gz: 46bffce34db7093e383caefffeba769df1598b15b4c7980881a3fa29e7f8eb23b3c9d243ab58ef0d7636f6e9a22080cdd72ce4eb29a3adbb82bf119ad8be42d0
data/README.md CHANGED
@@ -30,6 +30,38 @@ gem install slatedb
30
30
 
31
31
  Precompiled native gems are published for common Linux and Darwin platforms. Building from source still requires a Rust toolchain.
32
32
 
33
+ ## Upgrading to SlateDB 0.18.1
34
+
35
+ This update moves the Rust engine from 0.17.0 to 0.18.1. Existing database files
36
+ remain readable. New Ruby options keep the existing defaults unless enabled.
37
+
38
+ - Two-phase checkpoints: `begin_checkpoint` returns a handle with `id` and `wait`.
39
+ Both checkpoint methods accept `scope: :all` to capture pending writes, or
40
+ `scope: :durable` (the existing default) for writes already durable.
41
+ - Point reads accept `max_concurrent_sources:` on databases, readers, snapshots,
42
+ and transactions; database metadata reads accept it too.
43
+ - `Database.open` accepts `experimental_incremental_compaction:` and
44
+ `sst_block_alignment:`. Both are disabled by default; see the warnings below.
45
+ - Snapshot and transaction range/prefix scans now accept `order:`, like database
46
+ and reader scans. Reader `merge_operator:` is now applied rather than ignored.
47
+ - **Breaking:** Ruby Proc/lambda merge operators are no longer accepted. They
48
+ silently used concatenation on other threads, which could change stored values.
49
+ Keep `:concat` / `:string_concat` for concatenation. For custom updates, use a
50
+ serializable transaction that reads the value and writes the computed result;
51
+ handle `SlateDb::TransactionError` conflicts in the application. Databases with
52
+ unresolved custom merge operands need an application-specific migration before
53
+ upgrading. Do not reopen them with `:concat` unless concatenation was their
54
+ intended merge rule.
55
+ - Reader disk-cache keywords remain supported, but upstream deprecated the
56
+ underlying `CachedObjectStore` in 0.18.1. The new mirror and Rust trait-based
57
+ extension APIs are not exposed by this gem yet.
58
+
59
+ Upstream also fixes clone-compaction retention/data-loss cases, writer ownership
60
+ validation after WAL replay, write stalls, upload retry/concurrency behavior, and
61
+ iterator fetch cancellation. It improves memtable seeks and point-read/scan work.
62
+ See the full [0.18.0 release notes](https://github.com/slatedb/slatedb/releases/tag/v0.18.0)
63
+ and [0.18.1 release notes](https://github.com/slatedb/slatedb/releases/tag/v0.18.1).
64
+
33
65
  ## Usage
34
66
 
35
67
  ### Basic Operations
@@ -64,6 +96,73 @@ SlateDb::Database.open("/tmp/mydb") do |db|
64
96
  end # automatically closed
65
97
  ```
66
98
 
99
+ ### Point-Read Concurrency
100
+
101
+ ```ruby
102
+ db.get("key", max_concurrent_sources: 4, cache_blocks: false)
103
+ db.get_entry("key", max_concurrent_sources: 4) # value plus sequence/timestamps
104
+ ```
105
+
106
+ `max_concurrent_sources` defaults to 1 (serial fetching); zero also means 1.
107
+ SlateDB checks the newest source alone, then fetches up to this many sources
108
+ concurrently after a miss. Higher values can reduce latency but increase object
109
+ store requests and bandwidth. The same keyword works with `get` on readers,
110
+ snapshots, and transactions. It does not change isolation or durability.
111
+
112
+ ### Compaction Controls
113
+
114
+ SlateDB 0.17 adds controls for how long compaction inputs remain protected from
115
+ garbage collection and when to consolidate sorted runs:
116
+
117
+ ```ruby
118
+ SlateDb::Database.open("mydb",
119
+ url: "s3://mybucket/path",
120
+ compactor_checkpoint_lifetime: 30 * 60_000,
121
+ sorted_run_consolidation_threshold: 8) do |db|
122
+ db.put("key", "value")
123
+ end
124
+ ```
125
+
126
+ `compactor_checkpoint_lifetime` is in milliseconds and defaults to 15 minutes.
127
+ Reads, scans, snapshots, and transactions using old compaction inputs must finish
128
+ within that time; after it expires, garbage collection can remove those inputs.
129
+
130
+ `sorted_run_consolidation_threshold` starts fallback consolidation when the
131
+ projected sorted-run count exceeds the threshold. It is not a hard limit.
132
+ Zero disables it (the default); use at least two to enable it. SlateDB warns and
133
+ disables the trigger for one. Low thresholds can increase storage reads and writes.
134
+
135
+ SlateDB 0.18 adds an opt-in experimental mode:
136
+
137
+ ```ruby
138
+ SlateDb::Database.open("mydb", url: "s3://mybucket/path",
139
+ experimental_incremental_compaction: true) do |db|
140
+ db.put("key", "value")
141
+ end
142
+ ```
143
+
144
+ This publishes output from sorted-run compactions at most once a minute before
145
+ the full compaction finishes. It can reduce read amplification and allow earlier
146
+ storage reclamation after the compactor checkpoint expires. Compactions with L0
147
+ inputs still publish only when complete. It is disabled by default, experimental,
148
+ and may change or disappear upstream.
149
+
150
+ ### SST Block Alignment
151
+
152
+ `Database.open(..., sst_block_alignment: true)` pads SST data blocks to 4 KiB
153
+ boundaries, including oversized entries. It is disabled by default, uses more
154
+ storage, and cannot be combined with compression.
155
+
156
+ **Upgrade every reader before enabling alignment on a writer.** Older engines
157
+ that do not understand padded SSTs cannot read them. Once padded files exist,
158
+ rolling those readers back is not safe; disabling the option does not rewrite
159
+ existing files.
160
+
161
+ ### Key Size
162
+
163
+ Keys can now exceed the previous 65,535-byte limit, up to SlateDB's new
164
+ 4,294,967,295-byte limit. Very large keys still require enough memory to process.
165
+
67
166
  ### Persistent Storage
68
167
 
69
168
  For persistent storage, provide an object store URL:
@@ -313,53 +412,19 @@ db.batch do |b|
313
412
  end
314
413
  ```
315
414
 
316
- #### Custom Merge Operators
317
-
318
- You can provide a Ruby Proc/lambda as a custom merge operator:
319
-
320
- ```ruby
321
- # Counter merge operator (adds numbers)
322
- counter_merge = ->(key, existing, new_value) {
323
- existing_num = existing ? existing.to_i : 0
324
- (existing_num + new_value.to_i).to_s
325
- }
326
-
327
- SlateDb::Database.open("/tmp/mydb", merge_operator: counter_merge) do |db|
328
- db.merge("visits", "1")
329
- db.merge("visits", "1")
330
- db.merge("visits", "1")
331
-
332
- db.get("visits") # => "3"
333
- end
415
+ #### Available Merge Operators
334
416
 
335
- # Max value merge operator
336
- max_merge = ->(key, existing, new_value) {
337
- existing_num = existing ? existing.to_i : 0
338
- new_num = new_value.to_i
339
- [existing_num, new_num].max.to_s
340
- }
417
+ - `:string_concat` (or `:concat`) - Concatenates byte values (built-in)
341
418
 
342
- SlateDb::Database.open("/tmp/mydb", merge_operator: max_merge) do |db|
343
- db.merge("high_score", "100")
344
- db.merge("high_score", "250")
345
- db.merge("high_score", "150")
419
+ Ruby Proc/lambda operators are rejected. Use a serializable read/modify/write
420
+ transaction for custom logic, with application-level conflict handling:
346
421
 
347
- db.get("high_score") # => "250"
422
+ ```ruby
423
+ db.transaction(isolation: :serializable) do |txn|
424
+ txn.put("visits", (txn.get("visits").to_i + 1).to_s)
348
425
  end
349
426
  ```
350
427
 
351
- The proc receives three arguments:
352
- - `key` - The key being merged
353
- - `existing` - The existing value (nil if no value exists)
354
- - `new_value` - The new merge operand
355
-
356
- **Note:** Custom Proc merge operators work best with direct `db.merge()` calls. When used with transactions or batches, some merge operations may be processed on background threads and fall back to string concatenation.
357
-
358
- #### Available Merge Operators
359
-
360
- - `:string_concat` (or `:concat`) - Concatenates byte values (built-in)
361
- - Any `Proc` or `lambda` - Custom merge logic
362
-
363
428
  ### Write Batches
364
429
 
365
430
  Perform multiple writes atomically:
@@ -467,6 +532,29 @@ SlateDb::Database.open("/tmp/mydb", url: "file:///tmp/mydb") do |db|
467
532
  end
468
533
  ```
469
534
 
535
+ `create_checkpoint` waits until the checkpoint is stored. Its default
536
+ `scope: :durable` preserves the existing behavior. To include writes made with
537
+ `await_durable: false`, use `scope: :all`; no separate `flush` is needed.
538
+
539
+ To separate capturing the write boundary from waiting for storage:
540
+
541
+ ```ruby
542
+ db.put("key", "before", await_durable: false)
543
+ handle = db.begin_checkpoint(scope: :all, name: "backup", lifetime: 3_600_000)
544
+ id = handle.id # UUID is available immediately
545
+ db.put("key", "after") # not part of this checkpoint
546
+ checkpoint = handle.wait # => { id: id, manifest_id: ... }
547
+ ```
548
+
549
+ With `scope: :all`, writes completed before `begin_checkpoint` starts are included;
550
+ writes started after it returns are excluded. Concurrent writes during the call
551
+ may be included. Starting the checkpoint can still wait for internal flush work;
552
+ it is not a fully nonblocking Ruby call. `wait` releases the Ruby GVL while
553
+ waiting, consumes the handle, and raises `SlateDb::ClosedError` if called again.
554
+ `id` remains readable. Dropping the handle does not cancel or delete the checkpoint.
555
+ If waiting fails, use the UUID with `Admin#list_checkpoints` to check its outcome
556
+ before starting another checkpoint.
557
+
470
558
  ### Snapshots
471
559
 
472
560
  Point-in-time consistent reads:
@@ -489,6 +577,11 @@ value = snap.get("key")
489
577
  snap.close
490
578
  ```
491
579
 
580
+ Range and prefix scans on snapshots and transactions accept the same `order:`
581
+ keyword as databases and readers, including `:asc`/`:ascending` and
582
+ `:desc`/`:descending`. Bounds remain lower-inclusive and upper-exclusive in
583
+ either direction.
584
+
492
585
  ### Reader (Read-Only Access)
493
586
 
494
587
  Open a database in read-only mode, useful for replicas:
@@ -501,6 +594,14 @@ SlateDb::Reader.open("/tmp/mydb", url: "s3://bucket/path") do |reader|
501
594
  reader.scan("prefix:").each do |k, v|
502
595
  puts "#{k}: #{v}"
503
596
  end
597
+
598
+ # Range and prefix scans accept :asc/:ascending or :desc/:descending.
599
+ reader.scan("user:100", "user:200", order: :desc).each do |k, v|
600
+ puts "#{k}: #{v}"
601
+ end
602
+ reader.scan_prefix("user:", from: "100", to: "200", order: :desc).each do |k, v|
603
+ puts "#{k}: #{v}"
604
+ end
504
605
  end
505
606
 
506
607
  # Open at a specific checkpoint (pins the reader to that checkpoint's state)
@@ -532,6 +633,14 @@ SlateDb::Reader.open("/tmp/mydb",
532
633
  end
533
634
  ```
534
635
 
636
+ When the writer uses `merge_operator: :concat`, pass the same operator to
637
+ `Reader.open` so WAL replay and reads apply the correct merge rule. Unknown
638
+ operator names now raise `SlateDb::InvalidArgumentError` instead of being ignored.
639
+
640
+ The `cache_root` and `max_open_file_handles` keywords still work, but SlateDB
641
+ 0.18.1 deprecates this legacy disk cache in favor of its separate mirror crate or
642
+ tiered block cache. Those replacement APIs are not yet exposed here.
643
+
535
644
  ### Admin Operations
536
645
 
537
646
  Administrative operations for database management:
@@ -558,9 +667,6 @@ admin.delete_checkpoint("uuid")
558
667
  # Garbage Collection
559
668
  admin.run_gc # Run with default settings
560
669
  admin.run_gc(min_age: 3600_000) # Set min age for all directories (1 hour)
561
- admin.run_gc(manifest_min_age: 86400_000) # Custom age for manifest (1 day)
562
- admin.run_gc(wal_min_age: 60_000) # Custom age for WAL (1 minute)
563
- admin.run_gc(compacted_min_age: 60_000) # Custom age for compacted (1 minute)
564
670
 
565
671
  # Delete a database (SlateDB >= 0.16.0)
566
672
  would_delete = admin.delete_db # Dry run: returns the object paths
Binary file
Binary file
Binary file
@@ -9,9 +9,18 @@ module SlateDb
9
9
  #
10
10
  # @param path [String] The path identifier for the database
11
11
  # @param url [String, nil] Optional object store URL (e.g., "s3://bucket/path")
12
- # @param merge_operator [Symbol, String, Proc, nil] Optional merge operator.
13
- # Can be a symbol/string ("string_concat" or "concat") or a Proc/lambda
14
- # that takes (key, existing_value, new_value) and returns the merged value.
12
+ # @param merge_operator [Symbol, String, nil] Optional built-in merge operator
13
+ # (:string_concat or :concat). Ruby callbacks are not supported.
14
+ # @param compactor_checkpoint_lifetime [Integer, nil] How long compaction inputs
15
+ # are protected from garbage collection, in milliseconds (default: 15 minutes).
16
+ # Reads using those inputs must finish within this time.
17
+ # @param sorted_run_consolidation_threshold [Integer, nil] Sorted-run count above
18
+ # which fallback consolidation starts. Zero disables it (default); use at least
19
+ # two to enable it. One is disabled by SlateDB with a warning. This is not a hard limit.
20
+ # @param experimental_incremental_compaction [Boolean, nil] Publish sorted-run
21
+ # compaction output before completion (experimental, disabled by default).
22
+ # @param sst_block_alignment [Boolean, nil] Align SST data blocks to 4 KiB
23
+ # boundaries (disabled by default; incompatible with compression).
15
24
  # @yield [db] If a block is given, yields the database and ensures it's closed
16
25
  # @return [Database] The opened database (or block result if block given)
17
26
  #
@@ -28,28 +37,22 @@ module SlateDb
28
37
  # @example Open with S3 backend
29
38
  # db = SlateDb::Database.open("/tmp/mydb", url: "s3://mybucket/path")
30
39
  #
31
- # @example Open with a custom merge operator (Proc)
32
- # # Custom merge that adds numbers
33
- # db = SlateDb::Database.open("/tmp/mydb", merge_operator: ->(key, existing, new_val) {
34
- # existing_num = existing ? existing.to_i : 0
35
- # (existing_num + new_val.to_i).to_s
36
- # })
37
- # db.merge("counter", "5")
38
- # db.merge("counter", "3")
39
- # db.get("counter") # => "8"
40
- #
41
- def open(path, url: nil, merge_operator: nil)
42
- opts = {}
43
-
44
- case merge_operator
45
- when Symbol, String
46
- opts[:merge_operator] = merge_operator.to_s
47
- when Proc
48
- # Store the proc to prevent GC and pass to Rust
49
- @_merge_operator_proc = merge_operator
50
- opts[:merge_operator_proc] = merge_operator
40
+ def open(path, url: nil, merge_operator: nil,
41
+ compactor_checkpoint_lifetime: nil, sorted_run_consolidation_threshold: nil,
42
+ experimental_incremental_compaction: nil, sst_block_alignment: nil)
43
+ opts = {
44
+ compactor_checkpoint_lifetime: compactor_checkpoint_lifetime,
45
+ sorted_run_consolidation_threshold: sorted_run_consolidation_threshold,
46
+ experimental_incremental_compaction: experimental_incremental_compaction,
47
+ sst_block_alignment: sst_block_alignment
48
+ }.compact
49
+
50
+ unless merge_operator.nil? || merge_operator.is_a?(Symbol) || merge_operator.is_a?(String)
51
+ raise ArgumentError, "merge_operator must be :string_concat or :concat; Ruby callbacks are not supported"
51
52
  end
52
53
 
54
+ opts[:merge_operator] = merge_operator.to_s if merge_operator
55
+
53
56
  db = _open(path, url, opts)
54
57
 
55
58
  if block_given?
@@ -74,6 +77,8 @@ module SlateDb
74
77
  # @param durability_filter [String, nil] Filter by durability level ("remote" or "memory")
75
78
  # @param dirty [Boolean, nil] Whether to include uncommitted data
76
79
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
80
+ # @param max_concurrent_sources [Integer, nil] Concurrent sources to fetch after
81
+ # the first source misses (default: 1; zero also means serial fetching).
77
82
  # @return [String, nil] The value, or nil if not found
78
83
  #
79
84
  # @example Basic get
@@ -82,11 +87,12 @@ module SlateDb
82
87
  # @example Get with options
83
88
  # value = db.get("mykey", durability_filter: "memory", dirty: true)
84
89
  #
85
- def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
90
+ def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
86
91
  opts = {}
87
92
  opts[:durability_filter] = durability_filter.to_s if durability_filter
88
93
  opts[:dirty] = dirty unless dirty.nil?
89
94
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
95
+ opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
90
96
 
91
97
  if opts.empty?
92
98
  _get(key)
@@ -102,17 +108,20 @@ module SlateDb
102
108
  # @param dirty [Boolean, nil] Whether to include uncommitted data
103
109
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
104
110
  # @return [Hash, nil] A hash with :key, :value, :seq, :create_ts, and :expire_ts, or nil if not found
111
+ # @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
112
+ # miss. See {#get}.
105
113
  #
106
114
  # @example Inspect metadata
107
115
  # entry = db.get_key_value("mykey")
108
116
  # entry[:value] # => "myvalue"
109
117
  # entry[:seq] # => sequence number
110
118
  #
111
- def get_key_value(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
119
+ def get_key_value(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
112
120
  opts = {}
113
121
  opts[:durability_filter] = durability_filter.to_s if durability_filter
114
122
  opts[:dirty] = dirty unless dirty.nil?
115
123
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
124
+ opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
116
125
 
117
126
  if opts.empty?
118
127
  _get_key_value(key)
@@ -484,6 +493,8 @@ module SlateDb
484
493
  #
485
494
  # @param lifetime [Integer, nil] Checkpoint lifetime in milliseconds
486
495
  # @param name [String, nil] Optional name for the checkpoint
496
+ # @param scope [Symbol, String] :durable (default) captures durable writes;
497
+ # :all also flushes pending writes before storing the checkpoint.
487
498
  # @return [Hash] Hash with :id (UUID string) and :manifest_id (integer)
488
499
  #
489
500
  # @example Create a named checkpoint
@@ -493,13 +504,26 @@ module SlateDb
493
504
  # @example Create a checkpoint with lifetime
494
505
  # checkpoint = db.create_checkpoint(lifetime: 3600_000) # 1 hour
495
506
  #
496
- def create_checkpoint(lifetime: nil, name: nil)
497
- opts = {}
507
+ def create_checkpoint(lifetime: nil, name: nil, scope: :durable)
508
+ opts = { scope: scope.to_s }
498
509
  opts[:lifetime] = lifetime if lifetime
499
510
  opts[:name] = name if name
500
511
  _create_checkpoint(opts)
501
512
  end
502
513
 
514
+ # Start a checkpoint without waiting for it to be stored.
515
+ #
516
+ # With scope: :all, captures writes completed before this call and excludes
517
+ # writes started after it returns. Writes during the call may be included.
518
+ # Options match {#create_checkpoint}. Dropping the handle does not cancel it.
519
+ # @return [CheckpointHandle] #id is available immediately; #wait returns the
520
+ # :id/:manifest_id hash once stored and consumes the handle. A second wait
521
+ # raises ClosedError. #id remains available after waiting.
522
+ def begin_checkpoint(lifetime: nil, name: nil, scope: :durable)
523
+ opts = { scope: scope.to_s, lifetime: lifetime, name: name }.compact
524
+ _begin_checkpoint(opts)
525
+ end
526
+
503
527
  # Get database metrics registry.
504
528
  #
505
529
  # @return [Metrics] Metrics registry
@@ -21,6 +21,7 @@ module SlateDb
21
21
  # @param cache_root [String, nil] Root folder for the reader's local on-disk
22
22
  # object-store cache. Setting this enables the cached object store; when it is
23
23
  # not set the cache (and `max_open_file_handles`) has no effect.
24
+ # This legacy cache is deprecated upstream in SlateDB 0.18.1.
24
25
  # @param max_open_file_handles [Integer, nil] Maximum number of file handles to keep
25
26
  # open in the reader's file-handle cache. When the limit is reached, the least
26
27
  # recently used handle is closed (default: 1000). Only takes effect when
@@ -54,29 +55,31 @@ module SlateDb
54
55
  manifest_poll_interval: nil, checkpoint_lifetime: nil,
55
56
  max_memtable_bytes: nil, cache_root: nil, max_open_file_handles: nil,
56
57
  merge_operator: nil)
57
- opts = {}
58
- opts[:follow_latest] = follow_latest unless follow_latest.nil?
59
- opts[:manifest_poll_interval] = manifest_poll_interval if manifest_poll_interval
60
- opts[:checkpoint_lifetime] = checkpoint_lifetime if checkpoint_lifetime
61
- opts[:max_memtable_bytes] = max_memtable_bytes if max_memtable_bytes
62
- opts[:cache_root] = cache_root if cache_root
63
- opts[:max_open_file_handles] = max_open_file_handles if max_open_file_handles
64
- opts[:merge_operator] = merge_operator.to_s if merge_operator
58
+ unless merge_operator.nil? || merge_operator.is_a?(Symbol) || merge_operator.is_a?(String)
59
+ raise ArgumentError, "merge_operator must be :string_concat or :concat; Ruby callbacks are not supported"
60
+ end
61
+
62
+ opts = {
63
+ follow_latest: follow_latest,
64
+ manifest_poll_interval: manifest_poll_interval,
65
+ checkpoint_lifetime: checkpoint_lifetime,
66
+ max_memtable_bytes: max_memtable_bytes,
67
+ cache_root: cache_root,
68
+ max_open_file_handles: max_open_file_handles,
69
+ merge_operator: merge_operator&.to_s
70
+ }.compact
65
71
 
66
72
  reader = _open(path, url, checkpoint_id, opts)
73
+ return reader unless block_given?
67
74
 
68
- if block_given?
75
+ begin
76
+ yield reader
77
+ ensure
69
78
  begin
70
- yield reader
71
- ensure
72
- begin
73
- reader.close
74
- rescue StandardError
75
- nil
76
- end
79
+ reader.close
80
+ rescue StandardError
81
+ nil
77
82
  end
78
- else
79
- reader
80
83
  end
81
84
  end
82
85
  end
@@ -87,13 +90,16 @@ module SlateDb
87
90
  # @param durability_filter [String, nil] Filter by durability level ("remote" or "memory")
88
91
  # @param dirty [Boolean, nil] Whether to include uncommitted data
89
92
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
93
+ # @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
94
+ # miss (default: 1; zero also means serial fetching).
90
95
  # @return [String, nil] The value, or nil if not found
91
96
  #
92
- def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
97
+ def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
93
98
  opts = {}
94
99
  opts[:durability_filter] = durability_filter.to_s if durability_filter
95
100
  opts[:dirty] = dirty unless dirty.nil?
96
101
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
102
+ opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
97
103
 
98
104
  if opts.empty?
99
105
  _get(key)
@@ -106,16 +112,19 @@ module SlateDb
106
112
  #
107
113
  # @param start_key [String] The start key (inclusive)
108
114
  # @param end_key [String, nil] The end key (exclusive)
115
+ # @param order [Symbol, String, nil] Iteration order (:asc/:ascending or :desc/:descending)
109
116
  # @return [Iterator] An iterator over key-value pairs
110
117
  #
111
118
  def scan(start_key, end_key = nil, durability_filter: nil, dirty: nil,
112
- read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, &)
113
- opts = {}
114
- opts[:durability_filter] = durability_filter.to_s if durability_filter
115
- opts[:dirty] = dirty unless dirty.nil?
116
- opts[:read_ahead_bytes] = read_ahead_bytes if read_ahead_bytes
117
- opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
118
- opts[:max_fetch_tasks] = max_fetch_tasks if max_fetch_tasks
119
+ read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil, &)
120
+ opts = {
121
+ durability_filter: durability_filter&.to_s,
122
+ dirty: dirty,
123
+ read_ahead_bytes: read_ahead_bytes,
124
+ cache_blocks: cache_blocks,
125
+ max_fetch_tasks: max_fetch_tasks,
126
+ order: order&.to_s
127
+ }.compact
119
128
 
120
129
  iter = if opts.empty?
121
130
  _scan(start_key, end_key)
@@ -138,6 +147,7 @@ module SlateDb
138
147
  # @param read_ahead_bytes [Integer, nil] Number of bytes to read ahead
139
148
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
140
149
  # @param max_fetch_tasks [Integer, nil] Maximum number of fetch tasks
150
+ # @param order [Symbol, String, nil] Iteration order (:asc/:ascending or :desc/:descending)
141
151
  # @param from [String, nil] Inclusive lower bound suffix, appended to the
142
152
  # prefix, to start scanning from (e.g. prefix "user:" with from "100"
143
153
  # starts at "user:100"). Defaults to the start of the prefix.
@@ -146,7 +156,7 @@ module SlateDb
146
156
  # @return [Iterator] An iterator over key-value pairs
147
157
  #
148
158
  def scan_prefix(prefix, durability_filter: nil, dirty: nil,
149
- read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil,
159
+ read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil,
150
160
  from: nil, to: nil, &)
151
161
  opts = {
152
162
  durability_filter: durability_filter&.to_s,
@@ -154,6 +164,7 @@ module SlateDb
154
164
  read_ahead_bytes: read_ahead_bytes,
155
165
  cache_blocks: cache_blocks,
156
166
  max_fetch_tasks: max_fetch_tasks,
167
+ order: order&.to_s,
157
168
  subrange_from: from,
158
169
  subrange_to: to
159
170
  }.compact
@@ -8,13 +8,16 @@ module SlateDb
8
8
  # @param durability_filter [String, nil] Filter by durability level
9
9
  # @param dirty [Boolean, nil] Whether to include uncommitted data
10
10
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
11
+ # @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
12
+ # miss (default: 1; zero also means serial fetching).
11
13
  # @return [String, nil] The value, or nil if not found
12
14
  #
13
- def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
15
+ def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
14
16
  opts = {}
15
17
  opts[:durability_filter] = durability_filter.to_s if durability_filter
16
18
  opts[:dirty] = dirty unless dirty.nil?
17
19
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
20
+ opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
18
21
 
19
22
  if opts.empty?
20
23
  _get(key)
@@ -27,16 +30,19 @@ module SlateDb
27
30
  #
28
31
  # @param start_key [String] The start key (inclusive)
29
32
  # @param end_key [String, nil] The end key (exclusive)
33
+ # @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
30
34
  # @return [Iterator] An iterator over key-value pairs
31
35
  #
32
36
  def scan(start_key, end_key = nil, durability_filter: nil, dirty: nil,
33
- read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, &)
34
- opts = {}
35
- opts[:durability_filter] = durability_filter.to_s if durability_filter
36
- opts[:dirty] = dirty unless dirty.nil?
37
- opts[:read_ahead_bytes] = read_ahead_bytes if read_ahead_bytes
38
- opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
39
- opts[:max_fetch_tasks] = max_fetch_tasks if max_fetch_tasks
37
+ read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil, &)
38
+ opts = {
39
+ durability_filter: durability_filter&.to_s,
40
+ dirty: dirty,
41
+ read_ahead_bytes: read_ahead_bytes,
42
+ cache_blocks: cache_blocks,
43
+ max_fetch_tasks: max_fetch_tasks,
44
+ order: order&.to_s
45
+ }.compact
40
46
 
41
47
  iter = if opts.empty?
42
48
  _scan(start_key, end_key)
@@ -59,6 +65,7 @@ module SlateDb
59
65
  # @param read_ahead_bytes [Integer, nil] Number of bytes to read ahead
60
66
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
61
67
  # @param max_fetch_tasks [Integer, nil] Maximum number of fetch tasks
68
+ # @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
62
69
  # @param from [String, nil] Inclusive lower bound suffix, appended to the
63
70
  # prefix, to start scanning from. Defaults to the start of the prefix.
64
71
  # @param to [String, nil] Exclusive upper bound suffix, appended to the
@@ -66,7 +73,7 @@ module SlateDb
66
73
  # @return [Iterator] An iterator over key-value pairs
67
74
  #
68
75
  def scan_prefix(prefix, durability_filter: nil, dirty: nil,
69
- read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil,
76
+ read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil,
70
77
  from: nil, to: nil, &)
71
78
  opts = {
72
79
  durability_filter: durability_filter&.to_s,
@@ -74,6 +81,7 @@ module SlateDb
74
81
  read_ahead_bytes: read_ahead_bytes,
75
82
  cache_blocks: cache_blocks,
76
83
  max_fetch_tasks: max_fetch_tasks,
84
+ order: order&.to_s,
77
85
  subrange_from: from,
78
86
  subrange_to: to
79
87
  }.compact
@@ -8,13 +8,16 @@ module SlateDb
8
8
  # @param durability_filter [String, nil] Filter by durability level
9
9
  # @param dirty [Boolean, nil] Whether to include uncommitted data
10
10
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
11
+ # @param max_concurrent_sources [Integer, nil] Concurrent sources after the first
12
+ # miss (default: 1; zero also means serial fetching).
11
13
  # @return [String, nil] The value, or nil if not found
12
14
  #
13
- def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil)
15
+ def get(key, durability_filter: nil, dirty: nil, cache_blocks: nil, max_concurrent_sources: nil)
14
16
  opts = {}
15
17
  opts[:durability_filter] = durability_filter.to_s if durability_filter
16
18
  opts[:dirty] = dirty unless dirty.nil?
17
19
  opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
20
+ opts[:max_concurrent_sources] = max_concurrent_sources unless max_concurrent_sources.nil?
18
21
 
19
22
  if opts.empty?
20
23
  _get(key)
@@ -66,16 +69,19 @@ module SlateDb
66
69
  #
67
70
  # @param start_key [String] The start key (inclusive)
68
71
  # @param end_key [String, nil] The end key (exclusive)
72
+ # @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
69
73
  # @return [Iterator] An iterator over key-value pairs
70
74
  #
71
75
  def scan(start_key, end_key = nil, durability_filter: nil, dirty: nil,
72
- read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, &)
73
- opts = {}
74
- opts[:durability_filter] = durability_filter.to_s if durability_filter
75
- opts[:dirty] = dirty unless dirty.nil?
76
- opts[:read_ahead_bytes] = read_ahead_bytes if read_ahead_bytes
77
- opts[:cache_blocks] = cache_blocks unless cache_blocks.nil?
78
- opts[:max_fetch_tasks] = max_fetch_tasks if max_fetch_tasks
76
+ read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil, &)
77
+ opts = {
78
+ durability_filter: durability_filter&.to_s,
79
+ dirty: dirty,
80
+ read_ahead_bytes: read_ahead_bytes,
81
+ cache_blocks: cache_blocks,
82
+ max_fetch_tasks: max_fetch_tasks,
83
+ order: order&.to_s
84
+ }.compact
79
85
 
80
86
  iter = if opts.empty?
81
87
  _scan(start_key, end_key)
@@ -98,6 +104,7 @@ module SlateDb
98
104
  # @param read_ahead_bytes [Integer, nil] Number of bytes to read ahead
99
105
  # @param cache_blocks [Boolean, nil] Whether to cache blocks
100
106
  # @param max_fetch_tasks [Integer, nil] Maximum number of fetch tasks
107
+ # @param order [Symbol, String, nil] :asc/:ascending or :desc/:descending
101
108
  # @param from [String, nil] Inclusive lower bound suffix, appended to the
102
109
  # prefix, to start scanning from. Defaults to the start of the prefix.
103
110
  # @param to [String, nil] Exclusive upper bound suffix, appended to the
@@ -105,7 +112,7 @@ module SlateDb
105
112
  # @return [Iterator] An iterator over key-value pairs
106
113
  #
107
114
  def scan_prefix(prefix, durability_filter: nil, dirty: nil,
108
- read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil,
115
+ read_ahead_bytes: nil, cache_blocks: nil, max_fetch_tasks: nil, order: nil,
109
116
  from: nil, to: nil, &)
110
117
  opts = {
111
118
  durability_filter: durability_filter&.to_s,
@@ -113,6 +120,7 @@ module SlateDb
113
120
  read_ahead_bytes: read_ahead_bytes,
114
121
  cache_blocks: cache_blocks,
115
122
  max_fetch_tasks: max_fetch_tasks,
123
+ order: order&.to_s,
116
124
  subrange_from: from,
117
125
  subrange_to: to
118
126
  }.compact
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module SlateDb
4
- VERSION = "0.5.0"
4
+ VERSION = "0.7.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: slatedb
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.7.0
5
5
  platform: x86_64-linux
6
6
  authors:
7
7
  - SlateDB Contributors
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-10-26 00:00:00.000000000 Z
11
+ date: 2026-11-11 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -58,14 +58,14 @@ dependencies:
58
58
  requirements:
59
59
  - - "~>"
60
60
  - !ruby/object:Gem::Version
61
- version: '1.86'
61
+ version: '1.92'
62
62
  type: :development
63
63
  prerelease: false
64
64
  version_requirements: !ruby/object:Gem::Requirement
65
65
  requirements:
66
66
  - - "~>"
67
67
  - !ruby/object:Gem::Version
68
- version: '1.86'
68
+ version: '1.92'
69
69
  description: A cloud-native embedded key-value store built on object storage
70
70
  email:
71
71
  - slatedb@example.com