familia 2.9.0 → 2.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.rst +44 -0
- data/Gemfile.lock +1 -1
- data/docs/guides/datatype-collections.md +159 -0
- data/docs/migrating/v2.9.0.md +3 -3
- data/lib/familia/data_type/collection_base.rb +14 -17
- data/lib/familia/data_type/types/json_stringkey.rb +1 -1
- data/lib/familia/data_type/types/listkey.rb +7 -5
- data/lib/familia/data_type/types/sorted_set.rb +45 -0
- data/lib/familia/data_type/types/stringkey.rb +1 -1
- data/lib/familia/data_type/types/unsorted_set.rb +2 -1
- data/lib/familia/features/encrypted_fields/encrypted_field_type.rb +2 -20
- data/lib/familia/features/expiration.rb +2 -2
- data/lib/familia/field_type.rb +1 -19
- data/lib/familia/horreum/definition.rb +1 -19
- data/lib/familia/horreum/management.rb +1 -1
- data/lib/familia/utils.rb +48 -0
- data/lib/familia/version.rb +1 -1
- data/try/edge_cases/fast_writer_pipeline_support_try.rb +80 -0
- data/try/edge_cases/fast_writer_transaction_guard_try.rb +40 -59
- data/try/unit/data_types/each_record_try.rb +90 -13
- data/try/unit/data_types/sorted_set_try.rb +44 -0
- data/try/unit/horreum/destroy_related_fields_cleanup_try.rb +3 -3
- data/try/unit/utils/future_aware_helpers_try.rb +128 -0
- metadata +4 -2
- data/docs/archive/.gitignore +0 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d24c3b38092f1192f8e250553e8ef4d51b05c1bb00c48dfd7f6520d3a48a9a0e
|
|
4
|
+
data.tar.gz: 1dd0a2aa47682736209f116adf378eb4a0eccafffd7c151b02a8ef4573e52551
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 10d69edb30370c7dba83b387f53429878fded6bf736e04fbefec4c228baf375806eeefa6c1d44c45c296e5d79b82345d1bc9cfbb0992ee18fc4c204dac250e10
|
|
7
|
+
data.tar.gz: 9b52e601ae93d44a6db8b0f995828ae45d55872377a471658a8d63210607c56f70958e372654ac20a13b07344e9b9da0afcdaaa50203f0796326d5d08e93bf6b
|
data/CHANGELOG.rst
CHANGED
|
@@ -7,6 +7,50 @@ The format is based on `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`
|
|
|
7
7
|
|
|
8
8
|
<!--scriv-insert-here-->
|
|
9
9
|
|
|
10
|
+
.. _changelog-2.9.1:
|
|
11
|
+
|
|
12
|
+
2.9.1 — 2026-05-18
|
|
13
|
+
==================
|
|
14
|
+
|
|
15
|
+
Added
|
|
16
|
+
-----
|
|
17
|
+
|
|
18
|
+
- ``SortedSet#update`` (aliased ``merge!``) for bulk member insertion. A sorted
|
|
19
|
+
set is ``member => score`` -- the same pair shape as ``HashKey``'s
|
|
20
|
+
``field => value`` -- so it follows the established ``HashKey#update``/``merge!``
|
|
21
|
+
convention (a single Hash argument) rather than the variadic splat used by the
|
|
22
|
+
value-only ``UnsortedSet``/``ListKey``. Pass ``{member => score}`` to issue one
|
|
23
|
+
``ZADD`` instead of one round-trip per member. Validates the argument is a Hash
|
|
24
|
+
and that every score is ``Numeric`` (a missing/``nil`` score raises a clear
|
|
25
|
+
``ArgumentError`` instead of a low-level client error -- unlike single-value
|
|
26
|
+
``#add``, the bulk path does not default a missing score to ``Familia.now``).
|
|
27
|
+
Cascades expiration, and is a no-op returning ``0`` for empty input. The
|
|
28
|
+
single-value ``SortedSet#add`` (and its array-as-single-member contract) is
|
|
29
|
+
unchanged. PR #269
|
|
30
|
+
|
|
31
|
+
Changed
|
|
32
|
+
-------
|
|
33
|
+
|
|
34
|
+
- Bulk-write optimization for multi-value collection mutations. ``UnsortedSet#add``,
|
|
35
|
+
``ListKey#push``, and ``ListKey#unshift`` previously issued one Redis command per
|
|
36
|
+
element (a loop of ``SADD``/``RPUSH``/``LPUSH`` calls), making large populations
|
|
37
|
+
slow even when pipelined. They now serialize all values and issue a single bulk
|
|
38
|
+
``SADD``/``RPUSH``/``LPUSH`` command. Element ordering, ``nil`` compaction, nested
|
|
39
|
+
array flattening, return values, dirty-write warnings, and expiration cascading
|
|
40
|
+
are unchanged; empty calls remain no-ops. PR #269
|
|
41
|
+
|
|
42
|
+
AI Assistance
|
|
43
|
+
-------------
|
|
44
|
+
|
|
45
|
+
- AI investigated all collection ``DataType`` classes for the same per-element
|
|
46
|
+
loop anti-pattern, identified the three affected methods, verified
|
|
47
|
+
behavior-preservation (ordering, edge cases, chainability) at the Redis wire
|
|
48
|
+
level, and confirmed zero regressions against the existing test suites. The
|
|
49
|
+
``SortedSet#update`` API shape was chosen by priority order: existing Familia
|
|
50
|
+
conventions first (the ``HashKey#update``/``merge!`` precedent for keyed
|
|
51
|
+
collections), then the upstream redis-rb bulk ``ZADD`` form, then Ruby
|
|
52
|
+
``Hash#merge!`` semantics as confirmation.
|
|
53
|
+
|
|
10
54
|
.. _changelog-2.9.0:
|
|
11
55
|
|
|
12
56
|
2.9.0 — 2026-05-17
|
data/Gemfile.lock
CHANGED
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# docs/guides/datatype-collections.md
|
|
2
|
+
---
|
|
3
|
+
|
|
4
|
+
# DataType - Collection classes
|
|
5
|
+
|
|
6
|
+
UnsortedSet, Sorted Set, List, and Hash data types all include the `Collection` module. This guide covers two performance-sensitive concerns: writing many elements efficiently (a single bulk command instead of one round-trip per element), and iterating large collections efficiently via `each` and `each_record`.
|
|
7
|
+
|
|
8
|
+
## Bulk writes — single round-trip mutations
|
|
9
|
+
|
|
10
|
+
Collection mutations are **immediate** — every call hits Valkey/Redis right away, unlike scalar `field` setters which are deferred until `save`. Each call also runs `warn_if_dirty!` and cascades expiration. (See the write-model notes in `CLAUDE.md` for the deferred-vs-immediate split.)
|
|
11
|
+
|
|
12
|
+
Multi-element adds issue **one** command for the whole batch, not one per element. Populating a large collection is therefore a single round-trip even without an explicit pipeline.
|
|
13
|
+
|
|
14
|
+
The argument shape follows the collection's structure, and is consistent across the codebase:
|
|
15
|
+
|
|
16
|
+
- **Value-only** collections (`UnsortedSet`, `ListKey`) take a **variadic splat**; arguments are flattened and `nil`-compacted.
|
|
17
|
+
- **Keyed/pair** collections (`HashKey` is `field => value`, `SortedSet` is `member => score`) take a **single Hash** via `update` (aliased `merge!`), raising `ArgumentError` on a non-Hash.
|
|
18
|
+
|
|
19
|
+
| Type | Bulk method | Call shape | Redis command |
|
|
20
|
+
|---|---|---|---|
|
|
21
|
+
| `UnsortedSet` | `add(*values)` | `tags.add(:a, :b, :c)` | one `SADD` |
|
|
22
|
+
| `ListKey` | `push(*values)` / `unshift(*values)` | `log.push(1, 2, 3)` | one `RPUSH` / `LPUSH` |
|
|
23
|
+
| `HashKey` | `update(hash)` / `merge!` | `cfg.update(a: 1, b: 2)` | one `HMSET` |
|
|
24
|
+
| `SortedSet` | `update(hash)` / `merge!` | `board.update("alice" => 1000, "bob" => 850)` | one `ZADD` |
|
|
25
|
+
|
|
26
|
+
```ruby
|
|
27
|
+
tags.add(:ruby, :redis, :valkey) # 1 SADD, returns self
|
|
28
|
+
log.push("a", "b", "c") # 1 RPUSH → [a, b, c]
|
|
29
|
+
board.update("alice" => 1000, "bob" => 850) # 1 ZADD, returns new-member count (2)
|
|
30
|
+
board.merge!("alice" => 1200) # 1 ZADD, score updated → returns 0
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Behavior notes:
|
|
34
|
+
|
|
35
|
+
- **Ordering**: `push` preserves argument order; `unshift` prepends each element in turn, so `unshift(a, b, c)` leaves the list head as `c, b, a` (Redis `LPUSH` semantics — unchanged from the prior per-element implementation). Sets are unordered; sorted sets order by score.
|
|
36
|
+
- **Empty input is a no-op**: `add()` / `push()` / `update({})` issue no command. Set/list adds return `self`; `SortedSet#update` returns `0`.
|
|
37
|
+
- **`SortedSet#add(val, score, …)` is unchanged and not bulk** — it takes a single member plus score and the conditional ZADD options (`nx:`, `xx:`, `gt:`, `lt:`, `ch:`). An Array passed as `val` is stored as one JSON-encoded member, not exploded into many. Use `update`/`merge!` for bulk insertion.
|
|
38
|
+
|
|
39
|
+
The iteration methods `each` and `each_record` efficiently handle large collections by paginating through Valkey/Redis data structures, but they serve different purposes and yield different results. Here's how the two iterate, using `ModelClass.instances` (a `SortedSet` with `reference: true`) as the running example.
|
|
40
|
+
|
|
41
|
+
## `each` — yields **members** (identifiers, raw strings)
|
|
42
|
+
|
|
43
|
+
`each` is implemented per type. For the `instances` SortedSet, it pages through the ZSET with either `ZRANGEBYSCORE` (when `since:`/`until:` are given) or `ZSCAN` (unbounded), yielding one deserialized member at a time.
|
|
44
|
+
|
|
45
|
+
```mermaid
|
|
46
|
+
flowchart TD
|
|
47
|
+
Caller["ModelClass.instances.each { |id| ... }"] --> EachImpl["SortedSet#each"]
|
|
48
|
+
EachImpl --> Decide{since/until?}
|
|
49
|
+
Decide -- yes --> ZRBS["ZRANGEBYSCORE key min max LIMIT 0 batch_size WITHSCORES"]
|
|
50
|
+
Decide -- no --> ZSCAN["ZSCAN key cursor COUNT batch_size"]
|
|
51
|
+
ZRBS --> Page["Page of raw members"]
|
|
52
|
+
ZSCAN --> Page
|
|
53
|
+
Page --> Yield["yield deserialize_value(member)"]
|
|
54
|
+
Yield --> More{more pages?}
|
|
55
|
+
More -- yes --> Decide
|
|
56
|
+
More -- no --> Done["return self"]
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Per-type variations:
|
|
60
|
+
- `ListKey#each` — paginates with `LRANGE start stop` (no SCAN equivalent)
|
|
61
|
+
- `UnsortedSet#each` / `HashKey#each` — `SSCAN` / `HSCAN`, optional `matching:` glob
|
|
62
|
+
- `SortedSet#each` — `ZRANGEBYSCORE` (bounded) or `ZSCAN` (unbounded)
|
|
63
|
+
|
|
64
|
+
You get **identifiers only**. No record loading. One Redis round-trip per page.
|
|
65
|
+
|
|
66
|
+
## `each_record` — yields **loaded Horreum records**
|
|
67
|
+
|
|
68
|
+
`each_record` is defined once in `CollectionBase` and delegates to `each` to collect identifiers, then batches them into `record_class.load_multi` (pipelined `HGETALL`s), filters ghosts, and yields the live records.
|
|
69
|
+
|
|
70
|
+
```mermaid
|
|
71
|
+
flowchart TD
|
|
72
|
+
Caller["ModelClass.instances.each_record { |rec| ... }"] --> ER["each_record(batch_size, pipeline, **filters)"]
|
|
73
|
+
ER --> Validate{"pipeline <= batch_size?"}
|
|
74
|
+
Validate -- no --> Raise["raise ArgumentError"]
|
|
75
|
+
Validate -- yes --> CallEach["each(**filters) do |member|"]
|
|
76
|
+
CallEach --> Extract["id = member.is_a?(Array) ? member.first : member"]
|
|
77
|
+
Extract --> Buffer["buffer << id"]
|
|
78
|
+
Buffer --> Full{"buffer.size >= batch_size?"}
|
|
79
|
+
Full -- no --> CallEach
|
|
80
|
+
Full -- yes --> Load["record_class.load_multi(ids) -- pipelined HGETALLs"]
|
|
81
|
+
Load --> Compact["live = records.compact -- drop ghosts"]
|
|
82
|
+
Compact --> Mode{pipeline?}
|
|
83
|
+
Mode -- nil --> Serial["live.each { |r| block.call(r) }"]
|
|
84
|
+
Mode -- positive --> Pipe["live.each_slice(pipeline) do |group|<br/>record_class.pipelined { group.each &block }<br/>end"]
|
|
85
|
+
Serial --> Clear["buffer.clear; resume each"]
|
|
86
|
+
Pipe --> Clear
|
|
87
|
+
Clear --> CallEach
|
|
88
|
+
CallEach -. each exhausted .-> Flush["process_batch(buffer) if any remain"]
|
|
89
|
+
Flush --> Return["return self"]
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### Concrete timeline for `User.instances.each_record(batch_size: 100, pipeline: 25) { |u| u.touch! }`
|
|
93
|
+
|
|
94
|
+
```
|
|
95
|
+
SortedSet#each (ZSCAN page 1, 100 ids)
|
|
96
|
+
├─ buffer fills to 100
|
|
97
|
+
├─ load_multi(ids) → 1 pipeline of 100 HGETALLs
|
|
98
|
+
├─ compact ghosts → e.g. 97 live records
|
|
99
|
+
├─ slice(25):
|
|
100
|
+
│ pipelined { 25 × u.touch! } ← 1 Redis pipeline
|
|
101
|
+
│ pipelined { 25 × u.touch! } ← 1 Redis pipeline
|
|
102
|
+
│ pipelined { 25 × u.touch! } ← 1 Redis pipeline
|
|
103
|
+
│ pipelined { 22 × u.touch! } ← 1 Redis pipeline
|
|
104
|
+
└─ buffer.clear
|
|
105
|
+
SortedSet#each (ZSCAN page 2, 100 ids)
|
|
106
|
+
└─ … repeat …
|
|
107
|
+
SortedSet#each exhausted
|
|
108
|
+
└─ flush any remaining buffered ids the same way
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Key differences
|
|
112
|
+
|
|
113
|
+
| Aspect | `each` | `each_record` |
|
|
114
|
+
|---|---|---|
|
|
115
|
+
| Yields | raw identifier (or `[field, value]` for `HashKey`) | loaded Horreum instance |
|
|
116
|
+
| Redis ops per yield | 0 extra (already paged) | amortized `HGETALL` via `load_multi` batch |
|
|
117
|
+
| Requires `reference: true` + `:class` | no | yes (raises `Familia::Problem` otherwise) |
|
|
118
|
+
| Ghost handling | yields the dangling id | `compact` drops them silently |
|
|
119
|
+
| Write pipelining | not built-in | `pipeline:` groups block-body writes into `pipelined` blocks |
|
|
120
|
+
| Filters | type-specific (`since:`, `matching:`, …) | forwarded to underlying `each` |
|
|
121
|
+
|
|
122
|
+
So `each_record` is a thin orchestration layer: it leans on the type's own `each` for read pagination, then layers (1) batched record hydration and (2) optional write pipelining on top.
|
|
123
|
+
|
|
124
|
+
## Choosing a `pipeline` mode
|
|
125
|
+
|
|
126
|
+
`each_record` has two dispatch modes, controlled by `pipeline:`. The parameter answers a single question: **may the dispatch loop wrap your block in a `pipelined { }`?**
|
|
127
|
+
|
|
128
|
+
| Value | Dispatch | Use when the block… |
|
|
129
|
+
|---|---|---|
|
|
130
|
+
| `nil` (default) | Each record runs in its own connection context, no pipeline wrapper | …reads, OR calls `save` / `commit_fields` / `transaction` / anything with its own internal MULTI |
|
|
131
|
+
| positive integer | Groups of `pipeline` records run inside `record_class.pipelined { ... }` | …only issues fast writers (`record.field!`) that tolerate being queued |
|
|
132
|
+
|
|
133
|
+
Note: `pipeline: 0` raises `ArgumentError`. Use `pipeline: nil` to disable pipelining.
|
|
134
|
+
|
|
135
|
+
The read-only case and the serial-write case collapse into the same mode because both require **immediate** execution with real return values. Wrapping `save` in an outer `pipelined` would either return `Redis::Future` objects or raise `ConflictingContextError` when `save`'s internal transaction tries to open.
|
|
136
|
+
|
|
137
|
+
### The three idiomatic patterns
|
|
138
|
+
|
|
139
|
+
```ruby
|
|
140
|
+
# 1. Read-only iteration — the default (pipeline: nil) is correct
|
|
141
|
+
User.instances.each_record do |user|
|
|
142
|
+
puts "#{user.email} #{user.last_login}"
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
# 2. Serial writes — the default (pipeline: nil) is required for save / commit_fields / transaction
|
|
146
|
+
User.instances.each_record do |user|
|
|
147
|
+
user.score = recompute(user)
|
|
148
|
+
user.save
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
# 3. Pipelined fast writers — opt-in optimization
|
|
152
|
+
User.instances.each_record(pipeline: 50) do |user|
|
|
153
|
+
user.last_seen_at! Familia.now # single HSET, safe to queue in pipeline
|
|
154
|
+
end
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
### Pipelining footgun
|
|
158
|
+
|
|
159
|
+
If you enable pipelining and your block reads from a related collection (e.g. `user.sessions.size`), that read is queued into the pipeline and returns a `Redis::Future` rather than a value. Omit the `pipeline:` parameter (or explicitly pass `pipeline: nil`) whenever the block needs real return values from Redis.
|
data/docs/migrating/v2.9.0.md
CHANGED
|
@@ -82,12 +82,12 @@ Org.instances.each_record(batch_size: 100) do |org|
|
|
|
82
82
|
end
|
|
83
83
|
|
|
84
84
|
# Control pipelining depth separately from fetch batch size
|
|
85
|
-
Org.instances.each_record(batch_size: 500,
|
|
85
|
+
Org.instances.each_record(batch_size: 500, pipeline: 50) do |org|
|
|
86
86
|
org.status!("active")
|
|
87
87
|
end
|
|
88
88
|
|
|
89
|
-
# Serial execution (no pipelining)
|
|
90
|
-
Org.instances.each_record(batch_size: 100
|
|
89
|
+
# Serial execution (no pipelining) — this is the default
|
|
90
|
+
Org.instances.each_record(batch_size: 100) do |org|
|
|
91
91
|
org.complex_operation
|
|
92
92
|
end
|
|
93
93
|
```
|
|
@@ -45,10 +45,10 @@ module Familia
|
|
|
45
45
|
# filtered out.
|
|
46
46
|
#
|
|
47
47
|
# @param batch_size [Integer] Number of identifiers to load per batch
|
|
48
|
-
# @param
|
|
49
|
-
# in the block. When nil
|
|
50
|
-
# When positive, fast writers in the block will be pipelined
|
|
51
|
-
# groups of this size.
|
|
48
|
+
# @param pipeline [Integer, nil] Controls pipelining depth for writes
|
|
49
|
+
# in the block. When nil (default), writes are serial (no pipelining).
|
|
50
|
+
# When a positive integer, fast writers in the block will be pipelined
|
|
51
|
+
# in groups of this size. Must not exceed batch_size.
|
|
52
52
|
# @param filters [Hash] Additional filter parameters passed to `each`.
|
|
53
53
|
# Available filters depend on the collection type:
|
|
54
54
|
# - SortedSet: `since:`, `until:`, `cursor_batch_size:`
|
|
@@ -58,20 +58,17 @@ module Familia
|
|
|
58
58
|
# @yield [record] Each loaded Horreum record (non-nil)
|
|
59
59
|
# @return [Enumerator, self] Returns Enumerator if no block given, self otherwise
|
|
60
60
|
#
|
|
61
|
-
# @example Iterate over all records
|
|
61
|
+
# @example Iterate over all records (no pipelining, safe default)
|
|
62
62
|
# User.instances.each_record { |user| user.deactivate! }
|
|
63
63
|
#
|
|
64
64
|
# @example With time filter (for SortedSet)
|
|
65
65
|
# User.instances.each_record(since: 1.day.ago) { |u| notify(u) }
|
|
66
66
|
#
|
|
67
67
|
# @example Pipeline writes in groups
|
|
68
|
-
# items.each_record(batch_size: 500,
|
|
68
|
+
# items.each_record(batch_size: 500, pipeline: 50) { |r| r.foo! 'bar' }
|
|
69
69
|
#
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
#
|
|
73
|
-
def each_record(batch_size: 100, write_size: batch_size, **filters, &block)
|
|
74
|
-
return to_enum(:each_record, batch_size: batch_size, write_size: write_size, **filters) unless block
|
|
70
|
+
def each_record(batch_size: 100, pipeline: nil, **filters, &block)
|
|
71
|
+
return to_enum(:each_record, batch_size: batch_size, pipeline: pipeline, **filters) unless block
|
|
75
72
|
|
|
76
73
|
# Determine the class to load records from
|
|
77
74
|
# For reference DataTypes, @opts[:class] holds the Horreum class
|
|
@@ -80,10 +77,10 @@ module Familia
|
|
|
80
77
|
raise Familia::Problem, "each_record requires a reference DataType with a :class option that responds to load_multi"
|
|
81
78
|
end
|
|
82
79
|
|
|
83
|
-
# Validate
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
80
|
+
# Validate batch_size and pipeline constraints
|
|
81
|
+
raise ArgumentError, "batch_size must be a positive integer (got #{batch_size.inspect})" unless batch_size.is_a?(Integer) && batch_size.positive?
|
|
82
|
+
raise ArgumentError, "pipeline must be nil or a positive integer (got #{pipeline.inspect})" unless pipeline.nil? || (pipeline.is_a?(Integer) && pipeline.positive?)
|
|
83
|
+
raise ArgumentError, "pipeline (#{pipeline}) cannot exceed batch_size (#{batch_size})" if pipeline&.> batch_size
|
|
87
84
|
|
|
88
85
|
# Collect identifiers in batches
|
|
89
86
|
buffer = []
|
|
@@ -97,12 +94,12 @@ module Familia
|
|
|
97
94
|
# Filter out ghosts (nil results from expired keys)
|
|
98
95
|
live_records = records.compact
|
|
99
96
|
|
|
100
|
-
if
|
|
97
|
+
if pipeline.nil?
|
|
101
98
|
# Serial mode - no pipelining, execute block for each record directly
|
|
102
99
|
live_records.each { |record| block.call(record) }
|
|
103
100
|
else
|
|
104
101
|
# Pipelined mode - group records and wrap each group in a pipeline
|
|
105
|
-
live_records.each_slice(
|
|
102
|
+
live_records.each_slice(pipeline) do |group|
|
|
106
103
|
record_class.pipelined do
|
|
107
104
|
group.each { |record| block.call(record) }
|
|
108
105
|
end
|
|
@@ -25,7 +25,8 @@ module Familia
|
|
|
25
25
|
def push *values
|
|
26
26
|
warn_if_dirty!
|
|
27
27
|
echo :push, Familia.pretty_stack(limit: 1) if Familia.debug
|
|
28
|
-
values.flatten.compact.
|
|
28
|
+
serialized = values.flatten.compact.map { |v| serialize_value(v) }
|
|
29
|
+
dbclient.rpush(dbkey, serialized) unless serialized.empty?
|
|
29
30
|
dbclient.ltrim dbkey, -@opts[:maxlength], -1 if @opts[:maxlength]
|
|
30
31
|
update_expiration
|
|
31
32
|
self
|
|
@@ -43,7 +44,8 @@ module Familia
|
|
|
43
44
|
# scalar field changes, consider calling save first to avoid split-brain state.
|
|
44
45
|
def unshift *values
|
|
45
46
|
warn_if_dirty!
|
|
46
|
-
values.flatten.compact.
|
|
47
|
+
serialized = values.flatten.compact.map { |v| serialize_value(v) }
|
|
48
|
+
dbclient.lpush(dbkey, serialized) unless serialized.empty?
|
|
47
49
|
# TODO: test maxlength
|
|
48
50
|
dbclient.ltrim dbkey, 0, @opts[:maxlength] - 1 if @opts[:maxlength]
|
|
49
51
|
update_expiration
|
|
@@ -228,7 +230,7 @@ module Familia
|
|
|
228
230
|
raise ArgumentError, "position must be :before or :after, got #{position.inspect}"
|
|
229
231
|
end
|
|
230
232
|
result = dbclient.linsert dbkey, pos, serialize_value(pivot), serialize_value(value)
|
|
231
|
-
update_expiration if
|
|
233
|
+
update_expiration if Familia.positive?(result) == true
|
|
232
234
|
result
|
|
233
235
|
end
|
|
234
236
|
alias linsert insert
|
|
@@ -265,7 +267,7 @@ module Familia
|
|
|
265
267
|
|
|
266
268
|
warn_if_dirty!
|
|
267
269
|
result = dbclient.rpushx(dbkey, serialized_values)
|
|
268
|
-
update_expiration if
|
|
270
|
+
update_expiration if Familia.positive?(result) == true
|
|
269
271
|
result
|
|
270
272
|
end
|
|
271
273
|
alias rpushx pushx
|
|
@@ -281,7 +283,7 @@ module Familia
|
|
|
281
283
|
|
|
282
284
|
warn_if_dirty!
|
|
283
285
|
result = dbclient.lpushx(dbkey, serialized_values)
|
|
284
|
-
update_expiration if
|
|
286
|
+
update_expiration if Familia.positive?(result) == true
|
|
285
287
|
result
|
|
286
288
|
end
|
|
287
289
|
alias lpushx unshiftx
|
|
@@ -135,6 +135,51 @@ module Familia
|
|
|
135
135
|
end
|
|
136
136
|
alias add_element add
|
|
137
137
|
|
|
138
|
+
# Bulk-adds or updates multiple members in a single ZADD.
|
|
139
|
+
#
|
|
140
|
+
# Mirrors HashKey#update/merge! -- the established Familia pattern for
|
|
141
|
+
# bulk-setting keyed collections. A sorted set is member => score, the
|
|
142
|
+
# same pair shape as HashKey's field => value, so it takes a Hash rather
|
|
143
|
+
# than the variadic splat used by the value-only UnsortedSet/ListKey.
|
|
144
|
+
#
|
|
145
|
+
# Issues exactly one ZADD instead of one round-trip per member, which is
|
|
146
|
+
# what makes populating a large sorted set fast.
|
|
147
|
+
#
|
|
148
|
+
# @param hsh [Hash] Mapping of member => score (Object => Numeric)
|
|
149
|
+
# @return [Integer] Number of new members added (members whose score was
|
|
150
|
+
# merely updated are not counted), per redis-rb's bulk ZADD return value
|
|
151
|
+
# @raise [ArgumentError] If the argument is not a Hash, or any score is
|
|
152
|
+
# not Numeric
|
|
153
|
+
#
|
|
154
|
+
# @example
|
|
155
|
+
# board.update("alice" => 1000, "bob" => 850) #=> 2
|
|
156
|
+
# board.merge!("alice" => 1200) #=> 0 (score updated)
|
|
157
|
+
#
|
|
158
|
+
# @note Unlike single-value #add, scores are required: this bulk path does
|
|
159
|
+
# not default a missing score to Familia.now. A non-Numeric score (e.g.
|
|
160
|
+
# nil) raises ArgumentError rather than surfacing a low-level client
|
|
161
|
+
# error.
|
|
162
|
+
#
|
|
163
|
+
# @note Like #add, this executes immediately (not deferred) and cascades
|
|
164
|
+
# expiration. Empty input is a no-op returning 0.
|
|
165
|
+
def update(hsh = {})
|
|
166
|
+
warn_if_dirty!
|
|
167
|
+
raise ArgumentError, 'Argument to bulk add must be a hash' unless hsh.is_a?(Hash)
|
|
168
|
+
return 0 if hsh.empty?
|
|
169
|
+
|
|
170
|
+
pairs = hsh.map do |member, score|
|
|
171
|
+
unless score.is_a?(Numeric)
|
|
172
|
+
raise ArgumentError, "SortedSet#update score for #{member.inspect} must be Numeric, got #{score.class}"
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
[score, serialize_value(member)]
|
|
176
|
+
end
|
|
177
|
+
ret = dbclient.zadd(dbkey, pairs)
|
|
178
|
+
update_expiration
|
|
179
|
+
ret
|
|
180
|
+
end
|
|
181
|
+
alias merge! update
|
|
182
|
+
|
|
138
183
|
def score(val)
|
|
139
184
|
ret = dbclient.zscore dbkey, serialize_value(val)
|
|
140
185
|
ret&.to_f
|
|
@@ -26,7 +26,8 @@ module Familia
|
|
|
26
26
|
# scalar field changes, consider calling save first to avoid split-brain state.
|
|
27
27
|
def add *values
|
|
28
28
|
warn_if_dirty!
|
|
29
|
-
values.flatten.compact.
|
|
29
|
+
serialized = values.flatten.compact.map { |v| serialize_value(v) }
|
|
30
|
+
dbclient.sadd?(dbkey, serialized) unless serialized.empty?
|
|
30
31
|
update_expiration
|
|
31
32
|
self
|
|
32
33
|
end
|
|
@@ -108,25 +108,7 @@ module Familia
|
|
|
108
108
|
klass.define_method fast_method_name do |val|
|
|
109
109
|
raise ArgumentError, "#{fast_method_name} requires a value" if val.nil?
|
|
110
110
|
|
|
111
|
-
#
|
|
112
|
-
# would be Redis::Future which doesn't support zero?/positive? checks
|
|
113
|
-
if Fiber[:familia_transaction]
|
|
114
|
-
Familia.trace :FAST_WRITER_BLOCKED, dbkey,
|
|
115
|
-
"#{fast_method_name} blocked by active transaction context"
|
|
116
|
-
raise Familia::OperationModeError, <<~ERROR_MESSAGE.chomp
|
|
117
|
-
Cannot call fast writer #{fast_method_name} within a transaction.
|
|
118
|
-
Use multi_field_update or commit_fields instead.
|
|
119
|
-
ERROR_MESSAGE
|
|
120
|
-
elsif Fiber[:familia_pipeline]
|
|
121
|
-
Familia.trace :FAST_WRITER_BLOCKED, dbkey,
|
|
122
|
-
"#{fast_method_name} blocked by active pipeline context"
|
|
123
|
-
raise Familia::OperationModeError, <<~ERROR_MESSAGE.chomp
|
|
124
|
-
Cannot call fast writer #{fast_method_name} within a pipeline.
|
|
125
|
-
Restructure to call fast writers outside the pipeline.
|
|
126
|
-
ERROR_MESSAGE
|
|
127
|
-
end
|
|
128
|
-
|
|
129
|
-
# UnsortedSet via the setter method to get proper ConcealedString wrapping
|
|
111
|
+
# Use via the setter method to get proper ConcealedString wrapping
|
|
130
112
|
send(:"#{method_name}=", val) if method_name
|
|
131
113
|
|
|
132
114
|
# Get the ConcealedString and extract encrypted data for storage
|
|
@@ -136,7 +118,7 @@ module Familia
|
|
|
136
118
|
return false if encrypted_data.nil?
|
|
137
119
|
|
|
138
120
|
ret = hset(field_name, encrypted_data)
|
|
139
|
-
|
|
121
|
+
Familia.success?(ret)
|
|
140
122
|
end
|
|
141
123
|
end
|
|
142
124
|
end
|
|
@@ -342,7 +342,7 @@ module Familia
|
|
|
342
342
|
# @return [Boolean] true if TTL is set, false if data persists indefinitely
|
|
343
343
|
#
|
|
344
344
|
def expires?
|
|
345
|
-
|
|
345
|
+
Familia.positive?(ttl)
|
|
346
346
|
end
|
|
347
347
|
|
|
348
348
|
# Check if this object's data has expired or will expire soon
|
|
@@ -377,7 +377,7 @@ module Familia
|
|
|
377
377
|
#
|
|
378
378
|
def extend_expiration(duration)
|
|
379
379
|
current_ttl = ttl
|
|
380
|
-
return false unless
|
|
380
|
+
return false unless Familia.positive?(current_ttl) == true # no current expiration set
|
|
381
381
|
|
|
382
382
|
new_ttl = current_ttl + duration.to_f
|
|
383
383
|
expire(new_ttl)
|
data/lib/familia/field_type.rb
CHANGED
|
@@ -154,24 +154,6 @@ module Familia
|
|
|
154
154
|
# Handle Redis::Future objects during transactions
|
|
155
155
|
return hget(field_name) if val.nil? || val.is_a?(Redis::Future)
|
|
156
156
|
|
|
157
|
-
# Prevent fast writer within transaction/pipeline - the return value
|
|
158
|
-
# would be Redis::Future which doesn't support zero?/positive? checks
|
|
159
|
-
if Fiber[:familia_transaction]
|
|
160
|
-
Familia.trace :FAST_WRITER_BLOCKED, dbkey,
|
|
161
|
-
"#{fast_method_name} blocked by active transaction context"
|
|
162
|
-
raise Familia::OperationModeError, <<~ERROR_MESSAGE.chomp
|
|
163
|
-
Cannot call fast writer #{fast_method_name} within a transaction.
|
|
164
|
-
Use multi_field_update or commit_fields instead.
|
|
165
|
-
ERROR_MESSAGE
|
|
166
|
-
elsif Fiber[:familia_pipeline]
|
|
167
|
-
Familia.trace :FAST_WRITER_BLOCKED, dbkey,
|
|
168
|
-
"#{fast_method_name} blocked by active pipeline context"
|
|
169
|
-
raise Familia::OperationModeError, <<~ERROR_MESSAGE.chomp
|
|
170
|
-
Cannot call fast writer #{fast_method_name} within a pipeline.
|
|
171
|
-
Restructure to call fast writers outside the pipeline.
|
|
172
|
-
ERROR_MESSAGE
|
|
173
|
-
end
|
|
174
|
-
|
|
175
157
|
begin
|
|
176
158
|
# Trace the operation if debugging is enabled
|
|
177
159
|
Familia.trace :FAST_WRITER, nil, "#{field_name}: #{val.inspect}" if Familia.debug?
|
|
@@ -192,7 +174,7 @@ module Familia
|
|
|
192
174
|
|
|
193
175
|
clear_dirty!(field_name) if respond_to?(:clear_dirty!)
|
|
194
176
|
|
|
195
|
-
|
|
177
|
+
Familia.success?(ret)
|
|
196
178
|
rescue Familia::Problem => e
|
|
197
179
|
raise "#{fast_method_name} method failed: #{e.message}", e.backtrace
|
|
198
180
|
end
|
|
@@ -480,24 +480,6 @@ module Familia
|
|
|
480
480
|
# Handle Redis::Future objects during transactions
|
|
481
481
|
return hget field_name if val.nil? || val.is_a?(Redis::Future)
|
|
482
482
|
|
|
483
|
-
# Prevent fast writer within transaction/pipeline - the return value
|
|
484
|
-
# would be Redis::Future which doesn't support zero?/positive? checks
|
|
485
|
-
if Fiber[:familia_transaction]
|
|
486
|
-
Familia.trace :FAST_WRITER_BLOCKED, dbkey,
|
|
487
|
-
"#{fast_method_name} blocked by active transaction context"
|
|
488
|
-
raise Familia::OperationModeError, <<~ERROR_MESSAGE.chomp
|
|
489
|
-
Cannot call fast writer #{fast_method_name} within a transaction.
|
|
490
|
-
Use multi_field_update or commit_fields instead.
|
|
491
|
-
ERROR_MESSAGE
|
|
492
|
-
elsif Fiber[:familia_pipeline]
|
|
493
|
-
Familia.trace :FAST_WRITER_BLOCKED, dbkey,
|
|
494
|
-
"#{fast_method_name} blocked by active pipeline context"
|
|
495
|
-
raise Familia::OperationModeError, <<~ERROR_MESSAGE.chomp
|
|
496
|
-
Cannot call fast writer #{fast_method_name} within a pipeline.
|
|
497
|
-
Restructure to call fast writers outside the pipeline.
|
|
498
|
-
ERROR_MESSAGE
|
|
499
|
-
end
|
|
500
|
-
|
|
501
483
|
begin
|
|
502
484
|
# Trace the operation if debugging is enabled.
|
|
503
485
|
Familia.trace :FAST_WRITER, nil, "#{field_name}: #{val.inspect}" if Familia.debug?
|
|
@@ -518,7 +500,7 @@ module Familia
|
|
|
518
500
|
|
|
519
501
|
clear_dirty!(field_name) if respond_to?(:clear_dirty!)
|
|
520
502
|
|
|
521
|
-
|
|
503
|
+
Familia.success?(ret)
|
|
522
504
|
rescue Familia::Problem => e
|
|
523
505
|
# Raise a custom error message if an exception occurs during the execution of the method.
|
|
524
506
|
raise "#{fast_method_name} method failed: #{e.message}", e.backtrace
|
|
@@ -158,7 +158,7 @@ module Familia
|
|
|
158
158
|
# Safe mode: Check existence first (original behavior)
|
|
159
159
|
# We use a lower-level method here b/c we're working with the
|
|
160
160
|
# full key and not just the identifier.
|
|
161
|
-
does_exist = dbclient.exists(objkey)
|
|
161
|
+
does_exist = Familia.positive?(dbclient.exists(objkey))
|
|
162
162
|
|
|
163
163
|
Familia.debug "[find_by_key] #{self} from key #{objkey} (exists: #{does_exist})"
|
|
164
164
|
Familia.trace :FIND_BY_DBKEY_KEY, nil, objkey
|
data/lib/familia/utils.rb
CHANGED
|
@@ -8,6 +8,54 @@ module Familia
|
|
|
8
8
|
module Utils
|
|
9
9
|
using Familia::Refinements::TimeLiterals
|
|
10
10
|
|
|
11
|
+
# Future-aware success check for Redis command return values.
|
|
12
|
+
#
|
|
13
|
+
# Redis commands like HSET return 0 (updated existing) or 1 (created new).
|
|
14
|
+
# Both are success states. Inside a pipeline or transaction, the return
|
|
15
|
+
# value is a Redis::Future which cannot be inspected until the block
|
|
16
|
+
# completes.
|
|
17
|
+
#
|
|
18
|
+
# @param ret [Integer, Redis::Future] The return value from a Redis command
|
|
19
|
+
# @return [Boolean, Redis::Future] true/false for concrete values,
|
|
20
|
+
# passthrough for Futures
|
|
21
|
+
#
|
|
22
|
+
# @example Normal usage
|
|
23
|
+
# ret = dbclient.hset(key, field, value)
|
|
24
|
+
# Familia.success?(ret) #=> true (for 0 or 1)
|
|
25
|
+
#
|
|
26
|
+
# @example Inside pipeline
|
|
27
|
+
# pipelined do
|
|
28
|
+
# ret = dbclient.hset(key, field, value)
|
|
29
|
+
# Familia.success?(ret) #=> Redis::Future (passthrough)
|
|
30
|
+
# end
|
|
31
|
+
#
|
|
32
|
+
def success?(ret)
|
|
33
|
+
ret.is_a?(Redis::Future) ? ret : (ret.zero? || ret.positive?)
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# Future-aware positive check for Redis command return values.
|
|
37
|
+
#
|
|
38
|
+
# For commands where 0 means "nothing happened" and positive means
|
|
39
|
+
# "something happened" (e.g., EXISTS, DEL, LINSERT, TTL checks).
|
|
40
|
+
#
|
|
41
|
+
# @param ret [Integer, Redis::Future] The return value from a Redis command
|
|
42
|
+
# @return [Boolean, Redis::Future] true/false for concrete values,
|
|
43
|
+
# passthrough for Futures
|
|
44
|
+
#
|
|
45
|
+
# @example Check if key exists
|
|
46
|
+
# ret = dbclient.exists(key)
|
|
47
|
+
# Familia.positive?(ret) #=> true if key exists
|
|
48
|
+
#
|
|
49
|
+
# @example Check TTL inside pipeline
|
|
50
|
+
# pipelined do
|
|
51
|
+
# ret = dbclient.ttl(key)
|
|
52
|
+
# Familia.positive?(ret) #=> Redis::Future (passthrough)
|
|
53
|
+
# end
|
|
54
|
+
#
|
|
55
|
+
def positive?(ret)
|
|
56
|
+
ret.is_a?(Redis::Future) ? ret : ret.positive?
|
|
57
|
+
end
|
|
58
|
+
|
|
11
59
|
# Joins array elements with Familia delimiter
|
|
12
60
|
# @param val [Array] elements to join
|
|
13
61
|
# @return [String] joined string
|
data/lib/familia/version.rb
CHANGED
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
require_relative '../support/helpers/test_helpers'
|
|
2
|
+
|
|
3
|
+
Familia.debug = false
|
|
4
|
+
|
|
5
|
+
# Test class for fast writer pipeline/transaction support
|
|
6
|
+
class FastWriterPipelineTest < Familia::Horreum
|
|
7
|
+
identifier_field :testid
|
|
8
|
+
field :testid
|
|
9
|
+
field :name
|
|
10
|
+
field :planid
|
|
11
|
+
field :region
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
# Clean slate
|
|
15
|
+
@obj = FastWriterPipelineTest.new(testid: 'fw-pipeline-test', name: 'initial', planid: 'old_plan')
|
|
16
|
+
@obj.destroy!
|
|
17
|
+
@obj.save
|
|
18
|
+
|
|
19
|
+
## Fast writer inside pipeline returns Redis::Future
|
|
20
|
+
result = nil
|
|
21
|
+
@obj.pipelined do
|
|
22
|
+
result = @obj.planid!('new_plan_v1')
|
|
23
|
+
end
|
|
24
|
+
result.is_a?(Redis::Future)
|
|
25
|
+
#=> true
|
|
26
|
+
|
|
27
|
+
## Fast writer value is persisted after pipeline completes
|
|
28
|
+
@obj.refresh!
|
|
29
|
+
@obj.planid
|
|
30
|
+
#=> 'new_plan_v1'
|
|
31
|
+
|
|
32
|
+
## Multiple fast writers inside single pipeline all return Futures
|
|
33
|
+
results = []
|
|
34
|
+
@obj.pipelined do
|
|
35
|
+
results << @obj.planid!('plan_v2')
|
|
36
|
+
results << @obj.region!('ca-east-1')
|
|
37
|
+
end
|
|
38
|
+
results.all? { |r| r.is_a?(Redis::Future) }
|
|
39
|
+
#=> true
|
|
40
|
+
|
|
41
|
+
## Multiple fast writer values persisted after pipeline
|
|
42
|
+
@obj.refresh!
|
|
43
|
+
[@obj.planid, @obj.region]
|
|
44
|
+
#=> ['plan_v2', 'ca-east-1']
|
|
45
|
+
|
|
46
|
+
## Fast writer inside transaction returns Redis::Future
|
|
47
|
+
result = nil
|
|
48
|
+
@obj.transaction do
|
|
49
|
+
result = @obj.name!('tx-updated')
|
|
50
|
+
end
|
|
51
|
+
result.is_a?(Redis::Future)
|
|
52
|
+
#=> true
|
|
53
|
+
|
|
54
|
+
## Fast writer value is persisted after transaction completes
|
|
55
|
+
@obj.refresh!
|
|
56
|
+
@obj.name
|
|
57
|
+
#=> 'tx-updated'
|
|
58
|
+
|
|
59
|
+
## touch_instances! is called during pipelined fast writer
|
|
60
|
+
@obj.class.instances.remove(@obj)
|
|
61
|
+
raise "Setup failed: should not be member" if @obj.class.instances.member?(@obj.identifier)
|
|
62
|
+
@obj.pipelined do
|
|
63
|
+
@obj.planid!('plan_v3')
|
|
64
|
+
end
|
|
65
|
+
@obj.class.instances.member?(@obj.identifier)
|
|
66
|
+
#=> true
|
|
67
|
+
|
|
68
|
+
## touch_instances! is called during transaction fast writer
|
|
69
|
+
@obj.class.instances.remove(@obj)
|
|
70
|
+
raise "Setup failed: should not be member" if @obj.class.instances.member?(@obj.identifier)
|
|
71
|
+
@obj.transaction do
|
|
72
|
+
@obj.name!('tx-touch-test')
|
|
73
|
+
end
|
|
74
|
+
@obj.class.instances.member?(@obj.identifier)
|
|
75
|
+
#=> true
|
|
76
|
+
|
|
77
|
+
## Cleanup
|
|
78
|
+
@obj.destroy!
|
|
79
|
+
true
|
|
80
|
+
#=> true
|
|
@@ -8,7 +8,7 @@ test_keys = { v1: Base64.strict_encode64('a' * 32) }
|
|
|
8
8
|
Familia.config.encryption_keys = test_keys
|
|
9
9
|
Familia.config.current_key_version = :v1
|
|
10
10
|
|
|
11
|
-
# Test class for fast writer transaction/pipeline
|
|
11
|
+
# Test class for fast writer transaction/pipeline behavior
|
|
12
12
|
class FastWriterGuardTest < Familia::Horreum
|
|
13
13
|
identifier_field :testid
|
|
14
14
|
field :testid
|
|
@@ -16,7 +16,7 @@ class FastWriterGuardTest < Familia::Horreum
|
|
|
16
16
|
field :value
|
|
17
17
|
end
|
|
18
18
|
|
|
19
|
-
# Test class for encrypted fast writer
|
|
19
|
+
# Test class for encrypted fast writer behavior
|
|
20
20
|
class EncryptedFastWriterGuardTest < Familia::Horreum
|
|
21
21
|
feature :encrypted_fields
|
|
22
22
|
identifier_field :testid
|
|
@@ -29,61 +29,43 @@ end
|
|
|
29
29
|
@testobj.destroy!
|
|
30
30
|
@testobj.save
|
|
31
31
|
|
|
32
|
-
## Fast writer
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
end
|
|
37
|
-
:should_have_raised
|
|
38
|
-
rescue Familia::OperationModeError => e
|
|
39
|
-
e.message.include?('Cannot call fast writer')
|
|
40
|
-
end
|
|
41
|
-
#=> true
|
|
42
|
-
|
|
43
|
-
## Fast writer raises OperationModeError inside pipeline
|
|
44
|
-
begin
|
|
45
|
-
@testobj.pipelined do
|
|
46
|
-
@testobj.name!('inside-pipeline')
|
|
47
|
-
end
|
|
48
|
-
:should_have_raised
|
|
49
|
-
rescue Familia::OperationModeError => e
|
|
50
|
-
e.message.include?('Cannot call fast writer')
|
|
32
|
+
## Fast writer inside transaction returns Redis::Future
|
|
33
|
+
result = nil
|
|
34
|
+
@testobj.transaction do
|
|
35
|
+
result = @testobj.name!('inside-transaction')
|
|
51
36
|
end
|
|
37
|
+
result.is_a?(Redis::Future)
|
|
52
38
|
#=> true
|
|
53
39
|
|
|
54
|
-
##
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
end
|
|
59
|
-
:should_have_raised
|
|
60
|
-
rescue Familia::OperationModeError => e
|
|
61
|
-
e.message.include?('multi_field_update') && e.message.include?('commit_fields')
|
|
62
|
-
end
|
|
63
|
-
#=> true
|
|
40
|
+
## Fast writer value is persisted after transaction completes
|
|
41
|
+
@testobj.refresh
|
|
42
|
+
@testobj.name
|
|
43
|
+
#=> 'inside-transaction'
|
|
64
44
|
|
|
65
|
-
##
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
end
|
|
70
|
-
:should_have_raised
|
|
71
|
-
rescue Familia::OperationModeError => e
|
|
72
|
-
e.message.include?('Restructure') && !e.message.include?('multi_field_update')
|
|
45
|
+
## Fast writer inside pipeline returns Redis::Future
|
|
46
|
+
result = nil
|
|
47
|
+
@testobj.pipelined do
|
|
48
|
+
result = @testobj.value!('inside-pipeline')
|
|
73
49
|
end
|
|
50
|
+
result.is_a?(Redis::Future)
|
|
74
51
|
#=> true
|
|
75
52
|
|
|
76
|
-
## Fast writer
|
|
77
|
-
@testobj.
|
|
53
|
+
## Fast writer value is persisted after pipeline completes
|
|
54
|
+
@testobj.refresh
|
|
78
55
|
@testobj.value
|
|
79
|
-
#=> '
|
|
56
|
+
#=> 'inside-pipeline'
|
|
57
|
+
|
|
58
|
+
## Fast writer works normally outside transaction (returns boolean)
|
|
59
|
+
result = @testobj.name!('direct-write')
|
|
60
|
+
[true, false].include?(result)
|
|
61
|
+
#=> true
|
|
80
62
|
|
|
81
63
|
## Fast writer as getter works inside transaction (returns Future)
|
|
82
64
|
result = nil
|
|
83
65
|
@testobj.transaction do
|
|
84
66
|
result = @testobj.value!
|
|
85
67
|
end
|
|
86
|
-
result.is_a?(Redis::Future) || result == '
|
|
68
|
+
result.is_a?(Redis::Future) || result == 'inside-pipeline'
|
|
87
69
|
#=> true
|
|
88
70
|
|
|
89
71
|
## Fast writer works after transaction completes
|
|
@@ -94,29 +76,28 @@ end
|
|
|
94
76
|
@testobj.value
|
|
95
77
|
#=> 'after-transaction'
|
|
96
78
|
|
|
97
|
-
## Encrypted fast writer
|
|
79
|
+
## Encrypted fast writer inside transaction returns Redis::Future
|
|
98
80
|
@encrypted = EncryptedFastWriterGuardTest.new(testid: 'enc-fw-guard-test')
|
|
99
81
|
@encrypted.destroy!
|
|
100
82
|
@encrypted.save
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
end
|
|
105
|
-
:should_have_raised
|
|
106
|
-
rescue Familia::OperationModeError => e
|
|
107
|
-
e.message.include?('Cannot call fast writer')
|
|
83
|
+
result = nil
|
|
84
|
+
@encrypted.transaction do
|
|
85
|
+
result = @encrypted.secret!('sensitive-data')
|
|
108
86
|
end
|
|
87
|
+
result.is_a?(Redis::Future)
|
|
109
88
|
#=> true
|
|
110
89
|
|
|
111
|
-
## Encrypted fast writer
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
90
|
+
## Encrypted fast writer value is persisted after transaction
|
|
91
|
+
@encrypted.refresh
|
|
92
|
+
@encrypted.secret.to_s.length > 0
|
|
93
|
+
#=> true
|
|
94
|
+
|
|
95
|
+
## Encrypted fast writer inside pipeline returns Redis::Future
|
|
96
|
+
result = nil
|
|
97
|
+
@encrypted.pipelined do
|
|
98
|
+
result = @encrypted.secret!('updated-secret')
|
|
119
99
|
end
|
|
100
|
+
result.is_a?(Redis::Future)
|
|
120
101
|
#=> true
|
|
121
102
|
|
|
122
103
|
## Cleanup
|
|
@@ -89,26 +89,33 @@ records.size
|
|
|
89
89
|
#=> 5
|
|
90
90
|
|
|
91
91
|
# ============================================================
|
|
92
|
-
#
|
|
92
|
+
# pipeline parameter for pipelining (renamed from write_size)
|
|
93
93
|
# ============================================================
|
|
94
94
|
|
|
95
|
-
## each_record with
|
|
96
|
-
#
|
|
95
|
+
## each_record with no pipeline param uses serial execution (safe-by-default)
|
|
96
|
+
# Default behavior is no pipelining - each record processed individually
|
|
97
97
|
records = []
|
|
98
|
-
Customer.instances.each_record
|
|
98
|
+
Customer.instances.each_record { |r| records << r if r.custid.start_with?(@test_prefix) }
|
|
99
99
|
records.size
|
|
100
100
|
#=> 5
|
|
101
101
|
|
|
102
|
-
## each_record with
|
|
102
|
+
## each_record with pipeline: for batched writes
|
|
103
|
+
# pipeline controls how many records are processed before flushing writes
|
|
104
|
+
records = []
|
|
105
|
+
Customer.instances.each_record(pipeline: 2) { |r| records << r if r.custid.start_with?(@test_prefix) }
|
|
106
|
+
records.size
|
|
107
|
+
#=> 5
|
|
108
|
+
|
|
109
|
+
## each_record with pipeline: nil for explicit serial execution
|
|
103
110
|
# Serial execution processes one at a time without batching
|
|
104
111
|
records = []
|
|
105
|
-
Customer.instances.each_record(
|
|
112
|
+
Customer.instances.each_record(pipeline: nil) { |r| records << r if r.custid.start_with?(@test_prefix) }
|
|
106
113
|
records.size
|
|
107
114
|
#=> 5
|
|
108
115
|
|
|
109
|
-
## each_record with both batch_size and
|
|
116
|
+
## each_record with both batch_size and pipeline
|
|
110
117
|
records = []
|
|
111
|
-
Customer.instances.each_record(batch_size: 3,
|
|
118
|
+
Customer.instances.each_record(batch_size: 3, pipeline: 2) { |r| records << r if r.custid.start_with?(@test_prefix) }
|
|
112
119
|
records.size
|
|
113
120
|
#=> 5
|
|
114
121
|
|
|
@@ -244,24 +251,94 @@ records.size
|
|
|
244
251
|
# Argument validation
|
|
245
252
|
# ============================================================
|
|
246
253
|
|
|
247
|
-
## each_record raises ArgumentError when
|
|
254
|
+
## each_record raises ArgumentError when batch_size is nil
|
|
255
|
+
begin
|
|
256
|
+
Customer.instances.each_record(batch_size: nil) { |r| }
|
|
257
|
+
raised = false
|
|
258
|
+
rescue ArgumentError => e
|
|
259
|
+
raised = e.message.include?('batch_size') && e.message.include?('positive')
|
|
260
|
+
end
|
|
261
|
+
raised
|
|
262
|
+
#=> true
|
|
263
|
+
|
|
264
|
+
## each_record raises ArgumentError when batch_size is 0
|
|
265
|
+
begin
|
|
266
|
+
Customer.instances.each_record(batch_size: 0) { |r| }
|
|
267
|
+
raised = false
|
|
268
|
+
rescue ArgumentError => e
|
|
269
|
+
raised = e.message.include?('batch_size') && e.message.include?('positive')
|
|
270
|
+
end
|
|
271
|
+
raised
|
|
272
|
+
#=> true
|
|
273
|
+
|
|
274
|
+
## each_record raises ArgumentError when batch_size is negative
|
|
275
|
+
begin
|
|
276
|
+
Customer.instances.each_record(batch_size: -5) { |r| }
|
|
277
|
+
raised = false
|
|
278
|
+
rescue ArgumentError => e
|
|
279
|
+
raised = e.message.include?('batch_size') && e.message.include?('positive')
|
|
280
|
+
end
|
|
281
|
+
raised
|
|
282
|
+
#=> true
|
|
283
|
+
|
|
284
|
+
## each_record raises ArgumentError when batch_size is non-integer
|
|
285
|
+
begin
|
|
286
|
+
Customer.instances.each_record(batch_size: "100") { |r| }
|
|
287
|
+
raised = false
|
|
288
|
+
rescue ArgumentError => e
|
|
289
|
+
raised = e.message.include?('batch_size') && e.message.include?('positive')
|
|
290
|
+
end
|
|
291
|
+
raised
|
|
292
|
+
#=> true
|
|
293
|
+
|
|
294
|
+
## each_record raises ArgumentError when pipeline exceeds batch_size
|
|
248
295
|
begin
|
|
249
|
-
Customer.instances.each_record(batch_size: 10,
|
|
296
|
+
Customer.instances.each_record(batch_size: 10, pipeline: 20) { |r| }
|
|
250
297
|
raised = false
|
|
251
298
|
rescue ArgumentError => e
|
|
252
|
-
raised = e.message.include?('
|
|
299
|
+
raised = e.message.include?('pipeline') && e.message.include?('batch_size')
|
|
253
300
|
end
|
|
254
301
|
raised
|
|
255
302
|
#=> true
|
|
256
303
|
|
|
257
304
|
## each_record error message includes both values
|
|
258
305
|
begin
|
|
259
|
-
Customer.instances.each_record(batch_size: 10,
|
|
306
|
+
Customer.instances.each_record(batch_size: 10, pipeline: 20) { |r| }
|
|
260
307
|
''
|
|
261
308
|
rescue ArgumentError => e
|
|
262
309
|
e.message
|
|
263
310
|
end
|
|
264
|
-
#=~ /
|
|
311
|
+
#=~ /pipeline.*20.*batch_size.*10|batch_size.*10.*pipeline.*20/
|
|
312
|
+
|
|
313
|
+
## each_record raises ArgumentError when pipeline is 0
|
|
314
|
+
begin
|
|
315
|
+
Customer.instances.each_record(pipeline: 0) { |r| }
|
|
316
|
+
raised = false
|
|
317
|
+
rescue ArgumentError => e
|
|
318
|
+
raised = e.message.include?('pipeline') && e.message.include?('positive')
|
|
319
|
+
end
|
|
320
|
+
raised
|
|
321
|
+
#=> true
|
|
322
|
+
|
|
323
|
+
## each_record raises ArgumentError when pipeline is negative
|
|
324
|
+
begin
|
|
325
|
+
Customer.instances.each_record(pipeline: -1) { |r| }
|
|
326
|
+
raised = false
|
|
327
|
+
rescue ArgumentError => e
|
|
328
|
+
raised = e.message.include?('pipeline') && e.message.include?('positive')
|
|
329
|
+
end
|
|
330
|
+
raised
|
|
331
|
+
#=> true
|
|
332
|
+
|
|
333
|
+
## each_record raises ArgumentError when pipeline is non-integer
|
|
334
|
+
begin
|
|
335
|
+
Customer.instances.each_record(pipeline: "10") { |r| }
|
|
336
|
+
raised = false
|
|
337
|
+
rescue ArgumentError => e
|
|
338
|
+
raised = e.message.include?('pipeline') && e.message.include?('positive')
|
|
339
|
+
end
|
|
340
|
+
raised
|
|
341
|
+
#=> true
|
|
265
342
|
|
|
266
343
|
# ============================================================
|
|
267
344
|
# Non-reference DataType error handling
|
|
@@ -69,4 +69,48 @@ require_relative '../../support/helpers/test_helpers'
|
|
|
69
69
|
@a.metrics.members
|
|
70
70
|
#=> ['metric2']
|
|
71
71
|
|
|
72
|
+
## Familia::SortedSet#update bulk-adds a Hash of member => score in one ZADD, returns new count
|
|
73
|
+
@u = Bone.new 'zset_bulk_update'
|
|
74
|
+
@u.metrics.update(alpha: 1, gamma: 3, beta: 2)
|
|
75
|
+
#=> 3
|
|
76
|
+
|
|
77
|
+
## Familia::SortedSet#update orders members by their given scores
|
|
78
|
+
@u.metrics.members
|
|
79
|
+
#=> ['alpha', 'beta', 'gamma']
|
|
80
|
+
|
|
81
|
+
## Familia::SortedSet#merge! is an alias and updates existing scores (0 new members)
|
|
82
|
+
@u.metrics.merge!(alpha: 10)
|
|
83
|
+
#=> 0
|
|
84
|
+
|
|
85
|
+
## Familia::SortedSet#merge! re-scored member moves position
|
|
86
|
+
@u.metrics.members
|
|
87
|
+
#=> ['beta', 'gamma', 'alpha']
|
|
88
|
+
|
|
89
|
+
## Familia::SortedSet#update with empty Hash is a no-op returning 0
|
|
90
|
+
@u.metrics.update({})
|
|
91
|
+
#=> 0
|
|
92
|
+
|
|
93
|
+
## Familia::SortedSet#update raises ArgumentError on non-Hash argument
|
|
94
|
+
begin
|
|
95
|
+
@u.metrics.update([:not, :a, :hash])
|
|
96
|
+
:no_error
|
|
97
|
+
rescue ArgumentError => e
|
|
98
|
+
e.message
|
|
99
|
+
end
|
|
100
|
+
#=> 'Argument to bulk add must be a hash'
|
|
101
|
+
|
|
102
|
+
## Familia::SortedSet#update raises a clear ArgumentError on a non-Numeric score (not auto-defaulted like #add)
|
|
103
|
+
begin
|
|
104
|
+
@u.metrics.update('alice' => 1000, 'bob' => nil)
|
|
105
|
+
:no_error
|
|
106
|
+
rescue ArgumentError => e
|
|
107
|
+
e.message
|
|
108
|
+
end
|
|
109
|
+
#=> 'SortedSet#update score for "bob" must be Numeric, got NilClass'
|
|
110
|
+
|
|
111
|
+
## Familia::SortedSet#update rejects a bad score before issuing the ZADD (alice not added)
|
|
112
|
+
@u.metrics.member?('alice')
|
|
113
|
+
#=> false
|
|
114
|
+
|
|
115
|
+
@u.metrics.delete!
|
|
72
116
|
@a.metrics.delete!
|
|
@@ -314,19 +314,19 @@ class TestModelWithInit < Familia::Horreum
|
|
|
314
314
|
end
|
|
315
315
|
|
|
316
316
|
# Create object - init should set region based on user_id
|
|
317
|
-
init_obj = TestModelWithInit.new(user_id: "
|
|
317
|
+
init_obj = TestModelWithInit.new(user_id: "ca-west-123")
|
|
318
318
|
init_obj.save
|
|
319
319
|
init_obj.activities << "login"
|
|
320
320
|
init_obj.activities << "purchase"
|
|
321
321
|
|
|
322
322
|
# Verify init worked and region is set
|
|
323
|
-
region_set_correctly = init_obj.region == "
|
|
323
|
+
region_set_correctly = init_obj.region == "ca"
|
|
324
324
|
|
|
325
325
|
# Verify related field key includes region (would be nil without fix)
|
|
326
326
|
activities_key = init_obj.activities.dbkey
|
|
327
327
|
|
|
328
328
|
# Destroy using class method - temp instance init should execute with identifier
|
|
329
|
-
TestModelWithInit.destroy!("
|
|
329
|
+
TestModelWithInit.destroy!("ca-west-123")
|
|
330
330
|
|
|
331
331
|
# Verify all keys are cleaned up (including activities with correct key)
|
|
332
332
|
activities_cleaned = TestModelWithInit.dbclient.exists(activities_key).zero?
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
# try/unit/utils/future_aware_helpers_try.rb
|
|
2
|
+
#
|
|
3
|
+
# frozen_string_literal: true
|
|
4
|
+
|
|
5
|
+
# Direct unit tests for Familia.success? and Familia.positive? —
|
|
6
|
+
# the Future-aware utility methods added to Familia::Utils.
|
|
7
|
+
#
|
|
8
|
+
# These methods handle two cases:
|
|
9
|
+
# 1. Concrete Integer return values from Redis commands
|
|
10
|
+
# 2. Redis::Future objects inside pipelines/transactions (passthrough)
|
|
11
|
+
#
|
|
12
|
+
# Call sites include fast writers, DEL operations, EXISTS checks,
|
|
13
|
+
# LINSERT results, and TTL comparisons.
|
|
14
|
+
|
|
15
|
+
require_relative '../../support/helpers/test_helpers'
|
|
16
|
+
|
|
17
|
+
Familia.debug = false
|
|
18
|
+
|
|
19
|
+
@test_key = 'familia:test:future_aware_helpers'
|
|
20
|
+
|
|
21
|
+
##
|
|
22
|
+
## Familia.success? — concrete values
|
|
23
|
+
##
|
|
24
|
+
|
|
25
|
+
## success? returns true for positive integer (new key created)
|
|
26
|
+
Familia.success?(1)
|
|
27
|
+
#=> true
|
|
28
|
+
|
|
29
|
+
## success? returns true for zero (existing key updated)
|
|
30
|
+
Familia.success?(0)
|
|
31
|
+
#=> true
|
|
32
|
+
|
|
33
|
+
## success? returns false for negative integer
|
|
34
|
+
Familia.success?(-1)
|
|
35
|
+
#=> false
|
|
36
|
+
|
|
37
|
+
## success? returns false for large negative
|
|
38
|
+
Familia.success?(-100)
|
|
39
|
+
#=> false
|
|
40
|
+
|
|
41
|
+
##
|
|
42
|
+
## Familia.positive? — concrete values
|
|
43
|
+
##
|
|
44
|
+
|
|
45
|
+
## positive? returns true for positive integer
|
|
46
|
+
Familia.positive?(1)
|
|
47
|
+
#=> true
|
|
48
|
+
|
|
49
|
+
## positive? returns true for large positive integer
|
|
50
|
+
Familia.positive?(5)
|
|
51
|
+
#=> true
|
|
52
|
+
|
|
53
|
+
## positive? returns false for zero (nothing happened)
|
|
54
|
+
Familia.positive?(0)
|
|
55
|
+
#=> false
|
|
56
|
+
|
|
57
|
+
## positive? returns false for negative integer
|
|
58
|
+
Familia.positive?(-1)
|
|
59
|
+
#=> false
|
|
60
|
+
|
|
61
|
+
##
|
|
62
|
+
## Redis::Future passthrough — inside a pipeline
|
|
63
|
+
##
|
|
64
|
+
|
|
65
|
+
## success? returns the Future unchanged inside a pipeline
|
|
66
|
+
@success_future = nil
|
|
67
|
+
Familia.dbclient.pipelined do |pipe|
|
|
68
|
+
fut = pipe.hset(@test_key, 'field', 'val')
|
|
69
|
+
@success_future = Familia.success?(fut)
|
|
70
|
+
end
|
|
71
|
+
@success_future.is_a?(Redis::Future)
|
|
72
|
+
#=> true
|
|
73
|
+
|
|
74
|
+
## success? Future resolves to a truthy result after pipeline completes
|
|
75
|
+
@success_future.value.zero? || @success_future.value.positive?
|
|
76
|
+
#=> true
|
|
77
|
+
|
|
78
|
+
## positive? returns the Future unchanged inside a pipeline
|
|
79
|
+
@positive_future = nil
|
|
80
|
+
Familia.dbclient.pipelined do |pipe|
|
|
81
|
+
fut = pipe.exists(@test_key)
|
|
82
|
+
@positive_future = Familia.positive?(fut)
|
|
83
|
+
end
|
|
84
|
+
@positive_future.is_a?(Redis::Future)
|
|
85
|
+
#=> true
|
|
86
|
+
|
|
87
|
+
## positive? Future resolves to truthy for an existing key after pipeline
|
|
88
|
+
@positive_future.value.positive?
|
|
89
|
+
#=> true
|
|
90
|
+
|
|
91
|
+
##
|
|
92
|
+
## Edge cases — nil and non-numeric raise NoMethodError
|
|
93
|
+
##
|
|
94
|
+
|
|
95
|
+
## success? raises for nil input (no guard — explicit contract)
|
|
96
|
+
begin
|
|
97
|
+
Familia.success?(nil)
|
|
98
|
+
rescue NoMethodError => e
|
|
99
|
+
e.class
|
|
100
|
+
end
|
|
101
|
+
#=> NoMethodError
|
|
102
|
+
|
|
103
|
+
## positive? raises for nil input
|
|
104
|
+
begin
|
|
105
|
+
Familia.positive?(nil)
|
|
106
|
+
rescue NoMethodError => e
|
|
107
|
+
e.class
|
|
108
|
+
end
|
|
109
|
+
#=> NoMethodError
|
|
110
|
+
|
|
111
|
+
## success? raises for string input
|
|
112
|
+
begin
|
|
113
|
+
Familia.success?('ok')
|
|
114
|
+
rescue NoMethodError => e
|
|
115
|
+
e.class
|
|
116
|
+
end
|
|
117
|
+
#=> NoMethodError
|
|
118
|
+
|
|
119
|
+
## positive? raises for string input
|
|
120
|
+
begin
|
|
121
|
+
Familia.positive?('ok')
|
|
122
|
+
rescue NoMethodError => e
|
|
123
|
+
e.class
|
|
124
|
+
end
|
|
125
|
+
#=> NoMethodError
|
|
126
|
+
|
|
127
|
+
# Teardown
|
|
128
|
+
Familia.dbclient.del(@test_key)
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: familia
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 2.9.
|
|
4
|
+
version: 2.9.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Delano Mandelbaum
|
|
@@ -177,9 +177,9 @@ files:
|
|
|
177
177
|
- changelog.d/README.md
|
|
178
178
|
- changelog.d/scriv.ini
|
|
179
179
|
- docs/1106-participates_in-bidirectional-solution.md
|
|
180
|
-
- docs/archive/.gitignore
|
|
181
180
|
- docs/conf.py
|
|
182
181
|
- docs/guides/.gitignore
|
|
182
|
+
- docs/guides/datatype-collections.md
|
|
183
183
|
- docs/guides/encryption.md
|
|
184
184
|
- docs/guides/feature-encrypted-fields.md
|
|
185
185
|
- docs/guides/feature-expiration.md
|
|
@@ -366,6 +366,7 @@ files:
|
|
|
366
366
|
- try/audit/repair_score_correctness_try.rb
|
|
367
367
|
- try/audit/scan_keys_try.rb
|
|
368
368
|
- try/edge_cases/empty_identifiers_try.rb
|
|
369
|
+
- try/edge_cases/fast_writer_pipeline_support_try.rb
|
|
369
370
|
- try/edge_cases/fast_writer_transaction_guard_try.rb
|
|
370
371
|
- try/edge_cases/find_by_dbkey_race_condition_try.rb
|
|
371
372
|
- try/edge_cases/hash_symbolization_try.rb
|
|
@@ -656,6 +657,7 @@ files:
|
|
|
656
657
|
- try/unit/refinements/time_literals_numeric_methods_try.rb
|
|
657
658
|
- try/unit/refinements/time_literals_string_methods_try.rb
|
|
658
659
|
- try/unit/thread_safety_monitor_try.rb
|
|
660
|
+
- try/unit/utils/future_aware_helpers_try.rb
|
|
659
661
|
- try/valkey.conf
|
|
660
662
|
homepage: https://github.com/delano/familia
|
|
661
663
|
licenses:
|
data/docs/archive/.gitignore
DELETED