familia 2.10.0 → 2.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.rst +69 -0
- data/Gemfile.lock +45 -42
- data/changelog.d/20260605_220911_anthropic_sleepy-allen.rst +35 -0
- data/docs/guides/datatype-collections.md +30 -2
- data/docs/guides/feature-migrations.md +45 -0
- data/docs/guides/feature-relationships-indexing.md +10 -2
- data/docs/guides/feature-relationships-methods.md +41 -0
- data/docs/guides/feature-relationships-participation.md +43 -2
- data/docs/guides/feature-relationships.md +162 -1
- data/docs/migrating/{v2.10.0.md → v2.10.md} +128 -1
- data/lib/familia/connection/operations.rb +194 -0
- data/lib/familia/connection/transaction_core.rb +51 -0
- data/lib/familia/data_type/collection_base.rb +22 -7
- data/lib/familia/data_type/serialization.rb +21 -4
- data/lib/familia/data_type.rb +1 -1
- data/lib/familia/features/relationships/collection_operations.rb +23 -2
- data/lib/familia/features/relationships/indexing/multi_index_generators.rb +8 -9
- data/lib/familia/features/relationships/indexing/unique_index_generators.rb +5 -3
- data/lib/familia/features/relationships/participation/target_methods.rb +23 -5
- data/lib/familia/features/relationships/participation.rb +6 -1
- data/lib/familia/horreum/atomic_write.rb +24 -34
- data/lib/familia/horreum/management.rb +9 -7
- data/lib/familia/horreum/persistence.rb +54 -38
- data/lib/familia/index_descriptor.rb +258 -0
- data/lib/familia/migration/base.rb +1 -1
- data/lib/familia/migration/errors.rb +2 -0
- data/lib/familia/migration/model.rb +1 -1
- data/lib/familia/migration/pipeline.rb +1 -1
- data/lib/familia/migration/rake_tasks.rb +11 -17
- data/lib/familia/migration/registry.rb +4 -0
- data/lib/familia/migration/runner.rb +2 -0
- data/lib/familia/migration/script.rb +2 -0
- data/lib/familia/migration.rb +2 -0
- data/lib/familia/utils.rb +15 -0
- data/lib/familia/version.rb +1 -1
- data/lib/familia.rb +1 -0
- data/try/features/atomic_write_watch_try.rb +108 -18
- data/try/features/cross_model_atomic_write_try.rb +402 -0
- data/try/features/relationships/index_introspection_try.rb +304 -0
- data/try/features/relationships/multi_index_each_record_try.rb +211 -0
- data/try/features/relationships/participation_each_record_try.rb +247 -0
- data/try/features/relationships/participation_reverse_methods_try.rb +4 -2
- data/try/investigation/cross_model_atomic_poc_try.rb +130 -0
- data/try/unit/data_types/each_record_try.rb +1 -1
- metadata +10 -3
- data/docs/guides/writing-migrations.md +0 -345
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7903e14486c85385ad0b682c009b1f7d182e0cff24b107aa0264692529bb8dfd
|
|
4
|
+
data.tar.gz: 445219dfcd2df1cf2054b07d90e33902fc21532279730bc274629a2082a02f7f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 635d35b86d7c6a85332517e3b238b8b932d4823b0cfad2f58b7c349a530de44ae02b91930c9ddac7ed538c619cea3a6f8355eeaba20ebf52358ebd45f4547067
|
|
7
|
+
data.tar.gz: d96538d6fbb486fcd92bc329acd52cba07008e40db5df059f338e060cb6dbb4789c8fe0e8e52ee69891662b1b2353eec3832032aa3ccda50848a986d06ea6a4d
|
data/CHANGELOG.rst
CHANGED
|
@@ -7,6 +7,75 @@ The format is based on `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`
|
|
|
7
7
|
|
|
8
8
|
<!--scriv-insert-here-->
|
|
9
9
|
|
|
10
|
+
.. _changelog-2.10.1:
|
|
11
|
+
|
|
12
|
+
2.10.1 — 2026-06-06
|
|
13
|
+
===================
|
|
14
|
+
|
|
15
|
+
Added
|
|
16
|
+
-----
|
|
17
|
+
|
|
18
|
+
- ``record_class:`` option for collection DataTypes (``list``/``set``/
|
|
19
|
+
``sorted_set``/``hashkey``). This loading-only hint tells ``each_record`` which
|
|
20
|
+
class to hydrate via ``load_multi`` without changing how the collection
|
|
21
|
+
serializes or deserializes reads. Use this when you want ``each_record`` lookup
|
|
22
|
+
behavior but no changes to read behavior. Issue #297
|
|
23
|
+
|
|
24
|
+
- ``Familia.atomic_write(*instances)`` persists multiple Horreum instances in a
|
|
25
|
+
single ``MULTI/EXEC``. Includes an optional ``watch_keys:``/``pre_check:``
|
|
26
|
+
variant for race-safe, write-once semantics. All participating instances must
|
|
27
|
+
resolve to the same logical database (raising ``Familia::CrossDatabaseError``
|
|
28
|
+
otherwise) and must share a hash slot on Redis Cluster. #296
|
|
29
|
+
|
|
30
|
+
Changed
|
|
31
|
+
-------
|
|
32
|
+
|
|
33
|
+
- ``participates_in`` / ``class_participates_in`` collections now default to
|
|
34
|
+
using ``record_class:``. This change requires **no data migration and causes no
|
|
35
|
+
behavior changes**: existing collections already stored raw identifiers, and
|
|
36
|
+
read operations (``members``, ``to_a``, ``member?``, ``score``) behave exactly
|
|
37
|
+
as before. The only difference is that ``each_record`` is now supported. Pre-
|
|
38
|
+
declared collections are left untouched. Issue #297
|
|
39
|
+
|
|
40
|
+
Fixed
|
|
41
|
+
-----
|
|
42
|
+
|
|
43
|
+
- Enabled ``each_record`` on ``participates_in`` and ``class_participates_in``
|
|
44
|
+
collections by automatically declaring them with ``record_class: <participant
|
|
45
|
+
class>``. This resolves ``Familia::Problem`` exceptions and loads participant
|
|
46
|
+
records via ``load_multi`` across all collection types. Issue #297
|
|
47
|
+
|
|
48
|
+
- Suppressed per-member ``[deserialize] Raw fallback`` warning storm when
|
|
49
|
+
iterating ``record_class`` collections with non-JSON identifiers (such as UUIDs
|
|
50
|
+
or prefixed IDs). These expected raw values are now logged at the debug level
|
|
51
|
+
instead of warnings. Issue #297
|
|
52
|
+
|
|
53
|
+
- Resolved a connection-pooling bug where the ``WATCH``-based optimistic lock
|
|
54
|
+
in ``atomic_write(watch_keys:)``, ``save_if_not_exists!``, and ``create!`` was
|
|
55
|
+
silent/inert. The ``WATCH`` and ``MULTI/EXEC`` commands are now driven through
|
|
56
|
+
the same connection, ensuring concurrent modifications correctly abort and raise
|
|
57
|
+
as
|
|
58
|
+
documented. #296
|
|
59
|
+
|
|
60
|
+
AI Assistance
|
|
61
|
+
-------------
|
|
62
|
+
|
|
63
|
+
- AI diagnosed the participation iteration bug and identified that ``reference: true``
|
|
64
|
+
introduced unintended read-behavior changes. Designed and implemented the
|
|
65
|
+
``record_class:`` option to decouple ``each_record`` lookup from read deserialization,
|
|
66
|
+
suppressed a resulting per-member deserialize warning storm, kept intentional
|
|
67
|
+
raw-string semantics on ``instances`` and ``unique_index``, updated stale
|
|
68
|
+
flowcharts in ``datatype-collections.md``, and added regression coverage. Issue #297
|
|
69
|
+
|
|
70
|
+
- Root-caused and fixed a split-connection defect with Claude Code: implemented a
|
|
71
|
+
single-connection ``execute_watched_transaction`` primitive (avoiding fiber-pinning
|
|
72
|
+
that degrades atomic commands) and added real concurrent-modification tests to
|
|
73
|
+
replace simulated aborts. #296
|
|
74
|
+
|
|
75
|
+
- Designed and built multi-model atomic writes on top of the new ``WATCH`` primitive:
|
|
76
|
+
implemented the same-database guard, orchestration logic, and a test suite covering
|
|
77
|
+
two-model commits, rollback on error, cross-database rejection, and race conditions. #296
|
|
78
|
+
|
|
10
79
|
.. _changelog-2.10.0:
|
|
11
80
|
|
|
12
81
|
2.10.0 — 2026-06-04
|
data/Gemfile.lock
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
familia (2.10.
|
|
4
|
+
familia (2.10.1)
|
|
5
5
|
concurrent-ruby (~> 1.3)
|
|
6
6
|
connection_pool (>= 2.4, < 4.0)
|
|
7
7
|
csv (~> 3.3)
|
|
@@ -15,59 +15,59 @@ PATH
|
|
|
15
15
|
GEM
|
|
16
16
|
remote: https://rubygems.org/
|
|
17
17
|
specs:
|
|
18
|
-
addressable (2.
|
|
18
|
+
addressable (2.9.0)
|
|
19
19
|
public_suffix (>= 2.0.2, < 8.0)
|
|
20
20
|
ast (2.4.3)
|
|
21
21
|
base64 (0.3.0)
|
|
22
22
|
benchmark (0.5.0)
|
|
23
|
-
bigdecimal (
|
|
23
|
+
bigdecimal (4.1.2)
|
|
24
24
|
concurrent-ruby (1.3.6)
|
|
25
25
|
connection_pool (3.0.2)
|
|
26
26
|
csv (3.3.5)
|
|
27
|
-
date (3.5.
|
|
28
|
-
debug (1.11.
|
|
27
|
+
date (3.5.1)
|
|
28
|
+
debug (1.11.1)
|
|
29
29
|
irb (~> 1.10)
|
|
30
30
|
reline (>= 0.3.8)
|
|
31
31
|
diff-lcs (1.6.2)
|
|
32
|
-
dry-configurable (1.
|
|
33
|
-
dry-core (~> 1.
|
|
32
|
+
dry-configurable (1.4.0)
|
|
33
|
+
dry-core (~> 1.0)
|
|
34
34
|
zeitwerk (~> 2.6)
|
|
35
|
-
dry-core (1.
|
|
35
|
+
dry-core (1.2.0)
|
|
36
36
|
concurrent-ruby (~> 1.0)
|
|
37
37
|
logger
|
|
38
38
|
zeitwerk (~> 2.6)
|
|
39
|
-
dry-inflector (1.
|
|
39
|
+
dry-inflector (1.3.1)
|
|
40
40
|
dry-initializer (3.2.0)
|
|
41
41
|
dry-logic (1.6.0)
|
|
42
42
|
bigdecimal
|
|
43
43
|
concurrent-ruby (~> 1.0)
|
|
44
44
|
dry-core (~> 1.1)
|
|
45
45
|
zeitwerk (~> 2.6)
|
|
46
|
-
dry-schema (1.
|
|
46
|
+
dry-schema (1.16.0)
|
|
47
47
|
concurrent-ruby (~> 1.0)
|
|
48
48
|
dry-configurable (~> 1.0, >= 1.0.1)
|
|
49
49
|
dry-core (~> 1.1)
|
|
50
50
|
dry-initializer (~> 3.2)
|
|
51
|
-
dry-logic (~> 1.
|
|
52
|
-
dry-types (~> 1.
|
|
51
|
+
dry-logic (~> 1.6)
|
|
52
|
+
dry-types (~> 1.9, >= 1.9.1)
|
|
53
53
|
zeitwerk (~> 2.6)
|
|
54
|
-
dry-types (1.
|
|
55
|
-
bigdecimal (
|
|
54
|
+
dry-types (1.9.1)
|
|
55
|
+
bigdecimal (>= 3.0)
|
|
56
56
|
concurrent-ruby (~> 1.0)
|
|
57
57
|
dry-core (~> 1.0)
|
|
58
58
|
dry-inflector (~> 1.0)
|
|
59
59
|
dry-logic (~> 1.4)
|
|
60
60
|
zeitwerk (~> 2.6)
|
|
61
|
-
erb (
|
|
62
|
-
ffi (1.17.
|
|
63
|
-
ffi (1.17.
|
|
61
|
+
erb (6.0.4)
|
|
62
|
+
ffi (1.17.4)
|
|
63
|
+
ffi (1.17.4-arm64-darwin)
|
|
64
64
|
hana (1.3.7)
|
|
65
|
-
io-console (0.8.
|
|
65
|
+
io-console (0.8.2)
|
|
66
66
|
irb (1.15.3)
|
|
67
67
|
pp (>= 0.6.0)
|
|
68
68
|
rdoc (>= 4.0.0)
|
|
69
69
|
reline (>= 0.4.2)
|
|
70
|
-
json (2.
|
|
70
|
+
json (2.19.8)
|
|
71
71
|
json-schema (6.2.0)
|
|
72
72
|
addressable (~> 2.8)
|
|
73
73
|
bigdecimal (>= 3.1, < 5)
|
|
@@ -79,15 +79,15 @@ GEM
|
|
|
79
79
|
language_server-protocol (3.17.0.5)
|
|
80
80
|
lint_roller (1.1.0)
|
|
81
81
|
logger (1.7.0)
|
|
82
|
-
mcp (0.
|
|
82
|
+
mcp (0.18.0)
|
|
83
83
|
json-schema (>= 4.1)
|
|
84
84
|
minitest (5.27.0)
|
|
85
|
-
oj (3.
|
|
85
|
+
oj (3.17.3)
|
|
86
86
|
bigdecimal (>= 3.0)
|
|
87
87
|
ostruct (>= 0.2)
|
|
88
88
|
ostruct (0.6.3)
|
|
89
|
-
parallel (1.
|
|
90
|
-
parser (3.3.
|
|
89
|
+
parallel (1.28.0)
|
|
90
|
+
parser (3.3.11.1)
|
|
91
91
|
ast (~> 2.4.1)
|
|
92
92
|
racc
|
|
93
93
|
pastel (0.8.0)
|
|
@@ -96,25 +96,27 @@ GEM
|
|
|
96
96
|
prettyprint
|
|
97
97
|
prettyprint (0.2.0)
|
|
98
98
|
prism (1.9.0)
|
|
99
|
-
psych (5.
|
|
99
|
+
psych (5.4.0)
|
|
100
100
|
date
|
|
101
101
|
stringio
|
|
102
102
|
public_suffix (7.0.5)
|
|
103
103
|
racc (1.8.1)
|
|
104
104
|
rainbow (3.1.1)
|
|
105
|
-
rake (13.
|
|
105
|
+
rake (13.4.2)
|
|
106
106
|
rbnacl (7.1.2)
|
|
107
107
|
ffi (~> 1)
|
|
108
|
-
rbs (
|
|
108
|
+
rbs (4.0.2)
|
|
109
109
|
logger
|
|
110
|
-
|
|
110
|
+
prism (>= 1.6.0)
|
|
111
|
+
tsort
|
|
112
|
+
rdoc (7.2.0)
|
|
111
113
|
erb
|
|
112
114
|
psych (>= 4.0.0)
|
|
113
115
|
tsort
|
|
114
116
|
redcarpet (3.6.1)
|
|
115
117
|
redis (5.4.1)
|
|
116
118
|
redis-client (>= 0.22.0)
|
|
117
|
-
redis-client (0.
|
|
119
|
+
redis-client (0.29.0)
|
|
118
120
|
connection_pool
|
|
119
121
|
reek (6.5.0)
|
|
120
122
|
dry-schema (~> 1.13)
|
|
@@ -122,10 +124,10 @@ GEM
|
|
|
122
124
|
parser (~> 3.3.0)
|
|
123
125
|
rainbow (>= 2.0, < 4.0)
|
|
124
126
|
rexml (~> 3.1)
|
|
125
|
-
regexp_parser (2.
|
|
126
|
-
reline (0.6.
|
|
127
|
+
regexp_parser (2.12.0)
|
|
128
|
+
reline (0.6.3)
|
|
127
129
|
io-console (~> 0.5)
|
|
128
|
-
rexml (3.4.
|
|
130
|
+
rexml (3.4.4)
|
|
129
131
|
rspec (3.13.2)
|
|
130
132
|
rspec-core (~> 3.13.0)
|
|
131
133
|
rspec-expectations (~> 3.13.0)
|
|
@@ -135,10 +137,10 @@ GEM
|
|
|
135
137
|
rspec-expectations (3.13.5)
|
|
136
138
|
diff-lcs (>= 1.2.0, < 2.0)
|
|
137
139
|
rspec-support (~> 3.13.0)
|
|
138
|
-
rspec-mocks (3.13.
|
|
140
|
+
rspec-mocks (3.13.8)
|
|
139
141
|
diff-lcs (>= 1.2.0, < 2.0)
|
|
140
142
|
rspec-support (~> 3.13.0)
|
|
141
|
-
rspec-support (3.13.
|
|
143
|
+
rspec-support (3.13.7)
|
|
142
144
|
rubocop (1.85.1)
|
|
143
145
|
json (~> 2.3)
|
|
144
146
|
language_server-protocol (~> 3.17.0.2)
|
|
@@ -151,28 +153,29 @@ GEM
|
|
|
151
153
|
rubocop-ast (>= 1.49.0, < 2.0)
|
|
152
154
|
ruby-progressbar (~> 1.7)
|
|
153
155
|
unicode-display_width (>= 2.4.0, < 4.0)
|
|
154
|
-
rubocop-ast (1.49.
|
|
156
|
+
rubocop-ast (1.49.1)
|
|
155
157
|
parser (>= 3.3.7.2)
|
|
156
158
|
prism (~> 1.7)
|
|
157
|
-
rubocop-performance (1.
|
|
159
|
+
rubocop-performance (1.26.1)
|
|
158
160
|
lint_roller (~> 1.1)
|
|
159
161
|
rubocop (>= 1.75.0, < 2.0)
|
|
160
|
-
rubocop-ast (>= 1.
|
|
162
|
+
rubocop-ast (>= 1.47.1, < 2.0)
|
|
161
163
|
rubocop-thread_safety (0.7.3)
|
|
162
164
|
lint_roller (~> 1.1)
|
|
163
165
|
rubocop (~> 1.72, >= 1.72.1)
|
|
164
166
|
rubocop-ast (>= 1.44.0, < 2.0)
|
|
165
|
-
ruby-lsp (0.26.
|
|
167
|
+
ruby-lsp (0.26.9)
|
|
166
168
|
language_server-protocol (~> 3.17.0)
|
|
167
169
|
prism (>= 1.2, < 2.0)
|
|
168
170
|
rbs (>= 3, < 5)
|
|
169
|
-
ruby-prof (
|
|
171
|
+
ruby-prof (2.0.4)
|
|
170
172
|
base64
|
|
173
|
+
ostruct
|
|
171
174
|
ruby-progressbar (1.13.0)
|
|
172
175
|
simpleidn (0.2.3)
|
|
173
|
-
stackprof (0.2.
|
|
176
|
+
stackprof (0.2.28)
|
|
174
177
|
stringio (3.1.9)
|
|
175
|
-
timecop (0.9.
|
|
178
|
+
timecop (0.9.11)
|
|
176
179
|
tryouts (3.7.1)
|
|
177
180
|
concurrent-ruby (~> 1.0, < 2)
|
|
178
181
|
irb
|
|
@@ -190,8 +193,8 @@ GEM
|
|
|
190
193
|
unicode-emoji (~> 4.1)
|
|
191
194
|
unicode-emoji (4.2.0)
|
|
192
195
|
uri-valkey (1.4.0)
|
|
193
|
-
yard (0.9.
|
|
194
|
-
zeitwerk (2.
|
|
196
|
+
yard (0.9.44)
|
|
197
|
+
zeitwerk (2.8.2)
|
|
195
198
|
|
|
196
199
|
PLATFORMS
|
|
197
200
|
arm64-darwin-24
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
Added
|
|
2
|
+
-----
|
|
3
|
+
|
|
4
|
+
- Project-wide relationship introspection: ``Familia.index_descriptors``,
|
|
5
|
+
``Familia.unique_indexes``, ``Familia.multi_indexes``, and
|
|
6
|
+
``Familia.participation_descriptors`` aggregate index/participation metadata
|
|
7
|
+
across every loaded ``Horreum`` subclass, returning ``Familia::IndexDescriptor``
|
|
8
|
+
objects (``coordinate``, ``each_record``, ``rebuild!``, ``stale_format?``) that
|
|
9
|
+
act without the caller knowing index method-naming or storage layout.
|
|
10
|
+
|
|
11
|
+
- Stale unique-index boot guard: ``Familia.stale_indexes`` and
|
|
12
|
+
``Familia.assert_indexes_current!`` detect class-level unique indexes still
|
|
13
|
+
holding pre-2.10.0 JSON-encoded identifiers and fail fast (or warn) before an
|
|
14
|
+
un-rebuilt index silently breaks a ``find_by_*`` lookup. Rebuild them with
|
|
15
|
+
``Familia.stale_indexes.each(&:rebuild!)``.
|
|
16
|
+
|
|
17
|
+
- ``Familia.legacy_json_encoded?`` exposes the legacy-format predicate shared by
|
|
18
|
+
the read path (``strip_legacy_json_encoding``) and the introspection layer, so
|
|
19
|
+
detection and stripping never disagree.
|
|
20
|
+
|
|
21
|
+
Documentation
|
|
22
|
+
-------------
|
|
23
|
+
|
|
24
|
+
- Documented the relationship introspection API — per-class
|
|
25
|
+
(``indexing_relationships``/``participation_relationships``), project-wide, and
|
|
26
|
+
per-instance — plus the stale-index boot guard, across the relationships guide
|
|
27
|
+
and methods reference. Renamed ``docs/migrating/v2.10.0.md`` to
|
|
28
|
+
``docs/migrating/v2.10.md`` and noted that the introspection helpers require
|
|
29
|
+
2.10.1+.
|
|
30
|
+
|
|
31
|
+
AI Assistance
|
|
32
|
+
-------------
|
|
33
|
+
|
|
34
|
+
- The introspection helpers, the stale-index boot guard, their tryouts, and the
|
|
35
|
+
accompanying documentation were drafted with AI assistance.
|
|
@@ -73,7 +73,7 @@ flowchart TD
|
|
|
73
73
|
ER --> Validate{"pipeline <= batch_size?"}
|
|
74
74
|
Validate -- no --> Raise["raise ArgumentError"]
|
|
75
75
|
Validate -- yes --> CallEach["each(**filters) do |member|"]
|
|
76
|
-
CallEach --> Extract["id = member.is_a?(Array) ? member.
|
|
76
|
+
CallEach --> Extract["id = member.is_a?(Array) ? member.last : member"]
|
|
77
77
|
Extract --> Buffer["buffer << id"]
|
|
78
78
|
Buffer --> Full{"buffer.size >= batch_size?"}
|
|
79
79
|
Full -- no --> CallEach
|
|
@@ -114,13 +114,41 @@ SortedSet#each exhausted
|
|
|
114
114
|
|---|---|---|
|
|
115
115
|
| Yields | raw identifier (or `[field, value]` for `HashKey`) | loaded Horreum instance |
|
|
116
116
|
| Redis ops per yield | 0 extra (already paged) | amortized `HGETALL` via `load_multi` batch |
|
|
117
|
-
| Requires `
|
|
117
|
+
| Requires `record_class:` (or `class: + reference: true`) | no | yes (raises `Familia::Problem` otherwise) |
|
|
118
118
|
| Ghost handling | yields the dangling id | `compact` drops them silently |
|
|
119
119
|
| Write pipelining | not built-in | `pipeline:` groups block-body writes into `pipelined` blocks |
|
|
120
120
|
| Filters | type-specific (`since:`, `matching:`, …) | forwarded to underlying `each` |
|
|
121
121
|
|
|
122
122
|
So `each_record` is a thin orchestration layer: it leans on the type's own `each` for read pagination, then layers (1) batched record hydration and (2) optional write pipelining on top.
|
|
123
123
|
|
|
124
|
+
### Which collections support `each_record`?
|
|
125
|
+
|
|
126
|
+
`each_record` needs to know which class to hydrate. Two options supply it, and
|
|
127
|
+
the collections Familia generates for you already set one, so `each_record`
|
|
128
|
+
works on them out of the box:
|
|
129
|
+
|
|
130
|
+
- `ModelClass.instances` — the per-class timeline. Uses `class: + reference: true`.
|
|
131
|
+
- `unique_index` / `multi_index` lookups — the index hashkey/set points at the indexed class. Uses `class: + reference: true`.
|
|
132
|
+
- `participates_in` / `class_participates_in` collections — point at the participant class. Use `record_class:`.
|
|
133
|
+
|
|
134
|
+
The two options differ in scope:
|
|
135
|
+
|
|
136
|
+
- **`record_class: SomeClass`** — a *loading-only* hint. It enables `each_record`
|
|
137
|
+
but does **not** change how the collection serializes/deserializes reads
|
|
138
|
+
(`members`/`member?`/`score` keep the generic DataType semantics). Use this when
|
|
139
|
+
you want `each_record` without any read-behavior change. This is what
|
|
140
|
+
`participates_in` uses.
|
|
141
|
+
- **`class: SomeClass, reference: true`** — a full *reference type*. It enables
|
|
142
|
+
`each_record` **and** makes reads return raw-string identifiers (and `member?`
|
|
143
|
+
match raw strings). Use this when you also want raw-string read semantics. This
|
|
144
|
+
is what `instances` and the indexes use.
|
|
145
|
+
|
|
146
|
+
A collection you declare by hand (`sorted_set :foo`, `set :bar`, …) sets neither,
|
|
147
|
+
so calling `each_record` on it raises `Familia::Problem`. Add `record_class:` (or
|
|
148
|
+
`class: + reference: true`) to opt in. Note that if you pre-declare a collection
|
|
149
|
+
that `participates_in` would otherwise auto-create, your hand-declared options
|
|
150
|
+
win — add `record_class:` yourself if you want `each_record` on it.
|
|
151
|
+
|
|
124
152
|
## Choosing a `pipeline` mode
|
|
125
153
|
|
|
126
154
|
`each_record` has two dispatch modes, controlled by `pipeline:`. The parameter answers a single question: **may the dispatch loop wrap your block in a `pipelined { }`?**
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Migrations
|
|
2
|
+
|
|
3
|
+
Redis has no DDL, so Familia migrations operate on live data — renaming keys, reshaping hash fields, backfilling values, adjusting TTLs. The migration system provides idempotent execution, dry-run support, dependency ordering, and a Redis-backed registry that tracks what has run.
|
|
4
|
+
|
|
5
|
+
## Architecture
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
Base ← Model (SCAN + per-record callback)
|
|
9
|
+
← Pipeline (SCAN + batched Redis pipelining)
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
**Base** gives you raw `redis` access for key-level operations. **Model** iterates Horreum objects through `process_record(obj, key)`. **Pipeline** adds `should_process?` / `build_update_fields` for bulk HSET-style updates through Redis pipelines.
|
|
13
|
+
|
|
14
|
+
All three share the same lifecycle: `prepare` → `migration_needed?` → `migrate` (or `process_record` / pipeline dispatch) → optional `down` for rollback.
|
|
15
|
+
|
|
16
|
+
## Registry
|
|
17
|
+
|
|
18
|
+
Applied state lives in Redis, not in files or a relational table:
|
|
19
|
+
|
|
20
|
+
| Key | Type | Content |
|
|
21
|
+
|-----|------|---------|
|
|
22
|
+
| `{prefix}:applied` | Sorted Set | migration_id scored by timestamp |
|
|
23
|
+
| `{prefix}:metadata` | Hash | migration_id → JSON (duration, keys scanned/modified, reversibility) |
|
|
24
|
+
| `{prefix}:schema` | Hash | model_name → SHA256 digest of fields+types |
|
|
25
|
+
| `{prefix}:backup:{id}` | Hash (TTL) | field-level rollback data |
|
|
26
|
+
|
|
27
|
+
Default prefix: `familia:migrations`. The registry answers `applied?`, `pending`, `status`, `schema_changed?`, and `schema_drift` queries.
|
|
28
|
+
|
|
29
|
+
## Execution model
|
|
30
|
+
|
|
31
|
+
`Runner` resolves dependencies via topological sort (Kahn's algorithm), then runs pending migrations in order. Each migration is instantiated, prepared, checked with `migration_needed?`, and executed. The registry is updated only on non-dry-run success.
|
|
32
|
+
|
|
33
|
+
Rollback validates three preconditions: the migration is applied, no dependents are applied, and the migration implements `down`.
|
|
34
|
+
|
|
35
|
+
## Dry-run control
|
|
36
|
+
|
|
37
|
+
`for_realsies_this_time? { ... }` gates destructive operations. In dry-run mode the block is skipped and the registry is not updated. `dry_run_only? { ... }` provides the inverse gate for preview-only logging.
|
|
38
|
+
|
|
39
|
+
## Lua scripts
|
|
40
|
+
|
|
41
|
+
`Familia::Migration::Script` provides atomic Redis operations for common migration patterns: `rename_field`, `copy_field`, `delete_field`, `rename_key_preserve_ttl`, `backup_and_modify_field`. Custom scripts can be registered and executed through the same interface.
|
|
42
|
+
|
|
43
|
+
## Source files
|
|
44
|
+
|
|
45
|
+
`lib/familia/migration.rb` and `lib/familia/migration/`. Each file's first line states its purpose.
|
|
@@ -409,9 +409,12 @@ Index values (the object identifiers stored in hash keys and sets) are raw strin
|
|
|
409
409
|
### Debugging
|
|
410
410
|
|
|
411
411
|
```ruby
|
|
412
|
-
# Check configuration
|
|
412
|
+
# Check configuration — returns Array<IndexingRelationship> (Data objects),
|
|
413
|
+
# distinguished by .cardinality (:unique vs :multi)
|
|
413
414
|
User.indexing_relationships
|
|
414
|
-
# => [
|
|
415
|
+
# => [#<data IndexingRelationship field=:email, index_name=:email_lookup,
|
|
416
|
+
# cardinality=:unique, within=nil, ...>]
|
|
417
|
+
User.indexing_relationships.select { |r| r.cardinality == :unique }
|
|
415
418
|
|
|
416
419
|
# Inspect index contents
|
|
417
420
|
User.email_lookup.to_h
|
|
@@ -419,8 +422,13 @@ User.email_lookup.to_h
|
|
|
419
422
|
|
|
420
423
|
# Verify membership
|
|
421
424
|
employee.in_company_badge_index?(company) # => true/false
|
|
425
|
+
user.indexed_in?(:email_lookup) # => true/false (class-level indexes)
|
|
422
426
|
```
|
|
423
427
|
|
|
428
|
+
For the full introspection API — the `IndexingRelationship` fields, a
|
|
429
|
+
project-wide sweep over `Familia.members`, per-instance membership state, and
|
|
430
|
+
the audit/repair layer — see [Introspection](feature-relationships.md#introspection).
|
|
431
|
+
|
|
424
432
|
## See Also
|
|
425
433
|
|
|
426
434
|
- [**Relationships Overview**](feature-relationships.md) - Core concepts
|
|
@@ -228,6 +228,9 @@ domain.customer_count # => 2
|
|
|
228
228
|
customer.domains.size # Count
|
|
229
229
|
customer.domains.to_a # All IDs
|
|
230
230
|
customer.domains.range(0, 9) # First 10
|
|
231
|
+
|
|
232
|
+
# Iterate as loaded records (batched via load_multi, ghosts filtered)
|
|
233
|
+
customer.domains.each_record { |domain| domain.refresh! }
|
|
231
234
|
```
|
|
232
235
|
|
|
233
236
|
### Working with Indexes
|
|
@@ -251,6 +254,44 @@ employee.add_to_company_dept_index(company)
|
|
|
251
254
|
engineers = company.find_all_by_department('engineering')
|
|
252
255
|
```
|
|
253
256
|
|
|
257
|
+
## Introspection Methods
|
|
258
|
+
|
|
259
|
+
Unlike the methods above, these are always present on any class with
|
|
260
|
+
`feature :relationships` (and its instances) — they are not generated per
|
|
261
|
+
declaration. They report *what is declared* and *what an object currently
|
|
262
|
+
belongs to*.
|
|
263
|
+
|
|
264
|
+
### Class-Level (configuration)
|
|
265
|
+
|
|
266
|
+
| Method | Returns | Description |
|
|
267
|
+
|--------|---------|-------------|
|
|
268
|
+
| `indexing_relationships` | `Array<IndexingRelationship>` | All `unique_index` / `multi_index` declarations; `.cardinality` distinguishes `:unique` from `:multi` |
|
|
269
|
+
| `participation_relationships` | `Array<ParticipationRelationship>` | All `participates_in` / `class_participates_in` declarations |
|
|
270
|
+
|
|
271
|
+
### Instance-Level (current state)
|
|
272
|
+
|
|
273
|
+
| Method | Returns | Description |
|
|
274
|
+
|--------|---------|-------------|
|
|
275
|
+
| `current_indexings` | `Array<Hash>` | Indexes this object currently appears in |
|
|
276
|
+
| `indexed_in?(:index_name)` | Boolean | Whether the object is in the named class-level index |
|
|
277
|
+
| `current_participations` | `Array<Hash>` | Participation collections this object belongs to |
|
|
278
|
+
| `relationship_status` | Hash | `{ identifier:, current_participations:, index_memberships: }` |
|
|
279
|
+
|
|
280
|
+
### Project-Wide (every class)
|
|
281
|
+
|
|
282
|
+
| Method | Returns | Description |
|
|
283
|
+
|--------|---------|-------------|
|
|
284
|
+
| `Familia.index_descriptors(cardinality:, class_level:, owner:)` | `Array<IndexDescriptor>` | Every index across the clan, filterable |
|
|
285
|
+
| `Familia.unique_indexes` / `Familia.multi_indexes` | `Array<IndexDescriptor>` | Cardinality-filtered convenience helpers |
|
|
286
|
+
| `Familia.participation_descriptors(owner:)` | `Array<[Class, ParticipationRelationship]>` | Every participation, paired with its owner |
|
|
287
|
+
| `Familia.stale_indexes(sample:, owner:)` | `Array<IndexDescriptor>` | Class-level unique indexes holding pre-2.10.0 data |
|
|
288
|
+
| `Familia.assert_indexes_current!(on_stale:, owner:)` | Boolean | Boot guard: raise/warn if any index is stale |
|
|
289
|
+
|
|
290
|
+
Each `IndexDescriptor` exposes the relationship metadata plus `coordinate`,
|
|
291
|
+
`each_record(value:, scope:)`, `rebuild!(scope:)`, and `stale_format?`. For the
|
|
292
|
+
full breakdown and the audit/repair layer, see
|
|
293
|
+
[Introspection](feature-relationships.md#introspection).
|
|
294
|
+
|
|
254
295
|
## See Also
|
|
255
296
|
|
|
256
297
|
- [**Relationships Overview**](feature-relationships.md) - Core concepts and patterns
|
|
@@ -297,6 +297,42 @@ customer.domains.merge([id1, id2]) # Bulk ID operations
|
|
|
297
297
|
domain.customer_instances # Efficient bulk loading
|
|
298
298
|
```
|
|
299
299
|
|
|
300
|
+
### Iterating a Collection as Loaded Records
|
|
301
|
+
|
|
302
|
+
Participation collections carry a `record_class:` pointing at the participant
|
|
303
|
+
class, so you can iterate them with `each_record` — it batches the stored
|
|
304
|
+
identifiers through `load_multi` (pipelined `HGETALL`s), drops ghosts
|
|
305
|
+
(identifiers whose record has expired), and yields fully-loaded records.
|
|
306
|
+
(`record_class:` is a loading-only hint; it does not change how `members`,
|
|
307
|
+
`member?`, or `score` behave.)
|
|
308
|
+
|
|
309
|
+
```ruby
|
|
310
|
+
# Yields loaded Domain records, not raw identifiers
|
|
311
|
+
customer.domains.each_record { |domain| domain.refresh_dns! }
|
|
312
|
+
|
|
313
|
+
# Enumerator form composes with Enumerable
|
|
314
|
+
customer.domains.each_record.map(&:created_at)
|
|
315
|
+
|
|
316
|
+
# Filters forward to the underlying each (sorted sets accept since:/until:)
|
|
317
|
+
customer.domains.each_record(since: 1.day.ago) { |d| notify(d) }
|
|
318
|
+
|
|
319
|
+
# Class-level participation collections work too
|
|
320
|
+
User.all_users.each_record(pipeline: 50) { |u| u.last_seen_at! Familia.now }
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
This replaces the manual `load_multi` + `compact` workaround:
|
|
324
|
+
|
|
325
|
+
```ruby
|
|
326
|
+
# Before — manual batch load
|
|
327
|
+
Domain.load_multi(customer.domains.to_a).compact.each { |d| ... }
|
|
328
|
+
|
|
329
|
+
# After — each_record does the batching, ghost-filtering, and loading
|
|
330
|
+
customer.domains.each_record { |d| ... }
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
See [DataType Collections](datatype-collections.md) for the full `each` vs
|
|
334
|
+
`each_record` comparison and `pipeline:` tuning.
|
|
335
|
+
|
|
300
336
|
## Troubleshooting
|
|
301
337
|
|
|
302
338
|
### Common Issues
|
|
@@ -319,9 +355,10 @@ domain.customer_instances # Efficient bulk loading
|
|
|
319
355
|
### Debugging
|
|
320
356
|
|
|
321
357
|
```ruby
|
|
322
|
-
# Check configuration
|
|
358
|
+
# Check configuration — returns Array<ParticipationRelationship> (Data objects)
|
|
323
359
|
Domain.participation_relationships
|
|
324
|
-
# => [
|
|
360
|
+
# => [#<data ParticipationRelationship target_class=Customer,
|
|
361
|
+
# collection_name=:domains, type=:sorted_set, ...>]
|
|
325
362
|
|
|
326
363
|
# Inspect participations
|
|
327
364
|
domain.current_participations
|
|
@@ -330,6 +367,10 @@ domain.current_participations
|
|
|
330
367
|
domain.validate_relationships!
|
|
331
368
|
```
|
|
332
369
|
|
|
370
|
+
For the full introspection API — per-class metadata, a project-wide sweep over
|
|
371
|
+
`Familia.members`, per-instance membership state, and the audit/repair layer —
|
|
372
|
+
see [Introspection](feature-relationships.md#introspection).
|
|
373
|
+
|
|
333
374
|
## See Also
|
|
334
375
|
|
|
335
376
|
- [**Relationships Overview**](feature-relationships.md) - Core concepts
|