familia 2.0.0.pre26 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.rst +94 -0
- data/Gemfile +3 -0
- data/Gemfile.lock +12 -2
- data/README.md +1 -3
- data/docs/guides/feature-encrypted-fields.md +1 -1
- data/docs/guides/feature-expiration.md +1 -1
- data/docs/guides/feature-quantization.md +1 -1
- data/docs/guides/writing-migrations.md +345 -0
- data/docs/overview.md +7 -7
- data/docs/reference/api-technical.md +103 -7
- data/examples/migrations/v1_to_v2_serialization_migration.rb +374 -0
- data/examples/schemas/customer.json +33 -0
- data/examples/schemas/session.json +27 -0
- data/familia.gemspec +3 -2
- data/lib/familia/features/schema_validation.rb +139 -0
- data/lib/familia/migration/base.rb +447 -0
- data/lib/familia/migration/errors.rb +31 -0
- data/lib/familia/migration/model.rb +418 -0
- data/lib/familia/migration/pipeline.rb +226 -0
- data/lib/familia/migration/rake_tasks.rake +3 -0
- data/lib/familia/migration/rake_tasks.rb +160 -0
- data/lib/familia/migration/registry.rb +364 -0
- data/lib/familia/migration/runner.rb +311 -0
- data/lib/familia/migration/script.rb +234 -0
- data/lib/familia/migration.rb +43 -0
- data/lib/familia/schema_registry.rb +173 -0
- data/lib/familia/settings.rb +63 -1
- data/lib/familia/version.rb +1 -1
- data/lib/familia.rb +1 -0
- data/try/features/schema_registry_try.rb +193 -0
- data/try/features/schema_validation_feature_try.rb +218 -0
- data/try/migration/base_try.rb +226 -0
- data/try/migration/errors_try.rb +67 -0
- data/try/migration/integration_try.rb +451 -0
- data/try/migration/model_try.rb +431 -0
- data/try/migration/pipeline_try.rb +460 -0
- data/try/migration/rake_tasks_try.rb +61 -0
- data/try/migration/registry_try.rb +199 -0
- data/try/migration/runner_try.rb +311 -0
- data/try/migration/schema_validation_try.rb +201 -0
- data/try/migration/script_try.rb +192 -0
- data/try/migration/v1_to_v2_serialization_try.rb +513 -0
- data/try/performance/benchmarks_try.rb +11 -12
- metadata +45 -27
- data/docs/migrating/v2.0.0-pre.md +0 -84
- data/docs/migrating/v2.0.0-pre11.md +0 -253
- data/docs/migrating/v2.0.0-pre12.md +0 -306
- data/docs/migrating/v2.0.0-pre13.md +0 -95
- data/docs/migrating/v2.0.0-pre14.md +0 -37
- data/docs/migrating/v2.0.0-pre18.md +0 -58
- data/docs/migrating/v2.0.0-pre19.md +0 -197
- data/docs/migrating/v2.0.0-pre22.md +0 -241
- data/docs/migrating/v2.0.0-pre5.md +0 -131
- data/docs/migrating/v2.0.0-pre6.md +0 -154
- data/docs/migrating/v2.0.0-pre7.md +0 -222
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e83013bddd8ff985ad1cd9ef73421146b5fea6c35cbb77aa0cf12656dd270f68
|
|
4
|
+
data.tar.gz: ba7ebef4af806d823317fd41e1875733b12971b3c6321e841c9ecbf0c9d727a0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 126d79d60cad6c4ed4ed03a557f4435aec82d03ef2230ad64e4d2205f8c8856cc74b140c18fb5f1d567631f203f6b259e7d19d0bed8f36e09b5540fc004f4e6d
|
|
7
|
+
data.tar.gz: cb5a52b4622c9f193d3858b0f2766795f49000f56fccbffdc630dc5b05e68db0a8c200ad25137bf543d9fc64ee2f53cab891517a5d9f91c2dd8debeb2119b7e7
|
data/CHANGELOG.rst
CHANGED
|
@@ -7,6 +7,100 @@ The format is based on `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`
|
|
|
7
7
|
|
|
8
8
|
<!--scriv-insert-here-->
|
|
9
9
|
|
|
10
|
+
.. _changelog-2.1.0:
|
|
11
|
+
|
|
12
|
+
2.1.0 — 2026-02-01
|
|
13
|
+
==================
|
|
14
|
+
|
|
15
|
+
Added
|
|
16
|
+
-----
|
|
17
|
+
|
|
18
|
+
- Redis-native migration system with three patterns: Base (abstract foundation),
|
|
19
|
+
Model (record-by-record iteration via SCAN), and Pipeline (bulk updates with
|
|
20
|
+
Redis pipelining). Includes dependency resolution using topological sort,
|
|
21
|
+
dry-run mode, CLI support, and comprehensive Rake tasks.
|
|
22
|
+
|
|
23
|
+
- Migration registry for tracking applied migrations in Redis with rollback
|
|
24
|
+
support and schema drift detection.
|
|
25
|
+
|
|
26
|
+
- Lua script framework with atomic operations: rename_field, copy_field,
|
|
27
|
+
delete_field, rename_key_preserve_ttl, and backup_and_modify_field.
|
|
28
|
+
|
|
29
|
+
- Optional JSON Schema validation for Horreum models via ``feature :schema_validation``
|
|
30
|
+
with centralized SchemaRegistry supporting convention-based and explicit schema
|
|
31
|
+
discovery using the json_schemer gem.
|
|
32
|
+
|
|
33
|
+
- V1 to V2 serialization migration example at ``examples/migrations/v1_to_v2_serialization_migration.rb``
|
|
34
|
+
demonstrating how to upgrade Horreum objects from v1.x format (selective serialization
|
|
35
|
+
with type information loss) to v2.0 format (universal JSON encoding with type preservation).
|
|
36
|
+
Includes type detection heuristics, field type declarations, and batch processing.
|
|
37
|
+
|
|
38
|
+
Documentation
|
|
39
|
+
-------------
|
|
40
|
+
|
|
41
|
+
- Added comprehensive migration writing guide at ``docs/guides/writing-migrations.md``
|
|
42
|
+
covering all three migration patterns, CLI usage, dependencies, and best practices.
|
|
43
|
+
|
|
44
|
+
AI Assistance
|
|
45
|
+
-------------
|
|
46
|
+
|
|
47
|
+
- Claude Code assisted with test coverage analysis, identifying gaps in Model and
|
|
48
|
+
Pipeline test coverage. Implemented 67 new tests covering CLI entry points,
|
|
49
|
+
circular dependency detection, and comprehensive Model/Pipeline scenarios.
|
|
50
|
+
|
|
51
|
+
- Claude Code identified and fixed a bug where schema validation hooks were never
|
|
52
|
+
triggered in Model migrations, and optimized N+1 query patterns in Registry and
|
|
53
|
+
Runner classes.
|
|
54
|
+
|
|
55
|
+
.. _changelog-2.0.0:
|
|
56
|
+
|
|
57
|
+
2.0.0 — 2026-01-19
|
|
58
|
+
==================
|
|
59
|
+
|
|
60
|
+
Familia 2.0.0 represents a complete rewrite of the library with 26 pre-release
|
|
61
|
+
iterations incorporating community feedback and production testing.
|
|
62
|
+
|
|
63
|
+
Added
|
|
64
|
+
-----
|
|
65
|
+
|
|
66
|
+
- **Modular Feature System**: Autoloading features with ancestry chain traversal
|
|
67
|
+
(``feature :expiration``, ``feature :relationships``, etc.)
|
|
68
|
+
- **Unified Relationships API**: ``participates_in`` replaces ``tracked_in``/``member_of``
|
|
69
|
+
with bidirectional reverse lookups (``_instances`` suffix methods)
|
|
70
|
+
- **Type-Safe Serialization**: JSON encoding preserves Integer, Boolean, Float,
|
|
71
|
+
Hash, Array types across Redis boundary
|
|
72
|
+
- **Performance Optimizations**: Pipelined bulk loading (``load_multi``),
|
|
73
|
+
optional EXISTS check (``check_exists: false``), OJ JSON for 2-5× faster operations
|
|
74
|
+
- **Security Features**: VerifiableIdentifier with HMAC signatures,
|
|
75
|
+
ExternalIdentifier with format flexibility, encrypted fields with key rotation
|
|
76
|
+
- **Thread Safety**: Mutex initialization fixes, 56-test thread safety suite
|
|
77
|
+
- **Instrumentation**: ``Familia.on_command``, ``Familia.on_pipeline``,
|
|
78
|
+
``Familia.on_lifecycle`` hooks for monitoring
|
|
79
|
+
|
|
80
|
+
Changed
|
|
81
|
+
-------
|
|
82
|
+
|
|
83
|
+
- **BREAKING**: DataType class renaming to avoid Ruby namespace conflicts
|
|
84
|
+
(``Familia::String`` → ``Familia::StringKey``, etc.)
|
|
85
|
+
- **BREAKING**: Removed ``dump_method``/``load_method`` - JSON serialization is now standard
|
|
86
|
+
- **BREAKING**: Indexing API renamed (``class_indexed_by`` → ``unique_index``,
|
|
87
|
+
``indexed_by`` → ``multi_index``)
|
|
88
|
+
|
|
89
|
+
Documentation
|
|
90
|
+
-------------
|
|
91
|
+
|
|
92
|
+
- Archived 11 pre-release migration guides to ``docs/.archive/``
|
|
93
|
+
- Enhanced ``api-technical.md`` with bulk loading, EXISTS optimization,
|
|
94
|
+
per-class feature registration, and index rebuilding documentation
|
|
95
|
+
- Updated version references and fixed broken anchor links throughout docs
|
|
96
|
+
|
|
97
|
+
AI Assistance
|
|
98
|
+
-------------
|
|
99
|
+
|
|
100
|
+
- Claude Opus 4.5 coordinated 11 parallel code-explorer agents to evaluate
|
|
101
|
+
migration docs, identifying unique content to preserve before archiving.
|
|
102
|
+
Assisted with release statistics gathering and documentation consolidation.
|
|
103
|
+
|
|
10
104
|
.. _changelog-2.0.0.pre26:
|
|
11
105
|
|
|
12
106
|
2.0.0.pre26 — 2026-01-19
|
data/Gemfile
CHANGED
|
@@ -13,7 +13,10 @@ group :test do
|
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
group :development, :test do
|
|
16
|
+
gem 'benchmark', '~> 0.4', require: false
|
|
16
17
|
gem 'debug', require: false
|
|
18
|
+
gem 'json_schemer', '~> 2.0', require: false
|
|
19
|
+
gem 'rake', '~> 13.0', require: false
|
|
17
20
|
gem 'irb', '~> 1.15.2', require: false
|
|
18
21
|
gem 'redcarpet', require: false
|
|
19
22
|
gem 'reek', require: false
|
data/Gemfile.lock
CHANGED
|
@@ -1,8 +1,7 @@
|
|
|
1
1
|
PATH
|
|
2
2
|
remote: .
|
|
3
3
|
specs:
|
|
4
|
-
familia (2.
|
|
5
|
-
benchmark (~> 0.4)
|
|
4
|
+
familia (2.1.0)
|
|
6
5
|
concurrent-ruby (~> 1.3)
|
|
7
6
|
connection_pool (~> 2.5)
|
|
8
7
|
csv (~> 3.3)
|
|
@@ -59,12 +58,18 @@ GEM
|
|
|
59
58
|
erb (5.1.3)
|
|
60
59
|
ffi (1.17.2)
|
|
61
60
|
ffi (1.17.2-arm64-darwin)
|
|
61
|
+
hana (1.3.7)
|
|
62
62
|
io-console (0.8.1)
|
|
63
63
|
irb (1.15.3)
|
|
64
64
|
pp (>= 0.6.0)
|
|
65
65
|
rdoc (>= 4.0.0)
|
|
66
66
|
reline (>= 0.4.2)
|
|
67
67
|
json (2.15.1)
|
|
68
|
+
json_schemer (2.5.0)
|
|
69
|
+
bigdecimal
|
|
70
|
+
hana (~> 1.3)
|
|
71
|
+
regexp_parser (~> 2.0)
|
|
72
|
+
simpleidn (~> 0.2)
|
|
68
73
|
language_server-protocol (3.17.0.5)
|
|
69
74
|
lint_roller (1.1.0)
|
|
70
75
|
logger (1.7.0)
|
|
@@ -88,6 +93,7 @@ GEM
|
|
|
88
93
|
stringio
|
|
89
94
|
racc (1.8.1)
|
|
90
95
|
rainbow (3.1.1)
|
|
96
|
+
rake (13.3.1)
|
|
91
97
|
rbnacl (7.1.2)
|
|
92
98
|
ffi (~> 1)
|
|
93
99
|
rbs (3.9.5)
|
|
@@ -153,6 +159,7 @@ GEM
|
|
|
153
159
|
ruby-prof (1.7.2)
|
|
154
160
|
base64
|
|
155
161
|
ruby-progressbar (1.13.0)
|
|
162
|
+
simpleidn (0.2.3)
|
|
156
163
|
stackprof (0.2.27)
|
|
157
164
|
stringio (3.1.9)
|
|
158
165
|
timecop (0.9.10)
|
|
@@ -181,10 +188,13 @@ PLATFORMS
|
|
|
181
188
|
ruby
|
|
182
189
|
|
|
183
190
|
DEPENDENCIES
|
|
191
|
+
benchmark (~> 0.4)
|
|
184
192
|
concurrent-ruby (~> 1.3.5)
|
|
185
193
|
debug
|
|
186
194
|
familia!
|
|
187
195
|
irb (~> 1.15.2)
|
|
196
|
+
json_schemer (~> 2.0)
|
|
197
|
+
rake (~> 13.0)
|
|
188
198
|
rbnacl (~> 7.1, >= 7.1.1)
|
|
189
199
|
redcarpet
|
|
190
200
|
reek
|
data/README.md
CHANGED
|
@@ -4,8 +4,6 @@
|
|
|
4
4
|
|
|
5
5
|
Familia provides object-oriented access to Valkey/Redis using their database types. Unlike traditional ORMs that map objects to relational tables, Familia maps Ruby objects directly to Valkey's native data structures (strings, lists, sets, sorted sets, hashes) as instance variables.
|
|
6
6
|
|
|
7
|
-
> [!CAUTION]
|
|
8
|
-
> Familia 2 is in pre-release and not ready for production use. (October 2025)
|
|
9
7
|
## Traditional ORM vs Familia
|
|
10
8
|
|
|
11
9
|
**Traditional ORMs** convert your objects to SQL tables. A product with categories becomes two tables with a join table. Checking if a tag exists requires a query with joins.
|
|
@@ -140,7 +138,7 @@ user.tags? # Check if it's a Set type
|
|
|
140
138
|
|
|
141
139
|
## Prerequisites
|
|
142
140
|
|
|
143
|
-
- **Ruby**: 3.
|
|
141
|
+
- **Ruby**: 3.2+
|
|
144
142
|
- **Valkey/Redis**: 6.0+
|
|
145
143
|
- **Gems**: `redis` (automatically installed)
|
|
146
144
|
|
|
@@ -895,7 +895,7 @@ Familia::Encryption.validate_configuration!
|
|
|
895
895
|
## See Also
|
|
896
896
|
|
|
897
897
|
- **[Overview](../overview.md#encrypted-fields)** - Conceptual introduction to encrypted fields
|
|
898
|
-
- **[Technical Reference](../reference/api-technical.md#
|
|
898
|
+
- **[Technical Reference](../reference/api-technical.md#feature-system)** - Implementation details and advanced patterns
|
|
899
899
|
- **[Security Model Guide](security-model.md)** - Cryptographic design and threat model considerations
|
|
900
900
|
- **[Feature System Guide](feature-system.md)** - Understanding Familia's feature architecture
|
|
901
901
|
- **[Implementation Guide](implementation.md)** - Production deployment and configuration patterns
|
|
@@ -641,7 +641,7 @@ Get/set class-level default expiration.
|
|
|
641
641
|
|
|
642
642
|
## See Also
|
|
643
643
|
|
|
644
|
-
- **[Technical Reference](../reference/api-technical.md#
|
|
644
|
+
- **[Technical Reference](../reference/api-technical.md#feature-system)** - Implementation details and advanced patterns
|
|
645
645
|
- **[Overview](../overview.md#automatic-expiration)** - Conceptual introduction to expiration
|
|
646
646
|
- **[Feature System Guide](feature-system.md)** - Understanding Familia's feature architecture
|
|
647
647
|
- **[Implementation Guide](implementation.md)** - Production deployment and configuration patterns
|
|
@@ -891,7 +891,7 @@ BasicModel.qstamp() # Uses 10.minutes fallback (600 seconds)
|
|
|
891
891
|
|
|
892
892
|
## See Also
|
|
893
893
|
|
|
894
|
-
- **[Technical Reference](../reference/api-technical.md#
|
|
894
|
+
- **[Technical Reference](../reference/api-technical.md#feature-system)** - Implementation details and advanced patterns
|
|
895
895
|
- **[Overview](../overview.md#time-based-quantization)** - Conceptual introduction to quantization
|
|
896
896
|
- **[Time Utilities Guide](time-utilities.md)** - Time manipulation and formatting utilities
|
|
897
897
|
- **[Feature System Guide](feature-system.md)** - Understanding Familia's feature architecture
|
|
@@ -0,0 +1,345 @@
|
|
|
1
|
+
# Writing Migrations
|
|
2
|
+
|
|
3
|
+
## Quick Start
|
|
4
|
+
|
|
5
|
+
Minimal working migration:
|
|
6
|
+
|
|
7
|
+
```ruby
|
|
8
|
+
class NormalizeEmails < Familia::Migration::Base
|
|
9
|
+
self.migration_id = '20260131_120000_normalize_emails'
|
|
10
|
+
self.description = 'Lowercase all email addresses'
|
|
11
|
+
|
|
12
|
+
def migration_needed?
|
|
13
|
+
redis.exists('needs:normalization') > 0
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def migrate
|
|
17
|
+
redis.scan_each(match: 'user:*:object') do |key|
|
|
18
|
+
for_realsies_this_time? do
|
|
19
|
+
# perform changes
|
|
20
|
+
end
|
|
21
|
+
track_stat(:processed)
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Run it:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
bundle exec rake familia:migrate:dry_run # Preview
|
|
31
|
+
bundle exec rake familia:migrate # Apply
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
## Migration Types
|
|
35
|
+
|
|
36
|
+
| Type | Use When | Key Method |
|
|
37
|
+
|------|----------|------------|
|
|
38
|
+
| `Base` | Raw Redis operations, key patterns, config changes | `migrate` |
|
|
39
|
+
| `Model` | Iterating over Horreum objects with per-record logic | `process_record(obj, key)` |
|
|
40
|
+
| `Pipeline` | Bulk updates with Redis pipelining (1000+ records) | `should_process?(obj)`, `build_update_fields(obj)` |
|
|
41
|
+
|
|
42
|
+
### Base
|
|
43
|
+
|
|
44
|
+
Direct Redis access. Use for key renames, TTL changes, config migrations:
|
|
45
|
+
|
|
46
|
+
```ruby
|
|
47
|
+
class AddTTLToSessions < Familia::Migration::Base
|
|
48
|
+
self.migration_id = '20260131_add_ttl'
|
|
49
|
+
|
|
50
|
+
def migration_needed?
|
|
51
|
+
redis.exists('legacy:session:*') > 0
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def migrate
|
|
55
|
+
cursor = '0'
|
|
56
|
+
loop do
|
|
57
|
+
cursor, keys = redis.scan(cursor, match: 'legacy:session:*', count: 1000)
|
|
58
|
+
keys.each do |key|
|
|
59
|
+
for_realsies_this_time? { redis.expire(key, 3600) }
|
|
60
|
+
track_stat(:keys_expired)
|
|
61
|
+
end
|
|
62
|
+
break if cursor == '0'
|
|
63
|
+
end
|
|
64
|
+
end
|
|
65
|
+
end
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
### Model
|
|
69
|
+
|
|
70
|
+
SCAN-based iteration over Horreum objects. Use for per-record transformations with error handling:
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
class CustomerEmailMigration < Familia::Migration::Model
|
|
74
|
+
self.migration_id = '20260131_customer_emails'
|
|
75
|
+
|
|
76
|
+
def prepare
|
|
77
|
+
@model_class = Customer
|
|
78
|
+
@batch_size = 500 # optional, default: 1000
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
def process_record(customer, key)
|
|
82
|
+
return unless customer.email =~ /[A-Z]/
|
|
83
|
+
|
|
84
|
+
for_realsies_this_time? do
|
|
85
|
+
customer.email = customer.email.downcase
|
|
86
|
+
customer.save
|
|
87
|
+
end
|
|
88
|
+
track_stat(:records_updated)
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### Pipeline
|
|
94
|
+
|
|
95
|
+
Batched updates using Redis pipelining. Use for high-volume simple field updates:
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
class AddDefaultSettings < Familia::Migration::Pipeline
|
|
99
|
+
self.migration_id = '20260131_default_settings'
|
|
100
|
+
|
|
101
|
+
def prepare
|
|
102
|
+
@model_class = User
|
|
103
|
+
@batch_size = 100 # smaller batches for pipelines
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def should_process?(user)
|
|
107
|
+
user.settings.nil?
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
def build_update_fields(user)
|
|
111
|
+
{ 'settings' => '{}' }
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Override `execute_update` for custom pipeline operations:
|
|
117
|
+
|
|
118
|
+
```ruby
|
|
119
|
+
def execute_update(pipe, obj, fields, original_key)
|
|
120
|
+
dbkey = original_key || obj.dbkey
|
|
121
|
+
pipe.hmset(dbkey, *fields.flatten)
|
|
122
|
+
pipe.expire(dbkey, 86400) # also set TTL
|
|
123
|
+
end
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
## Class Attributes
|
|
127
|
+
|
|
128
|
+
| Attribute | Required | Description |
|
|
129
|
+
|-----------|----------|-------------|
|
|
130
|
+
| `migration_id` | Yes | Unique identifier. Format: `YYYYMMDD_HHMMSS_snake_case_name` |
|
|
131
|
+
| `description` | No | Human-readable summary for status output |
|
|
132
|
+
| `dependencies` | No | Array of migration IDs that must run first |
|
|
133
|
+
|
|
134
|
+
```ruby
|
|
135
|
+
class BuildEmailIndex < Familia::Migration::Base
|
|
136
|
+
self.migration_id = '20260131_150000_build_index'
|
|
137
|
+
self.description = 'Create secondary index for email lookups'
|
|
138
|
+
self.dependencies = ['20260131_120000_normalize_emails']
|
|
139
|
+
# ...
|
|
140
|
+
end
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
## Lifecycle Methods
|
|
144
|
+
|
|
145
|
+
| Method | Purpose | Required |
|
|
146
|
+
|--------|---------|----------|
|
|
147
|
+
| `prepare` | Initialize config, set `@model_class` | Model/Pipeline only |
|
|
148
|
+
| `migration_needed?` | Idempotency check. Return `false` to skip. | Yes |
|
|
149
|
+
| `migrate` | Core migration logic | Base only |
|
|
150
|
+
| `process_record(obj, key)` | Per-record logic | Model only |
|
|
151
|
+
| `should_process?(obj)` | Filter predicate | Pipeline only |
|
|
152
|
+
| `build_update_fields(obj)` | Return Hash of field updates | Pipeline only |
|
|
153
|
+
| `down` | Rollback logic | No (enables rollback) |
|
|
154
|
+
|
|
155
|
+
## Dry Run vs Live
|
|
156
|
+
|
|
157
|
+
Wrap destructive operations with `for_realsies_this_time?`:
|
|
158
|
+
|
|
159
|
+
```ruby
|
|
160
|
+
def migrate
|
|
161
|
+
redis.scan_each(match: 'session:*') do |key|
|
|
162
|
+
for_realsies_this_time? do
|
|
163
|
+
redis.del(key) # only executes with --run
|
|
164
|
+
end
|
|
165
|
+
track_stat(:deleted)
|
|
166
|
+
end
|
|
167
|
+
end
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
| Mode | `for_realsies_this_time?` | Registry Updated |
|
|
171
|
+
|------|---------------------------|------------------|
|
|
172
|
+
| Dry run (`:dry_run` task) | Block skipped | No |
|
|
173
|
+
| Live (`:run` task) | Block executes | Yes |
|
|
174
|
+
|
|
175
|
+
## Dependencies
|
|
176
|
+
|
|
177
|
+
Dependencies ensure execution order. The runner uses topological sort (Kahn's algorithm).
|
|
178
|
+
|
|
179
|
+
```ruby
|
|
180
|
+
class MigrationA < Familia::Migration::Base
|
|
181
|
+
self.migration_id = 'step_a'
|
|
182
|
+
self.dependencies = []
|
|
183
|
+
end
|
|
184
|
+
|
|
185
|
+
class MigrationB < Familia::Migration::Base
|
|
186
|
+
self.migration_id = 'step_b'
|
|
187
|
+
self.dependencies = ['step_a'] # runs after step_a
|
|
188
|
+
end
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
Rollback is blocked if dependents are still applied:
|
|
192
|
+
|
|
193
|
+
```ruby
|
|
194
|
+
runner.rollback('step_a')
|
|
195
|
+
# => Errors::HasDependents if step_b is applied
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
## Rollback
|
|
199
|
+
|
|
200
|
+
Implement `down` to enable rollback:
|
|
201
|
+
|
|
202
|
+
```ruby
|
|
203
|
+
class AddFeatureFlag < Familia::Migration::Base
|
|
204
|
+
self.migration_id = '20260131_feature_flag'
|
|
205
|
+
|
|
206
|
+
def migration_needed?
|
|
207
|
+
!redis.exists?('config:feature:enabled')
|
|
208
|
+
end
|
|
209
|
+
|
|
210
|
+
def migrate
|
|
211
|
+
for_realsies_this_time? do
|
|
212
|
+
redis.set('config:feature:enabled', 'true')
|
|
213
|
+
end
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
def down
|
|
217
|
+
redis.del('config:feature:enabled')
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
Check reversibility:
|
|
223
|
+
|
|
224
|
+
```ruby
|
|
225
|
+
instance = AddFeatureFlag.new
|
|
226
|
+
instance.reversible? # => true
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
## Lua Scripts
|
|
230
|
+
|
|
231
|
+
Use `Familia::Migration::Script` for atomic operations:
|
|
232
|
+
|
|
233
|
+
```ruby
|
|
234
|
+
# Rename hash field atomically
|
|
235
|
+
Familia::Migration::Script.execute(
|
|
236
|
+
redis,
|
|
237
|
+
:rename_field,
|
|
238
|
+
keys: ['user:123:object'],
|
|
239
|
+
argv: ['old_name', 'new_name']
|
|
240
|
+
)
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Built-in scripts:
|
|
244
|
+
|
|
245
|
+
| Script | Purpose | KEYS | ARGV |
|
|
246
|
+
|--------|---------|------|------|
|
|
247
|
+
| `:rename_field` | Rename hash field | `[hash_key]` | `[old, new]` |
|
|
248
|
+
| `:copy_field` | Copy field within hash | `[hash_key]` | `[src, dst]` |
|
|
249
|
+
| `:delete_field` | Delete hash field | `[hash_key]` | `[field]` |
|
|
250
|
+
| `:rename_key_preserve_ttl` | Rename key, keep TTL | `[src, dst]` | `[]` |
|
|
251
|
+
| `:backup_and_modify_field` | Backup old value, set new | `[hash, backup]` | `[field, value, ttl]` |
|
|
252
|
+
|
|
253
|
+
Register custom scripts:
|
|
254
|
+
|
|
255
|
+
```ruby
|
|
256
|
+
Familia::Migration::Script.register(:my_script, <<~LUA)
|
|
257
|
+
local key = KEYS[1]
|
|
258
|
+
return redis.call('GET', key)
|
|
259
|
+
LUA
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
## CLI Reference
|
|
263
|
+
|
|
264
|
+
```bash
|
|
265
|
+
# Status
|
|
266
|
+
bundle exec rake familia:migrate:status # Show applied/pending
|
|
267
|
+
bundle exec rake familia:migrate:validate # Check dependency issues
|
|
268
|
+
bundle exec rake familia:migrate:schema_drift # Models with changed schemas
|
|
269
|
+
|
|
270
|
+
# Execution
|
|
271
|
+
bundle exec rake familia:migrate:dry_run # Preview (no changes)
|
|
272
|
+
bundle exec rake familia:migrate # Apply all pending
|
|
273
|
+
bundle exec rake familia:migrate:run # Same as above
|
|
274
|
+
|
|
275
|
+
# Rollback
|
|
276
|
+
bundle exec rake "familia:migrate:rollback[20260131_120000_migration_id]"
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
## Statistics
|
|
280
|
+
|
|
281
|
+
Track operations with `track_stat`:
|
|
282
|
+
|
|
283
|
+
```ruby
|
|
284
|
+
def process_record(obj, key)
|
|
285
|
+
if obj.email.blank?
|
|
286
|
+
track_stat(:skipped_blank)
|
|
287
|
+
return
|
|
288
|
+
end
|
|
289
|
+
|
|
290
|
+
for_realsies_this_time? do
|
|
291
|
+
obj.email = obj.email.downcase
|
|
292
|
+
obj.save
|
|
293
|
+
end
|
|
294
|
+
track_stat(:records_updated)
|
|
295
|
+
end
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
Access stats:
|
|
299
|
+
|
|
300
|
+
```ruby
|
|
301
|
+
instance.stats[:records_updated] # => 42
|
|
302
|
+
instance.stats[:skipped_blank] # => 7
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
## Configuration
|
|
306
|
+
|
|
307
|
+
```ruby
|
|
308
|
+
Familia::Migration.configure do |config|
|
|
309
|
+
config.migrations_key = 'familia:migrations' # Registry key prefix
|
|
310
|
+
config.backup_ttl = 86_400 # Backup expiration (24h)
|
|
311
|
+
config.batch_size = 1000 # Default SCAN batch
|
|
312
|
+
end
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
## Best Practices
|
|
316
|
+
|
|
317
|
+
1. **Test locally first.** Run dry run, verify stats, then run live on staging before production.
|
|
318
|
+
|
|
319
|
+
2. **Deploy schema changes separately.** Avoid updating model definitions and running migrations in the same deploy. New model logic can break migration code.
|
|
320
|
+
|
|
321
|
+
3. **Keep migrations idempotent.** `migration_needed?` should return `false` after successful execution.
|
|
322
|
+
|
|
323
|
+
4. **Use descriptive IDs.** `20260131_120000_normalize_customer_emails` beats `20260131_fix_stuff`.
|
|
324
|
+
|
|
325
|
+
5. **Backup critical data.** Use `:backup_and_modify_field` or `registry.backup_field` before destructive changes.
|
|
326
|
+
|
|
327
|
+
## Error Reference
|
|
328
|
+
|
|
329
|
+
| Error | Cause |
|
|
330
|
+
|-------|-------|
|
|
331
|
+
| `NotReversible` | `down` not implemented |
|
|
332
|
+
| `NotApplied` | Rollback of unapplied migration |
|
|
333
|
+
| `DependencyNotMet` | Dependency not yet applied |
|
|
334
|
+
| `HasDependents` | Rollback blocked by dependents |
|
|
335
|
+
| `CircularDependency` | Dependency cycle detected |
|
|
336
|
+
| `PreconditionFailed` | `@model_class` not set in `prepare` |
|
|
337
|
+
|
|
338
|
+
## Source Files
|
|
339
|
+
|
|
340
|
+
- [`lib/familia/migration/base.rb`](../../lib/familia/migration/base.rb)
|
|
341
|
+
- [`lib/familia/migration/model.rb`](../../lib/familia/migration/model.rb)
|
|
342
|
+
- [`lib/familia/migration/pipeline.rb`](../../lib/familia/migration/pipeline.rb)
|
|
343
|
+
- [`lib/familia/migration/registry.rb`](../../lib/familia/migration/registry.rb)
|
|
344
|
+
- [`lib/familia/migration/runner.rb`](../../lib/familia/migration/runner.rb)
|
|
345
|
+
- [`lib/familia/migration/script.rb`](../../lib/familia/migration/script.rb)
|
data/docs/overview.md
CHANGED
|
@@ -242,7 +242,7 @@ This automatically groups metrics into 10-minute intervals formatted as "HH:MM",
|
|
|
242
242
|
- **Reduced Storage**: Aggregate similar data points to optimize memory usage
|
|
243
243
|
- **Analytics Ready**: Perfect for dashboards and time-series data visualization
|
|
244
244
|
|
|
245
|
-
> For advanced quantization strategies, value bucketing, geographic quantization, and performance patterns, see the [Technical Reference](reference/api-technical.md#
|
|
245
|
+
> For advanced quantization strategies, value bucketing, geographic quantization, and performance patterns, see the [Technical Reference](reference/api-technical.md#feature-system).
|
|
246
246
|
|
|
247
247
|
### Object Identifiers
|
|
248
248
|
|
|
@@ -273,7 +273,7 @@ session.objid # => "a1b2c3d4e5f6" (hex)
|
|
|
273
273
|
- `:uuid_v4` - Standard UUID format for global uniqueness
|
|
274
274
|
- `:hex` - Compact hexadecimal identifiers for internal use
|
|
275
275
|
|
|
276
|
-
> For custom generators, collision detection, and advanced identifier patterns, see the [Technical Reference](reference/api-technical.md#
|
|
276
|
+
> For custom generators, collision detection, and advanced identifier patterns, see the [Technical Reference](reference/api-technical.md#feature-system).
|
|
277
277
|
|
|
278
278
|
### Specialized Field Types
|
|
279
279
|
|
|
@@ -354,7 +354,7 @@ user = ExternalUser.create(external_id: "ext_12345", name: "Alice")
|
|
|
354
354
|
|
|
355
355
|
This feature helps maintain consistency when integrating with external APIs or legacy systems.
|
|
356
356
|
|
|
357
|
-
> For advanced external identifier patterns, batch operations, and sync status management, see the [Technical Reference](reference/api-technical.md#
|
|
357
|
+
> For advanced external identifier patterns, batch operations, and sync status management, see the [Technical Reference](reference/api-technical.md#feature-system).
|
|
358
358
|
|
|
359
359
|
### Relationships
|
|
360
360
|
|
|
@@ -435,7 +435,7 @@ Team.email_index_for("alice@example.com") # Direct index access
|
|
|
435
435
|
- **Automatic Indexing**: Efficient O(1) lookups with automatic index maintenance
|
|
436
436
|
- **Performance Optimized**: Bulk operations and efficient sorted set operations
|
|
437
437
|
|
|
438
|
-
> For advanced relationship patterns, permission-encoded relationships, time-series tracking, and performance optimization, see the [Technical Reference](reference/api-technical.md#
|
|
438
|
+
> For advanced relationship patterns, permission-encoded relationships, time-series tracking, and performance optimization, see the [Technical Reference](reference/api-technical.md#feature-system).
|
|
439
439
|
|
|
440
440
|
### Transient Fields
|
|
441
441
|
|
|
@@ -472,7 +472,7 @@ attempt.security_token.reveal # => "sensitive_data"
|
|
|
472
472
|
- **Transient Fields**: Exist only in memory, never persisted
|
|
473
473
|
- **Redacted Fields**: Return `[REDACTED]` when converted to strings for logging safety
|
|
474
474
|
|
|
475
|
-
> For RedactedString implementation details, single-use patterns, and security considerations, see the [Technical Reference](reference/api-technical.md#
|
|
475
|
+
> For RedactedString implementation details, single-use patterns, and security considerations, see the [Technical Reference](reference/api-technical.md#feature-system).
|
|
476
476
|
|
|
477
477
|
### Permission Management
|
|
478
478
|
|
|
@@ -638,7 +638,7 @@ user.encrypted_fields_status # Check encryption status
|
|
|
638
638
|
- **Key Rotation**: Seamless updates with backward compatibility
|
|
639
639
|
- **Multiple Algorithms**: XChaCha20-Poly1305 (preferred) with AES-256-GCM fallback
|
|
640
640
|
|
|
641
|
-
> For advanced encryption configuration, multiple providers, request caching, and key rotation procedures, see the [Technical Reference](reference/api-technical.md#
|
|
641
|
+
> For advanced encryption configuration, multiple providers, request caching, and key rotation procedures, see the [Technical Reference](reference/api-technical.md#feature-system).
|
|
642
642
|
|
|
643
643
|
### Open-ended Serialization
|
|
644
644
|
|
|
@@ -772,7 +772,7 @@ Familia.configure do |config|
|
|
|
772
772
|
end
|
|
773
773
|
```
|
|
774
774
|
|
|
775
|
-
> For production configuration patterns, advanced connection pooling, multi-database setup, and environment-based configuration, see the [Technical Reference](reference/api-technical.md#connection-management
|
|
775
|
+
> For production configuration patterns, advanced connection pooling, multi-database setup, and environment-based configuration, see the [Technical Reference](reference/api-technical.md#connection-management).
|
|
776
776
|
|
|
777
777
|
## Common Patterns
|
|
778
778
|
|