familia 2.0.0.pre25 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. checksums.yaml +4 -4
  2. data/.gitignore +1 -0
  3. data/CHANGELOG.rst +69 -0
  4. data/Gemfile +1 -0
  5. data/Gemfile.lock +2 -2
  6. data/README.md +1 -3
  7. data/docs/guides/feature-encrypted-fields.md +1 -1
  8. data/docs/guides/feature-expiration.md +1 -1
  9. data/docs/guides/feature-quantization.md +1 -1
  10. data/docs/overview.md +7 -7
  11. data/docs/reference/api-technical.md +103 -7
  12. data/familia.gemspec +1 -2
  13. data/lib/familia/data_type/types/hashkey.rb +238 -0
  14. data/lib/familia/data_type/types/listkey.rb +110 -4
  15. data/lib/familia/data_type/types/sorted_set.rb +365 -0
  16. data/lib/familia/data_type/types/stringkey.rb +139 -0
  17. data/lib/familia/data_type/types/unsorted_set.rb +122 -2
  18. data/lib/familia/features/relationships/indexing/rebuild_strategies.rb +2 -1
  19. data/lib/familia/features/relationships/participation/through_model_operations.rb +4 -3
  20. data/lib/familia/features/relationships/participation.rb +6 -6
  21. data/lib/familia/horreum/management.rb +29 -0
  22. data/lib/familia/version.rb +1 -1
  23. data/try/features/relationships/prefix_vs_config_name_try.rb +418 -0
  24. metadata +3 -27
  25. data/docs/migrating/v2.0.0-pre.md +0 -84
  26. data/docs/migrating/v2.0.0-pre11.md +0 -253
  27. data/docs/migrating/v2.0.0-pre12.md +0 -306
  28. data/docs/migrating/v2.0.0-pre13.md +0 -95
  29. data/docs/migrating/v2.0.0-pre14.md +0 -37
  30. data/docs/migrating/v2.0.0-pre18.md +0 -58
  31. data/docs/migrating/v2.0.0-pre19.md +0 -197
  32. data/docs/migrating/v2.0.0-pre22.md +0 -241
  33. data/docs/migrating/v2.0.0-pre5.md +0 -131
  34. data/docs/migrating/v2.0.0-pre6.md +0 -154
  35. data/docs/migrating/v2.0.0-pre7.md +0 -222
@@ -1,95 +0,0 @@
1
- # Migrating Guide: v2.0.0-pre13
2
-
3
- This version introduces the Feature Autoloading System for automatic discovery and loading of feature-specific configuration files, enabling cleaner separation between core model definitions and feature configurations.
4
-
5
- ## Feature Autoloading System
6
-
7
- ### What Changed
8
-
9
- Features now automatically discover and load extension files from your project directories using conventional file naming patterns. This eliminates the need to configure features in your main model files.
10
-
11
- ### Basic Migration
12
-
13
- #### Before
14
- ```ruby
15
- # app/models/user.rb
16
- class User < Familia::Horreum
17
- field :name, :email, :password
18
-
19
- feature :safe_dump
20
- safe_dump_fields :name, :email # Configuration mixed with model
21
- end
22
- ```
23
-
24
- #### After
25
- ```ruby
26
- # app/models/user.rb - Clean model definition
27
- class User < Familia::Horreum
28
- field :name, :email, :password
29
- feature :safe_dump # Configuration auto-loaded
30
- end
31
-
32
- # app/models/user/safe_dump_extensions.rb - Automatically discovered
33
- class User
34
- safe_dump_fields :name, :email
35
- end
36
- ```
37
-
38
- ### File Naming Convention
39
-
40
- Extension files follow the pattern: `{model_name}/{feature_name}_*.rb`
41
-
42
- ```
43
- app/models/
44
- ├── user.rb
45
- ├── user/
46
- │ ├── safe_dump_extensions.rb # SafeDump configuration
47
- │ └── expiration_config.rb # Expiration settings
48
- ```
49
-
50
- ### Migration Steps
51
-
52
- 1. **Create extension directories**: `mkdir -p app/models/user`
53
- 2. **Extract feature configuration** from main model files to separate extension files
54
- 3. **Verify autoloading**: Check that feature methods are available after migration
55
-
56
- ### Debugging
57
-
58
- Enable debug output to troubleshoot autoloading:
59
- ```ruby
60
- ENV['FAMILIA_DEBUG'] = '1' # Shows discovered and loaded files
61
- ```
62
-
63
- Common issues:
64
- - Files must follow `{feature_name}_*.rb` naming pattern
65
- - Extension files should reopen the same class as your model
66
-
67
- ## Architecture
68
-
69
- The Feature Autoloading System consists of two key components:
70
-
71
- ### Familia::Features::Autoloader
72
- A utility module providing shared file loading functionality:
73
- - Handles Dir.glob pattern matching and file loading
74
- - Provides consistent debug logging across all autoloading scenarios
75
- - Used by both feature-specific and general-purpose autoloading
76
-
77
- ### Familia::Features::Autoloadable
78
- A mixin for feature modules that enables post-inclusion autoloading:
79
- - Uses `Module.const_source_location` to find where user classes are defined
80
- - Discovers extension files using conventional patterns relative to the user class location
81
- - Integrates with the feature system's inclusion lifecycle
82
-
83
- When you call `feature :safe_dump`, the SafeDump module (which includes Autoloadable) triggers post-inclusion autoloading that searches for `user/safe_dump_*.rb` files and loads them automatically.
84
-
85
- ## New Capabilities
86
-
87
- 1. **Automatic Extension Discovery**: No manual require statements needed
88
- 2. **Conventional File Organization**: Standard patterns for consistent project structure
89
- 3. **Feature Isolation**: Clean separation between core models and feature configurations
90
- 4. **Shared Autoloader Infrastructure**: Consistent loading behavior across all features
91
- 5. **Debug Support**: Built-in debugging for troubleshooting autoloading issues
92
-
93
- ## Breaking Changes
94
-
95
- **None** - This release is fully backward compatible. Existing models and feature configurations continue to work without modification.
@@ -1,37 +0,0 @@
1
- # Migrating Guide: v2.0.0-pre14
2
-
3
- This version renames TimeUtils to TimeLiterals for semantic clarity and fixes an ExternalIdentifier bug.
4
-
5
- ## Breaking Change: TimeUtils → TimeLiterals
6
-
7
- **Migration Required:**
8
-
9
- Find and replace in your codebase:
10
- ```bash
11
- # Find files to update
12
- grep -r "using Familia::Refinements::TimeUtils" .
13
-
14
- # Replace the import
15
- sed -i 's/using Familia::Refinements::TimeUtils/using Familia::Refinements::TimeLiterals/g' *.rb
16
- ```
17
-
18
- **Before:**
19
- ```ruby
20
- using Familia::Refinements::TimeUtils
21
- ```
22
-
23
- **After:**
24
- ```ruby
25
- using Familia::Refinements::TimeLiterals
26
- ```
27
-
28
- All functionality remains identical - only the module name changed.
29
-
30
- ## Bug Fix: ExternalIdentifier
31
-
32
- Fixed `NoMethodError` when using ExternalIdentifier by replacing incorrect `.del()` calls with `.remove_field()` in HashKey operations. This affected:
33
- - Changing external identifier values
34
- - Looking up objects by external ID
35
- - Destroying objects with external identifiers
36
-
37
- No migration needed - the fix is automatic.
@@ -1,58 +0,0 @@
1
- # Migrating Guide: v2.0.0-pre18
2
-
3
- This version completes the JSON serialization implementation by removing the string-as-is optimization and fixes encrypted field visibility in serialization.
4
-
5
- ## JSON Serialization for All Types
6
-
7
- **What Changed:**
8
-
9
- All field values (including strings) are now JSON-encoded during storage for consistent type preservation.
10
-
11
- **Storage Format:**
12
-
13
- ```ruby
14
- # Before (v2.0.0-pre14):
15
- HGET user:123 name
16
- "John Doe" # Plain string
17
-
18
- # After (v2.0.0-pre18):
19
- HGET user:123 name
20
- "\"John Doe\"" # JSON-encoded string
21
- ```
22
-
23
- **Migration:**
24
-
25
- No migration needed. The deserializer automatically handles:
26
- - **New format**: `"\"value\""` → `"value"`
27
- - **Legacy format**: `"value"` → `"value"`
28
-
29
- **Why This Matters:**
30
-
31
- Prevents data corruption for edge cases:
32
- - Badge `"007"` stays `"007"` (not converted to integer `7`)
33
- - String `"true"` stays `"true"` (not converted to boolean)
34
- - String `"null"` stays `"null"` (not converted to `nil`)
35
-
36
- ## Encrypted Field Security Fix
37
-
38
- **What Changed:**
39
-
40
- Fields defined with `category: :encrypted` now correctly exclude encrypted data from `to_h()` output.
41
-
42
- **Before:**
43
- ```ruby
44
- field :secret, category: :encrypted
45
- user.to_h # => {"id" => "123", "secret" => {...}} # ❌ Exposed!
46
- ```
47
-
48
- **After:**
49
- ```ruby
50
- field :secret, category: :encrypted
51
- user.to_h # => {"id" => "123"} # ✅ Secure
52
- ```
53
-
54
- Both `encrypted_field` and `field :name, category: :encrypted` now behave identically for security.
55
-
56
- **Migration:**
57
-
58
- No code changes needed. Review any code relying on encrypted fields appearing in `to_h()` output.
@@ -1,197 +0,0 @@
1
- # Migrating Guide: v2.0.0-pre19
2
-
3
- This version introduces significant improvements to Familia's database operations, making them more atomic, reliable, and consistent with Rails conventions. The changes include enhanced error handling, optimistic locking support, and breaking API changes.
4
-
5
- ## Breaking Changes
6
-
7
- ### Management.create → create!
8
-
9
- **What Changed:**
10
-
11
- The `create` class method has been renamed to `create!` to follow Rails conventions and indicate that it raises exceptions on failure.
12
-
13
- **Before:**
14
- ```ruby
15
- user = User.create(email: "test@example.com")
16
- ```
17
-
18
- **After:**
19
- ```ruby
20
- user = User.create!(email: "test@example.com")
21
- ```
22
-
23
- **Why This Matters:**
24
-
25
- - Follows Rails naming conventions where `!` methods raise exceptions
26
- - Makes it clear that `CreationError` will be raised if object already exists
27
- - Prevents silent failures in object creation
28
-
29
- ### save_if_not_exists → save_if_not_exists!
30
-
31
- **What Changed:**
32
-
33
- The `save_if_not_exists` method has been renamed to `save_if_not_exists!` and now includes optimistic locking with automatic retry logic.
34
-
35
- **Before:**
36
- ```ruby
37
- success = user.save_if_not_exists
38
- ```
39
-
40
- **After:**
41
- ```ruby
42
- success = user.save_if_not_exists!
43
- ```
44
-
45
- **Behavior Changes:**
46
- - Now uses Redis WATCH/MULTI/EXEC for optimistic locking
47
- - Automatically retries up to 3 times on `OptimisticLockError`
48
- - Raises `RecordExistsError` if object already exists
49
- - More atomic and thread-safe
50
-
51
- ## Enhanced Error Handling
52
-
53
- ### New Error Hierarchy
54
-
55
- **What's New:**
56
-
57
- A structured error hierarchy provides better error categorization:
58
-
59
- ```ruby
60
- Familia::Problem # Base class
61
- ├── Familia::PersistenceError # Redis/database errors
62
- │ ├── Familia::NonUniqueKey
63
- │ ├── Familia::OptimisticLockError
64
- │ ├── Familia::OperationModeError
65
- │ ├── Familia::NoConnectionAvailable
66
- │ ├── Familia::NotFound
67
- │ └── Familia::NotConnected
68
- └── Familia::HorreumError # Model-related errors
69
- ├── Familia::CreationError
70
- ├── Familia::NoIdentifier
71
- ├── Familia::FieldTypeError
72
- ├── Familia::AutoloadError
73
- ├── Familia::SerializerError
74
- ├── Familia::UnknownFieldError
75
- └── Familia::NotDistinguishableError
76
- ```
77
-
78
- **Migration:**
79
-
80
- Update exception handling to use specific error classes:
81
-
82
- ```ruby
83
- # Before
84
- rescue Familia::Problem => e
85
- # Handle all errors
86
-
87
- # After - More granular handling
88
- rescue Familia::CreationError => e
89
- # Handle creation failures specifically
90
- rescue Familia::OptimisticLockError => e
91
- # Handle concurrent modification
92
- rescue Familia::PersistenceError => e
93
- # Handle database-related errors
94
- ```
95
-
96
- ### New Exception Types
97
-
98
- - **`CreationError`**: Raised when object creation fails (replaces generic errors)
99
- - **`OptimisticLockError`**: Raised when WATCH fails due to concurrent modification
100
- - **`UnknownFieldError`**: Raised when referencing non-existent fields
101
-
102
- ## Database Operation Improvements
103
-
104
- ### Atomic Save Operations
105
-
106
- **What Changed:**
107
-
108
- The `save` method now uses a single Redis transaction for complete atomicity:
109
-
110
- ```ruby
111
- # All operations now happen atomically:
112
- # 1. Save all fields (HMSET)
113
- # 2. Set expiration (EXPIRE)
114
- # 3. Update indexes
115
- # 4. Add to instances collection
116
- ```
117
-
118
- **Benefits:**
119
- - Eliminates race conditions during save operations
120
- - Ensures data consistency across related operations
121
- - Better performance with fewer round trips
122
-
123
- ### Enhanced Timestamp Precision
124
-
125
- **What Changed:**
126
-
127
- Created/updated timestamps now use float values instead of integers for higher precision:
128
-
129
- **Before:**
130
- ```ruby
131
- user.created # => 1697234567 (integer seconds)
132
- ```
133
-
134
- **After:**
135
- ```ruby
136
- user.created # => 1697234567.123 (float with milliseconds)
137
- ```
138
-
139
- **Migration:**
140
-
141
- No code changes needed. Existing integer timestamps continue to work.
142
-
143
- ## Redis Command Enhancements
144
-
145
- ### New Commands Available
146
-
147
- Added support for optimistic locking commands:
148
- - `watch(key)` - Watch key for changes
149
- - `unwatch()` - Remove all watches
150
- - `discard()` - Discard queued commands
151
-
152
- ### Improved Command Logging
153
-
154
- Database command logging now includes:
155
- - Structured format for better readability
156
- - Pipelined operation tracking
157
- - Transaction boundary markers
158
- - Command timing information
159
-
160
- ## Terminology Updates
161
-
162
- **What Changed:**
163
-
164
- Standardized on "pipelined" terminology throughout (previously mixed "pipeline"/"pipelined").
165
-
166
- **Files Affected:**
167
- - Method names now consistently use "pipelined"
168
- - Documentation updated to match Redis terminology
169
- - Log messages standardized
170
-
171
- **Migration:**
172
-
173
- No code changes needed - this was an internal consistency improvement.
174
-
175
- ## Recommended Actions
176
-
177
- 1. **Update method calls:**
178
- ```ruby
179
- # Replace all instances
180
- Model.create(...) → Model.create!(...)
181
- obj.save_if_not_exists → obj.save_if_not_exists!
182
- ```
183
-
184
- 2. **Review error handling:**
185
- ```ruby
186
- # Consider more specific error handling
187
- rescue Familia::CreationError
188
- rescue Familia::OptimisticLockError
189
- ```
190
-
191
- 3. **Test concurrent operations:**
192
- - The new optimistic locking provides better concurrency handling
193
- - Verify your application handles `OptimisticLockError` appropriately
194
-
195
- 4. **Review logging:**
196
- - Enhanced database command logging may affect log volume
197
- - Adjust log levels if needed for production environments
@@ -1,241 +0,0 @@
1
- # Migrating Guide: v2.0.0-pre22
2
-
3
- This version introduces significant performance optimizations for Redis operations, completes the bidirectional relationships feature, and improves flexibility for external identifiers.
4
-
5
- ## Major Features
6
-
7
- ### Bidirectional Relationship Methods
8
-
9
- **What's New:**
10
-
11
- The `participates_in` declarations now generate reverse collection methods with the `_instances` suffix, providing symmetric access to relationships from both directions.
12
-
13
- **Generated Methods:**
14
- ```ruby
15
- class User < Familia::Horreum
16
- participates_in Team, :members
17
- participates_in Organization, :employees
18
- end
19
-
20
- # New reverse collection methods:
21
- user.team_instances # => [team1, team2]
22
- user.team_ids # => ["team_123", "team_456"]
23
- user.team? # => true/false
24
- user.team_count # => 2
25
-
26
- user.organization_instances # => [org1]
27
- user.organization_ids # => ["org_789"]
28
- ```
29
-
30
- **Custom Names:**
31
- ```ruby
32
- class User < Familia::Horreum
33
- participates_in Organization, :contractors, as: :clients
34
- end
35
-
36
- user.clients_instances # Instead of organization_instances
37
- user.clients_ids # Instead of organization_ids
38
- ```
39
-
40
- **Migration:**
41
-
42
- No changes required for existing code. The new methods are additive and don't affect existing `participates_in` functionality.
43
-
44
- ### Pipelined Bulk Loading
45
-
46
- **What's New:**
47
-
48
- New `load_multi` methods provide up to 2× performance improvement for bulk object loading by using Redis pipelining.
49
-
50
- **Before (N×2 commands):**
51
- ```ruby
52
- users = ids.map { |id| User.find_by_id(id) }
53
- # For 14 objects: 28 Redis commands (14 EXISTS + 14 HGETALL)
54
- ```
55
-
56
- **After (1 round trip):**
57
- ```ruby
58
- users = User.load_multi(ids)
59
- # For 14 objects: 1 pipelined batch with 14 HGETALL commands
60
- ```
61
-
62
- **Additional Methods:**
63
- ```ruby
64
- # Load by full dbkeys
65
- users = User.load_multi_by_keys(['user:123:object', 'user:456:object'])
66
-
67
- # Filter out nils for missing objects
68
- existing_only = User.load_multi(ids).compact
69
- ```
70
-
71
- ### Optional EXISTS Check Optimization
72
-
73
- **What's New:**
74
-
75
- The `find_by_id` and related methods now support skipping the EXISTS check for 50% reduction in Redis commands.
76
-
77
- ```ruby
78
- # Default behavior (unchanged, 2 commands)
79
- user = User.find_by_id(123)
80
-
81
- # Optimized mode (1 command)
82
- user = User.find_by_id(123, check_exists: false)
83
- ```
84
-
85
- **When to Use:**
86
- - Performance-critical paths
87
- - Bulk operations with known-to-exist keys
88
- - High-throughput APIs
89
- - Loading from sorted set results
90
-
91
- ## Enhanced Features
92
-
93
- ### Flexible External Identifier Format
94
-
95
- **What's New:**
96
-
97
- The `external_identifier` feature now supports custom format templates.
98
-
99
- **Examples:**
100
- ```ruby
101
- # Default format (unchanged)
102
- class User < Familia::Horreum
103
- feature :external_identifier
104
- end
105
- user.extid # => "ext_abc123def456"
106
-
107
- # Custom prefix
108
- class Customer < Familia::Horreum
109
- feature :external_identifier, format: 'cust_%{id}'
110
- end
111
- customer.extid # => "cust_abc123def456"
112
-
113
- # Different separator
114
- class APIKey < Familia::Horreum
115
- feature :external_identifier, format: 'api-%{id}'
116
- end
117
- key.extid # => "api-abc123def456"
118
- ```
119
-
120
- ### Atomic Index Rebuilding
121
-
122
- **What's New:**
123
-
124
- Auto-generated rebuild methods for all unique and multi indexes with zero downtime.
125
-
126
- **Examples:**
127
-
128
- ```ruby
129
- # Class-level unique index
130
- User.rebuild_email_lookup
131
-
132
- # Instance-scoped unique index
133
- company.rebuild_badge_index
134
-
135
- # With progress tracking
136
- User.rebuild_email_lookup(batch_size: 100) do |progress|
137
- puts "#{progress[:completed]}/#{progress[:total]}"
138
- end
139
- ```
140
-
141
- When to Use:
142
- - After data migrations or bulk imports
143
- - Recovering from index corruption
144
- - Adding indexes to existing data
145
-
146
- Migration:
147
-
148
- Run rebuild methods once after upgrade to ensure index consistency. No code changes required—methods are auto-generated from existing
149
- unique_index and multi_index declarations.
150
-
151
-
152
- ## Bug Fixes
153
-
154
- ### Symbol/String Target Classes in participates_in
155
-
156
- **What Was Fixed:**
157
-
158
- Fixed multiple bugs when using Symbol or String class names in `participates_in`:
159
-
160
- ```ruby
161
- class Domain < Familia::Horreum
162
- # All forms now work correctly:
163
- participates_in Customer, :domains # Class object
164
- participates_in :Customer, :domains # Symbol (was broken)
165
- participates_in 'Customer', :domains # String (was broken)
166
- end
167
- ```
168
-
169
- **Errors Fixed:**
170
- - `NoMethodError: private method 'member_by_config_name'`
171
- - `NoMethodError: undefined method 'familia_name' for Symbol`
172
- - `NoMethodError: undefined method 'config_name' for Symbol`
173
- - Confusing nil errors for unloaded classes
174
-
175
- **New Behavior:**
176
-
177
- When a target class can't be resolved, you now get a helpful error:
178
- ```
179
- Target class 'Customer' could not be resolved.
180
- Possible causes:
181
- 1. The class hasn't been loaded yet (load order issue)
182
- 2. The class name is misspelled
183
- 3. The class doesn't inherit from Familia::Horreum
184
-
185
- Registered Familia classes: ["User", "Team", "Organization"]
186
- ```
187
-
188
- ## Performance Recommendations
189
-
190
- ### Use Bulk Loading for Collections
191
-
192
- ```ruby
193
- # ❌ Avoid N+1 queries
194
- team.members.to_a.map { |id| User.find_by_id(id) }
195
-
196
- # ✅ Use bulk loading
197
- User.load_multi(team.members.to_a)
198
- ```
199
-
200
- ### Skip EXISTS Checks When Safe
201
-
202
- ```ruby
203
- # When loading from sorted sets (keys guaranteed to exist)
204
- task_ids = project.tasks.range(0, 9)
205
- tasks = Task.load_multi(task_ids) # Or use check_exists: false
206
-
207
- # For known-existing keys
208
- user = User.find_by_id(session[:user_id], check_exists: false)
209
- ```
210
-
211
- ### Leverage Reverse Collection Methods
212
-
213
- ```ruby
214
- # ❌ Manual parsing of participations
215
- team_keys = user.participations.members.select { |k| k.start_with?("team:") }
216
- team_ids = team_keys.map { |k| k.split(':')[1] }
217
- teams = Team.load_multi(team_ids)
218
-
219
- # ✅ Use generated methods
220
- teams = user.team_instances
221
- ```
222
-
223
- ## Backwards Compatibility
224
-
225
- All changes in this version are backwards compatible:
226
-
227
- - New methods are additive and don't affect existing APIs
228
- - Default behaviors remain unchanged
229
- - Symbol/String fixes don't require code changes
230
-
231
- ## Recommended Actions
232
-
233
- 1. **Adopt bulk loading** for performance-critical paths
234
- 2. **Use reverse collection methods** to simplify relationship queries
235
- 3. **Consider check_exists: false** for guaranteed-existing keys
236
- 4. **Update external_identifier formats** if custom prefixes are needed
237
-
238
- ## See Also
239
-
240
- - [Relationships Guide](../guides/feature-relationships.md)
241
- - [Performance Optimization Guide](../guides/optimized-loading.md)