familia 2.0.0.pre25 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. checksums.yaml +4 -4
  2. data/.gitignore +1 -0
  3. data/CHANGELOG.rst +69 -0
  4. data/Gemfile +1 -0
  5. data/Gemfile.lock +2 -2
  6. data/README.md +1 -3
  7. data/docs/guides/feature-encrypted-fields.md +1 -1
  8. data/docs/guides/feature-expiration.md +1 -1
  9. data/docs/guides/feature-quantization.md +1 -1
  10. data/docs/overview.md +7 -7
  11. data/docs/reference/api-technical.md +103 -7
  12. data/familia.gemspec +1 -2
  13. data/lib/familia/data_type/types/hashkey.rb +238 -0
  14. data/lib/familia/data_type/types/listkey.rb +110 -4
  15. data/lib/familia/data_type/types/sorted_set.rb +365 -0
  16. data/lib/familia/data_type/types/stringkey.rb +139 -0
  17. data/lib/familia/data_type/types/unsorted_set.rb +122 -2
  18. data/lib/familia/features/relationships/indexing/rebuild_strategies.rb +2 -1
  19. data/lib/familia/features/relationships/participation/through_model_operations.rb +4 -3
  20. data/lib/familia/features/relationships/participation.rb +6 -6
  21. data/lib/familia/horreum/management.rb +29 -0
  22. data/lib/familia/version.rb +1 -1
  23. data/try/features/relationships/prefix_vs_config_name_try.rb +418 -0
  24. metadata +3 -27
  25. data/docs/migrating/v2.0.0-pre.md +0 -84
  26. data/docs/migrating/v2.0.0-pre11.md +0 -253
  27. data/docs/migrating/v2.0.0-pre12.md +0 -306
  28. data/docs/migrating/v2.0.0-pre13.md +0 -95
  29. data/docs/migrating/v2.0.0-pre14.md +0 -37
  30. data/docs/migrating/v2.0.0-pre18.md +0 -58
  31. data/docs/migrating/v2.0.0-pre19.md +0 -197
  32. data/docs/migrating/v2.0.0-pre22.md +0 -241
  33. data/docs/migrating/v2.0.0-pre5.md +0 -131
  34. data/docs/migrating/v2.0.0-pre6.md +0 -154
  35. data/docs/migrating/v2.0.0-pre7.md +0 -222
@@ -1,131 +0,0 @@
1
- # Migrating Guide: Security Features (v2.0.0-pre5)
2
-
3
- This guide covers adopting the security enhancements introduced in v2.0.0-pre5.
4
-
5
- ## Security Feature Adoption
6
-
7
- ### 1. Configure Encryption Keys
8
-
9
- Before using encrypted fields, configure encryption keys:
10
-
11
- ```ruby
12
- Familia.configure do |config|
13
- config.encryption_keys = {
14
- v1: 'your-32-byte-base64-encoded-key==',
15
- v2: 'newer-32-byte-base64-encoded-key=='
16
- }
17
- config.current_key_version = :v2
18
- end
19
- ```
20
-
21
- **Key Management:**
22
- - Use secure key storage (environment variables, key management services)
23
- - Rotate keys regularly by adding new versions
24
- - Never remove old key versions while data exists
25
-
26
- ### 2. Identify Sensitive Fields
27
-
28
- Mark fields that contain sensitive data:
29
-
30
- **For Encryption:**
31
- ```ruby
32
- class Vault < Familia::Horreum
33
- feature :encrypted_fields
34
-
35
- field :name # Plaintext
36
- encrypted_field :secret_key # Encrypted at rest
37
- encrypted_field :api_token # Transparent access
38
- end
39
- ```
40
-
41
- **For Transient Fields:**
42
- ```ruby
43
- class User < Familia::Horreum
44
- feature :transient_fields
45
-
46
- field :email # Persisted
47
- transient_field :password # Never persisted
48
- transient_field :session_token # Runtime only
49
- end
50
- ```
51
-
52
- ### 3. Update Serialization Code
53
-
54
- Handle `ConcealedString` in serialization:
55
-
56
- **Before:**
57
- ```ruby
58
- def to_json
59
- { name: name, password: password }.to_json
60
- end
61
- ```
62
-
63
- **After:**
64
- ```ruby
65
- def to_json
66
- # ConcealedString automatically excluded from serialization
67
- { name: name }.to_json # password field omitted if transient
68
- end
69
- ```
70
-
71
- **Manual ConcealedString Handling:**
72
- ```ruby
73
- # Access original value when needed
74
- password.reveal # Returns actual string value
75
- password.cleared? # Returns true if cleared from memory
76
- ```
77
-
78
- ### 4. Implement Key Rotation Procedures
79
-
80
- **Rotation Process:**
81
- 1. Add new key version to configuration
82
- 2. Update `current_key_version`
83
- 3. Re-encrypt existing data using `re_encrypt_fields!`
84
- 4. Verify migration completion
85
- 5. Remove old keys after migration complete
86
-
87
- **Example Rotation Script:**
88
- ```ruby
89
- # Step 1: Add new key version
90
- Familia.configure do |config|
91
- config.encryption_keys = {
92
- v2: ENV['OLD_ENCRYPTION_KEY'],
93
- v3: ENV['NEW_ENCRYPTION_KEY']
94
- }
95
- config.current_key_version = :v3
96
- end
97
-
98
- # Step 2: Validate configuration
99
- Familia::Encryption.validate_configuration!
100
-
101
- # Step 3: Re-encrypt existing records
102
- Vault.all.each do |vault|
103
- vault.re_encrypt_fields! # Re-encrypts with current key
104
- vault.save
105
- end
106
-
107
- # Step 4: Verify migration
108
- Vault.all.each do |vault|
109
- status = vault.encrypted_fields_status
110
- puts "Vault #{vault.identifier}: #{status}"
111
- end
112
-
113
- # Step 5: Remove old key (after verification)
114
- Familia.config.encryption_keys.delete(:v2)
115
- ```
116
-
117
- ## Security Best Practices
118
-
119
- - **Environment Variables:** Store keys in environment variables, not code
120
- - **Key Rotation:** Rotate encryption keys regularly (quarterly/annually)
121
- - **Field Selection:** Only encrypt fields that truly need protection
122
- - **Memory Clearing:** Use transient fields for temporary sensitive data
123
- - **Logging:** Verify ConcealedString prevents accidental logging
124
- - **Configuration Validation:** Use `validate_configuration!` before production
125
- - **Monitoring:** Use `encrypted_fields_status` to track encryption state
126
-
127
- ## Next Steps
128
-
129
- After implementing security features:
130
- 1. Review [Architecture Migration](v2.0.0-pre6.md) for persistence improvements
131
- 2. Explore [Relationships Migration](v2.0.0-pre7.md) for the relationship system
@@ -1,154 +0,0 @@
1
- # Migrating Guide: Architecture Improvements (v2.0.0-pre6)
2
-
3
- This guide covers the architecture enhancements and new persistence methods in v2.0.0-pre6.
4
-
5
- ## Architecture Improvements
6
-
7
- ### 1. Enhanced Persistence Operations
8
-
9
- **New `save_if_not_exists` Method:**
10
- ```ruby
11
- user = User.new(email: 'user@example.com')
12
-
13
- # Only save if the user doesn't already exist
14
- if user.save_if_not_exists
15
- puts "User created successfully"
16
- else
17
- puts "User already exists"
18
- end
19
- ```
20
-
21
- **Atomic Persistence with Transactions:**
22
- ```ruby
23
- user.transaction do |conn|
24
- conn.set(user.key, user.serialize)
25
- conn.sadd("all_users", user.identifier)
26
- conn.expire(user.key, user.ttl) if user.ttl
27
- end
28
- ```
29
-
30
- ### 2. Modular Class Structure
31
-
32
- The Horreum class structure was reorganized for better maintainability:
33
-
34
- **Core Modules:**
35
- - `Familia::Horreum::Core` - Essential functionality
36
- - `Familia::Horreum::ClassMethods` - Class-level methods
37
- - `Familia::Horreum::Serialization` - Object serialization
38
- - `Familia::Horreum::Commands` - Valkey/Redis command wrappers
39
-
40
- **Feature System Improvements:**
41
- - Dependency management between features
42
- - Cleaner feature activation
43
- - Better error handling for missing dependencies
44
-
45
- ### 3. Enhanced Error Handling
46
-
47
- **New Exception Types:**
48
- ```ruby
49
- begin
50
- user.save!
51
- rescue Familia::PersistenceError => e
52
- puts "Failed to save: #{e.message}"
53
- rescue Familia::ValidationError => e
54
- puts "Validation failed: #{e.message}"
55
- end
56
- ```
57
-
58
- **Improved Data Consistency:**
59
- - Automatic retry for transient Valkey/Redis connection issues
60
- - Better handling of concurrent modifications
61
- - Enhanced validation before persistence operations
62
-
63
- ## Migration Steps
64
-
65
- ### 1. Update Error Handling
66
-
67
- **Before (v2.0.0-pre5):**
68
- ```ruby
69
- begin
70
- user.save
71
- rescue => e
72
- puts "Something went wrong: #{e}"
73
- end
74
- ```
75
-
76
- **After (v2.0.0-pre6):**
77
- ```ruby
78
- begin
79
- user.save
80
- rescue Familia::PersistenceError => e
81
- puts "Persistence failed: #{e.message}"
82
- # Handle specific persistence issues
83
- rescue Familia::ValidationError => e
84
- puts "Invalid data: #{e.message}"
85
- # Handle validation failures
86
- end
87
- ```
88
-
89
- ### 2. Adopt Conditional Persistence
90
-
91
- Replace existence checks with atomic operations:
92
-
93
- **Before:**
94
- ```ruby
95
- user = User.new(email: email)
96
- unless User.exists?(email)
97
- user.save
98
- end
99
- ```
100
-
101
- **After:**
102
- ```ruby
103
- user = User.new(email: email)
104
- user.save_if_not_exists
105
- ```
106
-
107
- ### 3. Leverage Transaction Support
108
-
109
- For complex operations, use transactions:
110
-
111
- ```ruby
112
- # Before - Multiple separate operations
113
- user.save
114
- user.tags.add(tag)
115
- user.scores.add(score, timestamp)
116
-
117
- # After - Atomic transaction
118
- user.transaction do |conn|
119
- conn.set(user.key, user.serialize)
120
- conn.sadd(user.tags.key, tag)
121
- conn.zadd(user.scores.key, timestamp, score)
122
- end
123
- ```
124
-
125
- ## Performance Improvements
126
-
127
- ### Connection Management
128
- - Improved connection pooling with better resource utilization
129
- - Reduced connection overhead through intelligent connection reuse
130
- - Enhanced concurrent operation support
131
-
132
- ### Feature System
133
- - Lazy feature loading reduces memory footprint
134
- - Optimized method dispatch for feature methods
135
- - Better dependency resolution
136
-
137
- ## Breaking Changes
138
-
139
- ### Method Signatures
140
- - Some internal methods changed signatures for better consistency
141
- - Error handling improved with specific exception types
142
- - Transaction block interface standardized
143
-
144
- ### Feature Dependencies
145
- - Features now explicitly declare dependencies
146
- - Better error messages for missing feature requirements
147
- - Automatic dependency resolution where possible
148
-
149
- ## Next Steps
150
-
151
- After completing architecture migration:
152
- 1. Explore [Relationships Migration](v2.0.0-pre7.md) for the comprehensive relationship system
153
- 2. Review updated documentation for architectural patterns
154
- 3. Consider adopting new persistence patterns in your application
@@ -1,222 +0,0 @@
1
- # Migrating Guide: Relationships System (v2.0.0-pre7)
2
-
3
- This guide covers adopting the comprehensive relationships system introduced in v2.0.0-pre7.
4
-
5
- ## Relationships System Overview
6
-
7
- The v2.0.0-pre7 release introduces three powerful relationship types:
8
-
9
- - **`tracked_in`** - Multi-presence tracking with score encoding
10
- - **`indexed_by`** - O(1) hash-based lookups
11
- - **`member_of`** - Bidirectional membership with collision-free naming
12
-
13
- ## Migration Steps
14
-
15
- ### 1. Enable Relationships Feature
16
-
17
- Add the relationships feature to your models:
18
-
19
- ```ruby
20
- class Customer < Familia::Horreum
21
- feature :relationships # Required for all relationship types
22
-
23
- identifier_field :custid
24
- field :custid, :name, :email
25
- end
26
- ```
27
-
28
- ### 2. Choose Appropriate Relationship Types
29
-
30
- **For Tracking Collections (sorted sets):**
31
- ```ruby
32
- class Customer < Familia::Horreum
33
- feature :relationships
34
-
35
- # Global tracking with score-based sorting
36
- tracked_in :all_customers, type: :sorted_set, score: :created_at
37
-
38
- # Scoped tracking
39
- tracked_in :active_users, type: :sorted_set, score: :last_seen
40
- end
41
- ```
42
-
43
- **For Fast Lookups (hash indexes):**
44
- ```ruby
45
- class Customer < Familia::Horreum
46
- feature :relationships
47
-
48
- # Global email index for O(1) lookups
49
- indexed_by :email, :email_lookup, target: :global
50
-
51
- # Domain-specific indexes
52
- indexed_by :account_id, :account_lookup, target: Domain
53
- end
54
- ```
55
-
56
- **For Bidirectional Membership:**
57
- ```ruby
58
- class Domain < Familia::Horreum
59
- feature :relationships
60
-
61
- # Belongs to customer's domains collection
62
- participates_in Customer, :domains, type: :set
63
- end
64
- ```
65
-
66
- ### 3. Implement Permission Systems
67
-
68
- For access-controlled relationships:
69
-
70
- ```ruby
71
- class Document < Familia::Horreum
72
- feature :relationships
73
- include Familia::Features::Relationships::PermissionManagement
74
-
75
- # Permission-aware tracking
76
- tracked_in Customer, :documents, score: :created_at
77
-
78
- permission_tracking :user_permissions
79
-
80
- def permission_bits
81
- # Define permission levels (bit-encoded)
82
- @permission_bits || 1 # Default: read-only
83
- end
84
- end
85
- ```
86
-
87
- ### 4. Update Queries
88
-
89
- **Before (manual db operations):**
90
- ```ruby
91
- # Manual sorted set operations
92
- redis.zadd("all_customers", customer.created_at, customer.custid)
93
- customers = redis.zrange("all_customers", 0, -1)
94
- ```
95
-
96
- **After (relationship methods):**
97
- ```ruby
98
- # Automatic method generation
99
- Customer.add_to_all_customers(customer)
100
- customers = Customer.all_customers.to_a
101
- ```
102
-
103
- ## Relationship Method Patterns
104
-
105
- ### `tracked_in` Methods
106
- ```ruby
107
- # Class methods generated:
108
- Customer.add_to_all_customers(customer)
109
- Customer.remove_from_all_customers(customer)
110
- Customer.all_customers # Access sorted set directly
111
- ```
112
-
113
- ### `indexed_by` Methods
114
- ```ruby
115
- # Class methods generated:
116
- Customer.add_to_email_lookup(customer)
117
- Customer.remove_from_email_lookup(customer)
118
- Customer.email_lookup.get(email) # O(1) lookup
119
- ```
120
-
121
- ### `member_of` Methods
122
- ```ruby
123
- # Instance methods generated:
124
- domain.add_to_customer_domains(customer)
125
- domain.remove_from_customer_domains(customer)
126
- domain.in_customer_domains?(customer)
127
- ```
128
-
129
- ## Permission System Migration
130
-
131
- ### Basic Permission Encoding
132
- ```ruby
133
- # Permission levels (bit-encoded)
134
- READ = 1 # 001
135
- WRITE = 4 # 100
136
- DELETE = 32 # 100000
137
- ADMIN = 128 # 10000000
138
-
139
- # Combine permissions
140
- user_permissions = READ | WRITE # Can read and write
141
- admin_permissions = ADMIN # All permissions via hierarchy
142
- ```
143
-
144
- ### Time-Based Permission Scores
145
- ```ruby
146
- # Encode timestamp with permission bits
147
- timestamp = Familia.now.to_i
148
- permissions = READ | WRITE
149
- score = Familia::Features::Relationships::ScoreEncoding.encode_score(timestamp, permissions)
150
-
151
- # Query by permission level
152
- documents_with_write_access = customer.documents.accessible_items(
153
- permissions: WRITE,
154
- collection_key: customer.documents.key
155
- )
156
- ```
157
-
158
- ## Performance Considerations
159
-
160
- ### Relationship Type Selection
161
-
162
- **Use `indexed_by` for:**
163
- - Unique field lookups (email, username, ID)
164
- - Fields that need O(1) access
165
- - Reference relationships
166
-
167
- **Use `tracked_in` for:**
168
- - Time-ordered collections
169
- - Scored/ranked relationships
170
- - Permission-based access control
171
-
172
- **Use `member_of` for:**
173
- - Simple membership tracking
174
- - Bidirectional relationships
175
- - UnsortedSet-based collections
176
-
177
- ### Optimization Tips
178
-
179
- **Batch Operations:**
180
- ```ruby
181
- # Instead of multiple individual operations
182
- customers.each { |c| Customer.add_to_all_customers(c) }
183
-
184
- # Use batch operations where available
185
- Customer.all_customers.add_batch(customers.map(&:identifier),
186
- customers.map(&:created_at))
187
- ```
188
-
189
- **Permission Queries:**
190
- ```ruby
191
- # Efficient permission filtering
192
- accessible_docs = document.accessible_items(
193
- collection_key: customer.documents.key,
194
- minimum_permissions: READ
195
- )
196
- ```
197
-
198
- ## Breaking Changes
199
-
200
- ### Method Name Changes
201
- - Relationship methods follow new naming patterns
202
- - Previous manual db operations need updating
203
- - Collection access methods standardized
204
-
205
- ### Permission System
206
- - New bit-encoded permission system
207
- - Time-based scoring for temporal access
208
- - Permission hierarchy implementation required
209
-
210
- ## Next Steps
211
-
212
- 1. **Analyze Existing Relationships:** Review current db operations for optimization opportunities
213
- 2. **Choose Relationship Types:** Select appropriate types based on usage patterns
214
- 3. **Implement Permissions:** Add access control where needed
215
- 4. **Update Queries:** Replace manual operations with generated relationship methods
216
- 5. **Test Performance:** Benchmark relationship operations under load
217
-
218
- ## Resources
219
-
220
- - [Relationships Methods Guide](../guides/relationships-methods.md) - Complete method reference
221
- - [Relationships Guide](../guides/Relationships-Guide.md) - Implementation examples
222
- - [Performance Guide](../guides/Implementation-Guide.md) - Optimization strategies