familia 2.12.0 → 2.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (96) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/claude-code-review.yml +9 -10
  3. data/.github/workflows/claude.yml +1 -1
  4. data/.github/workflows/release-gem.yml +1 -1
  5. data/.gitignore +1 -0
  6. data/AGENTS.md +21 -0
  7. data/CHANGELOG.rst +48 -147
  8. data/Gemfile +1 -1
  9. data/Gemfile.lock +11 -11
  10. data/changelog.d/fragments/20260901_221800_delano_find_by_id_legacy_string.rst +17 -0
  11. data/changelog.d/fragments/20260902_043148_delano_406_valid_debug_log.md +7 -0
  12. data/changelog.d/fragments/20260902_043513_delano_407_deserialization_log_redaction.md +7 -0
  13. data/changelog.d/fragments/20260902_120000_delano_405_envelope_provenance.rst +32 -0
  14. data/changelog.d/fragments/20260902_130000_delano_408_envelope_encoding_validation.rst +26 -0
  15. data/changelog.d/fragments/20260904_120000_delano_security_audit_anchors.rst +4 -0
  16. data/changelog.d/fragments/20260904_150000_delano_encryption_identifier_precondition.rst +6 -0
  17. data/changelog.d/fragments/20260904_160000_delano_rebuild_via_scan_failure.rst +4 -0
  18. data/changelog.d/fragments/20260905_000000_delano_external_identifier_format_validation.rst +4 -0
  19. data/changelog.d/fragments/20260905_000500_delano_concurrent_rebuild_temp_key.rst +78 -0
  20. data/changelog.d/fragments/20260905_111500_delano_identifier_secret_env.rst +16 -0
  21. data/changelog.d/fragments/20260905_213000_delano_related_field_lifecycle.rst +104 -0
  22. data/changelog.d/fragments/20260906_203000_delano_unique_index_rebuild_fallback.rst +10 -0
  23. data/docs/guides/datatype-collections.md +57 -1
  24. data/docs/guides/feature-encrypted-fields.md +23 -0
  25. data/docs/guides/feature-external-identifiers.md +4 -0
  26. data/docs/guides/feature-relationships-indexing.md +65 -0
  27. data/docs/investigation/memory-audit.md +34 -0
  28. data/docs/investigation/rebuild-memory-incident.md +136 -0
  29. data/docs/investigation/v2-passphrase-not-bound-to-ciphertext.md +163 -0
  30. data/docs/migrating/identifier-secret.md +36 -0
  31. data/docs/reference/api-technical.md +17 -0
  32. data/docs/reference/migrations-specification.md +703 -0
  33. data/docs/security/2026-07-19-audit.md +61 -34
  34. data/docs/security/2026-07-26-audit.md +113 -0
  35. data/docs/security/2026-07-30-audit.md +38 -10
  36. data/docs/security/2026-08-06-audit.md +110 -0
  37. data/docs/security/2026-09-04-audit.md +106 -0
  38. data/docs/security/README.md +25 -0
  39. data/examples/encryption_upgrade_proof/README.md +6 -3
  40. data/examples/encryption_upgrade_proof/phase1_gem_upgrade_no_libsodium.rb +4 -1
  41. data/examples/encryption_upgrade_proof/phase2_libsodium_enabled.rb +11 -7
  42. data/lib/familia/atomic_operations.rb +399 -36
  43. data/lib/familia/data_type/class_methods.rb +33 -0
  44. data/lib/familia/data_type.rb +4 -16
  45. data/lib/familia/encryption/encrypted_data.rb +34 -4
  46. data/lib/familia/encryption/manager.rb +9 -2
  47. data/lib/familia/encryption/stored_envelope.rb +28 -0
  48. data/lib/familia/encryption.rb +1 -0
  49. data/lib/familia/errors.rb +33 -0
  50. data/lib/familia/features/encrypted_fields/encrypted_field_type.rb +66 -14
  51. data/lib/familia/features/expiration.rb +5 -4
  52. data/lib/familia/features/external_identifier.rb +24 -10
  53. data/lib/familia/features/relationships/indexing/rebuild_strategies.rb +110 -98
  54. data/lib/familia/features/relationships/indexing/unique_index_generators.rb +21 -14
  55. data/lib/familia/field_type.rb +8 -0
  56. data/lib/familia/horreum/atomic_write.rb +2 -1
  57. data/lib/familia/horreum/definition.rb +76 -0
  58. data/lib/familia/horreum/management/repair.rb +24 -23
  59. data/lib/familia/horreum/management.rb +3 -2
  60. data/lib/familia/horreum/persistence.rb +3 -2
  61. data/lib/familia/horreum/related_fields.rb +213 -13
  62. data/lib/familia/horreum/serialization.rb +39 -12
  63. data/lib/familia/horreum.rb +123 -45
  64. data/lib/familia/thread_safety/instrumented_mutex.rb +5 -2
  65. data/lib/familia/verifiable_identifier.rb +60 -25
  66. data/lib/familia/version.rb +1 -1
  67. data/try/edge_cases/legacy_data_detection/deserialization_edge_cases_try.rb +2 -1
  68. data/try/edge_cases/legacy_data_detection/deserialization_log_redaction_try.rb +144 -0
  69. data/try/edge_cases/legacy_data_detection/find_by_id_legacy_string_try.rb +76 -0
  70. data/try/features/encrypted_fields/envelope_provenance_try.rb +151 -0
  71. data/try/features/encrypted_fields/identifier_precondition_try.rb +80 -0
  72. data/try/features/encryption/algorithm_upgrade_try.rb +5 -3
  73. data/try/features/encryption/encrypted_data_valid_logging_try.rb +145 -0
  74. data/try/features/encryption/envelope_encoding_validation_try.rb +135 -0
  75. data/try/features/expiration/long_ttl_try.rb +196 -0
  76. data/try/features/external_identifier/external_identifier_try.rb +24 -5
  77. data/try/features/relationships/indexing_rebuild_try.rb +60 -0
  78. data/try/features/relationships/rebuild_via_scan_try.rb +102 -0
  79. data/try/features/relationships/unique_index_fallback_try.rb +132 -0
  80. data/try/integration/familia_extended_try.rb +1 -1
  81. data/try/integration/verifiable_identifier_try.rb +255 -0
  82. data/try/investigation/memory_leak_proof.rb +20 -13
  83. data/try/performance/transaction_safety_benchmark_try.rb +51 -27
  84. data/try/support/debugging/debug_aad_process.rb +2 -2
  85. data/try/thread_safety/instrumented_mutex_exclusion_try.rb +88 -0
  86. data/try/unit/atomic_operations_try.rb +487 -3
  87. data/try/unit/horreum/related_field_inheritance_snapshot_try.rb +122 -0
  88. data/try/unit/horreum/related_field_lifecycle_try.rb +1109 -0
  89. metadata +38 -8
  90. data/changelog.d/fragments/20260725_202500_delano_282_unsaved_index_guard.md +0 -7
  91. data/changelog.d/fragments/20260725_213000_delano_282_index_scope_tracker.md +0 -18
  92. data/changelog.d/fragments/20260728_120000_delano_365_stale_tracker_prune.rst +0 -30
  93. data/docs/qodo-merge-compliance.md +0 -96
  94. /data/docs/{schema-validation.md → guides/schema-validation.md} +0 -0
  95. /data/docs/{1106-participates_in-bidirectional-solution.md → investigation/participates_in-bidirectional-solution.md} +0 -0
  96. /data/docs/{transaction_safety.md → reference/transaction_safety.md} +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d860c29966392e82cf54d842d457353d6a7268c3bf5378e25af9cb850f59d37c
4
- data.tar.gz: 87fc36560a6cd68ffb614ef6b3ab11625300ebed8433508739f1ab4633f5b9ff
3
+ metadata.gz: 889cca334ccb762d4a5cb4fa64a400c3df8727a8219bf19129b6699bf4056069
4
+ data.tar.gz: c2e315e109a866d8b1943be21fb7cf215c3db767e801138fa4097229e98c5dcb
5
5
  SHA512:
6
- metadata.gz: 15b5c9028503566db28d179052982d3a38617463753ea360f22ea49eeb86d3fc896352f18c73a703fe62d3fd5a6cdcf65f254dab284a7671f3b9a0b9db413acb
7
- data.tar.gz: 633e8ae2bab11aaf362791f23f68ea7f0287e1388c38850200809c5b43ed79ae59039492258927023ea91821d39243a18c9eaa5a63d8c068fb011edc99120468
6
+ metadata.gz: d7e65ba3aa75c03d7c0328a34f7c6ad13f9c840efc2aefbe0c06bba678a41ac27704928962b5d9a3be769bd7df186bf1bc56647ee80aa173a1000ff6404d599e
7
+ data.tar.gz: e34f5200cac7681ff999b31cf842d326e49882ffd823602d8d677bec95b6376edca29f967f9da494eb431f9a857bc81758519dd4fd5969714ed478bd085cdf54
@@ -48,7 +48,7 @@ jobs:
48
48
 
49
49
  - name: Run Claude Code Review
50
50
  id: claude-review
51
- uses: anthropics/claude-code-action@be7b93b1907a4abad570368f3c74b6fe3807510b # v1.0.183
51
+ uses: anthropics/claude-code-action@833fb0f8c9f6686b33d963a8bae0a94f4936ab2a # v1.0.211
52
52
  with:
53
53
  claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
54
54
 
@@ -62,17 +62,18 @@ jobs:
62
62
  claude_args: |
63
63
  --model ${{ inputs.model || vars.CLAUDE_MODEL || 'claude-opus-4-6' }}
64
64
  --fallback-model ${{ vars.CLAUDE_FALLBACK_MODEL || 'claude-sonnet-4-6' }}
65
- --allowedTools "Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"
65
+ --allowedTools "Bash(gh issue view:*),Bash(gh search:*),Bash(gh issue list:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*)"
66
66
 
67
- # Optional: Use sticky comments to make Claude reuse the same comment on subsequent pushes to the same PR
67
+ # A bare `prompt` puts the action in agent mode, which creates no tracking
68
+ # comment, so use_sticky_comment alone did nothing and every push got a
69
+ # fresh `gh pr comment`. track_progress forces tag mode: the action owns
70
+ # one tracking comment, use_sticky_comment finds and reuses it on later
71
+ # pushes, and Claude writes the review into it via the action's built-in
72
+ # comment tool. The prompt becomes <custom_instructions> in tag mode.
73
+ track_progress: true
68
74
  use_sticky_comment: true
69
75
 
70
- # v1 renamed direct_prompt -> prompt. The REPO/PR NUMBER header is the
71
- # format the migration guide calls for so Claude reviews the right PR.
72
76
  prompt: |
73
- REPO: ${{ github.repository }}
74
- PR NUMBER: ${{ github.event.pull_request.number }}
75
-
76
77
  Please review this pull request and provide feedback on:
77
78
  - Code quality and best practices
78
79
  - Potential bugs or issues
@@ -81,5 +82,3 @@ jobs:
81
82
  - Test coverage
82
83
 
83
84
  Use the repository's AGENTS.md for guidance on style and conventions. Be constructive and helpful in your feedback.
84
-
85
- Use `gh pr comment` with your Bash tool to leave your review as a comment on the PR.
@@ -32,7 +32,7 @@ jobs:
32
32
 
33
33
  - name: Run Claude Code
34
34
  id: claude
35
- uses: anthropics/claude-code-action@be7b93b1907a4abad570368f3c74b6fe3807510b # v1.0.183
35
+ uses: anthropics/claude-code-action@833fb0f8c9f6686b33d963a8bae0a94f4936ab2a # v1.0.211
36
36
  with:
37
37
  claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
38
38
 
@@ -158,4 +158,4 @@ jobs:
158
158
  echo "Releasing familia ${gem_version} from tag ${RELEASE_TAG}"
159
159
 
160
160
  - name: Build and push gem to RubyGems
161
- uses: rubygems/release-gem@052cc82692552de3ef2b81fd670e41d13cba8092 # v1.4.0
161
+ uses: rubygems/release-gem@7f9650160c1a4e7989fdc9855807bdbd421d8b6b # v1.4.1
data/.gitignore CHANGED
@@ -18,6 +18,7 @@
18
18
  dump.rdb
19
19
  appendonlydir
20
20
  data
21
+ doc
21
22
  log
22
23
  tmp
23
24
  vendor
data/AGENTS.md CHANGED
@@ -11,6 +11,8 @@ Guidance for AI coding agents working in this repository.
11
11
 
12
12
  ### Testing (Tryouts v3)
13
13
 
14
+ Use `valkey-server try/valkey.conf` to start the test redis.
15
+
14
16
  Each file has optional setup, testcases, and optional teardown. A testcase is a
15
17
  `##` description line, Ruby code, then one or more expectation comments
16
18
  (`#=>`, `#==>`, `#=:>`, `#=!>`, ...). The last expression is the result.
@@ -203,3 +205,22 @@ last-write timestamps (ZADD score), not a registry.
203
205
  DataType instances are frozen (immutable). Configure module-level settings once
204
206
  at startup, before threads spawn. `Familia.start_monitoring!` tracks contention.
205
207
  Tests and contention patterns live in `try/thread_safety/`.
208
+
209
+ ## Project claims and source authority
210
+
211
+ ### Attribution
212
+
213
+ Do not infer project terminology or guarantees from repetition. Before attributing a claim to the project, locate an authoritative primary source and provide its exact wording. Treat delivery notes, commit messages, agent output, and documents created or modified during the current task as leads, not evidence. If the wording is absent, call it an interpretation or proposal. Never place paraphrases in quotation marks.
214
+
215
+ ### Authoritative sources
216
+
217
+ Authoritative sources must be identified explicitly; repository presence alone does not confer authority. Accepted specifications and ADRs may establish project claims only within their stated scope. Delivery notes, commit messages, issue discussions, summaries, and agent-authored text are non-authoritative unless an authoritative source incorporates them explicitly.
218
+
219
+ ### Normative claims
220
+
221
+ For normative claims concerning security, privacy, compatibility, persistence, or data loss:
222
+
223
+ 1. Cite the authoritative source and its exact wording.
224
+ 2. Distinguish quotations, paraphrases, interpretations, and proposals.
225
+ 3. Do not use material created or modified during the current task to validate that task’s claims.
226
+ 4. If no authoritative wording exists, report the claim as unsupported.
data/CHANGELOG.rst CHANGED
@@ -15,83 +15,63 @@ The format is based on `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`
15
15
  Added
16
16
  -----
17
17
 
18
- - Added ``Familia::HashKey#claim_field`` and ``#release_field`` for server-side compare-and-set/compare-and-delete on single hash fields. Raises ``Familia::OperationModeError`` in pipelines/transactions.
19
- - Added generated ``claim_unique_<index>!`` and ``release_unique_<index>!`` instance methods with automatic partial claim rollback on unique index save collisions. #353
20
- - Added tryout test helpers (``set_test_encryption_keys``, ``clear_test_encryption_keys``, ``with_test_encryption_keys``) to safely manage encryption key configurations during test runs. #363
21
- - Added ``Familia::MultiResult#aborted?`` to distinguish WATCH aborts from individual command errors.
22
- - Added ``Familia::MultiResult#inspect`` for log-safe, compact transaction outcome summaries.
23
- - Added ``max_length:`` option to ``SortedSet`` and ``ListKey`` to automatically cap collection sizes at write time (retaining newest N elements). #351
24
- - Added ``DataType#max_length`` to query configured collection caps, with definition-time validation of the parameter. #351
25
- - Added ``SortedSet#enforce_max_length!`` and ``ListKey#enforce_max_length!`` to trim existing collections. #351
26
- - Added ``max_length:`` option to ``participates_in`` and ``class_participates_in``. #351
27
- - Added ``Familia::Features::Housekeeping::EnforceCollectionCaps`` chore class to support bulk cap enforcement. #351
28
- - Added ``encryption_personalization_history`` setting to support key personalization rotation for XChaCha20-Poly1305 providers. #333
29
- - Added ``limit:``, ``offset:``, and ``each_<collection>_with_permission`` to stream permission-filtered collection members via ``ZSCAN`` with O(1) memory. #309
18
+ - Added ``Familia::HashKey#claim_field`` and ``#release_field`` for single-field compare-and-set and compare-and-delete. They raise ``Familia::OperationModeError`` in pipelines and transactions.
19
+ - Added generated ``claim_unique_<index>!`` and ``release_unique_<index>!`` methods for class-level unique indexes. #353
20
+ - Added ``Familia::MultiResult#aborted?`` and ``#inspect`` for transaction outcome handling and logging.
21
+ - Added ``max_length:`` to ``ListKey``, ``SortedSet``, ``participates_in``, and ``class_participates_in``. ``ListKey`` retains the N elements nearest the end written to, while ``SortedSet`` retains its N highest-scoring members. Existing oversized collections are trimmed on the next capped write; call ``enforce_max_length!`` or run ``Familia::Features::Housekeeping::EnforceCollectionCaps`` to enforce a new cap immediately. ``DataType#max_length`` exposes the configured cap. #351
22
+ - Added ``encryption_personalization_history`` to support XChaCha20-Poly1305 personalization rotation. See ``docs/guides/encryption.md``. #333
23
+ - Added ``limit:``, ``offset:``, and ``each_<collection>_with_permission`` for memory-bounded permission-filtered collection queries. #309
24
+ - Added the ``dirty_write_warnings:`` collection option to override a parent class's diagnostic mode. #282
30
25
 
31
26
  Changed
32
27
  -------
33
28
 
34
- - Unique index updates (``add_to_class_<index>``, ``update_in_class_<index>``) now raise ``Familia::OperationModeError`` inside transactions unless the written value was previously claimed. #353
35
- - Retained ``guard_unique_indexes!`` for fast-fail read checks before acquiring index claims. #353
36
- - Calls to ``remove_from_class_*`` and class-level ``destroy!`` no longer evict index entries that point to different identifiers. #353
37
- - ``Familia::MultiResult#results`` now always returns an ``Array`` (empty instead of ``nil`` on transaction abort) to prevent upstream crashes.
38
- - ``Familia::MultiResult#to_h`` now includes an ``:aborted`` boolean key.
39
- - ``Familia::MultiResult`` instances are now read-only, freezing both ``#results`` and ``#errors``.
40
- - ``SortedSet#increment`` now runs the ``warn_if_dirty!`` guard, warning or raising when called on an unsaved parent. #351
41
- - ``Manager#decrypt`` now handles XChaCha20 provider personalization candidates correctly during decryption walks, matching AES-GCM salt history behavior. #333
42
- - The request-scoped key cache key now includes a personalization segment to prevent collisions across different rotation candidates. #333
43
- - ``multi_field_fast_write`` now raises ``Familia::IndexedFieldFastWriteError`` if any written field backs a class-level index. #308
44
- - Fast writer ``field!`` on a class-indexed field now raises ``Familia::IndexedFieldFastWriteError`` inside transactions/pipelines, and raises ``Familia::PersistenceError`` on unsaved records. #308
45
- - With a blank ``encryption_hkdf_salt``, ``Manager#encrypt`` now raises ``EncryptionError`` when the request-cache key is built rather than at key derivation. The raise happens earlier and now also fires on what would previously have been a warm-cache hit; correctly configured deployments see no change. #380
29
+ - Class-level unique-index updates inside transactions now require a prior claim and raise ``Familia::OperationModeError`` otherwise. #353
30
+ - Class-level index removal and destruction no longer delete entries owned by other records. #353
31
+ - ``Familia::MultiResult#results`` now returns an empty ``Array`` on transaction aborts, ``#to_h`` includes ``:aborted``, and result objects are read-only.
32
+ - ``SortedSet#increment`` now applies the dirty-write guard. #351
33
+ - XChaCha20 decryption and request caching now support personalization rotation candidates. #333
34
+ - ``multi_field_fast_write`` rejects class-indexed fields, and ``field!`` rejects class-indexed fields in transactions or pipelines and unsaved records. #308
35
+ - Registered instance-scoped index memberships refresh on ``save``. ``atomic_write`` callers must update those indexes explicitly. #282
36
+ - Saving a record with an instance-scoped unique-index collision now raises ``Familia::RecordExistsError`` without replacing the existing entry. #282
37
+ - Direct instance- and class-scoped index mutations now require a persisted record. #282
38
+ - Instance-scoped index mutations now reject scopes without an identifier or a Symbol/String ``identifier_field``. #282
46
39
 
47
40
  Removed
48
41
  -------
49
42
 
50
- - Removed the unused ``ScoreEncoding.category_score_range`` method.
51
43
  - Removed unused ``csv`` and ``stringio`` runtime dependencies from the gemspec. #354
52
- - Removed dead private helper methods ``Horreum::define_attr_accessor_methods``, ``remove_stale_collection_member``, and ``define_fast_writer_method``. #347, #308
53
44
 
54
45
  Fixed
55
46
  -----
56
47
 
57
- - Fixed ``decrby`` and ``decr`` (and aliases) to correctly use ``HINCRBY`` with a negated amount and added client-side integer validation.
58
- - Fixed ``encryption_info`` to reference the correct provider APIs and accurately expose ``key_size``.
59
- - Fixed a TOCTOU race in ``unique_index`` by enforcing server-side CAS claims using Lua before opening transactions, raising ``Familia::RecordExistsError`` on collision. #353
60
- - Fixed ``remove_from_class_*`` and ``update_in_class_*`` to use ownership-checked deletes, preventing accidental deletion of other records' valid entries. #353
61
- - Fixed process-global encryption configuration leaks across tryouts by using scoped helpers and restoring baselines. #363
62
- - Fixed test isolation in ``real_feature_integration_try.rb`` and ``module_loading_try.rb`` by explicitly managing encryption key rings. #363
63
- - Fixed fiber-local key cache teardowns to clear the correct request cache key. #363
64
- - Fixed claim leaks on unpersisted records in ``update_in_<scope>_<index_name>`` by running the persisted-record guard before checking claims. #370
65
- - Fixed ``ConcealedString`` by removing a redundant and non-functional GC finalizer. #359
66
- - Fixed ``Familia::MultiResult`` to prevent ``NoMethodError`` crashes on WATCH-aborted transactions. #355
67
- - Fixed ``SecureXChaCha20Poly1305Provider#derive_key`` to avoid mutating the receiver context string and handle non-String contexts correctly. #356
68
- - Fixed ``SortedSet#increment`` inside transactions/pipelines to safely return the future instead of calling ``to_f`` prematurely. #351
69
- - Fixed long-ignored ``:maxlength`` spelling to explicitly warn at definition time, prompting rename to ``max_length:``. #351
70
- - Fixed ``Lock#acquire`` with a positive TTL to run as a single atomic ``SET NX EX`` command. #347
71
- - Fixed ``DatabaseLogger.sample_rate`` assignment to validate and clamp input values, preventing crashes on the hot path. #347
72
- - Fixed ``SingleUseRedactedString`` to be loaded automatically by ``require 'familia'``. #347
73
- - Fixed memory overhead in ``<collection>_with_permission`` queries by batching and paging internally via ``ZRANGEBYSCORE ... LIMIT``. #309
74
- - Fixed performance by removing redundant post-save ``update_all_indexes`` calls inside relationships save. #307
75
- - Fixed partial write paths (``commit_fields``, ``save_fields``, ``multi_field_update``) to safely guard and claim unique indexes before executing writes. #308
76
- - Fixed fast writers (``field!``) on indexed fields to claim and update the index atomically before execution. #308
48
+ - Fixed ``decrby`` and ``decr`` (and aliases) to decrement hash fields correctly and validate integer amounts.
49
+ - Fixed ``encryption_info`` to report correct provider details and key sizes.
50
+ - Fixed unique-index save races and ownership-checked removals, preventing collisions from replacing or deleting another record's entry. #353
51
+ - Fixed claim leaks when unpersisted records call ``update_in_<scope>_<index_name>``. #370
52
+ - Fixed ``Familia::MultiResult`` handling of WATCH-aborted transactions. #355
53
+ - Fixed XChaCha20 key derivation to preserve caller-provided context values. #356
54
+ - Fixed ``SortedSet#increment`` in transactions and pipelines to return the queued result. #351
55
+ - ``:maxlength`` now warns at definition time; rename it to ``max_length:`` to opt in to collection capping. #351
56
+ - Fixed ``Lock#acquire`` with a TTL to acquire and set the expiration atomically. #347
57
+ - Fixed ``DatabaseLogger.sample_rate`` validation and ``SingleUseRedactedString`` loading through ``require 'familia'``. #347
58
+ - Fixed memory use in ``<collection>_with_permission`` queries by processing results in bounded batches. #309
59
+ - Fixed invalid permission-symbol lookups to fail rather than silently misconfigure permission queries.
60
+ - Fixed partial and fast writes to maintain class-level unique indexes atomically. #308
61
+ - Fixed instance-scoped index cleanup after indexed values change, scope classes share an index name, or identifiers are reused. #282, #365
77
62
 
78
63
  Security
79
64
  --------
80
65
 
81
- - Enforced field-type semantics in ``multi_field_update`` and ``multi_field_fast_write`` to prevent writing plaintext to encrypted fields or persisting transient fields.
82
- - Enforced loud failures for invalid permission symbol lookups in ``Relationships::ScoreEncoding`` to prevent silent misconfigurations.
66
+ - ``multi_field_update`` and ``multi_field_fast_write`` now reject plaintext writes to encrypted fields and writes to transient fields.
83
67
  - Pinned GitHub Workflows holding ``CLAUDE_CODE_OAUTH_TOKEN`` to immutable release commit SHAs.
84
- - The encrypt-path request-cache key now resolves the HKDF salt through the fail-closed ``current_hkdf_salt`` accessor instead of the permissive candidate list's head (``hkdf_salts.first``). Previously, with a blank ``encryption_hkdf_salt``, a decrypt inside ``with_request_cache`` could warm an entry keyed on a historical salt that a subsequent encrypt would silently reuse -- because the cache lookup precedes derivation, the blank-salt refusal in the provider never fired. Encrypts now refuse a blank salt even against a warm cache. #380
68
+ - Encryption now fails closed when ``encryption_hkdf_salt`` is blank, including with a warm request cache. Configure a non-blank salt as described in ``docs/guides/encryption.md``. #380
85
69
 
86
70
  Documentation
87
71
  -------------
88
72
 
89
- - Replaced fabricated/incorrect examples in ``docs/overview.md`` and RDoc comments with accurate, tested examples for permission management and relationships.
90
- - Documented that ``<collection>_with_permission`` requires atomic flags and rejects role symbols.
91
- - Corrected the error message for ``guard_allowed_fields!``.
92
- - Documented ``PERMISSION_CATEGORIES`` and exclusive permission tiers.
93
- - Fixed incorrect connection provider examples in docs.
94
- - Updated encryption guides to cover personalization history and rotation.
73
+ - Corrected permission-management, relationship, and connection-provider examples.
74
+ - Documented ``<collection>_with_permission`` flags, permission categories, exclusive tiers, and encryption personalization rotation.
95
75
 
96
76
  AI Assistance
97
77
  -------------
@@ -107,25 +87,14 @@ AI Assistance
107
87
  Changed
108
88
  -------
109
89
 
110
- - ``to_h_for_storage`` omits nil fields; every write path removes a field that has
111
- become nil. ``save``/``commit_fields`` do a full overwrite (the stored hash
112
- matches the in-memory object, so a field nil in memory is deleted);
113
- ``save_fields``/``multi_field_update``/``multi_field_fast_write`` delete a named
114
- field passed as nil. ``to_h`` still returns every declared field, nils included.
90
+ - Nil-valued fields are now omitted from storage. Setting a field to ``nil`` removes it on every write path, while ``to_h`` continues to include declared fields with nil values.
115
91
 
116
- - Claim caveat: a full ``save``/``commit_fields`` of a stale copy clears a field
117
- another writer claimed via ``HSETNX``. To claim and update without disturbing
118
- it, use the targeted writers or ``refresh!`` first.
92
+ - Refresh stale instances before a full ``save`` or ``commit_fields`` when another writer may have claimed a field; use targeted writers to update only named fields.
119
93
 
120
94
  Fixed
121
95
  -----
122
96
 
123
- - Nil-valued fields are no longer stored as the JSON string ``"null"``. Because a
124
- hash has no native NULL, this left declared fields perpetually present, breaking
125
- ``HSETNX``/``HEXISTS`` atomic-claim patterns and wasting memory. Nil fields are
126
- now omitted, and clearing a field to nil removes it from storage (``HDEL``), so
127
- absence again means "no value". No migration required: stale ``"null"`` values
128
- are cleaned up on the next save and already decode back to ``nil`` on read.
97
+ - Fixed nil-valued fields being stored as the JSON string ``"null"``. No migration is required; existing values are cleaned up on the next save.
129
98
 
130
99
  AI Assistance
131
100
  -------------
@@ -140,96 +109,28 @@ AI Assistance
140
109
  Added
141
110
  -----
142
111
 
143
- - ``encrypted_field`` now honors a per-field ``algorithm:`` option, pinning that
144
- field's write algorithm to a specific registered provider (``'aes-256-gcm'`` or
145
- ``'xchacha20poly1305'``) independent of the registry's default-provider
146
- priority. The option was previously documented but silently ignored, so writes
147
- always used the default provider. Decryption stays envelope-driven, so a pin can
148
- be added, changed, or removed without breaking ciphertext already at rest, and
149
- ``re_encrypt_fields!`` re-encrypts under the pin rather than the default. This is
150
- the supported lever for a reader-before-writer format migration: deploy
151
- ``rbnacl`` fleet-wide so every node can *read* XChaCha20-Poly1305 while keeping
152
- *writes* pinned to AES-256-GCM until all readers are confirmed capable, then drop
153
- the pin. Issue #334
112
+ - ``encrypted_field`` now honors its per-field ``algorithm:`` option for new writes, including ``re_encrypt_fields!``. Existing ciphertext remains decryptable when the pin changes or is removed. See ``docs/guides/feature-encrypted-fields.md`` for rollout guidance. #334
154
113
 
155
114
  Changed
156
115
  -------
157
116
 
158
- - ``Familia::Encryption::Registry.get`` now distinguishes an unknown algorithm
159
- from a *known* algorithm whose provider is not available on the current node.
160
- Because ``Registry.register`` only stores providers whose runtime dependency is
161
- present, pinning ``encrypted_field ..., algorithm: 'xchacha20poly1305'`` on a
162
- node without ``rbnacl``/libsodium previously raised the misleading
163
- ``"Unsupported algorithm: xchacha20poly1305"`` -- pointing an operator at a typo
164
- when the real fix is a missing dependency. It now names the provider, explains
165
- the dependency is missing, and (because ``get`` runs on both the encrypt and
166
- decrypt paths) states that installing the dependency is what enables reading
167
- *and* writing the algorithm, framing an algorithm pin as a write-time
168
- workaround that cannot decrypt existing ciphertext. Each provider declares its
169
- own dependency via a new ``Provider.dependency_hint`` class method (nil for
170
- always-available providers like OpenSSL AES-256-GCM), so the generic error path
171
- names the correct library as providers are added rather than hardcoding any one
172
- of them. The set of registerable providers is centralized in
173
- ``Registry.known_providers``, the single source of truth shared by ``setup!``
174
- and ``get``. Error-message and internal-refactor only; the resolution of every
175
- available algorithm is unchanged. Issue #334
117
+ - Encryption errors now distinguish unsupported algorithms from known algorithms whose provider dependency is unavailable. #334
176
118
 
177
119
  Fixed
178
120
  -----
179
121
 
180
- - ``Familia::DataType#exists?`` no longer returns ``true`` for a deleted or
181
- never-created scalar key (``StringKey``, ``Counter``, ``Lock``,
182
- ``JsonStringKey``). The check was ``dbclient.exists(dbkey) && !size.zero?``,
183
- but ``EXISTS`` returns an Integer count and ``0`` is truthy in Ruby, so the
184
- guard never short-circuited on a missing key -- existence was decided
185
- entirely by the size check. ``exists?`` now uses a boolean-coerced ``EXISTS``
186
- count directly. Issue #331
187
-
188
- - Relatedly, ``StringKey#size``/``#length``/``#empty?`` (and ``Lock``'s) no
189
- longer reflect the never-nil ``#to_s`` fallback. ``#char_count`` derived from
190
- ``#to_s.size``, and ``#to_s`` intentionally returns ``Familia::Base``'s
191
- documented "never nil" inspect-string when the value is absent -- so
192
- ``#size`` was non-zero (and ``#empty?`` false) for a missing key.
193
- ``#char_count`` now reads ``#value`` directly; ``#to_s`` is left unchanged.
194
- Issue #331
195
-
196
- - ``encrypted_fields_status`` now reports each field's real algorithm for a live
197
- encrypted value (e.g. ``{ encrypted: true, algorithm: "aes-256-gcm", cleared:
198
- false }``), honoring any per-field pin. Previously it returned ``{ encrypted:
199
- false, value: "[CONCEALED]" }`` for every encrypted field, because
200
- ``ConcealedString`` had no ``concealed?`` predicate for the status check to
201
- match -- so the algorithm shown in the method's docstring and the guides was
202
- never actually produced. ``ConcealedString`` gains ``#concealed?`` and
203
- ``#algorithm`` readers (the latter reads the stored envelope). Issue #334
122
+ - Fixed ``Familia::DataType#exists?`` and missing ``StringKey`` and ``Lock`` size and emptiness checks. #331
123
+
124
+ - Fixed ``encrypted_fields_status`` to report the stored algorithm for encrypted fields, including per-field algorithm pins. #334
204
125
 
205
126
  Documentation
206
127
  -------------
207
128
 
208
- - Added an executable, multi-phase proof (``examples/encryption_upgrade_proof/``)
209
- demonstrating that installing ``rbnacl`` safely flips new writes to
210
- XChaCha20-Poly1305 while every existing AES-256-GCM envelope including
211
- ciphertext written by the released 2.10.1 gem, under the pre-#310 static HKDF
212
- salt, and under a retired master key version — keeps decrypting. Also pins,
213
- as deliberately-passing checks, two operational hazards: the XChaCha
214
- ``encryption_personalization`` cannot be rotated (no history/fallback like
215
- ``encryption_hkdf_salt_history``), and once any XChaCha envelope exists,
216
- every node that may read it needs libsodium installed. PR #330
217
-
218
- - The encrypted-fields guide previously showed a ``provider: :aes_gcm`` field
219
- option that was never implemented; those examples now use the real
220
- ``algorithm: 'aes-256-gcm'`` form, and the ``Familia::Encryption`` facade
221
- docstring documents the shipped behavior instead of a hypothetical
222
- implementation sketch. The ``encrypted_fields_status`` output examples across
223
- the guides and the overview were corrected to match what the method now
224
- returns. Issue #334
225
-
226
- - Added a memory-audit investigation (``docs/investigation/memory-audit.md``)
227
- diagnosing #309's ``<collection>_with_permission`` O(N) query as a transient,
228
- GC-reclaimable spike rather than a per-process leak, and auditing the rest of
229
- ``lib/`` for per-process growth (concluding Familia has no unconditional leak).
230
- Ships two executable proofs in ``try/investigation/`` — a pure-Ruby
231
- ``process_memory_leak_proof.rb`` and a live-Redis ``memory_leak_proof.rb``.
232
- Diagnosis only; no runtime behaviour is changed by the investigation. Issue #309
129
+ - Added ``examples/encryption_upgrade_proof/`` to demonstrate encrypted-field algorithm upgrades. PR #330
130
+
131
+ - Corrected encrypted-field option and status examples to match supported behavior. #334
132
+
133
+ - Documented memory characteristics of ``<collection>_with_permission`` queries in ``docs/investigation/memory-audit.md``. #309
233
134
 
234
135
  AI Assistance
235
136
  -------------
data/Gemfile CHANGED
@@ -26,7 +26,7 @@ group :development, :test do
26
26
  gem 'rake', '~> 13.0', require: false
27
27
  gem 'redcarpet', require: false
28
28
  gem 'reek', require: false
29
- gem 'rubocop', '~> 1.88.2', require: false
29
+ gem 'rubocop', '~> 1.90.0', require: false
30
30
  gem 'rubocop-performance', require: false
31
31
  gem 'rubocop-thread_safety', require: false
32
32
  gem 'ruby-lsp', require: false
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- familia (2.12.0)
4
+ familia (2.13.0)
5
5
  concurrent-ruby (~> 1.3)
6
6
  connection_pool (>= 2.4, < 4.0)
7
7
  json_schemer (~> 2.0)
@@ -63,7 +63,7 @@ GEM
63
63
  prism (>= 1.3.0)
64
64
  rdoc (>= 4.0.0)
65
65
  reline (>= 0.4.2)
66
- json (2.20.0)
66
+ json (2.21.2)
67
67
  json_schemer (2.5.0)
68
68
  bigdecimal
69
69
  hana (~> 1.3)
@@ -73,12 +73,12 @@ GEM
73
73
  lint_roller (1.1.0)
74
74
  logger (1.7.0)
75
75
  minitest (5.27.0)
76
- oj (3.17.4)
76
+ oj (3.17.6)
77
77
  bigdecimal (>= 3.0)
78
78
  ostruct (>= 0.2)
79
79
  ostruct (0.6.3)
80
80
  parallel (1.28.0)
81
- parser (3.3.11.1)
81
+ parser (3.3.12.0)
82
82
  ast (~> 2.4.1)
83
83
  racc
84
84
  pastel (0.8.0)
@@ -95,7 +95,7 @@ GEM
95
95
  rake (13.4.2)
96
96
  rbnacl (7.1.2)
97
97
  ffi (~> 1)
98
- rbs (4.0.3)
98
+ rbs (4.1.3)
99
99
  logger
100
100
  prism (>= 1.6.0)
101
101
  tsort
@@ -131,8 +131,8 @@ GEM
131
131
  diff-lcs (>= 1.2.0, < 2.0)
132
132
  rspec-support (~> 3.13.0)
133
133
  rspec-support (3.13.7)
134
- rubocop (1.88.2)
135
- json (~> 2.3)
134
+ rubocop (1.90.0)
135
+ json (>= 2.3)
136
136
  language_server-protocol (~> 3.17.0.2)
137
137
  lint_roller (~> 1.1.0)
138
138
  parallel (>= 1.10)
@@ -145,15 +145,15 @@ GEM
145
145
  rubocop-ast (1.50.0)
146
146
  parser (>= 3.3.7.2)
147
147
  prism (~> 1.7)
148
- rubocop-performance (1.26.1)
148
+ rubocop-performance (1.27.0)
149
149
  lint_roller (~> 1.1)
150
- rubocop (>= 1.75.0, < 2.0)
150
+ rubocop (>= 1.89.0, < 2.0)
151
151
  rubocop-ast (>= 1.47.1, < 2.0)
152
152
  rubocop-thread_safety (0.7.3)
153
153
  lint_roller (~> 1.1)
154
154
  rubocop (~> 1.72, >= 1.72.1)
155
155
  rubocop-ast (>= 1.44.0, < 2.0)
156
- ruby-lsp (0.26.10)
156
+ ruby-lsp (0.26.11)
157
157
  language_server-protocol (~> 3.17.0)
158
158
  prism (>= 1.2, < 2.0)
159
159
  rbs (>= 3, < 5)
@@ -201,7 +201,7 @@ DEPENDENCIES
201
201
  rbnacl (~> 7.1, >= 7.1.1)
202
202
  redcarpet
203
203
  reek
204
- rubocop (~> 1.88.2)
204
+ rubocop (~> 1.90.0)
205
205
  rubocop-performance
206
206
  rubocop-thread_safety
207
207
  ruby-lsp
@@ -0,0 +1,17 @@
1
+ Fixed
2
+ -----
3
+
4
+ - ``Model.find_by_id``, ``find_by_dbkey``, and ``load_multi`` no longer raise
5
+ ``Familia::NoIdentifier`` when the stored hash contains a field value that
6
+ is not valid JSON (a legacy plain string written by a pre-JSON serializer or
7
+ by hand via ``HSET``). ``log_deserialization_issue`` computed ``dbkey`` for
8
+ its log message before any field setter had run on the freshly allocated
9
+ instance, so the identifier was still nil. The lookup now falls back to
10
+ ``no dbkey`` and the record loads with the raw string as the field value,
11
+ logging the intended ``Legacy plain string in ...`` error. ``refresh!`` was
12
+ unaffected because the instance already had its identifier.
13
+
14
+ AI Assistance
15
+ -------------
16
+
17
+ - The fix and regression test were developed with AI assistance.
@@ -0,0 +1,7 @@
1
+ ### Fixed
2
+
3
+ - **`EncryptedData.valid?` no longer echoes the candidate value into the debug log**: `valid?` runs on every assignment to an encrypted field, so the string it inspects is normally the plaintext the application just supplied. With `Familia.debug` enabled (`FAMILIA_DEBUG=1`), its success-path debug line interpolated the entire parsed value, so any plaintext that happened to be a JSON object (an API credential blob, a serialized profile) landed in the log in full, keys and values alike. The rescue path had the same shape of problem: it logged the JSON parser's exception message, which on some `json` gem versions quotes the offending input, so ordinary non-JSON plaintexts could leak through that line too. The success line now reports only the outcome and which required envelope fields are missing (never the candidate's own keys), and the rescue line logs only the exception class. `EncryptedData.validate!` and `Encryption::Manager#decrypt` re-raised the same parser message inside their `EncryptionError` ("Invalid JSON structure: ..."); those now name only the parser error class as well, so a caller that logs the exception cannot echo the stored bytes. Return values are unchanged. (#406)
4
+
5
+ ### AI Assistance
6
+
7
+ - The fix, the log-capture regression tests, and this entry were developed with AI assistance.
@@ -0,0 +1,7 @@
1
+ ### Fixed
2
+
3
+ - **Deserialization failure logs no longer carry the stored value**: when `deserialize_value` falls back to returning a raw string (a legacy plain string or corrupted JSON), `log_deserialization_issue` truncated the value to 50 characters in the structured `value_preview` field but interpolated the full `val.inspect` into the message itself. That message is logged at ERROR (ungated) and also wrapped in the `StandardError` handed to `Familia::Instrumentation.notify_error`, so on every load of an affected record the complete value, which can include raw ciphertext migrated from older encryption schemes, reached the log and any error-reporting sink. The message now carries only the classification, the `Class#field` context and the dbkey, so the ERROR line and the `notify_error` payload contain no bytes of the value at default settings. A new `value_length` (byte size) is always reported in both the log context and the `notify_error` context; the bounded `value_preview` is only added to the log context when `Familia.debug?` is on, and is never sent to `notify_error`. The classification is also computed once per call instead of twice. The return value and control flow of `deserialize_value` are unchanged. (#407)
4
+
5
+ ### AI Assistance
6
+
7
+ - The redaction change, the debug-gated preview, and the accompanying log and instrumentation tests were developed with AI assistance.
@@ -0,0 +1,32 @@
1
+ Security
2
+ --------
3
+
4
+ - Encrypted field setters no longer duck-type the assigned value. Previously
5
+ any value that parsed as JSON with the envelope keys (``algorithm``,
6
+ ``nonce``, ``ciphertext``, ``auth_tag``, ``key_version``) was stored
7
+ verbatim, so caller-supplied plaintext shaped like an envelope landed at
8
+ rest unencrypted (and unreadable). A value is now taken verbatim only when
9
+ wrapped in the new ``Familia::Encryption::StoredEnvelope`` marker, which the
10
+ hydration path applies via ``EncryptedFieldType#deserialize``; every other
11
+ assignment (setter, fast writer, ``apply_fields``, keyword construction) is
12
+ encrypted whatever it looks like. ``multi_field_update`` and
13
+ ``multi_field_fast_write`` were unaffected: they already raise
14
+ ``ArgumentError`` for anything but a ``ConcealedString``. (#405)
15
+
16
+ Changed
17
+ -------
18
+
19
+ - ``Familia::FieldType#deserialize(value, record)`` is now called on the
20
+ storage-to-object path (``load``, ``find_by_id``, ``refresh!``,
21
+ ``naive_refresh``) for every persistent field, after JSON decoding and before
22
+ the setter. It was previously defined but never invoked. Custom field types
23
+ that override it will now see stored values pass through it. (#405)
24
+ - Constructing a record with raw envelope JSON as a keyword argument
25
+ (``Model.new(secret: envelope_json)``) now encrypts that JSON as plaintext.
26
+ Rehydrate from a raw storage hash with ``naive_refresh`` instead. (#405)
27
+
28
+ AI Assistance
29
+ -------------
30
+
31
+ - The provenance-marker design, the hook wiring and the tests were developed
32
+ with AI assistance.
@@ -0,0 +1,26 @@
1
+ Security
2
+ --------
3
+
4
+ - The ``encoding`` field of an encryption envelope is now validated before it
5
+ is applied to the decrypted plaintext. It is the one envelope field that is
6
+ neither covered by the AEAD tag nor checked by ``validate_decryptable!``, so
7
+ a rewritten value used to reach ``force_encoding`` directly: an unknown name
8
+ raised ``ArgumentError`` (a non-string raised ``TypeError``) which the
9
+ generic rescue in ``Manager#decrypt`` reported as ``Decryption failed``,
10
+ indistinguishable from corrupted ciphertext. The value must now be a String
11
+ in ``Familia::Encryption::ENVELOPE_ENCODINGS`` (``Encoding.name_list`` minus
12
+ the host-dependent ``locale``, ``external``, ``internal`` and ``filesystem``
13
+ pseudo names). Anything else raises a defined
14
+ ``EncryptionError: Unsupported encoding`` from ``validate_decryptable!``,
15
+ ``Manager#decrypt`` (before any key derivation) and ``ConcealedString``
16
+ wrapping, and ``decryptable?`` returns false. Envelopes without the field
17
+ still decrypt as UTF-8. Binding the field into the AAD so that a
18
+ valid-but-wrong name is also rejected remains open; it needs a new envelope
19
+ version. (#408)
20
+
21
+ AI Assistance
22
+ -------------
23
+
24
+ - The allowlist placement (on ``EncryptedData`` so every validation entry
25
+ point shares it) and the accompanying tests were developed with AI
26
+ assistance.
@@ -0,0 +1,4 @@
1
+ Documentation
2
+ -------------
3
+
4
+ - Linked security audit status entries directly to their specific July 26 audit findings.
@@ -0,0 +1,6 @@
1
+ Security
2
+ --------
3
+
4
+ - Assigning plaintext to an encrypted field now raises ``Familia::NoIdentifier``
5
+ when the record has no identifier, preventing ciphertext from becoming
6
+ undecryptable after an identifier is assigned.
@@ -0,0 +1,4 @@
1
+ Security
2
+ --------
3
+
4
+ - Made ``rebuild_via_scan`` propagate batch failures and preserve the live index instead of replacing it with incomplete rebuilt contents.
@@ -0,0 +1,4 @@
1
+ Fixed
2
+ -----
3
+
4
+ - ``ExternalIdentifier.extid?`` now rejects uppercase IDs and external identifier format declarations now require one ``%{id}`` placeholder. (#400, #401)