dalli 5.0.7 → 5.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 10a8b21fed4728b6f98929e983262dc1fc32c59711fce78a84b0df82b0348ca1
-  data.tar.gz: 0253d48293d3b482e8362830acd9323873302e202cb230bca3fc1b1e09b1b457
+  metadata.gz: e11a3903c6fa786ac8f175bbb95130760f3e5886dc24fdc0825abc84d743006c
+  data.tar.gz: b9b376166e61c2899f23ba4bd5367360cc14c9e10a5d21fdb3ac34558c4b7303
 SHA512:
-  metadata.gz: bda1d416cf1f88514d0181ed2e7f7b53e1b935fb07901398cb91183f63e42d0ab7c9e54c177264cf77fc793ead7b8183d79cf3c335d8dee11aae5049a6f0fff9
-  data.tar.gz: d81bec09db2c23cdc95204b0bec5c491fbcb60e12823245b9d9d338502c98b2e6af5f6e58a12b11c081889ea5d63cb079b6e8ef2fb869db147f67219db86ac55
+  metadata.gz: bfc8526fce280bcb5003b81128b9e005975ffff54c1a7c82ce9d7b1104954c4dc169507108df70e4c5d90f444c01001640af9fccc038d180954e81f7a7459a55
+  data.tar.gz: 3c083001313d3dbdbc91cb122a76f8bd0af1fd5de75cddb89d2985ec67b40c313124ba56447818c6000d05633c80c4fae34cbaa560515a9947ca4a0fbe542134

data/CHANGELOG.md

@@ -4,16 +4,62 @@ Dalli Changelog
 Unreleased
 ==========
 
-5.0.7
+5.1.0
 ==========
 
-Security:
+Features:
+
+- Add opaque routing tokens: `:p_token` and `:l_token` request options (#1147, #1154)
+  - `get`, `gat`, `get_cas`, `get_with_metadata`, `fetch_with_lock`, `set`/`add`/`replace`/`set_cas`/`replace_cas`, `append`/`prepend`, `incr`/`decr`, `cas`/`cas!`, `delete`/`delete_cas`, and the bulk operations (`get_multi`, `get_multi_cas`, `get_multi_with_metadata`, `set_multi`, `delete_multi`) all accept per-request `:p_token`/`:l_token` options, appended to the wire protocol as `P<token>`/`L<token>` -- applied to every key on the bulk methods
+  - memcached itself ignores these tokens; per the meta protocol spec they exist as hints for a proxy or router sitting between the client and memcached
+  - CRLF and NUL bytes raise `ArgumentError` before the request reaches the socket, both in `Dalli::Client` and in `RequestFormatter`, so a bad token can't be used for wire-protocol injection and can't close the connection out from under the caller the way a formatter-only check would
+  - The bulk methods and `delete`/`delete_cas` were deferred out of #1147 to avoid racing other in-flight PRs touching the same method signatures; #1154 completes them, including both the single-server fast path and the multi-server pipelined path for each
+  - Extracted from #1130; thanks to Nick Herson for the original idea and Jianbin Chen for porting it forward
+
+- Support tombstone (mark-stale) deletes on `delete`, `delete_cas`, and `delete_multi` (#1145, #1153)
+  - `:invalidate` marks the item stale instead of removing it, so `#get_with_metadata` / `#get_multi_with_metadata` report `stale: true` and a reader can tell "another process is repopulating this" apart from "this was never here" -- a tombstoned key is not a miss
+  - `:tombstone_ttl` controls how long the stale marker lives; requires `:invalidate`, since memcached only honors the TTL on a delete when it accompanies the invalidate flag
+  - `:drop_value` removes the item's value but leaves the item; on its own it is not a tombstone -- reads are an ordinary hit with an empty value
+  - `delete_multi` applies the same options to every key in the batch, on both the single-server and pipelined paths; its return value keeps counting keys the server found and acted on, so under `:invalidate` it reports how many keys were tombstoned rather than removed
+  - Extracted from #1130; thanks to Jianbin Chen for this contribution
+
+- Add `:miss` and `:return_ttl_remaining` to `get_with_metadata` (#1143)
+  - `:miss` is now always present in the returned Hash, distinguishing a true miss from a stored `nil` under `cache_nils` or a tombstoned, stale hit -- neither of which a `nil` `:value` alone can tell apart
+  - `:return_ttl_remaining` exposes the meta protocol's `t` flag as `:ttl_remaining` (seconds remaining, or `-1` for an item with no expiry), following the same opt-in shape as `:return_hit_status` / `:return_last_access`
+  - Extracted from #1130; thanks to Jianbin Chen for this contribution
+
+- Add `get_multi_with_metadata` for stale-aware bulk reads (#1144)
+  - Returns `{ key => { value:, cas:, stale:, miss: } }` for the keys that were found; genuine misses are omitted, matching `get_multi` / `get_multi_cas` -- a tombstoned item is a hit at the protocol level, so it is still returned, with `stale: true`
+  - Routes to the same single-server fast path / pipelined-getter split as `get_multi`
+  - Extracted from #1130; thanks to Jianbin Chen for this contribution
+
+Other changes:
+
+- Raise the documented minimum supported memcached version to 1.6.27 (#1140)
+  - Groundwork for the features above: `drop_value` tombstone deletes require 1.6.27, the highest floor of anything landing from #1130
+  - Not enforced at runtime -- `MIN_SUPPORTED_MEMCACHED_VERSION` only gates the test harness and the README's support statement, so this changes no running client's behavior
+
+Bug Fixes:
 
-- Fix memcached command injection through numeric arguments (GHSA-6wmv-xq9m-fmp7)
-  - The `default` argument of `incr`/`decr`, and `fetch_with_lock`'s `lock_ttl` and `recache_threshold`, were written into the meta protocol command without conversion, so a String containing CRLF injected additional memcached commands (e.g. `set`, `flush_all`) on the connection
-  - These arguments must now be Integers, or Strings of decimal digits; anything else raises `ArgumentError` before a request is sent
-  - As defense in depth, `RequestFormatter` now converts every numeric flag it writes (`D`, `J`, `N`, `R`, `T`) to an Integer
-  - Thanks to oss-security-shop for the report
+- Retry a transient network error during a liveness check instead of treating it as terminal (#1150)
+  - `Dalli::Protocol::Base#alive?` caught any `NetworkError` -- including `RetryableNetworkError`, a subclass -- and unconditionally converted it to `false`, with no retry. This didn't match the retry-then-raise contract every other network-facing path in Dalli follows: a single transient connection hiccup during the liveness check itself (as opposed to an actual request) permanently reported a healthy server as down for that check
+  - Now retries once on `RetryableNetworkError`, letting `error_on_request!`'s own fail-count threshold decide when to actually give up (the same mechanism the rest of the codebase relies on): it keeps retrying until `socket_max_failures` is reached, then raises a terminal `NetworkError`, which is still converted to `false` as before
+  - **Behavior change:** a server that was previously marked "down" (engaging the `down_retry_delay` cooldown) only via an actual request could now also reach that state via a liveness check (`alive?`, and anything that calls it -- `Dalli::Client#stats`, `#reset_stats`, and `Ring`'s own server selection) exhausting its retries. Previously, a solitary transient failure during a liveness check was silently forgotten rather than tracked, so the cooldown was inconsistently applied depending on which code path first observed the failure
+  - Likely a contributing cause of the same intermittently failing `get_multi` failover integration test noted in #1149: that fix addressed the send/receive phase, but the liveness-check retry it introduced can itself force a fresh `connect()` mid-retry, giving this separate, pre-existing gap in `alive?` more chances to fire
+  - Also fixed a test (`test_ring.rb`, "detect when a dead server is up again") that had been unknowingly relying on the old behavior: it never engaged the `down_retry_delay` cooldown from a single transient failure, so its 0.5s delay never actually gated anything. Updated to use a 0s delay, since the test's intent is to verify reconnection is detected, not to test cooldown timing
+
+- Base64-encode keys containing control characters, not just NUL (#1148)
+  - `KeyRegularizer.required?` decided whether a key needed base64 encoding using `/\s/`, which matches most whitespace but none of the C0 control range (0x00-0x1F) or DEL (0x7F) -- a key that was otherwise ASCII-only and contained no whitespace (e.g. `"foo\x00bar"` or a key with an embedded ESC byte) went out on the wire unencoded
+  - Not a command-injection risk: the text protocol splits commands on CRLF, not other control bytes. The risk is key confusion -- anything downstream that treats one of these bytes specially (a C string terminating at NUL, a terminal or log line interpreting an escape byte) could silently act on a different key than Dalli believes it sent
+  - A raw control byte in a key was never protocol-compliant in the first place: memcached's own spec (`protocol.txt`) states a key "must not include control characters or whitespace." The only sanctioned way to carry such content in a key is the meta protocol's base64 (`b` flag) path -- the one whitespace and non-ASCII keys already used, and the one these keys now use too. The check is now `/[\p{Cntrl}\s]/`, matching that rule directly rather than special-casing NUL
+  - **Behavior change:** a key containing a control character now produces different bytes on the wire (base64-encoded, per the meta protocol's `b` flag) than before. Existing cache entries stored under the old, unencoded form of such a key will read as a miss once every reader has upgraded. **During a rolling deploy, old and new Dalli versions disagree about which physical key such a logical key maps to** -- not just a one-time cutover, but ongoing inconsistency between the old-version and new-version server pools for the duration of the rollout. Harmless for an ordinary cached value (worst case, extra cache misses); worth accounting for if such a key ever backs something stateful, like a lock or counter. Expected to be rare in practice: embedding a raw control byte in a cache key is unusual, and doing so was already outside what the protocol permits
+  - Found while auditing `request_formatter.rb` during the routing-token work in #1130 / #1147; unrelated to that change and predates it
+
+- Retry transient network errors in `get_multi`, `set_multi` and `delete_multi` instead of silently swallowing them (#1149)
+  - All three methods group keys by server and issue one request per server. Each per-server rescue clause caught `DalliError` and `NetworkError` together and swallowed both, just debug-logging -- since `RetryableNetworkError < NetworkError`, this also silently swallowed transient, retryable failures, dropping that server's keys from the result instead of the whole operation retrying (`get_multi`/`set_multi`'s single-server fast path did not even attempt a retry, on any failure)
+  - Six rescue sites across `PipelinedGetter`, `PipelinedSetter`, `PipelinedDeleter` and the `single_server_*` fast paths now retry a transient `RetryableNetworkError`, matching sibling rescue sites in the same files that already did this correctly
+  - **Behavior change:** if a server remains unreachable after retrying (not just a transient blip), these three methods now raise `Dalli::NetworkError` instead of silently returning an incomplete or empty result. This matches how every other Dalli::Client method already behaves on a hard network failure, and is the retry-then-raise behavior `delete_multi`'s own docs already described; it was just not reliably true for this failure path before. Code that calls these methods without rescuing `Dalli::NetworkError` should account for this if it can reach a fully unreachable server
+  - Likely the cause of an intermittently failing `get_multi` failover integration test that recurred across multiple PRs, though this could not be directly confirmed: the failure (an empty result with no exception in the logs) never reproduced locally despite repeated attempts, consistent with needing a genuine transient network hiccup that is far more likely on a loaded CI runner than an idle dev machine
 
 5.0.6
 ==========

data/README.md

@@ -17,7 +17,12 @@ The name is a variant of Salvador Dali for his famous painting [The Persistence
 ## Requirements
 
 * Ruby 3.3 or later (JRuby also supported)
-* memcached 1.6 or later
+* memcached 1.6.27 or later
+
+Dalli is tested against both the minimum supported memcached version and the
+latest release. Earlier 1.6.x servers are not supported: the meta protocol
+rejects unknown flags outright, so features added after a server's release fail
+with `CLIENT_ERROR invalid flag` rather than degrading.
 
 ## Configuration Options