dalli 5.0.4 → 5.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +137 -0
- data/lib/dalli/client.rb +23 -7
- data/lib/dalli/options.rb +1 -1
- data/lib/dalli/pipelined_deleter.rb +17 -10
- data/lib/dalli/protocol/base.rb +15 -2
- data/lib/dalli/protocol/connection_manager.rb +53 -17
- data/lib/dalli/protocol/key_regularizer.rb +8 -9
- data/lib/dalli/protocol/meta.rb +46 -87
- data/lib/dalli/protocol/request_formatter.rb +80 -32
- data/lib/dalli/protocol/response_buffer.rb +21 -22
- data/lib/dalli/protocol/response_processor.rb +63 -27
- data/lib/dalli/socket.rb +47 -32
- data/lib/dalli/version.rb +1 -1
- metadata +2 -3
- data/lib/dalli/pid_cache.rb +0 -40
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 54f211dbcca14abc18e7e9ffc7e9fe5c992c1e95afab2b6a64e46b6ec32efa8a
|
|
4
|
+
data.tar.gz: f655e7c63cc622876ae47400afd59139eea3b8e19f6585af223fafccad94e659
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: de1470d516e61e2b970c0cfaa0de3afe572124081b300ce643405813c3ba232be08da32fbc3748fbc86b39e2aaea0d6e5803d34cd9722f50c3fc47b325970c67
|
|
7
|
+
data.tar.gz: 1835dc76b3cfebe33a4a8eca38d37dcfa8af84495e0d5af523402ddd4ffa1e3a2a8c808ce4c6e2605325f8d6eded6faa4a574022680528016f091e330e987a3e
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,143 @@ Dalli Changelog
|
|
|
4
4
|
Unreleased
|
|
5
5
|
==========
|
|
6
6
|
|
|
7
|
+
5.0.6
|
|
8
|
+
==========
|
|
9
|
+
|
|
10
|
+
Performance:
|
|
11
|
+
|
|
12
|
+
- Skip the cas-return flag on quiet `meta_set` requests (#1131)
|
|
13
|
+
- In quiet mode memcached suppresses the `ms` response entirely, so the CAS requested by the `c` flag can never be read; sending it only added two bytes to every request
|
|
14
|
+
- Applies to the bulk-write paths, where quiet sets are emitted: `Dalli::Client#multi` blocks and the pipelined setter
|
|
15
|
+
- Extracted from #1130; thanks to Jianbin Chen for this contribution
|
|
16
|
+
|
|
17
|
+
- Reduce allocations in `KeyRegularizer` and multi-key request paths (#1120)
|
|
18
|
+
- Decomposed `KeyRegularizer#encode` into separate `needs_encoding?` and `encode` calls so the common happy path avoids allocating an intermediate array for the two-element return value
|
|
19
|
+
- Refactored `multi_get`/`multi_set`/`multi_delete` command generation into `RequestFormatter` to share its key-encoding helpers
|
|
20
|
+
- Thanks to Jean Boussier for this contribution
|
|
21
|
+
|
|
22
|
+
- Reduce allocations in `ResponseBuffer` pipelined getk parsing (#1117)
|
|
23
|
+
- `process_single_getk_response` was building a fresh array to return results alongside the updated offset; refactored to store the offset as the last element of the existing tokens array and pop it, saving one allocation per response
|
|
24
|
+
- Also skips trailing nils in the token array
|
|
25
|
+
- Thanks to Jean Boussier for this contribution
|
|
26
|
+
|
|
27
|
+
- Enable frozen string literals in `RequestFormatter` (#1118)
|
|
28
|
+
- Frozen string literals had been inadvertently disabled; re-enabling reduces allocations by ~300,000 objects in a 10,000-iteration `get_multi_cas` benchmark (562 MB → 550 MB total allocated)
|
|
29
|
+
- Thanks to Jean Boussier for this contribution
|
|
30
|
+
|
|
31
|
+
- Reduce allocations in `ResponseProcessor#value_from_tokens` (#1113)
|
|
32
|
+
- `token[1..].to_i` was allocating a new string for every token parsed; replaced with in-place `slice!` followed by a token reset to avoid poisoning subsequent token comparisons
|
|
33
|
+
- Saves 4 allocations per entry in `get_multi_cas` workloads (a hotspot for IdentityCache)
|
|
34
|
+
- Thanks to Jean Boussier for this contribution
|
|
35
|
+
|
|
36
|
+
- Reduce allocations in common operation paths (#1111)
|
|
37
|
+
- Use `Symbol#name` over `Symbol#to_s` to return a frozen string without allocation
|
|
38
|
+
- Skip trace attribute hash construction when OpenTelemetry instrumentation is disabled
|
|
39
|
+
- Use argument forwarding (`...`) in `Client#perform` and `Threadsafe#request` to avoid splat array allocation
|
|
40
|
+
- Use `match?` in `KeyRegularizer#encode` to avoid `MatchData` object allocation
|
|
41
|
+
- Reduces objects allocated by ~26% and memory by ~6% for a simple `get` workload
|
|
42
|
+
- Thanks to Jean Boussier for this contribution
|
|
43
|
+
|
|
44
|
+
- Fix pathological memory behavior in `ResponseBuffer` (#1114)
|
|
45
|
+
- `compact_if_needed` was intended to reclaim memory by slicing off consumed bytes, but `buffer.byteslice(@offset..)` on an unfrozen string causes Ruby to allocate a hidden third string as the copy-on-write owner rather than freeing the original
|
|
46
|
+
- Redesigns buffer management to pass reusable buffer objects directly to `read`/`read_nonblock`, avoiding reallocation on each response read
|
|
47
|
+
- Reduces allocations from ~2.38 GB to ~649 MB in a `get_multi_cas` benchmark over 10,000 iterations
|
|
48
|
+
- Accompanied by new unit tests for `ResponseBuffer` (#1115)
|
|
49
|
+
- Thanks to Jean Boussier for this contribution
|
|
50
|
+
|
|
51
|
+
Features:
|
|
52
|
+
|
|
53
|
+
- `delete_multi` now returns the number of keys found and deleted (#1126)
|
|
54
|
+
- Previously the return value was unspecified; callers (e.g. Rails, see rails/rails#58071) had no way to tell how many keys were actually removed
|
|
55
|
+
- The count is derived from the meta protocol's quiet-mode delete responses with no extra round-trips: successful deletes are suppressed while misses report `NF`, so any response received before the terminator is a key that was not deleted
|
|
56
|
+
- The single-server fast path now shares the pipelined path's bounded retry on transient (`RetryableNetworkError`) network errors, so both paths behave consistently; the returned count is best-effort and may under-report if a network error triggers a retry, since keys deleted before the error are not recounted
|
|
57
|
+
- Thanks to Iliana Hadzhiatanasova for this contribution
|
|
58
|
+
|
|
59
|
+
Bug Fixes:
|
|
60
|
+
|
|
61
|
+
- Raise instead of returning a truncated value when the peer closes mid-response (#1135)
|
|
62
|
+
- `IO#read(count)` on a blocking socket accumulates across TCP chunks and hands back a shorter buffer (or `nil`) in only one case: the stream hit EOF. That short buffer was passed through as the response body, so a memcached restart, proxy drop, or load balancer timeout partway through a response could surface a truncated but still decodable value to the caller, indistinguishable from a real one
|
|
63
|
+
- A short read is now treated as the premature EOF it is, raising and closing the dirty socket so the request is retried on a fresh connection
|
|
64
|
+
- CRuby only; the JRuby path already used `Socket#readfull`, which enforces the same contract
|
|
65
|
+
- Extracted from #1130; thanks to Ian Ker-Seymer for the original fix and Jianbin Chen for the port
|
|
66
|
+
|
|
67
|
+
- Tear down the connection when a non-`StandardError` aborts a request (#1136)
|
|
68
|
+
- `Async::Stop` and `Thread#kill` descend from `Exception` rather than `StandardError`, so the rescue clauses in `Protocol::Base#request` never saw them; a scheduler cancelling a fiber parked on a response read skipped `close` entirely, leaving the connection marked as having a request in progress with partial response bytes still unread on the wire, and returning that half-used client to the pool under `connection_pool`
|
|
69
|
+
- `Protocol::Base#request` now closes in an `ensure` unless the request ran to completion, and `ConnectionManager#close` performs its state cleanup in an `ensure` so a second cancellation landing inside `@sock.close` cannot leave the socket non-nil with the request still marked in progress
|
|
70
|
+
- `Dalli::DalliError` and `Dalli::MarshalError` now close the connection at the point of failure rather than at the start of the next request; those paths already left the request in progress and `ConnectionManager#confirm_ready!` closed on the next call, so this changes when the close happens rather than adding one
|
|
71
|
+
- Extracted from #1130; thanks to Dan Mayer for the original fix and Jianbin Chen for the port
|
|
72
|
+
|
|
73
|
+
- Fix `ResponseBuffer` compaction logic (#1119)
|
|
74
|
+
- `COMPACT_THRESHOLD` was removed in #1116 as apparently unused, but the constant was referenced by the compaction guard; its absence silently disabled buffer compaction
|
|
75
|
+
- Restored the constant, corrected the compaction condition, and improved the buffer-shrinking implementation to use `String#bytesplice` (backed by `memmove`) for true in-place compaction
|
|
76
|
+
- Adds targeted tests covering the compaction threshold and shrink behavior
|
|
77
|
+
- Thanks to Jean Boussier for this contribution
|
|
78
|
+
|
|
79
|
+
Maintenance:
|
|
80
|
+
|
|
81
|
+
- Scope `StrictWarnings` to Dalli's own source (#1134)
|
|
82
|
+
- The test suite runs under `-w` and prepends a hook to `Warning.singleton_class` that turns warnings into failures, but that hook is global: a warning emitted while loading any third-party gem aborted the whole suite before a single test ran
|
|
83
|
+
- `json` 2.21.2's pure-Ruby generator (used on JRuby, where the C extension is unavailable) warns `method redefined; discarding old to_hash` at require time, which took the `jruby-10` CI job red with no change to Dalli
|
|
84
|
+
- Warnings are now attributed to a source file and only raise for `lib/` and `test/`; attribution prefers the location Ruby embeds in the message, since the stack at that point describes the require chain rather than the offending code
|
|
85
|
+
- Portable attribution of `Kernel#warn` callers also required walking the stack rather than indexing it (Ruby 3.3/3.4 push an `<internal:warning>` frame that 4.0 does not), skipping RubyGems' `Kernel#warn` shim (active on JRuby but not CRuby), and resolving relative backtrace paths
|
|
86
|
+
|
|
87
|
+
- Make raw and namespace fast path tests actually use those options (#1129)
|
|
88
|
+
- Followup to #1127: the `raw` and `namespace` variants passed those options to the helper that starts memcached, which configures the client the tests then discarded, so neither option was ever exercised
|
|
89
|
+
- Passes the options to the client under test and adds assertions that fail if they are absent
|
|
90
|
+
- Thanks to Iliana Hadzhiatanasova for this contribution
|
|
91
|
+
|
|
92
|
+
- Benchmark `set_multi` and add a `delete_multi` target (#1132)
|
|
93
|
+
- Enables the two `set_multi` reports that were commented out pending the arrival of `set_multi`, resolving the accompanying TODO
|
|
94
|
+
- Adds a `delete_multi` target comparing the pipelined path against N single deletes
|
|
95
|
+
- Extracted from #1130; thanks to Jianbin Chen for this contribution
|
|
96
|
+
|
|
97
|
+
- Bump CI memcached to 1.6.41 and run benchmarks on pull requests (#1133)
|
|
98
|
+
- The tests workflow moves from 1.6.40 to 1.6.41; the benchmarks and profile workflows had drifted back on 1.6.23
|
|
99
|
+
- Extracted from #1130; thanks to Jianbin Chen for this contribution
|
|
100
|
+
|
|
101
|
+
- Disable RuboCop metrics cops (#1128)
|
|
102
|
+
- Thanks to Jean Boussier for this contribution
|
|
103
|
+
|
|
104
|
+
- Remove `PIDCache` module (#1125)
|
|
105
|
+
- `Process.pid` is cached natively by Ruby 3.3+ (via https://bugs.ruby-lang.org/issues/19443), making the manual cache unnecessary now that Dalli requires Ruby 3.3+
|
|
106
|
+
- Thanks to Jean Boussier for this contribution
|
|
107
|
+
|
|
108
|
+
- Remove unused `COMPACT_THRESHOLD` constant from `ResponseBuffer` (#1116)
|
|
109
|
+
- Followup cleanup after the buffer management redesign in #1114
|
|
110
|
+
- Note: subsequently found to be in use; restored and corrected in #1119
|
|
111
|
+
- Thanks to Jean Boussier for this contribution
|
|
112
|
+
|
|
113
|
+
- Use `String#byteindex` instead of `String#index` when searching for the response terminator in `getk_response_from_buffer` (#1112)
|
|
114
|
+
- The result feeds directly into `byteslice`; `byteindex` makes the intent explicit, though both return the same value since the buffer encoding is always `BINARY`
|
|
115
|
+
- Thanks to Jean Boussier for this contribution
|
|
116
|
+
|
|
117
|
+
- Make single-server fast path tests actually exercise the fast path (#1127)
|
|
118
|
+
- The batch-operation tests built clients through a helper that registers two address aliases for the same memcached process, so every client had a 2-server ring and the tests always ran through the pipelined path instead of the single-server fast path
|
|
119
|
+
- Adds a `single_server_client` test helper that builds a client with a single address, and uses it in the affected `get_multi`, `set_multi`, and `delete_multi` tests
|
|
120
|
+
- Thanks to Iliana Hadzhiatanasova for this contribution
|
|
121
|
+
|
|
122
|
+
5.0.5
|
|
123
|
+
==========
|
|
124
|
+
|
|
125
|
+
Performance:
|
|
126
|
+
|
|
127
|
+
- Batch multi-key commands into a single write to reduce packet overhead (#1107)
|
|
128
|
+
- With `TCP_NODELAY` set on sockets, each `write` call emits a separate packet; the meta protocol was calling `write` up to 3 times per key in multi-key operations (`get_multi`, `set_multi`, `delete_multi`), significantly increasing network traffic compared to the old binary protocol
|
|
129
|
+
- Multi-key request paths now buffer all per-key commands into a single binary string and flush once; single-key paths combine the write and flush into one `flushed_write` call
|
|
130
|
+
- Thanks to Jean Boussier for this contribution
|
|
131
|
+
|
|
132
|
+
- Avoid repeated `RUBY_ENGINE` checks on every socket read (#1103)
|
|
133
|
+
- Moved the JRuby branch from a runtime `if` inside `ConnectionManager#read` to a class-level conditional method definition, so the check happens once at load time rather than on every read call
|
|
134
|
+
- Thanks to Jean Boussier for this contribution
|
|
135
|
+
|
|
136
|
+
- Eliminate per-call array allocations in `ResponseProcessor` (#1104)
|
|
137
|
+
- Token sets passed to `error_on_unexpected!` (e.g. `[VA, EN, HD]`) were allocated as new arrays on every invocation; replaced with frozen constants defined once at class load time
|
|
138
|
+
- Thanks to Jean Boussier for this contribution
|
|
139
|
+
|
|
140
|
+
- Avoid string copies when building request commands in `RequestFormatter` (#1106)
|
|
141
|
+
- Changed `cmd + TERMINATOR` to `cmd << TERMINATOR`; since `cmd` is always a mutable string, the in-place append avoids copying the entire command string just to append two bytes
|
|
142
|
+
- Thanks to Jean Boussier for this contribution
|
|
143
|
+
|
|
7
144
|
5.0.4
|
|
8
145
|
==========
|
|
9
146
|
|
data/lib/dalli/client.rb
CHANGED
|
@@ -364,12 +364,15 @@ module Dalli
|
|
|
364
364
|
# it batches requests by server and uses quiet mode.
|
|
365
365
|
#
|
|
366
366
|
# @param keys [Array<String>] keys to delete
|
|
367
|
-
# @return [
|
|
367
|
+
# @return [Integer] the number of keys that were found and deleted. This is
|
|
368
|
+
# best-effort: on a network error the operation is retried, and keys
|
|
369
|
+
# deleted before the error are not recounted, so the result may
|
|
370
|
+
# under-report the number actually removed when a failure occurs.
|
|
368
371
|
#
|
|
369
372
|
# Example:
|
|
370
373
|
# client.delete_multi(['key1', 'key2', 'key3'])
|
|
371
374
|
def delete_multi(keys)
|
|
372
|
-
return if keys.empty?
|
|
375
|
+
return 0 if keys.empty?
|
|
373
376
|
|
|
374
377
|
Instrumentation.trace('delete_multi', multi_trace_attrs('delete_multi', keys.size, keys)) do
|
|
375
378
|
if ring.servers.size == 1
|
|
@@ -569,11 +572,18 @@ module Dalli
|
|
|
569
572
|
|
|
570
573
|
def single_server_delete_multi(keys)
|
|
571
574
|
validated_keys = keys.map { |k| @key_manager.validate_key(k.to_s) }
|
|
572
|
-
return unless (server = single_server)
|
|
575
|
+
return 0 unless (server = single_server)
|
|
573
576
|
|
|
574
577
|
server.request(:delete_multi_req, validated_keys)
|
|
578
|
+
rescue Dalli::RetryableNetworkError => e
|
|
579
|
+
# Mirror the pipelined path: retry transient errors so a momentary blip
|
|
580
|
+
# still yields a real count. Bounded by the server's socket_max_failures,
|
|
581
|
+
# after which a hard NetworkError is raised and handled below.
|
|
582
|
+
Dalli.logger.debug { e.inspect }
|
|
583
|
+
Dalli.logger.debug { 'retrying single-server delete_multi because of network error' }
|
|
584
|
+
retry
|
|
575
585
|
rescue Dalli::NetworkError
|
|
576
|
-
|
|
586
|
+
0
|
|
577
587
|
end
|
|
578
588
|
|
|
579
589
|
def get_multi_attributes(keys)
|
|
@@ -651,7 +661,7 @@ module Dalli
|
|
|
651
661
|
# operation times out.
|
|
652
662
|
##
|
|
653
663
|
# rubocop:disable Naming/MethodParameterName
|
|
654
|
-
def perform(op, key,
|
|
664
|
+
def perform(op, key, ...)
|
|
655
665
|
# rubocop:enable Naming/MethodParameterName
|
|
656
666
|
return yield if block_given?
|
|
657
667
|
|
|
@@ -659,8 +669,14 @@ module Dalli
|
|
|
659
669
|
key = @key_manager.validate_key(key)
|
|
660
670
|
|
|
661
671
|
server = ring.server_for_key(key)
|
|
662
|
-
|
|
663
|
-
|
|
672
|
+
|
|
673
|
+
if Instrumentation.enabled?
|
|
674
|
+
op_name = op.name
|
|
675
|
+
Instrumentation.trace(op_name, trace_attrs(op_name, key, server)) do
|
|
676
|
+
server.request(op, key, ...)
|
|
677
|
+
end
|
|
678
|
+
else
|
|
679
|
+
server.request(op, key, ...)
|
|
664
680
|
end
|
|
665
681
|
rescue RetryableNetworkError => e
|
|
666
682
|
Dalli.logger.debug { e.inspect }
|
data/lib/dalli/options.rb
CHANGED
|
@@ -16,16 +16,19 @@ module Dalli
|
|
|
16
16
|
# Deletes multiple keys from memcached.
|
|
17
17
|
#
|
|
18
18
|
# @param keys [Array<String>] keys to delete
|
|
19
|
-
# @return [
|
|
19
|
+
# @return [Integer] the number of keys that were deleted. This is
|
|
20
|
+
# best-effort: on a network error the operation is retried, and keys
|
|
21
|
+
# deleted before the error are not recounted, so the result may
|
|
22
|
+
# under-report the number actually removed when a failure occurs.
|
|
20
23
|
##
|
|
21
24
|
def process(keys)
|
|
22
|
-
return if keys.empty?
|
|
25
|
+
return 0 if keys.empty?
|
|
23
26
|
|
|
24
27
|
@ring.lock do
|
|
25
|
-
|
|
26
|
-
finish_requests(
|
|
28
|
+
groups = setup_requests(keys)
|
|
29
|
+
finish_requests(groups)
|
|
27
30
|
end
|
|
28
|
-
rescue
|
|
31
|
+
rescue Dalli::RetryableNetworkError => e
|
|
29
32
|
Dalli.logger.debug { e.inspect }
|
|
30
33
|
Dalli.logger.debug { 'retrying pipelined deletes because of network error' }
|
|
31
34
|
retry
|
|
@@ -36,7 +39,7 @@ module Dalli
|
|
|
36
39
|
def setup_requests(keys)
|
|
37
40
|
groups = groups_for_keys(keys)
|
|
38
41
|
make_delete_requests(groups)
|
|
39
|
-
groups
|
|
42
|
+
groups
|
|
40
43
|
end
|
|
41
44
|
|
|
42
45
|
##
|
|
@@ -45,24 +48,28 @@ module Dalli
|
|
|
45
48
|
##
|
|
46
49
|
def make_delete_requests(groups)
|
|
47
50
|
groups.each do |server, keys_for_server|
|
|
48
|
-
keys_for_server.
|
|
51
|
+
keys_for_server.select! do |key|
|
|
49
52
|
server.request(:pipelined_delete, key)
|
|
53
|
+
true
|
|
50
54
|
rescue DalliError, NetworkError => e
|
|
51
55
|
Dalli.logger.debug { e.inspect }
|
|
52
56
|
Dalli.logger.debug { "unable to delete key #{key} for server #{server.name}" }
|
|
57
|
+
false
|
|
53
58
|
end
|
|
54
59
|
end
|
|
55
60
|
end
|
|
56
61
|
|
|
57
62
|
##
|
|
58
63
|
# Sends noop to each server to flush responses and ensure all deletes complete.
|
|
64
|
+
# Returns the total successful deletes across servers.
|
|
59
65
|
##
|
|
60
|
-
def finish_requests(
|
|
61
|
-
|
|
62
|
-
server.request(:
|
|
66
|
+
def finish_requests(groups)
|
|
67
|
+
groups.sum do |server, keys_for_server|
|
|
68
|
+
server.request(:finish_pipelined_delete, keys_for_server.size)
|
|
63
69
|
rescue DalliError, NetworkError => e
|
|
64
70
|
Dalli.logger.debug { e.inspect }
|
|
65
71
|
Dalli.logger.debug { "unable to complete pipelined delete on server #{server.name}" }
|
|
72
|
+
0
|
|
66
73
|
end
|
|
67
74
|
end
|
|
68
75
|
|
data/lib/dalli/protocol/base.rb
CHANGED
|
@@ -18,7 +18,7 @@ module Dalli
|
|
|
18
18
|
|
|
19
19
|
def_delegators :@value_marshaller, :serializer, :compressor, :compression_min_size, :compress_by_default?
|
|
20
20
|
def_delegators :@connection_manager, :name, :sock, :hostname, :port, :close, :connected?, :socket_timeout,
|
|
21
|
-
:socket_type, :up!, :down!, :write, :reconnect_down_server?, :raise_down_error
|
|
21
|
+
:socket_type, :up!, :down!, :write, :reconnect_down_server?, :raise_down_error, :flushed_write
|
|
22
22
|
|
|
23
23
|
def initialize(attribs, client_options = {})
|
|
24
24
|
hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
|
|
@@ -40,12 +40,14 @@ module Dalli
|
|
|
40
40
|
verify_state(opkey)
|
|
41
41
|
|
|
42
42
|
begin
|
|
43
|
+
request_completed = false
|
|
43
44
|
@connection_manager.start_request!
|
|
44
45
|
response = send(opkey, *args)
|
|
45
46
|
|
|
46
47
|
# pipelined_get/pipelined_get_interleaved emit query but don't read the response(s)
|
|
47
48
|
@connection_manager.finish_request! unless %i[pipelined_get pipelined_get_interleaved].include?(opkey)
|
|
48
49
|
|
|
50
|
+
request_completed = true
|
|
49
51
|
response
|
|
50
52
|
rescue Dalli::MarshalError => e
|
|
51
53
|
log_marshal_err(args.first, e)
|
|
@@ -56,6 +58,18 @@ module Dalli
|
|
|
56
58
|
log_unexpected_err(e)
|
|
57
59
|
close
|
|
58
60
|
raise
|
|
61
|
+
ensure
|
|
62
|
+
# If the begin block didn't complete -- any exception, including a
|
|
63
|
+
# non-StandardError such as Async::Stop or Thread#kill, which the
|
|
64
|
+
# rescue clauses above never see -- tear down the connection so a
|
|
65
|
+
# half-used client isn't returned to the pool.
|
|
66
|
+
#
|
|
67
|
+
# The local flag is deliberate: reading $ERROR_INFO here would see
|
|
68
|
+
# the *outer* exception when `request` is called from inside a
|
|
69
|
+
# rescue clause (a common cache-fallback shape), and would then
|
|
70
|
+
# falsely tear down the pipelined_get happy path, which leaves
|
|
71
|
+
# @request_in_progress true on purpose until the caller drains.
|
|
72
|
+
close unless request_completed
|
|
59
73
|
end
|
|
60
74
|
end
|
|
61
75
|
|
|
@@ -95,7 +109,6 @@ module Dalli
|
|
|
95
109
|
# When a block is given, yields (key, value, cas) for each response,
|
|
96
110
|
# avoiding intermediate Hash allocation. Returns nil.
|
|
97
111
|
# Without a block, returns a Hash of { key => [value, cas] }.
|
|
98
|
-
# rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
|
|
99
112
|
def pipeline_next_responses(&block)
|
|
100
113
|
reconnect_on_pipeline_complete!
|
|
101
114
|
values = nil
|
|
@@ -4,8 +4,6 @@ require 'English'
|
|
|
4
4
|
require 'socket'
|
|
5
5
|
require 'timeout'
|
|
6
6
|
|
|
7
|
-
require 'dalli/pid_cache'
|
|
8
|
-
|
|
9
7
|
module Dalli
|
|
10
8
|
module Protocol
|
|
11
9
|
##
|
|
@@ -54,7 +52,7 @@ module Dalli
|
|
|
54
52
|
|
|
55
53
|
@sock = memcached_socket
|
|
56
54
|
@sock.sync = false # Enable buffered I/O for better performance
|
|
57
|
-
@pid =
|
|
55
|
+
@pid = Process.pid
|
|
58
56
|
@request_in_progress = false
|
|
59
57
|
rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
|
|
60
58
|
# SocketError = DNS resolution failure
|
|
@@ -117,10 +115,16 @@ module Dalli
|
|
|
117
115
|
@sock.close
|
|
118
116
|
rescue StandardError
|
|
119
117
|
nil
|
|
118
|
+
ensure
|
|
119
|
+
# A non-StandardError (e.g. a second Async::Stop fired into the
|
|
120
|
+
# fiber while it is already inside this cleanup) can escape
|
|
121
|
+
# @sock.close; run the state cleanup unconditionally so the client
|
|
122
|
+
# isn't returned to the pool with a half-closed socket and
|
|
123
|
+
# @request_in_progress == true.
|
|
124
|
+
@sock = nil
|
|
125
|
+
@pid = nil
|
|
126
|
+
abort_request!
|
|
120
127
|
end
|
|
121
|
-
@sock = nil
|
|
122
|
-
@pid = nil
|
|
123
|
-
abort_request!
|
|
124
128
|
end
|
|
125
129
|
|
|
126
130
|
def connected?
|
|
@@ -155,18 +159,26 @@ module Dalli
|
|
|
155
159
|
error_on_request!(e)
|
|
156
160
|
end
|
|
157
161
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
+
# JRuby doesn't support IO#timeout=, so use custom readfull implementation
|
|
163
|
+
# CRuby 3.3+ has IO#timeout= which makes IO#read work with timeouts
|
|
164
|
+
if RUBY_ENGINE == 'jruby'
|
|
165
|
+
def read(count)
|
|
162
166
|
@sock.readfull(count)
|
|
163
|
-
|
|
164
|
-
|
|
167
|
+
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
|
|
168
|
+
error_on_request!(e)
|
|
169
|
+
end
|
|
170
|
+
else
|
|
171
|
+
def read(count)
|
|
172
|
+
read_bytes(count)
|
|
173
|
+
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
|
|
174
|
+
error_on_request!(e)
|
|
165
175
|
end
|
|
166
|
-
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
|
|
167
|
-
error_on_request!(e)
|
|
168
176
|
end
|
|
169
177
|
|
|
178
|
+
# Alias for callers that want to make the exact-length contract explicit
|
|
179
|
+
# at the call site.
|
|
180
|
+
alias read_exact read
|
|
181
|
+
|
|
170
182
|
def write(bytes)
|
|
171
183
|
@sock.write(bytes)
|
|
172
184
|
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
|
|
@@ -179,10 +191,18 @@ module Dalli
|
|
|
179
191
|
error_on_request!(e)
|
|
180
192
|
end
|
|
181
193
|
|
|
194
|
+
def flushed_write(bytes)
|
|
195
|
+
written = @sock.write(bytes)
|
|
196
|
+
@sock.flush
|
|
197
|
+
written
|
|
198
|
+
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
|
|
199
|
+
error_on_request!(e)
|
|
200
|
+
end
|
|
201
|
+
|
|
182
202
|
# Non-blocking read. Here to support the operation
|
|
183
203
|
# of the get_multi operation
|
|
184
|
-
def
|
|
185
|
-
@sock.read_available
|
|
204
|
+
def read_available(...)
|
|
205
|
+
@sock.read_available(...)
|
|
186
206
|
end
|
|
187
207
|
|
|
188
208
|
def max_allowed_failures
|
|
@@ -240,7 +260,7 @@ module Dalli
|
|
|
240
260
|
end
|
|
241
261
|
|
|
242
262
|
def fork_detected?
|
|
243
|
-
@pid && @pid !=
|
|
263
|
+
@pid && @pid != Process.pid
|
|
244
264
|
end
|
|
245
265
|
|
|
246
266
|
def log_down_detected
|
|
@@ -261,6 +281,22 @@ module Dalli
|
|
|
261
281
|
time = Time.now - @down_at
|
|
262
282
|
Dalli.logger.warn { format('%<name>s is back (downtime was %<time>.3f seconds)', name: name, time: time) }
|
|
263
283
|
end
|
|
284
|
+
|
|
285
|
+
private
|
|
286
|
+
|
|
287
|
+
# Reads exactly `count` bytes. IO#read(count) on a blocking socket blocks
|
|
288
|
+
# until it has `count` bytes, accumulating across TCP chunks internally,
|
|
289
|
+
# and only hands back a shorter (or nil) buffer when the stream hits EOF.
|
|
290
|
+
# So a short read means the peer closed mid-response: raise EOFError and
|
|
291
|
+
# let read's existing `rescue EOFError` route it through
|
|
292
|
+
# error_on_request!, which closes the dirty socket for a retry on a fresh
|
|
293
|
+
# connection (and preserves the $ERROR_INFO context down! relies on).
|
|
294
|
+
def read_bytes(count)
|
|
295
|
+
buffer = @sock.read(count)
|
|
296
|
+
return buffer if buffer && buffer.bytesize == count
|
|
297
|
+
|
|
298
|
+
raise EOFError, "EOF reading #{count} bytes; received #{buffer ? buffer.bytesize : 0}"
|
|
299
|
+
end
|
|
264
300
|
end
|
|
265
301
|
end
|
|
266
302
|
end
|
|
@@ -9,19 +9,18 @@ module Dalli
|
|
|
9
9
|
# allowed.
|
|
10
10
|
# memcached supports the use of base64 hashes for keys containing
|
|
11
11
|
# whitespace or non-ASCII characters, provided the 'b' flag is included in the request.
|
|
12
|
-
|
|
13
|
-
|
|
12
|
+
module KeyRegularizer
|
|
13
|
+
module_function
|
|
14
14
|
|
|
15
|
-
def
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
strict_base64_encoded = [key].pack('m0')
|
|
19
|
-
[strict_base64_encoded, true]
|
|
15
|
+
def required?(key)
|
|
16
|
+
!key.ascii_only? || /\s/.match?(key)
|
|
20
17
|
end
|
|
21
18
|
|
|
22
|
-
def
|
|
23
|
-
|
|
19
|
+
def encode(key)
|
|
20
|
+
[key].pack('m0')
|
|
21
|
+
end
|
|
24
22
|
|
|
23
|
+
def decode(encoded_key)
|
|
25
24
|
strict_base64_decoded = encoded_key.unpack1('m0')
|
|
26
25
|
strict_base64_decoded.force_encoding(Encoding::UTF_8)
|
|
27
26
|
end
|