dalli 4.1.0 → 4.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +20 -0
- data/README.md +41 -0
- data/lib/dalli/client.rb +76 -26
- data/lib/dalli/instrumentation.rb +139 -0
- data/lib/dalli/pipelined_getter.rb +5 -3
- data/lib/dalli/protocol/base.rb +8 -1
- data/lib/dalli/protocol/binary.rb +13 -0
- data/lib/dalli/protocol/connection_manager.rb +7 -0
- data/lib/dalli/protocol/meta/request_formatter.rb +5 -2
- data/lib/dalli/protocol/meta.rb +23 -3
- data/lib/dalli/version.rb +1 -1
- data/lib/dalli.rb +1 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7966ac393d9c0f41d61b244c6a5832fbcab5705ed4daa4189b6e1ffc92db600f
|
|
4
|
+
data.tar.gz: da320eac7a8d553f33fe81315750297f97edb75c6da79197ef1f974d47ac8081
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5d29cfbe8d2ee6e2993e8a292238f2762ac1de29b8d8623f0a521f810edf866ffd9d631905d5b7cb939d44216fb7e8d6edabf4c09ba388162fef97a772254352
|
|
7
|
+
data.tar.gz: ef1c636e6aab12a630ddb7e4d7081c29b0e257203b97f78ccde207170802d1c3619db7057093d4140fcc9cbc23f979bf6f60be4a64011bde983062a724ea59fc
|
data/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,26 @@
|
|
|
1
1
|
Dalli Changelog
|
|
2
2
|
=====================
|
|
3
3
|
|
|
4
|
+
4.2.0
|
|
5
|
+
==========
|
|
6
|
+
|
|
7
|
+
Performance:
|
|
8
|
+
|
|
9
|
+
- Buffered I/O: Use `socket.sync = false` with explicit flush to reduce syscalls for pipelined operations
|
|
10
|
+
- get_multi optimizations: Use Set for O(1) server tracking lookups
|
|
11
|
+
- Raw mode optimization: Skip bitflags request in meta protocol when in raw mode (saves 2 bytes per request)
|
|
12
|
+
|
|
13
|
+
New Features:
|
|
14
|
+
|
|
15
|
+
- OpenTelemetry tracing support: Automatically instruments operations when OpenTelemetry SDK is present
|
|
16
|
+
- Zero overhead when OpenTelemetry is not loaded
|
|
17
|
+
- Traces `get`, `set`, `delete`, `get_multi`, `set_multi`, `delete_multi`, `get_with_metadata`, and `fetch_with_lock`
|
|
18
|
+
- Spans include `db.system: memcached` and `db.operation` attributes
|
|
19
|
+
- Single-key operations include `server.address` attribute
|
|
20
|
+
- Multi-key operations include `db.memcached.key_count` attribute
|
|
21
|
+
- `get_multi` spans include `db.memcached.hit_count` and `db.memcached.miss_count` for cache efficiency metrics
|
|
22
|
+
- Exceptions are automatically recorded on spans with error status
|
|
23
|
+
|
|
4
24
|
4.1.0
|
|
5
25
|
==========
|
|
6
26
|
|
data/README.md
CHANGED
|
@@ -11,6 +11,7 @@ Dalli supports:
|
|
|
11
11
|
* Thread-safe operation (either through use of a connection pool, or by using the Dalli client in threadsafe mode)
|
|
12
12
|
* SSL/TLS connections to memcached
|
|
13
13
|
* SASL authentication
|
|
14
|
+
* OpenTelemetry distributed tracing (automatic when SDK is present)
|
|
14
15
|
|
|
15
16
|
The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
|
|
16
17
|
|
|
@@ -40,6 +41,46 @@ Dalli::Client.new('localhost:11211', serializer: JSON)
|
|
|
40
41
|
|
|
41
42
|
See the [4.0-Upgrade.md](4.0-Upgrade.md) guide for more information.
|
|
42
43
|
|
|
44
|
+
## OpenTelemetry Tracing
|
|
45
|
+
|
|
46
|
+
Dalli automatically instruments operations with [OpenTelemetry](https://opentelemetry.io/) when the SDK is present. No configuration is required - just add the OpenTelemetry gems to your application:
|
|
47
|
+
|
|
48
|
+
```ruby
|
|
49
|
+
# Gemfile
|
|
50
|
+
gem 'opentelemetry-sdk'
|
|
51
|
+
gem 'opentelemetry-exporter-otlp' # or your preferred exporter
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
When OpenTelemetry is loaded, Dalli creates spans for:
|
|
55
|
+
- Single key operations: `get`, `set`, `delete`, `add`, `replace`, `incr`, `decr`, etc.
|
|
56
|
+
- Multi-key operations: `get_multi`, `set_multi`, `delete_multi`
|
|
57
|
+
- Advanced operations: `get_with_metadata`, `fetch_with_lock`
|
|
58
|
+
|
|
59
|
+
### Span Attributes
|
|
60
|
+
|
|
61
|
+
All spans include:
|
|
62
|
+
- `db.system`: `memcached`
|
|
63
|
+
- `db.operation`: The operation name (e.g., `get`, `set_multi`)
|
|
64
|
+
|
|
65
|
+
Single-key operations also include:
|
|
66
|
+
- `server.address`: The memcached server that handled the request (e.g., `localhost:11211`)
|
|
67
|
+
|
|
68
|
+
Multi-key operations include cache efficiency metrics:
|
|
69
|
+
- `db.memcached.key_count`: Number of keys in the request
|
|
70
|
+
- `db.memcached.hit_count`: Number of keys found (for `get_multi`)
|
|
71
|
+
- `db.memcached.miss_count`: Number of keys not found (for `get_multi`)
|
|
72
|
+
|
|
73
|
+
### Error Handling
|
|
74
|
+
|
|
75
|
+
Exceptions are automatically recorded on spans with error status. When an operation fails:
|
|
76
|
+
1. The exception is recorded on the span via `span.record_exception(e)`
|
|
77
|
+
2. The span status is set to error with the exception message
|
|
78
|
+
3. The exception is re-raised to the caller
|
|
79
|
+
|
|
80
|
+
### Zero Overhead
|
|
81
|
+
|
|
82
|
+
When OpenTelemetry is not present, there is zero overhead - the tracing code checks once at startup and bypasses all instrumentation logic entirely when the SDK is not loaded.
|
|
83
|
+
|
|
43
84
|

|
|
44
85
|
|
|
45
86
|
|
data/lib/dalli/client.rb
CHANGED
|
@@ -134,8 +134,10 @@ module Dalli
|
|
|
134
134
|
key = key.to_s
|
|
135
135
|
key = @key_manager.validate_key(key)
|
|
136
136
|
|
|
137
|
-
|
|
138
|
-
|
|
137
|
+
Instrumentation.trace('get_with_metadata', { 'db.operation' => 'get_with_metadata' }) do
|
|
138
|
+
server = ring.server_for_key(key)
|
|
139
|
+
server.request(:meta_get, key, options)
|
|
140
|
+
end
|
|
139
141
|
rescue NetworkError => e
|
|
140
142
|
Dalli.logger.debug { e.inspect }
|
|
141
143
|
Dalli.logger.debug { 'retrying get_with_metadata with new server' }
|
|
@@ -146,20 +148,19 @@ module Dalli
|
|
|
146
148
|
# Fetch multiple keys efficiently.
|
|
147
149
|
# If a block is given, yields key/value pairs one at a time.
|
|
148
150
|
# Otherwise returns a hash of { 'key' => 'value', 'key2' => 'value1' }
|
|
151
|
+
# rubocop:disable Style/ExplicitBlockArgument
|
|
149
152
|
def get_multi(*keys)
|
|
150
153
|
keys.flatten!
|
|
151
154
|
keys.compact!
|
|
152
|
-
|
|
153
155
|
return {} if keys.empty?
|
|
154
156
|
|
|
155
157
|
if block_given?
|
|
156
|
-
|
|
158
|
+
get_multi_yielding(keys) { |k, v| yield k, v }
|
|
157
159
|
else
|
|
158
|
-
|
|
159
|
-
pipelined_getter.process(keys) { |k, data| hash[k] = data.first }
|
|
160
|
-
end
|
|
160
|
+
get_multi_hash(keys)
|
|
161
161
|
end
|
|
162
162
|
end
|
|
163
|
+
# rubocop:enable Style/ExplicitBlockArgument
|
|
163
164
|
|
|
164
165
|
##
|
|
165
166
|
# Fetch multiple keys efficiently, including available metadata such as CAS.
|
|
@@ -228,7 +229,7 @@ module Dalli
|
|
|
228
229
|
# expensive_operation
|
|
229
230
|
# end
|
|
230
231
|
#
|
|
231
|
-
def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil)
|
|
232
|
+
def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil, &block)
|
|
232
233
|
raise ArgumentError, 'Block is required for fetch_with_lock' unless block_given?
|
|
233
234
|
|
|
234
235
|
raise_unless_meta_protocol!
|
|
@@ -236,21 +237,8 @@ module Dalli
|
|
|
236
237
|
key = key.to_s
|
|
237
238
|
key = @key_manager.validate_key(key)
|
|
238
239
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
vivify_ttl: lock_ttl,
|
|
242
|
-
recache_ttl: recache_threshold
|
|
243
|
-
})
|
|
244
|
-
|
|
245
|
-
if result[:won_recache]
|
|
246
|
-
# This client won the race - regenerate the value
|
|
247
|
-
new_val = yield
|
|
248
|
-
set(key, new_val, ttl_or_default(ttl), req_options)
|
|
249
|
-
new_val
|
|
250
|
-
else
|
|
251
|
-
# Another client is regenerating, or value exists and isn't stale
|
|
252
|
-
# Return the existing value
|
|
253
|
-
result[:value]
|
|
240
|
+
Instrumentation.trace('fetch_with_lock', { 'db.operation' => 'fetch_with_lock' }) do
|
|
241
|
+
fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options, &block)
|
|
254
242
|
end
|
|
255
243
|
rescue NetworkError => e
|
|
256
244
|
Dalli.logger.debug { e.inspect }
|
|
@@ -330,7 +318,12 @@ module Dalli
|
|
|
330
318
|
def set_multi(hash, ttl = nil, req_options = nil)
|
|
331
319
|
return if hash.empty?
|
|
332
320
|
|
|
333
|
-
|
|
321
|
+
Instrumentation.trace('set_multi', {
|
|
322
|
+
'db.operation' => 'set_multi',
|
|
323
|
+
'db.memcached.key_count' => hash.size
|
|
324
|
+
}) do
|
|
325
|
+
pipelined_setter.process(hash, ttl_or_default(ttl), req_options)
|
|
326
|
+
end
|
|
334
327
|
end
|
|
335
328
|
|
|
336
329
|
##
|
|
@@ -385,7 +378,12 @@ module Dalli
|
|
|
385
378
|
def delete_multi(keys)
|
|
386
379
|
return if keys.empty?
|
|
387
380
|
|
|
388
|
-
|
|
381
|
+
Instrumentation.trace('delete_multi', {
|
|
382
|
+
'db.operation' => 'delete_multi',
|
|
383
|
+
'db.memcached.key_count' => keys.size
|
|
384
|
+
}) do
|
|
385
|
+
pipelined_deleter.process(keys)
|
|
386
|
+
end
|
|
389
387
|
end
|
|
390
388
|
|
|
391
389
|
##
|
|
@@ -517,6 +515,42 @@ module Dalli
|
|
|
517
515
|
|
|
518
516
|
private
|
|
519
517
|
|
|
518
|
+
# Records hit/miss metrics on a span for cache observability.
|
|
519
|
+
# @param span [OpenTelemetry::Trace::Span, nil] the span to record on
|
|
520
|
+
# @param key_count [Integer] total keys requested
|
|
521
|
+
# @param hit_count [Integer] keys found in cache
|
|
522
|
+
def record_hit_miss_metrics(span, key_count, hit_count)
|
|
523
|
+
return unless span
|
|
524
|
+
|
|
525
|
+
span.set_attribute('db.memcached.hit_count', hit_count)
|
|
526
|
+
span.set_attribute('db.memcached.miss_count', key_count - hit_count)
|
|
527
|
+
end
|
|
528
|
+
|
|
529
|
+
def get_multi_yielding(keys)
|
|
530
|
+
Instrumentation.trace_with_result('get_multi', get_multi_attributes(keys)) do |span|
|
|
531
|
+
hit_count = 0
|
|
532
|
+
pipelined_getter.process(keys) do |k, data|
|
|
533
|
+
hit_count += 1
|
|
534
|
+
yield k, data.first
|
|
535
|
+
end
|
|
536
|
+
record_hit_miss_metrics(span, keys.size, hit_count)
|
|
537
|
+
nil
|
|
538
|
+
end
|
|
539
|
+
end
|
|
540
|
+
|
|
541
|
+
def get_multi_hash(keys)
|
|
542
|
+
Instrumentation.trace_with_result('get_multi', get_multi_attributes(keys)) do |span|
|
|
543
|
+
{}.tap do |hash|
|
|
544
|
+
pipelined_getter.process(keys) { |k, data| hash[k] = data.first }
|
|
545
|
+
record_hit_miss_metrics(span, keys.size, hash.size)
|
|
546
|
+
end
|
|
547
|
+
end
|
|
548
|
+
end
|
|
549
|
+
|
|
550
|
+
def get_multi_attributes(keys)
|
|
551
|
+
{ 'db.operation' => 'get_multi', 'db.memcached.key_count' => keys.size }
|
|
552
|
+
end
|
|
553
|
+
|
|
520
554
|
def check_positive!(amt)
|
|
521
555
|
raise ArgumentError, "Positive values only: #{amt}" if amt.negative?
|
|
522
556
|
end
|
|
@@ -529,6 +563,17 @@ module Dalli
|
|
|
529
563
|
perform(:set, key, newvalue, ttl_or_default(ttl), cas, req_options)
|
|
530
564
|
end
|
|
531
565
|
|
|
566
|
+
def fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options)
|
|
567
|
+
server = ring.server_for_key(key)
|
|
568
|
+
result = server.request(:meta_get, key, { vivify_ttl: lock_ttl, recache_ttl: recache_threshold })
|
|
569
|
+
|
|
570
|
+
return result[:value] unless result[:won_recache]
|
|
571
|
+
|
|
572
|
+
new_val = yield
|
|
573
|
+
set(key, new_val, ttl_or_default(ttl), req_options)
|
|
574
|
+
new_val
|
|
575
|
+
end
|
|
576
|
+
|
|
532
577
|
##
|
|
533
578
|
# Uses the argument TTL or the client-wide default. Ensures
|
|
534
579
|
# that the value is an integer
|
|
@@ -571,7 +616,12 @@ module Dalli
|
|
|
571
616
|
key = @key_manager.validate_key(key)
|
|
572
617
|
|
|
573
618
|
server = ring.server_for_key(key)
|
|
574
|
-
|
|
619
|
+
Instrumentation.trace(op.to_s, {
|
|
620
|
+
'db.operation' => op.to_s,
|
|
621
|
+
'server.address' => server.name
|
|
622
|
+
}) do
|
|
623
|
+
server.request(op, key, *args)
|
|
624
|
+
end
|
|
575
625
|
rescue NetworkError => e
|
|
576
626
|
Dalli.logger.debug { e.inspect }
|
|
577
627
|
Dalli.logger.debug { 'retrying request with new server' }
|
|
@@ -0,0 +1,139 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Dalli
|
|
4
|
+
##
|
|
5
|
+
# Instrumentation support for Dalli. Provides hooks for distributed tracing
|
|
6
|
+
# via OpenTelemetry when the SDK is available.
|
|
7
|
+
#
|
|
8
|
+
# When OpenTelemetry is loaded, Dalli automatically creates spans for cache operations.
|
|
9
|
+
# When OpenTelemetry is not available, all tracing methods are no-ops with zero overhead.
|
|
10
|
+
#
|
|
11
|
+
# == Span Attributes
|
|
12
|
+
#
|
|
13
|
+
# All spans include the following default attributes:
|
|
14
|
+
# - +db.system+ - Always "memcached"
|
|
15
|
+
#
|
|
16
|
+
# Single-key operations (+get+, +set+, +delete+, +incr+, +decr+, etc.) add:
|
|
17
|
+
# - +db.operation+ - The operation name (e.g., "get", "set")
|
|
18
|
+
# - +server.address+ - The memcached server handling the request (e.g., "localhost:11211")
|
|
19
|
+
#
|
|
20
|
+
# Multi-key operations (+get_multi+) add:
|
|
21
|
+
# - +db.operation+ - "get_multi"
|
|
22
|
+
# - +db.memcached.key_count+ - Number of keys requested
|
|
23
|
+
# - +db.memcached.hit_count+ - Number of keys found in cache
|
|
24
|
+
# - +db.memcached.miss_count+ - Number of keys not found
|
|
25
|
+
#
|
|
26
|
+
# Bulk write operations (+set_multi+, +delete_multi+) add:
|
|
27
|
+
# - +db.operation+ - The operation name
|
|
28
|
+
# - +db.memcached.key_count+ - Number of keys in the operation
|
|
29
|
+
#
|
|
30
|
+
# == Error Handling
|
|
31
|
+
#
|
|
32
|
+
# When an exception occurs during a traced operation:
|
|
33
|
+
# - The exception is recorded on the span via +record_exception+
|
|
34
|
+
# - The span status is set to error with the exception message
|
|
35
|
+
# - The exception is re-raised to the caller
|
|
36
|
+
#
|
|
37
|
+
# @example Checking if tracing is enabled
|
|
38
|
+
# Dalli::Instrumentation.enabled? # => true if OpenTelemetry is loaded
|
|
39
|
+
#
|
|
40
|
+
##
|
|
41
|
+
module Instrumentation
|
|
42
|
+
# Default attributes included on all memcached spans.
|
|
43
|
+
# @return [Hash] frozen hash with 'db.system' => 'memcached'
|
|
44
|
+
DEFAULT_ATTRIBUTES = { 'db.system' => 'memcached' }.freeze
|
|
45
|
+
|
|
46
|
+
class << self
|
|
47
|
+
# Returns the OpenTelemetry tracer if available, nil otherwise.
|
|
48
|
+
#
|
|
49
|
+
# The tracer is cached after first lookup for performance.
|
|
50
|
+
# Uses the library name 'dalli' and current Dalli::VERSION.
|
|
51
|
+
#
|
|
52
|
+
# @return [OpenTelemetry::Trace::Tracer, nil] the tracer or nil if OTel unavailable
|
|
53
|
+
def tracer
|
|
54
|
+
return @tracer if defined?(@tracer)
|
|
55
|
+
|
|
56
|
+
@tracer = (OpenTelemetry.tracer_provider.tracer('dalli', Dalli::VERSION) if defined?(OpenTelemetry))
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# Returns true if instrumentation is enabled (OpenTelemetry SDK is available).
|
|
60
|
+
#
|
|
61
|
+
# @return [Boolean] true if tracing is active, false otherwise
|
|
62
|
+
def enabled?
|
|
63
|
+
!tracer.nil?
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Wraps a block with a span if instrumentation is enabled.
|
|
67
|
+
#
|
|
68
|
+
# Creates a client span with the given name and attributes merged with
|
|
69
|
+
# DEFAULT_ATTRIBUTES. The block is executed within the span context.
|
|
70
|
+
# If an exception occurs, it is recorded on the span before re-raising.
|
|
71
|
+
#
|
|
72
|
+
# When tracing is disabled (OpenTelemetry not loaded), this method
|
|
73
|
+
# simply yields directly with zero overhead.
|
|
74
|
+
#
|
|
75
|
+
# @param name [String] the span name (e.g., 'get', 'set', 'delete')
|
|
76
|
+
# @param attributes [Hash] span attributes to merge with defaults.
|
|
77
|
+
# Common attributes include:
|
|
78
|
+
# - 'db.operation' - the operation name
|
|
79
|
+
# - 'server.address' - the target server
|
|
80
|
+
# - 'db.memcached.key_count' - number of keys (for multi operations)
|
|
81
|
+
# @yield the cache operation to trace
|
|
82
|
+
# @return [Object] the result of the block
|
|
83
|
+
# @raise [StandardError] re-raises any exception from the block
|
|
84
|
+
#
|
|
85
|
+
# @example Tracing a set operation
|
|
86
|
+
# trace('set', { 'db.operation' => 'set', 'server.address' => 'localhost:11211' }) do
|
|
87
|
+
# server.set(key, value, ttl)
|
|
88
|
+
# end
|
|
89
|
+
#
|
|
90
|
+
def trace(name, attributes = {})
|
|
91
|
+
return yield unless enabled?
|
|
92
|
+
|
|
93
|
+
tracer.in_span(name, attributes: DEFAULT_ATTRIBUTES.merge(attributes), kind: :client) do |span|
|
|
94
|
+
yield
|
|
95
|
+
rescue StandardError => e
|
|
96
|
+
span.record_exception(e)
|
|
97
|
+
span.status = OpenTelemetry::Trace::Status.error(e.message)
|
|
98
|
+
raise
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# Like trace, but yields the span to allow adding attributes after execution.
|
|
103
|
+
#
|
|
104
|
+
# This is useful for operations where metrics are only known after the
|
|
105
|
+
# operation completes, such as get_multi where hit/miss counts depend
|
|
106
|
+
# on the cache response.
|
|
107
|
+
#
|
|
108
|
+
# When tracing is disabled, yields nil as the span argument.
|
|
109
|
+
#
|
|
110
|
+
# @param name [String] the span name (e.g., 'get_multi')
|
|
111
|
+
# @param attributes [Hash] initial span attributes to merge with defaults
|
|
112
|
+
# @yield [OpenTelemetry::Trace::Span, nil] the span object, or nil if disabled
|
|
113
|
+
# @return [Object] the result of the block
|
|
114
|
+
# @raise [StandardError] re-raises any exception from the block
|
|
115
|
+
#
|
|
116
|
+
# @example Recording hit/miss metrics after get_multi
|
|
117
|
+
# trace_with_result('get_multi', { 'db.operation' => 'get_multi' }) do |span|
|
|
118
|
+
# results = fetch_from_cache(keys)
|
|
119
|
+
# if span
|
|
120
|
+
# span.set_attribute('db.memcached.hit_count', results.size)
|
|
121
|
+
# span.set_attribute('db.memcached.miss_count', keys.size - results.size)
|
|
122
|
+
# end
|
|
123
|
+
# results
|
|
124
|
+
# end
|
|
125
|
+
#
|
|
126
|
+
def trace_with_result(name, attributes = {})
|
|
127
|
+
return yield(nil) unless enabled?
|
|
128
|
+
|
|
129
|
+
tracer.in_span(name, attributes: DEFAULT_ATTRIBUTES.merge(attributes), kind: :client) do |span|
|
|
130
|
+
yield(span)
|
|
131
|
+
rescue StandardError => e
|
|
132
|
+
span.record_exception(e)
|
|
133
|
+
span.status = OpenTelemetry::Trace::Status.error(e.message)
|
|
134
|
+
raise
|
|
135
|
+
end
|
|
136
|
+
end
|
|
137
|
+
end
|
|
138
|
+
end
|
|
139
|
+
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require 'set'
|
|
4
|
+
|
|
3
5
|
module Dalli
|
|
4
6
|
##
|
|
5
7
|
# Contains logic for the pipelined gets implemented by the client.
|
|
@@ -57,7 +59,7 @@ module Dalli
|
|
|
57
59
|
# our set, sending the noop to terminate the set of queries.
|
|
58
60
|
##
|
|
59
61
|
def finish_queries(servers)
|
|
60
|
-
deleted =
|
|
62
|
+
deleted = Set.new
|
|
61
63
|
|
|
62
64
|
servers.each do |server|
|
|
63
65
|
next unless server.connected?
|
|
@@ -67,7 +69,7 @@ module Dalli
|
|
|
67
69
|
rescue Dalli::NetworkError
|
|
68
70
|
raise
|
|
69
71
|
rescue Dalli::DalliError
|
|
70
|
-
deleted
|
|
72
|
+
deleted << server
|
|
71
73
|
end
|
|
72
74
|
end
|
|
73
75
|
|
|
@@ -94,7 +96,7 @@ module Dalli
|
|
|
94
96
|
|
|
95
97
|
def fetch_responses(servers, start_time, timeout, &block)
|
|
96
98
|
# Remove any servers which are not connected
|
|
97
|
-
servers.
|
|
99
|
+
servers.select!(&:connected?)
|
|
98
100
|
return [] if servers.empty?
|
|
99
101
|
|
|
100
102
|
time_left = remaining_time(start_time, timeout)
|
data/lib/dalli/protocol/base.rb
CHANGED
|
@@ -23,10 +23,17 @@ module Dalli
|
|
|
23
23
|
def initialize(attribs, client_options = {})
|
|
24
24
|
hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
|
|
25
25
|
@options = client_options.merge(user_creds)
|
|
26
|
-
@
|
|
26
|
+
@raw_mode = client_options[:raw]
|
|
27
|
+
@value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
|
|
27
28
|
@connection_manager = ConnectionManager.new(hostname, port, socket_type, @options)
|
|
28
29
|
end
|
|
29
30
|
|
|
31
|
+
# Returns true if client is in raw mode (no serialization/compression).
|
|
32
|
+
# In raw mode, we can skip requesting bitflags from the server.
|
|
33
|
+
def raw_mode?
|
|
34
|
+
@raw_mode
|
|
35
|
+
end
|
|
36
|
+
|
|
30
37
|
# Chokepoint method for error handling and ensuring liveness
|
|
31
38
|
def request(opkey, *args)
|
|
32
39
|
verify_state(opkey)
|
|
@@ -22,6 +22,7 @@ module Dalli
|
|
|
22
22
|
def get(key, options = nil)
|
|
23
23
|
req = RequestFormatter.standard_request(opkey: :get, key: key)
|
|
24
24
|
write(req)
|
|
25
|
+
@connection_manager.flush
|
|
25
26
|
response_processor.get(cache_nils: cache_nils?(options))
|
|
26
27
|
end
|
|
27
28
|
|
|
@@ -33,12 +34,14 @@ module Dalli
|
|
|
33
34
|
ttl = TtlSanitizer.sanitize(ttl)
|
|
34
35
|
req = RequestFormatter.standard_request(opkey: :gat, key: key, ttl: ttl)
|
|
35
36
|
write(req)
|
|
37
|
+
@connection_manager.flush
|
|
36
38
|
response_processor.get(cache_nils: cache_nils?(options))
|
|
37
39
|
end
|
|
38
40
|
|
|
39
41
|
def touch(key, ttl)
|
|
40
42
|
ttl = TtlSanitizer.sanitize(ttl)
|
|
41
43
|
write(RequestFormatter.standard_request(opkey: :touch, key: key, ttl: ttl))
|
|
44
|
+
@connection_manager.flush
|
|
42
45
|
response_processor.generic_response
|
|
43
46
|
end
|
|
44
47
|
|
|
@@ -47,6 +50,7 @@ module Dalli
|
|
|
47
50
|
def cas(key)
|
|
48
51
|
req = RequestFormatter.standard_request(opkey: :get, key: key)
|
|
49
52
|
write(req)
|
|
53
|
+
@connection_manager.flush
|
|
50
54
|
response_processor.data_cas_response
|
|
51
55
|
end
|
|
52
56
|
|
|
@@ -81,6 +85,7 @@ module Dalli
|
|
|
81
85
|
value: value, bitflags: bitflags,
|
|
82
86
|
ttl: ttl, cas: cas)
|
|
83
87
|
write(req)
|
|
88
|
+
@connection_manager.flush unless quiet?
|
|
84
89
|
response_processor.storage_response unless quiet?
|
|
85
90
|
end
|
|
86
91
|
# rubocop:enable Metrics/ParameterLists
|
|
@@ -97,6 +102,7 @@ module Dalli
|
|
|
97
102
|
|
|
98
103
|
def write_append_prepend(opkey, key, value)
|
|
99
104
|
write(RequestFormatter.standard_request(opkey: opkey, key: key, value: value))
|
|
105
|
+
@connection_manager.flush unless quiet?
|
|
100
106
|
response_processor.no_body_response unless quiet?
|
|
101
107
|
end
|
|
102
108
|
|
|
@@ -105,6 +111,7 @@ module Dalli
|
|
|
105
111
|
opkey = quiet? ? :deleteq : :delete
|
|
106
112
|
req = RequestFormatter.standard_request(opkey: opkey, key: key, cas: cas)
|
|
107
113
|
write(req)
|
|
114
|
+
@connection_manager.flush unless quiet?
|
|
108
115
|
response_processor.delete unless quiet?
|
|
109
116
|
end
|
|
110
117
|
|
|
@@ -139,6 +146,7 @@ module Dalli
|
|
|
139
146
|
initial ||= 0
|
|
140
147
|
write(RequestFormatter.decr_incr_request(opkey: opkey, key: key,
|
|
141
148
|
count: count, initial: initial, expiry: expiry))
|
|
149
|
+
@connection_manager.flush unless quiet?
|
|
142
150
|
response_processor.decr_incr unless quiet?
|
|
143
151
|
end
|
|
144
152
|
|
|
@@ -146,6 +154,7 @@ module Dalli
|
|
|
146
154
|
def flush(ttl = 0)
|
|
147
155
|
opkey = quiet? ? :flushq : :flush
|
|
148
156
|
write(RequestFormatter.standard_request(opkey: opkey, ttl: ttl))
|
|
157
|
+
@connection_manager.flush unless quiet?
|
|
149
158
|
response_processor.no_body_response unless quiet?
|
|
150
159
|
end
|
|
151
160
|
|
|
@@ -159,22 +168,26 @@ module Dalli
|
|
|
159
168
|
def stats(info = '')
|
|
160
169
|
req = RequestFormatter.standard_request(opkey: :stat, key: info)
|
|
161
170
|
write(req)
|
|
171
|
+
@connection_manager.flush
|
|
162
172
|
response_processor.stats
|
|
163
173
|
end
|
|
164
174
|
|
|
165
175
|
def reset_stats
|
|
166
176
|
write(RequestFormatter.standard_request(opkey: :stat, key: 'reset'))
|
|
177
|
+
@connection_manager.flush
|
|
167
178
|
response_processor.reset
|
|
168
179
|
end
|
|
169
180
|
|
|
170
181
|
def version
|
|
171
182
|
write(RequestFormatter.standard_request(opkey: :version))
|
|
183
|
+
@connection_manager.flush
|
|
172
184
|
response_processor.version
|
|
173
185
|
end
|
|
174
186
|
|
|
175
187
|
def write_noop
|
|
176
188
|
req = RequestFormatter.standard_request(opkey: :noop)
|
|
177
189
|
write(req)
|
|
190
|
+
@connection_manager.flush
|
|
178
191
|
end
|
|
179
192
|
|
|
180
193
|
require_relative 'binary/request_formatter'
|
|
@@ -53,6 +53,7 @@ module Dalli
|
|
|
53
53
|
Dalli.logger.debug { "Dalli::Server#connect #{name}" }
|
|
54
54
|
|
|
55
55
|
@sock = memcached_socket
|
|
56
|
+
@sock.sync = false # Enable buffered I/O for better performance
|
|
56
57
|
@pid = PIDCache.pid
|
|
57
58
|
@request_in_progress = false
|
|
58
59
|
rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
|
|
@@ -166,6 +167,12 @@ module Dalli
|
|
|
166
167
|
error_on_request!(e)
|
|
167
168
|
end
|
|
168
169
|
|
|
170
|
+
def flush
|
|
171
|
+
@sock.flush
|
|
172
|
+
rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS => e
|
|
173
|
+
error_on_request!(e)
|
|
174
|
+
end
|
|
175
|
+
|
|
169
176
|
# Non-blocking read. Here to support the operation
|
|
170
177
|
# of the get_multi operation
|
|
171
178
|
def read_nonblock
|
|
@@ -37,9 +37,12 @@ module Dalli
|
|
|
37
37
|
# - l<N>: Seconds since last access
|
|
38
38
|
def self.meta_get(key:, value: true, return_cas: false, ttl: nil, base64: false, quiet: false,
|
|
39
39
|
vivify_ttl: nil, recache_ttl: nil,
|
|
40
|
-
return_hit_status: false, return_last_access: false, skip_lru_bump: false
|
|
40
|
+
return_hit_status: false, return_last_access: false, skip_lru_bump: false,
|
|
41
|
+
skip_flags: false)
|
|
41
42
|
cmd = "mg #{key}"
|
|
42
|
-
|
|
43
|
+
# In raw mode (skip_flags: true), we don't request bitflags since they're not used.
|
|
44
|
+
# This saves 2 bytes per request and skips parsing on response.
|
|
45
|
+
cmd << (skip_flags ? ' v' : ' v f') if value
|
|
43
46
|
cmd << ' c' if return_cas
|
|
44
47
|
cmd << ' b' if base64
|
|
45
48
|
cmd << " T#{ttl}" if ttl
|
data/lib/dalli/protocol/meta.rb
CHANGED
|
@@ -25,21 +25,28 @@ module Dalli
|
|
|
25
25
|
# Retrieval Commands
|
|
26
26
|
def get(key, options = nil)
|
|
27
27
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
28
|
-
|
|
28
|
+
# Skip bitflags in raw mode - saves 2 bytes per request and skips parsing
|
|
29
|
+
skip_flags = raw_mode? || (options && options[:raw])
|
|
30
|
+
req = RequestFormatter.meta_get(key: encoded_key, base64: base64, skip_flags: skip_flags)
|
|
29
31
|
write(req)
|
|
32
|
+
@connection_manager.flush
|
|
30
33
|
response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
|
|
31
34
|
end
|
|
32
35
|
|
|
33
36
|
def quiet_get_request(key)
|
|
34
37
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
35
|
-
|
|
38
|
+
# Skip bitflags in raw mode - saves 2 bytes per request and skips parsing
|
|
39
|
+
RequestFormatter.meta_get(key: encoded_key, return_cas: true, base64: base64, quiet: true,
|
|
40
|
+
skip_flags: raw_mode?)
|
|
36
41
|
end
|
|
37
42
|
|
|
38
43
|
def gat(key, ttl, options = nil)
|
|
39
44
|
ttl = TtlSanitizer.sanitize(ttl)
|
|
40
45
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
41
|
-
|
|
46
|
+
skip_flags = raw_mode? || (options && options[:raw])
|
|
47
|
+
req = RequestFormatter.meta_get(key: encoded_key, ttl: ttl, base64: base64, skip_flags: skip_flags)
|
|
42
48
|
write(req)
|
|
49
|
+
@connection_manager.flush
|
|
43
50
|
response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
|
|
44
51
|
end
|
|
45
52
|
|
|
@@ -48,6 +55,7 @@ module Dalli
|
|
|
48
55
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
49
56
|
req = RequestFormatter.meta_get(key: encoded_key, ttl: ttl, value: false, base64: base64)
|
|
50
57
|
write(req)
|
|
58
|
+
@connection_manager.flush
|
|
51
59
|
response_processor.meta_get_without_value
|
|
52
60
|
end
|
|
53
61
|
|
|
@@ -57,6 +65,7 @@ module Dalli
|
|
|
57
65
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
58
66
|
req = RequestFormatter.meta_get(key: encoded_key, value: true, return_cas: true, base64: base64)
|
|
59
67
|
write(req)
|
|
68
|
+
@connection_manager.flush
|
|
60
69
|
response_processor.meta_get_with_value_and_cas
|
|
61
70
|
end
|
|
62
71
|
|
|
@@ -93,6 +102,7 @@ module Dalli
|
|
|
93
102
|
return_last_access: options[:return_last_access], skip_lru_bump: options[:skip_lru_bump]
|
|
94
103
|
)
|
|
95
104
|
write(req)
|
|
105
|
+
@connection_manager.flush
|
|
96
106
|
response_processor.meta_get_with_metadata(
|
|
97
107
|
cache_nils: cache_nils?(options), return_hit_status: options[:return_hit_status],
|
|
98
108
|
return_last_access: options[:return_last_access]
|
|
@@ -110,6 +120,7 @@ module Dalli
|
|
|
110
120
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
111
121
|
req = RequestFormatter.meta_delete(key: encoded_key, cas: cas, base64: base64, stale: true)
|
|
112
122
|
write(req)
|
|
123
|
+
@connection_manager.flush
|
|
113
124
|
response_processor.meta_delete
|
|
114
125
|
end
|
|
115
126
|
|
|
@@ -146,6 +157,7 @@ module Dalli
|
|
|
146
157
|
write(req)
|
|
147
158
|
write(value)
|
|
148
159
|
write(TERMINATOR)
|
|
160
|
+
@connection_manager.flush unless quiet
|
|
149
161
|
end
|
|
150
162
|
# rubocop:enable Metrics/ParameterLists
|
|
151
163
|
|
|
@@ -168,6 +180,7 @@ module Dalli
|
|
|
168
180
|
write(req)
|
|
169
181
|
write(value)
|
|
170
182
|
write(TERMINATOR)
|
|
183
|
+
@connection_manager.flush unless quiet?
|
|
171
184
|
end
|
|
172
185
|
# rubocop:enable Metrics/ParameterLists
|
|
173
186
|
|
|
@@ -177,6 +190,7 @@ module Dalli
|
|
|
177
190
|
req = RequestFormatter.meta_delete(key: encoded_key, cas: cas,
|
|
178
191
|
base64: base64, quiet: quiet?)
|
|
179
192
|
write(req)
|
|
193
|
+
@connection_manager.flush unless quiet?
|
|
180
194
|
response_processor.meta_delete unless quiet?
|
|
181
195
|
end
|
|
182
196
|
|
|
@@ -202,12 +216,14 @@ module Dalli
|
|
|
202
216
|
encoded_key, base64 = KeyRegularizer.encode(key)
|
|
203
217
|
write(RequestFormatter.meta_arithmetic(key: encoded_key, delta: delta, initial: initial, incr: incr, ttl: ttl,
|
|
204
218
|
quiet: quiet?, base64: base64))
|
|
219
|
+
@connection_manager.flush unless quiet?
|
|
205
220
|
response_processor.decr_incr unless quiet?
|
|
206
221
|
end
|
|
207
222
|
|
|
208
223
|
# Other Commands
|
|
209
224
|
def flush(delay = 0)
|
|
210
225
|
write(RequestFormatter.flush(delay: delay))
|
|
226
|
+
@connection_manager.flush unless quiet?
|
|
211
227
|
response_processor.flush unless quiet?
|
|
212
228
|
end
|
|
213
229
|
|
|
@@ -220,21 +236,25 @@ module Dalli
|
|
|
220
236
|
|
|
221
237
|
def stats(info = nil)
|
|
222
238
|
write(RequestFormatter.stats(info))
|
|
239
|
+
@connection_manager.flush
|
|
223
240
|
response_processor.stats
|
|
224
241
|
end
|
|
225
242
|
|
|
226
243
|
def reset_stats
|
|
227
244
|
write(RequestFormatter.stats('reset'))
|
|
245
|
+
@connection_manager.flush
|
|
228
246
|
response_processor.reset
|
|
229
247
|
end
|
|
230
248
|
|
|
231
249
|
def version
|
|
232
250
|
write(RequestFormatter.version)
|
|
251
|
+
@connection_manager.flush
|
|
233
252
|
response_processor.version
|
|
234
253
|
end
|
|
235
254
|
|
|
236
255
|
def write_noop
|
|
237
256
|
write(RequestFormatter.meta_noop)
|
|
257
|
+
@connection_manager.flush
|
|
238
258
|
end
|
|
239
259
|
|
|
240
260
|
def authenticate_connection
|
data/lib/dalli/version.rb
CHANGED
data/lib/dalli.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: dalli
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 4.
|
|
4
|
+
version: 4.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Peter M. Goldstein
|
|
@@ -40,6 +40,7 @@ files:
|
|
|
40
40
|
- lib/dalli/cas/client.rb
|
|
41
41
|
- lib/dalli/client.rb
|
|
42
42
|
- lib/dalli/compressor.rb
|
|
43
|
+
- lib/dalli/instrumentation.rb
|
|
43
44
|
- lib/dalli/key_manager.rb
|
|
44
45
|
- lib/dalli/options.rb
|
|
45
46
|
- lib/dalli/pid_cache.rb
|