emb 0.1.2 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cc1be692fe3d651429a292c2e2d0b912aa1bb8af22dbf56b9390d7e8a00fe516
4
- data.tar.gz: 433caba4f427a465b9717970884ea64e12c4169fca3f54df8e91858dc1ef21bb
3
+ metadata.gz: 424790b9a9b835dd7febbdfa1df2e5f8b6a2a9b89290b5e653467fcc09e4f68f
4
+ data.tar.gz: d21f28bb4c79deaefa9bab91247f41ff5e84f869b422727939ab1f07a940eb34
5
5
  SHA512:
6
- metadata.gz: 553e753f5a90217312bd7f6e2e9240c16747b6fcc01e876cdabdc2b33a2c430cb937cb214926ee8a4d3fb77a66a45439ad6fb6c1be1885617fb0fa68026ee336
7
- data.tar.gz: 478a17bbf82026da26580ead85fae29bd6f8b1350bbe841aa32b098ce0845e3005641a80f388fd7912df0ba3e2c9652a98d8dfda8769a8d48c3ceb1d3eed1ae4
6
+ metadata.gz: eacbde1ca9ccf217bd53cb9028f8a5192427754abcee678373d38f16b08ce4ae00b5b905909cb0d7ef0f07fa04b39c0ad112c820c6f4fd6d3c98ba06708af854
7
+ data.tar.gz: 8605fe8eaede9567d6506f5247ef85dfcb32b9c1c1fa719d00ce2446233b0518cbd336e6f496c8449ac41ab33545776b05d1febe2bf3d5a576aafa47afa60d9c
data/Gemfile CHANGED
@@ -9,3 +9,7 @@ gem 'redis-client'
9
9
 
10
10
  gem 'rake', require: false
11
11
  gem 'rspec', require: false
12
+
13
+ gem 'rubocop', require: false
14
+ gem 'rubocop-rake', require: false
15
+ gem 'rubocop-rspec', require: false
data/README.md CHANGED
@@ -20,16 +20,118 @@ gem install emb
20
20
 
21
21
  ## Setup
22
22
 
23
- Configure the connection pool (defaults shown):
23
+ The client connects to an emb server via the Redis protocol (RESP2). Configure with a URL,
24
+ host/port, or rely on defaults and environment variables:
24
25
 
25
26
  ```ruby
26
27
  require "emb"
27
28
 
28
- Emb.setup(host: "localhost", port: 6379, pool: 5)
29
+ # URL (Redis URL format)
30
+ Emb.setup(url: "redis://localhost:6379")
31
+
32
+ # Or individual params
33
+ Emb.setup(host: "localhost", port: 6379)
34
+
35
+ # Or rely on defaults
36
+ Emb.setup
29
37
  ```
30
38
 
31
39
  `Emb.config` is an alias for `Emb.setup`.
32
40
 
41
+ ### Configuration sources (priority order)
42
+
43
+ 1. Explicit `url:` or `host:`/`port:` arguments
44
+ 2. `EMB_URL` environment variable
45
+ 3. Default: `redis://localhost:6379`
46
+
47
+ ### Connection pool
48
+
49
+ ```ruby
50
+ Emb.setup(url: "redis://localhost:6379", pool: 10)
51
+ ```
52
+
53
+ ### Authentication
54
+
55
+ If the server is configured with a password, include it in the URL:
56
+
57
+ ```ruby
58
+ # Password as URL userinfo
59
+ Emb.setup(url: "redis://:hunter2@localhost:6379")
60
+ ```
61
+
62
+ The `RedisClient` gem handles `AUTH` automatically on connect when a password
63
+ is embedded in the URL. This works correctly with connection pooling — every
64
+ connection in the pool authenticates on creation.
65
+
66
+ Manual authentication is also possible but not recommended for pooled connections:
67
+
68
+ ```ruby
69
+ Emb.send_command("AUTH", "hunter2") # only authenticates one connection
70
+ ```
71
+
72
+ ### Redis client options
73
+
74
+ Any `RedisClient` option can be forwarded through `Emb.setup` or `Emb.new`:
75
+
76
+ ```ruby
77
+ Emb.setup(
78
+ url: "redis://localhost:6379",
79
+ pool: 10,
80
+ connect_timeout: 2,
81
+ read_timeout: 10,
82
+ write_timeout: 5,
83
+ reconnect_attempts: 5,
84
+ ssl: true,
85
+ ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_PEER },
86
+ driver: :hiredis,
87
+ inherit_socket: true
88
+ )
89
+ ```
90
+
91
+ See the [redis-client documentation](https://github.com/redis-rb/redis-client) for
92
+ all available options. Only `pool` is handled by the gem — everything else passes
93
+ through to `RedisClient.new`.
94
+
95
+ ## Instance-based clients
96
+
97
+ Create independent clients to connect to multiple servers or use different configurations:
98
+
99
+ ```ruby
100
+ default = Emb.setup(url: "redis://localhost:6379")
101
+ other = Emb.new(url: "redis://:hunter2@10.0.0.1:6380")
102
+
103
+ default.ping # => "PONG"
104
+ other.ping # => "PONG"
105
+ ```
106
+
107
+ Each client has its own connection pool and model proxy registry:
108
+
109
+ ```ruby
110
+ c1 = Emb.new(url: "redis://server1:6379")
111
+ c2 = Emb.new(url: "redis://server2:6379")
112
+
113
+ c1[:minilm] != c2[:minilm] # separate proxies
114
+ ```
115
+
116
+ ### Global convenience API
117
+
118
+ When you don't need multiple clients, use the module-level methods:
119
+
120
+ ```ruby
121
+ Emb.setup
122
+
123
+ Emb[:minilm]["hello"] # proxy access
124
+ Emb.models # list models
125
+ Emb.info(:minilm) # model info
126
+ Emb.stats # server stats
127
+ Emb.help # command reference
128
+ Emb.ping # health check
129
+ ```
130
+
131
+ These all delegate to a lazily-initialized default client. No explicit `setup` call
132
+ is required for simple cases — the default client connects to `redis://localhost:6379`
133
+ automatically.
134
+
33
135
  ## Usage
34
136
 
35
137
  ### Single text
@@ -39,6 +141,13 @@ result = Emb[:minilm]["hello world"]
39
141
  # => [0.0123, -0.0456, 0.0789, ...] (384 floats)
40
142
  ```
41
143
 
144
+ With an instance-based client:
145
+
146
+ ```ruby
147
+ client = Emb.new(url: "redis://localhost:6379")
148
+ result = client[:minilm]["hello world"]
149
+ ```
150
+
42
151
  ### Multiple texts
43
152
 
44
153
  ```ruby
@@ -51,11 +160,21 @@ results = Emb[:minilm]["hello", "world"]
51
160
  Send texts to different models in one round trip:
52
161
 
53
162
  ```ruby
54
- Emb.multi do |m|
163
+ results = Emb.multi do |m|
164
+ m[:minilm]["hello"]
165
+ m[:bge]["world"]
166
+ end
167
+ # => [[0.0123, ...], [-0.0456, ...]]
168
+ # Results are unpacked from float32 binary — same format as single embeddings
169
+ ```
170
+
171
+ Works the same on instance clients:
172
+
173
+ ```ruby
174
+ client.multi do |m|
55
175
  m[:minilm]["hello"]
56
176
  m[:bge]["world"]
57
177
  end
58
- # => EMB.MULTI minilm "hello" bge "world"
59
178
  ```
60
179
 
61
180
  ### Commands
@@ -68,7 +187,23 @@ Emb.help # => command reference string
68
187
  Emb.ping # => "PONG"
69
188
  ```
70
189
 
71
- ## Testing end to end
190
+ ## Development
191
+
192
+ ### Console
193
+
194
+ Start an IRB session with the gem loaded:
195
+
196
+ ```bash
197
+ bundle exec rake console
198
+ ```
199
+
200
+ ### Lint
201
+
202
+ ```bash
203
+ bundle exec rubocop
204
+ ```
205
+
206
+ ### Tests
72
207
 
73
208
  Start the emb server, then run the test suite:
74
209
 
@@ -80,4 +215,5 @@ Start the emb server, then run the test suite:
80
215
  bundle exec rake
81
216
  ```
82
217
 
83
- Tests cover all commands: `EMB`, `EMB.MODELS`, `EMB.INFO`, `EMB.HELP`, `PING`, and `EMB.MULTI`.
218
+ Tests cover all commands: `EMB`, `EMB.MODELS`, `EMB.INFO`, `EMB.HELP`, `PING`,
219
+ and `EMB.MULTI`, plus instance-based clients, URL configuration, and connection pooling.
data/lib/emb/client.rb CHANGED
@@ -1,36 +1,99 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "connection_pool"
4
- require "redis_client"
3
+ require 'connection_pool'
4
+ require 'redis_client'
5
5
 
6
6
  module Emb
7
- @pool = nil
7
+ DEFAULTS = { host: 'localhost', port: 6379, pool: 5 }.freeze
8
8
 
9
- DEFAULTS = { host: "localhost", port: 6379, pool: 5 }.freeze
9
+ class Client
10
+ attr_reader :pool
11
+
12
+ def initialize(pool: DEFAULTS[:pool], **redis_options)
13
+ url = extract_url!(redis_options)
14
+ redis_options[:host] ||= DEFAULTS[:host] unless url
15
+ redis_options[:port] ||= DEFAULTS[:port] unless url
16
+ redis_options[:protocol] ||= 2
17
+ redis_options[:reconnect_attempts] ||= 3
10
18
 
11
- class << self
12
- def setup(host: DEFAULTS[:host], port: DEFAULTS[:port], pool: DEFAULTS[:pool])
13
19
  @pool = ConnectionPool.new(size: pool) do
14
- RedisClient.new(host: host, port: port, protocol: 2, reconnect_attempts: 3)
20
+ RedisClient.new(url: url, **redis_options)
15
21
  end
16
- end
17
22
 
18
- alias_method :config, :setup
23
+ @registry = {}
24
+ end
19
25
 
20
26
  def send_command(*args)
21
- pool.with { |r| r.call(*args) }
22
- end
27
+ return @pool.with { |r| r.call(*args) } unless Emb.debug?
23
28
 
24
- private
29
+ start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
30
+ result = @pool.with { |r| r.call(*args) }
31
+ elapsed = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000
32
+
33
+ $stdout.puts "[EMB] #{args.map(&:inspect).join(' ')} (#{format('%.2f', elapsed)}ms)"
34
+
35
+ result
36
+ end
25
37
 
26
- def pool
27
- @pool ||= default_pool
38
+ def [](name)
39
+ @registry[name] ||= Proxy.new(self, name.to_sym)
28
40
  end
29
41
 
30
- def default_pool
31
- ConnectionPool.new(size: DEFAULTS[:pool]) do
32
- RedisClient.new(host: DEFAULTS[:host], port: DEFAULTS[:port], protocol: 2, reconnect_attempts: 3)
42
+ def models
43
+ raw = send_command('EMB.MODELS')
44
+ return [] if raw.nil?
45
+
46
+ raw.map do |name, dim, status|
47
+ { name: name, dim: dim.to_i, status: status }
33
48
  end
34
49
  end
50
+
51
+ def info(name)
52
+ raw = send_command('EMB.INFO', name.to_s)
53
+ return {} if raw.nil?
54
+
55
+ raw
56
+ .each_slice(2)
57
+ .to_h { |k, v| [k.to_sym, v] }
58
+ end
59
+
60
+ def stats = send_command('EMB.STATS')
61
+
62
+ def help = send_command('EMB.HELP')
63
+
64
+ def ping = send_command('PING')
65
+
66
+ def ready
67
+ send_command('EMB.READY')
68
+
69
+ "ready"
70
+ rescue RedisClient::CommandError => e
71
+ e.message
72
+ end
73
+
74
+ def ready?
75
+ ready
76
+
77
+ true
78
+ rescue RedisClient::CommandError
79
+ false
80
+ end
81
+
82
+ def reset_registry!
83
+ @registry = {}
84
+ end
85
+
86
+ def multi(&)
87
+ mp = MultiProxy.new(self)
88
+ yield mp
89
+ mp.run
90
+ end
91
+
92
+ private
93
+
94
+ def extract_url!(opts)
95
+ url = opts.delete(:url)
96
+ url.nil? ? ENV.fetch('EMB_URL', nil) : url
97
+ end
35
98
  end
36
99
  end
data/lib/emb/multi.rb CHANGED
@@ -2,7 +2,8 @@
2
2
 
3
3
  module Emb
4
4
  class MultiProxy
5
- def initialize
5
+ def initialize(client)
6
+ @client = client
6
7
  @pairs = []
7
8
  end
8
9
 
@@ -13,7 +14,9 @@ module Emb
13
14
  def run
14
15
  args = @pairs.flat_map { |pair| [pair[:model].to_s, pair[:text]] }
15
16
 
16
- Emb.send_command("EMB.MULTI", *args)
17
+ @client
18
+ .send_command('EMB.MULTI', *args)
19
+ .map { |entry| entry.unpack('e*') }
17
20
  end
18
21
 
19
22
  class PairCollector
@@ -23,19 +26,8 @@ module Emb
23
26
  end
24
27
 
25
28
  def [](text)
26
- @pairs << {
27
- model: @model, text: text }
29
+ @pairs << { model: @model, text: text }
28
30
  end
29
31
  end
30
32
  end
31
-
32
- class << self
33
- def multi
34
- mp = MultiProxy.new
35
-
36
- yield mp
37
-
38
- mp.run
39
- end
40
- end
41
33
  end
data/lib/emb/proxy.rb CHANGED
@@ -1,28 +1,17 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Emb
4
- @registry = {}
5
-
6
- class << self
7
- def [](name)
8
- @registry[name] ||= Proxy.new(name.to_sym)
9
- end
10
-
11
- def reset_registry!
12
- @registry.clear
13
- end
14
- end
15
-
16
4
  class Proxy
17
5
  attr_reader :name
18
6
 
19
- def initialize(name)
7
+ def initialize(client, name)
8
+ @client = client
20
9
  @name = name
21
10
  end
22
11
 
23
12
  def [](text, *texts)
24
- set = Array(Emb.send_command("EMB", @name.to_s, text, *texts))
25
- result = set.map { |entry| entry.unpack("e*") }
13
+ set = Array(@client.send_command('EMB', @name.to_s, text, *texts))
14
+ result = set.map { |entry| entry.unpack('e*') }
26
15
 
27
16
  return result.first if result.size == 1
28
17
 
data/lib/emb/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Emb
4
- VERSION = Gem.loaded_specs['emb'].version.to_s
4
+ VERSION = Gem.loaded_specs['emb']&.version&.to_s || '0.0.0'
5
5
  end
data/lib/emb.rb CHANGED
@@ -1,36 +1,41 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative "emb/version"
4
- require_relative "emb/client"
5
- require_relative "emb/proxy"
6
- require_relative "emb/multi"
3
+ require_relative 'emb/version'
4
+ require_relative 'emb/client'
5
+ require_relative 'emb/proxy'
6
+ require_relative 'emb/multi'
7
7
 
8
8
  module Emb
9
9
  class << self
10
- def models
11
- raw = send_command("EMB.MODELS")
10
+ def new(...) = Client.new(...)
12
11
 
13
- return [] if raw.nil?
14
-
15
- raw.map do |name, dim, status|
16
- { name: name, dim: dim.to_i, status: status }
17
- end
12
+ def setup(...)
13
+ @default_client = Client.new(...)
18
14
  end
19
15
 
20
- def info(name)
21
- raw = send_command("EMB.INFO", name.to_s)
22
-
23
- return {} if raw.nil?
24
-
25
- raw
26
- .each_slice(2)
27
- .to_h { |k, v| [k.to_sym, v] }
16
+ alias config setup
17
+
18
+ def [](name) = default_client[name]
19
+ def models = default_client.models
20
+ def info(name) = default_client.info(name)
21
+ def stats = default_client.stats
22
+ def help = default_client.help
23
+ def ping = default_client.ping
24
+ def ready = default_client.ready
25
+ def ready? = default_client.ready?
26
+ def multi(&) = default_client.multi(&)
27
+ def reset_registry! = default_client.reset_registry!
28
+ def debug? = @debug
29
+ def send_command(*) = default_client.send_command(*)
30
+
31
+ def debug!
32
+ @debug = true
28
33
  end
29
34
 
30
- def stats = send_command("EMB.STATS")
35
+ private
31
36
 
32
- def help = send_command("EMB.HELP")
33
-
34
- def ping = send_command("PING")
37
+ def default_client
38
+ @default_client ||= Client.new
39
+ end
35
40
  end
36
41
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: emb
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.2
4
+ version: 0.2.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - elcuervo
@@ -38,20 +38,6 @@ dependencies:
38
38
  - - "~>"
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0.24'
41
- - !ruby/object:Gem::Dependency
42
- name: rspec
43
- requirement: !ruby/object:Gem::Requirement
44
- requirements:
45
- - - "~>"
46
- - !ruby/object:Gem::Version
47
- version: '3.13'
48
- type: :development
49
- prerelease: false
50
- version_requirements: !ruby/object:Gem::Requirement
51
- requirements:
52
- - - "~>"
53
- - !ruby/object:Gem::Version
54
- version: '3.13'
55
41
  description:
56
42
  email:
57
43
  - elcuervo@elcuervo.net