taba-client 0.3.20.rc1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/README.md +81 -0
- data/lib/taba/calls.rb +176 -0
- data/lib/taba/cancel_token.rb +70 -0
- data/lib/taba/client.rb +359 -0
- data/lib/taba/config.rb +52 -0
- data/lib/taba/error.rb +54 -0
- data/lib/taba/json_fields.rb +175 -0
- data/lib/taba/models.rb +226 -0
- data/lib/taba/options.rb +103 -0
- data/lib/taba/transport.rb +347 -0
- data/lib/taba/version.rb +4 -0
- data/lib/taba.rb +18 -0
- metadata +56 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 0db7633a4ecc2e4f1a60e87cf00905f08e7141f4ddccf805975cdb9388608bd1
|
|
4
|
+
data.tar.gz: 165dc7a0564ff85e73c69610093ff5f8ef5f5eb27ba564087ca36762d76bc732
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: e16982c9ff2a74ac8a9e821b2a269ab423fa1f7566fc7a75a456f3da93ae9deb8b8a4eafb45c63aba3fdeb2b06237650e7c43b31a6cdd07eddcd3997938bc570
|
|
7
|
+
data.tar.gz: 6b84aa22836229f94e8747afbf7fbf90f28c6c431ec5518d17d3a84ae35d2d21a6b80450752f7c782680b9028c130ded4a93f99ca269bf0f7f35f2175888dd98
|
data/README.md
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# taba-client (Ruby)
|
|
2
|
+
|
|
3
|
+
The Ruby client of [Taba](https://github.com/rafaelpontani/taba): the nine operations of the client API, `subscribe`, consuming with a handler (with automatic renewal of the invisibility) and the call retries of the common contract. **No dependencies** (only the standard library: `net/http`, `json`).
|
|
4
|
+
|
|
5
|
+
> Other languages: [Português (Brasil)](https://github.com/rafaelpontani/taba/blob/main/doc/pt-BR/21-conectores.md) · [all languages](https://github.com/rafaelpontani/taba/blob/main/doc/README.md)
|
|
6
|
+
|
|
7
|
+
- **Ruby 2.1 or later** (it runs the same code on 2.1 and on 3.x).
|
|
8
|
+
- `Taba::Client` is safe for concurrent use: each thread has its own connection.
|
|
9
|
+
|
|
10
|
+
```ruby
|
|
11
|
+
require 'taba'
|
|
12
|
+
|
|
13
|
+
client = Taba::Client.new('http://localhost:8080', token: ENV['TABA_TOKEN'])
|
|
14
|
+
|
|
15
|
+
client.create_topic(name: 'orders', type: 'queue')
|
|
16
|
+
client.publish_json('orders', order: 9001)
|
|
17
|
+
|
|
18
|
+
# Consume with a block: the message is deleted if the block does not raise.
|
|
19
|
+
client.consume_with('orders', wait_time: 20) do |context|
|
|
20
|
+
puts context.message.body_text
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
# Or the whole loop, until the token is cancelled.
|
|
24
|
+
# token = Taba::CancelToken.new
|
|
25
|
+
# client.subscribe('orders', cancel: token) { |message| puts message.json }
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
## Installing
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
gem install taba-client
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Or, with Bundler, `gem 'taba-client'`. The gem has the version of the Taba release it came with and is published from the release workflow of this repository with RubyGems Trusted Publishing. A release candidate is published in the RubyGems pre-release form (`0.3.19.rc1`), which `gem` only installs when asked (`gem install --pre taba-client`). Each [Release](https://github.com/rafaelpontani/taba/releases) also has the same gem, `taba-client-<release>.gem`, with its checksum in `SHA256SUMS` and its provenance (`gh attestation verify <file> -R rafaelpontani/taba`).
|
|
35
|
+
|
|
36
|
+
With Bundler, straight from the source (a branch, a tag or a commit):
|
|
37
|
+
|
|
38
|
+
```ruby
|
|
39
|
+
gem 'taba-client', git: 'https://github.com/rafaelpontani/taba.git', branch: 'release',
|
|
40
|
+
glob: 'connectors/ruby/*.gemspec'
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Declaring a topic
|
|
44
|
+
|
|
45
|
+
`create_or_update_topic(request, recreate: nil)` (`PUT /topics/{topic}`; the fields can also go as options) creates the topic or brings it to the declared state, and returns a `Taba::TopicPut` with `topic` and `outcome` (`created`, `updated` or `recreated`). What is left out goes back to the server default (a full replacement, unlike the update); another type, `fifo` or `volatile` is the error `409 topic.immutable-mismatch` unless `recreate` is `if-empty` (only when the queue and its dead-letter are empty, `409 topic.not-empty` otherwise) or `always` (drops the messages; new `id`). After a `503` the call is repeated like the others. `delete_topic(name, if_empty: true)` only removes a queue with no message. `create_topic` does not change (`409` when it exists). Details: [Create or update a topic](https://github.com/rafaelpontani/taba/blob/main/doc/en-US/16-client-api.md#create-or-update-a-topic-put-topicstopic).
|
|
46
|
+
|
|
47
|
+
## Batches
|
|
48
|
+
|
|
49
|
+
Many messages in one request, next to the single calls (optional): `publish_batch(topic, [Taba::BatchMessage.new(body, content_type: ...)])`, `ack_batch(topic, messages)` and `change_visibility_batch(topic, messages, seconds)` return one `BatchResult` per item (`ok?`, `status`, `id`, `error_code`). Each item is answered as its single call, in order, and an item that failed comes back with its `status` and error code instead of raising. The bodies go in base64, so any type fits. Details: [Messages in batch](https://github.com/rafaelpontani/taba/blob/main/doc/en-US/16-client-api.md#messages-in-batch-post-messagesbatch-ack-and-visibility).
|
|
50
|
+
|
|
51
|
+
## Configuration
|
|
52
|
+
|
|
53
|
+
`Taba::Client.new(config)` with a `Taba::Config`, or `Taba::Client.new(base_url, **options)` with the same fields. The client does not read environment variables: `TABA_*` belongs to the server.
|
|
54
|
+
|
|
55
|
+
| Field | Default | |
|
|
56
|
+
|---|---|---|
|
|
57
|
+
| `base_url` | required | `http://host:port` or `https://...` |
|
|
58
|
+
| `token` | none | `Authorization: Bearer` |
|
|
59
|
+
| `connect_timeout` | `5.0` | seconds |
|
|
60
|
+
| `request_timeout` | `30.0` | seconds; `consume` uses `wait_time + 10` |
|
|
61
|
+
| `max_attempts` | `3` | counting the first |
|
|
62
|
+
| `user_agent` | `taba-ruby/<version>` | |
|
|
63
|
+
| `ca_file`, `ca_path` | system's | an own CA bundle (a private CA, for example); TLS 1.2 or later, the certificate is always verified |
|
|
64
|
+
|
|
65
|
+
## What differs from the other connectors
|
|
66
|
+
|
|
67
|
+
- Names are `snake_case` and predicates end in `?` (`ok?`, `lease_lost?`); options are keyword arguments (`client.consume('q', max_messages: 5, wait_time: 20)`; an unknown one is a `client.invalid-request` error). Handlers are blocks. `ack` and `change_visibility` take the `Message` you received, or its id plus the receipt token.
|
|
68
|
+
- A `TopicUpdate` field left out is not sent; `nil` sends `null` (back to the server default) for `max_message_bytes`, `ttl` and `max_deliveries`: `client.update_topic('q', ttl: nil)`.
|
|
69
|
+
- Errors are a `Taba::Error` with `status`, `code`, `reason` (the message of the `ErrorResponse`; `message` and `to_s` also carry the code and the request id), `request_id`, `retry_after`, `details`, `body`. Test the `code`, never the text.
|
|
70
|
+
- Cancelling: pass `cancel: token` (a `Taba::CancelToken`) and call `token.cancel` from another thread; the call in flight ends at once with `Taba::Cancelled` (not a `Taba::Error`), so a long polling does not wait for its timeout. `subscribe` returns without an exception.
|
|
71
|
+
- `each_topic` walks all the pages (an `Enumerator` without a block); `list_all_topics` returns the array. `consume_json` hands over the body already parsed in `context.body`.
|
|
72
|
+
- `request_timeout` limits each wait for the response (Ruby's `Net::HTTP` has no total time limit), and `Net::HTTP`'s own automatic retry is switched off, so only the retries of the contract happen.
|
|
73
|
+
|
|
74
|
+
## Tests
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
cd connectors/ruby
|
|
78
|
+
ruby -Ilib -Itest -e 'Dir["test/*_test.rb"].each { |f| require File.expand_path(f) }'
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
The tests start a local HTTP server that records every request and answers from a script (the equivalent of the C# `ScriptedHandler`). The **contract tests** run the same scenario against a real Taba (one node in Docker), on Ruby 2.1: `contract/run.sh` (`../contract/run.sh` for all the connectors; [how it works](https://github.com/rafaelpontani/taba/blob/main/connectors/contract/README.md)).
|
data/lib/taba/calls.rb
ADDED
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
require 'json'
|
|
2
|
+
require 'taba/error'
|
|
3
|
+
require 'taba/options'
|
|
4
|
+
require 'taba/transport'
|
|
5
|
+
|
|
6
|
+
module Taba
|
|
7
|
+
# Builds the +ApiCall+ of each operation.
|
|
8
|
+
module Calls
|
|
9
|
+
# +consume+ waits +wait_time+ plus this slack (seconds), not the +request_timeout+.
|
|
10
|
+
LONG_POLL_SLACK = 10.0
|
|
11
|
+
|
|
12
|
+
HANDLER_MAX_MESSAGES = 1
|
|
13
|
+
HANDLER_VISIBILITY_TIMEOUT = 120
|
|
14
|
+
HANDLER_WAIT_TIME = 20
|
|
15
|
+
|
|
16
|
+
module_function
|
|
17
|
+
|
|
18
|
+
def encode(value)
|
|
19
|
+
::JSON.generate(value)
|
|
20
|
+
rescue ::JSON::GeneratorError, EncodingError => e
|
|
21
|
+
raise Error.client('client.invalid-request', "cannot serialize the request: #{e.message}")
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def segment(value)
|
|
25
|
+
Transport.segment(value)
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
def create_topic(request)
|
|
29
|
+
body = { 'name' => request.name.to_s }.merge(topic_fields(request))
|
|
30
|
+
call = ApiCall.new('POST', '/topics').with_json(encode(body))
|
|
31
|
+
call.recover << [409, 'topic.already-exists']
|
|
32
|
+
call
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# +PUT /topics/{topic}+: the fields of +POST /topics+ without +name+ (it goes in the path).
|
|
36
|
+
def put_topic(request, recreate)
|
|
37
|
+
unless recreate.nil? || %w[if-empty always].include?(recreate.to_s)
|
|
38
|
+
raise Error.client('client.invalid-request', "recreate must be 'if-empty' or 'always'")
|
|
39
|
+
end
|
|
40
|
+
call = ApiCall.new('PUT', "/topics/#{segment(request.name.to_s)}").with_json(encode(topic_fields(request)))
|
|
41
|
+
call.with_query('recreate', recreate.to_s) unless recreate.nil?
|
|
42
|
+
call
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def topic_fields(request)
|
|
46
|
+
body = { 'type' => request.type.to_s }
|
|
47
|
+
body['tags'] = request.tags if request.tags && !request.tags.empty?
|
|
48
|
+
body['prefixes'] = request.prefixes.to_a if request.prefixes && !request.prefixes.empty?
|
|
49
|
+
[[:volatile, 'volatile'], [:idle_timeout, 'idleTimeout'], [:fifo, 'fifo'],
|
|
50
|
+
[:max_message_bytes, 'maxMessageBytes'], [:ttl, 'ttl'], [:max_deliveries, 'maxDeliveries']].each do |attribute, name|
|
|
51
|
+
value = request.send(attribute)
|
|
52
|
+
body[name] = value unless value.nil?
|
|
53
|
+
end
|
|
54
|
+
body
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def get_topic(name)
|
|
58
|
+
ApiCall.new('GET', "/topics/#{segment(name)}")
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def list_topics(limit, cursor, name)
|
|
62
|
+
call = ApiCall.new('GET', '/topics')
|
|
63
|
+
call.with_query('limit', limit) unless limit.nil?
|
|
64
|
+
call.with_query('cursor', cursor) unless cursor.nil?
|
|
65
|
+
call.with_query('name', name) unless name.nil?
|
|
66
|
+
call
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def update_topic(name, update)
|
|
70
|
+
body = {}
|
|
71
|
+
update.given.each do |key, value|
|
|
72
|
+
wire = key.to_s.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }
|
|
73
|
+
if TopicUpdate::NULLABLE.include?(key)
|
|
74
|
+
body[wire] = value # nil goes as JSON null: back to the server default
|
|
75
|
+
elsif !value.nil?
|
|
76
|
+
body[wire] = key == :prefixes ? value.to_a : value
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
ApiCall.new('PATCH', "/topics/#{segment(name)}").with_json(encode(body))
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def delete_topic(name, if_empty = false)
|
|
83
|
+
call = ApiCall.new('DELETE', "/topics/#{segment(name)}")
|
|
84
|
+
call.with_query('ifEmpty', 'true') if if_empty
|
|
85
|
+
call.recover << [404, 'topic.not-found']
|
|
86
|
+
call
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
def bytes(body)
|
|
90
|
+
body.nil? ? '' : body.to_s
|
|
91
|
+
end
|
|
92
|
+
|
|
93
|
+
def publish(topic, body, options)
|
|
94
|
+
call = ApiCall.new('POST', "/topics/#{segment(topic)}/messages")
|
|
95
|
+
call.body = bytes(body)
|
|
96
|
+
call.content_type = options.content_type || 'application/json'
|
|
97
|
+
call.request_id = options.request_id
|
|
98
|
+
call.traceparent = options.traceparent
|
|
99
|
+
call.retry_gateway = options.fifo ? true : false
|
|
100
|
+
call.with_query('groupId', options.group_id) unless options.group_id.nil?
|
|
101
|
+
call.with_query('deduplicationId', options.deduplication_id) unless options.deduplication_id.nil?
|
|
102
|
+
call
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
def publish_batch(topic, messages)
|
|
106
|
+
items = messages.map do |m|
|
|
107
|
+
unless m.is_a?(BatchMessage)
|
|
108
|
+
raise Error.client('client.invalid-request', 'publish_batch expects Taba::BatchMessage objects')
|
|
109
|
+
end
|
|
110
|
+
item = { 'body' => [bytes(m.body)].pack('m0'), 'bodyEncoding' => 'base64' }
|
|
111
|
+
item['contentType'] = m.content_type unless m.content_type.nil?
|
|
112
|
+
item['groupId'] = m.group_id unless m.group_id.nil?
|
|
113
|
+
item['deduplicationId'] = m.deduplication_id unless m.deduplication_id.nil?
|
|
114
|
+
item
|
|
115
|
+
end
|
|
116
|
+
ApiCall.new('POST', "/topics/#{segment(topic)}/messages/batch").with_json(encode('messages' => items))
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
# The ack (+route+ +ack+) or the change of invisibility (+visibility+) of many messages in one request.
|
|
120
|
+
def receipts(topic, route, messages, seconds = nil)
|
|
121
|
+
items = messages.map do |m|
|
|
122
|
+
item = { 'id' => m.id, 'receiptToken' => m.receipt_token }
|
|
123
|
+
item['visibilityTimeout'] = seconds.to_i unless seconds.nil?
|
|
124
|
+
item
|
|
125
|
+
end
|
|
126
|
+
ApiCall.new('POST', "/topics/#{segment(topic)}/messages/#{route}").with_json(encode('messages' => items))
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
def consume(topic, o)
|
|
130
|
+
call = ApiCall.new('GET', "/topics/#{segment(topic)}/messages")
|
|
131
|
+
call.with_query('maxMessages', o.max_messages) unless o.max_messages.nil?
|
|
132
|
+
call.with_query('visibilityTimeout', o.visibility_timeout) unless o.visibility_timeout.nil?
|
|
133
|
+
call.with_query('waitTime', o.wait_time) unless o.wait_time.nil?
|
|
134
|
+
if o.auto_create
|
|
135
|
+
call.with_query('autoCreate', 'true')
|
|
136
|
+
call.with_query('volatile', o.volatile) unless o.volatile.nil?
|
|
137
|
+
call.with_query('idleTimeout', o.idle_timeout) unless o.idle_timeout.nil?
|
|
138
|
+
call.with_query('fifo', o.fifo) unless o.fifo.nil?
|
|
139
|
+
(o.tags || {}).sort_by { |k, _| k.to_s }.each { |k, v| call.with_query("tag.#{k}", v) }
|
|
140
|
+
end
|
|
141
|
+
call.request_id = o.request_id
|
|
142
|
+
call.traceparent = o.traceparent
|
|
143
|
+
call.timeout = (o.wait_time || 0).to_f + LONG_POLL_SLACK
|
|
144
|
+
call
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
def ack(topic, id, receipt_token)
|
|
148
|
+
ApiCall.new('DELETE', "/topics/#{segment(topic)}/messages/#{segment(id)}").with_query('receiptToken', receipt_token)
|
|
149
|
+
end
|
|
150
|
+
|
|
151
|
+
def change_visibility(topic, id, receipt_token, seconds)
|
|
152
|
+
call = ApiCall.new('PUT', "/topics/#{segment(topic)}/messages/#{segment(id)}/visibility")
|
|
153
|
+
call.with_query('receiptToken', receipt_token)
|
|
154
|
+
call.with_json(encode('visibilityTimeout' => seconds.to_i))
|
|
155
|
+
end
|
|
156
|
+
|
|
157
|
+
# One message at a time, 20 s of long polling and 120 s of invisibility, which the renewal keeps.
|
|
158
|
+
def handler_defaults(o)
|
|
159
|
+
o.max_messages = HANDLER_MAX_MESSAGES if o.max_messages.nil?
|
|
160
|
+
o.visibility_timeout = HANDLER_VISIBILITY_TIMEOUT if o.visibility_timeout.nil?
|
|
161
|
+
o.wait_time = HANDLER_WAIT_TIME if o.wait_time.nil?
|
|
162
|
+
o
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
def subscribe_defaults(o)
|
|
166
|
+
o.wait_time = HANDLER_WAIT_TIME if o.wait_time.nil?
|
|
167
|
+
o.max_messages = HANDLER_MAX_MESSAGES if o.max_messages.nil?
|
|
168
|
+
o
|
|
169
|
+
end
|
|
170
|
+
|
|
171
|
+
# A third of the invisibility (and never all of it), so a failed renewal can be tried again in time.
|
|
172
|
+
def renewal_interval(lease_seconds)
|
|
173
|
+
[1, lease_seconds].max / 3.0
|
|
174
|
+
end
|
|
175
|
+
end
|
|
176
|
+
end
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
require 'thread'
|
|
2
|
+
|
|
3
|
+
module Taba
|
|
4
|
+
# Cancels a call or a +subscribe+ loop from another thread.
|
|
5
|
+
#
|
|
6
|
+
# token = Taba::CancelToken.new
|
|
7
|
+
# Thread.new { sleep 60; token.cancel }
|
|
8
|
+
# client.subscribe('orders', cancel: token) { |message| ... } # returns when cancelled
|
|
9
|
+
#
|
|
10
|
+
# Cancelling closes the connection of the call in flight, so a long polling does not wait for its timeout.
|
|
11
|
+
class CancelToken
|
|
12
|
+
def initialize
|
|
13
|
+
@mutex = Mutex.new
|
|
14
|
+
@condition = ConditionVariable.new
|
|
15
|
+
@cancelled = false
|
|
16
|
+
@callbacks = []
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def cancel
|
|
20
|
+
callbacks = @mutex.synchronize do
|
|
21
|
+
@cancelled = true
|
|
22
|
+
@condition.broadcast
|
|
23
|
+
@callbacks.dup
|
|
24
|
+
end
|
|
25
|
+
callbacks.each do |callback|
|
|
26
|
+
begin
|
|
27
|
+
callback.call
|
|
28
|
+
rescue StandardError, IOError
|
|
29
|
+
nil # closing a connection that is already closed
|
|
30
|
+
end
|
|
31
|
+
end
|
|
32
|
+
self
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def cancelled?
|
|
36
|
+
@mutex.synchronize { @cancelled }
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
# Waits up to +seconds+; true when cancelled.
|
|
40
|
+
def wait(seconds)
|
|
41
|
+
deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + seconds
|
|
42
|
+
@mutex.synchronize do
|
|
43
|
+
until @cancelled
|
|
44
|
+
left = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
45
|
+
break if left <= 0
|
|
46
|
+
@condition.wait(@mutex, left)
|
|
47
|
+
end
|
|
48
|
+
@cancelled
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def raise_if_cancelled
|
|
53
|
+
raise Cancelled if cancelled?
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# :nodoc:
|
|
57
|
+
def attach(callback)
|
|
58
|
+
already = @mutex.synchronize do
|
|
59
|
+
@callbacks << callback
|
|
60
|
+
@cancelled
|
|
61
|
+
end
|
|
62
|
+
callback.call if already
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# :nodoc:
|
|
66
|
+
def detach(callback)
|
|
67
|
+
@mutex.synchronize { @callbacks.delete(callback) }
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|