taba-client 0.3.20.rc1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 0db7633a4ecc2e4f1a60e87cf00905f08e7141f4ddccf805975cdb9388608bd1
4
+ data.tar.gz: 165dc7a0564ff85e73c69610093ff5f8ef5f5eb27ba564087ca36762d76bc732
5
+ SHA512:
6
+ metadata.gz: e16982c9ff2a74ac8a9e821b2a269ab423fa1f7566fc7a75a456f3da93ae9deb8b8a4eafb45c63aba3fdeb2b06237650e7c43b31a6cdd07eddcd3997938bc570
7
+ data.tar.gz: 6b84aa22836229f94e8747afbf7fbf90f28c6c431ec5518d17d3a84ae35d2d21a6b80450752f7c782680b9028c130ded4a93f99ca269bf0f7f35f2175888dd98
data/README.md ADDED
@@ -0,0 +1,81 @@
1
+ # taba-client (Ruby)
2
+
3
+ The Ruby client of [Taba](https://github.com/rafaelpontani/taba): the nine operations of the client API, `subscribe`, consuming with a handler (with automatic renewal of the invisibility) and the call retries of the common contract. **No dependencies** (only the standard library: `net/http`, `json`).
4
+
5
+ > Other languages: [Português (Brasil)](https://github.com/rafaelpontani/taba/blob/main/doc/pt-BR/21-conectores.md) · [all languages](https://github.com/rafaelpontani/taba/blob/main/doc/README.md)
6
+
7
+ - **Ruby 2.1 or later** (it runs the same code on 2.1 and on 3.x).
8
+ - `Taba::Client` is safe for concurrent use: each thread has its own connection.
9
+
10
+ ```ruby
11
+ require 'taba'
12
+
13
+ client = Taba::Client.new('http://localhost:8080', token: ENV['TABA_TOKEN'])
14
+
15
+ client.create_topic(name: 'orders', type: 'queue')
16
+ client.publish_json('orders', order: 9001)
17
+
18
+ # Consume with a block: the message is deleted if the block does not raise.
19
+ client.consume_with('orders', wait_time: 20) do |context|
20
+ puts context.message.body_text
21
+ end
22
+
23
+ # Or the whole loop, until the token is cancelled.
24
+ # token = Taba::CancelToken.new
25
+ # client.subscribe('orders', cancel: token) { |message| puts message.json }
26
+ ```
27
+
28
+ ## Installing
29
+
30
+ ```
31
+ gem install taba-client
32
+ ```
33
+
34
+ Or, with Bundler, `gem 'taba-client'`. The gem has the version of the Taba release it came with and is published from the release workflow of this repository with RubyGems Trusted Publishing. A release candidate is published in the RubyGems pre-release form (`0.3.19.rc1`), which `gem` only installs when asked (`gem install --pre taba-client`). Each [Release](https://github.com/rafaelpontani/taba/releases) also has the same gem, `taba-client-<release>.gem`, with its checksum in `SHA256SUMS` and its provenance (`gh attestation verify <file> -R rafaelpontani/taba`).
35
+
36
+ With Bundler, straight from the source (a branch, a tag or a commit):
37
+
38
+ ```ruby
39
+ gem 'taba-client', git: 'https://github.com/rafaelpontani/taba.git', branch: 'release',
40
+ glob: 'connectors/ruby/*.gemspec'
41
+ ```
42
+
43
+ ## Declaring a topic
44
+
45
+ `create_or_update_topic(request, recreate: nil)` (`PUT /topics/{topic}`; the fields can also go as options) creates the topic or brings it to the declared state, and returns a `Taba::TopicPut` with `topic` and `outcome` (`created`, `updated` or `recreated`). What is left out goes back to the server default (a full replacement, unlike the update); another type, `fifo` or `volatile` is the error `409 topic.immutable-mismatch` unless `recreate` is `if-empty` (only when the queue and its dead-letter are empty, `409 topic.not-empty` otherwise) or `always` (drops the messages; new `id`). After a `503` the call is repeated like the others. `delete_topic(name, if_empty: true)` only removes a queue with no message. `create_topic` does not change (`409` when it exists). Details: [Create or update a topic](https://github.com/rafaelpontani/taba/blob/main/doc/en-US/16-client-api.md#create-or-update-a-topic-put-topicstopic).
46
+
47
+ ## Batches
48
+
49
+ Many messages in one request, next to the single calls (optional): `publish_batch(topic, [Taba::BatchMessage.new(body, content_type: ...)])`, `ack_batch(topic, messages)` and `change_visibility_batch(topic, messages, seconds)` return one `BatchResult` per item (`ok?`, `status`, `id`, `error_code`). Each item is answered as its single call, in order, and an item that failed comes back with its `status` and error code instead of raising. The bodies go in base64, so any type fits. Details: [Messages in batch](https://github.com/rafaelpontani/taba/blob/main/doc/en-US/16-client-api.md#messages-in-batch-post-messagesbatch-ack-and-visibility).
50
+
51
+ ## Configuration
52
+
53
+ `Taba::Client.new(config)` with a `Taba::Config`, or `Taba::Client.new(base_url, **options)` with the same fields. The client does not read environment variables: `TABA_*` belongs to the server.
54
+
55
+ | Field | Default | |
56
+ |---|---|---|
57
+ | `base_url` | required | `http://host:port` or `https://...` |
58
+ | `token` | none | `Authorization: Bearer` |
59
+ | `connect_timeout` | `5.0` | seconds |
60
+ | `request_timeout` | `30.0` | seconds; `consume` uses `wait_time + 10` |
61
+ | `max_attempts` | `3` | counting the first |
62
+ | `user_agent` | `taba-ruby/<version>` | |
63
+ | `ca_file`, `ca_path` | system's | an own CA bundle (a private CA, for example); TLS 1.2 or later, the certificate is always verified |
64
+
65
+ ## What differs from the other connectors
66
+
67
+ - Names are `snake_case` and predicates end in `?` (`ok?`, `lease_lost?`); options are keyword arguments (`client.consume('q', max_messages: 5, wait_time: 20)`; an unknown one is a `client.invalid-request` error). Handlers are blocks. `ack` and `change_visibility` take the `Message` you received, or its id plus the receipt token.
68
+ - A `TopicUpdate` field left out is not sent; `nil` sends `null` (back to the server default) for `max_message_bytes`, `ttl` and `max_deliveries`: `client.update_topic('q', ttl: nil)`.
69
+ - Errors are a `Taba::Error` with `status`, `code`, `reason` (the message of the `ErrorResponse`; `message` and `to_s` also carry the code and the request id), `request_id`, `retry_after`, `details`, `body`. Test the `code`, never the text.
70
+ - Cancelling: pass `cancel: token` (a `Taba::CancelToken`) and call `token.cancel` from another thread; the call in flight ends at once with `Taba::Cancelled` (not a `Taba::Error`), so a long polling does not wait for its timeout. `subscribe` returns without an exception.
71
+ - `each_topic` walks all the pages (an `Enumerator` without a block); `list_all_topics` returns the array. `consume_json` hands over the body already parsed in `context.body`.
72
+ - `request_timeout` limits each wait for the response (Ruby's `Net::HTTP` has no total time limit), and `Net::HTTP`'s own automatic retry is switched off, so only the retries of the contract happen.
73
+
74
+ ## Tests
75
+
76
+ ```
77
+ cd connectors/ruby
78
+ ruby -Ilib -Itest -e 'Dir["test/*_test.rb"].each { |f| require File.expand_path(f) }'
79
+ ```
80
+
81
+ The tests start a local HTTP server that records every request and answers from a script (the equivalent of the C# `ScriptedHandler`). The **contract tests** run the same scenario against a real Taba (one node in Docker), on Ruby 2.1: `contract/run.sh` (`../contract/run.sh` for all the connectors; [how it works](https://github.com/rafaelpontani/taba/blob/main/connectors/contract/README.md)).
data/lib/taba/calls.rb ADDED
@@ -0,0 +1,176 @@
1
+ require 'json'
2
+ require 'taba/error'
3
+ require 'taba/options'
4
+ require 'taba/transport'
5
+
6
+ module Taba
7
+ # Builds the +ApiCall+ of each operation.
8
+ module Calls
9
+ # +consume+ waits +wait_time+ plus this slack (seconds), not the +request_timeout+.
10
+ LONG_POLL_SLACK = 10.0
11
+
12
+ HANDLER_MAX_MESSAGES = 1
13
+ HANDLER_VISIBILITY_TIMEOUT = 120
14
+ HANDLER_WAIT_TIME = 20
15
+
16
+ module_function
17
+
18
+ def encode(value)
19
+ ::JSON.generate(value)
20
+ rescue ::JSON::GeneratorError, EncodingError => e
21
+ raise Error.client('client.invalid-request', "cannot serialize the request: #{e.message}")
22
+ end
23
+
24
+ def segment(value)
25
+ Transport.segment(value)
26
+ end
27
+
28
+ def create_topic(request)
29
+ body = { 'name' => request.name.to_s }.merge(topic_fields(request))
30
+ call = ApiCall.new('POST', '/topics').with_json(encode(body))
31
+ call.recover << [409, 'topic.already-exists']
32
+ call
33
+ end
34
+
35
+ # +PUT /topics/{topic}+: the fields of +POST /topics+ without +name+ (it goes in the path).
36
+ def put_topic(request, recreate)
37
+ unless recreate.nil? || %w[if-empty always].include?(recreate.to_s)
38
+ raise Error.client('client.invalid-request', "recreate must be 'if-empty' or 'always'")
39
+ end
40
+ call = ApiCall.new('PUT', "/topics/#{segment(request.name.to_s)}").with_json(encode(topic_fields(request)))
41
+ call.with_query('recreate', recreate.to_s) unless recreate.nil?
42
+ call
43
+ end
44
+
45
+ def topic_fields(request)
46
+ body = { 'type' => request.type.to_s }
47
+ body['tags'] = request.tags if request.tags && !request.tags.empty?
48
+ body['prefixes'] = request.prefixes.to_a if request.prefixes && !request.prefixes.empty?
49
+ [[:volatile, 'volatile'], [:idle_timeout, 'idleTimeout'], [:fifo, 'fifo'],
50
+ [:max_message_bytes, 'maxMessageBytes'], [:ttl, 'ttl'], [:max_deliveries, 'maxDeliveries']].each do |attribute, name|
51
+ value = request.send(attribute)
52
+ body[name] = value unless value.nil?
53
+ end
54
+ body
55
+ end
56
+
57
+ def get_topic(name)
58
+ ApiCall.new('GET', "/topics/#{segment(name)}")
59
+ end
60
+
61
+ def list_topics(limit, cursor, name)
62
+ call = ApiCall.new('GET', '/topics')
63
+ call.with_query('limit', limit) unless limit.nil?
64
+ call.with_query('cursor', cursor) unless cursor.nil?
65
+ call.with_query('name', name) unless name.nil?
66
+ call
67
+ end
68
+
69
+ def update_topic(name, update)
70
+ body = {}
71
+ update.given.each do |key, value|
72
+ wire = key.to_s.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }
73
+ if TopicUpdate::NULLABLE.include?(key)
74
+ body[wire] = value # nil goes as JSON null: back to the server default
75
+ elsif !value.nil?
76
+ body[wire] = key == :prefixes ? value.to_a : value
77
+ end
78
+ end
79
+ ApiCall.new('PATCH', "/topics/#{segment(name)}").with_json(encode(body))
80
+ end
81
+
82
+ def delete_topic(name, if_empty = false)
83
+ call = ApiCall.new('DELETE', "/topics/#{segment(name)}")
84
+ call.with_query('ifEmpty', 'true') if if_empty
85
+ call.recover << [404, 'topic.not-found']
86
+ call
87
+ end
88
+
89
+ def bytes(body)
90
+ body.nil? ? '' : body.to_s
91
+ end
92
+
93
+ def publish(topic, body, options)
94
+ call = ApiCall.new('POST', "/topics/#{segment(topic)}/messages")
95
+ call.body = bytes(body)
96
+ call.content_type = options.content_type || 'application/json'
97
+ call.request_id = options.request_id
98
+ call.traceparent = options.traceparent
99
+ call.retry_gateway = options.fifo ? true : false
100
+ call.with_query('groupId', options.group_id) unless options.group_id.nil?
101
+ call.with_query('deduplicationId', options.deduplication_id) unless options.deduplication_id.nil?
102
+ call
103
+ end
104
+
105
+ def publish_batch(topic, messages)
106
+ items = messages.map do |m|
107
+ unless m.is_a?(BatchMessage)
108
+ raise Error.client('client.invalid-request', 'publish_batch expects Taba::BatchMessage objects')
109
+ end
110
+ item = { 'body' => [bytes(m.body)].pack('m0'), 'bodyEncoding' => 'base64' }
111
+ item['contentType'] = m.content_type unless m.content_type.nil?
112
+ item['groupId'] = m.group_id unless m.group_id.nil?
113
+ item['deduplicationId'] = m.deduplication_id unless m.deduplication_id.nil?
114
+ item
115
+ end
116
+ ApiCall.new('POST', "/topics/#{segment(topic)}/messages/batch").with_json(encode('messages' => items))
117
+ end
118
+
119
+ # The ack (+route+ +ack+) or the change of invisibility (+visibility+) of many messages in one request.
120
+ def receipts(topic, route, messages, seconds = nil)
121
+ items = messages.map do |m|
122
+ item = { 'id' => m.id, 'receiptToken' => m.receipt_token }
123
+ item['visibilityTimeout'] = seconds.to_i unless seconds.nil?
124
+ item
125
+ end
126
+ ApiCall.new('POST', "/topics/#{segment(topic)}/messages/#{route}").with_json(encode('messages' => items))
127
+ end
128
+
129
+ def consume(topic, o)
130
+ call = ApiCall.new('GET', "/topics/#{segment(topic)}/messages")
131
+ call.with_query('maxMessages', o.max_messages) unless o.max_messages.nil?
132
+ call.with_query('visibilityTimeout', o.visibility_timeout) unless o.visibility_timeout.nil?
133
+ call.with_query('waitTime', o.wait_time) unless o.wait_time.nil?
134
+ if o.auto_create
135
+ call.with_query('autoCreate', 'true')
136
+ call.with_query('volatile', o.volatile) unless o.volatile.nil?
137
+ call.with_query('idleTimeout', o.idle_timeout) unless o.idle_timeout.nil?
138
+ call.with_query('fifo', o.fifo) unless o.fifo.nil?
139
+ (o.tags || {}).sort_by { |k, _| k.to_s }.each { |k, v| call.with_query("tag.#{k}", v) }
140
+ end
141
+ call.request_id = o.request_id
142
+ call.traceparent = o.traceparent
143
+ call.timeout = (o.wait_time || 0).to_f + LONG_POLL_SLACK
144
+ call
145
+ end
146
+
147
+ def ack(topic, id, receipt_token)
148
+ ApiCall.new('DELETE', "/topics/#{segment(topic)}/messages/#{segment(id)}").with_query('receiptToken', receipt_token)
149
+ end
150
+
151
+ def change_visibility(topic, id, receipt_token, seconds)
152
+ call = ApiCall.new('PUT', "/topics/#{segment(topic)}/messages/#{segment(id)}/visibility")
153
+ call.with_query('receiptToken', receipt_token)
154
+ call.with_json(encode('visibilityTimeout' => seconds.to_i))
155
+ end
156
+
157
+ # One message at a time, 20 s of long polling and 120 s of invisibility, which the renewal keeps.
158
+ def handler_defaults(o)
159
+ o.max_messages = HANDLER_MAX_MESSAGES if o.max_messages.nil?
160
+ o.visibility_timeout = HANDLER_VISIBILITY_TIMEOUT if o.visibility_timeout.nil?
161
+ o.wait_time = HANDLER_WAIT_TIME if o.wait_time.nil?
162
+ o
163
+ end
164
+
165
+ def subscribe_defaults(o)
166
+ o.wait_time = HANDLER_WAIT_TIME if o.wait_time.nil?
167
+ o.max_messages = HANDLER_MAX_MESSAGES if o.max_messages.nil?
168
+ o
169
+ end
170
+
171
+ # A third of the invisibility (and never all of it), so a failed renewal can be tried again in time.
172
+ def renewal_interval(lease_seconds)
173
+ [1, lease_seconds].max / 3.0
174
+ end
175
+ end
176
+ end
@@ -0,0 +1,70 @@
1
+ require 'thread'
2
+
3
+ module Taba
4
+ # Cancels a call or a +subscribe+ loop from another thread.
5
+ #
6
+ # token = Taba::CancelToken.new
7
+ # Thread.new { sleep 60; token.cancel }
8
+ # client.subscribe('orders', cancel: token) { |message| ... } # returns when cancelled
9
+ #
10
+ # Cancelling closes the connection of the call in flight, so a long polling does not wait for its timeout.
11
+ class CancelToken
12
+ def initialize
13
+ @mutex = Mutex.new
14
+ @condition = ConditionVariable.new
15
+ @cancelled = false
16
+ @callbacks = []
17
+ end
18
+
19
+ def cancel
20
+ callbacks = @mutex.synchronize do
21
+ @cancelled = true
22
+ @condition.broadcast
23
+ @callbacks.dup
24
+ end
25
+ callbacks.each do |callback|
26
+ begin
27
+ callback.call
28
+ rescue StandardError, IOError
29
+ nil # closing a connection that is already closed
30
+ end
31
+ end
32
+ self
33
+ end
34
+
35
+ def cancelled?
36
+ @mutex.synchronize { @cancelled }
37
+ end
38
+
39
+ # Waits up to +seconds+; true when cancelled.
40
+ def wait(seconds)
41
+ deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + seconds
42
+ @mutex.synchronize do
43
+ until @cancelled
44
+ left = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
45
+ break if left <= 0
46
+ @condition.wait(@mutex, left)
47
+ end
48
+ @cancelled
49
+ end
50
+ end
51
+
52
+ def raise_if_cancelled
53
+ raise Cancelled if cancelled?
54
+ end
55
+
56
+ # :nodoc:
57
+ def attach(callback)
58
+ already = @mutex.synchronize do
59
+ @callbacks << callback
60
+ @cancelled
61
+ end
62
+ callback.call if already
63
+ end
64
+
65
+ # :nodoc:
66
+ def detach(callback)
67
+ @mutex.synchronize { @callbacks.delete(callback) }
68
+ end
69
+ end
70
+ end