pubsubc 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/README.md +236 -0
- data/exe/pubsubc +176 -0
- data/lib/pubsubc.rb +3 -0
- data/lib/pubsubc/version.rb +5 -0
- metadata +107 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: 688500fbb35695a35b4109e914ea2ba45277d64743d2372f865f65d5dd46ad6d
|
4
|
+
data.tar.gz: 8839a7ad9d76cfc88578ed8357c716fca88afe065e8bcfaa4e28806ccddd548f
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 4a8375467b9013c139fd84a371cb98a5b27782d9a802048dfada45d9cc8fab9384e826e4b1ef1b9779c4278772747d6540d838da21021f3610799ea723a0d3c6
|
7
|
+
data.tar.gz: b79699630a59e7e70639acc40b997442b7e273d9efabf873878b44c0110b4be010e2152e9d5b4682e3afe38099e53b0e9599e846a22dfb37e146ac309f990ab0
|
data/README.md
ADDED
@@ -0,0 +1,236 @@
|
|
1
|
+
# pubsubc
|
2
|
+
|
3
|
+
[![CI status](https://github.com/ZimbiX/pubsubc/actions/workflows/main.yml/badge.svg?branch=master)](https://github.com/ZimbiX/pubsubc/actions/workflows/main.yml) [![Gem Version](https://badge.fury.io/rb/pubsubc.svg)](https://rubygems.org/gems/pubsubc)
|
4
|
+
|
5
|
+
A simple command-line client for [Google Cloud Pub/Sub](https://cloud.google.com/pubsub).
|
6
|
+
|
7
|
+
Unlike the `gcloud pubsub` utility, `pubsubc`:
|
8
|
+
|
9
|
+
- Supports the [Pub/Sub emulator](https://cloud.google.com/pubsub/docs/emulator)
|
10
|
+
- Provides a continuous subscribe command - to see all messages as they come in
|
11
|
+
|
12
|
+
Since this was originally created as a learning exercise for integrating to Pub/Sub, the source code should be simple enough for you to copy and adapt into your own application.
|
13
|
+
|
14
|
+
## Contents
|
15
|
+
|
16
|
+
<!-- MarkdownTOC autolink=true -->
|
17
|
+
|
18
|
+
- [Intro](#intro)
|
19
|
+
- [Installation](#installation)
|
20
|
+
- [Direct](#direct)
|
21
|
+
- [Bundler](#bundler)
|
22
|
+
- [Usage](#usage)
|
23
|
+
- [Subscribe](#subscribe)
|
24
|
+
- [Publish message](#publish-message)
|
25
|
+
- [Publish generated messages](#publish-generated-messages)
|
26
|
+
- [Example](#example)
|
27
|
+
- [Communicating some messages](#communicating-some-messages)
|
28
|
+
- [Publishing from a message generator with several subscribers](#publishing-from-a-message-generator-with-several-subscribers)
|
29
|
+
- [Help info](#help-info)
|
30
|
+
- [Contributing](#contributing)
|
31
|
+
- [Development](#development)
|
32
|
+
- [Pub/Sub emulator](#pubsub-emulator)
|
33
|
+
- [Pre-push hook](#pre-push-hook)
|
34
|
+
- [Release](#release)
|
35
|
+
|
36
|
+
<!-- /MarkdownTOC -->
|
37
|
+
|
38
|
+
## Intro
|
39
|
+
|
40
|
+
Features:
|
41
|
+
|
42
|
+
- Subscribe to a topic, continuously logging all messages as they arrive
|
43
|
+
- Publish a string message to a topic (creating a subscription automatically)
|
44
|
+
- Publish automatically-generated messages continuously at a given interval
|
45
|
+
- Supports connecting to either a local emulator or the remote real Pub/Sub
|
46
|
+
|
47
|
+
## Installation
|
48
|
+
|
49
|
+
The executable is distributed through RubyGems.
|
50
|
+
|
51
|
+
### Direct
|
52
|
+
|
53
|
+
You can install it to your system directly using:
|
54
|
+
|
55
|
+
```bash
|
56
|
+
gem install pubsubc
|
57
|
+
```
|
58
|
+
|
59
|
+
And then if you're using rbenv:
|
60
|
+
|
61
|
+
```bash
|
62
|
+
rbenv rehash
|
63
|
+
```
|
64
|
+
|
65
|
+
You can then run it with:
|
66
|
+
|
67
|
+
```bash
|
68
|
+
pubsubc
|
69
|
+
```
|
70
|
+
|
71
|
+
### Bundler
|
72
|
+
|
73
|
+
Or, just add it to the `Gemfile` of your project:
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
gem 'pubsubc'
|
77
|
+
```
|
78
|
+
|
79
|
+
And then run it with:
|
80
|
+
|
81
|
+
```bash
|
82
|
+
bundle exec pubsubc
|
83
|
+
```
|
84
|
+
|
85
|
+
## Usage
|
86
|
+
|
87
|
+
### Subscribe
|
88
|
+
|
89
|
+
To subscribe and continuously log all messages from the topic:
|
90
|
+
|
91
|
+
```bash
|
92
|
+
pubsubc sub
|
93
|
+
```
|
94
|
+
|
95
|
+
### Publish message
|
96
|
+
|
97
|
+
To publish a single message to the topic:
|
98
|
+
|
99
|
+
```bash
|
100
|
+
pubsubc pub msg "G'day"
|
101
|
+
```
|
102
|
+
|
103
|
+
### Publish generated messages
|
104
|
+
|
105
|
+
To publish a generated message to the topic every second:
|
106
|
+
|
107
|
+
```bash
|
108
|
+
pubsubc pub gen --interval 1000
|
109
|
+
```
|
110
|
+
|
111
|
+
## Example
|
112
|
+
|
113
|
+
### Communicating some messages
|
114
|
+
|
115
|
+
```
|
116
|
+
➜ pubsubc pub msg "Sup wigga"
|
117
|
+
Initialising Pub/Sub client for emulator (localhost:8085)... done
|
118
|
+
Checking existence of topic 'my-topic'... creating... done
|
119
|
+
Checking existence of subscription 'my-subscription'... creating... done
|
120
|
+
Published message: Sup wigga
|
121
|
+
|
122
|
+
➜ pubsubc pub msg "Yo J-dog"
|
123
|
+
Initialising Pub/Sub client for emulator (localhost:8085)... done
|
124
|
+
Checking existence of topic 'my-topic'... exists
|
125
|
+
Checking existence of subscription 'my-subscription'... exists
|
126
|
+
Published message: Yo J-dog
|
127
|
+
|
128
|
+
➜ pubsubc sub
|
129
|
+
Initialising Pub/Sub client for emulator (localhost:8085)... done
|
130
|
+
Checking existence of topic 'my-topic'... exists
|
131
|
+
Checking existence of subscription 'my-subscription'... exists
|
132
|
+
Listening for messages
|
133
|
+
Data: Sup wigga, published at 2021-07-30 23:19:06 +1000
|
134
|
+
Data: Yo J-dog, published at 2021-07-30 23:19:41 +1000
|
135
|
+
^C
|
136
|
+
```
|
137
|
+
|
138
|
+
### Publishing from a message generator with several subscribers
|
139
|
+
|
140
|
+
![](doc/example-with-generator-and-several-subscribers.png)
|
141
|
+
|
142
|
+
The generator was run with:
|
143
|
+
|
144
|
+
```bash
|
145
|
+
pubsubc pub gen --interval 1
|
146
|
+
```
|
147
|
+
|
148
|
+
And each subscriber:
|
149
|
+
|
150
|
+
```bash
|
151
|
+
pubsubc sub
|
152
|
+
```
|
153
|
+
|
154
|
+
## Help info
|
155
|
+
|
156
|
+
Output of `pubsubc --help`:
|
157
|
+
|
158
|
+
<!-- Help -->
|
159
|
+
|
160
|
+
```
|
161
|
+
Usage:
|
162
|
+
pubsubc [OPTIONS] SUBCOMMAND [ARG] ...
|
163
|
+
|
164
|
+
A simple command-line client for Google Cloud Pub/Sub.
|
165
|
+
|
166
|
+
Created as a learning exercise, but also useful since the 'gcloud pubsub' commands don't support the emulator, and it doesn't provide a continuous subscribe command.
|
167
|
+
|
168
|
+
To clean up:
|
169
|
+
|
170
|
+
gcloud pubsub topics delete my-topic
|
171
|
+
gcloud pubsub subscriptions delete my-subscription
|
172
|
+
|
173
|
+
Parameters:
|
174
|
+
SUBCOMMAND subcommand
|
175
|
+
[ARG] ... subcommand arguments
|
176
|
+
|
177
|
+
Subcommands:
|
178
|
+
pub Publishing messages to the topic
|
179
|
+
sub Subscribe to messages on the topic
|
180
|
+
shell Get a Ruby shell to interact with Pub/Sub
|
181
|
+
|
182
|
+
Options:
|
183
|
+
--topic TOPIC_NAME The name of the Pub/Sub topic (default: "my-topic")
|
184
|
+
--subscription SUBSCRIPTION_NAME The name of the Pub/Sub subscription within the topic (default: "my-subscription")
|
185
|
+
--remote Connect to the real Pub/Sub, rather than a local Pub/Sub emulator
|
186
|
+
--emulator-host EMULATOR_HOST The host and port of the Pub/Sub emulator (default: $PUBSUB_EMULATOR_HOST, or "localhost:8085")
|
187
|
+
-h, --help print help
|
188
|
+
```
|
189
|
+
|
190
|
+
<!-- /Help -->
|
191
|
+
|
192
|
+
Suffix `--help` to any subcommand to see the usage info for it.
|
193
|
+
|
194
|
+
## Contributing
|
195
|
+
|
196
|
+
Pull requests welcome! =)
|
197
|
+
|
198
|
+
However, I don't intend to duplicate functionality that exists in the `gcloud pubsub` utility, unless it's an improvement over what's in that.
|
199
|
+
|
200
|
+
## Development
|
201
|
+
|
202
|
+
### Pub/Sub emulator
|
203
|
+
|
204
|
+
To run the Pub/Sub emulator (with Docker):
|
205
|
+
|
206
|
+
```bash
|
207
|
+
docker-compose up -d pubsub-emulator
|
208
|
+
```
|
209
|
+
|
210
|
+
### Pre-push hook
|
211
|
+
|
212
|
+
This hook runs style checks and tests.
|
213
|
+
|
214
|
+
To set up the pre-push hook:
|
215
|
+
|
216
|
+
```bash
|
217
|
+
echo -e "#\!/bin/bash\n\$(dirname \$0)/../../auto/pre-push-hook" > .git/hooks/pre-push
|
218
|
+
chmod +x .git/hooks/pre-push
|
219
|
+
```
|
220
|
+
|
221
|
+
### Release
|
222
|
+
|
223
|
+
To release a new version:
|
224
|
+
|
225
|
+
```bash
|
226
|
+
auto/release/update-version && auto/release/tag && auto/release/publish
|
227
|
+
```
|
228
|
+
|
229
|
+
This takes care of the whole process:
|
230
|
+
|
231
|
+
- Incrementing the version number (the patch number by default)
|
232
|
+
- Tagging & pushing commits
|
233
|
+
- Publishing the gem to RubyGems
|
234
|
+
- Creating a draft GitHub release
|
235
|
+
|
236
|
+
To increment the minor or major versions instead of the patch number, run `auto/release/update-version` with `--minor` or `--major`.
|
data/exe/pubsubc
ADDED
@@ -0,0 +1,176 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
# frozen_string_literal: true
|
4
|
+
|
5
|
+
ENV['GOOGLE_AUTH_SUPPRESS_CREDENTIALS_WARNINGS'] = 'true'
|
6
|
+
|
7
|
+
require 'google/cloud/pubsub'
|
8
|
+
require 'clamp'
|
9
|
+
require 'pry'
|
10
|
+
|
11
|
+
Clamp.allow_options_after_parameters = true
|
12
|
+
|
13
|
+
Clamp do # rubocop:disable Metrics/BlockLength
|
14
|
+
self.description = <<~TEXT
|
15
|
+
A simple command-line client for Google Cloud Pub/Sub.
|
16
|
+
|
17
|
+
Created as a learning exercise, but also useful since the 'gcloud pubsub' commands don't support the emulator, and it doesn't provide a continuous subscribe command.
|
18
|
+
|
19
|
+
To clean up:
|
20
|
+
|
21
|
+
gcloud pubsub topics delete my-topic
|
22
|
+
gcloud pubsub subscriptions delete my-subscription
|
23
|
+
TEXT
|
24
|
+
|
25
|
+
option '--topic',
|
26
|
+
'TOPIC_NAME',
|
27
|
+
'The name of the Pub/Sub topic',
|
28
|
+
attribute_name: :topic_name,
|
29
|
+
default: 'my-topic'
|
30
|
+
|
31
|
+
option '--subscription',
|
32
|
+
'SUBSCRIPTION_NAME',
|
33
|
+
'The name of the Pub/Sub subscription within the topic',
|
34
|
+
attribute_name: :subscription_name,
|
35
|
+
default: 'my-subscription'
|
36
|
+
|
37
|
+
option '--remote', :flag, 'Connect to the real Pub/Sub, rather than a local Pub/Sub emulator'
|
38
|
+
|
39
|
+
option '--emulator-host',
|
40
|
+
'EMULATOR_HOST',
|
41
|
+
'The host and port of the Pub/Sub emulator',
|
42
|
+
environment_variable: 'PUBSUB_EMULATOR_HOST',
|
43
|
+
default: 'localhost:8085' do |host|
|
44
|
+
ENV.delete('PUBSUB_EMULATOR_HOST')
|
45
|
+
ENV.delete('PUBSUB_PROJECT_ID')
|
46
|
+
host
|
47
|
+
end
|
48
|
+
|
49
|
+
subcommand 'pub', 'Publishing messages to the topic' do # rubocop:disable Metrics/BlockLength
|
50
|
+
subcommand 'msg', 'Publish one message' do
|
51
|
+
parameter 'MESSAGE', 'The message string to publish to the topic'
|
52
|
+
|
53
|
+
def execute
|
54
|
+
publish(message)
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
subcommand 'gen', 'Publish generated messages' do
|
59
|
+
option %w[-i --interval],
|
60
|
+
'INTERVAL_MS',
|
61
|
+
'How long we should sleep before generating each message',
|
62
|
+
attribute_name: :interval_ms,
|
63
|
+
required: true,
|
64
|
+
&method(:Integer)
|
65
|
+
|
66
|
+
def execute
|
67
|
+
(1..).each do |i|
|
68
|
+
message = "This is message #{i}"
|
69
|
+
publish(message)
|
70
|
+
sleep(interval_ms / 1000.0)
|
71
|
+
end
|
72
|
+
end
|
73
|
+
end
|
74
|
+
|
75
|
+
def publish(message)
|
76
|
+
# Ensure the subscription exists first so the message won't just get dropped
|
77
|
+
subscription
|
78
|
+
|
79
|
+
topic.publish(message)
|
80
|
+
|
81
|
+
puts "Published message: #{message}"
|
82
|
+
end
|
83
|
+
end
|
84
|
+
|
85
|
+
subcommand 'sub', 'Subscribe to messages on the topic' do
|
86
|
+
def execute
|
87
|
+
subscribe do |received_message|
|
88
|
+
puts "Data: #{received_message.message.data}, published at #{received_message.message.published_at}"
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
def subscribe
|
93
|
+
subscriber = subscription.listen(threads: { callback: 16 }) do |received_message|
|
94
|
+
yield received_message
|
95
|
+
received_message.acknowledge!
|
96
|
+
end
|
97
|
+
|
98
|
+
subscriber.on_error do |exception|
|
99
|
+
puts "Exception: #{exception.class} #{exception.message}"
|
100
|
+
end
|
101
|
+
|
102
|
+
# Gracefully shut down the subscriber on program exit, blocking until all received messages have been processed or 10 seconds have passed
|
103
|
+
at_exit do
|
104
|
+
subscriber.stop!(10)
|
105
|
+
end
|
106
|
+
|
107
|
+
puts 'Listening for messages'
|
108
|
+
|
109
|
+
# Start the background threads that will call the block passed to listen
|
110
|
+
subscriber.start
|
111
|
+
|
112
|
+
# Keep the program open while the threads handle receiving messages
|
113
|
+
sleep
|
114
|
+
end
|
115
|
+
end
|
116
|
+
|
117
|
+
subcommand 'shell', 'Get a Ruby shell to interact with Pub/Sub' do
|
118
|
+
def execute
|
119
|
+
binding.pry # rubocop:disable Lint/Debugger
|
120
|
+
end
|
121
|
+
end
|
122
|
+
|
123
|
+
def subscription
|
124
|
+
@subscription ||= get_or_create_topic_subscription(subscription_name)
|
125
|
+
end
|
126
|
+
|
127
|
+
def get_or_create_topic_subscription(subscription_name)
|
128
|
+
topic.tap do
|
129
|
+
print "Checking existence of subscription '%s'... " % subscription_name
|
130
|
+
end.get_subscription(subscription_name)&.tap do # rubocop:disable Style/MultilineBlockChain
|
131
|
+
puts 'exists'
|
132
|
+
end || begin
|
133
|
+
print 'creating... '
|
134
|
+
topic.create_subscription(subscription_name).tap { puts 'done' }
|
135
|
+
end
|
136
|
+
end
|
137
|
+
|
138
|
+
def topic
|
139
|
+
@topic ||= get_or_create_topic(topic_name)
|
140
|
+
end
|
141
|
+
|
142
|
+
def get_or_create_topic(topic_name)
|
143
|
+
pubsub.tap do
|
144
|
+
print "Checking existence of topic '%s'... " % topic_name
|
145
|
+
end.get_topic(topic_name)&.tap do # rubocop:disable Style/MultilineBlockChain
|
146
|
+
puts 'exists'
|
147
|
+
end || begin
|
148
|
+
print 'creating... '
|
149
|
+
pubsub.create_topic(topic_name).tap { puts 'done' }
|
150
|
+
end
|
151
|
+
end
|
152
|
+
|
153
|
+
def pubsub
|
154
|
+
@pubsub ||= begin
|
155
|
+
print 'Initialising Pub/Sub client for %s... ' % (remote? ? 'remote' : "emulator (#{emulator_host})")
|
156
|
+
Google::Cloud::PubSub.new(**pubsub_args).tap { puts 'done' }
|
157
|
+
end
|
158
|
+
end
|
159
|
+
|
160
|
+
def pubsub_args
|
161
|
+
remote? ? pubsub_remote_args : pubsub_emulator_args
|
162
|
+
end
|
163
|
+
|
164
|
+
def pubsub_remote_args
|
165
|
+
{
|
166
|
+
project_id: 'greensync-dex-dev',
|
167
|
+
}
|
168
|
+
end
|
169
|
+
|
170
|
+
def pubsub_emulator_args
|
171
|
+
{
|
172
|
+
project_id: 'greensync-dex-dev-nonexistent',
|
173
|
+
emulator_host: emulator_host,
|
174
|
+
}
|
175
|
+
end
|
176
|
+
end
|
data/lib/pubsubc.rb
ADDED
metadata
ADDED
@@ -0,0 +1,107 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: pubsubc
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Brendan Weibrecht
|
8
|
+
autorequire:
|
9
|
+
bindir: exe
|
10
|
+
cert_chain: []
|
11
|
+
date: 2021-07-30 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: clamp
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 1.3.2
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 1.3.2
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: google-cloud-pubsub
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: 2.7.1
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: 2.7.1
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: jazz_fingers
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - "~>"
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: 6.2.0
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - "~>"
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: 6.2.0
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: rainbow
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: 3.0.0
|
62
|
+
type: :runtime
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: 3.0.0
|
69
|
+
description:
|
70
|
+
email:
|
71
|
+
- brendan@weibrecht.net.au
|
72
|
+
executables:
|
73
|
+
- pubsubc
|
74
|
+
extensions: []
|
75
|
+
extra_rdoc_files: []
|
76
|
+
files:
|
77
|
+
- README.md
|
78
|
+
- exe/pubsubc
|
79
|
+
- lib/pubsubc.rb
|
80
|
+
- lib/pubsubc/version.rb
|
81
|
+
homepage: https://github.com/ZimbiX/pubsubc
|
82
|
+
licenses:
|
83
|
+
- GPL-3.0
|
84
|
+
metadata:
|
85
|
+
homepage_uri: https://github.com/ZimbiX/pubsubc
|
86
|
+
source_code_uri: https://github.com/ZimbiX/pubsubc
|
87
|
+
changelog_uri: https://github.com/ZimbiX/pubsubc/releases
|
88
|
+
post_install_message:
|
89
|
+
rdoc_options: []
|
90
|
+
require_paths:
|
91
|
+
- lib
|
92
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - ">="
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: 2.6.0
|
97
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
98
|
+
requirements:
|
99
|
+
- - ">="
|
100
|
+
- !ruby/object:Gem::Version
|
101
|
+
version: '0'
|
102
|
+
requirements: []
|
103
|
+
rubygems_version: 3.2.16
|
104
|
+
signing_key:
|
105
|
+
specification_version: 4
|
106
|
+
summary: A simple command-line client for Google Cloud Pub/Sub
|
107
|
+
test_files: []
|