pubsubc 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 688500fbb35695a35b4109e914ea2ba45277d64743d2372f865f65d5dd46ad6d
+  data.tar.gz: 8839a7ad9d76cfc88578ed8357c716fca88afe065e8bcfaa4e28806ccddd548f
+SHA512:
+  metadata.gz: 4a8375467b9013c139fd84a371cb98a5b27782d9a802048dfada45d9cc8fab9384e826e4b1ef1b9779c4278772747d6540d838da21021f3610799ea723a0d3c6
+  data.tar.gz: b79699630a59e7e70639acc40b997442b7e273d9efabf873878b44c0110b4be010e2152e9d5b4682e3afe38099e53b0e9599e846a22dfb37e146ac309f990ab0

data/README.md

@@ -0,0 +1,236 @@
+# pubsubc
+
+[![CI status](https://github.com/ZimbiX/pubsubc/actions/workflows/main.yml/badge.svg?branch=master)](https://github.com/ZimbiX/pubsubc/actions/workflows/main.yml) [![Gem Version](https://badge.fury.io/rb/pubsubc.svg)](https://rubygems.org/gems/pubsubc)
+
+A simple command-line client for [Google Cloud Pub/Sub](https://cloud.google.com/pubsub).
+
+Unlike the `gcloud pubsub` utility, `pubsubc`:
+
+- Supports the [Pub/Sub emulator](https://cloud.google.com/pubsub/docs/emulator)
+- Provides a continuous subscribe command - to see all messages as they come in
+
+Since this was originally created as a learning exercise for integrating to Pub/Sub, the source code should be simple enough for you to copy and adapt into your own application.
+
+## Contents
+
+<!-- MarkdownTOC autolink=true -->
+
+- [Intro](#intro)
+- [Installation](#installation)
+  - [Direct](#direct)
+  - [Bundler](#bundler)
+- [Usage](#usage)
+  - [Subscribe](#subscribe)
+  - [Publish message](#publish-message)
+  - [Publish generated messages](#publish-generated-messages)
+- [Example](#example)
+  - [Communicating some messages](#communicating-some-messages)
+  - [Publishing from a message generator with several subscribers](#publishing-from-a-message-generator-with-several-subscribers)
+- [Help info](#help-info)
+- [Contributing](#contributing)
+- [Development](#development)
+  - [Pub/Sub emulator](#pubsub-emulator)
+  - [Pre-push hook](#pre-push-hook)
+  - [Release](#release)
+
+<!-- /MarkdownTOC -->
+
+## Intro
+
+Features:
+
+- Subscribe to a topic, continuously logging all messages as they arrive
+- Publish a string message to a topic (creating a subscription automatically)
+- Publish automatically-generated messages continuously at a given interval
+- Supports connecting to either a local emulator or the remote real Pub/Sub
+
+## Installation
+
+The executable is distributed through RubyGems.
+
+### Direct
+
+You can install it to your system directly using:
+
+```bash
+gem install pubsubc
+```
+
+And then if you're using rbenv:
+
+```bash
+rbenv rehash
+```
+
+You can then run it with:
+
+```bash
+pubsubc
+```
+
+### Bundler
+
+Or, just add it to the `Gemfile` of your project:
+
+```ruby
+gem 'pubsubc'
+```
+
+And then run it with:
+
+```bash
+bundle exec pubsubc
+```
+
+## Usage
+
+### Subscribe
+
+To subscribe and continuously log all messages from the topic:
+
+```bash
+pubsubc sub
+```
+
+### Publish message
+
+To publish a single message to the topic:
+
+```bash
+pubsubc pub msg "G'day"
+```
+
+### Publish generated messages
+
+To publish a generated message to the topic every second:
+
+```bash
+pubsubc pub gen --interval 1000
+```
+
+## Example
+
+### Communicating some messages
+
+```
+➜ pubsubc pub msg "Sup wigga"    
+Initialising Pub/Sub client for emulator (localhost:8085)... done
+Checking existence of topic 'my-topic'... creating... done
+Checking existence of subscription 'my-subscription'... creating... done
+Published message: Sup wigga
+
+➜ pubsubc pub msg "Yo J-dog" 
+Initialising Pub/Sub client for emulator (localhost:8085)... done
+Checking existence of topic 'my-topic'... exists
+Checking existence of subscription 'my-subscription'... exists
+Published message: Yo J-dog
+
+➜ pubsubc sub                
+Initialising Pub/Sub client for emulator (localhost:8085)... done
+Checking existence of topic 'my-topic'... exists
+Checking existence of subscription 'my-subscription'... exists
+Listening for messages
+Data: Sup wigga, published at 2021-07-30 23:19:06 +1000
+Data: Yo J-dog, published at 2021-07-30 23:19:41 +1000
+^C
+```
+
+### Publishing from a message generator with several subscribers
+
+![](doc/example-with-generator-and-several-subscribers.png)
+
+The generator was run with:
+
+```bash
+pubsubc pub gen --interval 1
+```
+
+And each subscriber:
+
+```bash
+pubsubc sub
+```
+
+## Help info
+
+Output of `pubsubc --help`:
+
+<!-- Help -->
+
+```
+Usage:
+    pubsubc [OPTIONS] SUBCOMMAND [ARG] ...
+
+  A simple command-line client for Google Cloud Pub/Sub.
+
+  Created as a learning exercise, but also useful since the 'gcloud pubsub' commands don't support the emulator, and it doesn't provide a continuous subscribe command.
+
+  To clean up:
+
+      gcloud pubsub topics delete my-topic
+      gcloud pubsub subscriptions delete my-subscription
+
+Parameters:
+    SUBCOMMAND                          subcommand
+    [ARG] ...                           subcommand arguments
+
+Subcommands:
+    pub                                 Publishing messages to the topic
+    sub                                 Subscribe to messages on the topic
+    shell                               Get a Ruby shell to interact with Pub/Sub
+
+Options:
+    --topic TOPIC_NAME                  The name of the Pub/Sub topic (default: "my-topic")
+    --subscription SUBSCRIPTION_NAME    The name of the Pub/Sub subscription within the topic (default: "my-subscription")
+    --remote                            Connect to the real Pub/Sub, rather than a local Pub/Sub emulator
+    --emulator-host EMULATOR_HOST       The host and port of the Pub/Sub emulator (default: $PUBSUB_EMULATOR_HOST, or "localhost:8085")
+    -h, --help                          print help
+```
+
+<!-- /Help -->
+
+Suffix `--help` to any subcommand to see the usage info for it.
+
+## Contributing
+
+Pull requests welcome! =)
+
+However, I don't intend to duplicate functionality that exists in the `gcloud pubsub` utility, unless it's an improvement over what's in that.
+
+## Development
+
+### Pub/Sub emulator
+
+To run the Pub/Sub emulator (with Docker):
+
+```bash
+docker-compose up -d pubsub-emulator
+```
+
+### Pre-push hook
+
+This hook runs style checks and tests.
+
+To set up the pre-push hook:
+
+```bash
+echo -e "#\!/bin/bash\n\$(dirname \$0)/../../auto/pre-push-hook" > .git/hooks/pre-push
+chmod +x .git/hooks/pre-push
+```
+
+### Release
+
+To release a new version:
+
+```bash
+auto/release/update-version && auto/release/tag && auto/release/publish
+```
+
+This takes care of the whole process:
+
+- Incrementing the version number (the patch number by default)
+- Tagging & pushing commits
+- Publishing the gem to RubyGems
+- Creating a draft GitHub release
+
+To increment the minor or major versions instead of the patch number, run `auto/release/update-version` with `--minor` or `--major`.

data/exe/pubsubc

@@ -0,0 +1,176 @@
+#!/usr/bin/env ruby
+
+# frozen_string_literal: true
+
+ENV['GOOGLE_AUTH_SUPPRESS_CREDENTIALS_WARNINGS'] = 'true'
+
+require 'google/cloud/pubsub'
+require 'clamp'
+require 'pry'
+
+Clamp.allow_options_after_parameters = true
+
+Clamp do # rubocop:disable Metrics/BlockLength
+  self.description = <<~TEXT
+    A simple command-line client for Google Cloud Pub/Sub.
+
+    Created as a learning exercise, but also useful since the 'gcloud pubsub' commands don't support the emulator, and it doesn't provide a continuous subscribe command.
+
+    To clean up:
+
+        gcloud pubsub topics delete my-topic
+        gcloud pubsub subscriptions delete my-subscription
+  TEXT
+
+  option '--topic',
+         'TOPIC_NAME',
+         'The name of the Pub/Sub topic',
+         attribute_name: :topic_name,
+         default: 'my-topic'
+
+  option '--subscription',
+         'SUBSCRIPTION_NAME',
+         'The name of the Pub/Sub subscription within the topic',
+         attribute_name: :subscription_name,
+         default: 'my-subscription'
+
+  option '--remote', :flag, 'Connect to the real Pub/Sub, rather than a local Pub/Sub emulator'
+
+  option '--emulator-host',
+         'EMULATOR_HOST',
+         'The host and port of the Pub/Sub emulator',
+         environment_variable: 'PUBSUB_EMULATOR_HOST',
+         default: 'localhost:8085' do |host|
+    ENV.delete('PUBSUB_EMULATOR_HOST')
+    ENV.delete('PUBSUB_PROJECT_ID')
+    host
+  end
+
+  subcommand 'pub', 'Publishing messages to the topic' do # rubocop:disable Metrics/BlockLength
+    subcommand 'msg', 'Publish one message' do
+      parameter 'MESSAGE', 'The message string to publish to the topic'
+
+      def execute
+        publish(message)
+      end
+    end
+
+    subcommand 'gen', 'Publish generated messages' do
+      option %w[-i --interval],
+             'INTERVAL_MS',
+             'How long we should sleep before generating each message',
+             attribute_name: :interval_ms,
+             required: true,
+             &method(:Integer)
+
+      def execute
+        (1..).each do |i|
+          message = "This is message #{i}"
+          publish(message)
+          sleep(interval_ms / 1000.0)
+        end
+      end
+    end
+
+    def publish(message)
+      # Ensure the subscription exists first so the message won't just get dropped
+      subscription
+
+      topic.publish(message)
+
+      puts "Published message: #{message}"
+    end
+  end
+
+  subcommand 'sub', 'Subscribe to messages on the topic' do
+    def execute
+      subscribe do |received_message|
+        puts "Data: #{received_message.message.data}, published at #{received_message.message.published_at}"
+      end
+    end
+
+    def subscribe
+      subscriber = subscription.listen(threads: { callback: 16 }) do |received_message|
+        yield received_message
+        received_message.acknowledge!
+      end
+
+      subscriber.on_error do |exception|
+        puts "Exception: #{exception.class} #{exception.message}"
+      end
+
+      # Gracefully shut down the subscriber on program exit, blocking until all received messages have been processed or 10 seconds have passed
+      at_exit do
+        subscriber.stop!(10)
+      end
+
+      puts 'Listening for messages'
+
+      # Start the background threads that will call the block passed to listen
+      subscriber.start
+
+      # Keep the program open while the threads handle receiving messages
+      sleep
+    end
+  end
+
+  subcommand 'shell', 'Get a Ruby shell to interact with Pub/Sub' do
+    def execute
+      binding.pry # rubocop:disable Lint/Debugger
+    end
+  end
+
+  def subscription
+    @subscription ||= get_or_create_topic_subscription(subscription_name)
+  end
+
+  def get_or_create_topic_subscription(subscription_name)
+    topic.tap do
+      print "Checking existence of subscription '%s'... " % subscription_name
+    end.get_subscription(subscription_name)&.tap do # rubocop:disable Style/MultilineBlockChain
+      puts 'exists'
+    end || begin
+      print 'creating... '
+      topic.create_subscription(subscription_name).tap { puts 'done' }
+    end
+  end
+
+  def topic
+    @topic ||= get_or_create_topic(topic_name)
+  end
+
+  def get_or_create_topic(topic_name)
+    pubsub.tap do
+      print "Checking existence of topic '%s'... " % topic_name
+    end.get_topic(topic_name)&.tap do # rubocop:disable Style/MultilineBlockChain
+      puts 'exists'
+    end || begin
+      print 'creating... '
+      pubsub.create_topic(topic_name).tap { puts 'done' }
+    end
+  end
+
+  def pubsub
+    @pubsub ||= begin
+      print 'Initialising Pub/Sub client for %s... ' % (remote? ? 'remote' : "emulator (#{emulator_host})")
+      Google::Cloud::PubSub.new(**pubsub_args).tap { puts 'done' }
+    end
+  end
+
+  def pubsub_args
+    remote? ? pubsub_remote_args : pubsub_emulator_args
+  end
+
+  def pubsub_remote_args
+    {
+      project_id: 'greensync-dex-dev',
+    }
+  end
+
+  def pubsub_emulator_args
+    {
+      project_id: 'greensync-dex-dev-nonexistent',
+      emulator_host: emulator_host,
+    }
+  end
+end

data/lib/pubsubc.rb

@@ -0,0 +1,3 @@
+# frozen_string_literal: true
+
+require_relative 'pubsubc/version'

data/lib/pubsubc/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+class Pubsubc
+  VERSION = '0.1.0'
+end

metadata

@@ -0,0 +1,107 @@
+--- !ruby/object:Gem::Specification
+name: pubsubc
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Brendan Weibrecht
+autorequire:
+bindir: exe
+cert_chain: []
+date: 2021-07-30 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: clamp
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.3.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.3.2
+- !ruby/object:Gem::Dependency
+  name: google-cloud-pubsub
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.7.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 2.7.1
+- !ruby/object:Gem::Dependency
+  name: jazz_fingers
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.2.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.2.0
+- !ruby/object:Gem::Dependency
+  name: rainbow
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+description:
+email:
+- brendan@weibrecht.net.au
+executables:
+- pubsubc
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- exe/pubsubc
+- lib/pubsubc.rb
+- lib/pubsubc/version.rb
+homepage: https://github.com/ZimbiX/pubsubc
+licenses:
+- GPL-3.0
+metadata:
+  homepage_uri: https://github.com/ZimbiX/pubsubc
+  source_code_uri: https://github.com/ZimbiX/pubsubc
+  changelog_uri: https://github.com/ZimbiX/pubsubc/releases
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.6.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.2.16
+signing_key:
+specification_version: 4
+summary: A simple command-line client for Google Cloud Pub/Sub
+test_files: []