laboristo 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 3733281eacf2c77397c8e7f452273bfb33e852bc
+  data.tar.gz: efdd5e9f806e245917c28339ada758b331f0821f
+SHA512:
+  metadata.gz: 9ed914bf20b5799719c6e569f1845a9061bc78c52935065cc7bf63bd12d596f6fa11838bde6786af7e826a103e9e4702771d835fba1d2e7d05746e4cb77feeea
+  data.tar.gz: 7abb5b6db3f92b7c619b6cb270f155e6329f3c70925f2f524ff7dc985b0560db16727b827d877f8f3586eac220fe86c4e661887fa89a4c6d44a452a4fb28e330

data/.gems

@@ -0,0 +1,3 @@
+aws-sdk-core -v 2.0.33
+clap -v 1.0.0
+cutest -v 1.2.2

data/.gitignore

@@ -0,0 +1,37 @@
+ .gitignore
+*.gem
+*.gs
+*.rbc
+/.config
+/coverage/
+/InstalledFiles
+/pkg/
+/spec/reports/
+/test/tmp/
+/test/version_tmp/
+/tmp/
+.rubocop.yml
+
+## Specific to RubyMotion:
+.dat*
+.repl_history
+build/
+
+## Documentation cache and generated files:
+/.yardoc/
+/_yardoc/
+/doc/
+/rdoc/
+
+## Environment normalisation:
+/.bundle/
+/lib/bundler/man/
+
+# for a library or gem, you might want to ignore these files since the code is
+# intended to run in multiple environments; otherwise, check them in:
+# Gemfile.lock
+# .ruby-version
+# .ruby-gemset
+
+# unless supporting rvm < 1.11.0 or doing something fancy, ignore this:
+.rvmrc

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2015 matiasow
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,115 @@
+# Laboristo
+Laboristo is an attempt to port [Ost](https://github.com/soveran/ost) simplicity to AWS SQS.
+
+## Installation
+
+Simply run:
+
+	$ gem install laboristo
+
+### Using Bundler?
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'laboristo'
+```
+
+And then execute:
+
+    $ bundle
+
+
+## Configuration
+
+Before using Laboristo, you will need to set some environment variables:
+
+	AWS_REGION=<your_aws_region>
+	AWS_ACCOUNT_ID=<your_aws_account_id>
+	AWS_ACCESS_KEY_ID=<your_aws_access_key>
+	AWS_SECRET_ACCESS_KEY=<your_aws_secret_key>
+
+Please, refer to [AWS documentation](%28http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/AWSCredentials.html%29) to obtain the corresponding values for your account.
+
+## Considerations
+
+First of all, if you are not familiar with AWS SQS I hardly recommend that you begin reading the [AWS SQS Getting Started Guide](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/Welcome.html).
+
+Here are some considerations that you need to know when using Laboristo and SQS:
+
+ - Laboristo does not create queues. You will have to set them up before sending messages to it (pretty obvious, right?). For instance, you can use [AWS Console](http://console.aws.amazon.com/) to create quehes.
+ - When creating queues, you can set up queue attributes such as *DelaySeconds*, *MaximumMessageSize*, *MessageRetentionPeriod*, *Policy*, *ReceiveMessageWaitTimeSeconds* and *VisibilityTimeout*. Tune this settings according to your needs to efficiently use the service, and keep usage costs at a minimum level.
+ - Laboristo retrieves the message, yields the process to your code, and then deletes the message from the queue. If something fails in between, the message will go back to the queue. I encourage you to consider using [Dead Letter Queues](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/SQSDeadLetterQueue.html) to prevent eternal processing of invalid/wrong messages.
+
+## Usage
+
+To push a message to the queue:
+
+```ruby
+require 'laboristo'
+
+Laboristo['my_queue_name'] << 'some message'
+```
+
+And get messages from a queue:
+```ruby
+require 'laboristo'
+Laboristo['my_queue_name'].each do |msg|
+  # Do something with msg...
+end
+```
+
+### Workers
+
+Workers can be as simple as this:
+
+```ruby
+  require 'my_app'
+
+  Laboristo['my_queue'].each do |message|
+    # Do some magic stuff with the message
+  end
+```
+
+Place worker at ```./workers/my_worker.rb``` and run this code to run worker in foreground:
+
+```
+  $ laboristo my_worker
+```
+
+To run it in background you can use ```-d``` flag to daemonize the process:
+
+```
+  $ laboristo my_worker -d
+```
+
+You can stop the workers by killing the process. Keep in mind that, because of how SQS works, unprocessed messages will go back to the queue.
+
+## Development
+
+After cloning repository install dependencies using [dep](https://github.com/cyx/dep):
+
+```
+  $ dep install
+```
+
+Set environment variables as explained before in this document, and run the tests:
+
+```
+  $ make
+```
+
+### TO-DO:
+
+First of all, add more tests! This gem stated as an experiment, so you can expect bugs.
+
+In future versions, I'd like to improve the worker bin so that it can be started/stopped in a more elegant fashion than simply killing the process ;-)
+
+Feel free to report bugs, open issues, comments and pull requests. Only keep in mind that I just want to keep this gem neat and simple.
+
+## Credits
+
+Most of the credit for this gem goes to [@soveran](https://github.com/soveran/ost) and [@djanowski](https://github.com/djanowski/ost-bin), who created the code this gem is based in. Kudos to them!
+
+## License
+Laboristo is released under the [MIT License](http://www.opensource.org/licenses/MIT).

data/bin/laboristo

@@ -0,0 +1,46 @@
+#! /usr/bin/env ruby
+
+stop = proc do
+  if defined?(Laboristo)
+    Laboristo.stop
+  else
+    exit 0
+  end
+end
+
+trap(:INT,  &stop)
+trap(:TERM, &stop)
+
+usage = <<-EOS
+Usage:
+  laboristo start <worker> [-d|--daemonize]
+  laboristo stop <worker>
+EOS
+
+require 'clap'
+require 'laboristo'
+
+opts = {}
+
+args = Clap.run ARGV,
+  '-d' || '--daemonize' => -> {
+    opts[:daemonize] = true
+  }
+
+command = args.shift
+worker = args.shift
+worker_path = File.expand_path("workers/#{worker}")
+pid_path = File.expand_path("tmp/#{worker}.pid")
+
+$stdout.sync = true
+
+
+if opts[:daemonize]
+  Process.daemon(true)
+  File.open(pid_path, File::RDWR | File::EXCL | File::CREAT, 0600) { |io| io.write(Process.pid) }
+  at_exit do
+    File.delete(pid_path) if File.exist?(pid_path)
+  end
+end
+
+require worker_path

data/laboristo.gemspec

@@ -0,0 +1,18 @@
+Gem::Specification.new do |s|
+  s.name          = 'laboristo'
+  s.version       = '0.1.0'
+  s.summary       = 'Simple messages and workers for AWS SQS'
+  s.authors       = ['matiasow']
+  s.email         = ['matiasow@gmail.com']
+  s.homepage      = 'https://github.com/matiasow/laboristo'
+  s.license       = 'MIT'
+
+  s.files         = `git ls-files`.split("\n")
+  s.bindir        = 'bin'
+  s.executables   << 'laboristo'
+
+  s.add_dependency 'aws-sdk-core', '~> 2'
+  s.add_dependency 'clap', '~> 1'
+
+  s.add_development_dependency 'cutest', '~> 1.2'
+end

data/lib/laboristo.rb

@@ -0,0 +1,46 @@
+require 'aws-sdk-core'
+require 'base64'
+
+module Laboristo
+  class Queue
+    attr_accessor :name
+    attr_accessor :url
+    attr_accessor :sqs
+
+    def initialize(name)
+      @name = name
+      @sqs = Aws::SQS::Client.new
+      @url = "https://sqs.#{ENV['AWS_REGION']}.amazonaws.com/#{ENV['AWS_ACCOUNT_ID']}/#{@name}"
+    end
+
+    def push(message)
+      encoded = Base64.encode64(message.to_s)
+      @sqs.send_message(queue_url: @url, message_body: encoded)
+    end
+
+    def each(&block)
+      loop do
+        resp = @sqs.receive_message(queue_url: @url,
+                                    attribute_names: ['All'],
+                                    max_number_of_messages: 10).data.to_hash
+
+        resp[:messages] && resp[:messages].each do |msg|
+          begin
+            block.call(Base64.decode64 msg[:body])
+            @sqs.delete_message(queue_url: @url,
+                                receipt_handle: msg[:receipt_handle])
+          rescue StandardError => e
+            $stderr.puts "ERROR: Can't process message #{msg[:message_id]}.\n#{e}"
+          end
+        end
+      end
+    end
+
+    alias << push
+    alias pop each
+  end
+
+  def self.[](queue)
+    Queue.new(queue)
+  end
+end

data/makefile

@@ -0,0 +1,9 @@
+.PHONY: test
+
+test:
+	@echo Running tests... && \
+	RUBYLIB=./lib cutest test/*_test.rb
+
+console:
+	@echo Running console... && \
+	RUBYLIB=./lib irb -r laboristo

data/test/helper.rb

@@ -0,0 +1,42 @@
+require 'laboristo'
+require 'securerandom'
+
+if ENV['TRAVIS']
+  require 'codeclimate-test-reporter'
+  CodeClimate::TestReporter.start
+end
+
+def send_message(message)
+  @client.send_message(queue_url: @url, message_body: Base64.encode64(message))
+end
+
+def fetch_last_message
+  fetched = @client.receive_message(queue_url: @url,
+                                    max_number_of_messages: 1,
+                                    wait_time_seconds: 0).data.to_hash
+
+  fetched[:messages].first
+end
+
+def message_body(msg)
+  Base64.decode64(msg[:body])
+end
+
+def delete_message(msg)
+  @client.delete_message(queue_url: @url,
+                         receipt_handle: msg[:receipt_handle])
+end
+
+test 'environment' do
+  assert ENV['AWS_REGION']
+  assert ENV['AWS_ACCESS_KEY_ID']
+  assert ENV['AWS_SECRET_ACCESS_KEY']
+  assert ENV['AWS_ACCOUNT_ID']
+end
+
+@account = ENV['AWS_ACCOUNT_ID']
+@region = ENV['AWS_REGION']
+@client = Aws::SQS::Client.new
+
+@queue_name = SecureRandom.uuid
+@url = @client.create_queue(queue_name: @queue_name).data.to_hash[:queue_url]

data/test/queue_test.rb

@@ -0,0 +1,66 @@
+require_relative 'helper'
+
+test 'initialize' do
+  q = Laboristo::Queue.new(@queue_name)
+  assert q.name == @queue_name
+  assert q.sqs.is_a? Aws::SQS::Client
+  assert q.url == @url
+end
+
+test 'initialize through []' do
+  q = Laboristo[@queue_name]
+  assert q.name == @queue_name
+  assert q.sqs.is_a? Aws::SQS::Client
+  assert q.url == @url
+end
+
+test 'push message to queue' do
+  q = Laboristo[@queue_name]
+  msg = SecureRandom.uuid
+
+  q.push(msg)
+
+  fetched = fetch_last_message
+  delete_message(fetched)
+
+  assert msg == message_body(fetched)
+end
+
+test 'push message to queue using <<' do
+  q = Laboristo[@queue_name]
+  msg = SecureRandom.uuid
+
+  q << msg
+
+  fetched = fetch_last_message
+  delete_message(fetched)
+
+  assert msg == message_body(fetched)
+end
+
+test 'fetch messages using pop' do
+  q = Laboristo[@queue_name]
+  msg = SecureRandom.uuid
+
+  send_message(msg)
+
+  q.pop do |m|
+    assert m == msg
+    break # only fetch one message
+  end
+end
+
+test 'fetch messages using each' do
+  q = Laboristo[@queue_name]
+  msg = SecureRandom.uuid
+
+  send_message(msg)
+
+  q.each do |m|
+    assert m == msg
+    break # only fetch one message
+  end
+end
+
+# And finally destroy the queue...
+@client.delete_queue(queue_url: @url)

metadata

@@ -0,0 +1,97 @@
+--- !ruby/object:Gem::Specification
+name: laboristo
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- matiasow
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2015-04-03 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-core
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2'
+- !ruby/object:Gem::Dependency
+  name: clap
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+- !ruby/object:Gem::Dependency
+  name: cutest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+description: 
+email:
+- matiasow@gmail.com
+executables:
+- laboristo
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gems"
+- ".gitignore"
+- LICENSE
+- README.md
+- bin/laboristo
+- laboristo.gemspec
+- lib/laboristo.rb
+- makefile
+- test/helper.rb
+- test/queue_test.rb
+homepage: https://github.com/matiasow/laboristo
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.4.4
+signing_key: 
+specification_version: 4
+summary: Simple messages and workers for AWS SQS
+test_files: []