nectarine 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 10f4ed6346611714e8ceb3f08bb5cb4b4c5f34a94be6dc7eac181e6eab21bbcc
+  data.tar.gz: '0799a978010ccdbe25aa9b1ce0c1c09a6e2855a516dd81a212836199b194f0c3'
+SHA512:
+  metadata.gz: 541cb106a4f26e3f5033ef6e918fcc23a0c9acfcea833fe8d0e8c2ace759ac8b389298c32cc6fa29255ab079f155de7ee03fa8f72066e8839741c41e9d0520ce
+  data.tar.gz: 1e64d7e9916044490a17f4db046f7d7807267cea27597bbd0c830d4ea49aafc3207f83eb5d585f1ce55cb46bf1ec856db28f7b307fa9d160e692b8168a8a0b6a

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2018 n8
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,83 @@
+# Nectarine
+
+*Parallel processing*
+
+> a mode of computer operation in which a process is split into parts that execute simultaneously on different processors attached to the same computer.
+
+I find myself constantly working on apps that need a mechanism to split an array of objects into multiple threads or CPUs to perform things on those objects, then return processing back to the main thread. 
+
+But it's still something I find surprisingly difficult to do in Ruby/Rails. There's definitely libraries that successfully help, but they all suffer from problems around versions of Ruby being used (green threads, native threads, interpreter locks) or your OS/environment (Windows/Mac/Linux).
+
+It dawned on me though that we already have a really interesting place to do parallel processing and communication amongst multiple threads/CPUs that's fairly agnostic of environment AND has built in fault tolerance: our current Job queue. 
+
+Whether we're using Resque/Sidekiq/Delayed Job/etc. Why not send things there for parallel processing our main thread waits for execution? 
+
+So here's a proof of concept to do just that. It's called Nectarine. 
+
+Nectarine works over a collection of Active Record objects, sending each object to your Job system using Active Job, and executing a single method on that object. The main thread blocks until processing has completed on all the objects. 
+
+That's it. I'm using it now in an app ([FilmHope.com](https://filmhope.com/)) that takes a user's video, converts it into hundreds of static thumbnails, and then for each thumbnail, Nectarine jobs out processing on each image, calling a bunch of other APIs as needed. It's a super slow task, which is why having Nectarine handle those hundreds of thumbs in parallel is a nice workflow to have. 
+
+
+## Usage
+Once nectarine is installed, you can call it on your collections of Active Record objects.  
+
+Let's say a Snail model in our system has a super slow method like "start_racing". It's slow because start_racing calls external APIs, does a bunch of other magic, whatever. It's slow. 
+
+```ruby
+Snail.all.each(&:start_racing) 
+Rails.logger.debug("That took forever")
+```
+
+Would take forever. 
+
+Instead:  
+
+```ruby
+Snail.all.nectarine(:start_racing) 
+Rails.logger.debug("That took a lot less than forever")
+```
+
+That's it. In this example, the Rails.logger.debug line isn't executed until all the snails in Snail.all have executed their "start_racing" method. 
+
+## Installation
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'nectarine'
+```
+
+And then execute:
+```bash
+$ bundle
+```
+
+Or install it yourself as:
+```bash
+$ gem install nectarine
+```
+
+## What's next. 
+
+There's a lot of gaps in this library right now. 
+
+A total Timeout period is currently hard coded at 15 minutes. That should at a very minium be configurable. There's other things like status checks of the jobs that just happen every 2 seconds. 
+
+```ruby
+Timeout::timeout(15 * 60) do 
+    while(!all_job_statuses_complete?(all_jobs)) do 
+        sleep 2
+    end
+end
+```
+
+It might be smarter to do some exponential backoff checking for their completion. There's also not very good error control or handling. 
+
+This also just uses whatever your default queue is, which is something that should probably be configurable. 
+
+So all feedback and pull requests are more than welcome!
+
+
+## P.S. 
+
+If you need a some help building a software project, [**whether it's advice or just extra hands, please shout**](https://www.rockstarcoders.com/contact-us/). We've got a great crew at Rockstar Coders and we'd love to help. 

data/Rakefile

@@ -0,0 +1,27 @@
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+require 'rdoc/task'
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'Nectarine'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.md')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+require 'bundler/gem_tasks'
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+task default: :test

data/lib/nectarine.rb

@@ -0,0 +1,32 @@
+require "nectarine/railtie"
+require "nectarine/good_job"
+
+module Nectarine
+
+end
+
+class ActiveRecord::Relation
+  def nectarine(method)
+
+    method_to_run = method.to_s
+
+    all_jobs = []
+    self.each do |item|
+      all_jobs << Nectarine::GoodJob.perform_later(item, method_to_run)
+    end
+
+    Timeout::timeout(15 * 60) do 
+      while(!all_job_statuses_complete?(all_jobs)) do 
+        sleep 2
+      end
+    end
+
+    return 
+  end
+
+  def all_job_statuses_complete?(all_jobs)
+    done = all_jobs.select{|job| ActiveJob::Status.get(job).completed?}
+    done.size == all_jobs.size
+  end
+end
+

data/lib/nectarine/good_job.rb

@@ -0,0 +1,9 @@
+require "activejob-status"
+
+class Nectarine::GoodJob < ActiveJob::Base
+  include ActiveJob::Status
+
+  def perform(object, method)
+    object.try(method)
+  end
+end

data/lib/nectarine/railtie.rb

@@ -0,0 +1,4 @@
+module Nectarine
+  class Railtie < ::Rails::Railtie
+  end
+end

data/lib/nectarine/version.rb

@@ -0,0 +1,3 @@
+module Nectarine
+  VERSION = '0.1.0'
+end

data/lib/tasks/nectarine_tasks.rake

@@ -0,0 +1,4 @@
+# desc "Explaining what the task does"
+# task :nectarine do
+#   # Task goes here
+# end

metadata

@@ -0,0 +1,96 @@
+--- !ruby/object:Gem::Specification
+name: nectarine
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- n8
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2018-09-04 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 5.2.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 5.2.0
+- !ruby/object:Gem::Dependency
+  name: activejob-status
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.1.5
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 0.1.5
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Runs a method over a collection of Rails models. But does it in parallel
+  using Active Job.
+email:
+- nate.kontny@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- MIT-LICENSE
+- README.md
+- Rakefile
+- lib/nectarine.rb
+- lib/nectarine/good_job.rb
+- lib/nectarine/railtie.rb
+- lib/nectarine/version.rb
+- lib/tasks/nectarine_tasks.rake
+homepage: https://github.com/n8/nectarine
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.7.3
+signing_key: 
+specification_version: 4
+summary: Runs a method over a collection of Rails models. But does it in parallel
+  using Active Job.
+test_files: []