belated 0.1.0 → 0.3.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3faf2091255e3a556c9abed02f3694bbbcd1bba9250621181745ef0708986d2d
-  data.tar.gz: acbcd9dd1d921a22cb1242e5f2ec8713f4315e21a197176094433926c1cb9302
+  metadata.gz: 995560f345bb7644aa19035b27a68d43a1210ee36c21d13e21cc430b76d9fded
+  data.tar.gz: 103441a7e60e79319c65bacbedfb4dc337224c41c0e9560225f3b2421e219473
 SHA512:
-  metadata.gz: 4066d54a8bd2d8e8a5a148bac5d2757081ed9d1778804f023561eade8f5a0b985357f807de545d5298fae39970c2f035545a2cdaac95b3e6534d490b5a0227a5
-  data.tar.gz: df1f8c8dc6bcb6fa1e22e0a6ab430bb63d8e14b42a13500d7e185331d02f35a4b294642210f9581068f98ed213754064273a252c8526a303e1428e4155d2d24c
+  metadata.gz: ecc27c208d2fdb26c93ee39ebba7ada11236e9f564565d69258dfa6381a407c924e076365329f088ca4071955fd0eaeb8e9e1a1119f77d824473987590fee670
+  data.tar.gz: 57db27785351cbba33b6c704f877eacf4343d95a02267411e79bf63f1efadb49a67430342d145898a8571318bc61eafdd756bc2618388603122a0733bad7a3ed

data/.gitignore

@@ -11,4 +11,6 @@
 .rspec_status
 .byebug_history
 /dummy/tmp/cache/
-belated_dump
+/dummy/log/
+/dummy/log/test.log
+belated_dump

data/.rubocop.yml

@@ -31,6 +31,9 @@ Style/GlobalVars:
 Style/BlockDelimiters:
   EnforcedStyle: braces_for_chaining
 
+Style/ModuleFunction:
+  EnforcedStyle: extend_self
+
 Metrics/BlockLength:
   Exclude:
     - 'spec/**/*.rb'

data/CHANGELOG.md

@@ -1,25 +1,45 @@
 ## [Unreleased]
-## [0.0.1] - 2021-07-15
 
-- Initial release! :tada:
-  Things are almost working. You can enqueue jobs! That should be more than enough.
-  Rails integration to be done soon!
 
-## [0.0.2] - 2021-07-15
 
-- Bugfix: was requiring byebug.
+## [0.3.2] - 2021-07-31
 
-## [0.0.3] - 2021-07-16
+- Trap INT and TERM, so now the shutdown is a little bit more graceful. 
 
-- YAML Marshalled queue! So now the queue stays in memory even if restarted. Though kill and term signals still need catching.. that is not done yet.
-- Also, now you can use classes! As long as the class has a perform-method.
+## [0.3.1] - 2021-07-29
+
+- Remove dummy app from gem... size should go down quite a bit. 
+
+## [0.3.0] - 2021-07-29
+
+- Now there is logging! By default logs to stdout, but you can configure the logger by setting a different one through `Belated.config.logger`. 
+## [0.2.0] - 2021-07-25
+
+- Workers now rescue StandardError and keep on working! 
+Note that they only rescue StandardError, and as there is no logger yet just pp the inspected error class. 
+
+## [0.1.0] - 2021-07-24
+
+- Gem name changed to Belated!
+
+Below are changes when this gem was named HardWorker:
 
 ## [0.0.4] - 2021-07-24
 
 - Rails support!
 - Banner (for Belated though...)
 - Final release, as the name of this gem is going to change to Belated.
+## [0.0.3] - 2021-07-16
 
-## [0.1.0] - 2021-07-24
+- YAML Marshalled queue! So now the queue stays in memory even if restarted. Though kill and term signals still need catching.. that is not done yet.
+- Also, now you can use classes! As long as the class has a perform-method.
 
-- Gem name changed to Belated!
+## [0.0.2] - 2021-07-15
+
+- Bugfix: was requiring byebug.
+
+## [0.0.1] - 2021-07-15
+
+- Initial release! :tada:
+  Things are almost working. You can enqueue jobs! That should be more than enough.
+  Rails integration to be done soon!

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    belated (0.1.0)
+    belated (0.3.2)
       drb
       dry-configurable
 

data/README.md

@@ -1,24 +1,32 @@
-# Hardworker
+# Belated
 
 [![CodeFactor](https://www.codefactor.io/repository/github/sampokuokkanen/belated/badge)](https://www.codefactor.io/repository/github/sampokuokkanen/belated) [![Gem Version](https://badge.fury.io/rb/belated.svg)](https://badge.fury.io/rb/belated)
 
-This is Belated, a new Ruby backend job library! It supports running procs and classes in the background.
-~~Also, you lose all jobs if you restart the process.~~ It now uses YAML to load the queue into a file, which it then calls at startup to find the previous jobs.
+This is Belated, a new Ruby backend job library! It supports running procs and classes in the background. To deal with restarts, it uses YAML to load the queue into a file, which it then calls at startup to find the previous jobs.
+
 Note that Belated used to be called HardWorker. That name was already in use in Sidekiq documentation and a bit too generic anyway. 
 
 It uses dRuby to do the communication! Which is absolute great. No need for Redis or PostgreSQL, just Ruby standard libraries.
 
 TODO LIST:
 
-- ~~Marshal the job queue into a file so you don't lose all progress~~
-  (Ended up using YAML)
-- ~~Support Rails~~ (Supported!)
-- ~~Parse options from command line, eg. `--workers 10`~~(Done!)
+- Catch SIGTERM and friends
+  - Now supports it, partly. 
+- Don't crash on errors (Partially done)
+- Make it possible to schedule jobs
 - Maybe support ActiveJob?
 - Have a web UI
 - Do some performance testing
 - Add a section telling people to use Sidekiq if they can
 
+DONE
+
+- ~~Marshal the job queue into a file so you don't lose all progress~~
+  (Ended up using YAML)
+- ~~Add a logger~~
+- ~~Support Rails~~ (Supported!)
+- ~~Parse options from command line, eg. `--workers 10`~~(Done!)
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -68,7 +76,8 @@ dummy.queue.push(proc { 2 / 1 })
 dummy.queue.push(DumDum.new)
 ```
 
-Hardworker runs on localhost, port 8788. Should probably make that a value you can change...
+Belated runs on localhost, port 8788. 
+Going to make that an option in the future.
 
 ## Rails
 
@@ -77,14 +86,14 @@ First, start up Belated.
 Then,
 
 ```ruby
-$client = Belated::Client.new
+client = Belated::Client.new
 ```
 
 and you can use the client!
 Call
 
 ```ruby
-$client.perform_belated(job)
+client.perform_belated(job)
 ```
 
 If you want to pass a job to Belated.
@@ -124,7 +133,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/sampokuokkanen/belated. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/[USERNAME]/hardworker/blob/master/CODE_OF_CONDUCT.md).
+Bug reports and pull requests are welcome on GitHub at https://github.com/sampokuokkanen/belated. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/sampokuokkanen/belated/blob/master/CODE_OF_CONDUCT.md).
 
 ## License
 
@@ -132,4 +141,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 ## Code of Conduct
 
-Everyone interacting in the Belated project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/hardworker/blob/master/CODE_OF_CONDUCT.md).
+Everyone interacting in the Belated project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/sampokuokkanen/belated/blob/main/CODE_OF_CONDUCT.md).

data/belated.gemspec

@@ -24,7 +24,7 @@ Gem::Specification.new do |spec|
   # Specify which files should be added to the gem when it is released.
   # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
   spec.files = Dir.chdir(File.expand_path(__dir__)) do
-    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{\A(?:test|spec|features)/}) }
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{\A(?:test|dummy|spec|features)/}) }
   end
   spec.bindir = 'bin'
   spec.executables   = ['belated']

data/lib/belated.rb

@@ -2,12 +2,13 @@
 
 require_relative 'belated/version'
 require_relative 'belated/worker'
+require_relative 'belated/logging'
 require 'drb'
 require 'yaml'
 require 'singleton'
 require 'dry-configurable'
 require 'belated/client'
-require 'belated/rails' if defined?(::Rails::Engine)
+require 'logger'
 
 # Belated is a pure Ruby job backend.
 # It has limited functionality, as it only accepts
@@ -17,8 +18,9 @@ require 'belated/rails' if defined?(::Rails::Engine)
 # and reloads them when started again.
 class Belated
   extend Dry::Configurable
+  include Logging
   include Singleton unless $TESTING
-  URI = "druby://localhost:#{$TESTING ? Array.new(4) { rand(10) }.join : "8788"}"
+  URI = 'druby://localhost:8788'
   FILE_NAME = 'belated_dump'
   @@queue = Queue.new
 
@@ -27,25 +29,48 @@ class Belated
   setting :workers, 1
   setting :connect, true
   setting :environment, 'development'
+  setting :logger, Logger.new($stdout), reader: true
+  setting :log_level, :info, reader: true
 
   # Since it's running as a singleton, we need something to start it up.
   # Aliased for testing purposes.
   # This is only run from the bin file.
   def start
-    boot_app
-    load_jobs
+    boot_app && load_jobs
     @worker_list = []
     Belated.config.workers.times do |_i|
       @worker_list << Thread.new { Worker.new }
     end
     return unless Belated.config.connect
 
-    DRb.start_service(URI, @@queue, verbose: true)
-    puts banner_and_info
+    connect!
+    banner_and_info
+    trap_signals
     DRb.thread.join
   end
   alias initialize start
 
+  # Handles connection to DRb server.
+  def connect!
+    DRb.start_service(URI, @@queue, verbose: true)
+  rescue DRb::DRbConnError, Errno::EADDRINUSE
+    Belated.logger.error 'Could not connect to DRb server.'
+    retry
+  end
+
+  def trap_signals
+    %w[INT TERM].each do |signal|
+      Signal.trap(signal) do
+        @worker_list.length.times do
+          @@queue.push(:shutdown)
+        end
+        Thread.new { stop_workers }
+        sleep 0.1 until @@queue.empty?
+        exit
+      end
+    end
+  end
+
   def boot_app
     return unless rails?
 
@@ -60,6 +85,7 @@ class Belated
   end
 
   def load_jobs
+    log "reloading... if file exists #{File.exist?(Belated::FILE_NAME)}"
     return unless File.exist?(Belated::FILE_NAME)
 
     jobs = YAML.load(File.binread(FILE_NAME))
@@ -70,40 +96,27 @@ class Belated
   end
 
   def reload
+    log 'reloading...'
     load_jobs
   end
 
   def stop_workers
-    @worker_list.each do |worker|
-      Thread.kill(worker)
-    end
-    return if @@queue.empty?
-
-    class_array = []
-
-    @@queue.size.times do |_i|
-      next if (klass_or_proc = @@queue.pop).instance_of?(Proc)
-
-      class_array << klass_or_proc
-    end
-    File.open(FILE_NAME, 'wb') { |f| f.write(YAML.dump(class_array)) }
-  end
-
-  def self.stop_workers
     @worker_list&.each do |worker|
+      sleep 0.1 if worker.alive?
       Thread.kill(worker)
     end
     class_array = []
     @@queue.size.times do |_i|
-      next if (klass_or_proc = @@queue.pop).instance_of?(Proc)
+      next if (klass = @@queue.pop).instance_of?(Proc)
 
-      class_array << klass_or_proc
+      class_array << klass
     end
-    File.open(FILE_NAME, 'wb') { |f| f.write(YAML.dump(class_array)) }
+    pp File.open(FILE_NAME, 'wb') { |f| f.write(YAML.dump(class_array)) }
   end
 
   def banner
     <<-'BANNER'
+
     .----------------. .----------------. .----------------. .----------------. .----------------. .----------------. .----------------.
     | .--------------. | .--------------. | .--------------. | .--------------. | .--------------. | .--------------. | .--------------. |
     | |   ______     | | |  _________   | | |   _____      | | |      __      | | |  _________   | | |  _________   | | |  ________    | |
@@ -119,9 +132,9 @@ class Belated
   end
 
   def banner_and_info
-    puts banner
-    puts "Currently running Belated version #{Belated::VERSION}"
-    puts %(Belated running #{@worker_list&.length.to_i} workers on #{URI}...)
+    log banner
+    log "Currently running Belated version #{Belated::VERSION}"
+    log %(Belated running #{@worker_list&.length.to_i} workers on #{URI}...)
   end
 
   def stats
@@ -131,6 +144,13 @@ class Belated
     }
   end
 
+  def self.kill_and_clear_queue!
+    @worker_list&.each do |worker|
+      Thread.kill(worker)
+    end
+    clear_queue!
+  end
+
   def self.clear_queue!
     @@queue.clear
   end
@@ -145,3 +165,5 @@ class Belated
 
   class Error < StandardError; end
 end
+
+require 'belated/rails' if defined?(::Rails::Engine)

data/lib/belated/client.rb

@@ -4,13 +4,15 @@ class Belated
 
     def initialize
       server_uri = Belated::URI
+      # @bank =
       DRb.start_service
       self.queue = DRbObject.new_with_uri(server_uri)
     end
 
-    def perform_belated(job)
+    def perform(job)
       queue.push(job)
     end
-    alias perform_later perform_belated
+    alias perform_belated perform
+    alias perform_later perform
   end
 end

data/lib/belated/logging.rb

@@ -0,0 +1,17 @@
+class Belated
+  module Logging
+    extend self
+
+    def logger
+      @logger ||= Belated.logger
+    end
+
+    def log(message)
+      logger.__send__(Belated.log_level, message)
+    end
+
+    def logger=(logger)
+      @logger = logger
+    end
+  end
+end

data/lib/belated/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 class Belated
-  VERSION = '0.1.0'
+  VERSION = '0.3.2'
 end

data/lib/belated/worker.rb

@@ -1,27 +1,33 @@
+require_relative 'logging'
 class Belated
   # The worker class that actually gets the jobs from the queue
   # and calls them. Expects the jobs to be procs.
   class Worker
+    include Logging
+
     def initialize
       start_working
     end
 
     def start_working
       loop do
-        job = Belated.fetch_job
-        next unless job
+        next unless (job = Belated.fetch_job)
+
+        break if job == :shutdown
 
-        call_job(job)
-        puts 'fetching jobs...'
+        log call_job(job)
+        log 'fetching jobs...'
       end
     end
 
     def call_job(job)
       if job.respond_to?(:call)
-        pp job.call
+        job.call
       else
-        pp job&.perform
+        job.perform
       end
+    rescue StandardError => e
+      e.inspect
     end
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: belated
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.2
 platform: ruby
 authors:
 - Sampo Kuokkanen
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2021-07-24 00:00:00.000000000 Z
+date: 2021-07-31 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: drb
@@ -78,75 +78,9 @@ files:
 - bin/bundle
 - bin/console
 - bin/setup
-- dummy/Rakefile
-- dummy/app/assets/config/manifest.js
-- dummy/app/assets/images/.keep
-- dummy/app/assets/stylesheets/application.css
-- dummy/app/channels/application_cable/channel.rb
-- dummy/app/channels/application_cable/connection.rb
-- dummy/app/controllers/application_controller.rb
-- dummy/app/controllers/concerns/.keep
-- dummy/app/helpers/application_helper.rb
-- dummy/app/javascript/packs/application.js
-- dummy/app/jobs/application_job.rb
-- dummy/app/mailers/application_mailer.rb
-- dummy/app/models/application_record.rb
-- dummy/app/models/concerns/.keep
-- dummy/app/models/user.rb
-- dummy/app/views/application/index.html.erb
-- dummy/app/views/layouts/application.html.erb
-- dummy/app/views/layouts/mailer.html.erb
-- dummy/app/views/layouts/mailer.text.erb
-- dummy/bin/rails
-- dummy/bin/rake
-- dummy/bin/setup
-- dummy/config.ru
-- dummy/config/application.rb
-- dummy/config/boot.rb
-- dummy/config/cable.yml
-- dummy/config/credentials.yml.enc
-- dummy/config/database.yml
-- dummy/config/environment.rb
-- dummy/config/environments/development.rb
-- dummy/config/environments/production.rb
-- dummy/config/environments/test.rb
-- dummy/config/initializers/application_controller_renderer.rb
-- dummy/config/initializers/backtrace_silencers.rb
-- dummy/config/initializers/content_security_policy.rb
-- dummy/config/initializers/cookies_serializer.rb
-- dummy/config/initializers/filter_parameter_logging.rb
-- dummy/config/initializers/inflections.rb
-- dummy/config/initializers/mime_types.rb
-- dummy/config/initializers/permissions_policy.rb
-- dummy/config/initializers/wrap_parameters.rb
-- dummy/config/locales/en.yml
-- dummy/config/master.key
-- dummy/config/puma.rb
-- dummy/config/routes.rb
-- dummy/config/storage.yml
-- dummy/db/migrate/20210613015406_create_users.rb
-- dummy/db/production.sqlite3
-- dummy/db/schema.rb
-- dummy/db/test.sqlite3
-- dummy/lib/assets/.keep
-- dummy/log/.keep
-- dummy/log/development.log
-- dummy/log/production.log
-- dummy/log/test.log
-- dummy/public/404.html
-- dummy/public/422.html
-- dummy/public/500.html
-- dummy/public/apple-touch-icon-precomposed.png
-- dummy/public/apple-touch-icon.png
-- dummy/public/favicon.ico
-- dummy/storage/.keep
-- dummy/tmp/.keep
-- dummy/tmp/development_secret.txt
-- dummy/tmp/pids/.keep
-- dummy/tmp/storage/.keep
-- hard_worker_dump
 - lib/belated.rb
 - lib/belated/client.rb
+- lib/belated/logging.rb
 - lib/belated/rails.rb
 - lib/belated/version.rb
 - lib/belated/worker.rb