tantot 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 856c9b2b2373579c05f52dd5ab10b2728bf516ec
+  data.tar.gz: 527e2a10f265490b948a73a108a6ac42b6d02d09
+SHA512:
+  metadata.gz: 882fa4b710f08cad5c074fc73f3a9eef94ee3d84fe499347a607e7683826587488aa82b3660f9cf1e877e73c4908dc3158debb5cb1889f69c80ed680677755a2
+  data.tar.gz: c2a4008e85d2b887c6c08889b09145883450e1e31cfd63129d68a94ac1a19738fb440935113ad2b91cba005e9ad74313acde0a89b3d8e32b873c0374562a8299

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.1
+before_install: gem install bundler -v 1.13.3

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at fpbouchard@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in tantot.gemspec
+gemspec
+
+gem 'sidekiq', require: false

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 François-Pierre Bouchard
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,45 @@
+# Tantot
+
+[![Build Status](https://travis-ci.org/petalmd/tantot.svg?branch=master)](https://travis-ci.org/petalmd/tantot)
+
+Tantot (french for _shortly_/_soon_)
+
+Centralize and delay changes to multiple ActiveRecord models to offload processing of complex calculations caused by model mutations.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'tantot'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install tantot
+
+## Usage
+
+Tantot introduces new ActiveRecord callbacks that abstracts most of the usual boilerplate when managing changes, and also allows running callbacks in Sidekiq.
+
+_More documentation to come!_
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/petalmd/tantot. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
+

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "tantot"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/tantot/changes.rb

@@ -0,0 +1,65 @@
+module Tantot
+  module Changes
+
+    class ById
+      include Enumerable
+
+      attr_reader :changes_by_id
+
+      def initialize(changes_by_id)
+        @changes_by_id = changes_by_id
+      end
+
+      delegate :[], :keys, :each, :count, :size, to: :changes_by_id
+
+      def ==(other)
+        other.changes_by_id == @changes_by_id
+      end
+
+      def for_attribute(attribute, compact = true)
+        changes = @changes_by_id.values.collect {|changes_by_attribute| changes_by_attribute[attribute.to_s]}.flatten.uniq
+        compact ? changes.compact : changes
+      end
+
+      def ids
+        @changes_by_id.keys
+      end
+
+      def attributes
+        @changes_by_id.values.collect(&:keys).flatten.uniq.collect(&:to_sym)
+      end
+    end
+
+    class ByModel
+      include Enumerable
+
+      attr_reader :changes_by_model
+
+      def initialize(changes_by_model)
+        @changes_by_model = changes_by_model
+      end
+
+      delegate :==, :keys, :count, :size, to: :changes_by_model
+      alias_method :models, :keys
+
+      def ==(other)
+        other.changes_by_model == @changes_by_model
+      end
+
+      def [](model)
+        for_model(model)
+      end
+
+      def each(&block)
+        @changes_by_model.each do |model, changes|
+          block.call(model, Tantot::Changes::ById.new(changes))
+        end
+      end
+
+      def for_model(model)
+        Tantot::Changes::ById.new(@changes_by_model[model])
+      end
+    end
+
+  end
+end

data/lib/tantot/collector/base.rb

@@ -0,0 +1,6 @@
+module Tantot
+  module Collector
+    class Base
+    end
+  end
+end

data/lib/tantot/collector/block.rb

@@ -0,0 +1,78 @@
+module Tantot
+  module Collector
+    class Block < Base
+      def self.manages?(context)
+        context.key?(:block_id)
+      end
+
+      def initialize
+        @stash = Hash.new do |block_id_hash, block_id|
+          block_id_hash[block_id] = Hash.new do |id_hash, id|
+            id_hash[id] = {}
+          end
+        end
+      end
+
+      def register_watch(context, block)
+        Tantot.registry.watch_config[context[:block_id]] = {context: context, block: block}
+      end
+
+      def push(context, instance, mutations)
+        options = context.fetch(:options, {})
+        formatter = Tantot::Formatter.resolve(options[:format] || Tantot.config.default_watcher_options[:format]).new
+        attribute_hash = @stash[context[:block_id]][instance.id]
+        mutations.each do |attr, changes|
+          attribute_hash[attr] = formatter.push(attribute_hash[attr], context, changes)
+        end
+      end
+
+      def sweep(performer, context = {})
+        @stash.each do |block_id, changes|
+          performer.run({block_id: block_id}, changes)
+        end
+        @stash.clear
+      end
+
+      def perform(context, changes_by_id)
+        watch_config = Tantot.registry.watch_config[context[:block_id]]
+        watch_config[:context][:model].instance_exec(Tantot::Changes::ById.new(changes_by_id), &watch_config[:block])
+      end
+
+      def marshal(context, changes_by_id)
+        [context, changes_by_id]
+      end
+
+      def unmarshal(context, changes_by_id)
+        changes_by_id = changes_by_id.each.with_object({}) do |(id, changes), change_hash|
+          change_hash[id.to_i] = changes
+        end
+        [context, changes_by_id]
+      end
+
+      def debug_block(block)
+        location, line = block.source_location
+        short_path = defined?(Rails) ? Pathname.new(location).relative_path_from(Rails.root).to_s : location
+        "block @ #{short_path}##{line}"
+      end
+
+      def debug_context(context)
+        block = Tantot.registry.watch_config[context[:block_id]][:block]
+        debug_block(block)
+      end
+
+      def debug_state(context)
+        return false if @stash.empty?
+        @stash.collect do |block_id, changes_by_id|
+          watch_config = Tantot.registry.watch_config[block_id]
+          "#{watch_config[:context][:model].name}*#{changes_by_id.size} for #{debug_block(watch_config[:block])}"
+        end.join(" / ")
+      end
+
+      def debug_perform(context, changes_by_id)
+        watch_config = Tantot.registry.watch_config[context[:block_id]]
+        "#{watch_config[:context][:model].name}*#{changes_by_id.size} for #{debug_block(watch_config[:block])}"
+      end
+
+    end
+  end
+end

data/lib/tantot/collector/watcher.rb

@@ -0,0 +1,87 @@
+module Tantot
+  module Collector
+    class Watcher < Base
+      def self.manages?(context)
+        context.key?(:watcher)
+      end
+
+      def initialize
+        @stash = Hash.new do |watcher_hash, watcher|
+          watcher_hash[watcher] = Hash.new do |model_hash, model|
+            model_hash[model] = Hash.new do |id_hash, id|
+              id_hash[id] = {}
+            end
+          end
+        end
+      end
+
+      def register_watch(context, block)
+        # nop
+      end
+
+      def push(context, instance, mutations)
+        watcher = context[:watcher]
+        model = context[:model]
+        formatter = Tantot::Formatter.resolve(watcher.watcher_options[:format]).new
+        attribute_hash = @stash[watcher][model][instance.id]
+        mutations.each do |attr, changes|
+          attribute_hash[attr] = formatter.push(attribute_hash[attr], context, changes)
+        end
+      end
+
+      def sweep(performer, context = {})
+        watcher = context[:watcher]
+        filtered_stash = watcher ? @stash.select {|w, _c| w == watcher} : @stash
+        filtered_stash.each do |w, changes_by_model|
+          performer.run({watcher: w}, changes_by_model)
+        end
+        if watcher
+          @stash.delete(watcher)
+        else
+          @stash.clear
+        end
+      end
+
+      def perform(context, changes_by_model)
+        context[:watcher].new.perform(Tantot::Changes::ByModel.new(changes_by_model))
+      end
+
+      def marshal(context, changes_by_model)
+        changes_by_model = changes_by_model.each.with_object({}) do |(model_class, changes), hash|
+          hash[model_class.name] = changes
+        end
+        [context, changes_by_model]
+      end
+
+      def unmarshal(context, changes_by_model)
+        context[:watcher] = context[:watcher].constantize
+        changes_by_model = changes_by_model.each.with_object({}) do |(model_class_name, changes_by_id), model_hash|
+          model_hash[model_class_name.constantize] = changes_by_id.each.with_object({}) do |(id, changes), change_hash|
+            change_hash[id.to_i] = changes
+          end
+        end
+        [context, changes_by_model]
+      end
+
+      def debug_context(context)
+        context[:watcher].name.demodulize
+      end
+
+      def debug_state(context)
+        return false if @stash.empty?
+        @stash.collect do |watcher, changes_by_model|
+          "#{watcher.name}[#{changes_by_model.collect do |model, changes_by_id|
+            "#{model.name}*#{changes_by_id.size}"
+          end.join("&")}]"
+        end.flatten.join(" / ")
+      end
+
+      def debug_perform(context, changes_by_model)
+        "#{context[:watcher].name}[#{changes_by_model.collect do |model, changes_by_id|
+          "#{model.name}*#{changes_by_id.size}"
+        end.join("&")}]"
+      end
+
+    end
+  end
+end

data/lib/tantot/collector.rb

@@ -0,0 +1,76 @@
+require 'tantot/collector/base'
+require 'tantot/collector/watcher'
+require 'tantot/collector/block'
+
+module Tantot
+  module Collector
+    class Manager
+      def initialize
+        @collectors = {}
+      end
+
+      def register_watch(context, block)
+        resolve!(context).register_watch(context, block)
+      end
+
+      def run(&block)
+        yield
+      ensure
+        sweep
+      end
+
+      def push(context, instance, mutations)
+        collector = resolve!(context)
+        Tantot.logger.debug do
+          mutate = mutations.size.zero? ? 'destroy' : "#{mutations.size} mutations(s)"
+          "[Tantot] [Collecting] [#{collector.class.name.demodulize}] #{mutate} on <#{instance.class.name}:#{instance.id}> for <#{collector.debug_context(context)}>"
+        end
+        collector.push(context, instance, mutations)
+        sweep(context.merge(performer: :inline)) if Tantot.config.console_mode
+      end
+
+      def sweep(context = {})
+        performer = Tantot::Performer.resolve(context[:performer] || Tantot.config.performer).new
+        specific_collector = resolve(context)
+        collectors = specific_collector ? [specific_collector] : @collectors.values
+        collectors.each do |collector|
+          if Tantot.logger.debug? && (debug_state = collector.debug_state(context))
+            Tantot.logger.debug { "[Tantot] [Sweeping] [#{collector.class.name.demodulize}] [#{performer.class.name.demodulize}] #{debug_state}" }
+          end
+          collector.sweep(performer, context)
+        end
+      end
+
+      def perform(context, changes)
+        collector = resolve!(context)
+        Tantot.logger.debug { "[Tantot] [Performing] [#{collector.class.name.demodulize}] #{collector.debug_perform(context, changes)}" }
+        collector.perform(context, changes)
+      end
+
+      def marshal(context, changes)
+        collector = resolve!(context)
+        context, changes = collector.marshal(context, changes)
+        context[:collector_class] = collector.class
+        [context, changes]
+      end
+
+      def unmarshal(context, changes)
+        context.deep_symbolize_keys!
+        collector_class = context[:collector_class].constantize
+        collector = @collectors[collector_class] || @collectors[collector_class] = collector_class.new
+        collector.unmarshal(context, changes)
+      end
+
+      def resolve(context)
+        collector_class = Tantot::Collector::Base.descendants.find {|c| c.manages?(context)}
+        return nil unless collector_class
+        @collectors[collector_class] || @collectors[collector_class] = collector_class.new
+      end
+
+      def resolve!(context)
+        resolve(context) || (raise "No collector manages current context: #{context.inspect}")
+      end
+
+    end
+  end
+end

data/lib/tantot/config.rb

@@ -0,0 +1,17 @@
+module Tantot
+  class Config
+    include Singleton
+
+    attr_accessor :performer, :use_after_commit_callbacks, :default_watcher_options, :console_mode, :chewy_strategy
+
+    def initialize
+      @performer = :inline
+      @use_after_commit_callbacks = true
+      @default_watcher_options = {
+        format: :compact
+      }
+      @console_mode = false
+      @chewy_strategy = :atomic
+    end
+  end
+end

data/lib/tantot/errors.rb

@@ -0,0 +1,4 @@
+module Tantot
+  class UnderivableWatcher < StandardError
+  end
+end

data/lib/tantot/extensions/chewy.rb

@@ -0,0 +1,88 @@
+module Tantot
+  module Extensions
+    module Chewy
+      extend ActiveSupport::Concern
+
+      included do
+        class_attribute :_tantot_chewy_callbacks
+      end
+
+      class_methods do
+        # watch_index 'index#type', attribute, attribute, {method: [:self | :method | ignore and pass a block | ignore and don't pass a block, equivalent of :self]} [block]
+        def watch_index(type_name, *args, &block)
+          options = args.extract_options!
+          watch('tantot/extensions/chewy/chewy', *args)
+          Tantot::Extensions::Chewy.register_watch(self, type_name, options, block)
+        end
+      end
+
+      def self.register_watch(model, type_name, options, block)
+        method = options.delete(:method)
+        model._tantot_chewy_callbacks ||= {}
+        model._tantot_chewy_callbacks[type_name] ||= []
+        model._tantot_chewy_callbacks[type_name].push({method: method, options: options, block: block})
+      end
+
+      class ChewyWatcher
+        include Tantot::Watcher
+
+        def perform(changes_by_model)
+          changes_by_model.each do |model, changes_by_id|
+            model_watches = model._tantot_chewy_callbacks
+            model_watches.each do |type_name, watch_args_array|
+              # Find type
+              reference =
+                if type_name.is_a?(Proc)
+                  if type_name.arity.zero?
+                    instance_exec(&type_name)
+                  else
+                    type_name.call(self)
+                  end
+                else
+                  type_name
+                end
+
+              watch_args_array.each do |watch_args|
+                method = watch_args[:method]
+                options = watch_args[:options]
+                block = watch_args[:block]
+
+                # Find ids to update
+                backreference =
+                  if (method && method.to_sym == :self) || (!method && !block)
+                    # Simply extract keys from changes
+                    changes_by_id.keys
+                  elsif method
+                    # We need to call `method`.
+                    # Try to find it on the class. If so, call it once with all changes.
+                    # There is no API to call per-instance since objects can be already destroyed
+                    # when using the sidekiq performer
+                    model.send(method, changes_by_id)
+                  elsif block
+                    # Since we can be post-destruction of the model, we can't load models here
+                    # Thus, the signature of the block callback is |changes| which are all
+                    # the changes to all the models
+                    model.instance_exec(changes_by_id, &block)
+                  end
+
+                if backreference
+                  backreference.compact!
+
+                  # Make sure there are any backreferences
+                  if backreference.any?
+                    # Call update_index, will follow normal chewy strategy
+                    ::Chewy.strategy Tantot.config.chewy_strategy do
+                      Tantot.logger.debug "[Tantot] [Chewy] [update_index] #{reference} (#{backreference.count} objects): #{backreference.inspect}"
+                      ::Chewy.derive_type(reference).update_index(backreference, options)
+                    end
+                  end
+                end
+
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/tantot/extensions/grape/middleware.rb

@@ -0,0 +1,17 @@
+if defined?(::Grape)
+
+  module Tantot
+    module Extensions
+
+      class GrapeMiddleware < Grape::Middleware::Base
+        def call!(env)
+          Tantot.collector.run do
+            @app_response = super(env)
+          end
+        end
+      end
+
+    end
+  end
+
+end

data/lib/tantot/formatter/compact.rb

@@ -0,0 +1,9 @@
+module Tantot
+  module Formatter
+    class Compact
+      def push(change_array, context, changes)
+        change_array.nil? ? changes : change_array | changes
+      end
+    end
+  end
+end

data/lib/tantot/formatter/detailed.rb

@@ -0,0 +1,9 @@
+module Tantot
+  module Formatter
+    class Detailed
+      def push(change_array, context, changes)
+        change_array.nil? ? [changes] : change_array.push(changes)
+      end
+    end
+  end
+end