fragmentary 0.2.2 → 0.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 2747ef223962cc8eaa11b9d929febf77a5746bca
-  data.tar.gz: 8eb6ca9f3d487102cf23bb8944f46a856230371c
+SHA256:
+  metadata.gz: 0005ce07a2de70b95fc9b105b3ab12a7d763fb2ededa0423845bbfc3c0b45e8d
+  data.tar.gz: e2202f4d38a1189ba956e7b88150178e962581ed93e00f3d74fb8144b193055e
 SHA512:
-  metadata.gz: dcf5246ca4dd67f075f74b7d951e6e69d57dce8269ed49e6896cb92d45e2079500317345e95bb63208d7035e9988f6c03ff653eab2d88c410bfe6de3a9509b99
-  data.tar.gz: f0498b8f900c03dc05d0469fbf5cde755b91b7a941da85cfc9394c5d730cc40ab9f097a4e2e1c38bc73e62700cc0bf83d0360233fb6e5d580dd9984b45b2a527
+  metadata.gz: 1665f0be866246efc7a1ba157d1216f63a39162261a680e6e8542884461c2285137ce350efdd4ed233ee6cb4b88484664ab5b7436a0f84e5fbb80c892ccab7ae
+  data.tar.gz: 0e4d6a8bd99ae54ff6c3715e203cde760382ef9c48a5ee241ae645ce58dd07dcb1d17b819f10771edcf5d5d113016d0d115fd5a08b1218e3060fef0bf57553de

data/CHANGELOG.md

@@ -1,3 +1,8 @@
+### 0.3.0
+- Updates gem to support Rails 5.x (Rails 4.x and earlier are no longer supported due to Rails API changes).
+- Adds support for multiple application instances, allowing pre-release application code to be staged for testing.
+- Fixes a bug in Fragment.set_record_type affecting some application data handlers for fragment classes that are subclassed from another.
+
 ### 0.2.2
 - Removes validation of the fragment's root_id (only relevant to child fragments); ignore v0.2.1
 

data/README.md

@@ -5,11 +5,12 @@ Fragmentary augments the fragment caching capabilities of Ruby on Rails to suppo
 * multiple versions of individual fragments for different groups of users, e.g. admin vs regular users
 * post-cache insertion of user-specific content
 * automatic refreshing of cached content when application data changes, without an external client request
+* multiple application instances running concurrently with shared application data
 
-**Note**: Fragmentary has been extracted from [Persuasive Thinking](http://persuasivethinking.com) where it is currently in active use. See [Integration Issues](https://github.com/MarkMT/fragmentary/blob/master/README.md#integration-issues) for details of issues that should be considered when using it elsewhere.
+Fragmentary has been extracted from [Persuasive Thinking](http://persuasivethinking.com) where it is currently in active use.
 
 ## Background
-In simple cases, Rails' native support for fragment caching assumes that a fragment's content is a representation of a specific application data record. The content is stored in the cache with a key value derived from the `updated_at` attribute of that record. If any attributes of the record change, the cached entry automatically expires and on the next browser request for that content the fragment is re-rendered using the current data. In the view, the `cache` helper is used to specify the record used to determine the key and define the content to be rendered within the fragment, e.g.:
+In simple cases, Rails' native support for fragment caching assumes that a fragment's content is a representation of a specific application data record. The content is stored in the cache with a key value derived from the`id` and `updated_at` attributes of that record. If any attributes of the record change, the cached entry automatically expires and on the next browser request for that content the fragment is re-rendered using the current data. In the view, the `cache` helper is used to specify the record used to determine the key and define the content to be rendered within the fragment, e.g.:
 ```
 <% cache product do %>
   <%= render product %>
@@ -577,7 +578,7 @@ def send_queued_requests
   Fragmentary::RequestQueue.all.each{|q| q.send(:delay => delay += 10.seconds)}
 end
 ```
-The `send` method takes two optional named arguments, `delay` and `between`. If neither are present, all requests held in the queue are sent immediately. If either are present, sending of requests is off-loaded to an asynchronous process using the [Delayed::Job gem](https://github.com/collectiveidea/delayed_job) and scheduled according to the parameters provided: `delay` represents the delay before the queue begins sending requests and `between` represents the interval between individual requests in the queue being sent. In the example above, we choose to delay the sending of requests from each queue by 10 seconds each. You may customize as appropriate.
+The `send` method takes two optional named arguments, `delay` and `between`. If neither are present, all requests held in the queue are sent immediately. If either are present, sending of requests is off-loaded to an asynchronous process using the [Delayed::Job gem](https://github.com/collectiveidea/delayed_job) (i.e. we are not currently using Active Job) and scheduled according to the parameters provided: `delay` represents the delay before the queue begins sending requests and `between` represents the interval between individual requests in the queue being sent. In the example above, we choose to delay the sending of requests from each queue by 10 seconds each. You may customize as appropriate.
 
 #### Queuing Requests Explicitly
 
@@ -758,13 +759,89 @@ Note that if the partial page content being generated contains several nested ch
 
 It is possible to define a fragment without actually storing its content in the cache store. This can be useful, for example if you wish to cache several sibling children within a page but don't need to store the entire root fragment that contains them. Simply include the option `:no_cache => true` in the hash passed to `cache_fragment` or `cache_child`.
 
-## Integration Issues
+### Support for Multiple Application Instances
 
-There are some aspects of this pre-release version of Fragmentary that reflect the application context in which it was originally developed and may need adjustment before deployment elsewhere. Note in particular the following:
-1. Fragmentary was created in the context of a Rails 4.x application (for perfectly sound reasons! :)). There should be only minor adjustment required for use within a Rails 5.x application, but two specific issues we are aware of are the following:
-  - Rails 5.x changes the API for `ActionDispatch::Integration::Session` and now requires that HTTP request parameters be passed as a named parameter `:params`, rather than an unnamed hash in Rails 4.x. This affects the method `to_proc` in class `Fragmentary::Request` and the methods `sign_in` and `sign_out` in class `Fragmentary::UserSession`.
-  - In 'lib/fragmentary/fragment.rb', we set `cache_timestamp_format = :usec` to overcome a timestamp resolution problem when using caching with Postgres under Rails 4.x. We believe that this problem has been solved in Rails 5.x, so this setting will not be necessary. See https://github.com/rails/rails/issues/21815.
-1. Fragmentary uses the [Delayed::Job gem](https://github.com/collectiveidea/delayed_job) to execute background tasks asynchronously. Other alternatives exist within the Rails ecosystem, and in Rails 5.x it will probably make sense to use [Active Job](https://guides.rubyonrails.org/active_job_basics.html) as an abstraction layer.
+In many practical deployment scenarios it is desirable to maintain a live pre-release version of the application online separate from the public-facing production website. This allows new software releases to be staged for either internal or beta testing prior to final deployment to the production environment. For example, if the public-facing application is accessed at a root URL of http://myapp.com/, a separate pre-release version might be deployed say to http://prerelease.myapp.com/ or http://myapp.com/prerelease/.
+
+In the specific scenario in which Fragmentary was developed, it was important for the pre-release version of the application to share the same application database as the production site, i.e. both production and pre-release versions render exactly the same data, and any changes to application data initiated by a user of one version of the application will be reflected in the content seen by a user of the other version as well. For caching, this leads to some additional challenges which we discuss below.
+
+#### Storing Multiple Versions of Content in the Cache
+
+There is an important difference between the content rendered by each instance of an application: any HTML links to other pages on the site must be to URLs representing the particular version being rendered. For example, on a 'products' index page, a link to an individual product page on the production website might look like http://myapp.com/products/123, while the same link on the index page of the pre-release version might be http://myapp.com/prerelease/products/123.
+
+In general, as long as links contained in the view are created using Rails' standard path or url helpers, e.g. `product_path(@product)`  etc, they will automatically be based on the root URL of the application instance (production or pre-release) in which they are generated. However, in the context of caching, the fact that differences exist in the content generated by different application instances implies that for each fragment defined in the view by calling `cache_fragment` or `cache_child`, different versions of fragment content need to be stored in the cache. Also, each of these distinct versions needs to be associated with a unique record in the `fragments` database table identifying which application instance generated that content.
+
+In Fragmentary we can accomplish this simply by adding a column to the `fragments` table to store the root URL of the particular application instance that created the fragment content.
+
+```
+class AddAppUrlToFragment < ActiveRecord::Migration
+  def change
+    change_table :fragments do |t|
+      t.string :application_root_url
+    end
+  end
+end
+```
+
+The column name `application_root_url` shown above is the default assumed by Fragmentary. You can use a different column name if you wish, as long as you tell Fragmentary in initializers/fragmentary.rb, e.g.:
+
+```
+Fragmentary.setup do |config|
+  ...
+  config.application_root_url_column = 'app_root'
+end
+
+```
+
+Once this column has been added to the `fragments` table, any fragment records subsequently created by calls to `cache_fragment` or `cache_child` will have the column populated automatically based on the particular application instance in which the code is executed, and the content stored in the cache for each fragment record will be that generated by the corresponding instance. As long as any links rendered within the content are generated using Rails' path or url helpers, the cached content will be rendered correctly both when initially created and when retrieved subsequently.
+
+Note that it is possible to store cached content for each of the different application instances in different places. Simply set `config.cache_store` in config/environments/production.rb (or alternative environment file) as required.
+
+#### Automatically Refreshing Multiple Versions of Cached Content
+
+The fact that two versions of content exist in the cache for each fragment means that whenever a change in application data occurs that triggers the generation of an internal application request to refresh a piece of cached content (i.e. for any fragment that has a `request_path` method defined), _both_ (in general all) versions need to be refreshed. So for example, a change to application data caused by a user action on the production website needs to trigger internal requests to _both_ the production _and_ the pre-release instances in order to refresh their respective content. The converse is true for changes initiated from the pre-release website.
+
+(Note: for better or worse, we've stuck with the term 'internal request' to mean any request delivered to the application programmatically in order to refresh cached content. This includes requests created by one application instance that are intended to be processed by another instance.)
+
+Our approach to sending requests between application instances relies on requests being processed asynchronously (i.e. by passing a `delay` value to RequestQueue#send in the controller method `send_queued_requests` discussed earlier). The reason is that asynchronous tasks used to process internal requests can be directed to specific task queues associated with the application instance they are intended to be processed by. Each application instance has an associated asynchronous task process. By configuring that process to run tasks from just the queue(s) designated for it, we ensure that any application instance can direct requests to any other.
+
+As noted earlier, our current implementation relies on [Delayed::Job](https://github.com/collectiveidea/delayed_job) for creating and processing asynchronous tasks. Queued tasks are stored in a database table, so as long as each application and [Delayed::Job](https://github.com/collectiveidea/delayed_job) instance have access to the same database, this approach will be successful.
+
+To configure Fragmentary to automatically refresh cached content for multiple instances, first set `remote_urls` in `Fragmentary.config`. This is an array of root URLs for all _other_ instances of the application that requests should be sent to. For example, in order to allow requests to be sent from the production instance to the pre-release instance, in initializers/fragmentary.rb in the production code we would add the following configuration:
+
+```
+Fragmentary.setup do |config|
+  ...
+  config.remote_urls << 'http://myapp.com/prerelease/'
+end
+```
+
+Our current approach to application deployment is to maintain different branches in our source repository for each application instance. This allows us to keep custom configurations like `config.remote_urls` above for each instance on their own branches. So in contrast to the case above, to allow requests to be sent from the pre-release instance to the production instance, in initializers/fragmentary.rb on the pre-release branch the `remote_urls` array would be set to `['http://myapp.com/']` .
+
+As an alternative, it may be possible to create a separate environment for the pre-release deployment in your config/environments directory and thus maintain the configuration for all application instances in a single repository branch. We have not investigated this approach.
+
+By specifying `remote_urls` as described above, for each internal application request Fragmentary will automatically create tasks to process the request not only on the application instance where it was created but also on all other instances corresponding to the elements in `remote_urls`. Fragmentary sends these tasks to [Delayed::Job](https://github.com/collectiveidea/delayed_job) queues that have names formed from the domain name and path of both the root URL of the current instance and the values in `remote_urls`. So in the example, a request to '/products/123', created say by editing a product name on the production website, would be sent to [Delayed::Job](https://github.com/collectiveidea/delayed_job) queues with names 'myapp.com' and 'myapp.com/prerelease/'.
+
+The final step needed to allow internal requests to be processed by the instance they are intended for is to configure the [Delayed::Job](https://github.com/collectiveidea/delayed_job) process associated with each instance to process tasks from the correct queue.
+
+In our case we use Capistrano 2 for deployment and use `delayed_job_args` in config/deploy.rb to configure queue names (see documentation [here](https://github.com/collectiveidea/delayed_job/blob/master/lib/delayed/recipes.rb)):
+
+```
+queue_prefix = "myapp.com/prerelease/"
+set :delayed_job_args, "--queue=#{queue_prefix},#{queue_prefix}_overnight"
+after "deploy:stop",    "delayed_job:stop"
+after "deploy:start",   "delayed_job:start"
+after "deploy:restart", "delayed_job:restart"
+```
+
+Again, this configuration will be different for each instance and so in our case each one will be stored in different source repository branches.
+
+Note that if you configure [Delayed::Job](https://github.com/collectiveidea/delayed_job) to work only from specific queues, you'll need to make sure that _any_ asynchronous tasks created by your application are submitted to one of those queues.
+
+## Dependencies
+
+  - The current implementation of Fragmentary has been tested using Rails 5. It does not work with earlier versions of Rails due to a change in the API for Rails `ActionDispatch::Integration::Session` class. We do have a 'Rails.4.2' branch that uses the older API in the github repository. However this branch is no longer maintained.
+  - As noted, Fragmentary uses the [Delayed::Job](https://github.com/collectiveidea/delayed_job) gem to execute background tasks asynchronously.
 
 ## Contributing
 

data/fragmentary.gemspec

@@ -22,9 +22,11 @@ Gem::Specification.new do |spec|
   spec.bindir        = "exe"
   spec.require_paths = ["lib"]
 
-  spec.add_runtime_dependency "rails", ">= 4.0.0", "< 5"
+  spec.add_runtime_dependency "rails", "~> 5.0"
   spec.add_runtime_dependency "delayed_job_active_record", "~> 4.1"
   spec.add_runtime_dependency "wisper-activerecord", "~> 1.0"
+  spec.add_runtime_dependency "http", "~> 3.0.0"
+  spec.add_runtime_dependency "nokogiri"
   spec.add_development_dependency "bundler", "~> 1.17"
   spec.add_development_dependency "rake", "~> 10.0"
   spec.add_development_dependency "rspec", "~> 3.0"

data/lib/fragmentary/config.rb

@@ -2,12 +2,14 @@ module Fragmentary
 
   class Config
     include Singleton
-    attr_accessor :current_user_method, :get_sign_in_path, :post_sign_in_path,
-                  :sign_out_path, :users, :default_user_type_mapping, :session_users
+    attr_accessor :current_user_method, :get_sign_in_path, :post_sign_in_path, :sign_out_path,
+                  :users, :default_user_type_mapping, :session_users, :application_root_url_column, :remote_urls
 
     def initialize
       # default
       @current_user_method = :current_user
+      @application_root_url_column = :application_root_url
+      @remote_urls = []
     end
 
     def session_users=(session_users)
@@ -15,34 +17,47 @@ module Fragmentary
       Fragmentary.parse_session_users(session_users)
       @session_users = session_users
     end
+
+    def application_root_url_column=(column_name)
+      @application_root_url_column = column_name.to_sym
+    end
   end
 
   def self.current_user_method
     self.config.current_user_method
   end
 
-  # Parse a class-specific set of session_user options
-  # session_users can be an array of session_user keys, a hash of session_user definitions or an array
-  # containing a mixture of both. The method should return an array of keys. If session_users is an
-  # array, elements representing existing SessionUser objects should be included in the returned array.
+  # Parse a set of session_user options, creating session_users where needed, and return a set of user_type keys.
+  # session_users may take several forms:
+  #   (1) a hash whose keys are user_type strings and whose values have the form {:credentials => credentials},
+  #       where 'credentials' is either a hash of parameters to be submitted when logging in or a proc that
+  #       returns those parameters.
+  #   (2) an array of hashes as described in (1) above.
+  #   (3) an array of user_type strings corresponding to SessionUser objects already defined.
+  #   (4) an array containing a mixture of user_type strings and hashes as described in (1) above.
   # Non-hash elements that don't represent existing SessionUser objects should raise an exception. Array
-  # elements that are hashes should be parsed to create new SessionUser objects. Raise an exception if
+  # elements that are hashes should be parsed to create new SessionUser objects. Raise an exception on
   # any attempt to redefine an existing user_type.
   def self.parse_session_users(session_users = nil)
     return nil unless session_users
     if session_users.is_a?(Array)
-      # Fun fact: can't use 'each_with_object' here because 'acc += parse_session_users(v)' assigns a
-      # different object to 'acc', while 'each_with_object' passes the *same* object to the block on
-      # each iteration.
+      # Fun fact: can't use 'each_with_object' here because 'acc += parse_session_users(v)' would assign
+      # a different object to 'acc' on each iteration, while 'each_with_object' passes the *same* object
+      # to the block on each iteration.
       session_users.inject([]) do |acc, v|
         if v.is_a?(Hash)
           acc + parse_session_users(v)
         else
+          # v is a user_type, e.g. :admin
+          raise "No SessionUser exists for user_type '#{v}'" unless SessionUser.fetch(v)
           acc << v
         end
       end
     elsif session_users.is_a?(Hash)
       session_users.each_with_object([]) do |(k,v), acc|
+        # k is the user_type, v is an options hash that typically looks like  {:credentials => login_credentials} where
+        # login_credentials is either a hash of parameters to be submitted at login or a proc that returns those parameters.
+        # In the latter case, the proc is executed when we actually log in to create a new session for the specified user.
         acc << k if user = SessionUser.new(k,v)
       end
     end

data/lib/fragmentary/fragment.rb

@@ -24,8 +24,6 @@ module Fragmentary
         # redundant duplicate request.
         after_commit :touch_parent, :on => [:update, :destroy]
 
-        attr_accessible :parent_id, :root_id, :record_id, :user_id, :user_type, :key
-
         attr_accessor :indexed_children
 
         # Set cache timestamp format to :usec instead of :nsec because the latter is greater precision than Postgres supports,
@@ -73,8 +71,14 @@ module Fragmentary
         # Collect the attributes to be used when searching for an existing fragment. Fragments are unique by these values.
         search_attributes = {}
 
-        parent_id = options.delete(:parent_id)
-        search_attributes.merge!(:parent_id => parent_id) if parent_id
+        if (parent_id = options.delete(:parent_id))
+          search_attributes.merge!(:parent_id => parent_id)
+        else
+          application_root_url_column = Fragmentary.config.application_root_url_column
+          if (application_root_url = options.delete(application_root_url_column)) && column_names.include?(application_root_url_column.to_s)
+            search_attributes.merge!(application_root_url_column => application_root_url)
+          end
+        end
 
         [:record_id, :user_id, :user_type, :key].each do |attribute_name|
           if klass.needs?(attribute_name)
@@ -117,20 +121,53 @@ module Fragmentary
         self
       end
 
+      # There is one queue per user_type per application instance (the current app and any external instances). The queues
+      # for all fragments are held in common by the Fragment base class here in @@request_queues but are also indexed on a
+      # subclass basis by an individual subclass's user_types (see the inherited hook below). As well as being accessible
+      # here as Fragment.request_queues, the queues are also available without indexation as RequestQueue.all.
       def request_queues
-        @@request_queues ||= Hash.new do |hash, user_type|
-          hash[user_type] = RequestQueue.new(user_type)
+        @@request_queues ||= Hash.new do |hsh, host_url|
+          # As well as acting as a hash key to index the set of request queues for a given target application instance
+          # (for which its uniqueness is the only requirement), host_url is also passed to the RequestQueue constructor,
+          # from which it is used:
+          #   (i) by the RequestQueue::Sender to derive the name of the delayed_job queue that will be used to process the
+          #       queued requests if the sender is invoked in asynchronous mode - see RequestQueue::Sender#schedulerequests.
+          #   (ii) by the Fragmentary::InternalUserSession instantiated by the Sender to configure the session_host.
+          hsh[host_url] = Hash.new do |hsh2, user_type|
+            hsh2[user_type] = RequestQueue.new(user_type, host_url)
+          end
         end
-        if self == base_class
-          @@request_queues
-        else
-          return nil unless (requestable? or new.requestable?)
-          user_types.each_with_object({}){|user_type, queues| queues[user_type] = @@request_queues[user_type]}
+      end
+
+      # Subclass-specific request_queues
+      def inherited(subclass)
+        subclass.instance_eval do
+
+          def request_queues
+            super  # ensure that @@request_queues has been defined
+            @request_queues ||= begin
+              app_root_url = Rails.application.routes.url_helpers.root_url
+              remote_urls = Fragmentary.config.remote_urls
+              user_types.each_with_object( Hash.new {|hsh0, url| hsh0[url] = {}} ) do |user_type, hsh|
+                # Internal request queues
+                hsh[app_root_url][user_type] = @@request_queues[app_root_url][user_type]
+                # External request queues
+                if remote_urls.any?
+                  unless Rails.application.routes.default_url_options[:host]
+                    raise "Can't create external request queues without setting Rails.application.routes.default_url_options[:host]"
+                  end
+                  remote_urls.each {|remote_url| hsh[remote_url][user_type] = @@request_queues[remote_url][user_type]}
+                end
+              end
+            end
+          end
+
         end
+        super
       end
 
       def remove_queued_request(user:, request_path:)
-        request_queues[user_type(user)].remove_path(request_path)
+        request_queues.each{|key, hsh| hsh[user_type(user)].remove_path(request_path)}
       end
 
       def subscriber
@@ -158,6 +195,11 @@ module Fragmentary
       # signed in. When the fragment is instantiated using FragmentsHelper methods 'cache_fragment' or 'CacheBuilder.cache_child',
       # a :user option is added to the options hash automatically from the value of 'current_user'. The user_type is extracted
       # from this option in Fragment.attributes.
+      #
+      # For each class that declares 'needs_user_type', a set of user_types is defined that determines the set of request_queues
+      # that will be used to send requests to the application when a fragment is touched. By default these user_types are defined
+      # globally using 'Fragmentary.setup' but they can alternatively be set on a class-specific basis by passing a :session_users
+      # option to 'needs_user_type'. See 'Fragmentary.parse_session_users' for details.
       def needs_user_type(options = {})
         self.extend NeedsUserType
         instance_eval do
@@ -218,28 +260,25 @@ module Fragmentary
             # by the fragment.
             class << record_type_subscription
               set_callback :after_destroy, :after, ->{subscriber.client.remove_fragments_for_record(record.id)}
+              set_callback :after_create, :after, ->{subscriber.client.try_request_for_record(record.id)}
             end
           end
 
-          if requestable?
-            record_class = record_type.constantize
-            instance_eval <<-HEREDOC
-              subscribe_to #{record_class} do
-                def create_#{record_class.model_name.param_key}_successful(record)
-                  request = Fragmentary::Request.new(request_method, request_path(record.id),
-                                                     request_parameters(record.id), request_options)
-                  queue_request(request)
-                end
-              end
-            HEREDOC
-          end
-
+          self.extend RecordClassMethods
           define_method(:record){record_type.constantize.find(record_id)}
         end
       end
 
-      def remove_fragments_for_record(record_id)
-        where(:record_id => record_id).each(&:destroy)
+      module RecordClassMethods
+        def remove_fragments_for_record(record_id)
+          where(:record_id => record_id).each(&:destroy)
+        end
+
+        def try_request_for_record(record_id)
+          if requestable?
+            queue_request(request(record_id))
+          end
+        end
       end
 
       def needs_record_id?
@@ -288,9 +327,7 @@ module Fragmentary
       end
 
       def queue_request(request=nil)
-        if request
-          request_queues.each{|key, queue| queue << request}
-        end
+        request_queues.each{|key, hsh| hsh.each{|key2, queue| queue << request}} if request
       end
 
       def requestable?
@@ -308,7 +345,11 @@ module Fragmentary
 
       # The instance method 'request_options' is defined in terms of this.
       def request_options
-        nil
+        {}
+      end
+
+      def request
+        raise NotImplementedError
       end
 
       # This method defines the handler for the creation of new list items. The method takes:
@@ -453,7 +494,7 @@ module Fragmentary
     end
 
     def touch(*args, no_request: false)
-      request_queues.each{|key, queue| queue << request} if request && !no_request
+      request_queues.each{|key, hsh| hsh.each{|key2, queue| queue << request}} if request && !no_request
       super(*args)
     end
 

data/lib/fragmentary/fragments_helper.rb

@@ -3,6 +3,7 @@ module Fragmentary
   module FragmentsHelper
 
     def cache_fragment(options, &block)
+      options.reverse_merge!(Fragmentary.config.application_root_url_column => self.root_url.gsub(%r{https?://}, ''))
       CacheBuilder.new(self).cache_fragment(options, &block)
     end
 
@@ -10,6 +11,7 @@ module Fragmentary
       # the template option is deprecated but avoids breaking prior usage
       template = options.delete(:template) || self
       options.reverse_merge!(:user => Template.new(template).current_user)
+      options.reverse_merge!(Fragmentary.config.application_root_url_column => self.root_url.gsub(%r{https?://}, ''))
       CacheBuilder.new(template, Fragmentary::Fragment.base_class.existing(options))
     end
 

data/lib/fragmentary/publisher.rb

@@ -33,7 +33,9 @@ module Fragmentary
       end
 
       def after_update_broadcast
+        Rails.logger.info "\n***** #{start = Time.now} broadcasting :after_update from #{self.class.name} #{self.id}\n"
         broadcast(:after_update, self) if self.previous_changes.any?
+        Rails.logger.info "\n***** #{Time.now} broadcast :after_update from #{self.class.name} #{self.id} took #{(Time.now - start) * 1000} ms\n"
       end
 
       def after_destroy_broadcast

data/lib/fragmentary/request.rb

@@ -3,28 +3,13 @@ module Fragmentary
   class Request
     attr_reader :method, :path, :options, :parameters
 
-    def initialize(method, path, parameters=nil, options=nil)
+    def initialize(method, path, parameters=nil, options={})
       @method, @path, @parameters, @options = method, path, parameters, options
     end
 
     def ==(other)
       method == other.method and path == other.path and parameters == other.parameters and options == other.options
     end
-
-    def to_proc
-      method = @method; path = @path; parameters = @parameters; options = @options.try :dup
-      if @options.try(:[], :xhr)
-        Proc.new do
-          puts "      * Sending xhr request '#{method.to_s} #{path}'" + (!parameters.nil? ? " with #{parameters.inspect}" : "")
-          send(:xhr, method, path, parameters, options)
-        end
-      else
-        Proc.new do
-          puts "      * Sending request '#{method.to_s} #{path}'" + (!parameters.nil? ? " with #{parameters.inspect}" : "")
-          send(method, path, parameters, options)
-        end
-      end
-    end
   end
 
 end

data/lib/fragmentary/request_queue.rb

@@ -1,5 +1,3 @@
-require 'fragmentary/user_session'
-
 module Fragmentary
 
   class RequestQueue
@@ -8,12 +6,15 @@ module Fragmentary
       @@all ||= []
     end
 
-    attr_reader :requests, :user_type, :sender
+    attr_reader :requests, :user_type, :host_root_url
 
-    def initialize(user_type)
+    def initialize(user_type, host_root_url)
       @user_type = user_type
+      # host_root_url represents where the queued *requests* are to be processed. For internal sessions it also represents where
+      # the *queue* will be processed by delayed_job. For external requests, the queue will be processed by the host creating the
+      # queue and the requests will be explicitly sent to the host_root_url.
+      @host_root_url = host_root_url
       @requests = []
-      @sender = Sender.new(self)
       self.class.all << self
     end
 
@@ -40,6 +41,10 @@ module Fragmentary
       requests.delete_if{|r| r.path == path}
     end
 
+    def sender
+      @sender ||= Sender.new(self)
+    end
+
     def send(**args)
       sender.start(args)
     end
@@ -49,16 +54,39 @@ module Fragmentary
     end
 
     class Sender
+
       class << self
         def jobs
           ::Delayed::Job.where("(handler LIKE ?) OR (handler LIKE ?)", "--- !ruby/object:#{name} %", "--- !ruby/object:#{name}\n%")
         end
       end
 
+      class Target
+
+        attr_reader :url
+
+        def initialize(url)
+          @url = url
+        end
+
+        def queue_name
+          @url.gsub(%r{https?://}, '')
+        end
+      end
+
       attr_reader :queue
 
       def initialize(queue)
         @queue = queue
+        @target = Target.new(queue.host_root_url)
+      end
+
+      def session_user
+        @session_user ||= Fragmentary::SessionUser.fetch(queue.user_type)
+      end
+
+      def session
+        @session ||= InternalUserSession.new(@target.url, session_user)
       end
 
       # Send all requests, either directly or by schedule
@@ -80,22 +108,19 @@ module Fragmentary
         @between ? send_next_request : send_all_requests
       end
 
+      def send_next_request
+        if queue.size > 0
+          request = queue.next_request
+          session.send_request(:method => request.method, :path => request.path, :parameters => request.parameters, :options => request.options)
+        end
+      end
+
       def success
         schedule_requests(@between) if queue.size > 0
       end
 
       private
 
-      def next_request
-        queue.next_request.to_proc
-      end
-
-      def send_next_request
-        if queue.size > 0
-          session.instance_exec(&(next_request))
-        end
-      end
-
       def send_all_requests
         while queue.size > 0
           send_next_request
@@ -107,20 +132,11 @@ module Fragmentary
           clear_session
           Delayed::Job.transaction do
             self.class.jobs.destroy_all
-            Delayed::Job.enqueue self, :run_at => delay.from_now
+            Delayed::Job.enqueue self, :run_at => delay.from_now, :queue => @target.queue_name
           end
         end
       end
 
-      def session
-        @session ||= new_session
-      end
-
-      def new_session
-        session_user = Fragmentary::SessionUser.fetch(queue.user_type)
-        UserSession.new(session_user)
-      end
-
       def clear_session
         @session = nil
       end
@@ -128,4 +144,5 @@ module Fragmentary
     end
 
   end
+
 end

data/lib/fragmentary/session_user.rb

@@ -0,0 +1,38 @@
+module Fragmentary
+
+  class SessionUser
+
+    def self.all
+      @@all ||= Hash.new
+    end
+
+    def self.fetch(key)
+      all[key]
+    end
+
+    def initialize(user_type, options={})
+      if user = self.class.fetch(user_type)
+        if user.options != options
+          raise RangeError, "You can't redefine an existing SessionUser object: #{user_type.inspect}"
+        else
+          user
+        end
+      else
+        @user_type = user_type
+        @options = options
+        self.class.all.merge!({user_type => self})
+      end
+    end
+
+    def credentials
+      options[:credentials]
+    end
+
+    protected
+    def options
+      @options
+    end
+
+  end
+
+end

data/lib/fragmentary/subscription.rb

@@ -44,7 +44,7 @@ module Fragmentary
     end
 
     include ActiveSupport::Callbacks
-    define_callbacks :after_destroy
+    define_callbacks :after_create, :after_destroy
 
     attr_reader :subscriber
     attr_accessor :record
@@ -55,7 +55,10 @@ module Fragmentary
     end
 
     def after_create(record)
-      call_method(:"create_#{record.class.model_name.param_key}_successful", record)
+      run_callbacks :after_create do
+        @record = record
+        call_method(:"create_#{record.class.model_name.param_key}_successful", record)
+      end
     end
 
     def after_update(record)

data/lib/fragmentary/user_session.rb

@@ -1,38 +1,66 @@
 require 'rails/console/app'
+require 'http'
+require 'nokogiri'
 
 module Fragmentary
 
-  class UserSession
+  class InternalUserSession
 
     include Rails::ConsoleMethods
 
-    def initialize(user, &block)
+    def initialize(target, user=nil, &block)
       # app is from Rails::ConsoleMethods. It returns an object ActionDispatch::Integration::Session.new(Rails.application)
       # with some extensions. See https://github.com/rails/rails/blob/master/railties/lib/rails/console/app.rb
       # The session object has instance methods get, post etc.
       # See https://github.com/rails/rails/blob/master/actionpack/lib/action_dispatch/testing/integration.rb
       @session = app
-      sign_in if @credentials = session_credentials(user)
+      @user = user
+      @target = URI.parse(target)
+      @session.host! session_host
+      sign_in if session_credentials
       instance_eval(&block) if block_given?
     end
 
-    def session_credentials(user)
-      credentials = user.try(:credentials)
-      credentials.is_a?(Proc) ? credentials.call : credentials
+    def session_host
+      @session_host ||= @target.host + (port=@target.port ? ":#{port}" : "")
     end
 
-    def method_missing(method, *args)
-      @session.send(method, *args)
+    def session_sign_in_path
+      @sign_in_path ||= Fragmentary.config.get_sign_in_path
     end
 
-    def sign_out
-      post Fragmentary.config.sign_out_path, {:_method => 'delete', :authenticity_token => request.session[:_csrf_token]}
+    def session_sign_out_path
+      @sign_out_path ||= Fragmentary.config.sign_out_path
+    end
+
+    def session_credentials
+      return nil unless @user
+      @credentials ||= begin
+        credentials = @user.credentials
+        credentials.is_a?(Proc) ? credentials.call : credentials
+      end
+    end
+
+    def relative_url_root
+      @relative_url_root ||= Rails.application.config.relative_url_root
+    end
+
+    def session_options
+      @session_options ||= relative_url_root ? {:env => {'SCRIPT_NAME' => relative_url_root}} : {}
+    end
+
+    def method_missing(method, *args)
+      @session.send(method, *args)
     end
 
     def sign_in
-      get Fragmentary.config.get_sign_in_path  # necessary in order to get the csrf token
+      raise "Can't sign in without user credentials" unless session_credentials
+      send_request(:method => :get, :path => session_sign_in_path, :options => session_options)  # necessary in order to get the csrf token
       # NOTE: In Rails 5, params is changed to a named argument, i.e. :params => {...}. Will need to be changed.
-      post Fragmentary.config.post_sign_in_path, @credentials.merge(:authenticity_token => request.session[:_csrf_token])
+      # Note that request is called on session, returning an ActionDispatch::Request; request.session is an ActionDispatch::Request::Session
+      puts "      * Signing in as #{session_credentials.inspect}"
+      parameters = session_credentials.merge(:authenticity_token => request.session[:_csrf_token])
+      send_request(:method => :post, :path => session_sign_in_path, :parameters => parameters, :options => session_options)
       if @session.redirect?
         follow_redirect!
       else
@@ -40,39 +68,87 @@ module Fragmentary
       end
     end
 
-  end
+    def follow_redirect!
+      raise "not a redirect! #{status} #{status_message}" unless redirect?
+      if (url = response.location) =~ %r{://}
+        destination = URI.parse(url)
+        path = destination.query ? "#{destination.path}?#{destination.query}" : destination.path
+      end
+      path = relative_url_root ? path.gsub(Regexp.new("^#{relative_url_root}"), "") : path
+      send_request(:method => :get, :path => path, :options => session_options)
+      status
+    end
 
-  class SessionUser
+    def send_request(method:, path:, parameters: nil, options: {})
+      options.merge!({:params => parameters})
+      options.merge!(session_options)
+      if options.try(:[], :xhr)
+        Rails.logger.info "      * Sending xhr request '#{method.to_s} #{path}'" + (!parameters.nil? ? " with #{parameters.inspect}" : "")
+      else
+        Rails.logger.info "      * Sending request '#{method.to_s} #{path}'" + (!parameters.nil? ? " with #{parameters.inspect}" : "")
+      end
+      @session.send(method, path, options)
+    end
 
-    def self.all
-      @@all ||= Hash.new
+    def sign_out
+      # request is called on session, returning an ActionDispatch::Request; request.session is an ActionDispatch::Request::Session
+      parameters = {:_method => 'delete', :authenticity_token => request.session[:_csrf_token]}
+      send_request(:method => :post, :path => session_sign_out_path, :parameters => parameters, :options => session_options)
     end
 
-    def self.fetch(key)
-      all[key]
+  end
+
+  class ExternalUserSession
+
+    def initialize(target, user=nil)
+      @target = URI.parse(target)
+      @relative_url_root = @target.path
+      @session = HTTP.persistent(target)
+      @cookie = nil
+      @authenticity_token = nil
+      sign_in if @credentials = session_credentials(user)
     end
 
-    def initialize(user_type, options={})
-      if user = self.class.fetch(user_type)
-        if user.options != options
-          raise RangeError, "You can't redefine an existing SessionUser object: #{user_type.inspect}"
-        else
-          user
-        end
+    def send_request(method:, path:, parameters: nil, options: {})
+      if options.try(:[], :xhr)
+        Rails.logger.info "      * Sending xhr request '#{method.to_s} #{path}'" + (!parameters.nil? ? " with #{parameters.inspect}" : "")
       else
-        @user_type = user_type
-        @options = options
-        self.class.all.merge!({user_type => self})
+        Rails.logger.info "      * Sending request '#{method.to_s} #{path}'" + (!parameters.nil? ? " with #{parameters.inspect}" : "")
       end
+      unless path =~ %r{://}
+        path = @relative_url_root + path
+      end
+      cookies = @cookie ? {@cookie.name.to_sym => @cookie.value} : {}
+      headers = options.try(:delete, :headers) || {}
+      headers.merge!({:'X-Requested-With' => 'XMLHttpRequest'}) if options.try(:delete, :xhr)
+      response = @session.cookies(cookies).headers(headers).send(method, path, {:json => parameters})
+      @cookie = response.cookies.first
+      @authenticity_token = Nokogiri::HTML.parse(response.to_s).css('head meta[name="csrf-token"]').first.try(:[], 'content')
+      if (response.code >=300) && (response.code <=399)
+        location = response.headers[:location]
+        options = {:headers => {:accept => "text/html,application/xhtml+xml,application/xml"}}
+        response = send_request(:method => :get, :path => location, :parameters => nil, :options => options)
+      end
+      response
     end
 
-    def credentials
-      options[:credentials]
+    def session_credentials(user)
+      credentials = user.try(:credentials)
+      credentials.is_a?(Proc) ? credentials.call : credentials
     end
 
-    protected
-    def options
-      @options
+    def sign_in
+      # The first request retrieves the authentication token
+      response = send_request(:method => :get, :path => Fragmentary.config.get_sign_in_path)
+      puts "      * Signing in as #{@credentials.inspect}"
+      response = send_request(:method => :post, :path => Fragmentary.config.post_sign_in_path,
+                              :parameters => @credentials.merge(:authenticity_token => @authenticity_token),
+                              :options => {:headers => {:accept => "text/html,application/xhtml+xml,application/xml"}})
+    end
+
+    def sign_out
+      send_request(:method => :delete, :path => Fragmentary.config.sign_out_path, :parameters => {:authenticity_token => @authenticity_token})
+      @session.close
     end
   end
 

data/lib/fragmentary/version.rb

@@ -1,3 +1,3 @@
 module Fragmentary
-  VERSION = "0.2.2"
+  VERSION = "0.3"
 end

data/lib/fragmentary.rb

@@ -7,6 +7,7 @@ require 'fragmentary/request'
 require 'fragmentary/fragment'
 require 'fragmentary/handler'
 require 'fragmentary/user_session'
+require 'fragmentary/session_user'
 require 'fragmentary/widget_parser'
 require 'fragmentary/widget'
 require 'fragmentary/publisher'

metadata

@@ -1,35 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: fragmentary
 version: !ruby/object:Gem::Version
-  version: 0.2.2
+  version: '0.3'
 platform: ruby
 authors:
 - Mark Thomson
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-07-02 00:00:00.000000000 Z
+date: 2022-01-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 4.0.0
-    - - "<"
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '5'
+        version: '5.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 4.0.0
-    - - "<"
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '5'
+        version: '5.0'
 - !ruby/object:Gem::Dependency
   name: delayed_job_active_record
   requirement: !ruby/object:Gem::Requirement
@@ -58,6 +52,34 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: http
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 3.0.0
+- !ruby/object:Gem::Dependency
+  name: nokogiri
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -127,6 +149,7 @@ files:
 - lib/fragmentary/publisher.rb
 - lib/fragmentary/request.rb
 - lib/fragmentary/request_queue.rb
+- lib/fragmentary/session_user.rb
 - lib/fragmentary/subscriber.rb
 - lib/fragmentary/subscription.rb
 - lib/fragmentary/user_session.rb
@@ -152,8 +175,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.4.8
+rubygems_version: 3.0.8
 signing_key: 
 specification_version: 4
 summary: Fragment modeling and caching for Rails