wcc-contentful 1.0.7 → 1.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6ee244bf6dd08b9bdb26e9a4fd75e3b620b5204816e5e308bb645a53fd51c71e
-  data.tar.gz: df8e72ec7afa5ae987d85ffa19bc8d9841bc69016c0af7c12ca37700d3348674
+  metadata.gz: 3facd27f1d4401d01c3e42dc9ab11d75652cc6859a74c5c74ffb58647b13c28d
+  data.tar.gz: 48072021ae935b5c7d9a684f488dafb3ffa53b94ea153bfeeba7f1b5f5a46806
 SHA512:
-  metadata.gz: bc0d104dbbd9106e806f3eca34272888598b34f45e9b0e4b2f28ebd96d1771803cfdf02a04b59655ff8ab1d7f41043fc5fe11bea7cab6de14600e0180d3a8d31
-  data.tar.gz: 15d915db83f38cbb08262e5f1c7c5ee3278922dfa4d57c75d966a85fbc04fed5288c5cb0ea066e9d51cf271c77057c1d90cc4838ebef9eecb37e02190f200b3a
+  metadata.gz: b884394e95a3c722c8a3a1327780700f153f9e724c920d9d57e81409d836e5111777f3493b504bfe7b6321a1d83ed95f6fe2613b9f77677393d2caab631caa4e
+  data.tar.gz: 6891f2e8a291d4a72c725ce09a7270681ed8e2f7485d2f68d195f6fc0f55005e07408855f194d1ac130fe3c874c59c3bdc4e5aa87b512b776c37be792dd5e605

data/README.md

@@ -14,16 +14,20 @@ Table of Contents:
 2. [Installation](#installation)
 3. [Configuration](#configure)
 4. [Usage](#usage)
-  1. [Model API](#wcccontentfulmodel-api)
-  2. [Store API](#store-api)
-  3. [Direct CDN client](#direct-cdn-api-simpleclient)
-  4. [Accessing the APIs](#accessing-the-apis-within-application-code)
+   1. [Model API](#wcccontentfulmodel-api)
+   2. [Store API](#store-api)
+   3. [Direct CDN client](#direct-cdn-api-simpleclient)
+   4. [Accessing the APIs](#accessing-the-apis-within-application-code)
 5. [Architecture](#architecture)
+   1. [Client Layer](#client-layer)
+   2. [Store Layer](#store-layer)
+   3. [Model Layer](#model-layer)
 6. [Test Helpers](#test-helpers)
 7. [Advanced Configuration Example](#advanced-configuration-example)
-8. [Development](#development)
-9. [Contributing](#contributing)
-10. [License](#license)
+8. [Connecting to Multiple Spaces](#connecting-to-multiple-spaces-or-environments)
+9. [Development](#development)
+10. [Contributing](#contributing)
+11. [License](#license)
 
 
 ## Why did you rewrite the Contentful ruby stack?
@@ -45,7 +49,7 @@ The wcc-contentful gem enables caching at two levels: the HTTP response using [F
 By default, the contentful.rb gem requires the [HTTP library](https://rubygems.org/gems/http).  While simple and straightforward to use, it is not as powerful for caching.  We decided to make our client conform to the [Faraday gem's API](https://github.com/lostisland/faraday).  If you prefer not to use Faraday, you can choose to supply your own HTTP adapter that "quacks like" Faraday (see the [TyphoeusAdapter](https://github.com/watermarkchurch/wcc-contentful/blob/master/wcc-contentful/lib/wcc/contentful/simple_client/typhoeus_adapter.rb) for one implementation).
 
 Using Faraday makes it easy to add Middleware.  As an example, our flagship Rails app that powers watermark.org uses the following configuration in Production, which provides us with instrumentation through statsd, logging, and caching:
-```rb
+```ruby
 config.connection = Faraday.new do |builder|
   builder.use :http_cache,
     shared_cache: false,
@@ -77,7 +81,7 @@ We also took advantage of Rails' naming conventions to automatically infer the c
 
 All our models are automatically generated at startup which improves response times at the expense of initialization time.  In addition, our content model registry allows easy definition of custom models in your `app/models` directory to override fields.  This plays nice with other gems like algoliasearch-rails, which allows you to declaratively manage your Algolia indexes.  Another example from our flagship watermark.org:
 
-```rb
+```ruby
 class Page < WCC::Contentful::Model::Page
   include AlgoliaSearch
 
@@ -327,6 +331,98 @@ end
 
 ![wcc-contentful diagram](./doc-static/wcc-contentful.png)
 
+From the bottom up:
+
+### Client Layer
+
+The {WCC::Contentful::SimpleClient} provides methods to access the [Contentful 
+Content Delivery API](https://www.contentful.com/developers/docs/references/content-delivery-api/)
+through your favorite HTTP client gem.  The SimpleClient expects
+an Adapter that conforms to the Faraday interface.
+
+Creating a SimpleClient to connect using different credentials, or to connect
+without setting up all the rest of WCC::Contentful, is easy:
+
+```ruby
+WCC::Contentful::SimpleClient::Cdn.new(
+  # required
+  access_token: 'xxxx',
+  space: '1234',
+  # optional
+  environment: 'staging', # omit to use master
+  default_locale: '*',
+  rate_limit_wait_timeout: 10,
+  instrumentation: ActiveSupport::Notifications,
+  connection: Faraday.new { |builder| ... },
+)
+```
+
+You can also create a {WCC::Contentful::SimpleClient::Preview} to talk to the
+Preview API, or a {WCC::Contentful::SimpleClient::Management} to talk to the
+Management API.
+
+### Store Layer
+
+The Store Layer represents the data store where Contentful entries are kept for
+querying.  By default, `WCC::Contentful.init!` creates a {WCC::Contentful::Store::CDNAdapter}
+which uses a {WCC::Contentful::SimpleClient::Cdn} instance to query entries from
+the [Contentful Content Delivery API](https://www.contentful.com/developers/docs/references/content-delivery-api/).  You can also query entries from another
+source like Postgres or an in-memory hash if your data is small enough.
+
+You can also implement your own store if you want!  The gem contains a suite of
+RSpec shared examples that give you a baseline for implementing your own store.
+In your RSpec suite:
+```ruby
+# frozen_string_literal: true
+
+require 'my_store'
+require 'wcc/contentful/store/rspec_examples'
+
+RSpec.describe MyStore do
+  it_behaves_like 'contentful store', {
+    # Set which store features your store implements.  
+    nested_queries: true,  # Does your store implement JOINs?
+    include_param: true    # Does your store resolve links when given the :include option?
+  }
+```
+
+The store is kept up-to-date by the {WCC::Contentful::SyncEngine}.  The `SyncEngine#next` methodcalls the `#index` method on the configured store in order to update
+it with the latest data via the [Contentful Sync API](https://www.contentful.com/developers/docs/references/content-delivery-api/#/reference/synchronization).  For example,
+the {WCC::Contentful::Store::MemoryStore} uses this to update the hash with the
+newest version of an entry, or delete an entry out of the hash.
+
+#### Store Middleware
+
+The store layer is made up of a base store (which implements {WCC::Contentful::Store::Interface}),
+and optional middleware.  The middleware
+allows custom transformation of received entries via the `#select` and `#transform`
+methods.  To create your own middleware simply include {WCC::Contentful::Middleware::Store}
+and implement those methods, then call `use` when configuring the store:
+
+```ruby
+config.store :direct do
+  use MyMiddleware, param1: 'xxx'
+end
+```
+
+The most useful middleware is the {WCC::Contentful::Middleware::Store::CachingMiddleware},
+which enables `:lazy_sync` mode (see {WCC::Contentful::Configuration#store})
+
+### Model Layer
+
+This is the global top layer where your Rails app looks up content similarly to
+ActiveModel.  The models are namespaced under the root class {WCC::Contentful::Model}.
+Each model's implementation of `.find`, `.find_by`, and `.find_all` simply call
+into the configured Store.
+
+The main benefit of the Model layer is lazy link resolution.  When a model's
+property is accessed, if that property is a link that has not been resolved
+yet (for example using the `include: n` parameter on `.find_by`), the model
+will automatically call `#find` on the store to resolve that linked entry.
+
+Note that this can easily result in lots of CDN calls to Contentful!  To optimize
+this you should use the `include` parameter and/or use a different store.
+
 ## Test Helpers
 
 To use the test helpers, include the following in your rails_helper.rb:
@@ -460,6 +556,75 @@ rake wcc_contentful:download_schema
 All configuration options can be found [in the rubydoc](https://www.rubydoc.info/gems/wcc-contentful/WCC/Contentful/Configuration) under
 {WCC::Contentful::Configuration}
 
+## Connecting to multiple spaces or environments
+
+When initializing the WCC::Contentful gem using `WCC::Contentful.init!`, the gem will by default connect to the single
+Contentful space that you specify in the `WCC::Contentful.configure` step.  However the gem is also capable of connecting
+to multiple spaces within the same ruby process!  You just have to create and initialize a namespace.
+
+The {WCC::Contentful::ModelAPI} concern makes this straightforward.  Start by creating your Namespace
+and including the concern:
+```ruby
+# app/models/my_second_space.rb
+class MySecondSpace
+  include WCC::Contentful::ModelAPI
+end
+
+# app/models/other_page.rb
+class OtherPage < MySecondSpace::Page
+end
+```
+
+Then configure it in an initializer:
+```ruby
+# config/initializers/my_second_space.rb
+MySecondSpace.configure do |config|
+  # Make sure to point to a different schema file from your first space!
+  config.schema_file = Rails.root.join('db/second-contentful-schema.json')
+
+  config.access_token = ENV['SECOND_CONTENTFUL_ACCESS_TOKEN']
+  config.preview_token = ENV['SECOND_CONTENTFUL_PREVIEW_ACCESS_TOKEN']
+  config.space = ENV['SECOND_CONTENTFUL_SPACE_ID']
+  config.environment = ENV['CONTENTFUL_ENVIRONMENT']
+end
+```
+
+Finally, use it:
+```ruby
+OtherPage.find('1234')
+# GET https://cdn.contentful.com/spaces/other-space/environments/other-env/entries/1234
+# => #<OtherPage:0x0000000005c71a78 @created_at=2018-04-16 18:41:17 UTC...>
+
+Page.find('1234')
+# GET https://cdn.contentful.com/spaces/first-space/environments/first-env/entries/1234
+# => #<Page:0x0000000001271b70 @created_at=2018-04-15 12:02:14 UTC...>
+```
+
+The ModelAPI defines a second stack of services that you can access for lower level connections:
+```ruby
+store = MySecondSpace.services.store
+# => #<WCC::Contentful::Store::CDNAdapter:0x00007f888edac118
+client = MySecondSpace.services.client
+# => #<WCC::Contentful::SimpleClient::Cdn:0x00007f88942a8888
+preview_client = MySecondSpace.services.preview_client
+# => #<WCC::Contentful::SimpleClient::Preview:0x00007f888ccafa00
+sync_engine = MySecondSpace.services.sync_engine
+# => #<WCC::Contentful::SyncEngine:0x00007f88960b6b40
+```
+Note that the above services are not accessible on {WCC::Contentful::Services.instance}
+or via the {WCC::Contentful::ServiceAccessors}.
+
+### Using a sync store with a second space
+
+If you use something other than the CDNAdapter with your second space, you will
+need to find a way to trigger `MySecondSpace.services.sync_engine.next` to keep
+it up-to-date.  The {WCC::Contentful::Engine} will only manage the global SyncEngine
+configured by the global {WCC::Contentful.configure}.
+
+The easiest way to do this is to set up your own Rails route to respond to Contentful
+webhooks and then configure the second Contentful space to send webhooks to this route.
+You could also do this by triggering it periodically in a background job using
+Sidekiq and [sidekiq-scheduler](https://github.com/Moove-it/sidekiq-scheduler).
 
 ## Development
 

data/app/jobs/wcc/contentful/webhook_enable_job.rb

@@ -13,18 +13,17 @@ module WCC::Contentful
       client = WCC::Contentful::SimpleClient::Management.new(
         args
       )
-      enable_webhook(client, args.slice(:app_url, :webhook_username, :webhook_password))
+      enable_webhook(client, args.slice(:receive_url, :webhook_username, :webhook_password))
     end
 
-    def enable_webhook(client, app_url:, webhook_username: nil, webhook_password: nil)
-      expected_url = URI.join(app_url, 'webhook/receive').to_s
-      webhook = client.webhook_definitions.items.find { |w| w['url'] == expected_url }
+    def enable_webhook(client, receive_url:, webhook_username: nil, webhook_password: nil)
+      webhook = client.webhook_definitions.items.find { |w| w['url'] == receive_url }
       logger.debug "existing webhook: #{webhook.inspect}" if webhook
       return if webhook
 
       body = {
         'name' => 'WCC::Contentful webhook',
-        'url' => expected_url,
+        'url' => receive_url,
         'topics' => [
           '*.publish',
           '*.unpublish'
@@ -50,13 +49,14 @@ module WCC::Contentful
 
       {
         management_token: config.management_token,
-        app_url: config.app_url,
         space: config.space,
         environment: config.environment,
         default_locale: config.default_locale,
         connection: config.connection,
         webhook_username: config.webhook_username,
-        webhook_password: config.webhook_password
+        webhook_password: config.webhook_password,
+
+        receive_url: URI.join(config.app_url, 'webhook/receive').to_s
       }
     end
 

data/lib/wcc/contentful/instrumentation.rb

@@ -11,13 +11,31 @@ module WCC::Contentful
           .name.parameterize.split('-').reverse.join('.')
     end
 
+    attr_writer :_instrumentation
+    def _instrumentation
+      # look for per-instance instrumentation then try class level
+      @_instrumentation || self.class._instrumentation
+    end
+
     included do
       protected
 
       def _instrument(name, payload = {}, &block)
         name += _instrumentation_event_prefix
-        (@_instrumentation ||= WCC::Contentful::Services.instance.instrumentation)
-          .instrument(name, payload, &block)
+        _instrumentation&.instrument(name, payload, &block)
+      end
+    end
+
+    class_methods do
+      attr_writer :_instrumentation
+
+      def _instrumentation
+        @_instrumentation ||
+          # try looking up the class heierarchy
+          superclass.try(:_instrumentation) ||
+          # default to global
+          WCC::Contentful::Services.instance&.instrumentation ||
+          ActiveSupport::Notifications
       end
     end
 

data/lib/wcc/contentful/link_visitor.rb

@@ -5,32 +5,21 @@
 # But you can use it too!
 class WCC::Contentful::LinkVisitor
   attr_reader :entry
-  attr_reader :type
   attr_reader :fields
   attr_reader :depth
 
   # @param [Hash] entry The entry hash (resolved or unresolved) to walk
-  # @param [Array<String, Symbol>] The fields to select from the entry tree.
-  #         Use `:Link` to select only links, or `'slug'` to select all slugs in the tree.
+  # @param [Array<Symbol>] The type of fields to select from the entry tree.
+  #         Must be one of `:Link`, `:Entry`, `:Asset`.
   # @param [Fixnum] depth (optional) How far to walk down the tree of links.  Be careful of
   #         recursive trees!
-  # @example
-  #   entry = store.find_by(id: id, include: 3)
-  #   WCC::Contentful::LinkVisitor.new(entry, 'slug', depth: 3)
-  #     .map { |slug| 'https://mirror-site' + slug }
   def initialize(entry, *fields, depth: 0)
-    unless entry.is_a?(Hash) && entry.dig('sys', 'id')
+    unless entry.is_a?(Hash) && entry.dig('sys', 'type') == 'Entry'
       raise ArgumentError, "Please provide an entry as a hash value (got #{entry})"
     end
-    unless ct = entry.dig('sys', 'contentType', 'sys', 'id')
-      raise ArgumentError, 'Entry has no content type!'
-    end
-
-    @type = WCC::Contentful.types[ct]
-    raise ArgumentError, "Unknown content type '#{ct}'" unless @type
 
     @entry = entry
-    @fields = fields
+    @fields = fields.map(&:to_s)
     @depth = depth
   end
 
@@ -42,7 +31,7 @@ class WCC::Contentful::LinkVisitor
   # @returns nil
   def each(&block)
     _each do |val, field, locale, index|
-      yield(val, field, locale, index) if should_yield_field?(field)
+      yield(val, field, locale, index) if should_yield_field?(field, val)
 
       next unless should_walk_link?(field, val)
 
@@ -54,7 +43,7 @@ class WCC::Contentful::LinkVisitor
 
   def map!(&block)
     _each do |val, field, locale, index|
-      if should_yield_field?(field)
+      if should_yield_field?(field, val)
         val = yield(val, field, locale, index)
         set_field(field, locale, index, val)
       end
@@ -70,15 +59,15 @@ class WCC::Contentful::LinkVisitor
   private
 
   def _each(&block)
-    type.fields.each_value do |f|
-      each_field(f, &block)
+    (entry['fields'] || {}).each do |(k, _v)|
+      each_field(k, &block)
     end
   end
 
   def each_field(field)
     each_locale(field) do |val, locale|
-      if field.array
-        val&.each_with_index do |v, index|
+      if val&.is_a?(Array)
+        val.each_with_index do |v, index|
           yield(v, field, locale, index) unless v.nil?
         end
       else
@@ -88,7 +77,7 @@ class WCC::Contentful::LinkVisitor
   end
 
   def each_locale(field)
-    raw_value = entry.dig('fields', field.name)
+    raw_value = entry.dig('fields', field)
     if locale = entry.dig('sys', 'locale')
       if raw_value.is_a?(Hash) && raw_value[locale]
         # it's a locale=* entry, but they've added sys.locale to those now
@@ -102,21 +91,28 @@ class WCC::Contentful::LinkVisitor
     end
   end
 
-  def should_yield_field?(field)
-    fields.empty? || fields.include?(field.type) || fields.include?(field.name)
+  def should_yield_field?(_field, value)
+    return true if fields.empty?
+
+    case value
+    when Hash
+      fields.include?(value.dig('sys', 'type'))
+    when Array
+      value.any? { |v| v.is_a?(Hash) && fields.include?(v.dig('sys', 'type')) }
+    end
   end
 
-  def should_walk_link?(field, val, dep = depth)
-    dep > 0 && field.type == :Link && val.dig('sys', 'type') == 'Entry'
+  def should_walk_link?(_field, val, dep = depth)
+    dep > 0 && val.is_a?(Hash) && val.dig('sys', 'type') == 'Entry'
   end
 
   def set_field(field, locale, index, val)
-    current_field = (entry['fields'][field.name] ||= {})
+    current_field = (entry['fields'][field] ||= {})
 
-    if field.array
-      (current_field[locale] ||= [])[index] = val
-    else
+    if index.nil?
       current_field[locale] = val
+    else
+      (current_field[locale] ||= [])[index] = val
     end
   end
 end

data/lib/wcc/contentful/middleware/store.rb

@@ -55,7 +55,8 @@ module WCC::Contentful::Middleware::Store
   def resolve_includes(entry, depth)
     return entry unless entry && depth && depth > 0
 
-    WCC::Contentful::LinkVisitor.new(entry, :Link, depth: depth).map! do |val|
+    # We only care about entries (see #resolved_link?)
+    WCC::Contentful::LinkVisitor.new(entry, :Entry, depth: depth).map! do |val|
       resolve_link(val)
     end
   end

data/lib/wcc/contentful/model.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require_relative './model_api'
+
 # This is the top layer of the WCC::Contentful gem.  It exposes an API by which
 # you can query for data from Contentful.  The API is only accessible after calling
 # WCC::Contentful.init!
@@ -37,138 +39,13 @@
 #
 # @api Model
 class WCC::Contentful::Model
-  extend WCC::Contentful::Helpers
-
-  # The Model base class maintains a registry which is best expressed as a
-  # class var.
-  # rubocop:disable Style/ClassVars
+  include WCC::Contentful::ModelAPI
 
   class << self
-    include WCC::Contentful::ServiceAccessors
-
     def const_missing(name)
+      type = WCC::Contentful::Helpers.content_type_from_constant(name)
       raise WCC::Contentful::ContentTypeNotFoundError,
-        "Content type '#{content_type_from_constant(name)}' does not exist in the space"
-    end
-  end
-
-  @@registry = {}
-
-  def self.store(preview = false)
-    if preview
-      if preview_store.nil?
-        raise ArgumentError,
-          'You must include a contentful preview token in your WCC::Contentful.configure block'
-      end
-      preview_store
-    else
-      super()
-    end
-  end
-
-  # Finds an Entry or Asset by ID in the configured contentful space
-  # and returns an initialized instance of the appropriate model type.
-  #
-  # Makes use of the {WCC::Contentful::Services#store configured store}
-  # to access the Contentful CDN.
-  def self.find(id, options: nil)
-    options ||= {}
-    raw = store(options[:preview])
-      .find(id, options.except(*WCC::Contentful::ModelMethods::MODEL_LAYER_CONTEXT_KEYS))
-
-    new_from_raw(raw, options) if raw.present?
-  end
-
-  # Creates a new initialized instance of the appropriate model type for the
-  # given raw value.  The raw value must be the same format as returned from one
-  # of the stores for a given object.
-  def self.new_from_raw(raw, context = nil)
-    content_type = content_type_from_raw(raw)
-    const = resolve_constant(content_type)
-    const.new(raw, context)
-  end
-
-  # Accepts a content type ID as a string and returns the Ruby constant
-  # stored in the registry that represents this content type.
-  def self.resolve_constant(content_type)
-    raise ArgumentError, 'content_type cannot be nil' unless content_type
-
-    const = @@registry[content_type]
-    return const if const
-
-    const_name = constant_from_content_type(content_type).to_s
-    begin
-      # The app may have defined a model and we haven't loaded it yet
-      const = Object.const_missing(const_name)
-      return const if const && const < WCC::Contentful::Model
-    rescue NameError => e
-      raise e unless e.message =~ /uninitialized constant #{const_name}/
-
-      nil
-    end
-
-    # Autoloading couldn't find their model - we'll register our own.
-    const = WCC::Contentful::Model.const_get(constant_from_content_type(content_type))
-    register_for_content_type(content_type, klass: const)
-  end
-
-  # Registers a class constant to be instantiated when resolving an instance
-  # of the given content type.  This automatically happens for the first subclass
-  # of a generated model type, example:
-  #
-  #   class MyMenu < WCC::Contentful::Model::Menu
-  #   end
-  #
-  # In the above case, instances of MyMenu will be instantiated whenever a 'menu'
-  # content type is resolved.
-  # The mapping can be made explicit with the optional parameters.  Example:
-  #
-  #   class MyFoo < WCC::Contentful::Model::Foo
-  #     register_for_content_type 'bar' # MyFoo is assumed
-  #   end
-  #
-  #   # in initializers/wcc_contentful.rb
-  #   WCC::Contentful::Model.register_for_content_type('bar', klass: MyFoo)
-  def self.register_for_content_type(content_type = nil, klass: nil)
-    klass ||= self
-    raise ArgumentError, "#{klass} must be a class constant!" unless klass.respond_to?(:new)
-
-    content_type ||= content_type_from_constant(klass)
-
-    @@registry[content_type] = klass
-  end
-
-  # Returns the current registry of content type names to constants.
-  def self.registry
-    return {} unless @@registry
-
-    @@registry.dup.freeze
-  end
-
-  def self.reload!
-    registry = self.registry
-    registry.each do |(content_type, klass)|
-      const_name = klass.name
-      begin
-        const = Object.const_missing(const_name)
-        register_for_content_type(content_type, klass: const) if const
-      rescue NameError => e
-        msg = "Error when reloading constant #{const_name} - #{e}"
-        if defined?(Rails) && Rails.logger
-          Rails.logger.error msg
-        else
-          puts msg
-        end
-      end
+        "Content type '#{type}' does not exist in the space"
     end
   end
-
-  # Checks if a content type has already been registered to a class and returns
-  # that class.  If nil, the generated WCC::Contentful::Model::{content_type} class
-  # will be resolved for this content type.
-  def self.registered?(content_type)
-    @@registry[content_type]
-  end
 end
-
-# rubocop:enable Style/ClassVars