jekyll-contentful-data-import 1.5.1 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d3f3bfe77c63cdbb2428c0e4ad7ec8ce5fcd85d7
-  data.tar.gz: 6b611233877a8275d952ec8ac49a1aecc55ff2fb
+  metadata.gz: 6240e01bd316ec09f91d3ec93de0886cb6b550fb
+  data.tar.gz: a77e6bb3f7234b2d7aec8816251a23c627a27db7
 SHA512:
-  metadata.gz: a51238cb49f031d904ac24b87e2ca9ba102a73d617cba54f756aea1315682ff737c1bfe67f3f47c0475048b6f8a1f134d4ededc3128dfcb19cbe99e80fcf608e
-  data.tar.gz: 1a8f633a2f9b6dfa747137778f46a724f0dbc63435705c70580b1fe0ab4cac5a5207337dc2711b964aaad380686f918c425aba143d9fefe0c57a247079fd0767
+  metadata.gz: 479b3306559697cd362e4a371365c671b644c00f62cecc82d8eb3207a542fc999d547f800f421e3603fdea1b4107079233b42db81318ebfcacab93f5c9db07ca
+  data.tar.gz: 7cbae4efc303ff3b88cfa62227ad4c19ec828db56eab9f84424d084cad8d5eaa66ea066e1db89beb842acf52a1286f12cdebdf8f5f6da5eac77fe631f6aecfd9

data/CHANGELOG.md

@@ -2,6 +2,12 @@
 
 ## Unreleased
 
+## 1.6.0
+
+### Added
+* Independent file per entry [#45](https://github.com/contentful/jekyll-contentful-data-import/pull/45) [#10](https://github.com/contentful/jekyll-contentful-data-import/issues/10) [#23](https://github.com/contentful/jekyll-contentful-data-import/pull/23) [#25](https://github.com/contentful/jekyll-contentful-data-import/issues/25)
+* Added User Agent Integration Headers
+
 ## v1.5.1
 ### Fixed
 * Fixed wrong serialization for Location fields
@@ -10,6 +16,7 @@
 
 ### Changed
 * Updated CDA SDK to 2.0
+* Field IDs will now be `snake_cased` following the 2.0.1 Ruby CDA SDK Upgrade
 
 ### Added
 * Added more metadata to `sys` attributes in entries

data/README.md

@@ -55,19 +55,21 @@ contentful:
           api_url: 'preview.contentful.com' # Defaults to 'api.contentful.com' which is Production
         base_path: app_path         # Optional - Defaults to Current directory
         destination: destination_in_data # Optional - Defaults to _data/contentful/spaces
+        individual_entry_files: true # Optional - Defaults to false
 ```
 
-Parameter             | Description
-----------            | ------------
-space                 | Contentful Space ID
-access_token          | Contentful Delivery API access token
-cda_query             | Hash describing query configuration. See [contentful.rb](https://github.com/contentful/contentful.rb) for more info (look for filter options there). Note that by default only 100 entries will be fetched, this can be configured to up to 1000 entries using the `limit` option.
-all_entries           | Boolean, if true will run multiple queries to the API until it fetches all entries for the space
-all_entries_page_size | Integer, the amount of maximum entries per CDA Request when fetching :all_entries
-content_types         | Hash describing the mapping applied to entries of the imported content types
-client_options        | Hash describing Contentful::Client configuration. See [contentful.rb](https://github.com/contentful/contentful.rb) for more info.
-base_path             | String with path to your Jekyll Application, defaults to current directory. Path is relative to your current location.
-destination           | String with path within `_data` under which to store the output yaml file. Defaults to contentful/spaces
+Parameter              | Description
+----------             | ------------
+space                  | Contentful Space ID
+access_token           | Contentful Delivery API access token
+cda_query              | Hash describing query configuration. See [contentful.rb](https://github.com/contentful/contentful.rb) for more info (look for filter options there). Note that by default only 100 entries will be fetched, this can be configured to up to 1000 entries using the `limit` option.
+all_entries            | Boolean, if true will run multiple queries to the API until it fetches all entries for the space
+all_entries_page_size  | Integer, the amount of maximum entries per CDA Request when fetching :all_entries
+content_types          | Hash describing the mapping applied to entries of the imported content types
+client_options         | Hash describing Contentful::Client configuration. See [contentful.rb](https://github.com/contentful/contentful.rb) for more info.
+base_path              | String with path to your Jekyll Application, defaults to current directory. Path is relative to your current location.
+destination            | String with path within `_data` under which to store the output yaml file. Defaults to contentful/spaces
+individual_entry_files | Boolean, if true will create an individual file per entry separated in folders by content type, file path will be `{space_alias}/{content_type_id}/{entry_id}.yaml`. Default behavior is to create a file per space. Usage is affected when this is set to true, please look in the section below.
 
 You can add multiple spaces to your configuration
 
@@ -157,6 +159,26 @@ therefore you can do the following:
 This way, it is safe to share your code without having to worry
 about your credentials.
 
+### Using Multiple Entry Files
+
+When setting the `individual_entry_files` flag to true, the usage pattern changes a little,
+as Jekyll does not allow for variable unpacking when iterating.
+
+A usage example is as follows:
+
+```html
+<ul class="cat-list">
+  <!-- Each element in the array of entries for a content type is an array of the form ['entry_id', { ... entry_data ...}] -->
+  {% for cat_data in site.data.contentful.spaces.example.cat %}
+    {% assign cat_id = cat_data[0] %} <!-- Entry ID is the first element of the array -->
+    {% assign cat = cat_data[1] %} <!-- Entry data is the second element of the array -->
+    <li>
+      <p>{{ cat_id }}: {{ cat.name }}</p>
+    </li>
+  {% endfor %}
+</ul>
+```
+
 ## Examples
 
 You can find working examples of multiple uses [here](https://github.com/contentful/contentful_jekyll_examples).

data/jekyll-contentful.gemspec

@@ -22,7 +22,7 @@ Gem::Specification.new do |s|
   s.add_dependency("jekyll", ">= 2.5.0", "< 4")
 
   # Additional dependencies
-  s.add_dependency("contentful", '>= 2.0.1', '~> 2.0')
+  s.add_dependency("contentful", '~> 2.1')
 
   s.add_development_dependency 'rubygems-tasks', '~> 0.2'
   s.add_development_dependency "guard"

data/lib/jekyll-contentful-data-import/{data_exporter.rb → base_data_exporter.rb}

@@ -2,14 +2,10 @@ require 'jekyll-contentful-data-import/serializer'
 
 module Jekyll
   module Contentful
-    # Data Exporter Class
+    # Base Data Exporter Class
     #
-    # Serializes Contentful data into YAML files
-    class DataExporter
-      DATA_FOLDER = '_data'.freeze
-      CONTENTFUL_FOLDER = 'contentful'.freeze
-      SPACES_FOLDER = 'spaces'.freeze
-
+    # Generic Data Exporter Implementation
+    class BaseDataExporter
       attr_reader :name, :entries, :config
 
       def initialize(name, entries, config = {})
@@ -19,16 +15,7 @@ module Jekyll
       end
 
       def run
-        setup_directory
-
-        File.open(destination_file, 'w') do |file|
-          file.write(
-            ::Jekyll::Contentful::Serializer.new(
-              entries,
-              config
-            ).to_yaml
-          )
-        end
+        raise 'must implement'
       end
 
       def base_directory
@@ -45,24 +32,34 @@ module Jekyll
 
       def destination_directory
         destination_dir = File.join(
-          base_directory, DATA_FOLDER,
-          CONTENTFUL_FOLDER, SPACES_FOLDER
+          base_directory, data_folder,
+          contentful_folder, spaces_folder
         )
         if config.key?('destination')
           destination_dir = File.join(
-            base_directory, DATA_FOLDER, config['destination']
+            base_directory, data_folder, config['destination']
           )
         end
 
         destination_dir
       end
 
-      def destination_file
-        File.join(destination_directory, "#{name}.yaml")
+      def setup_directory(directory)
+        FileUtils.mkdir_p(directory)
+      end
+
+      protected
+
+      def data_folder
+        '_data'
+      end
+
+      def contentful_folder
+        'contentful'
       end
 
-      def setup_directory
-        FileUtils.mkdir_p destination_directory
+      def spaces_folder
+        'spaces'
       end
     end
   end

data/lib/jekyll-contentful-data-import/importer.rb

@@ -1,5 +1,6 @@
 require 'contentful'
-require 'jekyll-contentful-data-import/data_exporter'
+require 'jekyll-contentful-data-import/single_file_data_exporter'
+require 'jekyll-contentful-data-import/multi_file_data_exporter'
 
 module Jekyll
   module Contentful
@@ -26,9 +27,35 @@ module Jekyll
       end
 
       def export_data(name, space_client, options)
-        Jekyll::Contentful::DataExporter.new(
+        entries = get_entries(space_client, options)
+
+        if options.fetch('individual_entry_files', false)
+          export_data_multiple_files(
+            name,
+            entries,
+            options
+          )
+        else
+          export_data_single_file(
+            name,
+            entries,
+            options
+          )
+        end
+      end
+
+      def export_data_single_file(name, entries, options)
+        Jekyll::Contentful::SingleFileDataExporter.new(
+          name,
+          entries,
+          options
+        ).run
+      end
+
+      def export_data_multiple_files(name, entries, options)
+        Jekyll::Contentful::MultiFileDataExporter.new(
           name,
-          get_entries(space_client, options),
+          entries,
           options
         ).run
       end
@@ -68,7 +95,9 @@ module Jekyll
           space: space,
           access_token: access_token,
           dynamic_entries: :auto,
-          raise_errors: true
+          raise_errors: true,
+          integration_name: 'jekyll',
+          integration_version: Jekyll::Contentful::VERSION
         }.merge(options)
 
         ::Contentful::Client.new(options)

data/lib/jekyll-contentful-data-import/multi_file_data_exporter.rb

@@ -0,0 +1,35 @@
+require 'jekyll-contentful-data-import/base_data_exporter'
+require 'yaml'
+
+module Jekyll
+  module Contentful
+    # Single File Data Exporter Class
+    #
+    # Serializes Contentful data into a multiple YAML files
+    class MultiFileDataExporter < BaseDataExporter
+      def run
+        data = ::Jekyll::Contentful::Serializer.new(
+          entries,
+          config
+        ).serialize
+
+        data.each do |content_type, entries|
+          content_type_directory = File.join(destination_directory, name, content_type.to_s)
+          setup_directory(content_type_directory)
+
+          entries.each do |entry|
+            yaml_entry = YAML.dump(entry)
+
+            File.open(destination_file(content_type_directory, entry), 'w') do |file|
+              file.write(yaml_entry)
+            end
+          end
+        end
+      end
+
+      def destination_file(content_type_directory, entry)
+        File.join(content_type_directory, "#{entry['sys']['id']}.yaml")
+      end
+    end
+  end
+end

data/lib/jekyll-contentful-data-import/single_file_data_exporter.rb

@@ -0,0 +1,27 @@
+require 'jekyll-contentful-data-import/base_data_exporter'
+
+module Jekyll
+  module Contentful
+    # Single File Data Exporter Class
+    #
+    # Serializes Contentful data into a single YAML file
+    class SingleFileDataExporter < BaseDataExporter
+      def run
+        setup_directory(destination_directory)
+
+        File.open(destination_file, 'w') do |file|
+          file.write(
+            ::Jekyll::Contentful::Serializer.new(
+              entries,
+              config
+            ).to_yaml
+          )
+        end
+      end
+
+      def destination_file
+        File.join(destination_directory, "#{name}.yaml")
+      end
+    end
+  end
+end

data/lib/jekyll-contentful-data-import/version.rb

@@ -1,5 +1,5 @@
 module Jekyll
   module Contentful
-    VERSION = '1.5.1'.freeze
+    VERSION = '1.6.0'.freeze
   end
 end

data/spec/jekyll-contentful/importer_spec.rb

@@ -42,13 +42,27 @@ describe Jekyll::Contentful::Importer do
 
     describe '#client' do
       it 'creates client with some defaults' do
-        expect(::Contentful::Client).to receive(:new).with(space: 'foo', access_token: 'foobar', dynamic_entries: :auto, raise_errors: true)
+        expect(::Contentful::Client).to receive(:new).with(
+          space: 'foo',
+          access_token: 'foobar',
+          dynamic_entries: :auto,
+          raise_errors: true,
+          integration_name: 'jekyll',
+          integration_version: Jekyll::Contentful::VERSION
+        )
 
         subject.client('foo', 'foobar')
       end
 
       it 'can override the defaults' do
-        expect(::Contentful::Client).to receive(:new).with(space: 'foo', access_token: 'foobar', dynamic_entries: :auto, raise_errors: false)
+        expect(::Contentful::Client).to receive(:new).with(
+          space: 'foo',
+          access_token: 'foobar',
+          dynamic_entries: :auto,
+          raise_errors: false,
+          integration_name: 'jekyll',
+          integration_version: Jekyll::Contentful::VERSION
+        )
 
         subject.client('foo', 'foobar', raise_errors: false)
       end
@@ -70,7 +84,7 @@ describe Jekyll::Contentful::Importer do
         allow(subject).to receive(:spaces).and_return([['foo', {'space' => 'foo', 'access_token' => 'bar'}], ['bar', {'space' => 'bar', 'access_token' => 'foo'}]])
         allow(subject).to receive(:client).and_return(ClientDouble.new)
 
-        expect(Jekyll::Contentful::DataExporter).to receive(:new).and_return(ExporterDouble.new).twice
+        expect(Jekyll::Contentful::SingleFileDataExporter).to receive(:new).and_return(ExporterDouble.new).twice
 
         subject.run
       end
@@ -78,7 +92,27 @@ describe Jekyll::Contentful::Importer do
       it 'runs exporter with correct arguments' do
         allow(subject).to receive(:client).and_return(ClientDouble.new)
 
-        expect(Jekyll::Contentful::DataExporter).to receive(:new).with('example', [], config['spaces'].first['example']).and_return(ExporterDouble.new)
+        expect(Jekyll::Contentful::SingleFileDataExporter).to receive(:new).with('example', [], config['spaces'].first['example']).and_return(ExporterDouble.new)
+
+        subject.run
+      end
+
+      it 'runs multifile exporter when passed :individual_entry_files flag' do
+        config = {
+          'spaces' => [
+            {
+              'example' => {
+                'space' => 'cfexampleapi',
+                'access_token' => 'b4c0n73n7fu1',
+                'individual_entry_files' => true
+              }
+            }
+          ]
+        }
+        subject = described_class.new(config)
+        allow(subject).to receive(:client).and_return(ClientDouble.new)
+
+        expect(Jekyll::Contentful::MultiFileDataExporter).to receive(:new).with('example', [], config['spaces'].first['example']).and_return(ExporterDouble.new)
 
         subject.run
       end

data/spec/jekyll-contentful/multi_file_data_exporter_spec.rb

@@ -0,0 +1,85 @@
+require 'spec_helper'
+require 'stringio'
+
+describe Jekyll::Contentful::MultiFileDataExporter do
+  subject { described_class.new('foo', []) }
+
+  describe 'instance methods' do
+    describe '#base_directory' do
+      it 'default directory' do
+        expect(subject.base_directory).to eq(Dir.pwd)
+      end
+
+      it 'overridden directory' do
+        subject = described_class.new('foo', [], {'base_path' => 'foo_dir'})
+
+        expect(subject.base_directory).to eq(File.join(Dir.pwd, 'foo_dir'))
+      end
+    end
+
+    describe '#destination_directory' do
+      it 'default directory' do
+        expected = File.join(Dir.pwd, '_data', 'contentful', 'spaces')
+        expect(subject.destination_directory).to eq(expected)
+      end
+
+      it 'overridden directory' do
+        subject = described_class.new('foo', [], {'base_path' => 'foo_dir'})
+
+        expected = File.join(Dir.pwd, 'foo_dir', '_data', 'contentful', 'spaces')
+        expect(subject.destination_directory).to eq(expected)
+      end
+    end
+
+    it '#destination_file' do
+      entry_double = { 'sys' => { 'id' => 'bar' } }
+      expected = File.join('foo', 'bar.yaml')
+      expect(subject.destination_file('foo', entry_double)).to eq(expected)
+    end
+
+    describe '#setup_directory' do
+      it 'default directory' do
+        expected = File.join(Dir.pwd, '_data', 'contentful', 'spaces')
+        expect(FileUtils).to receive(:mkdir_p).with(expected)
+
+        subject.setup_directory(expected)
+      end
+
+      it 'overridden directory' do
+        subject = described_class.new('foo', [], {'base_path' => 'foo_dir'})
+
+        expected = File.join(Dir.pwd, 'foo_dir', '_data', 'contentful', 'spaces')
+        expect(FileUtils).to receive(:mkdir_p).with(expected)
+
+        subject.setup_directory(expected)
+      end
+    end
+
+    describe '#run' do
+      before do
+        allow(subject).to receive(:setup_directory)
+      end
+
+      it 'does nothing with no entries' do
+        subject.run
+      end
+
+      it 'serializes entries' do
+        expect_any_instance_of(::Jekyll::Contentful::Serializer).to receive(:serialize) { {} }
+
+        subject.run
+      end
+
+      it 'creates a file per entry' do
+        subject = described_class.new('foo', [EntryDouble.new('bar', ContentTypeDouble.new('bar_ct'))])
+
+        expected_directory_path = File.join(Dir.pwd, '_data', 'contentful', 'spaces', 'foo', 'bar_ct')
+        expect(FileUtils).to receive(:mkdir_p).with(expected_directory_path)
+        expect(File).to receive(:open).with(File.join(expected_directory_path, 'bar.yaml'), 'w')
+
+        subject.run
+      end
+    end
+  end
+end
+

data/spec/jekyll-contentful/{data_exporter_spec.rb → single_file_data_exporter_spec.rb}

@@ -1,7 +1,7 @@
 require 'spec_helper'
 require 'stringio'
 
-describe Jekyll::Contentful::DataExporter do
+describe Jekyll::Contentful::SingleFileDataExporter do
   subject { described_class.new('foo', []) }
 
   describe 'instance methods' do
@@ -10,7 +10,7 @@ describe Jekyll::Contentful::DataExporter do
         expect(subject.base_directory).to eq(Dir.pwd)
       end
 
-      it 'overriden directory' do
+      it 'overridden directory' do
         subject = described_class.new('foo', [], {'base_path' => 'foo_dir'})
 
         expect(subject.base_directory).to eq(File.join(Dir.pwd, 'foo_dir'))
@@ -57,7 +57,7 @@ describe Jekyll::Contentful::DataExporter do
         expected = File.join(Dir.pwd, '_data', 'contentful', 'spaces')
         expect(FileUtils).to receive(:mkdir_p).with(expected)
 
-        subject.setup_directory
+        subject.setup_directory(expected)
       end
 
       it 'overridden directory' do
@@ -66,7 +66,7 @@ describe Jekyll::Contentful::DataExporter do
         expected = File.join(Dir.pwd, 'foo_dir', '_data', 'contentful', 'spaces')
         expect(FileUtils).to receive(:mkdir_p).with(expected)
 
-        subject.setup_directory
+        subject.setup_directory(expected)
       end
     end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: jekyll-contentful-data-import
 version: !ruby/object:Gem::Version
-  version: 1.5.1
+  version: 1.6.0
 platform: ruby
 authors:
 - Contentful GmbH
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-04-14 00:00:00.000000000 Z
+date: 2017-06-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: jekyll
@@ -34,22 +34,16 @@ dependencies:
   name: contentful
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 2.0.1
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.0'
+        version: '2.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 2.0.1
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.0'
+        version: '2.1'
 - !ruby/object:Gem::Dependency
   name: rubygems-tasks
   requirement: !ruby/object:Gem::Requirement
@@ -239,18 +233,21 @@ files:
 - Rakefile
 - jekyll-contentful.gemspec
 - lib/jekyll-contentful-data-import.rb
-- lib/jekyll-contentful-data-import/data_exporter.rb
+- lib/jekyll-contentful-data-import/base_data_exporter.rb
 - lib/jekyll-contentful-data-import/importer.rb
 - lib/jekyll-contentful-data-import/mappers.rb
 - lib/jekyll-contentful-data-import/mappers/base.rb
+- lib/jekyll-contentful-data-import/multi_file_data_exporter.rb
 - lib/jekyll-contentful-data-import/serializer.rb
+- lib/jekyll-contentful-data-import/single_file_data_exporter.rb
 - lib/jekyll-contentful-data-import/version.rb
 - lib/jekyll/commands/contentful.rb
 - spec/fixtures/vcr_fixtures/entries.yml
-- spec/jekyll-contentful/data_exporter_spec.rb
 - spec/jekyll-contentful/importer_spec.rb
 - spec/jekyll-contentful/mappers/base_spec.rb
+- spec/jekyll-contentful/multi_file_data_exporter_spec.rb
 - spec/jekyll-contentful/serializer_spec.rb
+- spec/jekyll-contentful/single_file_data_exporter_spec.rb
 - spec/jekyll/commands/contentful_spec.rb
 - spec/spec_helper.rb
 homepage: https://www.contentful.com
@@ -280,9 +277,10 @@ summary: Include mangablable content from the Contentful CMS and API into your J
   projects
 test_files:
 - spec/fixtures/vcr_fixtures/entries.yml
-- spec/jekyll-contentful/data_exporter_spec.rb
 - spec/jekyll-contentful/importer_spec.rb
 - spec/jekyll-contentful/mappers/base_spec.rb
+- spec/jekyll-contentful/multi_file_data_exporter_spec.rb
 - spec/jekyll-contentful/serializer_spec.rb
+- spec/jekyll-contentful/single_file_data_exporter_spec.rb
 - spec/jekyll/commands/contentful_spec.rb
 - spec/spec_helper.rb