algoliasearch-jekyll 0.2.2 → 0.2.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8ba110a511e94a6806f7cea6863401e58e7ff2a8
-  data.tar.gz: 8c498d9b582c93c4ee721d4c27af5f29a11cf528
+  metadata.gz: cde772c27a05882aba2a47f5c0fa626440ad7755
+  data.tar.gz: d0c6dafb3d6b3ade9a35057f6c91b9c6b3c3bfdc
 SHA512:
-  metadata.gz: 0f85d1f891cc31d2221cb822af36c2c4e31e501630c31c6cdeab90b508f3740f5b36830624d5c655ba35278a1000b1498f4b86a80b47891b924dd32c406ce9c3
-  data.tar.gz: af03f61ef88213c089ead811b6dc42676245614f7e4fada7f016851be224dfc9072ba355bcba7cd3772e68449257d5a92751c41f7ab4929db1455810d07e9961
+  metadata.gz: ad814fa1f1eee3d87a6407740d9aad6501874a86b395bda116c0ddeb82d35dae9ed17c87bbb59d98c11fc32c5c75a52a187838b8a8bf7714ec158c0b9a8137a0
+  data.tar.gz: 43d778da3ad04dec941d1f46e7a1943aeb8e60e771171243f9f502994d52a18325d26a85497f5919ccf2437c57e1c0b5fd1400ddaab0e7b8bf883650628f9b2a

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--format progress

data/.rubocop.yml

@@ -0,0 +1,25 @@
+# Defaults:
+# https://github.com/bbatsov/rubocop/blob/master/config/default.yml
+Metrics/AbcSize:
+  Max: 100
+
+Metrics/ClassLength:
+  Max: 200
+
+Metrics/ModuleLength:
+  Max: 200
+
+Metrics/MethodLength:
+  Max: 50
+
+Metrics/CyclomaticComplexity:
+  Max: 10
+
+Metrics/PerceivedComplexity:
+  Max: 10
+
+Style/FileName:
+  Enabled: false
+
+Style/MultilineOperationIndentation:
+  Enabled: false

data/Gemfile

@@ -0,0 +1,15 @@
+source 'http://rubygems.org'
+
+gem 'algoliasearch', '~> 1.4'
+gem 'awesome_print', '~> 1.6'
+gem 'json', '~> 1.8'
+gem 'nokogiri', '~> 1.6'
+
+group :development do
+  gem 'guard-rspec', '~> 4.6'
+  gem 'jekyll', '~> 2.5'
+  gem 'jeweler', '~> 2.0'
+  gem 'rspec', '~> 3.0'
+  gem 'rubocop', '~> 0.31'
+  gem 'simplecov', '~> 0.10'
+end

data/Guardfile

@@ -0,0 +1,7 @@
+guard :rspec, cmd: 'bundle exec rspec --color --format documentation' do
+  watch(%r{^spec/.+_spec\.rb$})
+  watch(%r{^lib/(.+)\.rb$})     { |m| "spec/#{m[1]}_spec.rb" }
+  watch('spec/spec_helper.rb')  { 'spec' }
+end
+
+notification :off

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2015 Algolia
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,148 @@
+# Algolia Jekyll Plugin 
+
+[![Gem Version](https://badge.fury.io/rb/algoliasearch-jekyll.svg)](http://badge.fury.io/rb/algoliasearch-jekyll)
+
+Jekyll plugin to automatically index your Jekyll posts and pages into an
+Algolia index by simply running `jekyll algolia push`.
+
+## Usage
+
+```shell
+$ jekyll algolia push
+```
+
+This will push the content of your jekyll website to your Algolia index.
+
+You can specify any option you would normally pass to `jekyll build`, like
+`--config`, `--source`, `--destination`, etc.
+
+## Installation
+
+First, add the `algoliasearch-jekyll` gem to your `Gemfile`, in the
+`:jekyll_plugins` section. If you do not yet have a `Gemfile`, here is the
+minimum content to get your started.
+
+```ruby
+source 'https://rubygems.org'
+
+gem 'jekyll', '>=2.5.3'
+
+group :jekyll_plugins do
+  gem 'algoliasearch-jekyll'
+end
+```
+
+Once this is done, download all dependencies with `bundle install`. 
+
+Then, add `algoliasearch-jekyll` to your `_config.yml` file, under the `gems`
+section, like this:
+
+```yaml
+gems:
+  - algoliasearch-jekyll
+```
+
+If everything went well, you should be able to execute `jekyll help` and see the
+`algolia` subcommand listed.
+
+## Configuration
+
+Add information about your Algolia configuration into the `_config.yml` file,
+under the `algolia` section, like this:
+
+```yaml
+algolia:
+  application_id: 'your_application_id'
+  index_name:     'your_index_name'
+```
+
+You admin api key will be read from the `ALGOLIA_API_KEY` environment variable.
+You can define it on the same line as your command, allowing you to type
+`ALGOLIA_API_KEY='your_admin_api_key' jekyll algolia push`.
+
+### ⚠ Other, unsecure, method ⚠
+
+You can also store your admin api key in a file named `_algolia_api_key`, in
+your source directory. If you do this we __very, very, very strongly__ encourage
+you to make sure the file is not tracked in your versioning system.
+
+### Options
+
+The plugin uses sensible defaults, but you may want to override some of its
+configuration. Here are the various options you can add to your `_config.yml`
+file, under the `algolia` section:
+
+#### `excluded_files`
+
+Defines which files should not be indexed for search.
+
+```yml
+algolia:
+  excluded_files:
+    - index.html
+    - 2015-01-01-post.md
+```
+
+#### `record_css_selector`
+
+Defines the css selector inside a page/post used to choose which parts to index.
+It is set to all paragraphs (`<p>`) by default.
+
+If you would like to also index lists, you could set it like this:
+
+```yml
+algolia:
+  record_css_selector: 'p,ul'
+```
+
+#### `settings`
+
+Here you can pass any custom settings you would like to push to your Algolia
+index.
+
+If you want to activate `distinct` and some snippets for example, you would do:
+
+```yml
+algolia:
+  settings:
+    attributeForDistinct: 'hierarchy'
+    distinct: true
+    attributesToSnippet: ['text:20']
+```
+
+### Hooks
+
+The `AlgoliaSearchRecordExtractor` contains two methods (`custom_hook_each` and
+`custom_hook_all`) that are here so you can overwrite them to add your custom
+logic. They currently simply return the argument they take as input.
+
+```ruby
+class AlgoliaSearchRecordExtractor
+  # Hook to modify a record after extracting
+  # `node` refers to the Nokogiri HTML node of the element
+  def custom_hook_each(item, node)
+    item
+  end
+
+  # Hook to modify all records after extracting
+  def custom_hook_all(items)
+    items
+  end
+end
+```
+
+## Searching
+
+This plugin will only index your data in your Algolia index. Adding search
+capabilities is quite easy. You can follow [our tutorials][1] or use our forked
+version of the popular [Hyde theme][2].
+
+## GitHub Pages
+
+Unfortunatly, GitHub does not allow custom plugins to be run on GitHub Pages.
+This mean that you will have to manually run `jekyll algolia push` before
+pushing your content to GitHub.
+
+
+[1]: https://www.algolia.com/doc/javascript
+[2]: https://github.com/algolia/hyde

data/Rakefile

@@ -0,0 +1,44 @@
+# encoding: utf-8
+
+require 'rubygems'
+require 'bundler'
+begin
+  Bundler.setup(:default, :development)
+rescue Bundler::BundlerError => e
+  $stderr.puts e.message
+  $stderr.puts 'Run `bundle install` to install missing gems'
+  exit e.status_code
+end
+require 'rake'
+
+require 'jeweler'
+Jeweler::Tasks.new do |gem|
+  # gem is a Gem::Specification... see
+  # http://guides.rubygems.org/specification-reference/ for more options
+  gem.name = 'algoliasearch-jekyll'
+  gem.homepage = 'https://github.com/algolia/algoliasearch-jekyll'
+  gem.license = 'MIT'
+  gem.summary = 'AlgoliaSearch for Jekyll'
+  gem.description = 'Index all your pages and posts to an Algolia index with ' \
+                    '`jekyll algolia push`'
+  gem.email = 'tim@pixelastic.com'
+  gem.authors = ['Tim Carry']
+  # dependencies defined in Gemfile
+end
+Jeweler::RubygemsDotOrgTasks.new
+
+require 'rspec/core'
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec) do |spec|
+  # spec.rspec_opts = '--color --format documentation'
+  spec.pattern = FileList['spec/**/*_spec.rb']
+end
+task test: :spec
+
+desc 'Code coverage detail'
+task :coverage do
+  ENV['COVERAGE'] = 'true'
+  Rake::Task['spec'].execute
+end
+
+task default: :test

data/VERSION

@@ -0,0 +1 @@
+0.2.3

data/lib/push.rb

@@ -31,17 +31,19 @@ class AlgoliaSearchJekyllPush < Jekyll::Command
     def indexable?(file)
       return false if file.is_a?(Jekyll::StaticFile)
 
+      basename = File.basename(file.path)
+      extname = File.extname(basename)[1..-1]
+
       # Keep only markdown and html files
       allowed_extensions = %w(html)
       if @config['markdown_ext']
         allowed_extensions += @config['markdown_ext'].split(',')
       end
-      current_extension = File.extname(file.name)[1..-1]
-      return false unless allowed_extensions.include?(current_extension)
+      return false unless allowed_extensions.include?(extname)
 
       # Exclude files manually excluded from config
       excluded_files = @config['algolia']['excluded_files']
-      return false if excluded_files && excluded_files.include?(file.name)
+      return false if excluded_files && excluded_files.include?(basename)
 
       true
     end
@@ -55,7 +57,6 @@ class AlgoliaSearchJekyllPush < Jekyll::Command
         items = []
         each_site_file do |file|
           next unless AlgoliaSearchJekyllPush.indexable?(file)
-
           new_items = AlgoliaSearchRecordExtractor.new(file).extract
           next if new_items.nil?
 
@@ -162,27 +163,33 @@ class AlgoliaSearchJekyllPush < Jekyll::Command
 
     def push(items)
       check_credentials
-
-      index_name = @config['algolia']['index_name']
       Algolia.init(
         application_id: @config['algolia']['application_id'],
         api_key: api_key
       )
-      index = Algolia::Index.new(index_name)
-      configure_index(index)
-      index.clear_index
 
+      # Create a temporary index
+      index_name = @config['algolia']['index_name']
+      index_name_tmp = "#{index_name}_tmp"
+      index_tmp = Algolia::Index.new(index_name_tmp)
+      configure_index(index_tmp)
+
+      # Push to temporary index
       items.each_slice(1000) do |batch|
         Jekyll.logger.info "Indexing #{batch.size} items"
         begin
-          index.add_objects(batch)
+          index_tmp.add_objects!(batch)
         rescue StandardError => error
           Jekyll.logger.error 'Algolia Error: HTTP Error'
           Jekyll.logger.warn error.message
           exit 1
+
         end
       end
 
+      # Move temporary index to real one
+      Algolia.move_index(index_name_tmp, index_name)
+
       Jekyll.logger.info "Indexing of #{items.size} items " \
                          "in #{index_name} done."
     end

data/lib/record_extractor.rb

@@ -24,9 +24,24 @@ class AlgoliaSearchRecordExtractor
 
   # Returns metadata from the current file
   def metadata
-    return metadata_page if @file.is_a?(Jekyll::Page)
-    return metadata_post if @file.is_a?(Jekyll::Post)
-    {}
+    metadata = {}
+    @file.data.each { |key, value| metadata[key.to_sym] = value }
+
+    metadata[:type] = @file.class.name.split('::')[1].downcase
+    metadata[:url] = @file.url
+
+    if @file.respond_to? :slug
+      metadata[:slug] = @file.slug
+    else
+      basename = File.basename(@file.path)
+      extname = File.extname(basename)
+      metadata[:slug] = File.basename(basename, extname)
+    end
+
+    metadata[:posted_at] = @file.date.to_time.to_i if @file.respond_to? :date
+    metadata[:tags] = tags if @file.respond_to? :tags
+
+    metadata
   end
 
   # Extract a list of tags
@@ -37,41 +52,21 @@ class AlgoliaSearchRecordExtractor
     @file.tags.map(&:to_s)
   end
 
-  # Extract metadata from a post
-  def metadata_post
-    {
-      type: 'post',
-      url: @file.url,
-      title: @file.title,
-      slug: @file.slug,
-      posted_at: @file.date.to_time.to_i,
-      tags: tags
-    }
-  end
-
-  # Extract metadata from a page
-  def metadata_page
-    {
-      type: 'page',
-      url: @file.url,
-      title: @file['title'],
-      slug: @file.basename
-    }
-  end
-
   # Get the list of all HTML nodes to index
   def html_nodes
     document = Nokogiri::HTML(@file.content)
     document.css(@config['record_css_selector'])
   end
 
+  # Check if node is a heading
+  def node_heading?(node)
+    %w(h1 h2 h3 h4 h5 h6).include?(node.name)
+  end
+
   # Get the closest heading parent
   def node_heading_parent(node, level = 'h7')
-    headings = %w(h1 h2 h3 h4 h5 h6)
-
-    # If initially called on a heading, we must not accept it but only accept
-    # strong headings
-    level = node.name if level == 'h7' && headings.include?(node.name)
+    # If initially called on a heading, we only accept stronger headings
+    level = node.name if level == 'h7' && node_heading?(node)
 
     previous = node.previous_element
 
@@ -84,32 +79,31 @@ class AlgoliaSearchRecordExtractor
     end
 
     # This is a heading, we return it
-    return previous if headings.include?(previous.name) && previous.name < level
+    return previous if node_heading?(previous) && previous.name < level
 
     node_heading_parent(previous, level)
   end
 
   # Get all the parent headings of the specified node
-  def node_hierarchy(node, memo = { level: 7 })
-    previous = node_heading_parent(node)
+  # If the node itself is a heading, we include it
+  def node_hierarchy(node, state = { level: 7 })
+    tag_name = node.name
+    level = tag_name.gsub('h', '').to_i
 
-    # No previous heading, we can stop the recursion
-    unless previous
-      memo.delete(:level)
-      return memo
+    if node_heading?(node) && level < state[:level]
+      state[tag_name.to_sym] = node_text(node)
+      state[:level] = level
     end
 
-    tag_name = previous.name
-    level = tag_name.gsub('h', '').to_i
-    content = previous.content
+    heading = node_heading_parent(node)
 
-    # Skip if item already as title of a higher level
-    return node_hierarchy(previous, memo) if level >= memo[:level]
-    memo[:level] = level
+    # No previous heading, we can stop the recursion
+    unless heading
+      state.delete(:level)
+      return state
+    end
 
-    # Add to the memo and continue
-    memo[tag_name.to_sym] = content
-    node_hierarchy(previous, memo)
+    node_hierarchy(heading, state)
   end
 
   # Return the raw HTML of the element to index

data/scripts/git_hooks/pre-commit

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+
+# Do not commit any focused or excluded tests
+if grep --color -r 'spec' -E -e '^( |\t)*(fit|fdescribe|xit|xdescribe)'; then
+  echo '✘ You have focused and/or skipped tests'
+  exit 1
+fi
+
+# Match style guide
+rubocop './lib/' './spec'

data/scripts/git_hooks/pre-push

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+bundle exec rspec --format documentation

data/scripts/run_tests

@@ -0,0 +1,2 @@
+#!/usr/bin/env bash
+bundle exec rspec --color --format documentation

data/scripts/update_gem

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+
+build_output = `gem build algoliasearch-jekyll.gemspec`.split("\n")
+puts build_output
+build_output.select! { |line| line.match(/^  File:/) }
+
+gem_file = build_output[0].gsub('  File: ', '')
+puts 'Pushing to rubygems'
+push_output = `gem push #{gem_file}`
+puts push_output

data/spec/fixtures/_config.yml

@@ -0,0 +1,7 @@
+algolia:
+  application_id: 'APPID'
+  index_name:     'INDEXNAME'
+
+collections:
+  my-collection:
+    output: true

data/spec/fixtures/_my-collection/collection-item.html

@@ -0,0 +1,6 @@
+---
+title: Collection Item
+---
+
+<p>The grandest of omelettes. Those that feast on dragon eggs often find that there
+  is very little they would not dare to do.</p>

data/spec/fixtures/_my-collection/collection-item.md

@@ -0,0 +1,7 @@
+---
+title: Collection Item
+custom: Foo
+---
+
+The grandest of omelettes. Those that feast on dragon eggs often find that there
+is very little they would not dare to do.

data/spec/fixtures/_posts/2015-07-02-test-post.md

@@ -0,0 +1,32 @@
+---
+title: "Test post"
+tags: 
+  - tag
+  - another tag
+custom: Foo
+---
+
+Introduction text that also includes [some link](https://www.algolia.com). To
+add a bit of fancy, we will also __bold__ and _italicize_ some text.
+
+# Main title
+
+We like writing stuff and then indexing it in a very fast engine. Here is why
+a fast engine is good for you:
+
+* fast
+* fast
+* fast
+* and fast
+
+## Built with hands
+
+All this text was typed with my own hands, on my own keyboard. I also did use
+a Chair© and a Table™.
+
+## Features
+
+The whole plugin is composed of parts of `code`, and sometime even
+<code>&lt;code&gt;</code>.
+
+Code is __✔ checked__ and errors are __✘ deleted__.

data/spec/fixtures/about.md

@@ -0,0 +1,34 @@
+---
+title: About page
+custom: Foo
+---
+
+# Heading 1
+
+Text 1
+
+## Heading 2
+
+Text 2
+
+### Heading 3
+
+Text 3
+
+- item 1
+- item 2
+- item 3
+
+### Another Heading 3
+
+<p id="text4">Another text 4</p>
+
+<h2 id="heading2b">Another Heading 2</h2>
+
+Another `<text>` 5
+
+### Last Heading 3
+
+<div>Just a div</div>
+
+<div><p>Last text 6 </p></div>

data/spec/fixtures/api_key_dir/_algolia_api_key

@@ -0,0 +1 @@
+APIKEY_FROM_FILE

data/spec/fixtures/authors.html

@@ -0,0 +1,5 @@
+---
+title: Authors
+---
+
+<p>This is an HTML page</p>

data/spec/fixtures/excluded.html

@@ -0,0 +1,5 @@
+---
+title: Excluded file
+---
+
+<p>This should not be indexed</p>

data/spec/fixtures/hierarchy.md

@@ -0,0 +1,35 @@
+---
+title: Hierarchy test
+---
+
+# H1
+
+TEXT1-H1
+
+## H2A
+
+TEXT2-H2A-H1
+
+TEXT3-H2A-H1
+
+## H2B
+
+TEXT4-H2B-H1
+
+### H3A
+
+TEXT5-H3-H2B-H1
+
+<div>
+  <h4>H4</h4>
+  <p>TEXT7-H4-H3-H2B-H1</p>
+</div>
+
+## H2C
+
+TEXT8-H2C-H1
+
+### H3B `<code>`
+
+TEXT9-H3B-H2C-H1
+

data/spec/fixtures/weight.md

@@ -0,0 +1,19 @@
+---
+title: Weight test
+---
+
+# AAA BBB CCC DDD
+
+aaa xxx aaa xxx aaa
+
+## AAA BBB
+
+aaa bbb
+
+## CCC DDD
+
+ccc ddd
+
+### DDD
+
+aaa bbb ccc dddd