jekyll-algolia 0.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CONTRIBUTING.md +94 -0
- data/README.md +99 -0
- data/errors/invalid_credentials.txt +10 -0
- data/errors/invalid_credentials_for_tmp_index.txt +17 -0
- data/errors/invalid_index_name.txt +11 -0
- data/errors/missing_api_key.txt +17 -0
- data/errors/missing_application_id.txt +12 -0
- data/errors/missing_index_name.txt +19 -0
- data/errors/no_records_found.txt +20 -0
- data/errors/record_too_big.txt +25 -0
- data/errors/unknown_application_id.txt +20 -0
- data/errors/unknown_settings.txt +15 -0
- data/lib/jekyll-algolia.rb +107 -0
- data/lib/jekyll/algolia/configurator.rb +202 -0
- data/lib/jekyll/algolia/error_handler.rb +270 -0
- data/lib/jekyll/algolia/extractor.rb +64 -0
- data/lib/jekyll/algolia/file_browser.rb +269 -0
- data/lib/jekyll/algolia/hooks.rb +67 -0
- data/lib/jekyll/algolia/indexer.rb +258 -0
- data/lib/jekyll/algolia/logger.rb +63 -0
- data/lib/jekyll/algolia/utils.rb +68 -0
- data/lib/jekyll/algolia/version.rb +7 -0
- data/lib/jekyll/commands/algolia.rb +49 -0
- metadata +304 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 8760fc38d0bbef9d96d721d8a4a3434f6e92c90e
|
4
|
+
data.tar.gz: '026308472f72a208686432580d17d592a19cbbd1'
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 9c795d6962ddf9e0dcd5f7b323506272f5af0d26cc170a588de5f01183fad3cf5628b54be0c3dfe842b6c20d14de39fb903b28754c75d1541bae0b62126c07ae
|
7
|
+
data.tar.gz: db570e83b0874e0dc8652ef8411b62b36f932cb1678e933a01dd60ca8a92abf7ecc1d3d881d36f54cf4875ee823474306488818520c624957938de66454f646e
|
data/CONTRIBUTING.md
ADDED
@@ -0,0 +1,94 @@
|
|
1
|
+
Hi collaborator!
|
2
|
+
|
3
|
+
If you have a fix or a new feature, please start by checking in the
|
4
|
+
[issues](https://github.com/algolia/jekyll-algolia/issues) if it is
|
5
|
+
already referenced. If not, feel free to open one.
|
6
|
+
|
7
|
+
We use [pull requests](https://github.com/algolia/jekyll-algolia/pulls)
|
8
|
+
for collaboration. The workflow is as follow:
|
9
|
+
|
10
|
+
- Create a local branch, starting from `develop`
|
11
|
+
- Submit the PR on `develop`
|
12
|
+
- Wait for review
|
13
|
+
- Do the changes requested (if any)
|
14
|
+
- We may ask you to rebase the branch to latest `develop` if it gets out of sync
|
15
|
+
- Receive the thanks of the Algolia team :)
|
16
|
+
|
17
|
+
# Development workflow
|
18
|
+
|
19
|
+
Start by running `bundle install` to get all the dependencies up to date.
|
20
|
+
|
21
|
+
## Testing
|
22
|
+
|
23
|
+
Run `rake test` to launch all tests. You can run `rake test_details` to get an
|
24
|
+
output with more details about the tests.
|
25
|
+
|
26
|
+
## TDD
|
27
|
+
|
28
|
+
// TODO
|
29
|
+
|
30
|
+
## Testing different ruby versions
|
31
|
+
|
32
|
+
You can test the gem across all the supported Ruby versions by running
|
33
|
+
`./scripts/test_all_ruby_versions`. Note that you will need to have RVM
|
34
|
+
installed for this to work.
|
35
|
+
|
36
|
+
## Testing local changes on an existing Jekyll website
|
37
|
+
|
38
|
+
If you want to test the plugin on an existing Jekyll website while developping,
|
39
|
+
I suggest updating the website `Gemfile` to point to the correct local directory
|
40
|
+
|
41
|
+
```ruby
|
42
|
+
group :jekyll_plugins do
|
43
|
+
gem "jekyll-algolia", :path => "/path/to/local/gem/folder"
|
44
|
+
end
|
45
|
+
```
|
46
|
+
|
47
|
+
# Git Hooks
|
48
|
+
|
49
|
+
If you plan on submitting a PR, I suggest you install the git hooks located in
|
50
|
+
`./scripts/git_hook`.
|
51
|
+
|
52
|
+
The easiest way is to create a symlink from your `.git/hooks` folder:
|
53
|
+
|
54
|
+
```sh
|
55
|
+
$ git root
|
56
|
+
$ rm ./.git/hooks
|
57
|
+
$ ln -s ./scripts/git_hooks/ ./.git/hooks
|
58
|
+
```
|
59
|
+
|
60
|
+
# Tagging and releasing
|
61
|
+
|
62
|
+
If you need to release a new version of the gem, run `rake release` from the
|
63
|
+
`develop` branch. It will ask you for the new version and automatically create
|
64
|
+
the git tags, create the gem and push it to Rubygems.
|
65
|
+
|
66
|
+
# Documentation
|
67
|
+
|
68
|
+
## Requirements
|
69
|
+
|
70
|
+
To run this project, you will need:
|
71
|
+
|
72
|
+
- Node.js >= v9.2.0, use nvm - [install instructions](https://github.com/creationix/nvm#install-script)
|
73
|
+
- Yarn >= v1.3.2 - [install instructions](https://yarnpkg.com/en/docs/install#alternatives-tab)
|
74
|
+
|
75
|
+
## Development
|
76
|
+
|
77
|
+
```sh
|
78
|
+
yarn
|
79
|
+
yarn start
|
80
|
+
```
|
81
|
+
|
82
|
+
Go to <http://localhost:3000>.
|
83
|
+
|
84
|
+
## Update docs
|
85
|
+
|
86
|
+
```sh
|
87
|
+
yarn docs:update
|
88
|
+
git push
|
89
|
+
```
|
90
|
+
|
91
|
+
|
92
|
+
# Project owner
|
93
|
+
|
94
|
+
[@pixelastic](https://github.com/pixelastic)
|
data/README.md
ADDED
@@ -0,0 +1,99 @@
|
|
1
|
+
# Jekyll Algolia Plugin
|
2
|
+
|
3
|
+
[![Gem Version][1]](http://badge.fury.io/rb/jekyll-algolia) [![Build
|
4
|
+
Status][2]](https://travis-ci.org/algolia/jekyll-algolia) [![Coverage
|
5
|
+
Status][3]](https://coveralls.io/github/algolia/jekyll-algolia?branch=master)
|
6
|
+
[![Code Climate][4]](https://codeclimate.com/github/algolia/jekyll-algolia)
|
7
|
+
![Jekyll >= 3.6.0][5] ![Ruby >= 2.3.0][6]
|
8
|
+
|
9
|
+
Jekyll plugin to automatically index your content on Algolia.
|
10
|
+
|
11
|
+
## Usage
|
12
|
+
|
13
|
+
```shell
|
14
|
+
$ bundle exec jekyll algolia
|
15
|
+
```
|
16
|
+
|
17
|
+
This will push the content of your Jekyll website to your Algolia index.
|
18
|
+
|
19
|
+
## Documentation
|
20
|
+
|
21
|
+
Official documentation can be found on
|
22
|
+
[https://community.algolia.com/jekyll-algolia/](https://community.algolia.com/jekyll-algolia/)
|
23
|
+
|
24
|
+
## Installation
|
25
|
+
|
26
|
+
The plugin requires at least Jekyll 3.6.0 and Ruby 2.3.0.
|
27
|
+
|
28
|
+
First, add the `jekyll-algolia` gem to your `Gemfile`, in the `:jekyll_plugins`
|
29
|
+
section.
|
30
|
+
|
31
|
+
```ruby
|
32
|
+
# Gemfile
|
33
|
+
|
34
|
+
group :jekyll_plugins do
|
35
|
+
gem 'jekyll-algolia'
|
36
|
+
end
|
37
|
+
```
|
38
|
+
|
39
|
+
Once this is done, download all dependencies with `bundle install`.
|
40
|
+
|
41
|
+
## Basic configuration
|
42
|
+
|
43
|
+
You need to provide certain Algolia credentials for this plugin to *index* your
|
44
|
+
site.
|
45
|
+
|
46
|
+
*If you don't yet have an Algolia account, you can open a free [Community plan
|
47
|
+
here][9]. Once signed in, you can get your credentials from
|
48
|
+
[your dashboard][10].*
|
49
|
+
|
50
|
+
Once you have your credentials, you should define your `application_id` and
|
51
|
+
`index_name` inside your `_config.yml` file like this:
|
52
|
+
|
53
|
+
```yaml
|
54
|
+
# _config.yml
|
55
|
+
|
56
|
+
algolia:
|
57
|
+
application_id: 'your_application_id'
|
58
|
+
index_name: 'your_index_name'
|
59
|
+
```
|
60
|
+
|
61
|
+
## Run it
|
62
|
+
|
63
|
+
Once your credentials are setup, you can run the indexing by running the
|
64
|
+
following command:
|
65
|
+
|
66
|
+
```shell
|
67
|
+
ALGOLIA_API_KEY='{your_admin_api_key}' bundle exec jekyll algolia
|
68
|
+
```
|
69
|
+
|
70
|
+
Note that `ALGOLIA_API_KEY` should be set to your admin API key.
|
71
|
+
|
72
|
+
# Thanks
|
73
|
+
|
74
|
+
Thanks to [Anatoliy Yastreb][21] for a [great tutorial][22] on creating Jekyll
|
75
|
+
plugins.
|
76
|
+
|
77
|
+
|
78
|
+
[1]: https://badge.fury.io/rb/jekyll-algolia.svg
|
79
|
+
[2]: https://travis-ci.org/algolia/jekyll-algolia.svg?branch=master
|
80
|
+
[3]: https://coveralls.io/repos/algolia/jekyll-algolia/badge.svg?branch=master&service=github
|
81
|
+
[4]: https://codeclimate.com/github/algolia/jekyll-algolia/badges/gpa.svg
|
82
|
+
[5]: https://img.shields.io/badge/jekyll-%3E%3D%203.6.0-green.svg
|
83
|
+
[6]: https://img.shields.io/badge/ruby-%3E%3D%202.3.0-green.svg
|
84
|
+
[7]: https://pages.github.com/versions.json
|
85
|
+
[8]: http://bundler.io/
|
86
|
+
[9]: https://www.algolia.com/users/sign_up/hacker
|
87
|
+
[10]: https://www.algolia.com/licensing
|
88
|
+
[11]: http://www.methods.co.nz/asciidoc/
|
89
|
+
[12]: https://github.com/textile
|
90
|
+
[13]: https://www.algolia.com/doc/api-reference/api-methods/set-settings/?language=ruby#set-settings
|
91
|
+
[14]: https://www.algolia.com/doc/javascript
|
92
|
+
[15]: https://github.com/algolia/hyde
|
93
|
+
[16]: https://travis-ci.org/
|
94
|
+
[17]: https://travis-ci.org/
|
95
|
+
[18]: http://docs.travis-ci.com/user/environment-variables/
|
96
|
+
[19]: /docs/travis-settings.png
|
97
|
+
[20]: https://travis-ci.org
|
98
|
+
[21]: https://github.com/ayastreb/
|
99
|
+
[22]: https://ayastreb.me/writing-a-jekyll-plugin/
|
@@ -0,0 +1,10 @@
|
|
1
|
+
E: [✗ Error] Invalid credentials
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not connect to your application ID using the
|
4
|
+
E: API key your provided.
|
5
|
+
W:
|
6
|
+
W: Make sure your API key has access to your {application_id} application.
|
7
|
+
I:
|
8
|
+
I: You can find your API key in your Algolia dashboard here:
|
9
|
+
I: https://www.algolia.com/licensing
|
10
|
+
I:
|
@@ -0,0 +1,17 @@
|
|
1
|
+
E: [✗ Error] Invalid credentials for temporary index
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not access your index with the API key you
|
4
|
+
E: provided.
|
5
|
+
W:
|
6
|
+
W: When using the `atomic` indexing mode, we will push all your content to a
|
7
|
+
W: temporary index, then overwrite the actual index with it. The API key you
|
8
|
+
W: defined should have access rights on both.
|
9
|
+
I:
|
10
|
+
I: Make sure your API key has access:
|
11
|
+
I: - {index_name}
|
12
|
+
I: - {index_name_tmp}
|
13
|
+
I:
|
14
|
+
I: You can configure API keys from your dashboard:
|
15
|
+
I: https://www.algolia.com/apps/{application_id}/api-keys/restricted
|
16
|
+
I:
|
17
|
+
I:
|
@@ -0,0 +1,11 @@
|
|
1
|
+
E: [✗ Error] Invalid index name
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could push records to your index as its name
|
4
|
+
E: contains invalid characters.
|
5
|
+
W:
|
6
|
+
W: Some special characters are not allowed in the naming of indices and your
|
7
|
+
W: index {index_name} contains some of them.
|
8
|
+
I:
|
9
|
+
I: Please, check our FAQ for more details:
|
10
|
+
I: https://www.algolia.com/doc/faq/index-configuration/what-can-i-name-my-indices/
|
11
|
+
I:
|
@@ -0,0 +1,17 @@
|
|
1
|
+
E: [✗ Error] Missing API key
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not find your API key.
|
4
|
+
W:
|
5
|
+
W: Please, define your API key either by:
|
6
|
+
W:
|
7
|
+
W: 1/ Defining an ENV variable when calling `jekyll algolia`
|
8
|
+
W: $ ALGOLIA_API_KEY='{your_api_key}' jekyll algolia
|
9
|
+
W:
|
10
|
+
W: 2/ Save your API key in a named `_algolia_api_key` in your source directory.
|
11
|
+
W: If you do this, we strongly recommend you to NOT track this file in your
|
12
|
+
W: versionning system.
|
13
|
+
I:
|
14
|
+
I: You can find your API key in your Algolia dashboard here:
|
15
|
+
I: https://www.algolia.com/licensing
|
16
|
+
I:
|
17
|
+
I:
|
@@ -0,0 +1,12 @@
|
|
1
|
+
E: [✗ Error] No application ID defined
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not find your Algolia application ID.
|
4
|
+
W:
|
5
|
+
W: Please, define it in your Jekyll _config.yml file like this:
|
6
|
+
W:
|
7
|
+
W: algolia:
|
8
|
+
W: application_id: {your_application_id}
|
9
|
+
I:
|
10
|
+
I: You can find your application ID along with all your credentials in your
|
11
|
+
I: Algolia dashboard here:
|
12
|
+
I: https://www.algolia.com/licensing
|
@@ -0,0 +1,19 @@
|
|
1
|
+
E: [✗ Error] No index name defined
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not find the name of the Algolia index you
|
4
|
+
E: want to push your records to.
|
5
|
+
W:
|
6
|
+
W: Please, define it in your Jekyll _config.yml file like this:
|
7
|
+
W:
|
8
|
+
W: algolia:
|
9
|
+
W: index_name: {your_index_name}
|
10
|
+
W:
|
11
|
+
W: Alternatively, you can also define it as an ENV variable, like this:
|
12
|
+
W:
|
13
|
+
W: $ ALGOLIA_INDEX_NAME='{your_index_name}' jekyll algolia
|
14
|
+
W:
|
15
|
+
I: You can see all your indices from your Algolia dashboard here:
|
16
|
+
I: https://www.algolia.com/explorer
|
17
|
+
I:
|
18
|
+
I: Note that you don't have to create an index before pushing records.
|
19
|
+
I: It will be created automatically if it does not exist yet.
|
@@ -0,0 +1,20 @@
|
|
1
|
+
E: [✗ Error] No records found
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could find any records to index
|
4
|
+
W:
|
5
|
+
W: The plugin tried to extract records from the pages, posts and collections of
|
6
|
+
W: your site but could not find anything to index.
|
7
|
+
I:
|
8
|
+
I: Make sure you did not exclude too many files from indexing using the
|
9
|
+
I: `files_to_exclude` option. You are currently excluding the following files:
|
10
|
+
I: {files_to_exclude}
|
11
|
+
I:
|
12
|
+
I: Also double check that your current value for `nodes_to_index` can actually
|
13
|
+
I: match something in the page. You are current indexing the following nodes:
|
14
|
+
I: {nodes_to_index}
|
15
|
+
I:
|
16
|
+
I: Note that all the markup that is defined in the layouts won't be available
|
17
|
+
I: during extraction. Only the page content can be accessed. So if you defined
|
18
|
+
I: a layout markup that is used only for one page, you should move it to the
|
19
|
+
I: page instead.
|
20
|
+
I:
|
@@ -0,0 +1,25 @@
|
|
1
|
+
E: [✗ Error] Record is too big
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not push one of your records as it exceeds
|
4
|
+
E: the {size_limit} size limit.
|
5
|
+
W:
|
6
|
+
W: The plugin will create one record for each element matching your
|
7
|
+
W: `nodes_to_index` value (currently set to "{nodes_to_index}"). Each record
|
8
|
+
W: should not weight more than {size_limit}. One of your records weights {size}
|
9
|
+
W: and has been rejected.
|
10
|
+
W:
|
11
|
+
W: Here are more information about the rejected record:
|
12
|
+
W: {
|
13
|
+
W: "objectID": "{object_id}",
|
14
|
+
W: "title": "{object_title}",
|
15
|
+
W: "url": "{object_url}",
|
16
|
+
W: "text": "{object_hint}…",
|
17
|
+
W: […]
|
18
|
+
W: }
|
19
|
+
W:
|
20
|
+
I: This issue is sometimes caused by malformed HTML preventing the parser to
|
21
|
+
I: correctly grab the content of the nodes.
|
22
|
+
I:
|
23
|
+
I: If you're having trouble solving this issue, feel free to file a bug on
|
24
|
+
I: GitHub, ideally with a link to a repository where we can reproduce the issue.
|
25
|
+
I: https://github.com/algolia/jekyll-algolia/issues
|
@@ -0,0 +1,20 @@
|
|
1
|
+
E: [✗ Error] Unreachable server
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not contact the server hosting your
|
4
|
+
E: application.
|
5
|
+
W:
|
6
|
+
W: Make sure you correctly typed your application ID. As we are using the
|
7
|
+
W: application ID as part of the server url, any typo in the application ID will
|
8
|
+
W: prevent us from reaching your server.
|
9
|
+
W:
|
10
|
+
I: Here is the application ID you defined: {application_id}
|
11
|
+
I:
|
12
|
+
I: Make sure it's the same as the one displayed in your dashboard:
|
13
|
+
I: https://www.algolia.com/licensing
|
14
|
+
I:
|
15
|
+
I: Then, define it in your Jekyll _config.yml file like this:
|
16
|
+
I:
|
17
|
+
I: algolia:
|
18
|
+
I: application_id: {your_application_id}
|
19
|
+
I:
|
20
|
+
I:
|
@@ -0,0 +1,15 @@
|
|
1
|
+
E: [✗ Error] Unknown setting
|
2
|
+
E:
|
3
|
+
E: The jekyll-algolia plugin could not correctly configure your index as some of
|
4
|
+
E: the settings passed are not recognized.
|
5
|
+
W:
|
6
|
+
W: It seems that one of the custom index settings you defined was not recognized
|
7
|
+
W: by the API and was rejected:
|
8
|
+
W: {setting_name}: {setting_value}
|
9
|
+
I:
|
10
|
+
I: Make sure the setting name and value are correct. You can find a list of all
|
11
|
+
I: the available settings with their documentation in our API reference:
|
12
|
+
I: https://www.algolia.com/doc/api-reference/api-parameters/
|
13
|
+
I: Or specifically for this setting:
|
14
|
+
I: https://www.algolia.com/doc/api-reference/api-parameters/{setting_name}/
|
15
|
+
I:
|
@@ -0,0 +1,107 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'jekyll/commands/algolia'
|
4
|
+
|
5
|
+
module Jekyll
|
6
|
+
# Requirable file, loading all dependencies.
|
7
|
+
# Methods here are called by the main `jekyll algolia` command
|
8
|
+
module Algolia
|
9
|
+
require 'jekyll/algolia/version'
|
10
|
+
require 'jekyll/algolia/utils'
|
11
|
+
require 'jekyll/algolia/hooks'
|
12
|
+
require 'jekyll/algolia/configurator'
|
13
|
+
require 'jekyll/algolia/logger'
|
14
|
+
require 'jekyll/algolia/error_handler'
|
15
|
+
require 'jekyll/algolia/file_browser'
|
16
|
+
require 'jekyll/algolia/extractor'
|
17
|
+
require 'jekyll/algolia/indexer'
|
18
|
+
|
19
|
+
@config = {}
|
20
|
+
|
21
|
+
# Public: Init the Algolia module
|
22
|
+
#
|
23
|
+
# config - A hash of Jekyll config option (merge of _config.yml options and
|
24
|
+
# options passed on the command line)
|
25
|
+
#
|
26
|
+
# The gist of the plugin works by instanciating a Jekyll site,
|
27
|
+
# monkey-patching its `write` method and building it.
|
28
|
+
def self.init(config = {})
|
29
|
+
@config = config
|
30
|
+
@site = Jekyll::Algolia::Site.new(@config)
|
31
|
+
|
32
|
+
exit 1 unless Configurator.assert_valid_credentials
|
33
|
+
|
34
|
+
self
|
35
|
+
end
|
36
|
+
|
37
|
+
# Public: Run the main Algolia module
|
38
|
+
#
|
39
|
+
# Actually "process" the site, which will acts just like a regular `jekyll
|
40
|
+
# build` except that our monkey patched `write` method will be called
|
41
|
+
# instead.
|
42
|
+
#
|
43
|
+
# Note: The internal list of files to be processed will only be created when
|
44
|
+
# calling .process
|
45
|
+
def self.run
|
46
|
+
@site.process
|
47
|
+
end
|
48
|
+
|
49
|
+
# Public: Get access to the Jekyll config
|
50
|
+
#
|
51
|
+
# All other classes will need access to this config, so we make it publicly
|
52
|
+
# accessible
|
53
|
+
def self.config
|
54
|
+
@config
|
55
|
+
end
|
56
|
+
|
57
|
+
# Public: Get access to the Jekyll site
|
58
|
+
#
|
59
|
+
# Tests will need access to the inner Jekyll website so we expose it here
|
60
|
+
def self.site
|
61
|
+
@site
|
62
|
+
end
|
63
|
+
|
64
|
+
# A Jekyll::Site subclass that overrides #write from the parent class to
|
65
|
+
# create JSON records out of rendered documents and push those records
|
66
|
+
# to Algolia instead of writing files to disk.
|
67
|
+
class Site < Jekyll::Site
|
68
|
+
def write
|
69
|
+
if Configurator.dry_run?
|
70
|
+
Logger.log('W:==== THIS IS A DRY RUN ====')
|
71
|
+
Logger.log('W: - No records will be pushed to your index')
|
72
|
+
Logger.log('W: - No settings will be updated on your index')
|
73
|
+
end
|
74
|
+
|
75
|
+
records = []
|
76
|
+
files = []
|
77
|
+
each_site_file do |file|
|
78
|
+
# Skip files that should not be indexed
|
79
|
+
is_indexable = FileBrowser.indexable?(file)
|
80
|
+
unless is_indexable
|
81
|
+
Logger.verbose("W:Skipping #{file.path}")
|
82
|
+
next
|
83
|
+
end
|
84
|
+
|
85
|
+
path = FileBrowser.path_from_root(file)
|
86
|
+
Logger.verbose("I:Extracting records from #{path}")
|
87
|
+
file_records = Extractor.run(file)
|
88
|
+
|
89
|
+
files << file
|
90
|
+
records += file_records
|
91
|
+
end
|
92
|
+
|
93
|
+
# Applying the user hook on the whole list of records
|
94
|
+
records = Hooks.apply_all(records)
|
95
|
+
|
96
|
+
# Adding a unique objectID to each record
|
97
|
+
records.map! do |record|
|
98
|
+
Extractor.add_unique_object_id(record)
|
99
|
+
end
|
100
|
+
|
101
|
+
Logger.verbose("I:Found #{files.length} files")
|
102
|
+
|
103
|
+
Indexer.run(records)
|
104
|
+
end
|
105
|
+
end
|
106
|
+
end
|
107
|
+
end
|