middleman-robots 1.2.3 → 1.3.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (21) hide show
  1. checksums.yaml +4 -4
  2. data/.travis.yml +5 -3
  3. data/README.jp.md +17 -20
  4. data/README.md +17 -26
  5. data/Rakefile +5 -2
  6. data/features/{robots.feature → build.feature} +65 -94
  7. data/features/server.feature +22 -0
  8. data/fixtures/server-app/config.rb +14 -0
  9. data/fixtures/server-app/source/index.html.erb +11 -0
  10. data/fixtures/server-app/source/layouts/layout.erb +20 -0
  11. data/fixtures/server-app/source/stylesheets/all.css +56 -0
  12. data/fixtures/server-app/source/stylesheets/normalize.css +376 -0
  13. data/lib/middleman-robots/extension.rb +15 -53
  14. data/lib/middleman-robots/generator.rb +42 -0
  15. data/lib/middleman-robots/group.rb +50 -0
  16. data/lib/middleman-robots/templates/robots.txt.erb +1 -0
  17. data/lib/middleman-robots/version.rb +1 -1
  18. data/middleman-robots.gemspec +2 -2
  19. data/tests/test_generator.rb +37 -0
  20. data/tests/test_group.rb +121 -0
  21. metadata +19 -7
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: c2d26e3baf2a7ec3242c91d76fb58aaae7cc618f
4
- data.tar.gz: edfa98b114b84e2cc0187c6a27fe0681273d0ab5
3
+ metadata.gz: 5563b4e8d0e5602b0c96be8f6863d39fa0a85386
4
+ data.tar.gz: 03f2c4966146975d82e40ae45310ad1a4abbebe9
5
5
  SHA512:
6
- metadata.gz: 4d85535162e8a1107ac75d5725f29b4f29d4f1aeda9ce5bd667cdd8c4516cfb5e8d8790c852fb23931573ed68e41997b396b0ebbedf167992ed00d08b5d2bcca
7
- data.tar.gz: 6994c8fb0a973c7cf2376fd5192a1df8c886a24fad5b57de80a498b73865c3289cf16ca4878a277d330731a06e4ff06f2d39fda7381628d8f38fda528853f8e9
6
+ metadata.gz: 32fbac66c9f171aebf12e5e8cffa31770976842a82e0c65b54570695557db15c15337b265188d327352950e58e75a7bea72b209a58e555bb35699ee247ecf211
7
+ data.tar.gz: 17b99bc1cbb52ae6022211c683bf9d7fdbd8fd526399a4a163e9355d49a4b7a8a746ed747ff3db00487ea2b563f9a0a411862623eba1352ce002e624502258ed
data/.travis.yml CHANGED
@@ -1,17 +1,19 @@
1
1
  language: ruby
2
2
  sudo: false
3
+ cache: bundler
4
+ before_script:
5
+ - bundle update
3
6
  rvm:
4
7
  - ruby-head
8
+ - 2.4.0
5
9
  - 2.3.1
6
10
  - 2.2.4
7
- - 2.1
8
- - 2.0
9
11
  os:
10
12
  - linux
11
13
  matrix:
12
14
  fast_finish: true
13
15
  allow_failures:
14
- - rvm: jruby-head
16
+ - rvm: jruby-head
15
17
  script: bundle exec rake test
16
18
  cache: bundler
17
19
  env: TEST=true
data/README.jp.md CHANGED
@@ -3,7 +3,7 @@
3
3
  [![Gem Version](https://badge.fury.io/rb/middleman-robots.svg)](http://badge.fury.io/rb/middleman-robots)
4
4
  [![Build Status](https://travis-ci.org/yterajima/middleman-robots.svg?branch=master)](https://travis-ci.org/yterajima/middleman-robots)
5
5
 
6
- `middleman-robots` は [Middleman](http://middlemanapp.com/) の拡張機能です。 build 実行時に `config.rb` に記述されたルールに基いて `robots.txt` を作ります。
6
+ `middleman-robots` は [Middleman](http://middlemanapp.com/) の拡張機能です。 `config.rb` に記述されたルールに基いて `robots.txt` を作ります。
7
7
 
8
8
  ## Installation
9
9
 
@@ -27,12 +27,11 @@ Gemfile を使わずにインストールする場合は次のコマンドを実
27
27
 
28
28
  ```ruby
29
29
  # config.rb
30
- configure :build do
31
- activate :robots, :rules => [
30
+ activate :robots,
31
+ :rules => [
32
32
  {:user_agent => '*', :allow => %w(/)}
33
33
  ],
34
34
  :sitemap => "http://example.com/sitemap.xml"
35
- end
36
35
  ```
37
36
 
38
37
  作成される `robots.txt`:
@@ -50,22 +49,20 @@ Sitemap: http://example.com/sitemap.xml
50
49
 
51
50
  ```ruby
52
51
  # config.rb
53
- configure :build do
54
- activate :robots,
55
- :rules => [
56
- {
57
- :user_agent => 'Googlebot',
58
- :disallow => %w(tmp/ /something/dir/file_disallow.html),
59
- :allow => %w(allow/ /something/dir/file_allow.html)
60
- },
61
- {
62
- :user_agent => 'Googlebot-Image',
63
- :disallow => %w(tmp/ /something/dir/file_disallow.html),
64
- :allow => %w(allow/ /something/dir/file_allow.html)
65
- }
66
- ],
67
- :sitemap => "http://example.com/sitemap.xml"
68
- end
52
+ activate :robots,
53
+ :rules => [
54
+ {
55
+ :user_agent => 'Googlebot',
56
+ :disallow => %w(tmp/ /something/dir/file_disallow.html),
57
+ :allow => %w(allow/ /something/dir/file_allow.html)
58
+ },
59
+ {
60
+ :user_agent => 'Googlebot-Image',
61
+ :disallow => %w(tmp/ /something/dir/file_disallow.html),
62
+ :allow => %w(allow/ /something/dir/file_allow.html)
63
+ }
64
+ ],
65
+ :sitemap => "http://example.com/sitemap.xml"
69
66
  ```
70
67
 
71
68
  作成される `robots.txt`:
data/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  [![Gem Version](https://badge.fury.io/rb/middleman-robots.svg)](http://badge.fury.io/rb/middleman-robots)
4
4
  [![Build Status](https://travis-ci.org/yterajima/middleman-robots.svg?branch=master)](https://travis-ci.org/yterajima/middleman-robots)
5
5
 
6
- `middleman-robots` is an extension of [Middleman](http://middlemanapp.com/). This can create `robots.txt` when build.
6
+ `middleman-robots` is an extension of [Middleman](http://middlemanapp.com/). This can create `robots.txt`.
7
7
 
8
8
  This plugin support Middleman v3-stable and v4.
9
9
 
@@ -23,20 +23,13 @@ Or install it yourself as:
23
23
 
24
24
  $ gem install middleman-robots
25
25
 
26
- ## Usage
27
- Important - if you set the `:build_dir` variable, make sure you set it before the `middleman-robots` settings code, otherwise default `/build` directory is used.
28
- Basic usage:
29
-
30
26
  ```ruby
31
27
  # config.rb
32
- configure :build do
33
- set :build_dir, 'some-path/' #optional
34
-
35
- activate :robots, :rules => [
28
+ activate :robots,
29
+ :rules => [
36
30
  {:user_agent => '*', :allow => %w(/)}
37
31
  ],
38
32
  :sitemap => "http://example.com/sitemap.xml"
39
- end
40
33
  ```
41
34
 
42
35
  Created `robots.txt`:
@@ -53,22 +46,20 @@ You can use options, `:rules` {[`:user_agent`(string), `:allow`(array), `:disall
53
46
 
54
47
  ```ruby
55
48
  # config.rb
56
- configure :build do
57
- activate :robots,
58
- :rules => [
59
- {
60
- :user_agent => 'Googlebot',
61
- :disallow => %w(tmp/ /something/dir/file_disallow.html),
62
- :allow => %w(allow/ /something/dir/file_allow.html)
63
- },
64
- {
65
- :user_agent => 'Googlebot-Image',
66
- :disallow => %w(tmp/ /something/dir/file_disallow.html),
67
- :allow => %w(allow/ /something/dir/file_allow.html)
68
- }
69
- ],
70
- :sitemap => "http://example.com/sitemap.xml"
71
- end
49
+ activate :robots,
50
+ :rules => [
51
+ {
52
+ :user_agent => 'Googlebot',
53
+ :disallow => %w(tmp/ /something/dir/file_disallow.html),
54
+ :allow => %w(allow/ /something/dir/file_allow.html)
55
+ },
56
+ {
57
+ :user_agent => 'Googlebot-Image',
58
+ :disallow => %w(tmp/ /something/dir/file_disallow.html),
59
+ :allow => %w(allow/ /something/dir/file_allow.html)
60
+ }
61
+ ],
62
+ :sitemap => "http://example.com/sitemap.xml"
72
63
  ```
73
64
 
74
65
  Created `robots.txt`:
data/Rakefile CHANGED
@@ -1,8 +1,11 @@
1
- require "bundler/gem_tasks"
1
+ require 'bundler/gem_tasks'
2
2
 
3
3
  task default: :test
4
4
 
5
5
  desc 'test command'
6
6
  task :test do
7
- sh 'cucumber features/'
7
+ Dir.glob('tests/test_*.rb') do |f|
8
+ sh "bundle exec ruby #{f}"
9
+ end
10
+ sh 'bundle exec cucumber features/'
8
11
  end
data/features/{robots.feature → build.feature} RENAMED
@@ -1,46 +1,42 @@
1
- Feature: Middleman-Robots
1
+ Feature: Middleman-Robots on build
2
2
 
3
3
  Scenario: Empty Usage
4
4
  Given a fixture app "basic-app"
5
5
  And a file named "config.rb" with:
6
6
  """
7
- configure :build do
8
- activate :robots
9
- end
7
+ activate :robots
10
8
  """
11
9
  And a successfully built app at "basic-app"
12
10
  When I cd to "build"
13
11
  Then a file named "robots.txt" should exist
14
- And the output should contain "== middleman-robots: robots.txt created =="
12
+ And the output should contain "== middleman-robots: robots.txt added to resources =="
13
+ And the file "robots.txt" should contain exactly:
14
+ """
15
+ """
15
16
 
16
17
  Scenario: Rules option with user_agent
17
18
  Given a fixture app "basic-app"
18
19
  And a file named "config.rb" with:
19
20
  """
20
- configure :build do
21
- activate :robots, :rules => [
22
- {:user_agent => '*'}
23
- ]
24
- end
21
+ activate :robots, :rules => [
22
+ {:user_agent => '*'}
23
+ ]
25
24
  """
26
25
  And a successfully built app at "basic-app"
27
26
  When I cd to "build"
28
27
  Then the file "robots.txt" should contain exactly:
29
28
  """
30
29
  User-Agent: *
31
-
32
30
  """
33
31
 
34
32
  Scenario: Rules option with user_agent using block
35
33
  Given a fixture app "basic-app"
36
34
  And a file named "config.rb" with:
37
35
  """
38
- configure :build do
39
- activate :robots do |r|
40
- r.rules = [
41
- {:user_agent => '*'}
42
- ]
43
- end
36
+ activate :robots do |r|
37
+ r.rules = [
38
+ {:user_agent => '*'}
39
+ ]
44
40
  end
45
41
  """
46
42
  And a successfully built app at "basic-app"
@@ -48,57 +44,48 @@ Feature: Middleman-Robots
48
44
  Then the file "robots.txt" should contain exactly:
49
45
  """
50
46
  User-Agent: *
51
-
52
47
  """
53
48
 
54
49
  Scenario: Rules option with user_agent
55
50
  Given a fixture app "basic-app"
56
51
  And a file named "config.rb" with:
57
52
  """
58
- configure :build do
59
- activate :robots, :rules => [
60
- {:user_agent => '*'}
61
- ]
62
- end
53
+ activate :robots, :rules => [
54
+ {:user_agent => '*'}
55
+ ]
63
56
  """
64
57
  And a successfully built app at "basic-app"
65
58
  When I cd to "build"
66
59
  Then the file "robots.txt" should contain exactly:
67
60
  """
68
61
  User-Agent: *
69
-
70
62
  """
71
63
 
72
64
  Scenario: Rules option with user-agent
73
65
  Given a fixture app "basic-app"
74
66
  And a file named "config.rb" with:
75
67
  """
76
- configure :build do
77
- activate :robots, :rules => [
78
- {'user-agent' => '*'}
79
- ]
80
- end
68
+ activate :robots, :rules => [
69
+ {'user-agent' => '*'}
70
+ ]
81
71
  """
82
72
  And a successfully built app at "basic-app"
83
73
  When I cd to "build"
84
74
  Then the file "robots.txt" should contain exactly:
85
75
  """
86
76
  User-Agent: *
87
-
88
77
  """
89
78
 
90
79
  Scenario: Rules option with Disallow
91
80
  Given a fixture app "basic-app"
92
81
  And a file named "config.rb" with:
93
82
  """
94
- configure :build do
95
- activate :robots, :rules => [
96
- {
97
- :user_agent => '*',
98
- :disallow => %w(tmp/* /something/dir/file_disallow.html)
99
- }
100
- ]
101
- end
83
+ activate :robots, :rules => [
84
+ {
85
+ :user_agent => '*',
86
+ :disallow => %w(tmp/* /something/dir/file_disallow.html)
87
+ }
88
+ ]
102
89
  """
103
90
  And a successfully built app at "basic-app"
104
91
  When I cd to "build"
@@ -107,21 +94,18 @@ Feature: Middleman-Robots
107
94
  User-Agent: *
108
95
  Disallow: /tmp/*
109
96
  Disallow: /something/dir/file_disallow.html
110
-
111
97
  """
112
98
 
113
99
  Scenario: Rules option with Allow
114
100
  Given a fixture app "basic-app"
115
101
  And a file named "config.rb" with:
116
102
  """
117
- configure :build do
118
- activate :robots, :rules => [
119
- {
120
- :user_agent => '*',
121
- :allow => %w(allow/* /something/dir/file_allow.html)
122
- }
123
- ]
124
- end
103
+ activate :robots, :rules => [
104
+ {
105
+ :user_agent => '*',
106
+ :allow => %w(allow/* /something/dir/file_allow.html)
107
+ }
108
+ ]
125
109
  """
126
110
  And a successfully built app at "basic-app"
127
111
  When I cd to "build"
@@ -130,22 +114,19 @@ Feature: Middleman-Robots
130
114
  User-Agent: *
131
115
  Allow: /allow/*
132
116
  Allow: /something/dir/file_allow.html
133
-
134
117
  """
135
118
 
136
119
  Scenario: All Rules
137
120
  Given a fixture app "basic-app"
138
121
  And a file named "config.rb" with:
139
122
  """
140
- configure :build do
141
- activate :robots, :rules => [
142
- {
143
- :user_agent => '*',
144
- :disallow => %w(tmp/* /something/dir/file_disallow.html),
145
- :allow => %w(allow/* /something/dir/file_allow.html)
146
- }
147
- ]
148
- end
123
+ activate :robots, :rules => [
124
+ {
125
+ :user_agent => '*',
126
+ :disallow => %w(tmp/* /something/dir/file_disallow.html),
127
+ :allow => %w(allow/* /something/dir/file_allow.html)
128
+ }
129
+ ]
149
130
  """
150
131
  And a successfully built app at "basic-app"
151
132
  When I cd to "build"
@@ -156,27 +137,24 @@ Feature: Middleman-Robots
156
137
  Disallow: /something/dir/file_disallow.html
157
138
  Allow: /allow/*
158
139
  Allow: /something/dir/file_allow.html
159
-
160
140
  """
161
141
 
162
142
  Scenario: Multiple Rules
163
143
  Given a fixture app "basic-app"
164
144
  And a file named "config.rb" with:
165
145
  """
166
- configure :build do
167
- activate :robots, :rules => [
168
- {
169
- :user_agent => 'Googlebot',
170
- :disallow => %w(tmp/* /something/dir/file_disallow.html),
171
- :allow => %w(allow/* /something/dir/file_allow.html)
172
- },
173
- {
174
- :user_agent => 'Googlebot-Image',
175
- :disallow => %w(tmp/* /something/dir/file_disallow.html),
176
- :allow => %w(allow/* /something/dir/file_allow.html)
177
- }
178
- ]
179
- end
146
+ activate :robots, :rules => [
147
+ {
148
+ :user_agent => 'Googlebot',
149
+ :disallow => %w(tmp/* /something/dir/file_disallow.html),
150
+ :allow => %w(allow/* /something/dir/file_allow.html)
151
+ },
152
+ {
153
+ :user_agent => 'Googlebot-Image',
154
+ :disallow => %w(tmp/* /something/dir/file_disallow.html),
155
+ :allow => %w(allow/* /something/dir/file_allow.html)
156
+ }
157
+ ]
180
158
  """
181
159
  And a successfully built app at "basic-app"
182
160
  When I cd to "build"
@@ -193,45 +171,39 @@ Feature: Middleman-Robots
193
171
  Disallow: /something/dir/file_disallow.html
194
172
  Allow: /allow/*
195
173
  Allow: /something/dir/file_allow.html
196
-
197
174
  """
198
175
 
199
176
  Scenario: Sitemap option
200
177
  Given a fixture app "basic-app"
201
178
  And a file named "config.rb" with:
202
179
  """
203
- configure :build do
204
- activate :robots, :sitemap => "http://example.com/sitemap.xml"
205
- end
180
+ activate :robots, :sitemap => "http://example.com/sitemap.xml"
206
181
  """
207
182
  And a successfully built app at "basic-app"
208
183
  When I cd to "build"
209
184
  Then the file "robots.txt" should contain exactly:
210
185
  """
211
186
  Sitemap: http://example.com/sitemap.xml
212
-
213
187
  """
214
188
 
215
189
  Scenario: All options
216
190
  Given a fixture app "basic-app"
217
191
  And a file named "config.rb" with:
218
192
  """
219
- configure :build do
220
- activate :robots,
221
- :rules => [
222
- {
223
- :user_agent => 'Googlebot',
224
- :disallow => %w(tmp/* /something/dir/file_disallow.html),
225
- :allow => %w(allow/* /something/dir/file_allow.html)
226
- },
227
- {
228
- :user_agent => 'Googlebot-Image',
229
- :disallow => %w(tmp/* /something/dir/file_disallow.html),
230
- :allow => %w(allow/* /something/dir/file_allow.html)
231
- }
232
- ],
233
- :sitemap => "http://example.com/sitemap.xml"
234
- end
193
+ activate :robots,
194
+ :rules => [
195
+ {
196
+ :user_agent => 'Googlebot',
197
+ :disallow => %w(tmp/* /something/dir/file_disallow.html),
198
+ :allow => %w(allow/* /something/dir/file_allow.html)
199
+ },
200
+ {
201
+ :user_agent => 'Googlebot-Image',
202
+ :disallow => %w(tmp/* /something/dir/file_disallow.html),
203
+ :allow => %w(allow/* /something/dir/file_allow.html)
204
+ }
205
+ ],
206
+ :sitemap => "http://example.com/sitemap.xml"
235
207
  """
236
208
  And a successfully built app at "basic-app"
237
209
  When I cd to "build"
@@ -250,6 +222,5 @@ Feature: Middleman-Robots
250
222
  Allow: /something/dir/file_allow.html
251
223
 
252
224
  Sitemap: http://example.com/sitemap.xml
253
-
254
225
  """
255
226