sitemap_generator 6.3.0 → 7.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGES.md +107 -61
- data/README.md +176 -171
- data/VERSION +1 -1
- data/lib/capistrano/sitemap_generator.rb +2 -0
- data/lib/sitemap_generator/adapters/active_storage_adapter.rb +31 -0
- data/lib/sitemap_generator/adapters/aws_sdk_adapter.rb +29 -13
- data/lib/sitemap_generator/adapters/file_adapter.rb +12 -13
- data/lib/sitemap_generator/adapters/fog_adapter.rb +9 -7
- data/lib/sitemap_generator/adapters/google_storage_adapter.rb +8 -5
- data/lib/sitemap_generator/adapters/s3_adapter.rb +23 -18
- data/lib/sitemap_generator/adapters/wave_adapter.rb +8 -6
- data/lib/sitemap_generator/application.rb +9 -3
- data/lib/sitemap_generator/builder/sitemap_file.rb +41 -31
- data/lib/sitemap_generator/builder/sitemap_index_file.rb +31 -23
- data/lib/sitemap_generator/builder/sitemap_index_url.rb +10 -9
- data/lib/sitemap_generator/builder/sitemap_url.rb +130 -80
- data/lib/sitemap_generator/builder.rb +7 -3
- data/lib/sitemap_generator/core_ext/big_decimal.rb +40 -33
- data/lib/sitemap_generator/core_ext/numeric.rb +52 -46
- data/lib/sitemap_generator/core_ext.rb +2 -0
- data/lib/sitemap_generator/helpers/number_helper.rb +84 -76
- data/lib/sitemap_generator/interpreter.rb +22 -18
- data/lib/sitemap_generator/link_set.rb +114 -113
- data/lib/sitemap_generator/railtie.rb +4 -1
- data/lib/sitemap_generator/simple_namer.rb +8 -6
- data/lib/sitemap_generator/sitemap_location.rb +52 -40
- data/lib/sitemap_generator/tasks.rb +16 -26
- data/lib/sitemap_generator/templates.rb +11 -9
- data/lib/sitemap_generator/utilities.rb +46 -24
- data/lib/sitemap_generator.rb +42 -27
- data/lib/tasks/sitemap_generator_tasks.rake +2 -0
- data/rails/install.rb +2 -0
- data/rails/uninstall.rb +2 -0
- data/templates/sitemap.rb +3 -1
- metadata +7 -131
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# SitemapGenerator
|
|
2
2
|
|
|
3
|
-
[](https://github.com/kjvarga/sitemap_generator/actions/workflows/ci.yml)
|
|
4
4
|
|
|
5
5
|
SitemapGenerator is the easiest way to generate Sitemaps in Ruby. Rails integration provides access to the Rails route helpers within your sitemap config file and automatically makes the rake tasks available to you. Or if you prefer to use another framework, you can! You can use the rake tasks provided or run your sitemap configs as plain ruby scripts.
|
|
6
6
|
|
|
@@ -9,16 +9,17 @@ Sitemaps adhere to the [Sitemap 0.9 protocol][sitemap_protocol] specification.
|
|
|
9
9
|
## Features
|
|
10
10
|
|
|
11
11
|
* Framework agnostic
|
|
12
|
-
* Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images],
|
|
12
|
+
* Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images], Mobile sitemaps *(deprecated)*, [PageMap sitemaps][sitemap_pagemap] and [Alternate Links][alternate_links]
|
|
13
13
|
* Supports read-only filesystems like Heroku via uploading to a remote host like Amazon S3
|
|
14
|
-
* Compatible with all versions of Rails and Ruby
|
|
15
14
|
* Adheres to the [Sitemap 0.9 protocol][sitemap_protocol]
|
|
16
15
|
* Handles millions of links
|
|
17
16
|
* Customizable sitemap compression
|
|
18
|
-
*
|
|
17
|
+
* Optional HTTP notification of search engines when `search_engines` is configured
|
|
19
18
|
* Ensures your old sitemaps stay in place if the new sitemap fails to generate
|
|
20
19
|
* Gives you complete control over your sitemap contents and naming scheme
|
|
21
20
|
* Intelligent sitemap indexing
|
|
21
|
+
* Compatible with Ruby from version 2.6 to version 4.0
|
|
22
|
+
* Compatible with Rails from version 6.0 to version 8.1
|
|
22
23
|
|
|
23
24
|
### Show Me
|
|
24
25
|
|
|
@@ -26,7 +27,7 @@ This is a simple standalone example. For Rails installation see the [Rails inst
|
|
|
26
27
|
|
|
27
28
|
Install:
|
|
28
29
|
|
|
29
|
-
```
|
|
30
|
+
```sh
|
|
30
31
|
gem install sitemap_generator
|
|
31
32
|
```
|
|
32
33
|
|
|
@@ -41,96 +42,87 @@ SitemapGenerator::Sitemap.create do
|
|
|
41
42
|
add '/home', :changefreq => 'daily', :priority => 0.9
|
|
42
43
|
add '/contact_us', :changefreq => 'weekly'
|
|
43
44
|
end
|
|
44
|
-
SitemapGenerator::Sitemap.ping_search_engines # Not needed if you use the rake tasks
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
Run it:
|
|
48
48
|
|
|
49
|
-
```
|
|
49
|
+
```sh
|
|
50
50
|
ruby sitemap.rb
|
|
51
51
|
```
|
|
52
52
|
|
|
53
53
|
Output:
|
|
54
54
|
|
|
55
|
-
```
|
|
55
|
+
```text
|
|
56
56
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
57
57
|
+ sitemap.xml.gz 3 links / 364 Bytes
|
|
58
58
|
Sitemap stats: 3 links / 1 sitemaps / 0m00s
|
|
59
|
-
|
|
60
|
-
Successful ping of Google
|
|
61
59
|
```
|
|
62
60
|
|
|
63
61
|
## Contents
|
|
64
62
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
## Contribute
|
|
129
|
-
|
|
130
|
-
Does your website use SitemapGenerator to generate Sitemaps? Where would you be without Sitemaps? Probably still knocking rocks together. Consider donating to the project to keep it up-to-date and open source.
|
|
131
|
-
|
|
132
|
-
<a href='http://www.pledgie.com/campaigns/15267'><img alt='Click here to lend your support to: SitemapGenerator and make a donation at www.pledgie.com !' src='http://pledgie.com/campaigns/15267.png?skin_name=chrome' border='0' /></a>
|
|
133
|
-
|
|
63
|
+
* [SitemapGenerator](#sitemapgenerator)
|
|
64
|
+
* [Features](#features)
|
|
65
|
+
* [Show Me](#show-me)
|
|
66
|
+
* [Contents](#contents)
|
|
67
|
+
* [Foreword](#foreword)
|
|
68
|
+
* [Installation](#installation)
|
|
69
|
+
* [Ruby](#ruby)
|
|
70
|
+
* [Rails](#rails)
|
|
71
|
+
* [Getting Started](#getting-started)
|
|
72
|
+
* [Preventing Output](#preventing-output)
|
|
73
|
+
* [Rake Tasks](#rake-tasks)
|
|
74
|
+
* [Pinging Search Engines](#pinging-search-engines)
|
|
75
|
+
* [Crontab](#crontab)
|
|
76
|
+
* [Robots.txt](#robotstxt)
|
|
77
|
+
* [Ruby Modules](#ruby-modules)
|
|
78
|
+
* [Deployments \& Capistrano](#deployments--capistrano)
|
|
79
|
+
* [Sitemaps with no Index File](#sitemaps-with-no-index-file)
|
|
80
|
+
* [Upload Sitemaps to a Remote Host using Adapters](#upload-sitemaps-to-a-remote-host-using-adapters)
|
|
81
|
+
* [Supported Adapters](#supported-adapters)
|
|
82
|
+
* [`SitemapGenerator::FileAdapter`](#sitemapgeneratorfileadapter)
|
|
83
|
+
* [`SitemapGenerator::ActiveStorageAdapter`](#sitemapgeneratoractivestorageadapter)
|
|
84
|
+
* [`SitemapGenerator::FogAdapter`](#sitemapgeneratorfogadapter)
|
|
85
|
+
* [`SitemapGenerator::S3Adapter`](#sitemapgenerators3adapter)
|
|
86
|
+
* [`SitemapGenerator::AwsSdkAdapter`](#sitemapgeneratorawssdkadapter)
|
|
87
|
+
* [`SitemapGenerator::WaveAdapter`](#sitemapgeneratorwaveadapter)
|
|
88
|
+
* [`SitemapGenerator::GoogleStorageAdapter`](#sitemapgeneratorgooglestorageadapter)
|
|
89
|
+
* [An Example of Using an Adapter](#an-example-of-using-an-adapter)
|
|
90
|
+
* [Generating Multiple Sitemaps](#generating-multiple-sitemaps)
|
|
91
|
+
* [Sitemap Configuration](#sitemap-configuration)
|
|
92
|
+
* [A Simple Example](#a-simple-example)
|
|
93
|
+
* [Adding Links](#adding-links)
|
|
94
|
+
* [Supported Options to `add`](#supported-options-to-add)
|
|
95
|
+
* [Adding Links to the Sitemap Index](#adding-links-to-the-sitemap-index)
|
|
96
|
+
* [Accessing the LinkSet instance](#accessing-the-linkset-instance)
|
|
97
|
+
* [Using `create` without a block](#using-create-without-a-block)
|
|
98
|
+
* [Speeding Things Up](#speeding-things-up)
|
|
99
|
+
* [Customizing your Sitemaps](#customizing-your-sitemaps)
|
|
100
|
+
* [Sitemap Options](#sitemap-options)
|
|
101
|
+
* [Sitemap Groups](#sitemap-groups)
|
|
102
|
+
* [A Groups Example](#a-groups-example)
|
|
103
|
+
* [Using `group` without a block](#using-group-without-a-block)
|
|
104
|
+
* [Sitemap Extensions](#sitemap-extensions)
|
|
105
|
+
* [News Sitemaps](#news-sitemaps)
|
|
106
|
+
* [Example](#example)
|
|
107
|
+
* [Supported options](#supported-options)
|
|
108
|
+
* [Image Sitemaps](#image-sitemaps)
|
|
109
|
+
* [Example](#example-1)
|
|
110
|
+
* [Supported options](#supported-options-1)
|
|
111
|
+
* [Video Sitemaps](#video-sitemaps)
|
|
112
|
+
* [Example](#example-2)
|
|
113
|
+
* [Supported options](#supported-options-2)
|
|
114
|
+
* [PageMap Sitemaps](#pagemap-sitemaps)
|
|
115
|
+
* [Supported options](#supported-options-3)
|
|
116
|
+
* [Example:](#example-3)
|
|
117
|
+
* [Alternate Links](#alternate-links)
|
|
118
|
+
* [Example](#example-4)
|
|
119
|
+
* [Supported options](#supported-options-4)
|
|
120
|
+
* [Alternates Example](#alternates-example)
|
|
121
|
+
* [Mobile Sitemaps](#mobile-sitemaps)
|
|
122
|
+
* [Example](#example-5)
|
|
123
|
+
* [Supported options](#supported-options-5)
|
|
124
|
+
* [Compatibility](#compatibility)
|
|
125
|
+
* [Licence](#licence)
|
|
134
126
|
|
|
135
127
|
## Foreword
|
|
136
128
|
|
|
@@ -140,12 +132,11 @@ Those who knew him know what an amazing guy he was, and what an excellent Rails
|
|
|
140
132
|
|
|
141
133
|
The canonical repository is: [http://github.com/kjvarga/sitemap_generator][canonical_repo]
|
|
142
134
|
|
|
143
|
-
|
|
144
135
|
## Installation
|
|
145
136
|
|
|
146
137
|
### Ruby
|
|
147
138
|
|
|
148
|
-
```
|
|
139
|
+
```sh
|
|
149
140
|
gem install 'sitemap_generator'
|
|
150
141
|
```
|
|
151
142
|
|
|
@@ -159,7 +150,7 @@ The Rake tasks expect your sitemap to be at `config/sitemap.rb` but if you need
|
|
|
159
150
|
|
|
160
151
|
### Rails
|
|
161
152
|
|
|
162
|
-
SitemapGenerator
|
|
153
|
+
SitemapGenerator targets Rails 6.0 through 8.1 (see [Compatibility](#compatibility)).
|
|
163
154
|
|
|
164
155
|
Add the gem to your `Gemfile`:
|
|
165
156
|
|
|
@@ -204,10 +195,10 @@ SitemapGenerator.verbose = false
|
|
|
204
195
|
|
|
205
196
|
### Pinging Search Engines
|
|
206
197
|
|
|
207
|
-
Using `rake sitemap:refresh` will notify
|
|
198
|
+
Using `rake sitemap:refresh` will notify configured search engines to let them know that a new sitemap
|
|
208
199
|
is available. To generate new sitemaps without notifying search engines, use `rake sitemap:refresh:no_ping`.
|
|
209
200
|
|
|
210
|
-
If you want to customize the hash of search engines you can access it at:
|
|
201
|
+
By default no search engines are configured. If you want to customize the hash of search engines you can access it at:
|
|
211
202
|
|
|
212
203
|
```ruby
|
|
213
204
|
SitemapGenerator::Sitemap.search_engines
|
|
@@ -224,7 +215,7 @@ directly in the call, as in the following example:
|
|
|
224
215
|
SitemapGenerator::Sitemap.ping_search_engines(newengine: 'http://newengine.com/ping?url=%s')
|
|
225
216
|
```
|
|
226
217
|
|
|
227
|
-
The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the
|
|
218
|
+
The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the URL-encoded (percent-encoded) sitemap index URL. If you have any literal percent characters in your URL you need to escape them with `%%`.
|
|
228
219
|
|
|
229
220
|
If you are calling `SitemapGenerator::Sitemap.ping_search_engines` from outside of your sitemap config file, then you will need to set `SitemapGenerator::Sitemap.default_host` and any other options that you set in your sitemap config which affect the location of the sitemap index file. For example:
|
|
230
221
|
|
|
@@ -256,7 +247,7 @@ end
|
|
|
256
247
|
|
|
257
248
|
You should add the URL of the sitemap index file to `public/robots.txt` to help search engines find your sitemaps. The URL should be the complete URL to the sitemap index. For example:
|
|
258
249
|
|
|
259
|
-
```
|
|
250
|
+
```text
|
|
260
251
|
Sitemap: http://www.example.com/sitemap.xml.gz
|
|
261
252
|
```
|
|
262
253
|
|
|
@@ -317,6 +308,7 @@ To never create an index:
|
|
|
317
308
|
```ruby
|
|
318
309
|
SitemapGenerator::Sitemap.create_index = false
|
|
319
310
|
```
|
|
311
|
+
|
|
320
312
|
Your sitemaps will still be called `sitemap.xml.gz`, `sitemap1.xml.gz`, `sitemap2.xml.gz`, etc.
|
|
321
313
|
|
|
322
314
|
And the default "intelligent" behaviour:
|
|
@@ -342,6 +334,10 @@ directory.
|
|
|
342
334
|
|
|
343
335
|
Standard adapter, writes out to a file.
|
|
344
336
|
|
|
337
|
+
##### `SitemapGenerator::ActiveStorageAdapter`
|
|
338
|
+
|
|
339
|
+
Uses `ActiveStorage::Blob` to store the sitemap.
|
|
340
|
+
|
|
345
341
|
##### `SitemapGenerator::FogAdapter`
|
|
346
342
|
|
|
347
343
|
Uses `Fog::Storage` to upload to any service supported by Fog.
|
|
@@ -364,6 +360,7 @@ directory.
|
|
|
364
360
|
Where `options` is a Hash with any of the following keys:
|
|
365
361
|
* `aws_access_key_id` [String] Your AWS access key id
|
|
366
362
|
* `aws_secret_access_key` [String] Your AWS secret access key
|
|
363
|
+
* `aws_session_token` [String] Session token for temporary credentials (optional)
|
|
367
364
|
* `fog_provider` [String]
|
|
368
365
|
* `fog_directory` [String]
|
|
369
366
|
* `fog_region` [String]
|
|
@@ -372,7 +369,7 @@ directory.
|
|
|
372
369
|
* `fog_public` [Boolean] Whether the file is publicly accessible
|
|
373
370
|
|
|
374
371
|
Alternatively you can use an environment variable to configure each option (except `fog_storage_options`). The environment variables have the same
|
|
375
|
-
name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
372
|
+
name but capitalized, e.g. `AWS_SESSION_TOKEN`, `FOG_PATH_STYLE`.
|
|
376
373
|
|
|
377
374
|
##### `SitemapGenerator::AwsSdkAdapter`
|
|
378
375
|
|
|
@@ -388,14 +385,15 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
388
385
|
SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new('s3_bucket',
|
|
389
386
|
acl: 'public-read', # Optional. This is the default.
|
|
390
387
|
cache_control: 'private, max-age=0, no-cache', # Optional. This is the default.
|
|
391
|
-
access_key_id: '
|
|
392
|
-
secret_access_key: '
|
|
388
|
+
access_key_id: 'YOUR_AWS_ACCESS_KEY_ID',
|
|
389
|
+
secret_access_key: 'YOUR_AWS_SECRET_ACCESS_KEY',
|
|
390
|
+
session_token: 'YOUR_AWS_SESSION_TOKEN', # Optional; use with temporary credentials.
|
|
393
391
|
region: 'us-east-1',
|
|
394
392
|
endpoint: 'https://sfo2.digitaloceanspaces.com'
|
|
395
393
|
)
|
|
396
394
|
```
|
|
397
395
|
|
|
398
|
-
Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files;
|
|
396
|
+
Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files; `session_token` supports STS-style credentials (or set `AWS_SESSION_TOKEN` and rely on SDK defaults). All other keyword options are passed directly to the AWS client.
|
|
399
397
|
|
|
400
398
|
See [the `SitemapGenerator::AwsSdkAdapter` docs](https://github.com/kjvarga/sitemap_generator/blob/master/lib/sitemap_generator/adapters/aws_sdk_adapter.rb), and [https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method](https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method) for the full list of supported options.
|
|
401
399
|
|
|
@@ -425,6 +423,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
425
423
|
project_id: 'google_account_project_id',
|
|
426
424
|
)
|
|
427
425
|
```
|
|
426
|
+
|
|
428
427
|
Also, inline with Google Authentication options, it can also pick credentials from environment variables. All [supported environment variables][google_cloud_storage_authentication] can be used, for example: `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_CREDENTIALS`. An example of using this adapter with the environment variables is:
|
|
429
428
|
|
|
430
429
|
```ruby
|
|
@@ -440,7 +439,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
440
439
|
1. Please see [this wiki page][remote_hosts] for more information about setting up SitemapGenerator to upload to a
|
|
441
440
|
remote host.
|
|
442
441
|
|
|
443
|
-
|
|
442
|
+
1. This example uses the CarrierWave adapter. It shows some common settings that are used when the hostname hosting
|
|
444
443
|
the sitemaps differs from the hostname of the sitemap links.
|
|
445
444
|
|
|
446
445
|
```ruby
|
|
@@ -460,9 +459,9 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
460
459
|
SitemapGenerator::Sitemap.adapter = SitemapGenerator::WaveAdapter.new
|
|
461
460
|
```
|
|
462
461
|
|
|
463
|
-
|
|
462
|
+
1. Update your `robots.txt` file to point robots to the remote sitemap index file, e.g:
|
|
464
463
|
|
|
465
|
-
```
|
|
464
|
+
```text
|
|
466
465
|
Sitemap: http://s3.amazonaws.com/sitemap-generator/sitemaps/sitemap.xml.gz
|
|
467
466
|
```
|
|
468
467
|
|
|
@@ -473,7 +472,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
473
472
|
that would otherwise be included would point to a different host than the rest of the links
|
|
474
473
|
in the sitemap, something that the sitemap rules forbid.
|
|
475
474
|
|
|
476
|
-
|
|
475
|
+
1. Verify to Google that you own the S3 url
|
|
477
476
|
|
|
478
477
|
In order for Google to use your sitemap, you need to prove you own the S3 bucket through [google webmaster tools](https://www.google.com/webmasters/tools/home?hl=en). In the example above, you would add the site `http://s3.amazonaws.com/sitemap-generator/sitemaps`. Once you have verified you own the directory, then add your
|
|
479
478
|
sitemap index to the list of sitemaps for the site.
|
|
@@ -499,7 +498,7 @@ end
|
|
|
499
498
|
|
|
500
499
|
Outputs:
|
|
501
500
|
|
|
502
|
-
```
|
|
501
|
+
```text
|
|
503
502
|
+ sitemaps/google/sitemap1.xml.gz 2 links / 822 Bytes / 328 Bytes gzipped
|
|
504
503
|
+ sitemaps/google/sitemap.xml.gz 1 sitemaps / 389 Bytes / 217 Bytes gzipped
|
|
505
504
|
Sitemap stats: 2 links / 1 sitemaps / 0m00s
|
|
@@ -536,10 +535,9 @@ SitemapGenerator::Sitemap.create do
|
|
|
536
535
|
end
|
|
537
536
|
```
|
|
538
537
|
|
|
539
|
-
|
|
540
538
|
To generate each one specify the configuration file to run by passing the `CONFIG_FILE` option to `rake sitemap:refresh`, e.g.:
|
|
541
539
|
|
|
542
|
-
```
|
|
540
|
+
```sh
|
|
543
541
|
rake sitemap:refresh CONFIG_FILE="config/google_sitemap.rb"
|
|
544
542
|
rake sitemap:refresh CONFIG_FILE="config/apple_sitemap.rb"
|
|
545
543
|
rake sitemap:refresh CONFIG_FILE="config/bing_sitemap.rb"
|
|
@@ -551,7 +549,7 @@ A sitemap configuration file contains all the information needed to generate you
|
|
|
551
549
|
|
|
552
550
|
If you want to use a non-standard configuration file, or have multiple configuration files, you can specify which one to run by passing the `CONFIG_FILE` option like so:
|
|
553
551
|
|
|
554
|
-
```
|
|
552
|
+
```sh
|
|
555
553
|
rake sitemap:refresh CONFIG_FILE="config/geo_sitemap.rb"
|
|
556
554
|
```
|
|
557
555
|
|
|
@@ -575,7 +573,7 @@ A few things to note:
|
|
|
575
573
|
|
|
576
574
|
Now let's see what is output when we run this configuration with `rake sitemap:refresh:no_ping`:
|
|
577
575
|
|
|
578
|
-
```
|
|
576
|
+
```text
|
|
579
577
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
580
578
|
+ sitemap.xml.gz 2 links / 347 Bytes
|
|
581
579
|
Sitemap stats: 2 links / 1 sitemaps / 0m00s
|
|
@@ -585,7 +583,6 @@ Weird! The sitemap has two links, even though we only added one! This is becau
|
|
|
585
583
|
|
|
586
584
|
Now let's take a look at the file that was created. After uncompressing and XML-tidying the contents we have:
|
|
587
585
|
|
|
588
|
-
|
|
589
586
|
* `public/sitemap.xml.gz`
|
|
590
587
|
|
|
591
588
|
```xml
|
|
@@ -620,7 +617,7 @@ end
|
|
|
620
617
|
|
|
621
618
|
And the output:
|
|
622
619
|
|
|
623
|
-
```
|
|
620
|
+
```text
|
|
624
621
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
625
622
|
+ sitemap1.xml.gz 2 links / 347 Bytes
|
|
626
623
|
+ sitemap.xml.gz 1 sitemaps / 228 Bytes
|
|
@@ -665,7 +662,7 @@ In the example about we pass a `lastmod` (last modified) option with the value o
|
|
|
665
662
|
|
|
666
663
|
Looking at the output from running this sitemap, we see that we have a few more links than before:
|
|
667
664
|
|
|
668
|
-
```
|
|
665
|
+
```text
|
|
669
666
|
+ sitemap.xml.gz 12 links / 2.3 KB / 365 Bytes gzipped
|
|
670
667
|
Sitemap stats: 12 links / 1 sitemaps / 0m00s
|
|
671
668
|
```
|
|
@@ -773,18 +770,28 @@ end
|
|
|
773
770
|
|
|
774
771
|
The output looks something like this:
|
|
775
772
|
|
|
776
|
-
```
|
|
773
|
+
```text
|
|
777
774
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
778
775
|
+ sitemap4.xml.gz 3 links / 355 Bytes
|
|
779
776
|
+ sitemap.xml.gz 4 sitemaps / 242 Bytes
|
|
780
777
|
Sitemap stats: 3 links / 4 sitemaps / 0m00s
|
|
781
778
|
```
|
|
782
779
|
|
|
780
|
+
### Using `create` without a block
|
|
781
|
+
|
|
782
|
+
You can call `create` without a block, add links (and use `group` as needed), then call `finalize!` on the returned link set when finished.
|
|
783
|
+
|
|
784
|
+
```ruby
|
|
785
|
+
sitemap = SitemapGenerator::Sitemap.create(default_host: 'http://www.example.com')
|
|
786
|
+
sitemap.add('/home')
|
|
787
|
+
# ... add more links or use groups, then:
|
|
788
|
+
sitemap.finalize!
|
|
789
|
+
```
|
|
790
|
+
|
|
783
791
|
### Speeding Things Up
|
|
784
792
|
|
|
785
793
|
For large ActiveRecord collections with thousands of records it is advisable to iterate through them in batches to avoid loading all records into memory at once. For this reason in the example above we use `Content.find_each` which is a batched iterator available since Rails 2.3.2, rather than `Content.all`.
|
|
786
794
|
|
|
787
|
-
|
|
788
795
|
## Customizing your Sitemaps
|
|
789
796
|
|
|
790
797
|
SitemapGenerator supports a number of options which allow you to control every aspect of your sitemap generation. How they are named, where they are stored, the contents of the links and the location that the sitemaps will be hosted from can all be set.
|
|
@@ -841,8 +848,8 @@ The following options are supported.
|
|
|
841
848
|
* `:public_path` - String. A **full or relative path** to the `public` directory or the directory you want to write sitemaps into. Defaults to `public/` under your application root or relative to the current working directory.
|
|
842
849
|
|
|
843
850
|
* `:sitemaps_host` - String. **Host including protocol** to use when generating a link to a sitemap file i.e. the hostname of the server where the sitemaps are hosted. The value will differ from the hostname in your sitemap links. For example: `'http://amazon.aws.com/'`. Note that `include_index` is
|
|
844
|
-
automatically turned off when the `sitemaps_host` does not match `default_host`.
|
|
845
|
-
Because the link to the sitemap index file that would otherwise be added would point to a different host than the rest of the links in the sitemap. Something that the sitemap rules forbid.
|
|
851
|
+
automatically turned off when the `sitemaps_host` does not match `default_host`.
|
|
852
|
+
Because the link to the sitemap index file that would otherwise be added would point to a different host than the rest of the links in the sitemap. Something that the sitemap rules forbid.
|
|
846
853
|
|
|
847
854
|
* `:namer` - A `SitemapGenerator::SimpleNamer` instance **for generating sitemap names**. You can read about Sitemap Namers by reading the API docs. Allows you to set the name, extension and number sequence for sitemap files, as well as modify the name of the first file in the sequence, which is often the index file. A simple example if we want to generate files like 'newname.xml.gz', 'newname1.xml.gz', etc is `SitemapGenerator::SimpleNamer.new(:newname)`.
|
|
848
855
|
|
|
@@ -853,9 +860,9 @@ Because the link to the sitemap index file that would otherwise be added would p
|
|
|
853
860
|
* `:adapter` - Instance. The default adapter is a `SitemapGenerator::FileAdapter` which simply writes files to the filesystem. You can use a `SitemapGenerator::WaveAdapter` for uploading sitemaps to remote servers - useful for read-only hosts such as Heroku. Or you can provide an instance of your own class to provide custom behavior. Your class must define a write method which takes a `SitemapGenerator::Location` and raw XML data.
|
|
854
861
|
|
|
855
862
|
* `:compress` - Specifies which files to compress with gzip. Default is `true`. Accepted values:
|
|
856
|
-
|
|
857
|
-
|
|
858
|
-
|
|
863
|
+
* `true` - Boolean; compress all files.
|
|
864
|
+
* `false` - Boolean; Do not compress any files.
|
|
865
|
+
* `:all_but_first` - Symbol; leave the first file uncompressed but compress all remaining files.
|
|
859
866
|
|
|
860
867
|
The compression setting applies to groups too. So `:all_but_first` will have the same effect (the first file in the group will not be compressed, the rest will). So if you require different behaviour for your groups, pass in a `:compress` option e.g. `group(:compress => false) { add('/link') }`
|
|
861
868
|
|
|
@@ -896,7 +903,7 @@ end
|
|
|
896
903
|
|
|
897
904
|
And the output from running the above:
|
|
898
905
|
|
|
899
|
-
```
|
|
906
|
+
```text
|
|
900
907
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
901
908
|
+ en/english.xml.gz 1 links / 328 Bytes
|
|
902
909
|
+ fr/french.xml.gz 1 links / 329 Bytes
|
|
@@ -945,7 +952,7 @@ end
|
|
|
945
952
|
|
|
946
953
|
And the output from running the above:
|
|
947
954
|
|
|
948
|
-
```
|
|
955
|
+
```text
|
|
949
956
|
In '/Users/kvarga/Projects/sitemap_generator-test/public/':
|
|
950
957
|
+ odds.xml.gz 10 links / 371 Bytes
|
|
951
958
|
+ evens.xml.gz 10 links / 371 Bytes
|
|
@@ -980,14 +987,14 @@ end
|
|
|
980
987
|
#### Supported options
|
|
981
988
|
|
|
982
989
|
* `:news` - Hash
|
|
983
|
-
|
|
984
|
-
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
989
|
-
|
|
990
|
-
|
|
990
|
+
* `:publication_name`
|
|
991
|
+
* `:publication_language`
|
|
992
|
+
* `:publication_date`
|
|
993
|
+
* `:genres`
|
|
994
|
+
* `:access`
|
|
995
|
+
* `:title`
|
|
996
|
+
* `:keywords`
|
|
997
|
+
* `:stock_tickers`
|
|
991
998
|
|
|
992
999
|
### Image Sitemaps
|
|
993
1000
|
|
|
@@ -1007,11 +1014,11 @@ end
|
|
|
1007
1014
|
#### Supported options
|
|
1008
1015
|
|
|
1009
1016
|
* `:images` - Array of hashes
|
|
1010
|
-
|
|
1011
|
-
|
|
1012
|
-
|
|
1013
|
-
|
|
1014
|
-
|
|
1017
|
+
* `:loc` Required, location of the image
|
|
1018
|
+
* `:caption`
|
|
1019
|
+
* `:geo_location`
|
|
1020
|
+
* `:title`
|
|
1021
|
+
* `:license`
|
|
1015
1022
|
|
|
1016
1023
|
### Video Sitemaps
|
|
1017
1024
|
|
|
@@ -1038,32 +1045,32 @@ end
|
|
|
1038
1045
|
#### Supported options
|
|
1039
1046
|
|
|
1040
1047
|
* `:video` or `:videos` - Hash or array of hashes, respectively
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1046
|
-
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
|
|
1053
|
-
|
|
1054
|
-
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1060
|
-
|
|
1061
|
-
|
|
1062
|
-
|
|
1063
|
-
|
|
1064
|
-
|
|
1065
|
-
|
|
1066
|
-
|
|
1048
|
+
* `:thumbnail_loc` - Required. String, URL of the thumbnail image.
|
|
1049
|
+
* `:title` - Required. String, title of the video.
|
|
1050
|
+
* `:description` - Required. String, description of the video.
|
|
1051
|
+
* `:content_loc` - Depends. String, URL. One of content_loc or player_loc must be present.
|
|
1052
|
+
* `:player_loc` - Depends. String, URL. One of content_loc or player_loc must be present.
|
|
1053
|
+
* `:allow_embed` - Boolean, attribute of player_loc.
|
|
1054
|
+
* `:autoplay` - Boolean, default true. Attribute of player_loc.
|
|
1055
|
+
* `:duration` - Recommended. Integer or string. Duration in seconds.
|
|
1056
|
+
* `:expiration_date` - Recommended when applicable. The date after which the video will no longer be available.
|
|
1057
|
+
* `:rating` - Optional
|
|
1058
|
+
* `:view_count` - Optional. Integer or string.
|
|
1059
|
+
* `:publication_date` - Optional
|
|
1060
|
+
* `:tags` - Optional. Array of string tags.
|
|
1061
|
+
* `:tag` - Optional. String, single tag.
|
|
1062
|
+
* `:category` - Optional
|
|
1063
|
+
* `:family_friendly`- Optional. Boolean
|
|
1064
|
+
* `:gallery_loc` - Optional. String, URL.
|
|
1065
|
+
* `:gallery_title` - Optional. Title attribute of the gallery location element
|
|
1066
|
+
* `:uploader` - Optional.
|
|
1067
|
+
* `:uploader_info` - Optional. Info attribute of uploader element
|
|
1068
|
+
* `:price` - Optional. Only one price supported at this time
|
|
1069
|
+
* `:price_currency` - Required. In [ISO_4217][iso_4217] format.
|
|
1070
|
+
* `:price_type` - Optional. `rent` or `own`
|
|
1071
|
+
* `:price_resolution` - Optional. `HD` or `SD`
|
|
1072
|
+
* `:live` - Optional. Boolean.
|
|
1073
|
+
* `:requires_subscription` - Optional. Boolean.
|
|
1067
1074
|
|
|
1068
1075
|
### PageMap Sitemaps
|
|
1069
1076
|
|
|
@@ -1072,14 +1079,14 @@ Pagemaps can be added by passing a `:pagemap` hash to `add`. The hash must conta
|
|
|
1072
1079
|
#### Supported options
|
|
1073
1080
|
|
|
1074
1081
|
* `:pagemap` - Hash
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
|
|
1080
|
-
|
|
1082
|
+
* `:dataobjects` - Required, array of hashes
|
|
1083
|
+
* `:type` - Required, string, type of the object
|
|
1084
|
+
* `:id` - String, ID of the object
|
|
1085
|
+
* `:attributes` - Array of hashes
|
|
1086
|
+
* `:name` - Required, string, name of the attribute.
|
|
1087
|
+
* `:value` - String, value of the attribute.
|
|
1081
1088
|
|
|
1082
|
-
#### Example
|
|
1089
|
+
#### Example
|
|
1083
1090
|
|
|
1084
1091
|
```ruby
|
|
1085
1092
|
SitemapGenerator::Sitemap.default_host = "http://www.example.com"
|
|
@@ -1121,10 +1128,10 @@ end
|
|
|
1121
1128
|
#### Supported options
|
|
1122
1129
|
|
|
1123
1130
|
* `:alternate`/`:alternates` - Hash or array of hashes, respectively
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1131
|
+
* `:href` - Required, string.
|
|
1132
|
+
* `:lang` - Optional, string.
|
|
1133
|
+
* `:nofollow` - Optional, boolean. Used to mark link as "nofollow".
|
|
1134
|
+
* `:media` - Optional, string. Specify [media targets for responsive design pages][media].
|
|
1128
1135
|
|
|
1129
1136
|
#### Alternates Example
|
|
1130
1137
|
|
|
@@ -1150,7 +1157,7 @@ end
|
|
|
1150
1157
|
|
|
1151
1158
|
Mobile sitemaps include a specific `<mobile:mobile/>` tag.
|
|
1152
1159
|
|
|
1153
|
-
|
|
1160
|
+
> **Deprecated**: Google deprecated mobile sitemaps in August 2022. No other major search engines supported this format. Responsive design is the recommended approach for mobile-friendly content.
|
|
1154
1161
|
|
|
1155
1162
|
#### Example
|
|
1156
1163
|
|
|
@@ -1167,25 +1174,23 @@ end
|
|
|
1167
1174
|
|
|
1168
1175
|
## Compatibility
|
|
1169
1176
|
|
|
1170
|
-
|
|
1171
|
-
Ruby
|
|
1177
|
+
Ruby 2.6 through 4.0 and Rails 6.0 through 8.1 are supported.
|
|
1178
|
+
Ruby 2.5 and Rails 5.2 were dropped in v7.0.0.
|
|
1179
|
+
Ruby 1.9.3 support was dropped in v6.0.0.
|
|
1172
1180
|
|
|
1173
1181
|
## Licence
|
|
1174
1182
|
|
|
1175
|
-
Released under the MIT
|
|
1176
|
-
|
|
1177
|
-
MIT. See the LICENSE.md file.
|
|
1183
|
+
Released under the MIT license which is included in the [MIT-LICENSE](./MIT-LICENSE) file.
|
|
1178
1184
|
|
|
1179
|
-
Copyright (c) Karl Varga
|
|
1185
|
+
Copyright (c) Karl Varga
|
|
1180
1186
|
|
|
1181
1187
|
[canonical_repo]:http://github.com/kjvarga/sitemap_generator
|
|
1182
1188
|
[sitemap_images]:http://www.google.com/support/webmasters/bin/answer.py?answer=178636
|
|
1183
1189
|
[sitemap_video]:https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190
|
|
1184
1190
|
[sitemap_news]:https://support.google.com/news/publisher/topic/2527688?hl=en&ref_topic=4359874
|
|
1185
|
-
[sitemap_mobile]:http://support.google.com/webmasters/bin/answer.py?hl=en&answer=34648
|
|
1186
1191
|
[sitemap_pagemap]:https://developers.google.com/custom-search/docs/structured_data#addtositemap
|
|
1187
1192
|
[sitemap_protocol]:http://www.sitemaps.org/protocol.html
|
|
1188
|
-
[video_tags]:
|
|
1193
|
+
[video_tags]:https://developers.google.com/search/docs/crawling-indexing/sitemaps/video-sitemaps
|
|
1189
1194
|
[image_tags]:http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=178636
|
|
1190
1195
|
[news_tags]:http://www.google.com/support/news_pub/bin/answer.py?answer=74288
|
|
1191
1196
|
[remote_hosts]:https://github.com/kjvarga/sitemap_generator/wiki/Generate-Sitemaps-on-read-only-filesystems-like-Heroku
|
|
@@ -1193,7 +1198,7 @@ Copyright (c) Karl Varga released under the MIT license
|
|
|
1193
1198
|
[using_pagemaps]:https://developers.google.com/custom-search/docs/structured_data#pagemaps
|
|
1194
1199
|
[iso_4217]:http://en.wikipedia.org/wiki/ISO_4217
|
|
1195
1200
|
[media]:https://developers.google.com/webmasters/smartphone-sites/details
|
|
1196
|
-
[expires]:https://support.google.com/
|
|
1201
|
+
[expires]:https://support.google.com/webmasters/answer/9689846
|
|
1197
1202
|
[google_cloud_storage_gem]:https://rubygems.org/gems/google-cloud-storage
|
|
1198
1203
|
[google_cloud_storage_authentication]:https://googleapis.dev/ruby/google-cloud-storage/latest/file.AUTHENTICATION.html
|
|
1199
1204
|
[google_cloud_storage_initializer]:https://github.com/googleapis/google-cloud-ruby/blob/master/google-cloud-storage/lib/google/cloud/storage.rb
|