sitemap_generator 6.3.0 → 7.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGES.md +107 -61
  3. data/README.md +176 -171
  4. data/VERSION +1 -1
  5. data/lib/capistrano/sitemap_generator.rb +2 -0
  6. data/lib/sitemap_generator/adapters/active_storage_adapter.rb +31 -0
  7. data/lib/sitemap_generator/adapters/aws_sdk_adapter.rb +29 -13
  8. data/lib/sitemap_generator/adapters/file_adapter.rb +12 -13
  9. data/lib/sitemap_generator/adapters/fog_adapter.rb +9 -7
  10. data/lib/sitemap_generator/adapters/google_storage_adapter.rb +8 -5
  11. data/lib/sitemap_generator/adapters/s3_adapter.rb +23 -18
  12. data/lib/sitemap_generator/adapters/wave_adapter.rb +8 -6
  13. data/lib/sitemap_generator/application.rb +9 -3
  14. data/lib/sitemap_generator/builder/sitemap_file.rb +41 -31
  15. data/lib/sitemap_generator/builder/sitemap_index_file.rb +31 -23
  16. data/lib/sitemap_generator/builder/sitemap_index_url.rb +10 -9
  17. data/lib/sitemap_generator/builder/sitemap_url.rb +130 -80
  18. data/lib/sitemap_generator/builder.rb +7 -3
  19. data/lib/sitemap_generator/core_ext/big_decimal.rb +40 -33
  20. data/lib/sitemap_generator/core_ext/numeric.rb +52 -46
  21. data/lib/sitemap_generator/core_ext.rb +2 -0
  22. data/lib/sitemap_generator/helpers/number_helper.rb +84 -76
  23. data/lib/sitemap_generator/interpreter.rb +22 -18
  24. data/lib/sitemap_generator/link_set.rb +114 -113
  25. data/lib/sitemap_generator/railtie.rb +4 -1
  26. data/lib/sitemap_generator/simple_namer.rb +8 -6
  27. data/lib/sitemap_generator/sitemap_location.rb +52 -40
  28. data/lib/sitemap_generator/tasks.rb +16 -26
  29. data/lib/sitemap_generator/templates.rb +11 -9
  30. data/lib/sitemap_generator/utilities.rb +46 -24
  31. data/lib/sitemap_generator.rb +42 -27
  32. data/lib/tasks/sitemap_generator_tasks.rake +2 -0
  33. data/rails/install.rb +2 -0
  34. data/rails/uninstall.rb +2 -0
  35. data/templates/sitemap.rb +3 -1
  36. metadata +7 -131
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # SitemapGenerator
2
2
 
3
- [![CircleCI](https://circleci.com/gh/kjvarga/sitemap_generator/tree/master.svg?style=shield)](https://circleci.com/gh/kjvarga/sitemap_generator/tree/master)
3
+ [![CI](https://github.com/kjvarga/sitemap_generator/actions/workflows/ci.yml/badge.svg)](https://github.com/kjvarga/sitemap_generator/actions/workflows/ci.yml)
4
4
 
5
5
  SitemapGenerator is the easiest way to generate Sitemaps in Ruby. Rails integration provides access to the Rails route helpers within your sitemap config file and automatically makes the rake tasks available to you. Or if you prefer to use another framework, you can! You can use the rake tasks provided or run your sitemap configs as plain ruby scripts.
6
6
 
@@ -9,16 +9,17 @@ Sitemaps adhere to the [Sitemap 0.9 protocol][sitemap_protocol] specification.
9
9
  ## Features
10
10
 
11
11
  * Framework agnostic
12
- * Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images], [Mobile sitemaps][sitemap_mobile], [PageMap sitemaps][sitemap_pagemap] and [Alternate Links][alternate_links]
12
+ * Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images], Mobile sitemaps *(deprecated)*, [PageMap sitemaps][sitemap_pagemap] and [Alternate Links][alternate_links]
13
13
  * Supports read-only filesystems like Heroku via uploading to a remote host like Amazon S3
14
- * Compatible with all versions of Rails and Ruby
15
14
  * Adheres to the [Sitemap 0.9 protocol][sitemap_protocol]
16
15
  * Handles millions of links
17
16
  * Customizable sitemap compression
18
- * Notifies search engines (Google) of new sitemaps
17
+ * Optional HTTP notification of search engines when `search_engines` is configured
19
18
  * Ensures your old sitemaps stay in place if the new sitemap fails to generate
20
19
  * Gives you complete control over your sitemap contents and naming scheme
21
20
  * Intelligent sitemap indexing
21
+ * Compatible with Ruby from version 2.6 to version 4.0
22
+ * Compatible with Rails from version 6.0 to version 8.1
22
23
 
23
24
  ### Show Me
24
25
 
@@ -26,7 +27,7 @@ This is a simple standalone example. For Rails installation see the [Rails inst
26
27
 
27
28
  Install:
28
29
 
29
- ```
30
+ ```sh
30
31
  gem install sitemap_generator
31
32
  ```
32
33
 
@@ -41,96 +42,87 @@ SitemapGenerator::Sitemap.create do
41
42
  add '/home', :changefreq => 'daily', :priority => 0.9
42
43
  add '/contact_us', :changefreq => 'weekly'
43
44
  end
44
- SitemapGenerator::Sitemap.ping_search_engines # Not needed if you use the rake tasks
45
45
  ```
46
46
 
47
47
  Run it:
48
48
 
49
- ```
49
+ ```sh
50
50
  ruby sitemap.rb
51
51
  ```
52
52
 
53
53
  Output:
54
54
 
55
- ```
55
+ ```text
56
56
  In /Users/karl/projects/sitemap_generator-test/public/
57
57
  + sitemap.xml.gz 3 links / 364 Bytes
58
58
  Sitemap stats: 3 links / 1 sitemaps / 0m00s
59
-
60
- Successful ping of Google
61
59
  ```
62
60
 
63
61
  ## Contents
64
62
 
65
- - [SitemapGenerator](#sitemapgenerator)
66
- - [Features](#features)
67
- - [Show Me](#show-me)
68
- - [Contents](#contents)
69
- - [Contribute](#contribute)
70
- - [Foreword](#foreword)
71
- - [Installation](#installation)
72
- - [Ruby](#ruby)
73
- - [Rails](#rails)
74
- - [Getting Started](#getting-started)
75
- - [Preventing Output](#preventing-output)
76
- - [Rake Tasks](#rake-tasks)
77
- - [Pinging Search Engines](#pinging-search-engines)
78
- - [Crontab](#crontab)
79
- - [Robots.txt](#robotstxt)
80
- - [Ruby Modules](#ruby-modules)
81
- - [Deployments & Capistrano](#deployments--capistrano)
82
- - [Sitemaps with no Index File](#sitemaps-with-no-index-file)
83
- - [Upload Sitemaps to a Remote Host using Adapters](#upload-sitemaps-to-a-remote-host-using-adapters)
84
- - [Supported Adapters](#supported-adapters)
85
- - [`SitemapGenerator::FileAdapter`](#sitemapgeneratorfileadapter)
86
- - [`SitemapGenerator::FogAdapter`](#sitemapgeneratorfogadapter)
87
- - [`SitemapGenerator::S3Adapter`](#sitemapgenerators3adapter)
88
- - [`SitemapGenerator::AwsSdkAdapter`](#sitemapgeneratorawssdkadapter)
89
- - [`SitemapGenerator::WaveAdapter`](#sitemapgeneratorwaveadapter)
90
- - [`SitemapGenerator::GoogleStorageAdapter`](#sitemapgeneratorgooglestorageadapter)
91
- - [An Example of Using an Adapter](#an-example-of-using-an-adapter)
92
- - [Generating Multiple Sitemaps](#generating-multiple-sitemaps)
93
- - [Sitemap Configuration](#sitemap-configuration)
94
- - [A Simple Example](#a-simple-example)
95
- - [Adding Links](#adding-links)
96
- - [Supported Options to `add`](#supported-options-to-add)
97
- - [Adding Links to the Sitemap Index](#adding-links-to-the-sitemap-index)
98
- - [Accessing the LinkSet instance](#accessing-the-linkset-instance)
99
- - [Speeding Things Up](#speeding-things-up)
100
- - [Customizing your Sitemaps](#customizing-your-sitemaps)
101
- - [Sitemap Options](#sitemap-options)
102
- - [Sitemap Groups](#sitemap-groups)
103
- - [A Groups Example](#a-groups-example)
104
- - [Using `group` without a block](#using-group-without-a-block)
105
- - [Sitemap Extensions](#sitemap-extensions)
106
- - [News Sitemaps](#news-sitemaps)
107
- - [Example](#example)
108
- - [Supported options](#supported-options)
109
- - [Image Sitemaps](#image-sitemaps)
110
- - [Example](#example-1)
111
- - [Supported options](#supported-options-1)
112
- - [Video Sitemaps](#video-sitemaps)
113
- - [Example](#example-2)
114
- - [Supported options](#supported-options-2)
115
- - [PageMap Sitemaps](#pagemap-sitemaps)
116
- - [Supported options](#supported-options-3)
117
- - [Example:](#example-3)
118
- - [Alternate Links](#alternate-links)
119
- - [Example](#example-4)
120
- - [Supported options](#supported-options-4)
121
- - [Alternates Example](#alternates-example)
122
- - [Mobile Sitemaps](#mobile-sitemaps)
123
- - [Example](#example-5)
124
- - [Supported options](#supported-options-5)
125
- - [Compatibility](#compatibility)
126
- - [Licence](#licence)
127
-
128
- ## Contribute
129
-
130
- Does your website use SitemapGenerator to generate Sitemaps? Where would you be without Sitemaps? Probably still knocking rocks together. Consider donating to the project to keep it up-to-date and open source.
131
-
132
- <a href='http://www.pledgie.com/campaigns/15267'><img alt='Click here to lend your support to: SitemapGenerator and make a donation at www.pledgie.com !' src='http://pledgie.com/campaigns/15267.png?skin_name=chrome' border='0' /></a>
133
-
63
+ * [SitemapGenerator](#sitemapgenerator)
64
+ * [Features](#features)
65
+ * [Show Me](#show-me)
66
+ * [Contents](#contents)
67
+ * [Foreword](#foreword)
68
+ * [Installation](#installation)
69
+ * [Ruby](#ruby)
70
+ * [Rails](#rails)
71
+ * [Getting Started](#getting-started)
72
+ * [Preventing Output](#preventing-output)
73
+ * [Rake Tasks](#rake-tasks)
74
+ * [Pinging Search Engines](#pinging-search-engines)
75
+ * [Crontab](#crontab)
76
+ * [Robots.txt](#robotstxt)
77
+ * [Ruby Modules](#ruby-modules)
78
+ * [Deployments \& Capistrano](#deployments--capistrano)
79
+ * [Sitemaps with no Index File](#sitemaps-with-no-index-file)
80
+ * [Upload Sitemaps to a Remote Host using Adapters](#upload-sitemaps-to-a-remote-host-using-adapters)
81
+ * [Supported Adapters](#supported-adapters)
82
+ * [`SitemapGenerator::FileAdapter`](#sitemapgeneratorfileadapter)
83
+ * [`SitemapGenerator::ActiveStorageAdapter`](#sitemapgeneratoractivestorageadapter)
84
+ * [`SitemapGenerator::FogAdapter`](#sitemapgeneratorfogadapter)
85
+ * [`SitemapGenerator::S3Adapter`](#sitemapgenerators3adapter)
86
+ * [`SitemapGenerator::AwsSdkAdapter`](#sitemapgeneratorawssdkadapter)
87
+ * [`SitemapGenerator::WaveAdapter`](#sitemapgeneratorwaveadapter)
88
+ * [`SitemapGenerator::GoogleStorageAdapter`](#sitemapgeneratorgooglestorageadapter)
89
+ * [An Example of Using an Adapter](#an-example-of-using-an-adapter)
90
+ * [Generating Multiple Sitemaps](#generating-multiple-sitemaps)
91
+ * [Sitemap Configuration](#sitemap-configuration)
92
+ * [A Simple Example](#a-simple-example)
93
+ * [Adding Links](#adding-links)
94
+ * [Supported Options to `add`](#supported-options-to-add)
95
+ * [Adding Links to the Sitemap Index](#adding-links-to-the-sitemap-index)
96
+ * [Accessing the LinkSet instance](#accessing-the-linkset-instance)
97
+ * [Using `create` without a block](#using-create-without-a-block)
98
+ * [Speeding Things Up](#speeding-things-up)
99
+ * [Customizing your Sitemaps](#customizing-your-sitemaps)
100
+ * [Sitemap Options](#sitemap-options)
101
+ * [Sitemap Groups](#sitemap-groups)
102
+ * [A Groups Example](#a-groups-example)
103
+ * [Using `group` without a block](#using-group-without-a-block)
104
+ * [Sitemap Extensions](#sitemap-extensions)
105
+ * [News Sitemaps](#news-sitemaps)
106
+ * [Example](#example)
107
+ * [Supported options](#supported-options)
108
+ * [Image Sitemaps](#image-sitemaps)
109
+ * [Example](#example-1)
110
+ * [Supported options](#supported-options-1)
111
+ * [Video Sitemaps](#video-sitemaps)
112
+ * [Example](#example-2)
113
+ * [Supported options](#supported-options-2)
114
+ * [PageMap Sitemaps](#pagemap-sitemaps)
115
+ * [Supported options](#supported-options-3)
116
+ * [Example:](#example-3)
117
+ * [Alternate Links](#alternate-links)
118
+ * [Example](#example-4)
119
+ * [Supported options](#supported-options-4)
120
+ * [Alternates Example](#alternates-example)
121
+ * [Mobile Sitemaps](#mobile-sitemaps)
122
+ * [Example](#example-5)
123
+ * [Supported options](#supported-options-5)
124
+ * [Compatibility](#compatibility)
125
+ * [Licence](#licence)
134
126
 
135
127
  ## Foreword
136
128
 
@@ -140,12 +132,11 @@ Those who knew him know what an amazing guy he was, and what an excellent Rails
140
132
 
141
133
  The canonical repository is: [http://github.com/kjvarga/sitemap_generator][canonical_repo]
142
134
 
143
-
144
135
  ## Installation
145
136
 
146
137
  ### Ruby
147
138
 
148
- ```
139
+ ```sh
149
140
  gem install 'sitemap_generator'
150
141
  ```
151
142
 
@@ -159,7 +150,7 @@ The Rake tasks expect your sitemap to be at `config/sitemap.rb` but if you need
159
150
 
160
151
  ### Rails
161
152
 
162
- SitemapGenerator works with all versions of Rails and has been tested in Rails 2, 3 and 4.
153
+ SitemapGenerator targets Rails 6.0 through 8.1 (see [Compatibility](#compatibility)).
163
154
 
164
155
  Add the gem to your `Gemfile`:
165
156
 
@@ -204,10 +195,10 @@ SitemapGenerator.verbose = false
204
195
 
205
196
  ### Pinging Search Engines
206
197
 
207
- Using `rake sitemap:refresh` will notify Google to let them know that a new sitemap
198
+ Using `rake sitemap:refresh` will notify configured search engines to let them know that a new sitemap
208
199
  is available. To generate new sitemaps without notifying search engines, use `rake sitemap:refresh:no_ping`.
209
200
 
210
- If you want to customize the hash of search engines you can access it at:
201
+ By default no search engines are configured. If you want to customize the hash of search engines you can access it at:
211
202
 
212
203
  ```ruby
213
204
  SitemapGenerator::Sitemap.search_engines
@@ -224,7 +215,7 @@ directly in the call, as in the following example:
224
215
  SitemapGenerator::Sitemap.ping_search_engines(newengine: 'http://newengine.com/ping?url=%s')
225
216
  ```
226
217
 
227
- The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the CGI escaped sitemap index URL. If you have any literal percent characters in your URL you need to escape them with `%%`.
218
+ The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the URL-encoded (percent-encoded) sitemap index URL. If you have any literal percent characters in your URL you need to escape them with `%%`.
228
219
 
229
220
  If you are calling `SitemapGenerator::Sitemap.ping_search_engines` from outside of your sitemap config file, then you will need to set `SitemapGenerator::Sitemap.default_host` and any other options that you set in your sitemap config which affect the location of the sitemap index file. For example:
230
221
 
@@ -256,7 +247,7 @@ end
256
247
 
257
248
  You should add the URL of the sitemap index file to `public/robots.txt` to help search engines find your sitemaps. The URL should be the complete URL to the sitemap index. For example:
258
249
 
259
- ```
250
+ ```text
260
251
  Sitemap: http://www.example.com/sitemap.xml.gz
261
252
  ```
262
253
 
@@ -317,6 +308,7 @@ To never create an index:
317
308
  ```ruby
318
309
  SitemapGenerator::Sitemap.create_index = false
319
310
  ```
311
+
320
312
  Your sitemaps will still be called `sitemap.xml.gz`, `sitemap1.xml.gz`, `sitemap2.xml.gz`, etc.
321
313
 
322
314
  And the default "intelligent" behaviour:
@@ -342,6 +334,10 @@ directory.
342
334
 
343
335
  Standard adapter, writes out to a file.
344
336
 
337
+ ##### `SitemapGenerator::ActiveStorageAdapter`
338
+
339
+ Uses `ActiveStorage::Blob` to store the sitemap.
340
+
345
341
  ##### `SitemapGenerator::FogAdapter`
346
342
 
347
343
  Uses `Fog::Storage` to upload to any service supported by Fog.
@@ -364,6 +360,7 @@ directory.
364
360
  Where `options` is a Hash with any of the following keys:
365
361
  * `aws_access_key_id` [String] Your AWS access key id
366
362
  * `aws_secret_access_key` [String] Your AWS secret access key
363
+ * `aws_session_token` [String] Session token for temporary credentials (optional)
367
364
  * `fog_provider` [String]
368
365
  * `fog_directory` [String]
369
366
  * `fog_region` [String]
@@ -372,7 +369,7 @@ directory.
372
369
  * `fog_public` [Boolean] Whether the file is publicly accessible
373
370
 
374
371
  Alternatively you can use an environment variable to configure each option (except `fog_storage_options`). The environment variables have the same
375
- name but capitalized, e.g. `FOG_PATH_STYLE`.
372
+ name but capitalized, e.g. `AWS_SESSION_TOKEN`, `FOG_PATH_STYLE`.
376
373
 
377
374
  ##### `SitemapGenerator::AwsSdkAdapter`
378
375
 
@@ -388,14 +385,15 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
388
385
  SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new('s3_bucket',
389
386
  acl: 'public-read', # Optional. This is the default.
390
387
  cache_control: 'private, max-age=0, no-cache', # Optional. This is the default.
391
- access_key_id: 'AKIAI3SW5CRAZBL4WSTA',
392
- secret_access_key: 'asdfadsfdsafsadf',
388
+ access_key_id: 'YOUR_AWS_ACCESS_KEY_ID',
389
+ secret_access_key: 'YOUR_AWS_SECRET_ACCESS_KEY',
390
+ session_token: 'YOUR_AWS_SESSION_TOKEN', # Optional; use with temporary credentials.
393
391
  region: 'us-east-1',
394
392
  endpoint: 'https://sfo2.digitaloceanspaces.com'
395
393
  )
396
394
  ```
397
395
 
398
- Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files; all other options are passed directly to the AWS client.
396
+ Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files; `session_token` supports STS-style credentials (or set `AWS_SESSION_TOKEN` and rely on SDK defaults). All other keyword options are passed directly to the AWS client.
399
397
 
400
398
  See [the `SitemapGenerator::AwsSdkAdapter` docs](https://github.com/kjvarga/sitemap_generator/blob/master/lib/sitemap_generator/adapters/aws_sdk_adapter.rb), and [https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method](https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method) for the full list of supported options.
401
399
 
@@ -425,6 +423,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
425
423
  project_id: 'google_account_project_id',
426
424
  )
427
425
  ```
426
+
428
427
  Also, inline with Google Authentication options, it can also pick credentials from environment variables. All [supported environment variables][google_cloud_storage_authentication] can be used, for example: `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_CREDENTIALS`. An example of using this adapter with the environment variables is:
429
428
 
430
429
  ```ruby
@@ -440,7 +439,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
440
439
  1. Please see [this wiki page][remote_hosts] for more information about setting up SitemapGenerator to upload to a
441
440
  remote host.
442
441
 
443
- 2. This example uses the CarrierWave adapter. It shows some common settings that are used when the hostname hosting
442
+ 1. This example uses the CarrierWave adapter. It shows some common settings that are used when the hostname hosting
444
443
  the sitemaps differs from the hostname of the sitemap links.
445
444
 
446
445
  ```ruby
@@ -460,9 +459,9 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
460
459
  SitemapGenerator::Sitemap.adapter = SitemapGenerator::WaveAdapter.new
461
460
  ```
462
461
 
463
- 3. Update your `robots.txt` file to point robots to the remote sitemap index file, e.g:
462
+ 1. Update your `robots.txt` file to point robots to the remote sitemap index file, e.g:
464
463
 
465
- ```
464
+ ```text
466
465
  Sitemap: http://s3.amazonaws.com/sitemap-generator/sitemaps/sitemap.xml.gz
467
466
  ```
468
467
 
@@ -473,7 +472,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
473
472
  that would otherwise be included would point to a different host than the rest of the links
474
473
  in the sitemap, something that the sitemap rules forbid.
475
474
 
476
- 4. Verify to Google that you own the S3 url
475
+ 1. Verify to Google that you own the S3 url
477
476
 
478
477
  In order for Google to use your sitemap, you need to prove you own the S3 bucket through [google webmaster tools](https://www.google.com/webmasters/tools/home?hl=en). In the example above, you would add the site `http://s3.amazonaws.com/sitemap-generator/sitemaps`. Once you have verified you own the directory, then add your
479
478
  sitemap index to the list of sitemaps for the site.
@@ -499,7 +498,7 @@ end
499
498
 
500
499
  Outputs:
501
500
 
502
- ```
501
+ ```text
503
502
  + sitemaps/google/sitemap1.xml.gz 2 links / 822 Bytes / 328 Bytes gzipped
504
503
  + sitemaps/google/sitemap.xml.gz 1 sitemaps / 389 Bytes / 217 Bytes gzipped
505
504
  Sitemap stats: 2 links / 1 sitemaps / 0m00s
@@ -536,10 +535,9 @@ SitemapGenerator::Sitemap.create do
536
535
  end
537
536
  ```
538
537
 
539
-
540
538
  To generate each one specify the configuration file to run by passing the `CONFIG_FILE` option to `rake sitemap:refresh`, e.g.:
541
539
 
542
- ```
540
+ ```sh
543
541
  rake sitemap:refresh CONFIG_FILE="config/google_sitemap.rb"
544
542
  rake sitemap:refresh CONFIG_FILE="config/apple_sitemap.rb"
545
543
  rake sitemap:refresh CONFIG_FILE="config/bing_sitemap.rb"
@@ -551,7 +549,7 @@ A sitemap configuration file contains all the information needed to generate you
551
549
 
552
550
  If you want to use a non-standard configuration file, or have multiple configuration files, you can specify which one to run by passing the `CONFIG_FILE` option like so:
553
551
 
554
- ```
552
+ ```sh
555
553
  rake sitemap:refresh CONFIG_FILE="config/geo_sitemap.rb"
556
554
  ```
557
555
 
@@ -575,7 +573,7 @@ A few things to note:
575
573
 
576
574
  Now let's see what is output when we run this configuration with `rake sitemap:refresh:no_ping`:
577
575
 
578
- ```
576
+ ```text
579
577
  In /Users/karl/projects/sitemap_generator-test/public/
580
578
  + sitemap.xml.gz 2 links / 347 Bytes
581
579
  Sitemap stats: 2 links / 1 sitemaps / 0m00s
@@ -585,7 +583,6 @@ Weird! The sitemap has two links, even though we only added one! This is becau
585
583
 
586
584
  Now let's take a look at the file that was created. After uncompressing and XML-tidying the contents we have:
587
585
 
588
-
589
586
  * `public/sitemap.xml.gz`
590
587
 
591
588
  ```xml
@@ -620,7 +617,7 @@ end
620
617
 
621
618
  And the output:
622
619
 
623
- ```
620
+ ```text
624
621
  In /Users/karl/projects/sitemap_generator-test/public/
625
622
  + sitemap1.xml.gz 2 links / 347 Bytes
626
623
  + sitemap.xml.gz 1 sitemaps / 228 Bytes
@@ -665,7 +662,7 @@ In the example about we pass a `lastmod` (last modified) option with the value o
665
662
 
666
663
  Looking at the output from running this sitemap, we see that we have a few more links than before:
667
664
 
668
- ```
665
+ ```text
669
666
  + sitemap.xml.gz 12 links / 2.3 KB / 365 Bytes gzipped
670
667
  Sitemap stats: 12 links / 1 sitemaps / 0m00s
671
668
  ```
@@ -773,18 +770,28 @@ end
773
770
 
774
771
  The output looks something like this:
775
772
 
776
- ```
773
+ ```text
777
774
  In /Users/karl/projects/sitemap_generator-test/public/
778
775
  + sitemap4.xml.gz 3 links / 355 Bytes
779
776
  + sitemap.xml.gz 4 sitemaps / 242 Bytes
780
777
  Sitemap stats: 3 links / 4 sitemaps / 0m00s
781
778
  ```
782
779
 
780
+ ### Using `create` without a block
781
+
782
+ You can call `create` without a block, add links (and use `group` as needed), then call `finalize!` on the returned link set when finished.
783
+
784
+ ```ruby
785
+ sitemap = SitemapGenerator::Sitemap.create(default_host: 'http://www.example.com')
786
+ sitemap.add('/home')
787
+ # ... add more links or use groups, then:
788
+ sitemap.finalize!
789
+ ```
790
+
783
791
  ### Speeding Things Up
784
792
 
785
793
  For large ActiveRecord collections with thousands of records it is advisable to iterate through them in batches to avoid loading all records into memory at once. For this reason in the example above we use `Content.find_each` which is a batched iterator available since Rails 2.3.2, rather than `Content.all`.
786
794
 
787
-
788
795
  ## Customizing your Sitemaps
789
796
 
790
797
  SitemapGenerator supports a number of options which allow you to control every aspect of your sitemap generation. How they are named, where they are stored, the contents of the links and the location that the sitemaps will be hosted from can all be set.
@@ -841,8 +848,8 @@ The following options are supported.
841
848
  * `:public_path` - String. A **full or relative path** to the `public` directory or the directory you want to write sitemaps into. Defaults to `public/` under your application root or relative to the current working directory.
842
849
 
843
850
  * `:sitemaps_host` - String. **Host including protocol** to use when generating a link to a sitemap file i.e. the hostname of the server where the sitemaps are hosted. The value will differ from the hostname in your sitemap links. For example: `'http://amazon.aws.com/'`. Note that `include_index` is
844
- automatically turned off when the `sitemaps_host` does not match `default_host`.
845
- Because the link to the sitemap index file that would otherwise be added would point to a different host than the rest of the links in the sitemap. Something that the sitemap rules forbid.
851
+ automatically turned off when the `sitemaps_host` does not match `default_host`.
852
+ Because the link to the sitemap index file that would otherwise be added would point to a different host than the rest of the links in the sitemap. Something that the sitemap rules forbid.
846
853
 
847
854
  * `:namer` - A `SitemapGenerator::SimpleNamer` instance **for generating sitemap names**. You can read about Sitemap Namers by reading the API docs. Allows you to set the name, extension and number sequence for sitemap files, as well as modify the name of the first file in the sequence, which is often the index file. A simple example if we want to generate files like 'newname.xml.gz', 'newname1.xml.gz', etc is `SitemapGenerator::SimpleNamer.new(:newname)`.
848
855
 
@@ -853,9 +860,9 @@ Because the link to the sitemap index file that would otherwise be added would p
853
860
  * `:adapter` - Instance. The default adapter is a `SitemapGenerator::FileAdapter` which simply writes files to the filesystem. You can use a `SitemapGenerator::WaveAdapter` for uploading sitemaps to remote servers - useful for read-only hosts such as Heroku. Or you can provide an instance of your own class to provide custom behavior. Your class must define a write method which takes a `SitemapGenerator::Location` and raw XML data.
854
861
 
855
862
  * `:compress` - Specifies which files to compress with gzip. Default is `true`. Accepted values:
856
- * `true` - Boolean; compress all files.
857
- * `false` - Boolean; Do not compress any files.
858
- * `:all_but_first` - Symbol; leave the first file uncompressed but compress all remaining files.
863
+ * `true` - Boolean; compress all files.
864
+ * `false` - Boolean; Do not compress any files.
865
+ * `:all_but_first` - Symbol; leave the first file uncompressed but compress all remaining files.
859
866
 
860
867
  The compression setting applies to groups too. So `:all_but_first` will have the same effect (the first file in the group will not be compressed, the rest will). So if you require different behaviour for your groups, pass in a `:compress` option e.g. `group(:compress => false) { add('/link') }`
861
868
 
@@ -896,7 +903,7 @@ end
896
903
 
897
904
  And the output from running the above:
898
905
 
899
- ```
906
+ ```text
900
907
  In /Users/karl/projects/sitemap_generator-test/public/
901
908
  + en/english.xml.gz 1 links / 328 Bytes
902
909
  + fr/french.xml.gz 1 links / 329 Bytes
@@ -945,7 +952,7 @@ end
945
952
 
946
953
  And the output from running the above:
947
954
 
948
- ```
955
+ ```text
949
956
  In '/Users/kvarga/Projects/sitemap_generator-test/public/':
950
957
  + odds.xml.gz 10 links / 371 Bytes
951
958
  + evens.xml.gz 10 links / 371 Bytes
@@ -980,14 +987,14 @@ end
980
987
  #### Supported options
981
988
 
982
989
  * `:news` - Hash
983
- * `:publication_name`
984
- * `:publication_language`
985
- * `:publication_date`
986
- * `:genres`
987
- * `:access`
988
- * `:title`
989
- * `:keywords`
990
- * `:stock_tickers`
990
+ * `:publication_name`
991
+ * `:publication_language`
992
+ * `:publication_date`
993
+ * `:genres`
994
+ * `:access`
995
+ * `:title`
996
+ * `:keywords`
997
+ * `:stock_tickers`
991
998
 
992
999
  ### Image Sitemaps
993
1000
 
@@ -1007,11 +1014,11 @@ end
1007
1014
  #### Supported options
1008
1015
 
1009
1016
  * `:images` - Array of hashes
1010
- * `:loc` Required, location of the image
1011
- * `:caption`
1012
- * `:geo_location`
1013
- * `:title`
1014
- * `:license`
1017
+ * `:loc` Required, location of the image
1018
+ * `:caption`
1019
+ * `:geo_location`
1020
+ * `:title`
1021
+ * `:license`
1015
1022
 
1016
1023
  ### Video Sitemaps
1017
1024
 
@@ -1038,32 +1045,32 @@ end
1038
1045
  #### Supported options
1039
1046
 
1040
1047
  * `:video` or `:videos` - Hash or array of hashes, respectively
1041
- * `:thumbnail_loc` - Required. String, URL of the thumbnail image.
1042
- * `:title` - Required. String, title of the video.
1043
- * `:description` - Required. String, description of the video.
1044
- * `:content_loc` - Depends. String, URL. One of content_loc or player_loc must be present.
1045
- * `:player_loc` - Depends. String, URL. One of content_loc or player_loc must be present.
1046
- * `:allow_embed` - Boolean, attribute of player_loc.
1047
- * `:autoplay` - Boolean, default true. Attribute of player_loc.
1048
- * `:duration` - Recommended. Integer or string. Duration in seconds.
1049
- * `:expiration_date` - Recommended when applicable. The date after which the video will no longer be available.
1050
- * `:rating` - Optional
1051
- * `:view_count` - Optional. Integer or string.
1052
- * `:publication_date` - Optional
1053
- * `:tags` - Optional. Array of string tags.
1054
- * `:tag` - Optional. String, single tag.
1055
- * `:category` - Optional
1056
- * `:family_friendly`- Optional. Boolean
1057
- * `:gallery_loc` - Optional. String, URL.
1058
- * `:gallery_title` - Optional. Title attribute of the gallery location element
1059
- * `:uploader` - Optional.
1060
- * `:uploader_info` - Optional. Info attribute of uploader element
1061
- * `:price` - Optional. Only one price supported at this time
1062
- * `:price_currency` - Required. In [ISO_4217][iso_4217] format.
1063
- * `:price_type` - Optional. `rent` or `own`
1064
- * `:price_resolution` - Optional. `HD` or `SD`
1065
- * `:live` - Optional. Boolean.
1066
- * `:requires_subscription` - Optional. Boolean.
1048
+ * `:thumbnail_loc` - Required. String, URL of the thumbnail image.
1049
+ * `:title` - Required. String, title of the video.
1050
+ * `:description` - Required. String, description of the video.
1051
+ * `:content_loc` - Depends. String, URL. One of content_loc or player_loc must be present.
1052
+ * `:player_loc` - Depends. String, URL. One of content_loc or player_loc must be present.
1053
+ * `:allow_embed` - Boolean, attribute of player_loc.
1054
+ * `:autoplay` - Boolean, default true. Attribute of player_loc.
1055
+ * `:duration` - Recommended. Integer or string. Duration in seconds.
1056
+ * `:expiration_date` - Recommended when applicable. The date after which the video will no longer be available.
1057
+ * `:rating` - Optional
1058
+ * `:view_count` - Optional. Integer or string.
1059
+ * `:publication_date` - Optional
1060
+ * `:tags` - Optional. Array of string tags.
1061
+ * `:tag` - Optional. String, single tag.
1062
+ * `:category` - Optional
1063
+ * `:family_friendly`- Optional. Boolean
1064
+ * `:gallery_loc` - Optional. String, URL.
1065
+ * `:gallery_title` - Optional. Title attribute of the gallery location element
1066
+ * `:uploader` - Optional.
1067
+ * `:uploader_info` - Optional. Info attribute of uploader element
1068
+ * `:price` - Optional. Only one price supported at this time
1069
+ * `:price_currency` - Required. In [ISO_4217][iso_4217] format.
1070
+ * `:price_type` - Optional. `rent` or `own`
1071
+ * `:price_resolution` - Optional. `HD` or `SD`
1072
+ * `:live` - Optional. Boolean.
1073
+ * `:requires_subscription` - Optional. Boolean.
1067
1074
 
1068
1075
  ### PageMap Sitemaps
1069
1076
 
@@ -1072,14 +1079,14 @@ Pagemaps can be added by passing a `:pagemap` hash to `add`. The hash must conta
1072
1079
  #### Supported options
1073
1080
 
1074
1081
  * `:pagemap` - Hash
1075
- * `:dataobjects` - Required, array of hashes
1076
- * `:type` - Required, string, type of the object
1077
- * `:id` - String, ID of the object
1078
- * `:attributes` - Array of hashes
1079
- * `:name` - Required, string, name of the attribute.
1080
- * `:value` - String, value of the attribute.
1082
+ * `:dataobjects` - Required, array of hashes
1083
+ * `:type` - Required, string, type of the object
1084
+ * `:id` - String, ID of the object
1085
+ * `:attributes` - Array of hashes
1086
+ * `:name` - Required, string, name of the attribute.
1087
+ * `:value` - String, value of the attribute.
1081
1088
 
1082
- #### Example:
1089
+ #### Example
1083
1090
 
1084
1091
  ```ruby
1085
1092
  SitemapGenerator::Sitemap.default_host = "http://www.example.com"
@@ -1121,10 +1128,10 @@ end
1121
1128
  #### Supported options
1122
1129
 
1123
1130
  * `:alternate`/`:alternates` - Hash or array of hashes, respectively
1124
- * `:href` - Required, string.
1125
- * `:lang` - Optional, string.
1126
- * `:nofollow` - Optional, boolean. Used to mark link as "nofollow".
1127
- * `:media` - Optional, string. Specify [media targets for responsive design pages][media].
1131
+ * `:href` - Required, string.
1132
+ * `:lang` - Optional, string.
1133
+ * `:nofollow` - Optional, boolean. Used to mark link as "nofollow".
1134
+ * `:media` - Optional, string. Specify [media targets for responsive design pages][media].
1128
1135
 
1129
1136
  #### Alternates Example
1130
1137
 
@@ -1150,7 +1157,7 @@ end
1150
1157
 
1151
1158
  Mobile sitemaps include a specific `<mobile:mobile/>` tag.
1152
1159
 
1153
- Check out the Google specification [here][sitemap_mobile].
1160
+ > **Deprecated**: Google deprecated mobile sitemaps in August 2022. No other major search engines supported this format. Responsive design is the recommended approach for mobile-friendly content.
1154
1161
 
1155
1162
  #### Example
1156
1163
 
@@ -1167,25 +1174,23 @@ end
1167
1174
 
1168
1175
  ## Compatibility
1169
1176
 
1170
- Compatible with all versions of Rails and Ruby. Tested up to Ruby 3.1 and Rails 7.0.
1171
- Ruby 1.9.3 support was dropped in Version 6.0.0.
1177
+ Ruby 2.6 through 4.0 and Rails 6.0 through 8.1 are supported.
1178
+ Ruby 2.5 and Rails 5.2 were dropped in v7.0.0.
1179
+ Ruby 1.9.3 support was dropped in v6.0.0.
1172
1180
 
1173
1181
  ## Licence
1174
1182
 
1175
- Released under the MIT License. See the (MIT-LICENSE)[MIT-LICENSE] file.
1176
-
1177
- MIT. See the LICENSE.md file.
1183
+ Released under the MIT license which is included in the [MIT-LICENSE](./MIT-LICENSE) file.
1178
1184
 
1179
- Copyright (c) Karl Varga released under the MIT license
1185
+ Copyright (c) Karl Varga
1180
1186
 
1181
1187
  [canonical_repo]:http://github.com/kjvarga/sitemap_generator
1182
1188
  [sitemap_images]:http://www.google.com/support/webmasters/bin/answer.py?answer=178636
1183
1189
  [sitemap_video]:https://support.google.com/webmasters/answer/80471?hl=en&ref_topic=4581190
1184
1190
  [sitemap_news]:https://support.google.com/news/publisher/topic/2527688?hl=en&ref_topic=4359874
1185
- [sitemap_mobile]:http://support.google.com/webmasters/bin/answer.py?hl=en&answer=34648
1186
1191
  [sitemap_pagemap]:https://developers.google.com/custom-search/docs/structured_data#addtositemap
1187
1192
  [sitemap_protocol]:http://www.sitemaps.org/protocol.html
1188
- [video_tags]:http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=80472#4
1193
+ [video_tags]:https://developers.google.com/search/docs/crawling-indexing/sitemaps/video-sitemaps
1189
1194
  [image_tags]:http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=178636
1190
1195
  [news_tags]:http://www.google.com/support/news_pub/bin/answer.py?answer=74288
1191
1196
  [remote_hosts]:https://github.com/kjvarga/sitemap_generator/wiki/Generate-Sitemaps-on-read-only-filesystems-like-Heroku
@@ -1193,7 +1198,7 @@ Copyright (c) Karl Varga released under the MIT license
1193
1198
  [using_pagemaps]:https://developers.google.com/custom-search/docs/structured_data#pagemaps
1194
1199
  [iso_4217]:http://en.wikipedia.org/wiki/ISO_4217
1195
1200
  [media]:https://developers.google.com/webmasters/smartphone-sites/details
1196
- [expires]:https://support.google.com/customsearch/answer/2631051?hl=en
1201
+ [expires]:https://support.google.com/webmasters/answer/9689846
1197
1202
  [google_cloud_storage_gem]:https://rubygems.org/gems/google-cloud-storage
1198
1203
  [google_cloud_storage_authentication]:https://googleapis.dev/ruby/google-cloud-storage/latest/file.AUTHENTICATION.html
1199
1204
  [google_cloud_storage_initializer]:https://github.com/googleapis/google-cloud-ruby/blob/master/google-cloud-storage/lib/google/cloud/storage.rb