sitemap_generator 6.2.1 → 7.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGES.md +23 -0
- data/README.md +43 -37
- data/VERSION +1 -1
- data/lib/sitemap_generator/adapters/active_storage_adapter.rb +26 -0
- data/lib/sitemap_generator/adapters/aws_sdk_adapter.rb +14 -8
- data/lib/sitemap_generator/adapters/file_adapter.rb +2 -0
- data/lib/sitemap_generator/adapters/fog_adapter.rb +7 -5
- data/lib/sitemap_generator/adapters/google_storage_adapter.rb +7 -3
- data/lib/sitemap_generator/adapters/s3_adapter.rb +10 -6
- data/lib/sitemap_generator/adapters/wave_adapter.rb +3 -1
- data/lib/sitemap_generator/application.rb +4 -0
- data/lib/sitemap_generator/builder/sitemap_file.rb +17 -12
- data/lib/sitemap_generator/builder/sitemap_index_file.rb +12 -9
- data/lib/sitemap_generator/builder/sitemap_index_url.rb +6 -4
- data/lib/sitemap_generator/builder/sitemap_url.rb +59 -55
- data/lib/sitemap_generator/builder.rb +3 -1
- data/lib/sitemap_generator/core_ext/big_decimal.rb +2 -0
- data/lib/sitemap_generator/core_ext/numeric.rb +2 -0
- data/lib/sitemap_generator/core_ext.rb +2 -0
- data/lib/sitemap_generator/helpers/number_helper.rb +35 -33
- data/lib/sitemap_generator/interpreter.rb +6 -4
- data/lib/sitemap_generator/link_set.rb +65 -63
- data/lib/sitemap_generator/railtie.rb +3 -1
- data/lib/sitemap_generator/simple_namer.rb +8 -5
- data/lib/sitemap_generator/sitemap_location.rb +13 -9
- data/lib/sitemap_generator/tasks.rb +10 -8
- data/lib/sitemap_generator/templates.rb +8 -5
- data/lib/sitemap_generator/utilities.rb +12 -9
- data/lib/sitemap_generator.rb +13 -9
- data/templates/sitemap.rb +1 -1
- metadata +6 -130
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b26cd553fe4ecc452da26ac6aff3262238f065dc12d9f328959eb3c1edc79f6e
|
|
4
|
+
data.tar.gz: 877828e867880ed215df8b3e1f6acb130e4e8c5856a349e61e6af17a5c641ac9
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d957d69f7a4c2fe8e98cecb84045d58fe59f9f8143764c88695a7561a63d54e9d515c80bdc2b2cf496b1704ffb2ca4610b54f5889cb246e2ed45aac88029abdd
|
|
7
|
+
data.tar.gz: 884fc7be3809f6670cce09afdddfa64cdf6b4c5b25cdc7ad5c7c3b784d08f83bbc96bdb671a1b52de7f0f59fac8618f1bc83122c01664ee574a52c4edd791699
|
data/CHANGES.md
CHANGED
|
@@ -1,3 +1,26 @@
|
|
|
1
|
+
### 7.0.1
|
|
2
|
+
|
|
3
|
+
* **Breaking:** Default search engines list is empty. `rake sitemap:refresh` and `ping_search_engines` perform no HTTP pings unless you configure engine URLs on `search_engines` or pass them into `ping_search_engines` (Google’s ping endpoint is deprecated upstream). [#444](https://github.com/kjvarga/sitemap_generator/pull/444)
|
|
4
|
+
* **Breaking:** `LinkSet#create` runs `finalize!` only when a block is given. Calling `create` without a block requires `finalize!` when you are done adding links (supported workflow for programmatic builds). [#463](https://github.com/kjvarga/sitemap_generator/pull/463)
|
|
5
|
+
* **Breaking:** Drop Ruby 2.5, Rails 5.2 [#438](https://github.com/kjvarga/sitemap_generator/pull/438)
|
|
6
|
+
* Migrate continuous integration from CircleCI to GitHub Actions with expanded Ruby × Rails CI matrix.
|
|
7
|
+
* Support AWS temporary credentials (`aws_session_token` / `AWS_SESSION_TOKEN`) for S3 uploads in AWS Lambda [#415](https://github.com/kjvarga/sitemap_generator/pull/415)
|
|
8
|
+
* Add `frozen_string_literal: true` magic comments [#430](https://github.com/kjvarga/sitemap_generator/pull/430)
|
|
9
|
+
* Add support for Rails 8.0 [#441](https://github.com/kjvarga/sitemap_generator/pull/441)
|
|
10
|
+
* Fix uninitialized constant ActiveSupport::LoggerThreadSafeLevel::Logger [#449](https://github.com/kjvarga/sitemap_generator/pull/449)
|
|
11
|
+
* Add support for Ruby 3.4 [#451](https://github.com/kjvarga/sitemap_generator/pull/451)
|
|
12
|
+
* Add support for Rails 8.1 [#461](https://github.com/kjvarga/sitemap_generator/pull/461)
|
|
13
|
+
* Replace CGI-based escaping with `URI.encode_www_form_component` for ping URLs; remove reliance on `cgi` (Ruby 4 compatibility) [#465](https://github.com/kjvarga/sitemap_generator/pull/465)
|
|
14
|
+
* Add support for Ruby 4.0 [#466](https://github.com/kjvarga/sitemap_generator/pull/466)
|
|
15
|
+
* Add ActiveStorage adapter (`ActiveStorage::Blob`) [#467](https://github.com/kjvarga/sitemap_generator/pull/467)
|
|
16
|
+
|
|
17
|
+
### 6.3.0
|
|
18
|
+
|
|
19
|
+
* Remove Bing's deprecated sitemap submission [#400](https://github.com/kjvarga/sitemap_generator/pull/400).
|
|
20
|
+
* `SitemapGenerator::AwsSdkAdapter`: Support configuring ACL and caching on the uploaded files [#409](https://github.com/kjvarga/sitemap_generator/pull/409).
|
|
21
|
+
* `SitemapGenerator::GoogleStorageAdapter`: Support configuring ACL on the uploaded files [#410](https://github.com/kjvarga/sitemap_generator/pull/410).
|
|
22
|
+
* Fix CircleCI specs for Ruby 3 [#407](https://github.com/kjvarga/sitemap_generator/pull/407).
|
|
23
|
+
|
|
1
24
|
### 6.2.1
|
|
2
25
|
|
|
3
26
|
* Bugfix: Improve handling of deprecated options in `AwsSdkAdapter`. Fixes bug where `:region` was being set to `nil`. [#390](https://github.com/kjvarga/sitemap_generator/pull/390).
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# SitemapGenerator
|
|
2
2
|
|
|
3
|
-
[](https://github.com/kjvarga/sitemap_generator/actions/workflows/ci.yml)
|
|
4
4
|
|
|
5
5
|
SitemapGenerator is the easiest way to generate Sitemaps in Ruby. Rails integration provides access to the Rails route helpers within your sitemap config file and automatically makes the rake tasks available to you. Or if you prefer to use another framework, you can! You can use the rake tasks provided or run your sitemap configs as plain ruby scripts.
|
|
6
6
|
|
|
@@ -11,14 +11,15 @@ Sitemaps adhere to the [Sitemap 0.9 protocol][sitemap_protocol] specification.
|
|
|
11
11
|
* Framework agnostic
|
|
12
12
|
* Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images], [Mobile sitemaps][sitemap_mobile], [PageMap sitemaps][sitemap_pagemap] and [Alternate Links][alternate_links]
|
|
13
13
|
* Supports read-only filesystems like Heroku via uploading to a remote host like Amazon S3
|
|
14
|
-
* Compatible with all versions of Rails and Ruby
|
|
15
14
|
* Adheres to the [Sitemap 0.9 protocol][sitemap_protocol]
|
|
16
15
|
* Handles millions of links
|
|
17
16
|
* Customizable sitemap compression
|
|
18
|
-
*
|
|
17
|
+
* Optional HTTP notification of search engines when `search_engines` is configured
|
|
19
18
|
* Ensures your old sitemaps stay in place if the new sitemap fails to generate
|
|
20
19
|
* Gives you complete control over your sitemap contents and naming scheme
|
|
21
20
|
* Intelligent sitemap indexing
|
|
21
|
+
* Compatible with Ruby from version 2.6 to version 4.0
|
|
22
|
+
* Compatible with Rails from version 6.0 to version 8.1
|
|
22
23
|
|
|
23
24
|
### Show Me
|
|
24
25
|
|
|
@@ -41,7 +42,6 @@ SitemapGenerator::Sitemap.create do
|
|
|
41
42
|
add '/home', :changefreq => 'daily', :priority => 0.9
|
|
42
43
|
add '/contact_us', :changefreq => 'weekly'
|
|
43
44
|
end
|
|
44
|
-
SitemapGenerator::Sitemap.ping_search_engines # Not needed if you use the rake tasks
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
Run it:
|
|
@@ -56,9 +56,6 @@ Output:
|
|
|
56
56
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
57
57
|
+ sitemap.xml.gz 3 links / 364 Bytes
|
|
58
58
|
Sitemap stats: 3 links / 1 sitemaps / 0m00s
|
|
59
|
-
|
|
60
|
-
Successful ping of Google
|
|
61
|
-
Successful ping of Bing
|
|
62
59
|
```
|
|
63
60
|
|
|
64
61
|
## Contents
|
|
@@ -67,7 +64,6 @@ Successful ping of Bing
|
|
|
67
64
|
- [Features](#features)
|
|
68
65
|
- [Show Me](#show-me)
|
|
69
66
|
- [Contents](#contents)
|
|
70
|
-
- [Contribute](#contribute)
|
|
71
67
|
- [Foreword](#foreword)
|
|
72
68
|
- [Installation](#installation)
|
|
73
69
|
- [Ruby](#ruby)
|
|
@@ -79,11 +75,12 @@ Successful ping of Bing
|
|
|
79
75
|
- [Crontab](#crontab)
|
|
80
76
|
- [Robots.txt](#robotstxt)
|
|
81
77
|
- [Ruby Modules](#ruby-modules)
|
|
82
|
-
- [Deployments
|
|
78
|
+
- [Deployments \& Capistrano](#deployments--capistrano)
|
|
83
79
|
- [Sitemaps with no Index File](#sitemaps-with-no-index-file)
|
|
84
80
|
- [Upload Sitemaps to a Remote Host using Adapters](#upload-sitemaps-to-a-remote-host-using-adapters)
|
|
85
81
|
- [Supported Adapters](#supported-adapters)
|
|
86
82
|
- [`SitemapGenerator::FileAdapter`](#sitemapgeneratorfileadapter)
|
|
83
|
+
- [`SitemapGenerator::ActiveStorageAdapter`](#sitemapgeneratoractivestorageadapter)
|
|
87
84
|
- [`SitemapGenerator::FogAdapter`](#sitemapgeneratorfogadapter)
|
|
88
85
|
- [`SitemapGenerator::S3Adapter`](#sitemapgenerators3adapter)
|
|
89
86
|
- [`SitemapGenerator::AwsSdkAdapter`](#sitemapgeneratorawssdkadapter)
|
|
@@ -97,6 +94,7 @@ Successful ping of Bing
|
|
|
97
94
|
- [Supported Options to `add`](#supported-options-to-add)
|
|
98
95
|
- [Adding Links to the Sitemap Index](#adding-links-to-the-sitemap-index)
|
|
99
96
|
- [Accessing the LinkSet instance](#accessing-the-linkset-instance)
|
|
97
|
+
- [Using `create` without a block](#using-create-without-a-block)
|
|
100
98
|
- [Speeding Things Up](#speeding-things-up)
|
|
101
99
|
- [Customizing your Sitemaps](#customizing-your-sitemaps)
|
|
102
100
|
- [Sitemap Options](#sitemap-options)
|
|
@@ -126,13 +124,6 @@ Successful ping of Bing
|
|
|
126
124
|
- [Compatibility](#compatibility)
|
|
127
125
|
- [Licence](#licence)
|
|
128
126
|
|
|
129
|
-
## Contribute
|
|
130
|
-
|
|
131
|
-
Does your website use SitemapGenerator to generate Sitemaps? Where would you be without Sitemaps? Probably still knocking rocks together. Consider donating to the project to keep it up-to-date and open source.
|
|
132
|
-
|
|
133
|
-
<a href='http://www.pledgie.com/campaigns/15267'><img alt='Click here to lend your support to: SitemapGenerator and make a donation at www.pledgie.com !' src='http://pledgie.com/campaigns/15267.png?skin_name=chrome' border='0' /></a>
|
|
134
|
-
|
|
135
|
-
|
|
136
127
|
## Foreword
|
|
137
128
|
|
|
138
129
|
Adam Salter first created SitemapGenerator while we were working together in Sydney, Australia. Unfortunately, he passed away in 2009. Since then I have taken over development of SitemapGenerator.
|
|
@@ -160,7 +151,7 @@ The Rake tasks expect your sitemap to be at `config/sitemap.rb` but if you need
|
|
|
160
151
|
|
|
161
152
|
### Rails
|
|
162
153
|
|
|
163
|
-
SitemapGenerator
|
|
154
|
+
SitemapGenerator targets Rails 6.0 through 8.1 (see [Compatibility](#compatibility)).
|
|
164
155
|
|
|
165
156
|
Add the gem to your `Gemfile`:
|
|
166
157
|
|
|
@@ -205,10 +196,10 @@ SitemapGenerator.verbose = false
|
|
|
205
196
|
|
|
206
197
|
### Pinging Search Engines
|
|
207
198
|
|
|
208
|
-
Using `rake sitemap:refresh` will notify
|
|
199
|
+
Using `rake sitemap:refresh` will notify configured search engines to let them know that a new sitemap
|
|
209
200
|
is available. To generate new sitemaps without notifying search engines, use `rake sitemap:refresh:no_ping`.
|
|
210
201
|
|
|
211
|
-
If you want to customize the hash of search engines you can access it at:
|
|
202
|
+
By default no search engines are configured. If you want to customize the hash of search engines you can access it at:
|
|
212
203
|
|
|
213
204
|
```ruby
|
|
214
205
|
SitemapGenerator::Sitemap.search_engines
|
|
@@ -225,7 +216,7 @@ directly in the call, as in the following example:
|
|
|
225
216
|
SitemapGenerator::Sitemap.ping_search_engines(newengine: 'http://newengine.com/ping?url=%s')
|
|
226
217
|
```
|
|
227
218
|
|
|
228
|
-
The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the
|
|
219
|
+
The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the URL-encoded (percent-encoded) sitemap index URL. If you have any literal percent characters in your URL you need to escape them with `%%`.
|
|
229
220
|
|
|
230
221
|
If you are calling `SitemapGenerator::Sitemap.ping_search_engines` from outside of your sitemap config file, then you will need to set `SitemapGenerator::Sitemap.default_host` and any other options that you set in your sitemap config which affect the location of the sitemap index file. For example:
|
|
231
222
|
|
|
@@ -343,6 +334,10 @@ directory.
|
|
|
343
334
|
|
|
344
335
|
Standard adapter, writes out to a file.
|
|
345
336
|
|
|
337
|
+
##### `SitemapGenerator::ActiveStorageAdapter`
|
|
338
|
+
|
|
339
|
+
Uses `ActiveStorage::Blob` to store the sitemap.
|
|
340
|
+
|
|
346
341
|
##### `SitemapGenerator::FogAdapter`
|
|
347
342
|
|
|
348
343
|
Uses `Fog::Storage` to upload to any service supported by Fog.
|
|
@@ -365,6 +360,7 @@ directory.
|
|
|
365
360
|
Where `options` is a Hash with any of the following keys:
|
|
366
361
|
* `aws_access_key_id` [String] Your AWS access key id
|
|
367
362
|
* `aws_secret_access_key` [String] Your AWS secret access key
|
|
363
|
+
* `aws_session_token` [String] Session token for temporary credentials (optional)
|
|
368
364
|
* `fog_provider` [String]
|
|
369
365
|
* `fog_directory` [String]
|
|
370
366
|
* `fog_region` [String]
|
|
@@ -373,7 +369,7 @@ directory.
|
|
|
373
369
|
* `fog_public` [Boolean] Whether the file is publicly accessible
|
|
374
370
|
|
|
375
371
|
Alternatively you can use an environment variable to configure each option (except `fog_storage_options`). The environment variables have the same
|
|
376
|
-
name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
372
|
+
name but capitalized, e.g. `AWS_SESSION_TOKEN`, `FOG_PATH_STYLE`.
|
|
377
373
|
|
|
378
374
|
##### `SitemapGenerator::AwsSdkAdapter`
|
|
379
375
|
|
|
@@ -387,18 +383,19 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
387
383
|
|
|
388
384
|
```ruby
|
|
389
385
|
SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new('s3_bucket',
|
|
390
|
-
|
|
391
|
-
|
|
386
|
+
acl: 'public-read', # Optional. This is the default.
|
|
387
|
+
cache_control: 'private, max-age=0, no-cache', # Optional. This is the default.
|
|
388
|
+
access_key_id: 'YOUR_AWS_ACCESS_KEY_ID',
|
|
389
|
+
secret_access_key: 'YOUR_AWS_SECRET_ACCESS_KEY',
|
|
390
|
+
session_token: 'YOUR_AWS_SESSION_TOKEN', # Optional; use with temporary credentials.
|
|
392
391
|
region: 'us-east-1',
|
|
393
392
|
endpoint: 'https://sfo2.digitaloceanspaces.com'
|
|
394
393
|
)
|
|
395
394
|
```
|
|
396
395
|
|
|
397
|
-
Where the first argument is the S3 bucket name, and the rest are keyword argument options
|
|
398
|
-
are passed directly to the AWS client.
|
|
396
|
+
Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files; `session_token` supports STS-style credentials (or set `AWS_SESSION_TOKEN` and rely on SDK defaults). All other keyword options are passed directly to the AWS client.
|
|
399
397
|
|
|
400
|
-
See https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method
|
|
401
|
-
for a full list of supported options.
|
|
398
|
+
See [the `SitemapGenerator::AwsSdkAdapter` docs](https://github.com/kjvarga/sitemap_generator/blob/master/lib/sitemap_generator/adapters/aws_sdk_adapter.rb), and [https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method](https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method) for the full list of supported options.
|
|
402
399
|
|
|
403
400
|
##### `SitemapGenerator::WaveAdapter`
|
|
404
401
|
|
|
@@ -420,9 +417,10 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
420
417
|
|
|
421
418
|
```ruby
|
|
422
419
|
SitemapGenerator::Sitemap.adapter = SitemapGenerator::GoogleStorageAdapter.new(
|
|
420
|
+
acl: 'public', # Optional. This is the default value.
|
|
421
|
+
bucket: 'name_of_bucket'
|
|
423
422
|
credentials: 'path/to/keyfile.json',
|
|
424
423
|
project_id: 'google_account_project_id',
|
|
425
|
-
bucket: 'name_of_bucket'
|
|
426
424
|
)
|
|
427
425
|
```
|
|
428
426
|
Also, inline with Google Authentication options, it can also pick credentials from environment variables. All [supported environment variables][google_cloud_storage_authentication] can be used, for example: `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_CREDENTIALS`. An example of using this adapter with the environment variables is:
|
|
@@ -433,7 +431,7 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
433
431
|
)
|
|
434
432
|
```
|
|
435
433
|
|
|
436
|
-
All options other than the `:bucket`
|
|
434
|
+
All options other than the `:bucket` and `:acl` options are passed to the `Google::Cloud::Storage.new` initializer giving you maximum configurability. See the [Google Cloud Storage initializer][google_cloud_storage_initializer] for supported options.
|
|
437
435
|
|
|
438
436
|
#### An Example of Using an Adapter
|
|
439
437
|
|
|
@@ -536,7 +534,6 @@ SitemapGenerator::Sitemap.create do
|
|
|
536
534
|
end
|
|
537
535
|
```
|
|
538
536
|
|
|
539
|
-
|
|
540
537
|
To generate each one specify the configuration file to run by passing the `CONFIG_FILE` option to `rake sitemap:refresh`, e.g.:
|
|
541
538
|
|
|
542
539
|
```
|
|
@@ -780,11 +777,21 @@ In /Users/karl/projects/sitemap_generator-test/public/
|
|
|
780
777
|
Sitemap stats: 3 links / 4 sitemaps / 0m00s
|
|
781
778
|
```
|
|
782
779
|
|
|
780
|
+
### Using `create` without a block
|
|
781
|
+
|
|
782
|
+
You can call `create` without a block, add links (and use `group` as needed), then call `finalize!` on the returned link set when finished.
|
|
783
|
+
|
|
784
|
+
```ruby
|
|
785
|
+
sitemap = SitemapGenerator::Sitemap.create(default_host: 'http://www.example.com')
|
|
786
|
+
sitemap.add('/home')
|
|
787
|
+
# ... add more links or use groups, then:
|
|
788
|
+
sitemap.finalize!
|
|
789
|
+
```
|
|
790
|
+
|
|
783
791
|
### Speeding Things Up
|
|
784
792
|
|
|
785
793
|
For large ActiveRecord collections with thousands of records it is advisable to iterate through them in batches to avoid loading all records into memory at once. For this reason in the example above we use `Content.find_each` which is a batched iterator available since Rails 2.3.2, rather than `Content.all`.
|
|
786
794
|
|
|
787
|
-
|
|
788
795
|
## Customizing your Sitemaps
|
|
789
796
|
|
|
790
797
|
SitemapGenerator supports a number of options which allow you to control every aspect of your sitemap generation. How they are named, where they are stored, the contents of the links and the location that the sitemaps will be hosted from can all be set.
|
|
@@ -1167,16 +1174,15 @@ end
|
|
|
1167
1174
|
|
|
1168
1175
|
## Compatibility
|
|
1169
1176
|
|
|
1170
|
-
|
|
1171
|
-
Ruby
|
|
1177
|
+
Ruby 2.6 through 4.0 and Rails 6.0 through 8.1 are supported.
|
|
1178
|
+
Ruby 2.5 and Rails 5.2 were dropped in v7.0.0.
|
|
1179
|
+
Ruby 1.9.3 support was dropped in v6.0.0.
|
|
1172
1180
|
|
|
1173
1181
|
## Licence
|
|
1174
1182
|
|
|
1175
|
-
Released under the MIT
|
|
1176
|
-
|
|
1177
|
-
MIT. See the LICENSE.md file.
|
|
1183
|
+
Released under the MIT license which is included in the [MIT-LICENSE](./MIT-LICENSE) file.
|
|
1178
1184
|
|
|
1179
|
-
Copyright (c) Karl Varga
|
|
1185
|
+
Copyright (c) Karl Varga
|
|
1180
1186
|
|
|
1181
1187
|
[canonical_repo]:http://github.com/kjvarga/sitemap_generator
|
|
1182
1188
|
[sitemap_images]:http://www.google.com/support/webmasters/bin/answer.py?answer=178636
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
7.0.1
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
module SitemapGenerator
|
|
2
|
+
# Class for uploading sitemaps to ActiveStorage.
|
|
3
|
+
class ActiveStorageAdapter
|
|
4
|
+
attr_reader :key, :filename
|
|
5
|
+
|
|
6
|
+
def initialize key: :sitemap, filename: 'sitemap.xml.gz'
|
|
7
|
+
@key, @filename = key, filename
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
def write(location, raw_data)
|
|
11
|
+
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
12
|
+
|
|
13
|
+
ActiveStorage::Blob.transaction do
|
|
14
|
+
ActiveStorage::Blob.where(key: key).destroy_all
|
|
15
|
+
|
|
16
|
+
ActiveStorage::Blob.create_and_upload!(
|
|
17
|
+
key: key,
|
|
18
|
+
io: open(location.path, 'rb'),
|
|
19
|
+
filename: filename,
|
|
20
|
+
content_type: 'application/gzip',
|
|
21
|
+
identify: false
|
|
22
|
+
)
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Aws::S3::Resource) or !defined?(Aws::Credentials)
|
|
2
|
-
raise LoadError, "Error: `Aws::S3::Resource` and/or `Aws::Credentials` are not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Aws::S3::Resource` and/or `Aws::Credentials` are not defined.\n\n" \
|
|
3
5
|
"Please `require 'aws-sdk'` - or another library that defines these classes."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -18,32 +20,36 @@ module SitemapGenerator
|
|
|
18
20
|
# Options:
|
|
19
21
|
# **Deprecated, use :endpoint instead** :aws_endpoint [String] The object storage endpoint,
|
|
20
22
|
# if not AWS, e.g. 'https://sfo2.digitaloceanspaces.com'
|
|
21
|
-
# **Deprecated, use :access_key_id instead** :
|
|
23
|
+
# **Deprecated, use :access_key_id instead** :aws_access_key_id [String] Your AWS access key id
|
|
22
24
|
# **Deprecated, use :secret_access_key instead** :aws_secret_access_key [String] Your AWS secret access key
|
|
23
25
|
# **Deprecated, use :region instead** :aws_region [String] Your AWS region
|
|
26
|
+
# :acl [String] The ACL to apply to the uploaded files. Defaults to 'public-read'.
|
|
27
|
+
# :cache_control [String] The cache control headder to apply to the uploaded files. Defaults to 'private, max-age=0, no-cache'.
|
|
24
28
|
#
|
|
25
29
|
# All other options you provide are passed directly to the AWS client.
|
|
26
30
|
# See https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method
|
|
27
31
|
# for a full list of supported options.
|
|
28
|
-
def initialize(bucket, aws_access_key_id: nil, aws_secret_access_key: nil, aws_region: nil, aws_endpoint: nil, **options)
|
|
32
|
+
def initialize(bucket, aws_access_key_id: nil, aws_secret_access_key: nil, aws_session_token: nil, aws_region: nil, aws_endpoint: nil, acl: 'public-read', cache_control: 'private, max-age=0, no-cache', **options)
|
|
29
33
|
@bucket = bucket
|
|
34
|
+
@acl = acl
|
|
35
|
+
@cache_control = cache_control
|
|
30
36
|
@options = options
|
|
31
37
|
set_option_unless_set(:access_key_id, aws_access_key_id)
|
|
32
38
|
set_option_unless_set(:secret_access_key, aws_secret_access_key)
|
|
39
|
+
set_option_unless_set(:session_token, aws_session_token)
|
|
33
40
|
set_option_unless_set(:region, aws_region)
|
|
34
41
|
set_option_unless_set(:endpoint, aws_endpoint)
|
|
35
42
|
end
|
|
36
43
|
|
|
37
|
-
|
|
38
44
|
# Call with a SitemapLocation and string data
|
|
39
45
|
def write(location, raw_data)
|
|
40
46
|
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
41
47
|
s3_object = s3_resource.bucket(@bucket).object(location.path_in_public)
|
|
42
|
-
s3_object.upload_file(location.path,
|
|
43
|
-
acl:
|
|
44
|
-
cache_control:
|
|
48
|
+
s3_object.upload_file(location.path, {
|
|
49
|
+
acl: @acl,
|
|
50
|
+
cache_control: @cache_control,
|
|
45
51
|
content_type: location[:compress] ? 'application/x-gzip' : 'application/xml'
|
|
46
|
-
)
|
|
52
|
+
}.compact)
|
|
47
53
|
end
|
|
48
54
|
|
|
49
55
|
private
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Fog::Storage)
|
|
2
|
-
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'fog'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -21,11 +23,11 @@ module SitemapGenerator
|
|
|
21
23
|
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
22
24
|
|
|
23
25
|
storage = Fog::Storage.new(@fog_credentials)
|
|
24
|
-
directory = storage.directories.new(:
|
|
26
|
+
directory = storage.directories.new(key: @fog_directory)
|
|
25
27
|
directory.files.create(
|
|
26
|
-
:
|
|
27
|
-
:
|
|
28
|
-
:
|
|
28
|
+
key: location.path_in_public,
|
|
29
|
+
body: File.open(location.path),
|
|
30
|
+
public: true
|
|
29
31
|
)
|
|
30
32
|
end
|
|
31
33
|
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Google::Cloud::Storage)
|
|
2
|
-
raise LoadError, "Error: `Google::Cloud::Storage` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Google::Cloud::Storage` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'google/cloud/storage'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -10,8 +12,9 @@ module SitemapGenerator
|
|
|
10
12
|
#
|
|
11
13
|
# @param [Hash] opts Google::Cloud::Storage configuration options.
|
|
12
14
|
# @option :bucket [String] Required. Name of Google Storage Bucket where the file is to be uploaded.
|
|
15
|
+
# @option :acl [String] Optional. Access control which is applied to the uploaded file(s). Default value is 'public'.
|
|
13
16
|
#
|
|
14
|
-
# All options other than the `:bucket`
|
|
17
|
+
# All options other than the `:bucket` and `:acl` options are passed to the `Google::Cloud::Storage.new`
|
|
15
18
|
# initializer. See https://googleapis.dev/ruby/google-cloud-storage/latest/file.AUTHENTICATION.html
|
|
16
19
|
# for all the supported environment variables and https://github.com/googleapis/google-cloud-ruby/blob/master/google-cloud-storage/lib/google/cloud/storage.rb
|
|
17
20
|
# for supported options.
|
|
@@ -22,6 +25,7 @@ module SitemapGenerator
|
|
|
22
25
|
def initialize(opts = {})
|
|
23
26
|
opts = opts.clone
|
|
24
27
|
@bucket = opts.delete(:bucket)
|
|
28
|
+
@acl = opts.has_key?(:acl) ? opts.delete(:acl) : 'public'
|
|
25
29
|
@storage_options = opts
|
|
26
30
|
end
|
|
27
31
|
|
|
@@ -31,7 +35,7 @@ module SitemapGenerator
|
|
|
31
35
|
|
|
32
36
|
storage = Google::Cloud::Storage.new(**@storage_options)
|
|
33
37
|
bucket = storage.bucket(@bucket)
|
|
34
|
-
bucket.create_file(location.path, location.path_in_public, acl:
|
|
38
|
+
bucket.create_file(location.path, location.path_in_public, acl: @acl)
|
|
35
39
|
end
|
|
36
40
|
end
|
|
37
41
|
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Fog::Storage)
|
|
2
|
-
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'fog-aws'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -23,6 +25,7 @@ module SitemapGenerator
|
|
|
23
25
|
def initialize(opts = {})
|
|
24
26
|
@aws_access_key_id = opts[:aws_access_key_id] || ENV['AWS_ACCESS_KEY_ID']
|
|
25
27
|
@aws_secret_access_key = opts[:aws_secret_access_key] || ENV['AWS_SECRET_ACCESS_KEY']
|
|
28
|
+
@aws_session_token = opts[:aws_session_token] || ENV['AWS_SESSION_TOKEN']
|
|
26
29
|
@fog_provider = opts[:fog_provider] || ENV['FOG_PROVIDER']
|
|
27
30
|
@fog_directory = opts[:fog_directory] || ENV['FOG_DIRECTORY']
|
|
28
31
|
@fog_region = opts[:fog_region] || ENV['FOG_REGION']
|
|
@@ -36,11 +39,12 @@ module SitemapGenerator
|
|
|
36
39
|
def write(location, raw_data)
|
|
37
40
|
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
38
41
|
|
|
39
|
-
credentials = { :
|
|
42
|
+
credentials = { provider: @fog_provider }
|
|
40
43
|
|
|
41
44
|
if @aws_access_key_id && @aws_secret_access_key
|
|
42
45
|
credentials[:aws_access_key_id] = @aws_access_key_id
|
|
43
46
|
credentials[:aws_secret_access_key] = @aws_secret_access_key
|
|
47
|
+
credentials[:aws_session_token] = @aws_session_token if @aws_session_token
|
|
44
48
|
else
|
|
45
49
|
credentials[:use_iam_profile] = true
|
|
46
50
|
end
|
|
@@ -49,11 +53,11 @@ module SitemapGenerator
|
|
|
49
53
|
credentials[:path_style] = @fog_path_style if @fog_path_style
|
|
50
54
|
|
|
51
55
|
storage = Fog::Storage.new(@fog_storage_options.merge(credentials))
|
|
52
|
-
directory = storage.directories.new(:
|
|
56
|
+
directory = storage.directories.new(key: @fog_directory)
|
|
53
57
|
directory.files.create(
|
|
54
|
-
:
|
|
55
|
-
:
|
|
56
|
-
:
|
|
58
|
+
key: location.path_in_public,
|
|
59
|
+
body: File.open(location.path),
|
|
60
|
+
public: @fog_public
|
|
57
61
|
)
|
|
58
62
|
end
|
|
59
63
|
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(::CarrierWave::Uploader::Base)
|
|
2
|
-
raise LoadError, "Error: `CarrierWave::Uploader::Base` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `CarrierWave::Uploader::Base` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'carrierwave'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require 'pathname'
|
|
2
4
|
|
|
3
5
|
module SitemapGenerator
|
|
@@ -29,6 +31,7 @@ module SitemapGenerator
|
|
|
29
31
|
def rails_root
|
|
30
32
|
return ::Rails.root.to_s if defined?(::Rails.root) && ::Rails.root
|
|
31
33
|
return RAILS_ROOT.to_s if defined?(RAILS_ROOT)
|
|
34
|
+
|
|
32
35
|
nil
|
|
33
36
|
end
|
|
34
37
|
|
|
@@ -40,6 +43,7 @@ module SitemapGenerator
|
|
|
40
43
|
def rails_env
|
|
41
44
|
return ::Rails.env.to_s if defined?(::Rails.env)
|
|
42
45
|
return RAILS_ENV.to_s if defined?(RAILS_ENV)
|
|
46
|
+
|
|
43
47
|
nil
|
|
44
48
|
end
|
|
45
49
|
end
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require 'zlib'
|
|
2
4
|
require 'fileutils'
|
|
3
5
|
require 'sitemap_generator/helpers/number_helper'
|
|
@@ -20,12 +22,12 @@ module SitemapGenerator
|
|
|
20
22
|
# * <tt>location</tt> - a SitemapGenerator::SitemapLocation instance or a Hash of options
|
|
21
23
|
# from which a SitemapLocation will be created for you. See `SitemapGenerator::SitemapLocation` for
|
|
22
24
|
# the supported list of options.
|
|
23
|
-
def initialize(opts={})
|
|
25
|
+
def initialize(opts = {})
|
|
24
26
|
@location = opts.is_a?(Hash) ? SitemapGenerator::SitemapLocation.new(opts) : opts
|
|
25
27
|
@link_count = 0
|
|
26
28
|
@news_count = 0
|
|
27
|
-
@xml_content = '' # XML urlset content
|
|
28
|
-
@xml_wrapper_start =
|
|
29
|
+
@xml_content = +'' # XML urlset content
|
|
30
|
+
@xml_wrapper_start = +<<-HTML
|
|
29
31
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
30
32
|
<urlset
|
|
31
33
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
|
@@ -41,7 +43,7 @@ module SitemapGenerator
|
|
|
41
43
|
>
|
|
42
44
|
HTML
|
|
43
45
|
@xml_wrapper_start.gsub!(/\s+/, ' ').gsub!(/ *> */, '>').strip!
|
|
44
|
-
@xml_wrapper_end
|
|
46
|
+
@xml_wrapper_end = '</urlset>'
|
|
45
47
|
@filesize = SitemapGenerator::Utilities.bytesize(@xml_wrapper_start) + SitemapGenerator::Utilities.bytesize(@xml_wrapper_end)
|
|
46
48
|
@written = false
|
|
47
49
|
@reserved_name = nil # holds the name reserved from the namer
|
|
@@ -89,15 +91,16 @@ module SitemapGenerator
|
|
|
89
91
|
#
|
|
90
92
|
# The link added to the sitemap will use the host from its location object
|
|
91
93
|
# if no host has been specified.
|
|
92
|
-
def add(link, options={})
|
|
94
|
+
def add(link, options = {})
|
|
93
95
|
raise SitemapGenerator::SitemapFinalizedError if finalized?
|
|
94
96
|
|
|
95
|
-
sitemap_url =
|
|
96
|
-
link
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
97
|
+
sitemap_url =
|
|
98
|
+
if link.is_a?(SitemapUrl)
|
|
99
|
+
link
|
|
100
|
+
else
|
|
101
|
+
options[:host] ||= @location.host
|
|
102
|
+
SitemapUrl.new(link, options)
|
|
103
|
+
end
|
|
101
104
|
|
|
102
105
|
xml = sitemap_url.to_xml
|
|
103
106
|
raise SitemapGenerator::SitemapFullError if !file_can_fit?(xml)
|
|
@@ -118,6 +121,7 @@ module SitemapGenerator
|
|
|
118
121
|
# has already been finalized.
|
|
119
122
|
def finalize!
|
|
120
123
|
raise SitemapGenerator::SitemapFinalizedError if finalized?
|
|
124
|
+
|
|
121
125
|
@frozen = true
|
|
122
126
|
end
|
|
123
127
|
|
|
@@ -133,7 +137,8 @@ module SitemapGenerator
|
|
|
133
137
|
# A SitemapGenerator::SitemapError exception is raised if the file has
|
|
134
138
|
# already been written.
|
|
135
139
|
def write
|
|
136
|
-
raise SitemapGenerator::SitemapError.new(
|
|
140
|
+
raise SitemapGenerator::SitemapError.new('Sitemap already written!') if written?
|
|
141
|
+
|
|
137
142
|
finalize! unless finalized?
|
|
138
143
|
reserve_name
|
|
139
144
|
@location.write(@xml_wrapper_start + @xml_content + @xml_wrapper_end, link_count)
|