sitemap_generator 6.3.0 → 7.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGES.md +16 -0
- data/README.md +37 -31
- data/VERSION +1 -1
- data/lib/sitemap_generator/adapters/active_storage_adapter.rb +26 -0
- data/lib/sitemap_generator/adapters/aws_sdk_adapter.rb +5 -3
- data/lib/sitemap_generator/adapters/file_adapter.rb +2 -0
- data/lib/sitemap_generator/adapters/fog_adapter.rb +7 -5
- data/lib/sitemap_generator/adapters/google_storage_adapter.rb +3 -1
- data/lib/sitemap_generator/adapters/s3_adapter.rb +10 -6
- data/lib/sitemap_generator/adapters/wave_adapter.rb +3 -1
- data/lib/sitemap_generator/application.rb +4 -0
- data/lib/sitemap_generator/builder/sitemap_file.rb +17 -12
- data/lib/sitemap_generator/builder/sitemap_index_file.rb +12 -9
- data/lib/sitemap_generator/builder/sitemap_index_url.rb +6 -4
- data/lib/sitemap_generator/builder/sitemap_url.rb +59 -55
- data/lib/sitemap_generator/builder.rb +3 -1
- data/lib/sitemap_generator/core_ext/big_decimal.rb +2 -0
- data/lib/sitemap_generator/core_ext/numeric.rb +2 -0
- data/lib/sitemap_generator/core_ext.rb +2 -0
- data/lib/sitemap_generator/helpers/number_helper.rb +35 -33
- data/lib/sitemap_generator/interpreter.rb +6 -4
- data/lib/sitemap_generator/link_set.rb +65 -62
- data/lib/sitemap_generator/railtie.rb +3 -1
- data/lib/sitemap_generator/simple_namer.rb +8 -5
- data/lib/sitemap_generator/sitemap_location.rb +13 -9
- data/lib/sitemap_generator/tasks.rb +10 -8
- data/lib/sitemap_generator/templates.rb +8 -5
- data/lib/sitemap_generator/utilities.rb +12 -9
- data/lib/sitemap_generator.rb +13 -9
- data/templates/sitemap.rb +1 -1
- metadata +6 -130
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b26cd553fe4ecc452da26ac6aff3262238f065dc12d9f328959eb3c1edc79f6e
|
|
4
|
+
data.tar.gz: 877828e867880ed215df8b3e1f6acb130e4e8c5856a349e61e6af17a5c641ac9
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d957d69f7a4c2fe8e98cecb84045d58fe59f9f8143764c88695a7561a63d54e9d515c80bdc2b2cf496b1704ffb2ca4610b54f5889cb246e2ed45aac88029abdd
|
|
7
|
+
data.tar.gz: 884fc7be3809f6670cce09afdddfa64cdf6b4c5b25cdc7ad5c7c3b784d08f83bbc96bdb671a1b52de7f0f59fac8618f1bc83122c01664ee574a52c4edd791699
|
data/CHANGES.md
CHANGED
|
@@ -1,3 +1,19 @@
|
|
|
1
|
+
### 7.0.1
|
|
2
|
+
|
|
3
|
+
* **Breaking:** Default search engines list is empty. `rake sitemap:refresh` and `ping_search_engines` perform no HTTP pings unless you configure engine URLs on `search_engines` or pass them into `ping_search_engines` (Google’s ping endpoint is deprecated upstream). [#444](https://github.com/kjvarga/sitemap_generator/pull/444)
|
|
4
|
+
* **Breaking:** `LinkSet#create` runs `finalize!` only when a block is given. Calling `create` without a block requires `finalize!` when you are done adding links (supported workflow for programmatic builds). [#463](https://github.com/kjvarga/sitemap_generator/pull/463)
|
|
5
|
+
* **Breaking:** Drop Ruby 2.5, Rails 5.2 [#438](https://github.com/kjvarga/sitemap_generator/pull/438)
|
|
6
|
+
* Migrate continuous integration from CircleCI to GitHub Actions with expanded Ruby × Rails CI matrix.
|
|
7
|
+
* Support AWS temporary credentials (`aws_session_token` / `AWS_SESSION_TOKEN`) for S3 uploads in AWS Lambda [#415](https://github.com/kjvarga/sitemap_generator/pull/415)
|
|
8
|
+
* Add `frozen_string_literal: true` magic comments [#430](https://github.com/kjvarga/sitemap_generator/pull/430)
|
|
9
|
+
* Add support for Rails 8.0 [#441](https://github.com/kjvarga/sitemap_generator/pull/441)
|
|
10
|
+
* Fix uninitialized constant ActiveSupport::LoggerThreadSafeLevel::Logger [#449](https://github.com/kjvarga/sitemap_generator/pull/449)
|
|
11
|
+
* Add support for Ruby 3.4 [#451](https://github.com/kjvarga/sitemap_generator/pull/451)
|
|
12
|
+
* Add support for Rails 8.1 [#461](https://github.com/kjvarga/sitemap_generator/pull/461)
|
|
13
|
+
* Replace CGI-based escaping with `URI.encode_www_form_component` for ping URLs; remove reliance on `cgi` (Ruby 4 compatibility) [#465](https://github.com/kjvarga/sitemap_generator/pull/465)
|
|
14
|
+
* Add support for Ruby 4.0 [#466](https://github.com/kjvarga/sitemap_generator/pull/466)
|
|
15
|
+
* Add ActiveStorage adapter (`ActiveStorage::Blob`) [#467](https://github.com/kjvarga/sitemap_generator/pull/467)
|
|
16
|
+
|
|
1
17
|
### 6.3.0
|
|
2
18
|
|
|
3
19
|
* Remove Bing's deprecated sitemap submission [#400](https://github.com/kjvarga/sitemap_generator/pull/400).
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# SitemapGenerator
|
|
2
2
|
|
|
3
|
-
[](https://github.com/kjvarga/sitemap_generator/actions/workflows/ci.yml)
|
|
4
4
|
|
|
5
5
|
SitemapGenerator is the easiest way to generate Sitemaps in Ruby. Rails integration provides access to the Rails route helpers within your sitemap config file and automatically makes the rake tasks available to you. Or if you prefer to use another framework, you can! You can use the rake tasks provided or run your sitemap configs as plain ruby scripts.
|
|
6
6
|
|
|
@@ -11,14 +11,15 @@ Sitemaps adhere to the [Sitemap 0.9 protocol][sitemap_protocol] specification.
|
|
|
11
11
|
* Framework agnostic
|
|
12
12
|
* Supports [News sitemaps][sitemap_news], [Video sitemaps][sitemap_video], [Image sitemaps][sitemap_images], [Mobile sitemaps][sitemap_mobile], [PageMap sitemaps][sitemap_pagemap] and [Alternate Links][alternate_links]
|
|
13
13
|
* Supports read-only filesystems like Heroku via uploading to a remote host like Amazon S3
|
|
14
|
-
* Compatible with all versions of Rails and Ruby
|
|
15
14
|
* Adheres to the [Sitemap 0.9 protocol][sitemap_protocol]
|
|
16
15
|
* Handles millions of links
|
|
17
16
|
* Customizable sitemap compression
|
|
18
|
-
*
|
|
17
|
+
* Optional HTTP notification of search engines when `search_engines` is configured
|
|
19
18
|
* Ensures your old sitemaps stay in place if the new sitemap fails to generate
|
|
20
19
|
* Gives you complete control over your sitemap contents and naming scheme
|
|
21
20
|
* Intelligent sitemap indexing
|
|
21
|
+
* Compatible with Ruby from version 2.6 to version 4.0
|
|
22
|
+
* Compatible with Rails from version 6.0 to version 8.1
|
|
22
23
|
|
|
23
24
|
### Show Me
|
|
24
25
|
|
|
@@ -41,7 +42,6 @@ SitemapGenerator::Sitemap.create do
|
|
|
41
42
|
add '/home', :changefreq => 'daily', :priority => 0.9
|
|
42
43
|
add '/contact_us', :changefreq => 'weekly'
|
|
43
44
|
end
|
|
44
|
-
SitemapGenerator::Sitemap.ping_search_engines # Not needed if you use the rake tasks
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
Run it:
|
|
@@ -56,8 +56,6 @@ Output:
|
|
|
56
56
|
In /Users/karl/projects/sitemap_generator-test/public/
|
|
57
57
|
+ sitemap.xml.gz 3 links / 364 Bytes
|
|
58
58
|
Sitemap stats: 3 links / 1 sitemaps / 0m00s
|
|
59
|
-
|
|
60
|
-
Successful ping of Google
|
|
61
59
|
```
|
|
62
60
|
|
|
63
61
|
## Contents
|
|
@@ -66,7 +64,6 @@ Successful ping of Google
|
|
|
66
64
|
- [Features](#features)
|
|
67
65
|
- [Show Me](#show-me)
|
|
68
66
|
- [Contents](#contents)
|
|
69
|
-
- [Contribute](#contribute)
|
|
70
67
|
- [Foreword](#foreword)
|
|
71
68
|
- [Installation](#installation)
|
|
72
69
|
- [Ruby](#ruby)
|
|
@@ -78,11 +75,12 @@ Successful ping of Google
|
|
|
78
75
|
- [Crontab](#crontab)
|
|
79
76
|
- [Robots.txt](#robotstxt)
|
|
80
77
|
- [Ruby Modules](#ruby-modules)
|
|
81
|
-
- [Deployments
|
|
78
|
+
- [Deployments \& Capistrano](#deployments--capistrano)
|
|
82
79
|
- [Sitemaps with no Index File](#sitemaps-with-no-index-file)
|
|
83
80
|
- [Upload Sitemaps to a Remote Host using Adapters](#upload-sitemaps-to-a-remote-host-using-adapters)
|
|
84
81
|
- [Supported Adapters](#supported-adapters)
|
|
85
82
|
- [`SitemapGenerator::FileAdapter`](#sitemapgeneratorfileadapter)
|
|
83
|
+
- [`SitemapGenerator::ActiveStorageAdapter`](#sitemapgeneratoractivestorageadapter)
|
|
86
84
|
- [`SitemapGenerator::FogAdapter`](#sitemapgeneratorfogadapter)
|
|
87
85
|
- [`SitemapGenerator::S3Adapter`](#sitemapgenerators3adapter)
|
|
88
86
|
- [`SitemapGenerator::AwsSdkAdapter`](#sitemapgeneratorawssdkadapter)
|
|
@@ -96,6 +94,7 @@ Successful ping of Google
|
|
|
96
94
|
- [Supported Options to `add`](#supported-options-to-add)
|
|
97
95
|
- [Adding Links to the Sitemap Index](#adding-links-to-the-sitemap-index)
|
|
98
96
|
- [Accessing the LinkSet instance](#accessing-the-linkset-instance)
|
|
97
|
+
- [Using `create` without a block](#using-create-without-a-block)
|
|
99
98
|
- [Speeding Things Up](#speeding-things-up)
|
|
100
99
|
- [Customizing your Sitemaps](#customizing-your-sitemaps)
|
|
101
100
|
- [Sitemap Options](#sitemap-options)
|
|
@@ -125,13 +124,6 @@ Successful ping of Google
|
|
|
125
124
|
- [Compatibility](#compatibility)
|
|
126
125
|
- [Licence](#licence)
|
|
127
126
|
|
|
128
|
-
## Contribute
|
|
129
|
-
|
|
130
|
-
Does your website use SitemapGenerator to generate Sitemaps? Where would you be without Sitemaps? Probably still knocking rocks together. Consider donating to the project to keep it up-to-date and open source.
|
|
131
|
-
|
|
132
|
-
<a href='http://www.pledgie.com/campaigns/15267'><img alt='Click here to lend your support to: SitemapGenerator and make a donation at www.pledgie.com !' src='http://pledgie.com/campaigns/15267.png?skin_name=chrome' border='0' /></a>
|
|
133
|
-
|
|
134
|
-
|
|
135
127
|
## Foreword
|
|
136
128
|
|
|
137
129
|
Adam Salter first created SitemapGenerator while we were working together in Sydney, Australia. Unfortunately, he passed away in 2009. Since then I have taken over development of SitemapGenerator.
|
|
@@ -159,7 +151,7 @@ The Rake tasks expect your sitemap to be at `config/sitemap.rb` but if you need
|
|
|
159
151
|
|
|
160
152
|
### Rails
|
|
161
153
|
|
|
162
|
-
SitemapGenerator
|
|
154
|
+
SitemapGenerator targets Rails 6.0 through 8.1 (see [Compatibility](#compatibility)).
|
|
163
155
|
|
|
164
156
|
Add the gem to your `Gemfile`:
|
|
165
157
|
|
|
@@ -204,10 +196,10 @@ SitemapGenerator.verbose = false
|
|
|
204
196
|
|
|
205
197
|
### Pinging Search Engines
|
|
206
198
|
|
|
207
|
-
Using `rake sitemap:refresh` will notify
|
|
199
|
+
Using `rake sitemap:refresh` will notify configured search engines to let them know that a new sitemap
|
|
208
200
|
is available. To generate new sitemaps without notifying search engines, use `rake sitemap:refresh:no_ping`.
|
|
209
201
|
|
|
210
|
-
If you want to customize the hash of search engines you can access it at:
|
|
202
|
+
By default no search engines are configured. If you want to customize the hash of search engines you can access it at:
|
|
211
203
|
|
|
212
204
|
```ruby
|
|
213
205
|
SitemapGenerator::Sitemap.search_engines
|
|
@@ -224,7 +216,7 @@ directly in the call, as in the following example:
|
|
|
224
216
|
SitemapGenerator::Sitemap.ping_search_engines(newengine: 'http://newengine.com/ping?url=%s')
|
|
225
217
|
```
|
|
226
218
|
|
|
227
|
-
The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the
|
|
219
|
+
The key gives the name of the search engine, as a string or symbol, and the value is the full URL to ping, with a string interpolation that will be replaced by the URL-encoded (percent-encoded) sitemap index URL. If you have any literal percent characters in your URL you need to escape them with `%%`.
|
|
228
220
|
|
|
229
221
|
If you are calling `SitemapGenerator::Sitemap.ping_search_engines` from outside of your sitemap config file, then you will need to set `SitemapGenerator::Sitemap.default_host` and any other options that you set in your sitemap config which affect the location of the sitemap index file. For example:
|
|
230
222
|
|
|
@@ -342,6 +334,10 @@ directory.
|
|
|
342
334
|
|
|
343
335
|
Standard adapter, writes out to a file.
|
|
344
336
|
|
|
337
|
+
##### `SitemapGenerator::ActiveStorageAdapter`
|
|
338
|
+
|
|
339
|
+
Uses `ActiveStorage::Blob` to store the sitemap.
|
|
340
|
+
|
|
345
341
|
##### `SitemapGenerator::FogAdapter`
|
|
346
342
|
|
|
347
343
|
Uses `Fog::Storage` to upload to any service supported by Fog.
|
|
@@ -364,6 +360,7 @@ directory.
|
|
|
364
360
|
Where `options` is a Hash with any of the following keys:
|
|
365
361
|
* `aws_access_key_id` [String] Your AWS access key id
|
|
366
362
|
* `aws_secret_access_key` [String] Your AWS secret access key
|
|
363
|
+
* `aws_session_token` [String] Session token for temporary credentials (optional)
|
|
367
364
|
* `fog_provider` [String]
|
|
368
365
|
* `fog_directory` [String]
|
|
369
366
|
* `fog_region` [String]
|
|
@@ -372,7 +369,7 @@ directory.
|
|
|
372
369
|
* `fog_public` [Boolean] Whether the file is publicly accessible
|
|
373
370
|
|
|
374
371
|
Alternatively you can use an environment variable to configure each option (except `fog_storage_options`). The environment variables have the same
|
|
375
|
-
name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
372
|
+
name but capitalized, e.g. `AWS_SESSION_TOKEN`, `FOG_PATH_STYLE`.
|
|
376
373
|
|
|
377
374
|
##### `SitemapGenerator::AwsSdkAdapter`
|
|
378
375
|
|
|
@@ -388,14 +385,15 @@ name but capitalized, e.g. `FOG_PATH_STYLE`.
|
|
|
388
385
|
SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new('s3_bucket',
|
|
389
386
|
acl: 'public-read', # Optional. This is the default.
|
|
390
387
|
cache_control: 'private, max-age=0, no-cache', # Optional. This is the default.
|
|
391
|
-
access_key_id: '
|
|
392
|
-
secret_access_key: '
|
|
388
|
+
access_key_id: 'YOUR_AWS_ACCESS_KEY_ID',
|
|
389
|
+
secret_access_key: 'YOUR_AWS_SECRET_ACCESS_KEY',
|
|
390
|
+
session_token: 'YOUR_AWS_SESSION_TOKEN', # Optional; use with temporary credentials.
|
|
393
391
|
region: 'us-east-1',
|
|
394
392
|
endpoint: 'https://sfo2.digitaloceanspaces.com'
|
|
395
393
|
)
|
|
396
394
|
```
|
|
397
395
|
|
|
398
|
-
Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files;
|
|
396
|
+
Where the first argument is the S3 bucket name, and the rest are keyword argument options. Options `:acl` and `:cache_control` configure access and caching of the uploaded files; `session_token` supports STS-style credentials (or set `AWS_SESSION_TOKEN` and rely on SDK defaults). All other keyword options are passed directly to the AWS client.
|
|
399
397
|
|
|
400
398
|
See [the `SitemapGenerator::AwsSdkAdapter` docs](https://github.com/kjvarga/sitemap_generator/blob/master/lib/sitemap_generator/adapters/aws_sdk_adapter.rb), and [https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method](https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method) for the full list of supported options.
|
|
401
399
|
|
|
@@ -536,7 +534,6 @@ SitemapGenerator::Sitemap.create do
|
|
|
536
534
|
end
|
|
537
535
|
```
|
|
538
536
|
|
|
539
|
-
|
|
540
537
|
To generate each one specify the configuration file to run by passing the `CONFIG_FILE` option to `rake sitemap:refresh`, e.g.:
|
|
541
538
|
|
|
542
539
|
```
|
|
@@ -780,11 +777,21 @@ In /Users/karl/projects/sitemap_generator-test/public/
|
|
|
780
777
|
Sitemap stats: 3 links / 4 sitemaps / 0m00s
|
|
781
778
|
```
|
|
782
779
|
|
|
780
|
+
### Using `create` without a block
|
|
781
|
+
|
|
782
|
+
You can call `create` without a block, add links (and use `group` as needed), then call `finalize!` on the returned link set when finished.
|
|
783
|
+
|
|
784
|
+
```ruby
|
|
785
|
+
sitemap = SitemapGenerator::Sitemap.create(default_host: 'http://www.example.com')
|
|
786
|
+
sitemap.add('/home')
|
|
787
|
+
# ... add more links or use groups, then:
|
|
788
|
+
sitemap.finalize!
|
|
789
|
+
```
|
|
790
|
+
|
|
783
791
|
### Speeding Things Up
|
|
784
792
|
|
|
785
793
|
For large ActiveRecord collections with thousands of records it is advisable to iterate through them in batches to avoid loading all records into memory at once. For this reason in the example above we use `Content.find_each` which is a batched iterator available since Rails 2.3.2, rather than `Content.all`.
|
|
786
794
|
|
|
787
|
-
|
|
788
795
|
## Customizing your Sitemaps
|
|
789
796
|
|
|
790
797
|
SitemapGenerator supports a number of options which allow you to control every aspect of your sitemap generation. How they are named, where they are stored, the contents of the links and the location that the sitemaps will be hosted from can all be set.
|
|
@@ -1167,16 +1174,15 @@ end
|
|
|
1167
1174
|
|
|
1168
1175
|
## Compatibility
|
|
1169
1176
|
|
|
1170
|
-
|
|
1171
|
-
Ruby
|
|
1177
|
+
Ruby 2.6 through 4.0 and Rails 6.0 through 8.1 are supported.
|
|
1178
|
+
Ruby 2.5 and Rails 5.2 were dropped in v7.0.0.
|
|
1179
|
+
Ruby 1.9.3 support was dropped in v6.0.0.
|
|
1172
1180
|
|
|
1173
1181
|
## Licence
|
|
1174
1182
|
|
|
1175
|
-
Released under the MIT
|
|
1176
|
-
|
|
1177
|
-
MIT. See the LICENSE.md file.
|
|
1183
|
+
Released under the MIT license which is included in the [MIT-LICENSE](./MIT-LICENSE) file.
|
|
1178
1184
|
|
|
1179
|
-
Copyright (c) Karl Varga
|
|
1185
|
+
Copyright (c) Karl Varga
|
|
1180
1186
|
|
|
1181
1187
|
[canonical_repo]:http://github.com/kjvarga/sitemap_generator
|
|
1182
1188
|
[sitemap_images]:http://www.google.com/support/webmasters/bin/answer.py?answer=178636
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
7.0.1
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
module SitemapGenerator
|
|
2
|
+
# Class for uploading sitemaps to ActiveStorage.
|
|
3
|
+
class ActiveStorageAdapter
|
|
4
|
+
attr_reader :key, :filename
|
|
5
|
+
|
|
6
|
+
def initialize key: :sitemap, filename: 'sitemap.xml.gz'
|
|
7
|
+
@key, @filename = key, filename
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
def write(location, raw_data)
|
|
11
|
+
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
12
|
+
|
|
13
|
+
ActiveStorage::Blob.transaction do
|
|
14
|
+
ActiveStorage::Blob.where(key: key).destroy_all
|
|
15
|
+
|
|
16
|
+
ActiveStorage::Blob.create_and_upload!(
|
|
17
|
+
key: key,
|
|
18
|
+
io: open(location.path, 'rb'),
|
|
19
|
+
filename: filename,
|
|
20
|
+
content_type: 'application/gzip',
|
|
21
|
+
identify: false
|
|
22
|
+
)
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Aws::S3::Resource) or !defined?(Aws::Credentials)
|
|
2
|
-
raise LoadError, "Error: `Aws::S3::Resource` and/or `Aws::Credentials` are not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Aws::S3::Resource` and/or `Aws::Credentials` are not defined.\n\n" \
|
|
3
5
|
"Please `require 'aws-sdk'` - or another library that defines these classes."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -27,18 +29,18 @@ module SitemapGenerator
|
|
|
27
29
|
# All other options you provide are passed directly to the AWS client.
|
|
28
30
|
# See https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/S3/Client.html#initialize-instance_method
|
|
29
31
|
# for a full list of supported options.
|
|
30
|
-
def initialize(bucket, aws_access_key_id: nil, aws_secret_access_key: nil, aws_region: nil, aws_endpoint: nil, acl: 'public-read', cache_control: 'private, max-age=0, no-cache', **options)
|
|
32
|
+
def initialize(bucket, aws_access_key_id: nil, aws_secret_access_key: nil, aws_session_token: nil, aws_region: nil, aws_endpoint: nil, acl: 'public-read', cache_control: 'private, max-age=0, no-cache', **options)
|
|
31
33
|
@bucket = bucket
|
|
32
34
|
@acl = acl
|
|
33
35
|
@cache_control = cache_control
|
|
34
36
|
@options = options
|
|
35
37
|
set_option_unless_set(:access_key_id, aws_access_key_id)
|
|
36
38
|
set_option_unless_set(:secret_access_key, aws_secret_access_key)
|
|
39
|
+
set_option_unless_set(:session_token, aws_session_token)
|
|
37
40
|
set_option_unless_set(:region, aws_region)
|
|
38
41
|
set_option_unless_set(:endpoint, aws_endpoint)
|
|
39
42
|
end
|
|
40
43
|
|
|
41
|
-
|
|
42
44
|
# Call with a SitemapLocation and string data
|
|
43
45
|
def write(location, raw_data)
|
|
44
46
|
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Fog::Storage)
|
|
2
|
-
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'fog'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -21,11 +23,11 @@ module SitemapGenerator
|
|
|
21
23
|
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
22
24
|
|
|
23
25
|
storage = Fog::Storage.new(@fog_credentials)
|
|
24
|
-
directory = storage.directories.new(:
|
|
26
|
+
directory = storage.directories.new(key: @fog_directory)
|
|
25
27
|
directory.files.create(
|
|
26
|
-
:
|
|
27
|
-
:
|
|
28
|
-
:
|
|
28
|
+
key: location.path_in_public,
|
|
29
|
+
body: File.open(location.path),
|
|
30
|
+
public: true
|
|
29
31
|
)
|
|
30
32
|
end
|
|
31
33
|
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Google::Cloud::Storage)
|
|
2
|
-
raise LoadError, "Error: `Google::Cloud::Storage` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Google::Cloud::Storage` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'google/cloud/storage'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(Fog::Storage)
|
|
2
|
-
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `Fog::Storage` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'fog-aws'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -23,6 +25,7 @@ module SitemapGenerator
|
|
|
23
25
|
def initialize(opts = {})
|
|
24
26
|
@aws_access_key_id = opts[:aws_access_key_id] || ENV['AWS_ACCESS_KEY_ID']
|
|
25
27
|
@aws_secret_access_key = opts[:aws_secret_access_key] || ENV['AWS_SECRET_ACCESS_KEY']
|
|
28
|
+
@aws_session_token = opts[:aws_session_token] || ENV['AWS_SESSION_TOKEN']
|
|
26
29
|
@fog_provider = opts[:fog_provider] || ENV['FOG_PROVIDER']
|
|
27
30
|
@fog_directory = opts[:fog_directory] || ENV['FOG_DIRECTORY']
|
|
28
31
|
@fog_region = opts[:fog_region] || ENV['FOG_REGION']
|
|
@@ -36,11 +39,12 @@ module SitemapGenerator
|
|
|
36
39
|
def write(location, raw_data)
|
|
37
40
|
SitemapGenerator::FileAdapter.new.write(location, raw_data)
|
|
38
41
|
|
|
39
|
-
credentials = { :
|
|
42
|
+
credentials = { provider: @fog_provider }
|
|
40
43
|
|
|
41
44
|
if @aws_access_key_id && @aws_secret_access_key
|
|
42
45
|
credentials[:aws_access_key_id] = @aws_access_key_id
|
|
43
46
|
credentials[:aws_secret_access_key] = @aws_secret_access_key
|
|
47
|
+
credentials[:aws_session_token] = @aws_session_token if @aws_session_token
|
|
44
48
|
else
|
|
45
49
|
credentials[:use_iam_profile] = true
|
|
46
50
|
end
|
|
@@ -49,11 +53,11 @@ module SitemapGenerator
|
|
|
49
53
|
credentials[:path_style] = @fog_path_style if @fog_path_style
|
|
50
54
|
|
|
51
55
|
storage = Fog::Storage.new(@fog_storage_options.merge(credentials))
|
|
52
|
-
directory = storage.directories.new(:
|
|
56
|
+
directory = storage.directories.new(key: @fog_directory)
|
|
53
57
|
directory.files.create(
|
|
54
|
-
:
|
|
55
|
-
:
|
|
56
|
-
:
|
|
58
|
+
key: location.path_in_public,
|
|
59
|
+
body: File.open(location.path),
|
|
60
|
+
public: @fog_public
|
|
57
61
|
)
|
|
58
62
|
end
|
|
59
63
|
end
|
|
@@ -1,5 +1,7 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
if !defined?(::CarrierWave::Uploader::Base)
|
|
2
|
-
raise LoadError, "Error: `CarrierWave::Uploader::Base` is not defined.\n\n"\
|
|
4
|
+
raise LoadError, "Error: `CarrierWave::Uploader::Base` is not defined.\n\n" \
|
|
3
5
|
"Please `require 'carrierwave'` - or another library that defines this class."
|
|
4
6
|
end
|
|
5
7
|
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require 'pathname'
|
|
2
4
|
|
|
3
5
|
module SitemapGenerator
|
|
@@ -29,6 +31,7 @@ module SitemapGenerator
|
|
|
29
31
|
def rails_root
|
|
30
32
|
return ::Rails.root.to_s if defined?(::Rails.root) && ::Rails.root
|
|
31
33
|
return RAILS_ROOT.to_s if defined?(RAILS_ROOT)
|
|
34
|
+
|
|
32
35
|
nil
|
|
33
36
|
end
|
|
34
37
|
|
|
@@ -40,6 +43,7 @@ module SitemapGenerator
|
|
|
40
43
|
def rails_env
|
|
41
44
|
return ::Rails.env.to_s if defined?(::Rails.env)
|
|
42
45
|
return RAILS_ENV.to_s if defined?(RAILS_ENV)
|
|
46
|
+
|
|
43
47
|
nil
|
|
44
48
|
end
|
|
45
49
|
end
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require 'zlib'
|
|
2
4
|
require 'fileutils'
|
|
3
5
|
require 'sitemap_generator/helpers/number_helper'
|
|
@@ -20,12 +22,12 @@ module SitemapGenerator
|
|
|
20
22
|
# * <tt>location</tt> - a SitemapGenerator::SitemapLocation instance or a Hash of options
|
|
21
23
|
# from which a SitemapLocation will be created for you. See `SitemapGenerator::SitemapLocation` for
|
|
22
24
|
# the supported list of options.
|
|
23
|
-
def initialize(opts={})
|
|
25
|
+
def initialize(opts = {})
|
|
24
26
|
@location = opts.is_a?(Hash) ? SitemapGenerator::SitemapLocation.new(opts) : opts
|
|
25
27
|
@link_count = 0
|
|
26
28
|
@news_count = 0
|
|
27
|
-
@xml_content = '' # XML urlset content
|
|
28
|
-
@xml_wrapper_start =
|
|
29
|
+
@xml_content = +'' # XML urlset content
|
|
30
|
+
@xml_wrapper_start = +<<-HTML
|
|
29
31
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
30
32
|
<urlset
|
|
31
33
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
|
@@ -41,7 +43,7 @@ module SitemapGenerator
|
|
|
41
43
|
>
|
|
42
44
|
HTML
|
|
43
45
|
@xml_wrapper_start.gsub!(/\s+/, ' ').gsub!(/ *> */, '>').strip!
|
|
44
|
-
@xml_wrapper_end
|
|
46
|
+
@xml_wrapper_end = '</urlset>'
|
|
45
47
|
@filesize = SitemapGenerator::Utilities.bytesize(@xml_wrapper_start) + SitemapGenerator::Utilities.bytesize(@xml_wrapper_end)
|
|
46
48
|
@written = false
|
|
47
49
|
@reserved_name = nil # holds the name reserved from the namer
|
|
@@ -89,15 +91,16 @@ module SitemapGenerator
|
|
|
89
91
|
#
|
|
90
92
|
# The link added to the sitemap will use the host from its location object
|
|
91
93
|
# if no host has been specified.
|
|
92
|
-
def add(link, options={})
|
|
94
|
+
def add(link, options = {})
|
|
93
95
|
raise SitemapGenerator::SitemapFinalizedError if finalized?
|
|
94
96
|
|
|
95
|
-
sitemap_url =
|
|
96
|
-
link
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
97
|
+
sitemap_url =
|
|
98
|
+
if link.is_a?(SitemapUrl)
|
|
99
|
+
link
|
|
100
|
+
else
|
|
101
|
+
options[:host] ||= @location.host
|
|
102
|
+
SitemapUrl.new(link, options)
|
|
103
|
+
end
|
|
101
104
|
|
|
102
105
|
xml = sitemap_url.to_xml
|
|
103
106
|
raise SitemapGenerator::SitemapFullError if !file_can_fit?(xml)
|
|
@@ -118,6 +121,7 @@ module SitemapGenerator
|
|
|
118
121
|
# has already been finalized.
|
|
119
122
|
def finalize!
|
|
120
123
|
raise SitemapGenerator::SitemapFinalizedError if finalized?
|
|
124
|
+
|
|
121
125
|
@frozen = true
|
|
122
126
|
end
|
|
123
127
|
|
|
@@ -133,7 +137,8 @@ module SitemapGenerator
|
|
|
133
137
|
# A SitemapGenerator::SitemapError exception is raised if the file has
|
|
134
138
|
# already been written.
|
|
135
139
|
def write
|
|
136
|
-
raise SitemapGenerator::SitemapError.new(
|
|
140
|
+
raise SitemapGenerator::SitemapError.new('Sitemap already written!') if written?
|
|
141
|
+
|
|
137
142
|
finalize! unless finalized?
|
|
138
143
|
reserve_name
|
|
139
144
|
@location.write(@xml_wrapper_start + @xml_content + @xml_wrapper_end, link_count)
|
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
module SitemapGenerator
|
|
2
4
|
module Builder
|
|
3
5
|
class SitemapIndexFile < SitemapFile
|
|
@@ -6,12 +8,12 @@ module SitemapGenerator
|
|
|
6
8
|
#
|
|
7
9
|
# * <tt>location</tt> - a SitemapGenerator::SitemapIndexLocation instance or a Hash of options
|
|
8
10
|
# from which a SitemapLocation will be created for you.
|
|
9
|
-
def initialize(opts={})
|
|
11
|
+
def initialize(opts = {})
|
|
10
12
|
@location = opts.is_a?(Hash) ? SitemapGenerator::SitemapIndexLocation.new(opts) : opts
|
|
11
13
|
@link_count = 0
|
|
12
14
|
@sitemaps_link_count = 0
|
|
13
|
-
@xml_content = '' # XML urlset content
|
|
14
|
-
@xml_wrapper_start =
|
|
15
|
+
@xml_content = +'' # XML urlset content
|
|
16
|
+
@xml_wrapper_start = +<<-HTML
|
|
15
17
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
16
18
|
<sitemapindex
|
|
17
19
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
|
@@ -21,7 +23,7 @@ module SitemapGenerator
|
|
|
21
23
|
>
|
|
22
24
|
HTML
|
|
23
25
|
@xml_wrapper_start.gsub!(/\s+/, ' ').gsub!(/ *> */, '>').strip!
|
|
24
|
-
@xml_wrapper_end
|
|
26
|
+
@xml_wrapper_end = '</sitemapindex>'
|
|
25
27
|
@filesize = SitemapGenerator::Utilities.bytesize(@xml_wrapper_start) + SitemapGenerator::Utilities.bytesize(@xml_wrapper_end)
|
|
26
28
|
@written = false
|
|
27
29
|
@reserved_name = nil # holds the name reserved from the namer
|
|
@@ -45,7 +47,7 @@ module SitemapGenerator
|
|
|
45
47
|
# can assume that the index is required (unless create_index is false of course).
|
|
46
48
|
# This seems like the logical thing to do.
|
|
47
49
|
alias_method :super_add, :add
|
|
48
|
-
def add(link, options={})
|
|
50
|
+
def add(link, options = {})
|
|
49
51
|
if file = link.is_a?(SitemapFile) && link
|
|
50
52
|
@sitemaps_link_count += file.link_count
|
|
51
53
|
file.finalize! unless file.finalized?
|
|
@@ -59,7 +61,7 @@ module SitemapGenerator
|
|
|
59
61
|
# for there to be an index.
|
|
60
62
|
if @link_count == 0
|
|
61
63
|
@first_sitemap = SitemapGenerator::Builder::LinkHolder.new(file, options)
|
|
62
|
-
@link_count += 1
|
|
64
|
+
@link_count += 1 # pretend it's added, but don't add it yet
|
|
63
65
|
else
|
|
64
66
|
# need an index so make sure name is reserved and first sitemap is written out
|
|
65
67
|
reserve_name unless @location.create_index == false
|
|
@@ -94,13 +96,14 @@ module SitemapGenerator
|
|
|
94
96
|
@sitemaps_link_count
|
|
95
97
|
end
|
|
96
98
|
|
|
97
|
-
def stats_summary(opts={})
|
|
99
|
+
def stats_summary(opts = {})
|
|
98
100
|
str = "Sitemap stats: #{number_with_delimiter(@sitemaps_link_count)} links / #{@link_count} sitemaps"
|
|
99
|
-
str +=
|
|
101
|
+
str += ' / %dm%02ds' % opts[:time_taken].divmod(60) if opts[:time_taken]
|
|
100
102
|
end
|
|
101
103
|
|
|
102
104
|
def finalize!
|
|
103
105
|
raise SitemapGenerator::SitemapFinalizedError if finalized?
|
|
106
|
+
|
|
104
107
|
reserve_name if create_index?
|
|
105
108
|
write_first_sitemap
|
|
106
109
|
@frozen = true
|
|
@@ -138,7 +141,7 @@ module SitemapGenerator
|
|
|
138
141
|
if @first_sitemap
|
|
139
142
|
@first_sitemap.link.write unless @first_sitemap.link.written?
|
|
140
143
|
super_add(SitemapGenerator::Builder::SitemapIndexUrl.new(@first_sitemap.link, @first_sitemap.options))
|
|
141
|
-
@link_count -= 1
|
|
144
|
+
@link_count -= 1 # we already counted it, don't count it twice
|
|
142
145
|
# Store the URL because if create_index is false, this is the
|
|
143
146
|
# "index" URL
|
|
144
147
|
@first_sitemap_url = @first_sitemap.link.location.url
|
|
@@ -1,12 +1,14 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
1
3
|
require 'builder'
|
|
2
4
|
|
|
3
5
|
module SitemapGenerator
|
|
4
6
|
module Builder
|
|
5
7
|
class SitemapIndexUrl < SitemapUrl
|
|
6
8
|
|
|
7
|
-
def initialize(path, options={})
|
|
9
|
+
def initialize(path, options = {})
|
|
8
10
|
if index = path.is_a?(SitemapGenerator::Builder::SitemapIndexFile) && path
|
|
9
|
-
options = SitemapGenerator::Utilities.reverse_merge(options, :
|
|
11
|
+
options = SitemapGenerator::Utilities.reverse_merge(options, host: index.location.host, lastmod: Time.now, priority: 1.0)
|
|
10
12
|
path = index.location.path_in_public
|
|
11
13
|
super(path, options)
|
|
12
14
|
else
|
|
@@ -15,11 +17,11 @@ module SitemapGenerator
|
|
|
15
17
|
end
|
|
16
18
|
|
|
17
19
|
# Return the URL as XML
|
|
18
|
-
def to_xml(builder=nil)
|
|
20
|
+
def to_xml(builder = nil)
|
|
19
21
|
builder = ::Builder::XmlMarkup.new if builder.nil?
|
|
20
22
|
builder.sitemap do
|
|
21
23
|
builder.loc self[:loc]
|
|
22
|
-
builder.lastmod w3c_date(self[:lastmod])
|
|
24
|
+
builder.lastmod w3c_date(self[:lastmod]) if self[:lastmod]
|
|
23
25
|
end
|
|
24
26
|
builder << '' # force to string
|
|
25
27
|
end
|