indexmap 0.12.0 → 0.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +21 -0
- data/README.md +34 -3
- data/UPGRADE.md +6 -4
- data/docs/search-engine-ping.md +62 -11
- data/docs/storage.md +1 -1
- data/lib/indexmap/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 3a6ec34705c1e8d3157b30768ba23f214c61985194b467dc851faaefe59d18bb
|
|
4
|
+
data.tar.gz: 37d6d9e215aa61cdd1a42208ae8be342a1769ad2b59d99457df5117f463c2ca2
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 637f065fd4266c1f43065c8114470fb2397a1844a2db4b7ecd286b461361d78e803d265ca2eea3afe87b9b92e8538591ab451945482933aeabd9314a3a541534
|
|
7
|
+
data.tar.gz: 92e9a2e58ce4e00f3a9d29df51e79ec51153ee276f1f6a284a3200bdf54c11c781dbad6bbb7102889010571e0b695ef6b95e87197fa3eeb92ec274ecdc21ffa0
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,27 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.12.1] - 2026-11-08
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
### Documentation
|
|
12
|
+
|
|
13
|
+
- recommend independent queued deliveries
|
|
14
|
+
|
|
15
|
+
- focus queue guidance on published URLs
|
|
16
|
+
|
|
17
|
+
- recommend independent queued deliveries ([#39](https://github.com/ethos-link/indexmap/pull/39))
|
|
18
|
+
|
|
19
|
+
- make Ethos Link the canonical documentation home
|
|
20
|
+
|
|
21
|
+
- add canonical documentation and branded social preview ([#40](https://github.com/ethos-link/indexmap/pull/40))
|
|
22
|
+
|
|
23
|
+
- keep audited technical guidance alongside the code
|
|
24
|
+
|
|
25
|
+
- keep audited technical guidance alongside the code ([#41](https://github.com/ethos-link/indexmap/pull/41))
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
|
|
8
29
|
## [0.12.0] - 2026-11-08
|
|
9
30
|
|
|
10
31
|
|
data/README.md
CHANGED
|
@@ -17,6 +17,11 @@ By default, `indexmap` writes a sitemap index plus one or more child sitemap fil
|
|
|
17
17
|
See [CHANGELOG.md](CHANGELOG.md) for published changes and [UPGRADE.md](UPGRADE.md)
|
|
18
18
|
for host migration steps.
|
|
19
19
|
|
|
20
|
+
## Requirements
|
|
21
|
+
|
|
22
|
+
Ruby 4.0 or newer. Active Support 8 is a runtime dependency; Rails itself is
|
|
23
|
+
optional. The Rails integration accepts Rails 8.0 or newer, below Rails 9.
|
|
24
|
+
|
|
20
25
|
## Installation
|
|
21
26
|
|
|
22
27
|
Add this line to your application's Gemfile:
|
|
@@ -47,6 +52,10 @@ directory prefix such as `sitemaps/`.
|
|
|
47
52
|
|
|
48
53
|
## Documentation
|
|
49
54
|
|
|
55
|
+
This README and the linked repository guides describe the checked-out code.
|
|
56
|
+
The [Ethos Link project page](https://www.ethos-link.com/opensource/indexmap)
|
|
57
|
+
provides the canonical public project guide, overview and resources.
|
|
58
|
+
|
|
50
59
|
- [How Indexmap works](docs/architecture.md): pipeline diagram and entrypoints.
|
|
51
60
|
- [Domain terms](docs/domain-terms.md): sections, partitions, outputs, publications,
|
|
52
61
|
roots, and entry types.
|
|
@@ -86,6 +95,12 @@ end
|
|
|
86
95
|
Indexmap.create
|
|
87
96
|
```
|
|
88
97
|
|
|
98
|
+
Run this from your project directory. It writes `public/sitemap.xml`, which
|
|
99
|
+
references `public/sitemap-marketing.xml`; the child contains the home and
|
|
100
|
+
pricing URLs. `Indexmap.create` generates and validates the selected output.
|
|
101
|
+
Use `Indexmap::TaskRunner#create` when you also need `after_create` callbacks
|
|
102
|
+
and IndexNow key-file handling.
|
|
103
|
+
|
|
89
104
|
## Rails Usage
|
|
90
105
|
|
|
91
106
|
Run the install generator after adding the gem:
|
|
@@ -96,10 +111,14 @@ bin/rails generate indexmap:install
|
|
|
96
111
|
|
|
97
112
|
The generator creates `config/initializers/indexmap.rb` using Rails' native
|
|
98
113
|
load hook and adds `require "indexmap/tasks"` to the application's `Rakefile`.
|
|
99
|
-
|
|
114
|
+
With the generated setup, normal web and worker boot does not load Indexmap.
|
|
115
|
+
Explicit runtime requires or mounted Indexmap controllers load it deliberately.
|
|
116
|
+
Rakefiles run only for tasks,
|
|
100
117
|
so task discovery needs no argument parsing.
|
|
101
118
|
|
|
102
|
-
Customize the generated initializer
|
|
119
|
+
Customize the generated initializer. `Sitemap.sections` below is an
|
|
120
|
+
application-owned method returning `Indexmap::Section` objects; replace it
|
|
121
|
+
with your own manifest or inline section definitions.
|
|
103
122
|
|
|
104
123
|
```ruby
|
|
105
124
|
ActiveSupport.on_load(:indexmap) do
|
|
@@ -508,7 +527,7 @@ fail after the default root is published, so monitor it separately.
|
|
|
508
527
|
```ruby
|
|
509
528
|
parser = Indexmap::Parser.new(source: "sitemap.xml")
|
|
510
529
|
parser.paths
|
|
511
|
-
# => ["/", "/
|
|
530
|
+
# => ["/", "/pricing"]
|
|
512
531
|
|
|
513
532
|
Indexmap::Validator.new.validate!
|
|
514
533
|
```
|
|
@@ -580,3 +599,15 @@ Use a disposable `indexmap_test` database; the integration suite recreates it.
|
|
|
580
599
|
|
|
581
600
|
Release preparation, publication, registry setup and recovery are documented
|
|
582
601
|
in the [Gemkit release guide](https://github.com/ethos-link/ethos-gemkit#releasing-a-gem).
|
|
602
|
+
|
|
603
|
+
## About Ethos Link
|
|
604
|
+
|
|
605
|
+
[Ethos Link](https://www.ethos-link.com) is a software company founded by
|
|
606
|
+
[Paulo Fidalgo](https://www.fidalgo.pt). This gem is part of the open-source
|
|
607
|
+
tooling we publish alongside our business products:
|
|
608
|
+
|
|
609
|
+
- [Reviato](https://www.reviato.com): collect private customer feedback and analyse public reviews.
|
|
610
|
+
- [Nuntio](https://www.nuntiohq.com): prepare, review and schedule social posts.
|
|
611
|
+
- [EthosPlaces](https://www.ethos.place): find places, opening hours and contact details.
|
|
612
|
+
|
|
613
|
+
Explore our [open-source projects](https://www.ethos-link.com/opensource).
|
data/UPGRADE.md
CHANGED
|
@@ -12,10 +12,12 @@ GitHub Actions runs commitlint and release packaging/publication only.
|
|
|
12
12
|
|
|
13
13
|
`Indexmap::Pinger::IndexNow#ping` and `.ping` now accept `entries:` containing
|
|
14
14
|
`Indexmap::Entry` objects. Existing calls without this keyword retain sitemap
|
|
15
|
-
selection. An explicit list submits only those URLs
|
|
16
|
-
|
|
17
|
-
submission. Replace app-owned IndexNow HTTP clients with this API
|
|
18
|
-
per
|
|
15
|
+
selection. An explicit list submits only those URLs without reading sitemap
|
|
16
|
+
files or applying date filters. Empty lists skip
|
|
17
|
+
submission. Replace app-owned IndexNow HTTP clients with this API. Prefer
|
|
18
|
+
independent queued deliveries per provider and URL, raising retryable failure
|
|
19
|
+
results into the host queue policy. A per-provider delivery table is optional,
|
|
20
|
+
not required; larger batches can use per-attempt receipts to retain progress. See
|
|
19
21
|
[direct change submission](docs/search-engine-ping.md#submit-known-changes-directly).
|
|
20
22
|
|
|
21
23
|
### Sitemap origin override
|
data/docs/search-engine-ping.md
CHANGED
|
@@ -31,7 +31,7 @@ IndexNow without generating or reading a sitemap:
|
|
|
31
31
|
```ruby
|
|
32
32
|
entries = [
|
|
33
33
|
Indexmap::Entry.new(loc: "https://example.com/articles/updated"),
|
|
34
|
-
Indexmap::Entry.new(loc: "https://example.com/
|
|
34
|
+
Indexmap::Entry.new(loc: "https://example.com/products/new")
|
|
35
35
|
]
|
|
36
36
|
|
|
37
37
|
result = Indexmap::Pinger::IndexNow.new.ping(entries: entries) do |event|
|
|
@@ -41,9 +41,9 @@ result = Indexmap::Pinger::IndexNow.new.ping(entries: entries) do |event|
|
|
|
41
41
|
end
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
Use this for publication events, imports
|
|
45
|
-
|
|
46
|
-
|
|
44
|
+
Use this for publication events, imports and edits. Applications can enqueue
|
|
45
|
+
known changes directly instead of comparing sitemap inventories. Entries must
|
|
46
|
+
use absolute HTTP(S) URLs matching the
|
|
47
47
|
configured base URL's scheme, host and port, without queries, fragments or
|
|
48
48
|
userinfo. All explicit entries are validated before the first submission, and
|
|
49
49
|
repeated URLs are submitted once.
|
|
@@ -55,11 +55,61 @@ fall back to the sitemap. Omitting `entries` retains sitemap selection.
|
|
|
55
55
|
Explicit entries reuse configured keys, key locations, endpoints, batch limits,
|
|
56
56
|
dry runs, bounded retries and result/event reporting. The class-level
|
|
57
57
|
`Indexmap::Pinger::IndexNow.ping(entries: entries)` form is also supported.
|
|
58
|
-
For durable delivery,
|
|
58
|
+
For durable delivery, use the host application's existing queue. Enqueue a
|
|
59
|
+
separate job for each provider and URL (or independently retryable batch). One
|
|
60
|
+
job class can take a `provider:` argument; each provider still gets its own job
|
|
61
|
+
instance. This isolates retries without requiring a per-provider delivery table.
|
|
62
|
+
GoodJob and Solid Queue can both use Active Job's `retry_on` policy. Keep
|
|
63
|
+
Indexmap's `max_attempts` at its default of one when the queue owns retries.
|
|
64
|
+
|
|
65
|
+
Indexmap returns failure results rather than raising for HTTP errors, timeouts
|
|
66
|
+
and connection failures. The application must turn retryable results into
|
|
67
|
+
exceptions for queue retries to run. A successful HTTP 202 is a receipt, not an
|
|
68
|
+
error. Treat permanent rejections as failed jobs for operator inspection; avoid
|
|
69
|
+
retrying invalid URLs indefinitely. A missing key needs correction and must not
|
|
70
|
+
be counted as successful delivery. A deliberate dry run makes no submission.
|
|
71
|
+
|
|
72
|
+
For example, an application can enqueue one URL at a time:
|
|
73
|
+
|
|
74
|
+
```ruby
|
|
75
|
+
class IndexNowPingJob < ApplicationJob
|
|
76
|
+
class RetryableError < StandardError; end
|
|
77
|
+
class RejectedError < StandardError; end
|
|
78
|
+
|
|
79
|
+
retry_on RetryableError, wait: :polynomially_longer, attempts: 10
|
|
80
|
+
|
|
81
|
+
def perform(url)
|
|
82
|
+
require "indexmap"
|
|
83
|
+
result = Indexmap::Pinger::IndexNow.new.ping(entries: [Indexmap::Entry.new(loc: url)])
|
|
84
|
+
return if [:submitted, :dry_run].include?(result.fetch(:status))
|
|
85
|
+
|
|
86
|
+
if result.fetch(:status) == :skipped
|
|
87
|
+
raise RetryableError, result.fetch(:reason).to_s
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
failure = result.fetch(:failures).first
|
|
91
|
+
status = failure[:status_code]
|
|
92
|
+
if failure[:reason] == :unknown_delivery || [401, 403, 429].include?(status) || (status && status >= 500)
|
|
93
|
+
raise RetryableError, "IndexNow delivery failed: #{status || failure[:reason]}"
|
|
94
|
+
else
|
|
95
|
+
raise RejectedError, "IndexNow rejected the URL: #{status}"
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
end
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Pass URLs directly in queued arguments. Applications select currently published
|
|
102
|
+
canonical URLs before enqueueing; queued URL snapshots can outlive the content.
|
|
103
|
+
Existing redirects and HTTP responses let search engines discover retired URLs
|
|
104
|
+
without a separate removal workflow. Do not infer deletion from sitemap absence.
|
|
105
|
+
An optional outbox can bridge content changes and enqueueing without tracking
|
|
106
|
+
provider retries. Delivery is at least once: a crash or timeout after remote
|
|
107
|
+
receipt can cause duplicates.
|
|
108
|
+
|
|
109
|
+
Larger jobs must isolate failed batches instead of retrying the whole inventory.
|
|
59
110
|
Each `attempt_finished` event identifies its URLs and result; later attempts for
|
|
60
|
-
the same URLs supersede earlier receipts.
|
|
61
|
-
|
|
62
|
-
those notifications pending.
|
|
111
|
+
the same URLs supersede earlier receipts. Persist per-batch progress only when
|
|
112
|
+
the workflow needs it, rather than requiring a separate ledger for every app.
|
|
63
113
|
|
|
64
114
|
## Google Search Console
|
|
65
115
|
|
|
@@ -337,9 +387,10 @@ Permanent errors and HTTP 202 do not retry. Both report events include an
|
|
|
337
387
|
`attempt` number, so an initially uncertain attempt remains visible even when a
|
|
338
388
|
later attempt succeeds. Summary counts describe final batch outcomes.
|
|
339
389
|
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
390
|
+
Use queue retries for one URL or one independently retryable batch. Do not wrap
|
|
391
|
+
a partially successful inventory submission in whole-job retries: that can
|
|
392
|
+
resend earlier successful batches. Indexmap's in-process retries do not survive
|
|
393
|
+
process restarts and may duplicate a remotely received request after a timeout. Generation and
|
|
343
394
|
submission remain separate. Preview and sending validate and spool the complete
|
|
344
395
|
reachable inventory before yielding selection or issuing requests; selected URLs
|
|
345
396
|
are then read in bounded batches instead of retaining an inventory-sized array.
|
data/docs/storage.md
CHANGED
|
@@ -36,7 +36,7 @@ These optional methods avoid whole-file read/write fallbacks during generation
|
|
|
36
36
|
and validation. Both are mandatory for the sitemap HTTP controller's reads:
|
|
37
37
|
|
|
38
38
|
```ruby
|
|
39
|
-
storage.open(filename) { |io|
|
|
39
|
+
storage.open(filename) { |io| io.read(1024) } # readable binary IO, closed after the block
|
|
40
40
|
storage.metadata(filename) # {etag:, bytes:, last_modified:}
|
|
41
41
|
storage.write_io(filename, io, content_type: "application/xml")
|
|
42
42
|
```
|
data/lib/indexmap/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: indexmap
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.12.
|
|
4
|
+
version: 0.12.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Paulo Fidalgo
|
|
@@ -201,8 +201,8 @@ metadata:
|
|
|
201
201
|
homepage_uri: https://www.ethos-link.com/opensource/indexmap
|
|
202
202
|
source_code_uri: https://github.com/ethos-link/indexmap
|
|
203
203
|
bug_tracker_uri: https://github.com/ethos-link/indexmap/issues
|
|
204
|
-
changelog_uri: https://github.com/ethos-link/indexmap/blob/v0.12.
|
|
205
|
-
documentation_uri: https://
|
|
204
|
+
changelog_uri: https://github.com/ethos-link/indexmap/blob/v0.12.1/CHANGELOG.md
|
|
205
|
+
documentation_uri: https://www.ethos-link.com/opensource/indexmap
|
|
206
206
|
funding_uri: https://www.reviato.com/
|
|
207
207
|
github_repo: ssh://github.com/ethos-link/indexmap
|
|
208
208
|
allowed_push_host: https://rubygems.org
|