site_maps 0.0.1.beta3 → 0.1.1

data/README.md

@@ -1,46 +1,65 @@
 # SiteMaps
 
-SiteMaps is a gem that helps you to generate sitemaps for your Rails application.
+A concurrent, incremental sitemap generator for Ruby. Framework-agnostic with built-in Rails support.
+
+Generates SEO-optimized XML sitemaps with support for sitemap indexes, XSL stylesheets, gzip compression, image/video/news extensions, search engine pinging, and Rack middleware for serving sitemaps with proper HTTP headers.
+
+## Documentation
+
+Full guides, adapter reference, CLI docs, and recipes are published at **[gems.marcosz.com.br/site_maps](https://gems.marcosz.com.br/site_maps/)** — part of the [marcosgz Ruby gem catalogue](https://gems.marcosz.com.br).
+
+## Table of Contents
+
+- [Documentation](#documentation)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Configuration](#configuration)
+- [Processes](#processes)
+- [Multi-Tenant Configuration](#multi-tenant-configuration)
+- [URL Filtering](#url-filtering)
+- [External Sitemaps](#external-sitemaps)
+- [Sitemap Extensions](#sitemap-extensions)
+- [XSL Stylesheets](#xsl-stylesheets)
+- [Rack Middleware](#rack-middleware)
+- [robots.txt](#robotstxt)
+- [Search Engine Ping](#search-engine-ping)
+- [Adapters](#adapters)
+- [CLI](#cli)
+- [Notifications](#notifications)
+- [Mixins](#mixins)
+- [Development](#development)
+- [License](#license)
 
 ## Installation
 
-Add this line to your application's Gemfile:
+Add to your Gemfile:
 
 ```ruby
-gem 'site_maps'
+gem "site_maps"
 ```
 
-And then execute:
+Then run `bundle install`.
 
-```bash
-bundle install
-```
-
-Or install it yourself as:
-
-```bash
-gem install site_maps
-```
+## Quick Start
 
-## Usage
-
-Create a configuration file where you will define the sitemap logic. You can use the following DSL to define the sitemap generation. Below is the minimum configuration required to generate a sitemap:
+Create a configuration file:
 
 ```ruby
 # config/sitemap.rb
 SiteMaps.use(:file_system) do
   configure do |config|
-    config.url = "https://example.com/sitemaps/sitemap.xml.gz" # Location of main sitemap index file
-    config.directory = "/home/www/public"
+    config.url = "https://example.com/sitemap.xml"
+    config.directory = Rails.public_path.to_s
   end
+
   process do |s|
-    s.add('/', priority: 1.0, changefreq: "daily")
-    s.add('/about', priority: 0.9, changefreq: "weekly")
+    s.add("/", lastmod: Time.now)
+    s.add("/about", lastmod: Time.now)
   end
 end
 ```
 
-After creating the configuration file, you can run the following command to generate the sitemap:
+Generate sitemaps:
 
 ```ruby
 SiteMaps.generate(config_file: "config/sitemap.rb")
@@ -48,544 +67,668 @@ SiteMaps.generate(config_file: "config/sitemap.rb")
   .run
 ```
 
-or you can use the CLI to generate the sitemap:
+Or via CLI:
 
 ```bash
 bundle exec site_maps generate --config-file config/sitemap.rb
 ```
 
-### Configuration
+## Configuration
 
-Configuration can be defined using the `configure` block or by passing the configuration options to the `use` method. Each adapter may have specific configuration options, but the following options are common to all adapters:
-
-* `url` - URL of the main sitemap index file. This URL must ends with `.xml` or `.xml.gz`.
-* `directory` - Directory where the sitemap files will be stored.
-
-Configuration using the `#configure` block
+Configuration can be set inside the `SiteMaps.use` block using `configure`, `config`, or by passing options directly:
 
 ```ruby
+# Block style
 SiteMaps.use(:file_system) do
   configure do |config|
-    config.url = "https://example.com/sitemaps/sitemap.xml.gz"
-    config.directory = "/home/www/public"
+    config.url = "https://example.com/sitemap.xml.gz"
+    config.directory = "/var/www/public"
   end
-  # define sitemap processes..
 end
-```
-
-Configuration using `#config` method
 
-```ruby
+# Inline style
 SiteMaps.use(:file_system) do
-  config.url = "https://example.com/sitemaps/sitemap.xml.gz"
-  config.directory = "/home/www/public"
-  # define sitemap processes..
+  config.url = "https://example.com/sitemap.xml.gz"
+  config.directory = "/var/www/public"
 end
+
+# Options style
+SiteMaps.use(:file_system, url: "https://example.com/sitemap.xml.gz", directory: "/var/www/public")
 ```
 
-Configuration passing options to the `#use` method
+### Common Options
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `url` | *required* | URL of the main sitemap index file. Must end with `.xml` or `.xml.gz`. |
+| `directory` | `"/tmp/sitemaps"` | Local directory for generated sitemap files. |
+| `max_links` | `50_000` | Maximum URLs per sitemap file before splitting. Set to `1_000` for Yoast-style performance. |
+| `emit_priority` | `true` | Include `<priority>` in XML output. Google ignores this — set to `false` to omit. |
+| `emit_changefreq` | `true` | Include `<changefreq>` in XML output. Google ignores this — set to `false` to omit. |
+| `xsl_stylesheet_url` | `nil` | URL of the XSL stylesheet for URL set sitemaps. Enables human-readable browser display. |
+| `xsl_index_stylesheet_url` | `nil` | URL of the XSL stylesheet for the sitemap index. |
+| `ping_search_engines` | `false` | Ping search engines after sitemap generation. |
+| `ping_engines` | `nil` | Custom engines hash. Defaults to Bing when `nil`. |
+
+### Gzip Compression
+
+Append `.gz` to the sitemap URL to enable automatic gzip compression:
 
 ```ruby
-SiteMaps.use(:file_system, url: "https://example.com/sitemaps/sitemap.xml.gz", directory: "/home/www/public") do
-  # define sitemap processes..
-end
+config.url = "https://example.com/sitemap.xml.gz"
 ```
 
-Refer to the adapter documentation to see the specific configuration options.
-
-### Gzip Compression
+### Priority and Change Frequency
 
-The sitemap files can be automatically compressed using the gzip algorithm. To enable the gzip compression, just pass the sitemap url with the `.gz` extension.
+Google and most search engines ignore `<priority>` and `<changefreq>` — only `<lastmod>` is meaningful. You can disable them:
 
 ```ruby
-# config/sitemap.rb
 SiteMaps.use(:file_system) do
   configure do |config|
-    config.url = "https://example.com/sitemaps/sitemap.xml.gz" # Location of main sitemap index file
-    config.directory = "/home/www/public"
-  end
-  process do |s|
-    # Add sitemap links
+    config.url = "https://example.com/sitemap.xml"
+    config.emit_priority = false
+    config.emit_changefreq = false
   end
 end
 ```
 
-### Sitemap Index
+When disabled, default values (`priority: 0.5`, `changefreq: "weekly"`) are not included in the XML output. If you explicitly pass `priority:` or `changefreq:` to `s.add`, they are still emitted regardless of the flag.
 
-For small websites, you can use a single sitemap file to store all the links. However, for large websites with thousands of links, you should use a sitemap index file to store the sitemap links. This library will automatically generate the sitemap index file if you define multiple processes or if the amount of links exceeds the maximum limit of links or file size.
+## Processes
 
+Processes define units of work for sitemap generation. Each process runs in a separate thread for concurrent generation.
 
-Criteria to generate the sitemap index file:
-* Multiple processes defined in the configuration file.
-* The amount of links exceeds the maximum limit of links (50,000 links).
-* The amount of news links exceeds the maximum limit of news links (1,000 links).
-* The uncompressed file size exceeds the maximum limit of file size (50MB).
+### Static Processes
 
-### Static and Dynamic Processes
-
-Sitemap links are defined in the `process` block because the gem is designed to generate sitemaps for large websites in parallel. It means that each process will be executed in a separate thread, which will improve the performance of the sitemap generation.
-
-Each process can have a unique name and a unique sitemap file location. By omitting the name and the file location, the process will use the `:default` value.
-
-Bellow is an example of a configuration file with multiple processes:
+Execute once with a fixed location:
 
 ```ruby
-# config/sitemap.rb
 SiteMaps.use(:file_system) do
-  configure do |config|
-    config.url = "https://example.com/sitemaps/sitemap.xml" # Location of main sitemap index file
-    config.directory = "/home/www/public"
-  end
-  # Static Processes
+  config.url = "https://example.com/sitemap.xml"
+
   process do |s|
-    s.add('/', priority: 1.0, changefreq: "daily")
-    s.add('/about', priority: 0.9, changefreq: "weekly")
+    s.add("/", lastmod: Time.now)
+    s.add("/about", lastmod: Time.now)
   end
+
   process :categories, "categories/sitemap.xml" do |s|
     Category.find_each do |category|
-      s.add(category_path(category), priority: 0.7)
+      s.add(category_path(category), lastmod: category.updated_at)
     end
   end
-  # Dynamic Processes
-  process :posts, "posts/%{year}-%{month}/sitemap.xml", year: Date.today.year, month: Date.today.month do |s, year:, month:|
+end
+```
+
+### Dynamic Processes
+
+Execute multiple times with different parameters. The location supports `%{placeholder}` interpolation:
+
+```ruby
+SiteMaps.use(:file_system) do
+  config.url = "https://example.com/sitemap.xml"
+
+  process :posts, "posts/%{year}-%{month}/sitemap.xml", year: Date.today.year, month: Date.today.month do |s, year:, month:, **|
     Post.where(year: year.to_i, month: month.to_i).find_each do |post|
-      s.add(post_path(post), priority: 0.8)
+      s.add(post_path(post), lastmod: post.updated_at)
     end
   end
 end
 ```
 
-Dynamic `process` are defined by passing a process name, a location, and a list of extra arguments that will be dinamically replaced by the given values in the `enqueue` method.
-
-Location can contain placeholders that will be replaced by the values passed to the process block(The `%{year}` and `%{month}` of example bellow). Both relative and absolute paths are supported. Note that when using relative paths, the base dir of main sitemap index file will be used as the root directory.
-
-It will let you enqueue the same process multiple times with different values.
+Enqueue dynamic processes with specific values:
 
 ```ruby
 SiteMaps.generate(config_file: "config/sitemap.rb")
-  .enqueue(:posts, year: "2021", month: "01")
-  .enqueue(:posts, year: "2021", month: "02")
-  .enqueue_remaining # Enqueue all remaining processes (default and categories)
+  .enqueue(:posts, year: "2024", month: "01")
+  .enqueue(:posts, year: "2024", month: "02")
+  .enqueue_remaining  # enqueue all other non-enqueued processes
   .run
 ```
 
-**Important Considerations:**
+**Note:** Dynamic process arguments may be strings when coming from CLI or external sources. Add `.to_i` or other conversions in the process block as needed.
+
+### Automatic Splitting
 
-* The values of the extra arguments may be strings when they are coming from the CLI or other sources.
-* By omitting the extra arguments, the process will be enqueued with the default values defined in the configuration file. So make sure you define default values or properly add nil checks in the process block to avoid errors.
+Sitemaps are automatically split into multiple files and a sitemap index is generated when:
 
-### Sitemap Extensions
+- Multiple processes are defined.
+- URL count exceeds `max_links` (default 50,000).
+- News URL count exceeds 1,000.
+- Uncompressed file size exceeds 50MB.
 
-The sitemap builder supports the following sitemap extensions:
+Split files are named sequentially: `sitemap1.xml`, `sitemap2.xml`, etc.
 
-* [Alternate](http://support.google.com/webmasters/bin/answer.py?hl=en&answer=2620865)
-* [Image](https://support.google.com/webmasters/answer/178636?hl=en)
-* [Mobile](http://support.google.com/webmasters/bin/answer.py?hl=en&answer=34648)
-* [News](https://support.google.com/news/publisher-center/answer/9606710?hl=en)
-* [PageMap](https://developers.google.com/custom-search/docs/structured_data?csw=1#pagemaps)
-* [Video](https://support.google.com/webmasters/answer/80471?hl=en)
+## Multi-Tenant Configuration
 
-You can add the sitemap links with the extensions by passing a hash with the extension name as the key and the extension attributes as the value.
+For multi-tenant applications where each site shares a config file but needs runtime context (like a `Site` model loaded from the database), use `SiteMaps.define` with the `context:` kwarg.
 
-#### Image
+The `context:` value must be a `Hash`. Its keys are passed as keyword arguments to the `define` block:
 
-Images can be added to the sitemap links by passing `images` attributes to the `add` method. The `images` attribute should be an array of hashes with the image attributes.
+```ruby
+# config/sitemap.rb
+SiteMaps.define do |site:, **|
+  use(:file_system) do
+    configure do |config|
+      config.url = "https://#{site.domain}/sitemap.xml"
+      config.directory = site.public_path
+    end
 
-Check out the Google specification [here](https://support.google.com/webmasters/answer/178636?hl=en).
+    process do |s|
+      site.pages.find_each { |p| s.add(p.path, lastmod: p.updated_at) }
+    end
+
+    process :posts, "posts/sitemap.xml" do |s|
+      site.posts.published.find_each { |p| s.add(p.path, lastmod: p.updated_at) }
+    end
+  end
+end
+```
 
 ```ruby
-config = { ... }
-SiteMaps.use(:file_system, **config) do
-  process do |s|
-    s.add(
-      '/',
-      priority: 1.0,
-      changefreq: "daily",
-      images: [
-        { loc: "https://example.com/image.jpg" }
-      ],
-    )
+# Usage — iterate sites, each gets its own isolated adapter
+Site.find_each do |site|
+  SiteMaps.generate(config_file: "config/sitemap.rb", context: {site: site}).enqueue_all.run
+end
+```
+
+Multiple context values are passed as additional Hash keys:
+
+```ruby
+SiteMaps.define do |site:, locale:|
+  use(:file_system) do
+    config.url = "https://#{site.domain}/#{locale}/sitemap.xml"
+    # ...
   end
 end
+
+SiteMaps.generate(config_file: "config/sitemap.rb", context: {site: site, locale: "en"}).run
 ```
 
-Supported attributes:
-* `loc` - URL of the image.
-* `caption` - Image caption.
-* `geo_location` - Image geo location.
-* `title` - Image title.
-* `license` - Image license.
+### Serving with Rack Middleware
 
-#### Video
+`SiteMaps::Middleware` supports multi-tenant setups via a callable `adapter:`. Because the adapter is resolved per-request, you can derive it from thread-local state set by an upstream middleware (e.g. `Current.site`):
 
-Videos can be added to the sitemap links by passing `videos` attributes to the `add` method. The `videos` attribute should be an array of hashes with the video attributes.
+```ruby
+# Insert after your multitenancy middleware so Current.site is already set
+Rails.application.middleware.insert_after MultitenancyMiddleware, SiteMaps::Middleware,
+  adapter: -> {
+    site = Current.site
+    next unless site
 
-Check out the Google specification [here](https://support.google.com/webmasters/answer/80471?hl=en).
+    SiteMaps::Adapters::FileSystem.new(url: site.sitemap_url, directory: "tmp/")
+  }
+```
+
+Both `adapter:` and the prefix options accept a 0-arg lambda (reads thread-local state) or a 1-arg lambda (receives the Rack `env`).
+
+#### Path mapping options
+
+Use these when the public URL path and the storage path differ:
+
+| Option | Direction | Example |
+|---|---|---|
+| `public_prefix:` | Public URL has an extra prefix → strip it to find the file | Stored at `/sitemap.xml`, served at `/sitemaps/tenant/sitemap.xml` |
+| `storage_prefix:` | Storage has an extra prefix → prepend it to the public path | Stored at `/sitemaps/tenant/sitemap.xml`, served at `/sitemap.xml` |
 
 ```ruby
-config = { ... }
-SiteMaps.use(:file_system, **config) do
-  process do |s|
-    s.add(
-      '/',
-      priority: 1.0,
-      changefreq: "daily",
-      videos: [
-        {
-          thumbnail_loc: "https://example.com/thumbnail.jpg",
-          title: "Video Title",
-          description: "Video Description",
-          content_loc: "https://example.com/video.mp4",
-          player_loc: "https://example.com/player.swf",
-          allow_embed: "yes",
-          autoplay: "ap=1",
-          # ...
-        }
-      ],
-    )
+# Sitemaps stored at /sitemaps/{slug}/sitemap.xml, served at /sitemap.xml
+# (subdomain identifies the tenant, no prefix needed in the public URL)
+Rails.application.middleware.insert_after MultitenancyMiddleware, SiteMaps::Middleware,
+  storage_prefix: -> { site = Current.site; "/sitemaps/#{site.slug}" if site },
+  adapter: -> { ... }
+
+# Sitemaps stored at root, served at /sitemaps/{slug}/sitemap.xml
+Rails.application.middleware.insert_after MultitenancyMiddleware, SiteMaps::Middleware,
+  public_prefix: -> { site = Current.site; "/sitemaps/#{site.slug}" if site },
+  adapter: -> { ... }
+```
+
+XSL stylesheet requests (`/_sitemap-stylesheet.xsl`, `/_sitemap-index-stylesheet.xsl`) are served directly without resolving the adapter or prefix.
+
+### Thread safety
+
+`SiteMaps.generate(config_file:, context:)` is thread-safe. Each call uses a thread-local scope to isolate adapter construction during `load(config_file)`, so concurrent calls from different threads don't race on module-level state:
+
+```ruby
+Site.find_each.map do |site|
+  Thread.new do
+    SiteMaps.generate(config_file: "config/sitemap.rb", context: {site: site}).enqueue_all.run
   end
+end.each(&:join)
+```
+
+Each thread's `Runner` gets its own isolated adapter. Note that `SiteMaps.current_adapter` (the module singleton) exhibits last-writer-wins semantics under concurrency — use the `Runner`'s `#adapter` attribute if you need a specific generation's adapter.
+
+For cases where you want to skip the config file entirely (e.g., everything dynamic from the database), instantiate adapters directly:
+
+```ruby
+adapter = SiteMaps::Adapters::FileSystem.new do
+  config.url = "https://#{site.domain}/sitemap.xml"
+  # ...
 end
+SiteMaps::Runner.new(adapter).enqueue_all.run
 ```
-Supported attributes:
-* `thumbnail_loc` - URL of the thumbnail image.
-* `title` - Title of the video.
-* `description` - Description of the video.
-* `content_loc` - URL of the video content.
-* `player_loc` - URL of the video player.
-* `allow_embed` - Allow embed attribute of the player location.
-* `autoplay` - Autoplay attribute of the player location.
-* `duration` - Duration of the video in seconds.
-* `expiration_date` - Expiration date of the video.
-* `rating` - Rating of the video.
-* `view_count` - View count of the video.
-* `publication_date` - Publication date of the video.
-* `tags` - Tags of the video.
-* `tag` - Single tag of the video.
-* `category` - Category of the video.
-* `family_friendly` - Family friendly attribute of the video.
-* `gallery_loc` - URL of the video gallery.
-* `gallery_title` - Title of the video gallery.
-* `uploader` - Uploader of the video.
-* `uploader_info` - Uploader info of the video.
-* `price` - Price of the video.
-* `price_currency` - Currency of the video price.
-* `price_type` - Type of the video price.
-* `price_resolution` - Resolution of the video price.
-* `live` - Live attribute of the video.
-* `requires_subscription` - Requires subscription attribute of the video.
-
-#### PageMap
-
-PageMap sitemaps can be added to the sitemap links by passing `pagemap` attributes to the `add` method. The `pagemap` attribute should be a hash with the pagemap attributes.
-
-Check out the Google specification [here](https://developers.google.com/custom-search/docs/structured_data?csw=1#pagemaps).
-
-```ruby
-config = { ... }
-SiteMaps.use(:file_system, **config) do
+
+## URL Filtering
+
+Use `url_filter` to exclude or modify URLs before they enter the sitemap. Filters receive the full URL string and the options hash. Return `false` to exclude, or a modified hash to change options:
+
+```ruby
+SiteMaps.use(:file_system) do
+  config.url = "https://example.com/sitemap.xml"
+
+  # Exclude admin URLs
+  url_filter { |url, _options| false if url.include?("/admin") }
+
+  # Override priority for blog posts
+  url_filter do |url, options|
+    if url.include?("/blog/")
+      options.merge(priority: 0.9)
+    else
+      options
+    end
+  end
+
   process do |s|
-    s.add(
-      '/',
-      priority: 1.0,
-      changefreq: "daily",
-      pagemap: {
-        dataobjects: [
-          {
-            type: "document",
-            id: "1",
-            attributes: [
-              { name: "title", value: "Page Title" },
-              { name: "description", value: "Page Description" },
-              { name: "url", value: "https://example.com" },
-            ]
-          }
-        ]
-      }
-    )
+    s.add("/", lastmod: Time.now)
+    s.add("/admin/dashboard")  # excluded by filter
+    s.add("/blog/hello-world", lastmod: Time.now)  # priority overridden to 0.9
   end
 end
 ```
 
-Supported attributes:
-* `dataobjects` - Array of hashes with the data objects.
-    * `type` - Type of the object.
-    * `id` - ID of the object.
-    * `attributes` - Array of hashes with the attributes.
-        * `name` - Name of the attribute.
-        * `value` - Value of the attribute.
-
-#### News
+Multiple filters are chained in order. If any filter returns `false`, the URL is excluded and subsequent filters are not called.
 
-News sitemaps can be added to the sitemap links by passing `news` attributes to the `add` method. The `news` attribute should be a hash with the news attributes.
+## External Sitemaps
 
-Check out the Google specification [here](https://support.google.com/news/publisher-center/answer/9606710?hl=en).
+Add third-party or externally-hosted sitemaps to your sitemap index using `external_sitemap`:
 
 ```ruby
-config = { ... }
-SiteMaps.use(:file_system, **config) do
+SiteMaps.use(:file_system) do
+  config.url = "https://example.com/sitemap.xml"
+
+  external_sitemap "https://cdn.example.com/products-sitemap.xml", lastmod: Time.now
+  external_sitemap "https://blog.example.com/sitemap.xml"
+
   process do |s|
-    s.add(
-      '/',
-      priority: 1.0,
-      changefreq: "daily",
-      news: {
-        publication_name: "Publication Name",
-        publication_language: "en",
-        publication_date: Time.now,
-        genres: "PressRelease",
-        access: "Subscription",
-        title: "News Title",
-        keywords: "News Keywords",
-        stock_tickers: "NASDAQ:GOOG",
-      }
-    )
+    s.add("/", lastmod: Time.now)
   end
 end
 ```
 
-Supported attributes:
-* `publication_name` - Name of the publication.
-* `publication_language` - Language of the publication.
-* `publication_date` - Publication date of the news.
-* `genres` - Genres of the news.
-* `access` - Access of the news.
-* `title` - Title of the news.
-* `keywords` - Keywords of the news.
-* `stock_tickers` - Stock tickers of the news.
+External sitemaps appear in the sitemap index alongside your generated sitemaps. When external sitemaps are present, the index is always generated (even with a single process).
 
-#### Alternates
+## Sitemap Extensions
 
-You can add alternate links to the sitemap links by passing `alternates` attributes to the `add` method. The `alternates` attribute should be an array of hashes with the alternate attributes.
+### Image
 
-Check out the Google specification [here](http://support.google.com/webmasters/bin/answer.py?hl=en&answer=2620865).
+Up to 1,000 images per URL. See [Google specification](https://support.google.com/webmasters/answer/178636?hl=en).
 
 ```ruby
-config = { ... }
-SiteMaps.use(:file_system, **config) do
-  process do |s|
-    s.add(
-      '/',
-      priority: 1.0,
-      changefreq: "daily",
-      alternates: [
-        { href: "https://example.com/en", lang: "en" },
-        { href: "https://example.com/es", lang: "es" },
-      ],
-    )
-  end
-end
+s.add("/gallery",
+  lastmod: Time.now,
+  images: [
+    { loc: "https://example.com/photo1.jpg", title: "Photo 1", caption: "A photo" },
+    { loc: "https://example.com/photo2.jpg", title: "Photo 2" }
+  ]
+)
 ```
 
-Supported attributes:
-* `href` - URL of the alternate link. (Required)
-* `lang` - Language of the alternate link. (Optional)
-* `nofollow` - Nofollow attribute of the alternate link. (Optional)
-* `media` - Media targets for responsive design pages. (Optional)
+Attributes: `loc`, `caption`, `geo_location`, `title`, `license`.
 
-#### Mobile
+### Video
 
-Mobile sitemaps include a specific <mobile:mobile/> tag.
+See [Google specification](https://support.google.com/webmasters/answer/80471?hl=en).
 
-Check out the Google specification [here](http://support.google.com/webmasters/bin/answer.py?hl=en&answer=34648).
+```ruby
+s.add("/videos/example",
+  lastmod: Time.now,
+  videos: [
+    {
+      thumbnail_loc: "https://example.com/thumb.jpg",
+      title: "Example Video",
+      description: "An example video",
+      content_loc: "https://example.com/video.mp4",
+      duration: 600,
+      publication_date: Time.now
+    }
+  ]
+)
+```
+
+Attributes: `thumbnail_loc`, `title`, `description`, `content_loc`, `player_loc`, `allow_embed`, `autoplay`, `duration`, `expiration_date`, `rating`, `view_count`, `publication_date`, `tags`, `tag`, `category`, `family_friendly`, `gallery_loc`, `gallery_title`, `uploader`, `uploader_info`, `price`, `live`, `requires_subscription`.
+
+### News
+
+Up to 1,000 news URLs per sitemap. See [Google specification](https://support.google.com/news/publisher-center/answer/9606710?hl=en).
 
 ```ruby
-config = { ... }
-SiteMaps.use(:file_system, **config) do
-  process do |s|
-    s.add('/', mobile: true)
-  end
-end
+s.add("/article/breaking-news",
+  lastmod: Time.now,
+  news: {
+    publication_name: "Example Times",
+    publication_language: "en",
+    publication_date: Time.now,
+    title: "Breaking News Story",
+    keywords: "breaking, news",
+    genres: "PressRelease",
+    access: "Subscription",
+    stock_tickers: "NASDAQ:GOOG"
+  }
+)
 ```
 
-Supported attributes:
+Attributes: `publication_name`, `publication_language`, `publication_date`, `genres`, `access`, `title`, `keywords`, `stock_tickers`.
 
-* `mobile` - Mobile attribute of the sitemap link.
+### Alternates (hreflang)
 
-## Adapters
+For multi-language sites. See [Google specification](https://support.google.com/webmasters/answer/189077).
 
-The gem provides adapters to store the sitemaps in different locations. The following adapters are available:
+```ruby
+s.add("/",
+  lastmod: Time.now,
+  alternates: [
+    { href: "https://example.com/en", lang: "en" },
+    { href: "https://example.com/es", lang: "es" },
+    { href: "https://example.com/fr", lang: "fr" }
+  ]
+)
+```
 
-* File System
-* AWS S3
+Attributes: `href` (required), `lang`, `nofollow`, `media`.
 
-### File System
+### Mobile
 
-You can use the file system adapter to store the sitemaps in the file system. The configuration is simple, you just need to provide the directory where the sitemaps will be stored.
+See [Google specification](https://support.google.com/webmasters/answer/34648).
 
 ```ruby
+s.add("/mobile-page", mobile: true)
+```
+
+### PageMap
+
+For Google Custom Search. See [Google specification](https://developers.google.com/custom-search/docs/structured_data#pagemaps).
+
+```ruby
+s.add("/product",
+  lastmod: Time.now,
+  pagemap: {
+    dataobjects: [
+      {
+        type: "product",
+        id: "sku-123",
+        attributes: [
+          { name: "name", value: "Widget" },
+          { name: "price", value: "19.99" }
+        ]
+      }
+    ]
+  }
+)
+```
+
+## XSL Stylesheets
+
+XSL stylesheets transform raw XML into styled HTML tables when sitemaps are opened in a browser — making them human-readable for debugging and review.
+
+The gem ships with built-in stylesheets for both URL set sitemaps and sitemap indexes.
 
+### Using with Rack Middleware
+
+The simplest setup — the middleware serves both sitemaps and stylesheets:
+
+```ruby
 SiteMaps.use(:file_system) do
   configure do |config|
-    config.url = "https://example.com/sitemaps/sitemap.xml.gz"
-    config.directory = "/home/www/public"
+    config.url = "https://example.com/sitemap.xml"
+    config.xsl_stylesheet_url = "/_sitemap-stylesheet.xsl"
+    config.xsl_index_stylesheet_url = "/_sitemap-index-stylesheet.xsl"
   end
-  process do |s|
-    # Add sitemap links
+end
+```
+
+### Using with Static Files
+
+Generate the XSL files and serve them as static assets:
+
+```ruby
+# Write stylesheets to disk
+File.write("public/sitemap-style.xsl", SiteMaps::Builder::XSLStylesheet.urlset_xsl)
+File.write("public/sitemap-index-style.xsl", SiteMaps::Builder::XSLStylesheet.index_xsl)
+```
+
+Then point the config to the static URLs:
+
+```ruby
+config.xsl_stylesheet_url = "https://example.com/sitemap-style.xsl"
+config.xsl_index_stylesheet_url = "https://example.com/sitemap-index-style.xsl"
+```
+
+## Rack Middleware
+
+`SiteMaps::Middleware` serves sitemaps over HTTP with SEO-appropriate headers:
+
+- `Content-Type: text/xml; charset=UTF-8`
+- `X-Robots-Tag: noindex, follow` — prevents search engines from indexing the sitemap itself
+- `Cache-Control: public, max-age=3600`
+
+It also serves the built-in XSL stylesheets at `/_sitemap-stylesheet.xsl` and `/_sitemap-index-stylesheet.xsl`.
+
+### Rails
+
+```ruby
+# config/application.rb
+config.middleware.use SiteMaps::Middleware
+```
+
+### Rack
+
+```ruby
+# config.ru
+use SiteMaps::Middleware
+run MyApp
+```
+
+### Options
+
+```ruby
+use SiteMaps::Middleware,
+  adapter: SiteMaps.current_adapter,        # defaults to SiteMaps.current_adapter
+  public_prefix: nil,                       # strip this prefix from the public URL before lookup
+  storage_prefix: nil,                      # prepend this prefix to the public URL for storage lookup
+  x_robots_tag: "noindex, follow",          # default
+  cache_control: "public, max-age=3600"     # default
+```
+
+Non-matching requests pass through to the next middleware.
+
+## robots.txt
+
+`SiteMaps::RobotsTxt` generates the `Sitemap:` directive for your `robots.txt`:
+
+```ruby
+# Get just the directive line
+SiteMaps::RobotsTxt.sitemap_directive("https://example.com/sitemap.xml")
+# => "Sitemap: https://example.com/sitemap.xml"
+
+# Auto-detect from current adapter
+SiteMaps::RobotsTxt.sitemap_directive
+# => "Sitemap: https://example.com/sitemap.xml"
+
+# Generate a complete robots.txt
+SiteMaps::RobotsTxt.render(
+  sitemap_url: "https://example.com/sitemap.xml",
+  extra_directives: ["Disallow: /admin/"]
+)
+# => "User-agent: *\nAllow: /\nDisallow: /admin/\nSitemap: https://example.com/sitemap.xml\n"
+```
+
+In a Rails controller:
+
+```ruby
+class RobotsController < ApplicationController
+  def show
+    render plain: SiteMaps::RobotsTxt.render
   end
 end
 ```
 
-### AWS S3
+## Search Engine Ping
 
-You can use the AWS S3 adapter to store the sitemaps in an S3 bucket. The configuration is similar to the file system adapter, but you need to provide the AWS SDK options.
+After sitemap generation, ping search engines to notify them of updates:
 
 ```ruby
-SiteMaps.use(:aws_sdk) do
+SiteMaps.use(:file_system) do
   configure do |config|
-    config.url = "https://my-bucket.s3.amazonaws.com/sitemaps/sitemap.xml"
-    config.directory = "/tmp" # Local directory to store the sitemaps before uploading to S3
-    # AWS S3 specific options
-    config.bucket = "my-bucket"
-    config.region = "us-east-1"
-    config.aws_access_key = ENV["AWS_ACCESS_KEY_ID"]
-    config.aws_secret_key = ENV["AWS_SECRET_ACCESS_KEY"]
-    # Optional parameters (default values)
-    config.acl = "public-read"
-    config.cache_control = "private, max-age=0, no-cache"
-  end
-  process do |s|
-    # Add sitemap links
+    config.url = "https://example.com/sitemap.xml"
+    config.ping_search_engines = true
   end
 end
 ```
 
-If you want to let your rails application as a proxy to the sitemap files, you can create a controller to serve the sitemap files from the S3 bucket.
+By default, only Bing is pinged (`https://www.bing.com/ping?sitemap=...`). Google deprecated their ping endpoint in 2023 — they discover sitemaps via `robots.txt` and Search Console.
+
+### Custom Engines
 
 ```ruby
-# config/routes.rb
-get "sitemaps/*relative_path", to: "sitemaps#show", as: :sitemap
+config.ping_engines = {
+  bing: "https://www.bing.com/ping?sitemap=%{url}",
+  google: "https://www.google.com/ping?sitemap=%{url}",
+  custom: "https://search.example.com/ping?url=%{url}"
+}
 ```
 
+### Ping via generate / CLI
+
+Use the `ping:` option to trigger a ping for a specific run without changing the config file:
+
 ```ruby
-# app/controllers/sitemaps_controller.rb
-class SitemapsController < ApplicationController
-  def show
-    location = params.permit("relative_path", "format").to_h.values.join(".")
+SiteMaps.generate(config_file: "config/sitemap.rb", ping: true).enqueue_all.run
+```
 
-    unless location =~ /\.xml(\.gz)?$/ # You may want add more validations here
-      raise ActionController::RoutingError, "Not found"
-    end
+```bash
+bundle exec site_maps generate --config-file config/sitemap.rb --ping
+```
 
-    data, meta = SiteMaps.current_adapter.read(File.join("sitemaps", location))
-    if location.ends_with?(".xml")
-      render xml: data
-    else
-      send_data(data, disposition: "attachment", type: meta[:content_type])
-    end
-  rescue SiteMaps::FileNotFoundError
-    raise ActionController::RoutingError, "Not found"
+`ping: true` overrides `config.ping_search_engines`. `ping: false` suppresses pinging even if the config enables it. Omitting `ping:` (the default) defers to the config value.
+
+### Manual Ping
+
+```ruby
+SiteMaps::Ping.ping("https://example.com/sitemap.xml")
+# => { bing: { status: 200, url: "https://www.bing.com/ping?sitemap=..." } }
+```
+
+## Adapters
+
+### File System
+
+Writes sitemaps to the local filesystem:
+
+```ruby
+SiteMaps.use(:file_system) do
+  configure do |config|
+    config.url = "https://example.com/sitemap.xml.gz"
+    config.directory = "/var/www/public"
   end
 end
 ```
 
-Make sure to let sitemap config in the initializer. You may want to add some caching to avoid hitting the S3 bucket on every request.
+### AWS S3
 
+Writes sitemaps to an S3 bucket:
 
-### Custom Adapters
+```ruby
+SiteMaps.use(:aws_sdk) do
+  configure do |config|
+    config.url = "https://my-bucket.s3.amazonaws.com/sitemaps/sitemap.xml"
+    config.directory = "/tmp"
+    config.bucket = "my-bucket"
+    config.region = "us-east-1"
+    config.access_key_id = ENV["AWS_ACCESS_KEY_ID"]
+    config.secret_access_key = ENV["AWS_SECRET_ACCESS_KEY"]
+    config.acl = "public-read"                          # default
+    config.cache_control = "private, max-age=0, no-cache" # default
+  end
+end
+```
 
-You can create custom adapters to store the sitemaps in different locations. You just need to create a class that implements the `SiteMaps::Adapters::Adapter` interface. The adapter should implement the following methods:
+### Custom Adapters
 
-* `write(url, raw_data, **extra)` - Write the sitemap data to the storage.
-* `read(url)` - Read the sitemap data from the storage.
-* `delete(url)` - Delete the sitemap data from the storage.
+Implement the `SiteMaps::Adapters::Adapter` interface:
 
 ```ruby
 class MyAdapter < SiteMaps::Adapters::Adapter
-  def write(url, raw_data, **extra)
-    # Write the sitemap data to the storage
+  def write(url, raw_data, **kwargs)
+    # Write sitemap data to storage
   end
 
   def read(url)
-    # Read the sitemap data from the storage
+    # Return [raw_data, { content_type: "application/xml" }]
   end
 
   def delete(url)
-    # Delete the sitemap data from the storage
+    # Delete sitemap from storage
   end
 end
 
-SiteMaps.use(MyAdapter, **config) do
-  process do |s|
-    # Add sitemap links
-  end
-end
-```
-
-#### Adapter Configuration
-
-If you adapter requires additional configuration, you can define a `<adapter class>::Config` inheriting from `SiteMaps::Configuration` and implement the required configuration options.
-
-```ruby
-class MyAdapter::Config < SiteMaps::Configuration
-  attribute :api_key, String
+SiteMaps.use(MyAdapter) do
+  config.url = "https://example.com/sitemap.xml"
 end
 ```
 
-During the adapter initialization, it will automatically detect the configuration class and use it to load the configuration options.
+For adapter-specific configuration, define a nested `Config` class:
 
 ```ruby
-SiteMaps.use(MyAdapter) do
-  configure do |config|
-    # ...
-    config.api_key = "my-api-key"
-  end
-  process do |s|
-    # Add sitemap links
+class MyAdapter < SiteMaps::Adapters::Adapter
+  class Config < SiteMaps::Configuration
+    attribute :api_key, default: -> { ENV["MY_API_KEY"] }
   end
 end
 ```
 
 ## CLI
 
-You can use the CLI to generate the sitemap. The CLI will load the configuration file and run the sitemap generation.
-
 ```bash
+# Generate all sitemaps
 bundle exec site_maps generate --config-file config/sitemap.rb
-```
-
-To enqueue dynamic processes, you can pass the process name with the context values.
 
-```bash
+# Enqueue a dynamic process with context
 bundle exec site_maps generate monthly_posts \
   --config-file config/sitemap.rb \
-  --context=year:2021 month:1
-```
-
-Enqueue dynamic + remaining processes
+  --context=year:2024 month:1
 
-```bash
+# Enqueue dynamic + remaining processes
 bundle exec site_maps generate monthly_posts \
   --config-file config/sitemap.rb \
-  --context=year:2021 month:1 \
+  --context=year:2024 month:1 \
   --enqueue-remaining
-```
 
-passing max threads to run the processes in parallel
-
-```bash
+# Control concurrency
 bundle exec site_maps generate \
   --config-file config/sitemap.rb \
   --max-threads 10
 ```
 
-## Notification
+## Notifications
 
-You can subscribe to the internal events to receive notifications about the sitemap generation. The following events are available:
+Subscribe to internal events for monitoring sitemap generation:
 
-* `sitemaps.enqueue_process` - Triggered when a process is enqueued.
-* `sitemaps.before_process_execution` - Triggered before a process starts execution
-* `sitemaps.process_execution` - Triggered when a process finishes execution.
-* `sitemaps.finalize_urlset` - Triggered when the sitemap builder finishes the URL set.
-
-You can subscribe to the events using the following code:
+| Event | Description |
+|-------|-------------|
+| `sitemaps.enqueue_process` | A process was enqueued |
+| `sitemaps.before_process_execution` | A process is about to start |
+| `sitemaps.process_execution` | A process finished execution |
+| `sitemaps.finalize_urlset` | A URL set was finalized and written |
+| `sitemaps.ping` | Search engines were pinged |
 
 ```ruby
-SiteMaps::Notification.subscribe("sitemaps.enqueue_process") do |event|
-  puts "Enqueueing process #{event.payload[:name]}"
+SiteMaps::Notification.subscribe("sitemaps.finalize_urlset") do |event|
+  puts "Wrote #{event.payload[:links_count]} links to #{event.payload[:url]}"
 end
 ```
 
-We have the standard event handler `SiteMaps::Runner::EventListener` that will print the events to the standard output. You can use it to view the progress of the sitemap generation.
+Use the built-in event listener for console output:
 
 ```ruby
 SiteMaps::Notification.subscribe(SiteMaps::Runner::EventListener)
@@ -596,52 +739,47 @@ SiteMaps.generate(config_file: "config/sitemap.rb")
 
 ## Mixins
 
-You can use mixins to extend the sitemap builder with additional methods. The mixins can be used to define common methods that will be used in multiple processes. Make sure they are thread-safe, otherwise I recommend to define them in the process block.
+Extend the sitemap builder with custom methods shared across processes:
 
 ```ruby
-module MyMixin
+module SitemapHelpers
   def repository
     Repository.new
   end
-
-  def post_path(post)
-    "/posts/#{post.slug}"
-  end
 end
 
 SiteMaps.use(:file_system) do
-  include_module(MyMixin)
+  extend_processes_with(SitemapHelpers)
+
   process do |s|
-    repository.posts.each do |post|
-      s.add(post_path(post), priority: 0.8)
+    s.repository.posts.each do |post|
+      s.add("/posts/#{post.slug}", lastmod: post.updated_at)
     end
   end
 end
 ```
 
-We already have a built-in mixin for Rails applications that provides the url helpers through the `route` method.
+Rails applications get a built-in mixin with URL helpers via the `route` method:
 
 ```ruby
-SiteMaps.use(:file_system) do
-  include_module(SiteMaps::Mixins::Rails)
-  process do |s|
-    s.add(route.root_path, priority: 1.0)
-    s.add(route.about_path, priority: 0.9)
-  end
+process do |s|
+  s.add(s.route.root_path, lastmod: Time.now)
+  s.add(s.route.about_path, lastmod: Time.now)
 end
 ```
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
-
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+After checking out the repo, run `bin/setup` to install dependencies. Run `bin/console` for an interactive prompt.
 
-## Contributing
-
-Bug reports and pull requests are welcome on GitHub at https://github.com/marcosgz/site_maps.
+```bash
+bundle exec rspec        # run tests
+bundle exec rubocop      # run linter
+bundle exec rake install # install locally
+```
 
+Bug reports and pull requests are welcome on [GitHub](https://github.com/marcosgz/site_maps).
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+Available as open source under the [MIT License](https://opensource.org/licenses/MIT).