urlpipe 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ff32c6fa848dfe556a24917001d75e73954deaca028fd68cdc9996b739f1acb7
-  data.tar.gz: 241d66afca02a3cd80c2a9f71b922eb9e9e5f95c5855bd57bf0e12a3a2a4d545
+  metadata.gz: 4bfaf07a0fb70edbee60c2467ec2f6123bc8c879c4eced1567f9ecbd51cf26d7
+  data.tar.gz: fc6ad0f280bc76a769a38383b4848ab98cf578360a7cbbb74a793dd94910571f
 SHA512:
-  metadata.gz: 673c4a11f37ba6663153d34fe4d072e78bf936707399c8c3c7b93b29ca3e9406b921b9ddeb304735978a741c714344c5042f420faa2cd4b7d4a54c3f7ac0bb15
-  data.tar.gz: b843a48d2ddaf91657a8dec8229c0bf2d54ce8f8fdce348bf849f81e16be978b546a9fca734b545c6e4f8f4c0becd50eb70701bb6f19773ba8aa5f5586e8c66b
+  metadata.gz: 6f8edccde9e41a55d017aa54c7d9da319212b6c074f37cc405d3efa37387345dc1ee956d690bf9d12cf840dbd5721f962cf233e491981b31f404836e8a58e0f5
+  data.tar.gz: fd35420d0c9087f63e65431b3c06092619201b538569a7d608b9c89559d6c7ba038a9712f16d6b5beae6563fa3d37002405154c04d66ea07c9686ca44c7930b9

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 0.2.0
+
+- `pdf` returns the whole rendered page as a PDF (`POST /pdf`): a `Urlpipe::Pdf` with the decoded bytes, `mime_type`, `result_url` and `save`. `pdf_options:` choose the paper, orientation, margins, scale, backgrounds, the print or screen stylesheet and `single_page`.
+- `extract` returns data from a page in a schema you define (`POST /extract`): pass `fields` and get a Hash with exactly those keys, in order, `nil` where the page gives no value. `instructions:` adds guidance in plain words.
+- `scrape` takes `pdf_options:`, `fields:` and `instructions:` for its `pdf` and `extract` operations.
+- `user_agent:` on `Urlpipe::Client.new` names the program making the requests; the User-Agent then reads `<product> urlpipe-ruby/<version>`.
+- `technologies` detects what a page is built with (`POST /technologies`): its CMS, frameworks, analytics, CDN and web server, each with its version, a confidence and the signals it was detected by.
+- `Meta#final_url` is the address the page ended on, after its redirects (the `X-Final-Url` header). A `/meta` result, a `/scrape` result and a webhook payload carry it as `"final_url"` too.
+- `meta` returns the fixed shape of `/meta`: every key always present — `canonical_url`, `title`, `description`, `site_name`, `type`, `language`, `locale`, `authors`, `published_at`, `modified_at`, `image`, `video`, `favicon_url`, `icons`, `logo_url`, `feeds`, `keywords`, `theme_color`, `generator`, `manifest_url`, `robots`, `open_graph`, `twitter`, `alternates`, `oembed_url`, `structured_data_types`. `main_image_url`, `author_name`, `publication_date`, `feed_url` and `additional_author_information` are gone: read `image["url"]`, `authors`, `published_at` and `feeds`.
+
 ## 0.1.1
 
 - The gem's author is listed as URLpipe.

data/README.md

@@ -1,6 +1,6 @@
 # urlpipe
 
-Turn any URL into clean Markdown, rendered HTML, a full-page screenshot, metadata, a summary, keywords, console errors or a Lighthouse audit, from Ruby. Pages are rendered in real Chrome, so JavaScript-heavy sites come back complete.
+Turn any URL into clean Markdown, rendered HTML, a full-page screenshot, a PDF, metadata, a summary, keywords, data in a schema you define, console errors, a Lighthouse audit or the technologies it is built with, from Ruby. Pages are rendered in real Chrome, so JavaScript-heavy sites come back complete.
 
 This is the official Ruby client for [URLpipe](https://urlpipe.dev). It has no runtime dependencies and runs on Ruby 3.1 and later.
 
@@ -37,7 +37,8 @@ Urlpipe::Client.new(
   timeout: 90,           # seconds per HTTP request; the API holds a sync request up to 60 s
   max_retries: 2,        # see "Retries and idempotency"
   wait_timeout: 300,     # how long a long analysis is polled before Urlpipe::WaitTimeoutError
-  poll_interval: 2       # seconds between polls of GET /result/:token
+  poll_interval: 2,      # seconds between polls of GET /result/:token
+  user_agent: nil        # your program's product token, e.g. "my-app/1.2"; it goes in front of the client's own
 )
 ```
 
@@ -54,7 +55,7 @@ Every method takes the URL first. These options work on all of them and are sent
 | `labels:` | Your own ids for the request, e.g. `{ client: "acme" }`. Returned with the result and the webhook. |
 | `residential:` | `true` fetches the page from a home broadband address. |
 | `report_to:` | A webhook URL the result is POSTed to (async calls). |
-| `page_options:` | `wait_for_selector`, `delay`, `block_ads`, `block_cookie_banners`, `remove_selectors`. Not on `lighthouse`. |
+| `page_options:` | `wait_for_selector`, `delay`, `block_ads`, `block_cookie_banners`, `block_popups`, `remove_selectors`. Not on `lighthouse`. |
 | `idempotency_key:` | Sent as the `Idempotency-Key` header. |
 | `extra:` | A Hash merged into the request body, for API options this version doesn't name yet. |
 
@@ -82,12 +83,25 @@ shot.mime_type  # => "image/webp"
 shot.result_url # => "https://..." - put it straight in an <img> tag
 ```
 
+### pdf
+
+`data` is a `Urlpipe::Pdf`: the whole rendered page as decoded PDF bytes, and `result_url`, a link to the PDF that needs no API key. `pdf_options:` choose `paper_size`, `landscape`, `margin`, `scale`, `print_background`, `media` and `single_page`.
+
+```ruby
+pdf = client.pdf("https://example.com/invoice", pdf_options: { paper_size: "letter", landscape: true }).data
+pdf.save("invoice.pdf")
+pdf.result_url # => "https://..."
+```
+
 ### meta
 
-`data` is a Hash with `title`, `description`, `language`, `main_image_url`, `favicon_url`, `author_name`, `feed_url`, `publication_date` and `additional_author_information`. Any of them can be `nil`.
+`data` is a Hash with the same keys for every page: `final_url` (where the page ended up, after its redirects; relative URLs are resolved against it), `title`, `description`, `site_name`, `type`, `language`, `locale`, `authors`, `published_at`, `modified_at`, `image`, `video`, `favicon_url`, `icons`, `logo_url`, `feeds`, `keywords`, `canonical_url`, `robots`, `alternates`, `oembed_url`, `structured_data_types`, and the raw `open_graph` and `twitter` tags. A value the page doesn't declare is `nil` or an empty Array. URLs are absolute, dates ISO 8601.
 
 ```ruby
-client.meta("https://example.com").data["title"] # => "Example Domain"
+page = client.meta("https://example.com").data
+page["title"]                   # => "Example Domain"
+page.dig("image", "url")        # => "https://example.com/cover.jpg", or nil
+page["authors"].map { _1["name"] }
 ```
 
 ### keywords
@@ -98,6 +112,24 @@ client.meta("https://example.com").data["title"] # => "Example Domain"
 client.keywords("https://example.com").data # => ["example domain", "documentation", ...]
 ```
 
+### extract
+
+Data in a schema you define. Describe the fields you want and `data` is a Hash with exactly those keys, in that order, `nil` where the page gives no value. A field's `type` is `string`, `number`, `integer`, `boolean`, `date`, `datetime`, `url`, `email`, `object` (with its own `fields`) or `array` (with `items`).
+
+```ruby
+product = client.extract(
+  "https://example.com/products/trail-shoe",
+  [
+    { name: "name", type: "string" },
+    { name: "price", type: "number", description: "The current price, without the currency" },
+    { name: "in_stock", type: "boolean" },
+    { name: "sizes", type: "array", items: "string" }
+  ],
+  instructions: "Ignore the related products."
+).data
+product["price"] # => 89.5
+```
+
 ### console
 
 `data` is an Array of `{"type" => "error" | "warning" | "exception", "text" => "..."}`.
@@ -106,6 +138,15 @@ client.keywords("https://example.com").data # => ["example domain", "documentati
 client.console("https://example.com").data.select { |entry| entry["type"] == "exception" }
 ```
 
+### technologies
+
+`data` is a Hash: `"final_url"` and `"technologies"`, an Array of what the page is built with, most certain first.
+
+```ruby
+client.technologies("https://example.com").data["technologies"].map { |tech| [tech["name"], tech["version"]] }
+# => [["WordPress", "6.4.2"], ["PHP", nil], ...]
+```
+
 ### lighthouse
 
 `data` is the audit as a Hash. `device:` is `"mobile"` (the default) or `"desktop"`; `include_audits: true` adds the full audits object.
@@ -117,7 +158,7 @@ report.dig("categories", "performance", "score")
 
 ### scrape
 
-Several operations off one page visit, so you get them all much sooner. `data` is `{"url" => ..., "operations" => {"markdown" => {"success", "result", "error", "cached"}, ...}}`. Each operation succeeds or fails on its own. A screenshot inside a scrape stays Base64, as the API returns it.
+Several operations off one page visit, so you get them all much sooner. `data` is `{"url" => ..., "operations" => {"markdown" => {"success", "result", "error", "cached"}, ...}}`. Each operation succeeds or fails on its own. A screenshot or a PDF inside a scrape stays Base64, as the API returns it. `scrape` also takes `screenshot_options:`, `pdf_options:`, and `fields:` and `instructions:` for its `extract` operation.
 
 ```ruby
 page = client.scrape("https://example.com", %w[markdown meta screenshot],
@@ -135,7 +176,7 @@ Every method returns a `Urlpipe::Response`:
 | `data` | The result when completed, `nil` otherwise. |
 | `token` | The request's result token. Fetching the result again with it is free for 30 days. |
 | `labels` | The labels the request was made with; `{}` when it had none. |
-| `meta` | `cache` (`"hit"`, `"miss"`, `"partial"`), `cache_age`, `processing_time_ms`, `quota` (`cost`, `limit`, `remaining`, `overage`, `resets_at`), `concurrency_limit`, `result_url`, `idempotent_replayed`. |
+| `meta` | `cache` (`"hit"`, `"miss"`, `"partial"`), `cache_age`, `processing_time_ms`, `quota` (`cost`, `limit`, `remaining`, `overage`, `resets_at`), `concurrency_limit`, `result_url`, `final_url` (where the page ended up, after its redirects), `idempotent_replayed`. |
 
 A value the API didn't send is `nil`; `idempotent_replayed` is always `true` or `false`. On an unlimited plan `quota.limit`, `quota.remaining` and `concurrency_limit` are the String `"unlimited"`.
 
@@ -156,7 +197,7 @@ report = client.wait(accepted.token, operation: "lighthouse").data
 
 `wait(token, operation: nil, timeout: wait_timeout, interval: poll_interval)` polls until the result is ready. `result(token, operation: nil)` checks once and returns a `"processing"` Response while it is still running.
 
-The API doesn't say which operation produced a token, so pass `operation:` to get `data` typed the way that method returns it. Without it JSON comes back parsed and text stays a String, which means a screenshot stays Base64: `client.wait(token, operation: "screenshot")` decodes it into a `Urlpipe::Screenshot`.
+The API doesn't say which operation produced a token, so pass `operation:` to get `data` typed the way that method returns it. Without it JSON comes back parsed and text stays a String, which means a screenshot or a PDF stays Base64: `client.wait(token, operation: "screenshot")` decodes it into a `Urlpipe::Screenshot`, and `operation: "pdf"` into a `Urlpipe::Pdf`.
 
 ## Webhooks
 
@@ -176,7 +217,7 @@ class UrlpipeWebhooksController < ActionController::API
 end
 ```
 
-`verify(raw_body, headers, secret, tolerance: 300)` returns the payload Hash (`token`, `operation`, `labels`, `success`, `result`, `result_url`, `error`, `meta`), or raises `Urlpipe::WebhookVerificationError` saying why. It accepts any signature in the header that matches, so deliveries keep verifying through a secret rotation, and it refuses a timestamp more than `tolerance` seconds from now.
+`verify(raw_body, headers, secret, tolerance: 300)` returns the payload Hash (`token`, `operation`, `labels`, `success`, `result`, `result_url`, `final_url`, `error`, `meta`), or raises `Urlpipe::WebhookVerificationError` saying why. It accepts any signature in the header that matches, so deliveries keep verifying through a secret rotation, and it refuses a timestamp more than `tolerance` seconds from now.
 
 Pass the **raw** request body, exactly the bytes received: `request.raw_post` in Rails, `request.body.read` in Rack. Parsed and re-serialized JSON has different bytes and will not verify. `headers` can be Rails' `request.headers`, a Rack env or a plain Hash.
 

data/lib/urlpipe/client.rb

@@ -18,7 +18,7 @@ module Urlpipe
     USER_AGENT = "urlpipe-ruby/#{VERSION}".freeze
 
     TEXT_OPERATIONS = %w[markdown html summarize].freeze
-    JSON_OPERATIONS = %w[meta keywords console lighthouse scrape].freeze
+    JSON_OPERATIONS = %w[meta keywords extract console technologies lighthouse scrape].freeze
 
     RETRYABLE_STATUSES = [500, 502, 503].freeze
     MAX_RETRY_DELAY = 60
@@ -45,8 +45,10 @@ module Urlpipe
     # - +wait_timeout+: how long a sync call or +wait+ keeps polling a long
     #   analysis before raising WaitTimeoutError
     # - +poll_interval+: seconds between polls of GET /result/:token
+    # - +user_agent+: your program's product token, such as "my-app/1.2". It
+    #   goes in front of the client's own, so requests identify both
     def initialize(api_key: nil, base_url: DEFAULT_BASE_URL, timeout: 90, max_retries: 2,
-                   wait_timeout: 300, poll_interval: 2)
+                   wait_timeout: 300, poll_interval: 2, user_agent: nil)
       api_key = ENV.fetch("URLPIPE_API_KEY", nil) if api_key.nil? || api_key.to_s.strip.empty?
       if api_key.nil? || api_key.strip.empty?
         raise ConfigurationError,
@@ -61,6 +63,7 @@ module Urlpipe
       @max_retries = Integer(max_retries)
       @wait_timeout = wait_timeout
       @poll_interval = poll_interval
+      @user_agent = [user_agent.to_s.strip, USER_AGENT].reject(&:empty?).join(" ")
     end
 
     def inspect
@@ -98,9 +101,22 @@ module Urlpipe
               idempotency_key, extra)
     end
 
-    # Page metadata. +data+ is a Hash: title, description, language,
-    # main_image_url, favicon_url, author_name, feed_url, publication_date,
-    # additional_author_information.
+    # The whole rendered page as a PDF. +data+ is a Pdf (decoded bytes,
+    # +mime_type+, +result_url+). +pdf_options+ choose the paper, orientation,
+    # margins, scale and stylesheet.
+    def pdf(url, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
+            page_options: nil, pdf_options: nil, idempotency_key: nil, extra: {})
+      analyse("pdf", url,
+              { sync:, max_age:, labels:, residential:, report_to:, page_options:, pdf_options: },
+              idempotency_key, extra)
+    end
+
+    # Page metadata. +data+ is a Hash with the same keys for every page:
+    # final_url (where the page ended up, after its redirects), canonical_url,
+    # title, description, site_name, type, language, locale,
+    # authors, published_at, modified_at, image, video, favicon_url, icons,
+    # logo_url, feeds, keywords, theme_color, generator, manifest_url, robots,
+    # open_graph, twitter, alternates, oembed_url, structured_data_types.
     def meta(url, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
              page_options: nil, idempotency_key: nil, extra: {})
       analyse("meta", url, { sync:, max_age:, labels:, residential:, report_to:, page_options: },
@@ -114,6 +130,18 @@ module Urlpipe
               idempotency_key, extra)
     end
 
+    # Data from the page in the schema +fields+ describes: an Array of Hashes
+    # with :name, :type and optionally :description, :fields (an object's) and
+    # :items (an array's). +data+ is a Hash with exactly those names as keys,
+    # in order, nil where the page gives no value. +instructions+ adds
+    # guidance in plain words.
+    def extract(url, fields, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
+                page_options: nil, instructions: nil, idempotency_key: nil, extra: {})
+      analyse("extract", url,
+              { fields:, instructions:, sync:, max_age:, labels:, residential:, report_to:, page_options: },
+              idempotency_key, extra)
+    end
+
     # Console errors, warnings and uncaught exceptions. +data+ is an Array of
     # Hashes with "type" ("error", "warning" or "exception") and "text".
     def console(url, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
@@ -122,6 +150,15 @@ module Urlpipe
               idempotency_key, extra)
     end
 
+    # What the page is built with. +data+ is a Hash with "final_url" and
+    # "technologies", an Array of Hashes with "name", "version", "confidence",
+    # "categories", "detected_by" and the rest, most certain first.
+    def technologies(url, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
+                     page_options: nil, idempotency_key: nil, extra: {})
+      analyse("technologies", url, { sync:, max_age:, labels:, residential:, report_to:, page_options: },
+              idempotency_key, extra)
+    end
+
     # A Lighthouse audit. +data+ is a Hash. +device+ is "mobile" (the API's
     # default) or "desktop".
     def lighthouse(url, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
@@ -132,14 +169,15 @@ module Urlpipe
 
     # Several operations off one page visit. +operations+ is an Array such as
     # ["markdown", "meta"]. +data+ is a Hash:
-    # {"url" => ..., "operations" => {"markdown" => {"success", "result", "error", "cached"}}}.
-    # A screenshot inside a scrape stays Base64, as the API returns it.
+    # {"url" => ..., "final_url" => ..., "operations" => {"markdown" => {"success", "result", "error", "cached"}}}.
+    # A screenshot or a PDF inside a scrape stays Base64, as the API returns
+    # it. +fields+ and +instructions+ are for its "extract" operation.
     def scrape(url, operations, sync: true, max_age: nil, labels: nil, residential: nil, report_to: nil,
                page_options: nil, device: nil, include_audits: nil, screenshot_options: nil,
-               idempotency_key: nil, extra: {})
+               pdf_options: nil, fields: nil, instructions: nil, idempotency_key: nil, extra: {})
       analyse("scrape", url,
               { operations:, sync:, max_age:, labels:, residential:, report_to:, page_options:,
-                device:, include_audits:, screenshot_options: },
+                device:, include_audits:, screenshot_options:, pdf_options:, fields:, instructions: },
               idempotency_key, extra)
     end
 
@@ -148,8 +186,9 @@ module Urlpipe
     #
     # +operation+ names the operation that produced the token, so +data+ is
     # typed as that operation returns it. Without it, JSON stays parsed JSON
-    # and text stays a String (a screenshot stays Base64: pass
-    # <tt>operation: "screenshot"</tt> to get a Screenshot).
+    # and text stays a String (a screenshot or a PDF stays Base64: pass
+    # <tt>operation: "screenshot"</tt> or <tt>"pdf"</tt> to get a Screenshot
+    # or a Pdf).
     def result(token, operation: nil)
       raw = execute(:get, "/result/#{URI.encode_www_form_component(token)}")
 
@@ -234,6 +273,8 @@ module Urlpipe
         raw.text
       elsif operation == "screenshot"
         Screenshot.from_base64(raw.text, result_url: raw.headers["x-result-url"])
+      elsif operation == "pdf"
+        Pdf.from_base64(raw.text, result_url: raw.headers["x-result-url"])
       elsif JSON_OPERATIONS.include?(operation) || (operation.nil? && raw.json?)
         parse_result_json(raw)
       else
@@ -316,7 +357,7 @@ module Urlpipe
 
       request = method == :post ? Net::HTTP::Post.new(uri) : Net::HTTP::Get.new(uri)
       request["Authorization"] = "Bearer #{@api_key}"
-      request["User-Agent"] = USER_AGENT
+      request["User-Agent"] = @user_agent
       request["Accept"] = "*/*"
       request["Idempotency-Key"] = idempotency_key if idempotency_key
       if body

data/lib/urlpipe/pdf.rb

@@ -0,0 +1,36 @@
+# frozen_string_literal: true
+
+module Urlpipe
+  # A decoded PDF of the page: +data+ is the PDF as a binary String,
+  # +mime_type+ is "application/pdf", and +result_url+ is a link to the same
+  # PDF that needs no API key.
+  class Pdf
+    MIME_TYPE = "application/pdf"
+
+    attr_reader :data, :result_url
+
+    # Builds a Pdf from the API's Base64 body.
+    def self.from_base64(text, result_url: nil)
+      new(text.to_s.unpack1("m"), result_url: result_url)
+    end
+
+    def initialize(data, result_url: nil)
+      @data = data.b
+      @result_url = result_url
+    end
+
+    def mime_type = MIME_TYPE
+
+    # Writes the PDF to +path+ and returns the path.
+    def save(path)
+      File.binwrite(path, data)
+      path
+    end
+
+    def bytesize = data.bytesize
+
+    def inspect
+      "#<#{self.class.name} #{bytesize} bytes result_url=#{result_url.inspect}>"
+    end
+  end
+end

data/lib/urlpipe/response.rb

@@ -16,11 +16,12 @@ module Urlpipe
   # - +quota+: a Quota
   # - +concurrency_limit+: Integer or "unlimited"
   # - +result_url+: screenshots only, a link to the image that needs no key
+  # - +final_url+: where the page ended up, after its redirects
   # - +idempotent_replayed+: +true+ when this answers an earlier request with
   #   the same Idempotency-Key, +false+ otherwise
   Meta = Struct.new(
     :cache, :cache_age, :processing_time_ms, :quota, :concurrency_limit,
-    :result_url, :idempotent_replayed,
+    :result_url, :final_url, :idempotent_replayed,
     keyword_init: true
   ) do
     # +headers+ is a Hash of lowercase header names to values.
@@ -38,6 +39,7 @@ module Urlpipe
         ),
         concurrency_limit: integer_or_given(headers["x-concurrency-limit"]),
         result_url: headers["x-result-url"],
+        final_url: headers["x-final-url"],
         idempotent_replayed: headers["idempotent-replayed"].to_s.strip.casecmp?("true")
       )
     end

data/lib/urlpipe/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Urlpipe
-  VERSION = "0.1.1"
+  VERSION = "0.2.0"
 end

data/lib/urlpipe/webhook.rb

@@ -22,7 +22,8 @@ module Urlpipe
 
     # Checks the delivery's signature and freshness and returns the parsed
     # payload, a Hash with "token", "operation", "labels", "success",
-    # "result", "result_url", "error" and "meta".
+    # "result", "result_url", "final_url" (where the page ended up, after its
+    # redirects), "error" and "meta".
     #
     # - +raw_body+: the request body as received (a String, or an IO)
     # - +headers+: the request headers. Accepts a plain Hash, Rails'

data/lib/urlpipe.rb

@@ -4,12 +4,14 @@ require_relative "urlpipe/version"
 require_relative "urlpipe/errors"
 require_relative "urlpipe/response"
 require_relative "urlpipe/screenshot"
+require_relative "urlpipe/pdf"
 require_relative "urlpipe/client"
 require_relative "urlpipe/webhook"
 
 # Ruby client for the URLpipe API (https://urlpipe.dev): turn a URL into
-# Markdown, rendered HTML, a screenshot, metadata, a summary, keywords,
-# console errors or a Lighthouse audit.
+# Markdown, rendered HTML, a screenshot, a PDF, metadata, a summary, keywords,
+# data in a schema you define, console errors, a Lighthouse audit or the
+# technologies a site is built with.
 #
 #   client = Urlpipe::Client.new # reads URLPIPE_API_KEY
 #   client.markdown("https://example.com").data

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: urlpipe
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.0
 platform: ruby
 authors:
 - URLpipe
@@ -10,9 +10,9 @@ cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies: []
 description: Official Ruby client for URLpipe. Get a page's Markdown, rendered HTML,
-  a full-page screenshot, metadata, a summary, keywords, console errors or a Lighthouse
-  audit, rendered in real Chrome. Handles long analyses, retries with idempotency
-  keys, and webhook signature verification. No runtime dependencies.
+  a full-page screenshot, a PDF, metadata, a summary, keywords, structured data, console
+  errors or a Lighthouse audit, rendered in real Chrome. Handles long analyses, retries
+  with idempotency keys, and webhook signature verification. No runtime dependencies.
 email:
 - contact@urlpipe.dev
 executables: []
@@ -25,6 +25,7 @@ files:
 - lib/urlpipe.rb
 - lib/urlpipe/client.rb
 - lib/urlpipe/errors.rb
+- lib/urlpipe/pdf.rb
 - lib/urlpipe/response.rb
 - lib/urlpipe/screenshot.rb
 - lib/urlpipe/version.rb