capturekitdev 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/LICENSE +21 -0
- data/README.md +154 -0
- data/examples/README.md +12 -0
- data/examples/binary.rb +12 -0
- data/examples/json.rb +8 -0
- data/examples/request.rb +11 -0
- data/examples/wait_for_job.rb +8 -0
- data/lib/capturekit/client.rb +266 -0
- data/lib/capturekit/endpoints.rb +58 -0
- data/lib/capturekit/errors.rb +56 -0
- data/lib/capturekit/operations.rb +147 -0
- data/lib/capturekit/responses.rb +45 -0
- data/lib/capturekit/serialization.rb +66 -0
- data/lib/capturekit/transport.rb +27 -0
- data/lib/capturekit/version.rb +5 -0
- data/lib/capturekit.rb +14 -0
- data/lib/capturekitdev.rb +4 -0
- data/sig/capturekit.rbs +85 -0
- metadata +67 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 36bf43c1c8df4739b21e44aa3ad4b8ed8e98b59f63e830010afe113548c29008
|
|
4
|
+
data.tar.gz: 89e7d371a68f2c7128dfdff9e835edd38fce839b842b2f843cfb316a257d95d5
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: cd6f9777e9d60e5775dff60fcf64c4b8b78ab4eb8c19bcb0bcceccea42eee41f095339a3f3843a98f0b34d1f18ce2c85997f57cf93d973e18a01ae245da742a0
|
|
7
|
+
data.tar.gz: 26993322f333bde5b1fc19bd150730af26198ea360c8c696325b83b49ef7542e3a011ca8479cfbf5b7cfc1e1f3a2713f1ebf8bfe82e6d9a1c98b4eed2940ec8c
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 CaptureKit
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
# CaptureKit Ruby SDK
|
|
2
|
+
|
|
3
|
+
The official Ruby SDK for the [CaptureKit](https://www.capturekit.dev) screenshot API.
|
|
4
|
+
|
|
5
|
+
Capture websites as PNG, JPEG, WebP or PDF, extract their content, or analyse them with AI.
|
|
6
|
+
|
|
7
|
+
The SDK exposes **7 methods**, one per documented endpoint, plus a generic `request`. It only uses the Ruby standard library.
|
|
8
|
+
|
|
9
|
+
## Requirements
|
|
10
|
+
|
|
11
|
+
- Ruby 3.1 or newer
|
|
12
|
+
|
|
13
|
+
## Installation
|
|
14
|
+
|
|
15
|
+
```sh
|
|
16
|
+
gem install capturekitdev
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Or in your `Gemfile`:
|
|
20
|
+
|
|
21
|
+
```ruby
|
|
22
|
+
gem "capturekitdev"
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
## Get your API key
|
|
26
|
+
|
|
27
|
+
[Sign up for CaptureKit](https://app.capturekit.dev/register) or sign in, then copy your API key from the [dashboard](https://app.capturekit.dev). The client reads `ENV["CAPTUREKIT_API_KEY"]` when no key is passed.
|
|
28
|
+
|
|
29
|
+
## Quick start
|
|
30
|
+
|
|
31
|
+
```ruby
|
|
32
|
+
require "capturekit"
|
|
33
|
+
|
|
34
|
+
client = CaptureKit::Client.new(api_key: ENV.fetch("CAPTUREKIT_API_KEY"))
|
|
35
|
+
response = client.content(url: "https://example.com")
|
|
36
|
+
puts response.data
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Keyword arguments use the parameter names of the [API reference](https://app.capturekit.dev/openapi). Unknown names raise `CaptureKit::SerializationError`; use `request` for undocumented parameters.
|
|
40
|
+
|
|
41
|
+
## Binary responses
|
|
42
|
+
|
|
43
|
+
Endpoints that can return a file (image, PDF…) return a `JsonResponse` or a `BinaryResponse`:
|
|
44
|
+
|
|
45
|
+
```ruby
|
|
46
|
+
response = client.capture(url: "https://example.com", format: "png")
|
|
47
|
+
response.save("screenshot.png") if response.binary?
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
## Async jobs
|
|
51
|
+
|
|
52
|
+
Any call accepts `is_async: true` (useful past the 60-second server limit): the response is then a `job_id`. `wait_for_job` polls `GET /v1/jobs/{id}` until a terminal status (`CompletedJob`, `Error`, `NotFound`, `BadRequest`):
|
|
53
|
+
|
|
54
|
+
```ruby
|
|
55
|
+
submitted = client.content(url: "https://example.com", is_async: true)
|
|
56
|
+
job = client.wait_for_job(submitted.data["job_id"]) { |j| puts "#{j['status']} #{j['progress']}%" }
|
|
57
|
+
puts job.data
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
## GET or POST
|
|
61
|
+
|
|
62
|
+
Methods use GET when the endpoint supports it (query string). POST sends the same parameters as JSON:
|
|
63
|
+
|
|
64
|
+
```ruby
|
|
65
|
+
client.analyze(url: "test", request_options: { method: "POST" })
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
`nil` values are omitted, `false` and `0` are sent, and arrays repeat the parameter name.
|
|
69
|
+
|
|
70
|
+
## Any endpoint
|
|
71
|
+
|
|
72
|
+
```ruby
|
|
73
|
+
response = client.request("GET", "/v1/usage")
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
## Errors
|
|
77
|
+
|
|
78
|
+
```ruby
|
|
79
|
+
begin
|
|
80
|
+
client.content(url: "https://example.com")
|
|
81
|
+
rescue CaptureKit::APIError => e
|
|
82
|
+
warn "HTTP #{e.status}: #{e.message}"
|
|
83
|
+
end
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
| Error | When |
|
|
87
|
+
| --- | --- |
|
|
88
|
+
| `CaptureKit::ConfigurationError` | Missing API key, invalid options or unsupported method. |
|
|
89
|
+
| `CaptureKit::SerializationError` | A required parameter is missing, unknown or can't be sent. |
|
|
90
|
+
| `CaptureKit::TransportError` | Network failure or timeout. |
|
|
91
|
+
| `CaptureKit::APIError` | Non-2xx status (`status`, `error_message`, `response_body`, `raw_response`). |
|
|
92
|
+
| `CaptureKit::ResponseDecodingError` | A success response could not be decoded. |
|
|
93
|
+
|
|
94
|
+
All of them inherit from `CaptureKit::Error`.
|
|
95
|
+
|
|
96
|
+
## Retries
|
|
97
|
+
|
|
98
|
+
Requests are retried up to 3 times after a network error, a `408`, `500`, `502`, `503` or `504`, or a rate limit — which the API returns as `401` (or `429`) with a `Rate limit exceeded` message. Per-key quota errors (`…: quota daily`) are not retried, nor are `400`, `402`, `404` or client-side timeouts. The delay starts at 1 second and grows 1.5× per retry (capped at 30 seconds); a `Retry-After` header wins.
|
|
99
|
+
|
|
100
|
+
```ruby
|
|
101
|
+
client = CaptureKit::Client.new(max_retries: 5, retry_delay: 0.5)
|
|
102
|
+
client.content(url: "https://example.com", request_options: { max_retries: 0 }) # one request without retries
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Batch requests
|
|
106
|
+
|
|
107
|
+
`batch` runs the block for every item on 10 threads (`concurrency:`) and returns one `BatchResult` per item, in input order. A failing item never stops the others.
|
|
108
|
+
|
|
109
|
+
```ruby
|
|
110
|
+
inputs = ["https://example.com", "https://www.capturekit.dev", "https://github.com"]
|
|
111
|
+
results = client.batch(inputs, concurrency: 5) { |value| client.content(url: value) }
|
|
112
|
+
results.each do |result|
|
|
113
|
+
if result.ok?
|
|
114
|
+
puts result.value.data
|
|
115
|
+
else
|
|
116
|
+
warn "#{inputs[result.index]}: #{result.error.message}"
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
| Constant | Default | Description |
|
|
122
|
+
| --- | --- | --- |
|
|
123
|
+
| `CaptureKit::Client::DEFAULT_CONCURRENCY` | `10` | Tasks run at the same time by batch |
|
|
124
|
+
| `CaptureKit::Client::DEFAULT_TIMEOUT` | `120` | Request timeout (seconds) |
|
|
125
|
+
| `CaptureKit::Client::MAX_RETRIES` | `3` | Retry attempts after a network error, a 408/5xx or a rate limit |
|
|
126
|
+
| `CaptureKit::Client::RETRY_BACKOFF_FACTOR` | `1.5` | Exponential backoff multiplier |
|
|
127
|
+
| `CaptureKit::Client::DEFAULT_RETRY_DELAY` | `1.0` | Delay before the first retry (seconds) |
|
|
128
|
+
|
|
129
|
+
## Timeouts and custom transports
|
|
130
|
+
|
|
131
|
+
```ruby
|
|
132
|
+
client = CaptureKit::Client.new(timeout: 60) # seconds, default 120
|
|
133
|
+
client.content(url: "https://example.com", request_options: { timeout: 30 })
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
A transport is any object with `call(method:, url:, headers:, body:, timeout:)` returning `[status, headers, body]`; the default is `CaptureKit::NetHTTPTransport`.
|
|
137
|
+
|
|
138
|
+
## Endpoints
|
|
139
|
+
|
|
140
|
+
7 methods, generated from the [OpenAPI document](https://app.capturekit.dev/openapi).
|
|
141
|
+
|
|
142
|
+
| Method | HTTP | Description |
|
|
143
|
+
| --- | --- | --- |
|
|
144
|
+
| `analyze` | GET / POST `/v1/analyze` | AI Content API analyzes any webpage and returns AI insights summary, value proposition, features, categories, use cases, and optional conta… |
|
|
145
|
+
| `capture` | GET / POST `/v1/capture` | Browser-like Screenshot API that captures webpages as PNG/JPEG/WebP or PDF for previews, QA, reporting, monitoring, and archiving no headle… |
|
|
146
|
+
| `content` | GET / POST `/v1/content` | CaptureKit Content API extracts structured webpage data, metadata, links, HTML/Markdown: in one request for SEO, monitoring, indexing and a… |
|
|
147
|
+
| `jobs` | GET `/v1/jobs/{id}` | Poll the status, progress (0–100) and result of a job started with is_async=true. Polling is free. Poll until status is terminal: Completed… |
|
|
148
|
+
| `stats` | GET `/v1/stats` | Infrastructure health for an engine or endpoint over the last hour. Not scoped to your account or tenant. Enabled per account: HTTP 403 mea… |
|
|
149
|
+
| `usage` | GET `/v1/usage` | Retrieve real-time usage statistics and credit balance for your CaptureKit account. |
|
|
150
|
+
| `website_crawler` | GET / POST `/v1/website/crawler` | Website Crawler API fetches raw HTML from any webpage in request mode (no rendering) for fast scraping, audits, monitoring, and link/metada… |
|
|
151
|
+
|
|
152
|
+
## License
|
|
153
|
+
|
|
154
|
+
MIT — see [LICENSE](LICENSE).
|
data/examples/README.md
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Examples
|
|
2
|
+
|
|
3
|
+
```sh
|
|
4
|
+
gem install capturekitdev
|
|
5
|
+
export CAPTUREKIT_API_KEY="YOUR_API_KEY"
|
|
6
|
+
ruby examples/json.rb
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
- `json.rb` — call the featured endpoint and print the JSON result.
|
|
10
|
+
- `binary.rb` — save a binary response to a file.
|
|
11
|
+
- `request.rb` — call any path with `request` and handle API errors.
|
|
12
|
+
- `wait_for_job.rb` — start an async job and wait for its result.
|
data/examples/binary.rb
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "capturekit"
|
|
4
|
+
|
|
5
|
+
client = CaptureKit::Client.new # reads CAPTUREKIT_API_KEY
|
|
6
|
+
response = client.capture(url: "https://example.com", format: "png")
|
|
7
|
+
if response.binary?
|
|
8
|
+
response.save("screenshot.png")
|
|
9
|
+
puts "Saved screenshot.png (#{response.content_type}, #{response.bytes.bytesize} bytes)"
|
|
10
|
+
else
|
|
11
|
+
puts response.data
|
|
12
|
+
end
|
data/examples/json.rb
ADDED
data/examples/request.rb
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "capturekit"
|
|
4
|
+
|
|
5
|
+
client = CaptureKit::Client.new # reads CAPTUREKIT_API_KEY
|
|
6
|
+
begin
|
|
7
|
+
response = client.request("GET", "/v1/usage")
|
|
8
|
+
puts response.data if response.json?
|
|
9
|
+
rescue CaptureKit::APIError => e
|
|
10
|
+
warn "HTTP #{e.status}: #{e.message}"
|
|
11
|
+
end
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "capturekit"
|
|
4
|
+
|
|
5
|
+
client = CaptureKit::Client.new # reads CAPTUREKIT_API_KEY
|
|
6
|
+
submitted = client.content(url: "https://example.com", is_async: true)
|
|
7
|
+
job = client.wait_for_job(submitted.data["job_id"]) { |j| puts "#{j['status']} #{j['progress']}%" }
|
|
8
|
+
puts job.data
|
|
@@ -0,0 +1,266 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module CaptureKit
|
|
6
|
+
# Official CaptureKit API client. Create one instance and reuse it.
|
|
7
|
+
class Client
|
|
8
|
+
include Operations
|
|
9
|
+
|
|
10
|
+
DEFAULT_BASE_URL = "https://api.capturekit.dev"
|
|
11
|
+
DEFAULT_TIMEOUT = 120
|
|
12
|
+
# Tasks run at the same time by #batch.
|
|
13
|
+
DEFAULT_CONCURRENCY = 10
|
|
14
|
+
# Retries after a network error or a 408, 429, 500, 502, 503 or 504 response.
|
|
15
|
+
MAX_RETRIES = 3
|
|
16
|
+
# Each retry waits this many times longer than the previous one (a Retry-After header wins).
|
|
17
|
+
RETRY_BACKOFF_FACTOR = 1.5
|
|
18
|
+
# Seconds before the first retry.
|
|
19
|
+
DEFAULT_RETRY_DELAY = 1.0
|
|
20
|
+
MAX_RETRY_DELAY = 30.0
|
|
21
|
+
RETRYABLE_STATUSES = [408, 500, 502, 503, 504].freeze
|
|
22
|
+
API_KEY_ENVIRONMENT_VARIABLE = "CAPTUREKIT_API_KEY"
|
|
23
|
+
JOB_TERMINAL_STATUSES = %w[CompletedJob Error NotFound BadRequest].freeze
|
|
24
|
+
ERROR_MESSAGE_KEYS = %w[error message error_message detail].freeze
|
|
25
|
+
|
|
26
|
+
# @param api_key [String, nil] defaults to ENV["CAPTUREKIT_API_KEY"]
|
|
27
|
+
# @param base_url [String] API origin
|
|
28
|
+
# @param timeout [Numeric, nil] seconds per request; nil disables it
|
|
29
|
+
# @param headers [Hash] extra headers for every request
|
|
30
|
+
# @param max_retries [Integer] retries after a network error or a 408/429/5xx response; 0 disables them
|
|
31
|
+
# @param retry_delay [Numeric] seconds before the first retry, growing 1.5x per retry
|
|
32
|
+
# @param transport [#call] custom transport, see {NetHTTPTransport}
|
|
33
|
+
def initialize(api_key: nil, base_url: DEFAULT_BASE_URL, timeout: DEFAULT_TIMEOUT, headers: {},
|
|
34
|
+
max_retries: MAX_RETRIES, retry_delay: DEFAULT_RETRY_DELAY, transport: nil)
|
|
35
|
+
api_key = ENV.fetch(API_KEY_ENVIRONMENT_VARIABLE, nil) if api_key.nil?
|
|
36
|
+
if !api_key.is_a?(String) || api_key.strip.empty?
|
|
37
|
+
raise ConfigurationError,
|
|
38
|
+
"A non-blank CaptureKit API key is required. Pass api_key: or set #{API_KEY_ENVIRONMENT_VARIABLE}."
|
|
39
|
+
end
|
|
40
|
+
raise ConfigurationError, "The CaptureKit API key cannot be used in an HTTP header." if api_key.match?(/[\r\n]/)
|
|
41
|
+
|
|
42
|
+
uri = URI.parse(base_url.to_s)
|
|
43
|
+
unless %w[http https].include?(uri.scheme) && uri.host
|
|
44
|
+
raise ConfigurationError, "base_url must be an absolute http or https URL."
|
|
45
|
+
end
|
|
46
|
+
raise ConfigurationError, "timeout must be a positive number or nil." if timeout && !(timeout.is_a?(Numeric) && timeout.positive?)
|
|
47
|
+
raise ConfigurationError, "transport must respond to #call." if transport && !transport.respond_to?(:call)
|
|
48
|
+
|
|
49
|
+
@api_key = api_key
|
|
50
|
+
@base_url = base_url.to_s.sub(%r{/+\z}, "")
|
|
51
|
+
@timeout = timeout
|
|
52
|
+
@headers = headers.to_h { |name, value| [name.to_s, value.to_s] }.freeze
|
|
53
|
+
raise ConfigurationError, "max_retries must be a non-negative Integer." unless max_retries.is_a?(Integer) && max_retries >= 0
|
|
54
|
+
raise ConfigurationError, "retry_delay must be a non-negative number." unless retry_delay.is_a?(Numeric) && retry_delay >= 0
|
|
55
|
+
|
|
56
|
+
@max_retries = max_retries
|
|
57
|
+
@retry_delay = retry_delay
|
|
58
|
+
@transport = transport || NetHTTPTransport.new
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def inspect
|
|
62
|
+
"#<#{self.class.name} base_url=#{@base_url.inspect}>"
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# Sends any request, including endpoints released after this SDK version.
|
|
66
|
+
# GET parameters travel in the query string; POST parameters travel in a JSON body.
|
|
67
|
+
# @return [JsonResponse, BinaryResponse]
|
|
68
|
+
def request(method, path, params = nil, timeout: nil, headers: nil, max_retries: nil)
|
|
69
|
+
send_request(method.to_s.upcase, path, params, { timeout: timeout, headers: headers, max_retries: max_retries }, :any)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Runs the block for every item on +concurrency+ threads (default 10) and returns one {BatchResult} per item,
|
|
73
|
+
# in input order. A failing item never stops the others; the client is thread-safe.
|
|
74
|
+
#
|
|
75
|
+
# results = client.batch(ids, concurrency: 5) { |id| client.some_endpoint(query: id) }
|
|
76
|
+
# @return [Array<BatchResult>]
|
|
77
|
+
def batch(items, concurrency: DEFAULT_CONCURRENCY)
|
|
78
|
+
raise ConfigurationError, "concurrency must be a positive Integer." unless concurrency.is_a?(Integer) && concurrency.positive?
|
|
79
|
+
raise ArgumentError, "batch needs a block" unless block_given?
|
|
80
|
+
|
|
81
|
+
list = items.to_a
|
|
82
|
+
results = Array.new(list.size)
|
|
83
|
+
queue = Queue.new
|
|
84
|
+
list.each_with_index { |item, index| queue << [item, index] }
|
|
85
|
+
queue.close
|
|
86
|
+
workers = Array.new([concurrency, list.size].min) do
|
|
87
|
+
Thread.new do
|
|
88
|
+
while (job = queue.pop)
|
|
89
|
+
item, index = job
|
|
90
|
+
results[index] = begin
|
|
91
|
+
BatchResult.new(index: index, value: yield(item), error: nil)
|
|
92
|
+
rescue StandardError => e
|
|
93
|
+
BatchResult.new(index: index, value: nil, error: e)
|
|
94
|
+
end
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
workers.each(&:join)
|
|
99
|
+
results
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# Polls GET /v1/jobs/{id} until the job reaches a terminal status. The delay starts at +interval+ seconds
|
|
103
|
+
# and grows 1.5x per poll up to 60 s; +timeout+ defaults to 12 hours.
|
|
104
|
+
# @yieldparam job [Hash] the latest job document after every poll
|
|
105
|
+
# @return [JsonResponse]
|
|
106
|
+
def wait_for_job(job_id, interval: 3, timeout: 43_200)
|
|
107
|
+
raise SerializationError.new("A non-blank job id is required.", "id") if job_id.to_s.strip.empty?
|
|
108
|
+
|
|
109
|
+
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
110
|
+
loop do
|
|
111
|
+
last = request("GET", "/v1/jobs/{id}", { "id" => job_id })
|
|
112
|
+
yield last.data if block_given?
|
|
113
|
+
return last if last.data.is_a?(Hash) && JOB_TERMINAL_STATUSES.include?(last.data["status"])
|
|
114
|
+
|
|
115
|
+
elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started
|
|
116
|
+
raise JobTimeoutError.new(job_id, last) if elapsed + interval > timeout
|
|
117
|
+
|
|
118
|
+
sleep(interval)
|
|
119
|
+
interval = (interval * 1.5).clamp(interval, [interval, 60].max)
|
|
120
|
+
end
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
private
|
|
124
|
+
|
|
125
|
+
def call_operation(name, params, options)
|
|
126
|
+
endpoint = ENDPOINTS.fetch(name)
|
|
127
|
+
options = (options || {}).transform_keys(&:to_sym)
|
|
128
|
+
method = (options[:method] || endpoint[:methods].first).to_s.upcase
|
|
129
|
+
unless endpoint[:methods].include?(method)
|
|
130
|
+
raise ConfigurationError, "#{endpoint[:path]} supports #{endpoint[:methods].join(', ')}, not #{method}."
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
params = params.transform_keys(&:to_s)
|
|
134
|
+
unknown = params.keys - endpoint[:params]
|
|
135
|
+
unless unknown.empty?
|
|
136
|
+
raise SerializationError.new(
|
|
137
|
+
"Unknown parameter(s) for #{name}: #{unknown.join(', ')}. Use #request for undocumented parameters.",
|
|
138
|
+
unknown.first
|
|
139
|
+
)
|
|
140
|
+
end
|
|
141
|
+
endpoint[:required].each do |required|
|
|
142
|
+
raise SerializationError.new("The \"#{required}\" parameter is required.", required) if params[required].nil?
|
|
143
|
+
end
|
|
144
|
+
send_request(method, endpoint[:path], params, options, endpoint[:response])
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
def send_request(method, path, params, options, expected)
|
|
148
|
+
raise ConfigurationError, "Unsupported HTTP method: #{method.inspect}." unless %w[GET POST].include?(method)
|
|
149
|
+
raise ConfigurationError, 'The request path must start with "/".' unless path.is_a?(String) && path.start_with?("/")
|
|
150
|
+
|
|
151
|
+
path, pairs, body = Serialization.serialize(method, path, params)
|
|
152
|
+
url = @base_url + path
|
|
153
|
+
url += "?#{Serialization.encode_query(pairs)}" unless pairs.empty?
|
|
154
|
+
headers = {
|
|
155
|
+
"Accept" => "application/json, */*;q=0.8",
|
|
156
|
+
"User-Agent" => "capturekit-ruby/#{VERSION}"
|
|
157
|
+
}.merge(@headers).merge((options[:headers] || {}).to_h { |k, v| [k.to_s, v.to_s] })
|
|
158
|
+
headers["x-api-key"] = @api_key
|
|
159
|
+
payload = nil
|
|
160
|
+
if body
|
|
161
|
+
headers["Content-Type"] = "application/json"
|
|
162
|
+
payload = JSON.generate(body)
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
max_retries = options[:max_retries] || @max_retries
|
|
166
|
+
attempt = 0
|
|
167
|
+
loop do
|
|
168
|
+
begin
|
|
169
|
+
status, response_headers, response_body = @transport.call(
|
|
170
|
+
method: method, url: url, headers: headers, body: payload, timeout: options[:timeout] || @timeout
|
|
171
|
+
)
|
|
172
|
+
rescue Timeout::Error => e
|
|
173
|
+
raise TransportError, "CaptureKit request timed out: #{e.message}"
|
|
174
|
+
rescue StandardError => e
|
|
175
|
+
raise TransportError, "CaptureKit request failed before an HTTP response was received: #{e.message}" if attempt >= max_retries
|
|
176
|
+
|
|
177
|
+
sleep(retry_delay(attempt, nil))
|
|
178
|
+
attempt += 1
|
|
179
|
+
next
|
|
180
|
+
end
|
|
181
|
+
raw = build_raw_response(status, response_headers, response_body)
|
|
182
|
+
return decode(raw, expected) if (200..299).cover?(raw.status)
|
|
183
|
+
raise api_error(raw) unless retryable?(raw) && attempt < max_retries
|
|
184
|
+
|
|
185
|
+
sleep(retry_delay(attempt, raw.headers["retry-after"]&.first))
|
|
186
|
+
attempt += 1
|
|
187
|
+
end
|
|
188
|
+
end
|
|
189
|
+
|
|
190
|
+
# Server errors and plan rate limits (401/429 "Rate limit exceeded") are retried; per-key quotas are not.
|
|
191
|
+
def retryable?(raw)
|
|
192
|
+
return true if RETRYABLE_STATUSES.include?(raw.status)
|
|
193
|
+
return false unless [401, 429].include?(raw.status)
|
|
194
|
+
|
|
195
|
+
message = api_error(raw).error_message.to_s.downcase
|
|
196
|
+
return false if message.include?("quota")
|
|
197
|
+
|
|
198
|
+
message.include?("rate limit") || (raw.status == 429 && message.empty?)
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
def retry_delay(attempt, retry_after)
|
|
202
|
+
seconds = Float(retry_after, exception: false) if retry_after
|
|
203
|
+
return [seconds, 60].min if seconds && seconds >= 0
|
|
204
|
+
|
|
205
|
+
[@retry_delay * (RETRY_BACKOFF_FACTOR**attempt), MAX_RETRY_DELAY].min
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
def build_raw_response(status, headers, body)
|
|
209
|
+
normalized = {}
|
|
210
|
+
(headers || {}).each { |name, values| normalized[name.to_s.downcase] = Array(values).map(&:to_s).freeze }
|
|
211
|
+
RawResponse.new(
|
|
212
|
+
status: status.to_i,
|
|
213
|
+
headers: normalized.freeze,
|
|
214
|
+
content_type: normalized["content-type"]&.first,
|
|
215
|
+
body: body.to_s.b
|
|
216
|
+
)
|
|
217
|
+
end
|
|
218
|
+
|
|
219
|
+
def api_error(raw)
|
|
220
|
+
parsed = parse_json(raw.body)
|
|
221
|
+
message = nil
|
|
222
|
+
if parsed.is_a?(Hash)
|
|
223
|
+
ERROR_MESSAGE_KEYS.each do |key|
|
|
224
|
+
value = parsed[key]
|
|
225
|
+
if value.is_a?(String) && !value.empty?
|
|
226
|
+
message = value
|
|
227
|
+
break
|
|
228
|
+
end
|
|
229
|
+
end
|
|
230
|
+
end
|
|
231
|
+
APIError.new(raw_response: raw, response_body: parsed == NOT_JSON ? nil : parsed, error_message: message)
|
|
232
|
+
end
|
|
233
|
+
|
|
234
|
+
def decode(raw, expected)
|
|
235
|
+
unless json_media_type?(raw.content_type)
|
|
236
|
+
if expected == :json
|
|
237
|
+
raise ResponseDecodingError.new("CaptureKit returned a successful non-JSON response for a JSON operation.", raw)
|
|
238
|
+
end
|
|
239
|
+
|
|
240
|
+
return BinaryResponse.new(bytes: raw.body, content_type: raw.content_type, raw_response: raw)
|
|
241
|
+
end
|
|
242
|
+
data = parse_json(raw.body)
|
|
243
|
+
if data == NOT_JSON
|
|
244
|
+
raise ResponseDecodingError.new("CaptureKit returned a successful JSON response that could not be decoded.", raw)
|
|
245
|
+
end
|
|
246
|
+
|
|
247
|
+
JsonResponse.new(data: data, raw_response: raw)
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
NOT_JSON = Object.new.freeze
|
|
251
|
+
private_constant :NOT_JSON
|
|
252
|
+
|
|
253
|
+
def parse_json(body)
|
|
254
|
+
JSON.parse(body.dup.force_encoding(Encoding::UTF_8))
|
|
255
|
+
rescue JSON::ParserError, EncodingError
|
|
256
|
+
NOT_JSON
|
|
257
|
+
end
|
|
258
|
+
|
|
259
|
+
def json_media_type?(content_type)
|
|
260
|
+
return false if content_type.nil?
|
|
261
|
+
|
|
262
|
+
media_type = content_type.split(";", 2).first.strip.downcase
|
|
263
|
+
media_type == "application/json" || media_type.end_with?("+json")
|
|
264
|
+
end
|
|
265
|
+
end
|
|
266
|
+
end
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Generated from the CaptureKit OpenAPI document. Do not edit by hand.
|
|
4
|
+
|
|
5
|
+
module CaptureKit
|
|
6
|
+
# Path, methods, response kind and parameter names of every documented operation.
|
|
7
|
+
ENDPOINTS = {
|
|
8
|
+
"analyze" => {
|
|
9
|
+
path: "/v1/analyze",
|
|
10
|
+
methods: ["GET", "POST"].freeze,
|
|
11
|
+
response: :json,
|
|
12
|
+
required: ["url"].freeze,
|
|
13
|
+
params: ["url", "custom_response", "custom_prompt", "proxy", "cache", "cache_ttl", "fields", "is_async", "soft_fail"].freeze
|
|
14
|
+
}.freeze,
|
|
15
|
+
"capture" => {
|
|
16
|
+
path: "/v1/capture",
|
|
17
|
+
methods: ["GET", "POST"].freeze,
|
|
18
|
+
response: :any,
|
|
19
|
+
required: ["url"].freeze,
|
|
20
|
+
params: ["url", "format", "device", "full_page", "full_page_scroll", "full_page_scroll_duration", "viewport_width", "viewport_height", "scale_factor", "image_quality", "selector", "cache", "cache_ttl", "remove_selectors", "remove_cookie_banners", "remove_ads", "block_resources", "block_urls", "delay", "wait_until", "wait_for_selector", "proxy", "s3_secret_key", "s3_access_key_id", "s3_bucket", "s3_object_key", "s3_region", "storage_endpoint", "s3_url", "fields", "is_async", "soft_fail"].freeze
|
|
21
|
+
}.freeze,
|
|
22
|
+
"content" => {
|
|
23
|
+
path: "/v1/content",
|
|
24
|
+
methods: ["GET", "POST"].freeze,
|
|
25
|
+
response: :json,
|
|
26
|
+
required: ["url"].freeze,
|
|
27
|
+
params: ["url", "include_html", "include_markdown", "include_sitemap", "use_defuddle", "remove_selectors", "remove_cookie_banners", "remove_ads", "block_resources", "block_urls", "delay", "wait_until", "wait_for_selector", "proxy", "cache", "cache_ttl", "fields", "is_async", "soft_fail"].freeze
|
|
28
|
+
}.freeze,
|
|
29
|
+
"jobs" => {
|
|
30
|
+
path: "/v1/jobs/{id}",
|
|
31
|
+
methods: ["GET"].freeze,
|
|
32
|
+
response: :json,
|
|
33
|
+
required: ["id"].freeze,
|
|
34
|
+
params: ["id"].freeze
|
|
35
|
+
}.freeze,
|
|
36
|
+
"stats" => {
|
|
37
|
+
path: "/v1/stats",
|
|
38
|
+
methods: ["GET"].freeze,
|
|
39
|
+
response: :json,
|
|
40
|
+
required: [].freeze,
|
|
41
|
+
params: ["engine", "path"].freeze
|
|
42
|
+
}.freeze,
|
|
43
|
+
"usage" => {
|
|
44
|
+
path: "/v1/usage",
|
|
45
|
+
methods: ["GET"].freeze,
|
|
46
|
+
response: :json,
|
|
47
|
+
required: [].freeze,
|
|
48
|
+
params: [].freeze
|
|
49
|
+
}.freeze,
|
|
50
|
+
"website_crawler" => {
|
|
51
|
+
path: "/v1/website/crawler",
|
|
52
|
+
methods: ["GET", "POST"].freeze,
|
|
53
|
+
response: :any,
|
|
54
|
+
required: ["query"].freeze,
|
|
55
|
+
params: ["query", "allow_redirects", "return_page_source", "fields", "is_async", "soft_fail"].freeze
|
|
56
|
+
}.freeze
|
|
57
|
+
}.freeze
|
|
58
|
+
end
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module CaptureKit
|
|
4
|
+
# Base class for every SDK-defined error.
|
|
5
|
+
class Error < StandardError; end
|
|
6
|
+
|
|
7
|
+
# The client configuration is missing or unusable.
|
|
8
|
+
class ConfigurationError < Error; end
|
|
9
|
+
|
|
10
|
+
# A request value cannot be represented safely on the wire.
|
|
11
|
+
class SerializationError < Error
|
|
12
|
+
attr_reader :param
|
|
13
|
+
|
|
14
|
+
def initialize(message, param = nil)
|
|
15
|
+
super(message)
|
|
16
|
+
@param = param
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
# The request failed before a complete HTTP response was received.
|
|
21
|
+
class TransportError < Error; end
|
|
22
|
+
|
|
23
|
+
# CaptureKit returned a non-success HTTP status.
|
|
24
|
+
class APIError < Error
|
|
25
|
+
attr_reader :status, :error_message, :response_body, :raw_response
|
|
26
|
+
|
|
27
|
+
def initialize(raw_response:, response_body: nil, error_message: nil)
|
|
28
|
+
super(error_message || "CaptureKit API request failed with status #{raw_response.status}.")
|
|
29
|
+
@status = raw_response.status
|
|
30
|
+
@error_message = error_message
|
|
31
|
+
@response_body = response_body
|
|
32
|
+
@raw_response = raw_response
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# A successful response could not be decoded as the documented representation.
|
|
37
|
+
class ResponseDecodingError < Error
|
|
38
|
+
attr_reader :raw_response
|
|
39
|
+
|
|
40
|
+
def initialize(message, raw_response)
|
|
41
|
+
super(message)
|
|
42
|
+
@raw_response = raw_response
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# An async job did not reach a terminal status before the deadline.
|
|
47
|
+
class JobTimeoutError < Error
|
|
48
|
+
attr_reader :job_id, :last_response
|
|
49
|
+
|
|
50
|
+
def initialize(job_id, last_response = nil)
|
|
51
|
+
super("Job #{job_id} did not finish before the deadline.")
|
|
52
|
+
@job_id = job_id
|
|
53
|
+
@last_response = last_response
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Generated from the CaptureKit OpenAPI document. Do not edit by hand.
|
|
4
|
+
|
|
5
|
+
module CaptureKit
|
|
6
|
+
# One method per documented endpoint. Mixed into {Client}.
|
|
7
|
+
module Operations
|
|
8
|
+
# Analyze (AI). AI Content API analyzes any webpage and returns AI insights summary, value proposition, features, categories, use cases, and optional contact signals. Credit cost: 1 credit per call.
|
|
9
|
+
#
|
|
10
|
+
# +GET / POST /v1/analyze+
|
|
11
|
+
#
|
|
12
|
+
# @param url [String] (required) The URL of the webpage to capture.
|
|
13
|
+
# @param custom_response [String] Pass custom response fields to be included in the AI response.
|
|
14
|
+
# @param custom_prompt [String] Add your own instructions to guide the AI’s analysis.
|
|
15
|
+
# @param proxy [String] Specify a proxy server to route your request through.
|
|
16
|
+
# @param cache [Boolean] Cache the response.
|
|
17
|
+
# @param cache_ttl [Numeric] Cache the response for a custom TTL (in seconds).
|
|
18
|
+
# @param fields [String] Comma-separated list of fields to keep in the response, with dot notation for nested fields (e.g. name,location.city). Does not change the credit cost.
|
|
19
|
+
# @param is_async [Boolean] Process the request asynchronously: the response is a job_id to poll on /v1/jobs/{id} (see waitForJob).
|
|
20
|
+
# @param soft_fail [Boolean] Return HTTP 200 even when an error occurs; the error is then in the response body.
|
|
21
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
22
|
+
# @return [JsonResponse]
|
|
23
|
+
def analyze(request_options: nil, **params)
|
|
24
|
+
call_operation("analyze", params, request_options)
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# Capture (Screenshot). Browser-like Screenshot API that captures webpages as PNG/JPEG/WebP or PDF for previews, QA, reporting, monitoring, and archiving no headless setup. Credit cost: 1 credit per call.
|
|
28
|
+
#
|
|
29
|
+
# +GET / POST /v1/capture+
|
|
30
|
+
#
|
|
31
|
+
# @param url [String] (required) The URL of the webpage to capture. Examples: https://www.amazon.com/s?k=pokemon
|
|
32
|
+
# @param format [String] The output format of the screenshot.
|
|
33
|
+
# @param device [String] The device type to emulate for the screenshot.
|
|
34
|
+
# @param full_page [Boolean] Capture the entire page instead of just the visible viewport.
|
|
35
|
+
# @param full_page_scroll [Boolean] Scroll the page to fully load lazy-loaded elements before capturing a full-page screenshot.
|
|
36
|
+
# @param full_page_scroll_duration [Numeric] Time in milliseconds for scrolling before capturing the full page.
|
|
37
|
+
# @param viewport_width [Numeric] The width of the browser viewport in pixels.
|
|
38
|
+
# @param viewport_height [Numeric] The height of the browser viewport in pixels.
|
|
39
|
+
# @param scale_factor [Numeric] The scale factor for high-resolution screenshots.
|
|
40
|
+
# @param image_quality [String] Generate images with custom quality settings.
|
|
41
|
+
# @param selector [String] Capture a specific element on the page instead of the full viewport.
|
|
42
|
+
# @param cache [Boolean] Cache the response.
|
|
43
|
+
# @param cache_ttl [Numeric] Cache the response for a custom TTL (in seconds).
|
|
44
|
+
# @param remove_selectors [String] A comma-separated list of elements to hide before capturing (e.g., ads, popups).
|
|
45
|
+
# @param remove_cookie_banners [Boolean] Automatically remove cookie banners before capturing.
|
|
46
|
+
# @param remove_ads [Boolean] Automatically remove ads before capturing.
|
|
47
|
+
# @param block_resources [String] Comma-separated list of resource types to block (e.g., “image,stylesheet,font”).
|
|
48
|
+
# @param block_urls [String] Comma-separated list of URL patterns to block (e.g., “analytics,tracking,advertisement”).
|
|
49
|
+
# @param delay [Numeric] Delay in seconds before capturing the screenshot.
|
|
50
|
+
# @param wait_until [Array<String>] Define when to capture.
|
|
51
|
+
# @param wait_for_selector [String] Wait for a specific element to appear before taking the screenshot.
|
|
52
|
+
# @param proxy [String] Specify a proxy server to route your request through.
|
|
53
|
+
# @param s3_secret_key [String] AWS S3 secret access key.
|
|
54
|
+
# @param s3_access_key_id [String] AWS S3 access key ID.
|
|
55
|
+
# @param s3_bucket [String] The S3 bucket name where the screenshot will be uploaded.
|
|
56
|
+
# @param s3_object_key [String] The filename for the S3 object (auto-generated if not provided).
|
|
57
|
+
# @param s3_region [String] AWS S3 region for storage.
|
|
58
|
+
# @param storage_endpoint [String] Leave this field empty for AWS S3. Only specify a value if required. S3-compatible storage services are supported, for example, use https://<accountId>.r2.cloudflarestorage.com for Cloudflare R2.
|
|
59
|
+
# @param s3_url [Boolean] Use S3 storage for screenshots. The S3 URL will be returned in the response.
|
|
60
|
+
# @param fields [String] Comma-separated list of fields to keep in the response, with dot notation for nested fields (e.g. name,location.city). Does not change the credit cost.
|
|
61
|
+
# @param is_async [Boolean] Process the request asynchronously: the response is a job_id to poll on /v1/jobs/{id} (see waitForJob).
|
|
62
|
+
# @param soft_fail [Boolean] Return HTTP 200 even when an error occurs; the error is then in the response body.
|
|
63
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
64
|
+
# @return [JsonResponse, BinaryResponse]
|
|
65
|
+
def capture(request_options: nil, **params)
|
|
66
|
+
call_operation("capture", params, request_options)
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
# Content. CaptureKit Content API extracts structured webpage data, metadata, links, HTML/Markdown: in one request for SEO, monitoring, indexing and automation. Credit cost: 1 credit per call.
|
|
70
|
+
#
|
|
71
|
+
# +GET / POST /v1/content+
|
|
72
|
+
#
|
|
73
|
+
# @param url [String] (required) The URL of the webpage to capture.
|
|
74
|
+
# @param include_html [Boolean] Include the raw HTML of the webpage in the response.
|
|
75
|
+
# @param include_markdown [Boolean] Include the Markdown of the webpage in the response.
|
|
76
|
+
# @param include_sitemap [Boolean] Include the sitemap of the website in the response.
|
|
77
|
+
# @param use_defuddle [Boolean] Use Defuddle to clean and extract the main content from web pages. This popular library removes unnecessary elements like comments, sidebars, headers, footers, and other non-essential elements, leavi…
|
|
78
|
+
# @param remove_selectors [String] A comma-separated list of elements to hide before capturing (e.g., ads, popups).
|
|
79
|
+
# @param remove_cookie_banners [Boolean] Automatically remove cookie banners before capturing.
|
|
80
|
+
# @param remove_ads [Boolean] Automatically remove ads before capturing.
|
|
81
|
+
# @param block_resources [String] Comma-separated list of resource types to block (e.g., “image,stylesheet,font”).
|
|
82
|
+
# @param block_urls [String] Comma-separated list of URL patterns to block (e.g., “analytics,tracking,advertisement”).
|
|
83
|
+
# @param delay [Numeric] Delay in seconds before capturing the screenshot.
|
|
84
|
+
# @param wait_until [Array<String>] Define when to capture.
|
|
85
|
+
# @param wait_for_selector [String] Wait for a specific element to appear before taking the screenshot.
|
|
86
|
+
# @param proxy [String] Specify a proxy server to route your request through.
|
|
87
|
+
# @param cache [Boolean] Cache the response.
|
|
88
|
+
# @param cache_ttl [Numeric] Cache the response for a custom TTL (in seconds).
|
|
89
|
+
# @param fields [String] Comma-separated list of fields to keep in the response, with dot notation for nested fields (e.g. name,location.city). Does not change the credit cost.
|
|
90
|
+
# @param is_async [Boolean] Process the request asynchronously: the response is a job_id to poll on /v1/jobs/{id} (see waitForJob).
|
|
91
|
+
# @param soft_fail [Boolean] Return HTTP 200 even when an error occurs; the error is then in the response body.
|
|
92
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
93
|
+
# @return [JsonResponse]
|
|
94
|
+
def content(request_options: nil, **params)
|
|
95
|
+
call_operation("content", params, request_options)
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
# Jobs. Poll the status, progress (0–100) and result of a job started with is_async=true. Polling is free. Poll until status is terminal: CompletedJob, Error, NotFound or BadRequest.
|
|
99
|
+
#
|
|
100
|
+
# +GET /v1/jobs/{id}+
|
|
101
|
+
#
|
|
102
|
+
# @param id [String] (required) The job_id returned by an endpoint called with is_async=true.
|
|
103
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
104
|
+
# @return [JsonResponse]
|
|
105
|
+
def jobs(request_options: nil, **params)
|
|
106
|
+
call_operation("jobs", params, request_options)
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
# Stats. Infrastructure health for an engine or endpoint over the last hour. Not scoped to your account or tenant. Enabled per account: HTTP 403 means contact support to activate it.
|
|
110
|
+
#
|
|
111
|
+
# +GET /v1/stats+
|
|
112
|
+
#
|
|
113
|
+
# @param engine [String] API family to check, for example capture. Provide at least engine or path.
|
|
114
|
+
# @param path [String] Specific endpoint to check, for example /v1/capture. If you also pass engine, use an endpoint from that API.
|
|
115
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
116
|
+
# @return [JsonResponse]
|
|
117
|
+
def stats(request_options: nil, **params)
|
|
118
|
+
call_operation("stats", params, request_options)
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
# Usage. Retrieve real-time usage statistics and credit balance for your CaptureKit account.
|
|
122
|
+
#
|
|
123
|
+
# +GET /v1/usage+
|
|
124
|
+
#
|
|
125
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
126
|
+
# @return [JsonResponse]
|
|
127
|
+
def usage(request_options: nil, **params)
|
|
128
|
+
call_operation("usage", params, request_options)
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
# Website Crawler. Website Crawler API fetches raw HTML from any webpage in request mode (no rendering) for fast scraping, audits, monitoring, and link/metadata extraction. Credit cost: 1 credit per call.
|
|
132
|
+
#
|
|
133
|
+
# +GET / POST /v1/website/crawler+
|
|
134
|
+
#
|
|
135
|
+
# @param query [String] (required) A website URL with either the http or https protocol.
|
|
136
|
+
# @param allow_redirects [Boolean] If set to false, do not follow redirects.
|
|
137
|
+
# @param return_page_source [Boolean] If set to true, the response will be the HTML source of the page.
|
|
138
|
+
# @param fields [String] Comma-separated list of fields to keep in the response, with dot notation for nested fields (e.g. name,location.city). Does not change the credit cost.
|
|
139
|
+
# @param is_async [Boolean] Process the request asynchronously: the response is a job_id to poll on /v1/jobs/{id} (see waitForJob).
|
|
140
|
+
# @param soft_fail [Boolean] Return HTTP 200 even when an error occurs; the error is then in the response body.
|
|
141
|
+
# @param request_options [Hash, nil] +:method+ ("GET"/"POST"), +:timeout+ (seconds), +:headers+
|
|
142
|
+
# @return [JsonResponse, BinaryResponse]
|
|
143
|
+
def website_crawler(request_options: nil, **params)
|
|
144
|
+
call_operation("website_crawler", params, request_options)
|
|
145
|
+
end
|
|
146
|
+
end
|
|
147
|
+
end
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module CaptureKit
|
|
4
|
+
# Buffered HTTP response: status, lower-cased multi-value headers, content type and body bytes.
|
|
5
|
+
RawResponse = Struct.new(:status, :headers, :content_type, :body, keyword_init: true)
|
|
6
|
+
|
|
7
|
+
# Outcome of one Client#batch item: +value+ when +ok?+, else +error+.
|
|
8
|
+
BatchResult = Struct.new(:index, :value, :error, keyword_init: true) do
|
|
9
|
+
def ok? = error.nil?
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
# A successful JSON response; +data+ is the parsed body (string keys).
|
|
13
|
+
class JsonResponse
|
|
14
|
+
attr_reader :data, :raw_response
|
|
15
|
+
|
|
16
|
+
def initialize(data:, raw_response:)
|
|
17
|
+
@data = data
|
|
18
|
+
@raw_response = raw_response
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def kind = :json
|
|
22
|
+
def json? = true
|
|
23
|
+
def binary? = false
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# A successful non-JSON response (image, PDF, HTML…).
|
|
27
|
+
class BinaryResponse
|
|
28
|
+
attr_reader :bytes, :content_type, :raw_response
|
|
29
|
+
|
|
30
|
+
def initialize(bytes:, content_type:, raw_response:)
|
|
31
|
+
@bytes = bytes
|
|
32
|
+
@content_type = content_type
|
|
33
|
+
@raw_response = raw_response
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def kind = :binary
|
|
37
|
+
def json? = false
|
|
38
|
+
def binary? = true
|
|
39
|
+
|
|
40
|
+
# Writes the body to +path+ and returns the number of bytes written.
|
|
41
|
+
def save(path)
|
|
42
|
+
File.binwrite(path, bytes)
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
require "uri"
|
|
5
|
+
|
|
6
|
+
module CaptureKit
|
|
7
|
+
# Request serialization: path filling, query pairs and JSON bodies.
|
|
8
|
+
module Serialization
|
|
9
|
+
QUERY_STRING_PARAMS = %w[fields is_async soft_fail].freeze
|
|
10
|
+
|
|
11
|
+
module_function
|
|
12
|
+
|
|
13
|
+
# Fills {name} path segments, then returns [path, query_pairs, body].
|
|
14
|
+
# nil values are omitted; false and 0 are sent.
|
|
15
|
+
def serialize(method, path_template, params)
|
|
16
|
+
raise SerializationError, "Request parameters must be a Hash." unless params.nil? || params.is_a?(Hash)
|
|
17
|
+
|
|
18
|
+
remaining = {}
|
|
19
|
+
(params || {}).each { |name, value| remaining[name.to_s] = value unless value.nil? }
|
|
20
|
+
path = path_template.gsub(/\{([^}]+)\}/) do
|
|
21
|
+
name = Regexp.last_match(1)
|
|
22
|
+
raise SerializationError.new("The \"#{name}\" path parameter is required.", name) unless remaining.key?(name)
|
|
23
|
+
|
|
24
|
+
URI.encode_www_form_component(wire_string(name, remaining.delete(name))).gsub("+", "%20")
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
if method == "POST"
|
|
28
|
+
# The API reads the global parameters from the query string, whatever the method.
|
|
29
|
+
globals = QUERY_STRING_PARAMS.filter_map { |name| [name, wire_string(name, remaining.delete(name))] if remaining.key?(name) }
|
|
30
|
+
return [path, globals, remaining]
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
pairs = []
|
|
34
|
+
remaining.each do |name, value|
|
|
35
|
+
if value.is_a?(Array)
|
|
36
|
+
value.each { |item| pairs << [name, wire_string(name, item)] unless item.nil? }
|
|
37
|
+
else
|
|
38
|
+
pairs << [name, wire_string(name, value)]
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
[path, pairs, nil]
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def wire_string(name, value)
|
|
45
|
+
case value
|
|
46
|
+
when String then value
|
|
47
|
+
when true, false, Integer, Symbol then value.to_s
|
|
48
|
+
when Float
|
|
49
|
+
raise SerializationError.new("The \"#{name}\" parameter must be a finite number.", name) unless value.finite?
|
|
50
|
+
|
|
51
|
+
value == value.floor ? value.to_i.to_s : value.to_s
|
|
52
|
+
when Hash, Array then JSON.generate(value)
|
|
53
|
+
else
|
|
54
|
+
raise SerializationError.new("The \"#{name}\" parameter has an unsupported type (#{value.class}).", name)
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
def encode_query(pairs)
|
|
59
|
+
pairs.map { |name, value| "#{escape(name)}=#{escape(value)}" }.join("&")
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def escape(value)
|
|
63
|
+
URI.encode_www_form_component(value).gsub("+", "%20")
|
|
64
|
+
end
|
|
65
|
+
end
|
|
66
|
+
end
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "net/http"
|
|
4
|
+
require "uri"
|
|
5
|
+
|
|
6
|
+
module CaptureKit
|
|
7
|
+
# Default transport on Net::HTTP. A custom transport is any object with
|
|
8
|
+
# +call(method:, url:, headers:, body:, timeout:)+ returning +[status, headers, body]+,
|
|
9
|
+
# where headers is a Hash of lower-cased names to arrays of values.
|
|
10
|
+
class NetHTTPTransport
|
|
11
|
+
def call(method:, url:, headers:, body:, timeout:)
|
|
12
|
+
uri = URI(url)
|
|
13
|
+
request = (method == "POST" ? Net::HTTP::Post : Net::HTTP::Get).new(uri)
|
|
14
|
+
headers.each { |name, value| request[name] = value }
|
|
15
|
+
request.body = body if body
|
|
16
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
|
17
|
+
http.use_ssl = uri.scheme == "https"
|
|
18
|
+
if timeout
|
|
19
|
+
http.open_timeout = timeout
|
|
20
|
+
http.read_timeout = timeout
|
|
21
|
+
http.write_timeout = timeout
|
|
22
|
+
end
|
|
23
|
+
response = http.start { |connection| connection.request(request) }
|
|
24
|
+
[response.code.to_i, response.to_hash, response.body || ""]
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
data/lib/capturekit.rb
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "capturekit/version"
|
|
4
|
+
require_relative "capturekit/errors"
|
|
5
|
+
require_relative "capturekit/responses"
|
|
6
|
+
require_relative "capturekit/serialization"
|
|
7
|
+
require_relative "capturekit/transport"
|
|
8
|
+
require_relative "capturekit/endpoints"
|
|
9
|
+
require_relative "capturekit/operations"
|
|
10
|
+
require_relative "capturekit/client"
|
|
11
|
+
|
|
12
|
+
# Official Ruby SDK for the CaptureKit API (https://www.capturekit.dev).
|
|
13
|
+
module CaptureKit
|
|
14
|
+
end
|
data/sig/capturekit.rbs
ADDED
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
module CaptureKit
|
|
2
|
+
VERSION: String
|
|
3
|
+
ENDPOINTS: Hash[String, Hash[Symbol, untyped]]
|
|
4
|
+
|
|
5
|
+
class Error < StandardError
|
|
6
|
+
end
|
|
7
|
+
class ConfigurationError < Error
|
|
8
|
+
end
|
|
9
|
+
class SerializationError < Error
|
|
10
|
+
attr_reader param: String?
|
|
11
|
+
def initialize: (String message, ?String? param) -> void
|
|
12
|
+
end
|
|
13
|
+
class TransportError < Error
|
|
14
|
+
end
|
|
15
|
+
class APIError < Error
|
|
16
|
+
attr_reader status: Integer
|
|
17
|
+
attr_reader error_message: String?
|
|
18
|
+
attr_reader response_body: untyped
|
|
19
|
+
attr_reader raw_response: RawResponse
|
|
20
|
+
end
|
|
21
|
+
class ResponseDecodingError < Error
|
|
22
|
+
attr_reader raw_response: RawResponse
|
|
23
|
+
end
|
|
24
|
+
class JobTimeoutError < Error
|
|
25
|
+
attr_reader job_id: String
|
|
26
|
+
attr_reader last_response: JsonResponse?
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
class RawResponse < Struct[untyped]
|
|
30
|
+
attr_accessor status: Integer
|
|
31
|
+
attr_accessor headers: Hash[String, Array[String]]
|
|
32
|
+
attr_accessor content_type: String?
|
|
33
|
+
attr_accessor body: String
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
class JsonResponse
|
|
37
|
+
attr_reader data: untyped
|
|
38
|
+
attr_reader raw_response: RawResponse
|
|
39
|
+
def kind: () -> :json
|
|
40
|
+
def json?: () -> true
|
|
41
|
+
def binary?: () -> false
|
|
42
|
+
end
|
|
43
|
+
class BinaryResponse
|
|
44
|
+
attr_reader bytes: String
|
|
45
|
+
attr_reader content_type: String?
|
|
46
|
+
attr_reader raw_response: RawResponse
|
|
47
|
+
def kind: () -> :binary
|
|
48
|
+
def json?: () -> false
|
|
49
|
+
def binary?: () -> true
|
|
50
|
+
def save: (String path) -> Integer
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
class BatchResult < Struct[untyped]
|
|
54
|
+
attr_accessor index: Integer
|
|
55
|
+
attr_accessor value: untyped
|
|
56
|
+
attr_accessor error: StandardError?
|
|
57
|
+
def ok?: () -> bool
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
type response = JsonResponse | BinaryResponse
|
|
61
|
+
type request_options = { ?method: String, ?timeout: Numeric?, ?headers: Hash[String, String], ?max_retries: Integer }
|
|
62
|
+
|
|
63
|
+
module Operations
|
|
64
|
+
def analyze: (url: String, ?custom_response: String, ?custom_prompt: String, ?proxy: String, ?cache: bool, ?cache_ttl: Numeric, ?fields: String, ?is_async: bool, ?soft_fail: bool, ?request_options: request_options?) -> JsonResponse
|
|
65
|
+
def capture: (url: String, ?format: String, ?device: String, ?full_page: bool, ?full_page_scroll: bool, ?full_page_scroll_duration: Numeric, ?viewport_width: Numeric, ?viewport_height: Numeric, ?scale_factor: Numeric, ?image_quality: String, ?selector: String, ?cache: bool, ?cache_ttl: Numeric, ?remove_selectors: String, ?remove_cookie_banners: bool, ?remove_ads: bool, ?block_resources: String, ?block_urls: String, ?delay: Numeric, ?wait_until: Array[untyped], ?wait_for_selector: String, ?proxy: String, ?s3_secret_key: String, ?s3_access_key_id: String, ?s3_bucket: String, ?s3_object_key: String, ?s3_region: String, ?storage_endpoint: String, ?s3_url: bool, ?fields: String, ?is_async: bool, ?soft_fail: bool, ?request_options: request_options?) -> response
|
|
66
|
+
def content: (url: String, ?include_html: bool, ?include_markdown: bool, ?include_sitemap: bool, ?use_defuddle: bool, ?remove_selectors: String, ?remove_cookie_banners: bool, ?remove_ads: bool, ?block_resources: String, ?block_urls: String, ?delay: Numeric, ?wait_until: Array[untyped], ?wait_for_selector: String, ?proxy: String, ?cache: bool, ?cache_ttl: Numeric, ?fields: String, ?is_async: bool, ?soft_fail: bool, ?request_options: request_options?) -> JsonResponse
|
|
67
|
+
def jobs: (id: String, ?request_options: request_options?) -> JsonResponse
|
|
68
|
+
def stats: (?engine: String, ?path: String, ?request_options: request_options?) -> JsonResponse
|
|
69
|
+
def usage: (?request_options: request_options?) -> JsonResponse
|
|
70
|
+
def website_crawler: (query: String, ?allow_redirects: bool, ?return_page_source: bool, ?fields: String, ?is_async: bool, ?soft_fail: bool, ?request_options: request_options?) -> response
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
class Client
|
|
74
|
+
include Operations
|
|
75
|
+
DEFAULT_CONCURRENCY: Integer
|
|
76
|
+
MAX_RETRIES: Integer
|
|
77
|
+
RETRY_BACKOFF_FACTOR: Float
|
|
78
|
+
DEFAULT_RETRY_DELAY: Float
|
|
79
|
+
DEFAULT_TIMEOUT: Integer
|
|
80
|
+
def initialize: (?api_key: String?, ?base_url: String, ?timeout: Numeric?, ?headers: Hash[untyped, untyped], ?max_retries: Integer, ?retry_delay: Numeric, ?transport: untyped) -> void
|
|
81
|
+
def request: (String | Symbol method, String path, ?Hash[untyped, untyped]? params, ?timeout: Numeric?, ?headers: Hash[untyped, untyped]?, ?max_retries: Integer?) -> response
|
|
82
|
+
def batch: [I, T] (Enumerable[I] items, ?concurrency: Integer) { (I item) -> T } -> Array[BatchResult]
|
|
83
|
+
def wait_for_job: (String job_id, ?interval: Numeric, ?timeout: Numeric) ?{ (untyped job) -> void } -> JsonResponse
|
|
84
|
+
end
|
|
85
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: capturekitdev
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- CaptureKit
|
|
8
|
+
autorequire:
|
|
9
|
+
bindir: bin
|
|
10
|
+
cert_chain: []
|
|
11
|
+
date: 2026-11-06 00:00:00.000000000 Z
|
|
12
|
+
dependencies: []
|
|
13
|
+
description: Capture websites as PNG, JPEG, WebP or PDF, extract their content, or
|
|
14
|
+
analyse them with AI.
|
|
15
|
+
email:
|
|
16
|
+
- contact@capturekit.dev
|
|
17
|
+
executables: []
|
|
18
|
+
extensions: []
|
|
19
|
+
extra_rdoc_files: []
|
|
20
|
+
files:
|
|
21
|
+
- LICENSE
|
|
22
|
+
- README.md
|
|
23
|
+
- examples/README.md
|
|
24
|
+
- examples/binary.rb
|
|
25
|
+
- examples/json.rb
|
|
26
|
+
- examples/request.rb
|
|
27
|
+
- examples/wait_for_job.rb
|
|
28
|
+
- lib/capturekit.rb
|
|
29
|
+
- lib/capturekit/client.rb
|
|
30
|
+
- lib/capturekit/endpoints.rb
|
|
31
|
+
- lib/capturekit/errors.rb
|
|
32
|
+
- lib/capturekit/operations.rb
|
|
33
|
+
- lib/capturekit/responses.rb
|
|
34
|
+
- lib/capturekit/serialization.rb
|
|
35
|
+
- lib/capturekit/transport.rb
|
|
36
|
+
- lib/capturekit/version.rb
|
|
37
|
+
- lib/capturekitdev.rb
|
|
38
|
+
- sig/capturekit.rbs
|
|
39
|
+
homepage: https://www.capturekit.dev
|
|
40
|
+
licenses:
|
|
41
|
+
- MIT
|
|
42
|
+
metadata:
|
|
43
|
+
bug_tracker_uri: https://github.com/capturekitdev/capturekit-ruby/issues
|
|
44
|
+
documentation_uri: https://app.capturekit.dev/openapi
|
|
45
|
+
homepage_uri: https://www.capturekit.dev
|
|
46
|
+
rubygems_mfa_required: 'true'
|
|
47
|
+
source_code_uri: https://github.com/capturekitdev/capturekit-ruby
|
|
48
|
+
post_install_message:
|
|
49
|
+
rdoc_options: []
|
|
50
|
+
require_paths:
|
|
51
|
+
- lib
|
|
52
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
53
|
+
requirements:
|
|
54
|
+
- - ">="
|
|
55
|
+
- !ruby/object:Gem::Version
|
|
56
|
+
version: '3.1'
|
|
57
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
58
|
+
requirements:
|
|
59
|
+
- - ">="
|
|
60
|
+
- !ruby/object:Gem::Version
|
|
61
|
+
version: '0'
|
|
62
|
+
requirements: []
|
|
63
|
+
rubygems_version: 3.5.22
|
|
64
|
+
signing_key:
|
|
65
|
+
specification_version: 4
|
|
66
|
+
summary: Official Ruby SDK for the CaptureKit screenshot API.
|
|
67
|
+
test_files: []
|