api2convert 10.2.1 → 10.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/AGENTS.md +1 -1
- data/README.md +5 -0
- data/docs/SDK_CONTRACT.md +57 -6
- data/lib/api2convert/client.rb +29 -5
- data/lib/api2convert/cloud_provider.rb +29 -0
- data/lib/api2convert/http/net_http_sender.rb +31 -3
- data/lib/api2convert/http/request.rb +7 -5
- data/lib/api2convert/http/transport.rb +18 -3
- data/lib/api2convert/model/cloud_input.rb +108 -0
- data/lib/api2convert/model/conversion.rb +7 -3
- data/lib/api2convert/model/input_file.rb +8 -3
- data/lib/api2convert/model/output_target.rb +78 -0
- data/lib/api2convert/resource/jobs.rb +6 -2
- data/lib/api2convert/resource/stats.rb +2 -1
- data/lib/api2convert/result.rb +51 -20
- data/lib/api2convert/support/data.rb +8 -1
- data/lib/api2convert/support/redactor.rb +83 -0
- data/lib/api2convert/upload/file_uploader.rb +2 -2
- data/lib/api2convert/version.rb +1 -1
- data/lib/api2convert.rb +4 -0
- data/openapi/api2convert.openapi.json +51 -51
- metadata +6 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ab0c201184fae5e154e8eb8dda08b1dd523f6721b8c0ea63882f36b0d08d69a8
|
|
4
|
+
data.tar.gz: 51d41801caf9b731b48b2f988b2f76605638b455b90a337c4ce6ce2acbfda4d6
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e1a07542793b7fe4f016907ba28b6e25bc6d887bf0549714b3233350994f561534604a79f1488df8c69efc7cd40f76340a73486938fa5c82fb9424a8f63417d6
|
|
7
|
+
data.tar.gz: bd3a8cdc13f9d5092a54f91d7e6ef3caa5e7627b2bee607e8203afda5a7991d5356749ce35da48afcd0ae8420e3e43cf6b52f359b37bdfc6feba030111fefb24
|
data/AGENTS.md
CHANGED
|
@@ -63,7 +63,7 @@ junior-friendly surface — one-call `convert` — and use AI to keep it current
|
|
|
63
63
|
key/token in errors. Secret-scan before any release.
|
|
64
64
|
- **The contract is law.** Public method names, signatures and semantics match `docs/SDK_CONTRACT.md`
|
|
65
65
|
across every SDK language. Adapt only to Ruby idiom (see divergences below).
|
|
66
|
-
- **Upload uses the per-job `X-
|
|
66
|
+
- **Upload uses the per-job `X-Api2convert-Token`, never the account key.** There is a spec for this.
|
|
67
67
|
- **Secret-bearing requests never follow redirects.** The key/token/download-password ride in custom
|
|
68
68
|
`X-Oc-*` headers. `Net::HTTP` does not follow redirects by default — the SDK relies on that and
|
|
69
69
|
only opts the no-secret download path into following redirects. `spec/security` proves the
|
data/README.md
CHANGED
|
@@ -1,5 +1,10 @@
|
|
|
1
1
|
# API2Convert Ruby SDK
|
|
2
2
|
|
|
3
|
+
[](https://github.com/QaamGo/api2convert-ruby/actions/workflows/ci.yml)
|
|
4
|
+
[](https://rubygems.org/gems/api2convert)
|
|
5
|
+

|
|
6
|
+

|
|
7
|
+
|
|
3
8
|
Official Ruby SDK for the [API2Convert](https://www.api2convert.com) file-conversion API.
|
|
4
9
|
Convert, compress and transform images, documents, audio, video, ebooks, archives and CAD —
|
|
5
10
|
and run operations like OCR, merge, thumbnail and website capture — in one line of code.
|
data/docs/SDK_CONTRACT.md
CHANGED
|
@@ -10,27 +10,40 @@ Two layers make up an SDK:
|
|
|
10
10
|
(`openapi/api2convert.openapi.json`); an AI update may freely change it to match the spec.
|
|
11
11
|
- **Hand-authored layer** — the ergonomics below (`convert`, upload, polling, download, webhook
|
|
12
12
|
verification). These flows are **not** in the spec. Their **public signatures and semantics are
|
|
13
|
-
fixed**: change them only when this document changes
|
|
13
|
+
fixed**: change them only when this document changes. Adding a new **optional** parameter or
|
|
14
|
+
options-bag field that preserves every existing call site and behavior is an additive **minor**;
|
|
15
|
+
changing or removing an existing parameter, return type, or documented semantic is a **major**.
|
|
14
16
|
|
|
15
17
|
> When the two disagree, this contract wins for the hand-authored layer; the spec wins for the
|
|
16
18
|
> derived layer.
|
|
17
19
|
|
|
18
20
|
## Protocol facts (the API the SDK speaks)
|
|
19
21
|
|
|
20
|
-
- Base URL `https://api.api2convert.com/v2`. Auth header `X-
|
|
22
|
+
- Base URL `https://api.api2convert.com/v2`. Auth header `X-Api2convert-Api-Key: <key>` on account requests (canonical `X-Api2convert-*` names; the legacy `X-Oc-*` headers remain accepted by the API as permanent aliases).
|
|
21
23
|
- **Create job** `POST /jobs` `{ conversion:[{category?,target,options?}], input?:[…], process:bool,
|
|
22
24
|
callback?, notify_status?, download_passwords?:[…] }` → response includes `id`, per-job `token`,
|
|
23
25
|
per-job upload `server`, and `status.code`. `download_passwords` protects every output of the job;
|
|
24
26
|
any password in the list then unlocks its downloads. The API never returns the plaintext back.
|
|
25
27
|
- **Upload** (not in the spec): `POST {server}/upload-file/{job_id}`, `multipart/form-data` field
|
|
26
|
-
`file`, authenticated with the per-job **`X-
|
|
28
|
+
`file`, authenticated with the per-job **`X-Api2convert-Token`** header — never the account key.
|
|
27
29
|
- **Add remote input**: `POST /jobs/{id}/input` `{ type:'remote', source:'https://…' }`.
|
|
30
|
+
- **Cloud input** (import from customer storage): `POST /jobs/{id}/input` (or inline in `create`)
|
|
31
|
+
`{ type:'cloud', source:<provider>, parameters:{…}, credentials:{…} }`, `<provider> ∈ {amazons3,
|
|
32
|
+
azure, ftp, googlecloud}`; plus `{ type:'gdrive_picker', source:<drive-file-id>,
|
|
33
|
+
credentials:{token}, content_type? }` for Google Drive. The API validates a cloud descriptor
|
|
34
|
+
**asynchronously** — it accepts any descriptor on create (`201`) and later fails a bad one on the
|
|
35
|
+
input (`status:failed`, generic `code 99`); it never echoes a credential value.
|
|
36
|
+
- **Cloud output** (deliver to customer storage): a `conversion[]` may carry
|
|
37
|
+
`output_target:[{ type:<provider>, parameters:{…}, credentials:{…} }]`, `<provider> ∈ {amazons3,
|
|
38
|
+
googlecloud, azure, ftp, youtube, gdrive}`. `status` (`waiting|uploading|completed|failed`) is
|
|
39
|
+
server-set and read-only — never sent on create. The job reaches `completed` only after the upload
|
|
40
|
+
succeeds (a failed upload → `failed`); an output-target conversion produces **no** local output.
|
|
28
41
|
- **Start**: `PATCH /jobs/{id}` `{ process:true }`.
|
|
29
42
|
- **Poll**: `GET /jobs/{id}` → terminal when `status.code ∈ {completed, failed, canceled}`
|
|
30
43
|
(`failed`/`canceled` are unsuccessful terminals; non-terminal: `created`, `incomplete`,
|
|
31
44
|
`downloading`, `queued`, `processing`, and any unknown code). Poll with backoff; clamp the
|
|
32
45
|
interval to a floor (never busy-loop) and the total wait to a ceiling (never poll unbounded).
|
|
33
|
-
- **Download**: `GET output.uri` — self-contained, no auth; `X-
|
|
46
|
+
- **Download**: `GET output.uri` — self-contained, no auth; `X-Api2convert-Download-Password` header if set.
|
|
34
47
|
- **Discover options**: `GET /conversions?category=&target=`.
|
|
35
48
|
- **Errors**: HTTP body `{ "message": "…" }`; job-level `errors[]` / `warnings[]` of
|
|
36
49
|
`{ source, id_source, code, message, details }`.
|
|
@@ -64,7 +77,7 @@ Two layers make up an SDK:
|
|
|
64
77
|
- `contents(downloadPassword?) → binary`, `url() → string`, `output()`, `outputs()`.
|
|
65
78
|
- **Download-password transparency**: a password supplied at conversion time (`convert(...,
|
|
66
79
|
downloadPassword)`) or to `download(output, downloadPassword)` is remembered and sent as the
|
|
67
|
-
`X-
|
|
80
|
+
`X-Api2convert-Download-Password` header on every download from that result/handle — callers do not
|
|
68
81
|
re-supply it. An explicit `downloadPassword` argument to `save()` / `contents()` overrides the
|
|
69
82
|
remembered one for that call.
|
|
70
83
|
|
|
@@ -82,8 +95,39 @@ Two layers make up an SDK:
|
|
|
82
95
|
missing/wrong signature. Empty secret skips verification.
|
|
83
96
|
- `parse(rawBody) → WebhookEvent` — deserialize without verifying (pre-signed-webhooks).
|
|
84
97
|
|
|
98
|
+
## Cloud storage connectors
|
|
99
|
+
|
|
100
|
+
The API imports inputs from and delivers outputs to customer-owned cloud storage. The SDK models the
|
|
101
|
+
wire descriptors above; per-provider keys are **not** validated synchronously server-side, so the
|
|
102
|
+
typed surface is the client's only pre-flight structure.
|
|
103
|
+
|
|
104
|
+
- **Provider vocabulary** — one shared `CloudProvider` concept (per-language spelling): `amazons3,
|
|
105
|
+
azure, ftp, gdrive, googlecloud, youtube`. It is **build-side vocabulary only** — read models keep
|
|
106
|
+
`source`/`type`/`status` as raw strings, and an unknown provider from the server round-trips
|
|
107
|
+
untyped (never throws).
|
|
108
|
+
- **Cloud input** — a `CloudInput` builder emits `{ type:cloud, source, parameters, credentials }`
|
|
109
|
+
and hands off to `addInput` / the create path. It ships per-provider named constructors whose
|
|
110
|
+
signatures carry each provider's required keys **verbatim** (flat/lowercase, as the API expects):
|
|
111
|
+
`amazonS3(bucket, file, accesskeyid, secretaccesskey)`,
|
|
112
|
+
`azure(container, file, accountname, accountkey)`, `ftp(host, file, username, password)`,
|
|
113
|
+
`googleCloud(projectid, bucket, file, keyfile)`. The required keys are **constructor arguments**,
|
|
114
|
+
not a runtime gate — the builder never rejects a descriptor the permissive server would accept, and
|
|
115
|
+
a generic `parameters`/`credentials` map stays reachable for optional/forward-compat keys. Google
|
|
116
|
+
Drive input uses `type:gdrive_picker` (`source` = the Drive file id, token in `credentials.token`),
|
|
117
|
+
carried by the generic `addInput` raw-map path this wave (no typed builder yet). `gdrive` and
|
|
118
|
+
`youtube` are **output-only** — they validate as an input `source` but have no downloader.
|
|
119
|
+
- **Cloud output** — an `OutputTarget` model (`type` = a `CloudProvider` + free-form
|
|
120
|
+
`parameters`/`credentials`) attaches to a conversion, both via `convert`/`convertAsync` (a new
|
|
121
|
+
optional `outputTargets` control, never merged into the options map) and the raw `jobs().create`
|
|
122
|
+
conversion map. It serializes `{ type, parameters, credentials }` and **omits `status`** on create.
|
|
123
|
+
Per-provider output factories are **not** in this wave (their keys live in a separate service and
|
|
124
|
+
diverge per provider). When any output target is set, `convert` returns the completed `Job` and
|
|
125
|
+
does **not** download — the conversion has no local output.
|
|
126
|
+
- **Read semantics** — `parameters` and the per-target `status` round-trip on read; `credentials`
|
|
127
|
+
are **never** surfaced (the API returns them empty; the SDK does not hydrate them).
|
|
128
|
+
|
|
85
129
|
## Cross-cutting semantics
|
|
86
|
-
- **Auth**: account key as `X-
|
|
130
|
+
- **Auth**: account key as `X-Api2convert-Api-Key`; uploads use the per-job token.
|
|
87
131
|
- **Retries**: automatically retry with capped, jittered exponential backoff, honoring `Retry-After`
|
|
88
132
|
(delay-seconds or HTTP-date form, clamped to a ceiling). `429` is retried for every method; `5xx`
|
|
89
133
|
and network errors are retried only for idempotent methods (`GET`/`HEAD`/`PUT`/`DELETE`/`OPTIONS`/
|
|
@@ -97,3 +141,10 @@ Two layers make up an SDK:
|
|
|
97
141
|
conversion-failed (carrying the job and its `errors`).
|
|
98
142
|
- **Naming**: method names and option keys are identical across languages, adapted only to each
|
|
99
143
|
language's idiom (camelCase / snake_case).
|
|
144
|
+
- **Credential redaction**: cloud `credentials` travel in the request body — never derive error text
|
|
145
|
+
from the request body, and mask the **whole `credentials` object** to `[REDACTED]` wherever a value
|
|
146
|
+
object could surface it: object inspection (`toString`/`repr`/`Debug`/`inspect`), any SDK-emitted
|
|
147
|
+
request log, and the decoded error body (belt-and-suspenders — the API does not echo credential
|
|
148
|
+
values). Also mask any `parameters` leaf whose key contains a sensitive token (`token, password,
|
|
149
|
+
passwd, secret, key, keyfile, credential, passphrase, sas, sig, signature`, case-insensitive).
|
|
150
|
+
Credentials ride in the plaintext body — user-attached request logging must redact its own.
|
data/lib/api2convert/client.rb
CHANGED
|
@@ -57,9 +57,18 @@ module Api2Convert
|
|
|
57
57
|
# remembered and applied automatically on download.
|
|
58
58
|
#
|
|
59
59
|
# @return [Result::ConversionResult]
|
|
60
|
+
# A {Model::CloudInput} imports the source straight from customer storage (a
|
|
61
|
+
# started job, like a remote URL). Pass +output_targets+ (a list of
|
|
62
|
+
# {Model::OutputTarget}) to deliver the output(s) to customer storage instead
|
|
63
|
+
# of producing a downloadable file — the job then completes with **no** local
|
|
64
|
+
# output and the returned result is not downloaded (calling `output`/`save`
|
|
65
|
+
# on it would have nothing to fetch). Output targets are attached to the
|
|
66
|
+
# conversion's `output_target` and never merged into +options+.
|
|
60
67
|
def convert(source, to, options = nil, category: nil, timeout: nil,
|
|
61
|
-
output_index: nil, filename: nil, download_password: nil
|
|
62
|
-
|
|
68
|
+
output_index: nil, filename: nil, download_password: nil,
|
|
69
|
+
output_targets: nil)
|
|
70
|
+
job = start_conversion(source, to, options, category, nil, filename,
|
|
71
|
+
download_password, output_targets)
|
|
63
72
|
done = @jobs.wait(job.id, timeout)
|
|
64
73
|
Result::ConversionResult.new(done, @transport, output_index.nil? ? 0 : output_index, download_password)
|
|
65
74
|
end
|
|
@@ -71,8 +80,9 @@ module Api2Convert
|
|
|
71
80
|
#
|
|
72
81
|
# @return [Model::Job]
|
|
73
82
|
def convert_async(source, to, options = nil, callback: nil, category: nil,
|
|
74
|
-
filename: nil, download_password: nil)
|
|
75
|
-
start_conversion(source, to, options, category, callback, filename,
|
|
83
|
+
filename: nil, download_password: nil, output_targets: nil)
|
|
84
|
+
start_conversion(source, to, options, category, callback, filename,
|
|
85
|
+
download_password, output_targets)
|
|
76
86
|
end
|
|
77
87
|
|
|
78
88
|
# A {Result::FileDownload} for an output file. A +download_password+ is
|
|
@@ -112,10 +122,16 @@ module Api2Convert
|
|
|
112
122
|
|
|
113
123
|
private
|
|
114
124
|
|
|
115
|
-
def start_conversion(source, to, options, category, callback, filename,
|
|
125
|
+
def start_conversion(source, to, options, category, callback, filename,
|
|
126
|
+
download_password, output_targets = nil)
|
|
116
127
|
conversion = { "target" => to }
|
|
117
128
|
conversion["category"] = category unless category.nil?
|
|
118
129
|
conversion["options"] = options if !options.nil? && !options.empty?
|
|
130
|
+
# Cloud delivery targets attach to the conversion's `output_target` — never
|
|
131
|
+
# merged into the options map.
|
|
132
|
+
unless output_targets.nil? || output_targets.empty?
|
|
133
|
+
conversion["output_target"] = Array(output_targets).map(&:to_h)
|
|
134
|
+
end
|
|
119
135
|
|
|
120
136
|
payload = { "conversion" => [conversion] }
|
|
121
137
|
unless callback.nil?
|
|
@@ -124,6 +140,14 @@ module Api2Convert
|
|
|
124
140
|
end
|
|
125
141
|
payload["download_passwords"] = [download_password] unless download_password.nil?
|
|
126
142
|
|
|
143
|
+
# A cloud input imports from customer storage — a started job with the
|
|
144
|
+
# descriptor inline, exactly like a remote URL (never staged/uploaded).
|
|
145
|
+
if source.is_a?(Model::CloudInput)
|
|
146
|
+
payload["process"] = true
|
|
147
|
+
payload["input"] = [source.to_h]
|
|
148
|
+
return @jobs.create(payload)
|
|
149
|
+
end
|
|
150
|
+
|
|
127
151
|
if source.is_a?(String) && source =~ URL_RE
|
|
128
152
|
payload["process"] = true
|
|
129
153
|
payload["input"] = [{ "type" => "remote", "source" => source }]
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Api2Convert
|
|
4
|
+
# The cloud storage providers the API can import inputs from and deliver
|
|
5
|
+
# outputs to — the values of a cloud descriptor's `source` (input) / `type`
|
|
6
|
+
# (output) field.
|
|
7
|
+
#
|
|
8
|
+
# This is **build-side vocabulary only**: it types the input builder
|
|
9
|
+
# ({Model::CloudInput}) and output-target serialization ({Model::OutputTarget}).
|
|
10
|
+
# Read models keep `source`/`type`/`status` as raw strings, so an unknown
|
|
11
|
+
# provider string returned by the server round-trips untyped and never raises —
|
|
12
|
+
# there is deliberately no strict parse.
|
|
13
|
+
#
|
|
14
|
+
# Import support (a {Model::CloudInput} factory) exists for {AMAZON_S3},
|
|
15
|
+
# {AZURE}, {FTP} and {GOOGLE_CLOUD}. {GDRIVE} and {YOUTUBE} are **output-only**
|
|
16
|
+
# (they validate as an output `type` but have no downloader); Google Drive
|
|
17
|
+
# *input* uses the separate `gdrive_picker` input type.
|
|
18
|
+
module CloudProvider
|
|
19
|
+
AMAZON_S3 = "amazons3"
|
|
20
|
+
AZURE = "azure"
|
|
21
|
+
FTP = "ftp"
|
|
22
|
+
GDRIVE = "gdrive"
|
|
23
|
+
GOOGLE_CLOUD = "googlecloud"
|
|
24
|
+
YOUTUBE = "youtube"
|
|
25
|
+
|
|
26
|
+
# The full provider vocabulary, in canonical order.
|
|
27
|
+
ALL = [AMAZON_S3, AZURE, FTP, GDRIVE, GOOGLE_CLOUD, YOUTUBE].freeze
|
|
28
|
+
end
|
|
29
|
+
end
|
|
@@ -13,7 +13,7 @@ module Api2Convert
|
|
|
13
13
|
# `Net::HTTP` does not follow redirects on its own — this SDK relies on that.
|
|
14
14
|
# A request only follows a 3xx when {Request#follow_redirects} is true (the
|
|
15
15
|
# no-secret download path); a redirect is then re-issued as a bare GET carrying
|
|
16
|
-
# only `Accept`/`User-Agent`, so a custom `X-
|
|
16
|
+
# only `Accept`/`User-Agent`, so a custom `X-Api2convert-*` secret header can never be
|
|
17
17
|
# forwarded to the redirect target.
|
|
18
18
|
#
|
|
19
19
|
# An {HttpSender} is any object responding to `call(request) -> Response`. Unit
|
|
@@ -23,6 +23,15 @@ module Api2Convert
|
|
|
23
23
|
MAX_REDIRECTS = 5
|
|
24
24
|
REDIRECT_CODES = [301, 302, 303, 307, 308].freeze
|
|
25
25
|
|
|
26
|
+
# Caps how much of a control-plane (API / error) JSON body the SDK buffers
|
|
27
|
+
# into memory, so a hostile or buggy server cannot force an unbounded read
|
|
28
|
+
# (OOM) on that path. Mirrors the shipped Go SDK's `maxResponseBytes = 16 <<
|
|
29
|
+
# 20` (transport.go), whose readAllAndClose reads through an
|
|
30
|
+
# io.LimitReader(rc, maxResponseBytes). File downloads are streamed straight
|
|
31
|
+
# to the sink (never buffered) and bounded separately, so this cap covers the
|
|
32
|
+
# buffered control-plane path only.
|
|
33
|
+
MAX_RESPONSE_BYTES = 16 * 1024 * 1024 # 16 MiB
|
|
34
|
+
|
|
26
35
|
# Errors worth surfacing when they strike mid-stream (after bytes have
|
|
27
36
|
# already reached the sink). They are re-raised as {NetworkError} so the
|
|
28
37
|
# transport does NOT retry — replaying would re-stream the whole body and
|
|
@@ -83,13 +92,13 @@ module Api2Convert
|
|
|
83
92
|
stream_body(res, sink)
|
|
84
93
|
result = to_response(res, "")
|
|
85
94
|
else
|
|
86
|
-
result = to_response(res, res
|
|
95
|
+
result = to_response(res, read_capped_body(res))
|
|
87
96
|
end
|
|
88
97
|
end
|
|
89
98
|
end
|
|
90
99
|
|
|
91
100
|
if redirect_to
|
|
92
|
-
# Re-issue as a bare GET carrying only non-secret headers, so no X-
|
|
101
|
+
# Re-issue as a bare GET carrying only non-secret headers, so no X-Api2convert-*
|
|
93
102
|
# secret header is ever forwarded to the redirect target.
|
|
94
103
|
safe = {}
|
|
95
104
|
%w[Accept User-Agent].each { |k| safe[k] = headers[k] unless headers[k].nil? }
|
|
@@ -100,6 +109,25 @@ module Api2Convert
|
|
|
100
109
|
result
|
|
101
110
|
end
|
|
102
111
|
|
|
112
|
+
# Buffer a control-plane (API / error) response body into memory, bounded by
|
|
113
|
+
# {MAX_RESPONSE_BYTES}. Because `Net::HTTP` is used in block form the body is
|
|
114
|
+
# not read until this runs, so — unlike an SDK-side cap applied after a whole
|
|
115
|
+
# `res.body` — we accumulate chunk-by-chunk and abort the instant the total
|
|
116
|
+
# crosses the cap, before an over-cap body is ever fully resident. Mirrors the
|
|
117
|
+
# Go SDK's io.LimitReader-wrapped read. Only the cap is raised here; genuine
|
|
118
|
+
# transport errors propagate exactly as `res.body` would, so the retry loop
|
|
119
|
+
# still classifies them.
|
|
120
|
+
def read_capped_body(res)
|
|
121
|
+
buffer = +""
|
|
122
|
+
res.read_body do |chunk|
|
|
123
|
+
buffer << chunk
|
|
124
|
+
next unless buffer.bytesize > MAX_RESPONSE_BYTES
|
|
125
|
+
|
|
126
|
+
raise Api2Convert::NetworkError, "API response body exceeds 16 MiB"
|
|
127
|
+
end
|
|
128
|
+
buffer
|
|
129
|
+
end
|
|
130
|
+
|
|
103
131
|
def stream_body(res, sink)
|
|
104
132
|
res.read_body { |chunk| sink.write(chunk) }
|
|
105
133
|
rescue *STREAM_ERRORS => e
|
|
@@ -11,7 +11,7 @@ module Api2Convert
|
|
|
11
11
|
class Request
|
|
12
12
|
attr_reader :method, :url, :headers, :body, :body_stream, :content_length, :response_sink
|
|
13
13
|
# Whether the sender may follow a 3xx redirect. Set by the transport per
|
|
14
|
-
# request: authenticated requests carry a secret in a custom `X-
|
|
14
|
+
# request: authenticated requests carry a secret in a custom `X-Api2convert-*` header,
|
|
15
15
|
# so they must NOT follow redirects (the header could leak to another host).
|
|
16
16
|
attr_accessor :follow_redirects
|
|
17
17
|
|
|
@@ -29,12 +29,14 @@ module Api2Convert
|
|
|
29
29
|
@response_sink = response_sink
|
|
30
30
|
end
|
|
31
31
|
|
|
32
|
-
# Redacted representation — {#headers} carries the raw `X-
|
|
33
|
-
# `X-
|
|
34
|
-
# them in cleartext in a log line or a backtrace. Mask every `X-
|
|
32
|
+
# Redacted representation — {#headers} carries the raw `X-Api2convert-Api-Key` /
|
|
33
|
+
# `X-Api2convert-Download-Password` secrets, so the default `#inspect` would print
|
|
34
|
+
# them in cleartext in a log line or a backtrace. Mask every `X-Api2convert-*` value
|
|
35
|
+
# (and the legacy `X-Oc-*` prefix, so a redaction gap can never open up).
|
|
35
36
|
def inspect
|
|
36
37
|
safe = @headers.to_h do |key, value|
|
|
37
|
-
|
|
38
|
+
secret = key.to_s.downcase.start_with?("x-api2convert-", "x-oc-")
|
|
39
|
+
[key, secret ? Support::Secret.mask(value) : value]
|
|
38
40
|
end
|
|
39
41
|
"#<#{self.class.name} method=#{@method.inspect} url=#{@url.inspect} headers=#{safe.inspect}>"
|
|
40
42
|
end
|
|
@@ -61,7 +61,7 @@ module Api2Convert
|
|
|
61
61
|
|
|
62
62
|
# Perform an authenticated JSON request and return the decoded body.
|
|
63
63
|
def request(method, path, body = nil, query = nil, headers = nil)
|
|
64
|
-
request_headers = { "X-
|
|
64
|
+
request_headers = { "X-Api2convert-Api-Key" => @config.api_key }
|
|
65
65
|
request_headers.merge!(headers) unless headers.nil?
|
|
66
66
|
content = nil
|
|
67
67
|
unless body.nil?
|
|
@@ -83,7 +83,7 @@ module Api2Convert
|
|
|
83
83
|
# a non-seekable body so it is sent once.
|
|
84
84
|
#
|
|
85
85
|
# +follow_redirects+ defaults to false: authenticated requests carry a secret
|
|
86
|
-
# in a custom `X-
|
|
86
|
+
# in a custom `X-Api2convert-*` header, which a redirect could leak to another host.
|
|
87
87
|
# Only the self-contained download path (no account key) opts in.
|
|
88
88
|
def send_request(build, replayable: true, follow_redirects: false)
|
|
89
89
|
attempt = 0
|
|
@@ -125,6 +125,15 @@ module Api2Convert
|
|
|
125
125
|
def interpret(response)
|
|
126
126
|
ensure_successful(response)
|
|
127
127
|
|
|
128
|
+
# Every API request rides the no-follow path (secrets travel in X-Api2convert-* headers), so a 3xx
|
|
129
|
+
# passes ensure_successful (status < 400) but was deliberately not followed; decoding its
|
|
130
|
+
# body would yield an empty model. Surface it as a typed error instead.
|
|
131
|
+
status = response.status
|
|
132
|
+
if status >= 300 && status < 400
|
|
133
|
+
raise NetworkError, "API2Convert returned an unexpected redirect (HTTP #{status}); " \
|
|
134
|
+
"the request was not followed."
|
|
135
|
+
end
|
|
136
|
+
|
|
128
137
|
raw = response.body
|
|
129
138
|
return {} if raw.nil? || raw.empty?
|
|
130
139
|
|
|
@@ -143,7 +152,13 @@ module Api2Convert
|
|
|
143
152
|
status = response.status
|
|
144
153
|
return if status < 400
|
|
145
154
|
|
|
146
|
-
body
|
|
155
|
+
# Belt-and-suspenders: deep-redact the decoded error body before it lands
|
|
156
|
+
# on the exception. Cloud credentials ride in the plaintext request body;
|
|
157
|
+
# the API only ever echoes field *names* (never a value), but a future
|
|
158
|
+
# server/proxy change must not be able to surface a secret through
|
|
159
|
+
# `error.body`. The `message` is server-provided text, never derived from
|
|
160
|
+
# the request body.
|
|
161
|
+
body = Support::Redactor.redact_body(decode_safe(response))
|
|
147
162
|
api_message = body["message"]
|
|
148
163
|
message = api_message.is_a?(String) ? api_message : fallback_message(response)
|
|
149
164
|
request_id = response.header("x-request-id")
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Api2Convert
|
|
4
|
+
module Model
|
|
5
|
+
# A cloud-storage input descriptor:
|
|
6
|
+
# `{ type:"cloud", source:<provider>, parameters, credentials }`.
|
|
7
|
+
#
|
|
8
|
+
# Hand it to `client.convert` / `convert_async` as the input, or to
|
|
9
|
+
# `client.jobs.add_input(job_id, cloud_input)`; either way it emits the wire
|
|
10
|
+
# descriptor via {#to_h}. Like a remote URL, a cloud input is a **started**
|
|
11
|
+
# job (`process => true`), not a staged upload.
|
|
12
|
+
#
|
|
13
|
+
# The per-provider factories carry each provider's required keys **verbatim**
|
|
14
|
+
# — flat and lowercase, exactly as the API expects (`accesskeyid`, not
|
|
15
|
+
# `access_key_id`). The required keys are constructor arguments (structural
|
|
16
|
+
# correctness), **not** a runtime gate: the builder never rejects a descriptor
|
|
17
|
+
# the permissive, asynchronously-validating server would accept. Optional and
|
|
18
|
+
# forward-compat keys go through the trailing +parameters+ / +credentials+
|
|
19
|
+
# maps, or the generic {.of} escape hatch.
|
|
20
|
+
#
|
|
21
|
+
# Google Drive *input* uses the `gdrive_picker` input type (the generic
|
|
22
|
+
# `add_input` raw-map path this wave); `gdrive`/`youtube` are output-only.
|
|
23
|
+
#
|
|
24
|
+
# `credentials` ride in the plaintext body, so {#inspect} masks the **whole**
|
|
25
|
+
# credentials object to `[REDACTED]` and any sensitive `parameters` leaf.
|
|
26
|
+
class CloudInput
|
|
27
|
+
attr_reader :source, :parameters, :credentials
|
|
28
|
+
|
|
29
|
+
# +source+ is the provider string; +parameters+ are non-secret locator keys
|
|
30
|
+
# (`bucket`, `file`, `host`, …); +credentials+ are secret keys.
|
|
31
|
+
def initialize(source:, parameters: {}, credentials: {})
|
|
32
|
+
@source = source
|
|
33
|
+
@parameters = parameters
|
|
34
|
+
@credentials = credentials
|
|
35
|
+
freeze
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
# Generic escape hatch: any provider (a {CloudProvider} constant or a
|
|
39
|
+
# forward-compat string) with free-form maps.
|
|
40
|
+
def self.of(source, parameters: {}, credentials: {})
|
|
41
|
+
new(source: source.to_s, parameters: parameters, credentials: credentials)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# Import from Amazon S3. Extra/forward-compat keys merge into
|
|
45
|
+
# +parameters+ / +credentials+.
|
|
46
|
+
def self.amazon_s3(bucket:, file:, accesskeyid:, secretaccesskey:,
|
|
47
|
+
parameters: {}, credentials: {})
|
|
48
|
+
new(
|
|
49
|
+
source: CloudProvider::AMAZON_S3,
|
|
50
|
+
parameters: { "bucket" => bucket, "file" => file }.merge(parameters),
|
|
51
|
+
credentials: { "accesskeyid" => accesskeyid, "secretaccesskey" => secretaccesskey }
|
|
52
|
+
.merge(credentials)
|
|
53
|
+
)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# Import from Azure Blob Storage.
|
|
57
|
+
def self.azure(container:, file:, accountname:, accountkey:,
|
|
58
|
+
parameters: {}, credentials: {})
|
|
59
|
+
new(
|
|
60
|
+
source: CloudProvider::AZURE,
|
|
61
|
+
parameters: { "container" => container, "file" => file }.merge(parameters),
|
|
62
|
+
credentials: { "accountname" => accountname, "accountkey" => accountkey }.merge(credentials)
|
|
63
|
+
)
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Import from an FTP server.
|
|
67
|
+
def self.ftp(host:, file:, username:, password:, parameters: {}, credentials: {})
|
|
68
|
+
new(
|
|
69
|
+
source: CloudProvider::FTP,
|
|
70
|
+
parameters: { "host" => host, "file" => file }.merge(parameters),
|
|
71
|
+
credentials: { "username" => username, "password" => password }.merge(credentials)
|
|
72
|
+
)
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
# Import from Google Cloud Storage.
|
|
76
|
+
def self.google_cloud(projectid:, bucket:, file:, keyfile:, parameters: {}, credentials: {})
|
|
77
|
+
new(
|
|
78
|
+
source: CloudProvider::GOOGLE_CLOUD,
|
|
79
|
+
parameters: { "projectid" => projectid, "bucket" => bucket, "file" => file }.merge(parameters),
|
|
80
|
+
credentials: { "keyfile" => keyfile }.merge(credentials)
|
|
81
|
+
)
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
# The wire descriptor sent to `POST /jobs` (inline `input`) or
|
|
85
|
+
# `POST /jobs/{id}/input`.
|
|
86
|
+
def to_h
|
|
87
|
+
{
|
|
88
|
+
"type" => InputType::CLOUD,
|
|
89
|
+
"source" => @source,
|
|
90
|
+
"parameters" => @parameters,
|
|
91
|
+
"credentials" => @credentials
|
|
92
|
+
}
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
# Redacted representation — the whole `credentials` object renders as
|
|
96
|
+
# `[REDACTED]`; sensitive `parameters` leaves are masked too. Safe to log.
|
|
97
|
+
def inspect
|
|
98
|
+
"#<#{self.class.name} type=cloud source=#{@source.inspect} " \
|
|
99
|
+
"parameters=#{Support::Redactor.parameters(@parameters).inspect} " \
|
|
100
|
+
"credentials=#{Support::Redactor::MARKER}>"
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def to_s
|
|
104
|
+
inspect
|
|
105
|
+
end
|
|
106
|
+
end
|
|
107
|
+
end
|
|
108
|
+
end
|
|
@@ -4,14 +4,17 @@ module Api2Convert
|
|
|
4
4
|
module Model
|
|
5
5
|
# A single conversion within a job: the target format plus its options.
|
|
6
6
|
class Conversion
|
|
7
|
-
attr_reader :target, :id, :category, :options, :metadata
|
|
7
|
+
attr_reader :target, :id, :category, :options, :metadata, :output_targets
|
|
8
8
|
|
|
9
|
-
def initialize(target: "", id: nil, category: nil, options: {}, metadata: {}
|
|
9
|
+
def initialize(target: "", id: nil, category: nil, options: {}, metadata: {},
|
|
10
|
+
output_targets: [])
|
|
10
11
|
@target = target
|
|
11
12
|
@id = id
|
|
12
13
|
@category = category
|
|
13
14
|
@options = options
|
|
14
15
|
@metadata = metadata
|
|
16
|
+
# Cloud delivery targets for this conversion's output, if any.
|
|
17
|
+
@output_targets = output_targets
|
|
15
18
|
freeze
|
|
16
19
|
end
|
|
17
20
|
|
|
@@ -22,7 +25,8 @@ module Api2Convert
|
|
|
22
25
|
id: Support::Data.nullable_str(d["id"]),
|
|
23
26
|
category: Support::Data.nullable_str(d["category"]),
|
|
24
27
|
options: Support::Data.as_object(d["options"]),
|
|
25
|
-
metadata: Support::Data.as_object(d["metadata"])
|
|
28
|
+
metadata: Support::Data.as_object(d["metadata"]),
|
|
29
|
+
output_targets: Support::Data.map_objects(d["output_target"]) { |x| OutputTarget.from_hash(x) }
|
|
26
30
|
)
|
|
27
31
|
end
|
|
28
32
|
end
|
|
@@ -4,10 +4,11 @@ module Api2Convert
|
|
|
4
4
|
module Model
|
|
5
5
|
# An input file attached to a job.
|
|
6
6
|
class InputFile
|
|
7
|
-
attr_reader :id, :type, :source, :status, :filename, :size, :content_type, :options
|
|
7
|
+
attr_reader :id, :type, :source, :status, :filename, :size, :content_type, :options,
|
|
8
|
+
:parameters
|
|
8
9
|
|
|
9
10
|
def initialize(id: nil, type: "", source: nil, status: nil, filename: nil,
|
|
10
|
-
size: nil, content_type: nil, options: {})
|
|
11
|
+
size: nil, content_type: nil, options: {}, parameters: {})
|
|
11
12
|
@id = id
|
|
12
13
|
@type = type
|
|
13
14
|
@source = source
|
|
@@ -16,6 +17,9 @@ module Api2Convert
|
|
|
16
17
|
@size = size
|
|
17
18
|
@content_type = content_type
|
|
18
19
|
@options = options
|
|
20
|
+
# Cloud-input locator keys (`bucket`, `file`, `host`, …); empty for
|
|
21
|
+
# non-cloud inputs. Credentials are never surfaced on read.
|
|
22
|
+
@parameters = parameters
|
|
19
23
|
freeze
|
|
20
24
|
end
|
|
21
25
|
|
|
@@ -29,7 +33,8 @@ module Api2Convert
|
|
|
29
33
|
filename: Support::Data.nullable_str(d["filename"]),
|
|
30
34
|
size: Support::Data.nullable_int(d["size"]),
|
|
31
35
|
content_type: Support::Data.nullable_str(d["content_type"]),
|
|
32
|
-
options: Support::Data.as_object(d["options"])
|
|
36
|
+
options: Support::Data.as_object(d["options"]),
|
|
37
|
+
parameters: Support::Data.as_object(d["parameters"])
|
|
33
38
|
)
|
|
34
39
|
end
|
|
35
40
|
end
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Api2Convert
|
|
4
|
+
module Model
|
|
5
|
+
# A cloud-storage delivery target for a conversion's output:
|
|
6
|
+
# `{ type:<provider>, parameters, credentials }`.
|
|
7
|
+
#
|
|
8
|
+
# Attach one (or more) to a conversion via
|
|
9
|
+
# `client.convert(..., output_targets: [...])` / `convert_async(...)`, or
|
|
10
|
+
# inline in a raw `jobs.create` conversion map. When any output target is set
|
|
11
|
+
# the conversion delivers straight to your storage and produces **no** local
|
|
12
|
+
# output — so `convert` returns the completed job without downloading.
|
|
13
|
+
#
|
|
14
|
+
# This wave ships the **generic** shape only (`type` + free-form
|
|
15
|
+
# `parameters`/`credentials`); the per-provider output keys live in a separate
|
|
16
|
+
# service and diverge per provider, so there are no per-provider output
|
|
17
|
+
# factories yet.
|
|
18
|
+
#
|
|
19
|
+
# Serialization ({#to_h}) emits `{ type, parameters, credentials }` and
|
|
20
|
+
# **omits `status`** (server-set, read-only). On read ({.from_hash}) `type`,
|
|
21
|
+
# `parameters` and `status` round-trip as raw values; `credentials` are
|
|
22
|
+
# **never** surfaced (the API returns them empty). `credentials` ride in the
|
|
23
|
+
# plaintext body, so {#inspect} masks the whole object to `[REDACTED]`.
|
|
24
|
+
class OutputTarget
|
|
25
|
+
attr_reader :type, :parameters, :credentials, :status
|
|
26
|
+
|
|
27
|
+
# +status+ is server-set on read (`waiting|uploading|completed|failed`) and
|
|
28
|
+
# never sent on create.
|
|
29
|
+
def initialize(type:, parameters: {}, credentials: {}, status: nil)
|
|
30
|
+
@type = type
|
|
31
|
+
@parameters = parameters
|
|
32
|
+
@credentials = credentials
|
|
33
|
+
@status = status
|
|
34
|
+
freeze
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# Generic constructor accepting a {CloudProvider} constant or a
|
|
38
|
+
# forward-compat string.
|
|
39
|
+
def self.of(type, parameters: {}, credentials: {})
|
|
40
|
+
new(type: type.to_s, parameters: parameters, credentials: credentials)
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# The wire descriptor sent on create — `{ type, parameters, credentials }`,
|
|
44
|
+
# with `status` omitted (server-set, read-only).
|
|
45
|
+
def to_h
|
|
46
|
+
{
|
|
47
|
+
"type" => @type,
|
|
48
|
+
"parameters" => @parameters,
|
|
49
|
+
"credentials" => @credentials
|
|
50
|
+
}
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
# Hydrate from a `GET /jobs/{id}` `output_target[]` element. `type`/`status`
|
|
54
|
+
# stay raw strings (an unknown provider round-trips untyped); `credentials`
|
|
55
|
+
# are deliberately not surfaced.
|
|
56
|
+
def self.from_hash(data)
|
|
57
|
+
d = Support::Data.as_object(data)
|
|
58
|
+
new(
|
|
59
|
+
type: Support::Data.as_str(d["type"]),
|
|
60
|
+
parameters: Support::Data.as_object(d["parameters"]),
|
|
61
|
+
credentials: {},
|
|
62
|
+
status: Support::Data.nullable_str(d["status"])
|
|
63
|
+
)
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
# Redacted representation — credentials masked. Safe to log.
|
|
67
|
+
def inspect
|
|
68
|
+
"#<#{self.class.name} type=#{@type.inspect} " \
|
|
69
|
+
"parameters=#{Support::Redactor.parameters(@parameters).inspect} " \
|
|
70
|
+
"credentials=#{Support::Redactor::MARKER} status=#{@status.inspect}>"
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def to_s
|
|
74
|
+
inspect
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
|
@@ -52,10 +52,14 @@ module Api2Convert
|
|
|
52
52
|
end
|
|
53
53
|
|
|
54
54
|
# Attach an input by descriptor, e.g. a remote URL:
|
|
55
|
-
# `add_input(job_id, { "type" => "remote", "source" => "https://..." })
|
|
55
|
+
# `add_input(job_id, { "type" => "remote", "source" => "https://..." })`,
|
|
56
|
+
# a Google Drive picker
|
|
57
|
+
# (`{ "type" => "gdrive_picker", "source" => file_id, "credentials" => { "token" => ... } }`),
|
|
58
|
+
# or a {Model::CloudInput} builder.
|
|
56
59
|
def add_input(job_id, descriptor)
|
|
60
|
+
payload = descriptor.is_a?(Model::CloudInput) ? descriptor.to_h : descriptor
|
|
57
61
|
Model::InputFile.from_hash(
|
|
58
|
-
@transport.request("POST", "/jobs/#{Support::Data.encode_segment(job_id)}/input",
|
|
62
|
+
@transport.request("POST", "/jobs/#{Support::Data.encode_segment(job_id)}/input", payload)
|
|
59
63
|
)
|
|
60
64
|
end
|
|
61
65
|
|