calmlens 0.20.0 → 0.22.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b3b2b65d6e8ec874bb4971fd4e4568e6cc424ecce6f060c6ce796bc2eb1a1c6d
4
- data.tar.gz: 54a14554c90bec933b2ebaf0482918f3902497c6bdff6d8204e6b374fb881aab
3
+ metadata.gz: 5fd7cb8e811f68c6be7989deb980a259d4a5643bc6f4402324c64236d258661b
4
+ data.tar.gz: f5e03fc720335e9c2ee2116549ddad9407a627449aed230f31eacf9a31743a3d
5
5
  SHA512:
6
- metadata.gz: 899100986bd5d7534a535a92f1af8c91a06fe8eadd123dfb9ef03a63d22193536c6d3bdaf72936905bee03379e88a3bb53e8197ae259fddec8b32e1d0eb951b8
7
- data.tar.gz: 33d8fa683ba8d59117f9be22611f0ba5a12b4b2d557a4bbf61cdfd3b782f1d4186a3197bdaf3e26df8e0ff07c8ed53ca79e0bdeb84feae3d1f9ce904976ae7c6
6
+ metadata.gz: ea06ee7b551c7e4a4336f3fb972e0e1181a55b5a148cb89c3b281dff789efe60289db280c18ffe700f1e883f763be60b10560b6a8321773c776aa40df2c57c13
7
+ data.tar.gz: b50571100307aaa792130cf4ca1d980232185613d9376a244b7857a7f052e0e43ffcbaa584fa661e869d349c3971e9d5199a07ac053844ee3ddbaa496db5aa81
data/README.md CHANGED
@@ -16,6 +16,8 @@ gem "calmlens"
16
16
 
17
17
  The gem has no runtime dependencies: it uses only the standard library (Net::HTTP, JSON, OpenSSL). It ships YARD documentation and RBS signatures in `sig/`, so Steep and other RBS tools can check your calls and the response keys you read.
18
18
 
19
+ Versions follow semantic versioning: see [Versioning and support](https://calmlens.com/docs/sdk#versioning-and-support).
20
+
19
21
  ## Quick start
20
22
 
21
23
  Create an API key and copy your project ID in the CalmLens portal, and keep both on your server.
@@ -67,6 +69,7 @@ New statuses and fields may appear in later API versions. The SDK returns them u
67
69
  - [Serve content from the CDN](https://calmlens.com/docs/sdk/ruby#cdn-methods), with [lenses, transforms and signed URLs](https://calmlens.com/docs/sdk/ruby/reference#cdn-urls) and [age-restricted delivery](https://calmlens.com/docs/guides/delivering-assets#age-restricted-assets).
68
70
  - [Organise assets in folders](https://calmlens.com/docs/sdk/ruby/reference#folders), [search them](https://calmlens.com/docs/guides/search) and [page through lists](https://calmlens.com/docs/sdk/ruby#pagination).
69
71
  - [Match images against hash lists](https://calmlens.com/docs/sdk/ruby/reference#hash-lists).
72
+ - Keep [upload presets](https://calmlens.com/docs/guides/upload-presets), project settings and metadata fields in your code: `upload_presets` (`list`, `each`, `ensure`, `delete`), `project_settings` (`get`, `update`) and `metadata_fields` (`list`, `ensure`, `delete`). `ensure` and `update` create or replace by key or ID, and the `revision` you read refuses a change made since.
70
73
  - [Read the events feed](https://calmlens.com/docs/sdk/ruby#pagination) and [verify webhooks in Rails or Rack](https://calmlens.com/docs/sdk/ruby#webhooks).
71
74
  - [Handle errors](https://calmlens.com/docs/sdk/ruby/reference#errors) with automatic [retries, timeouts and idempotency keys](https://calmlens.com/docs/sdk/ruby/reference#retries-and-timeouts).
72
75
  - [Type-check your code with RBS signatures](https://calmlens.com/docs/sdk/ruby/reference#types).
@@ -77,7 +80,7 @@ Every method is listed in the [Ruby SDK documentation](https://calmlens.com/docs
77
80
 
78
81
  `CalmLens::MockServerClient` from `calmlens/testing` runs the real client against an in-memory stand-in for the API and CDN, so you can test your integration offline. See [Ruby testing](https://calmlens.com/docs/sdk/ruby/testing).
79
82
 
80
- `CalmLens::TestInputs` signs test inputs with your project's test input key from the portal: `generate_text_code` turns a scenario, preset or signal into a text code, `paint_image` paints it as a test image, and `generate_media_code` gives a phrase for audio and video, so the real API returns the scenario you declared.
83
+ `CalmLens::TestInputs` signs test inputs with your project's test input key from the portal: `generate_text_code` turns a scenario, preset or signal into a text code, `paint_image` paints it as a test image, and `generate_media_code` gives a phrase for audio and video, so the real API returns the scenario you declared. Submit a text code as the whole text and a test image as the whole upload.
81
84
 
82
85
  ## Links
83
86
 
@@ -15,30 +15,30 @@ module CalmLens
15
15
  # The prefix of this resource's operations in Internal::OPTION_KEYS.
16
16
  OPERATIONS = 'assets'
17
17
 
18
- SUBMIT_QUERY_KEYS = %w[name description processingOptions visibility metadata authorId context externalId externalUrl
19
- folderId uploadPresetId].freeze
18
+ # The asset fields an upload carries in its body: the JSON body for text, the `options` part of
19
+ # an upload form for files. They never go in the URL, which proxies and servers log, and the
20
+ # API refuses them there.
21
+ UPLOAD_OPTION_KEYS = %w[name description processingOptions visibility metadata tags authorId context externalId
22
+ externalUrl folderId uploadPresetId].freeze
20
23
  MAX_MULTIPART_IMAGE_BYTES = 20 * 1024 * 1024
21
24
  # The API's limits for one-request uploads: images up to 10 MiB (larger ones are uploaded in
22
25
  # parts), audio and video up to 50 MiB and other files up to 5,000 MiB.
23
26
  MAX_DIRECT_IMAGE_BYTES = 10 * 1024 * 1024
24
27
  MAX_MEDIA_BYTES = 50 * 1024 * 1024
25
28
  MAX_FILE_BYTES = 5000 * 1024 * 1024
29
+ # Audio and video sent in parts can be up to 2 GiB; the API holds free uploads, made before
30
+ # billing is set up, to MAX_MEDIA_BYTES itself.
31
+ MAX_LONG_MEDIA_BYTES = 2 * 1024 * 1024 * 1024
26
32
  # The longest `idempotency_key` of an upload sent in parts: each part's key adds `:part-<n>`.
27
33
  MAX_SPLIT_IDEMPOTENCY_KEY_LENGTH = 240
28
- # The moderation status before a decision; {#wait_for_decision} returns once an asset has any
29
- # other, or once its classification status is in {FINISHED_PROCESSING_STATUSES}.
30
- PENDING_MODERATION_STATUS = 'pending'
31
- # Processing statuses after which no decision will arrive without a retry.
32
- FINISHED_PROCESSING_STATUSES = %w[failed].freeze
34
+ # The classification statuses of a settled asset: `completed` (with its decision in
35
+ # `classification.moderation.status`) or `failed`. {#wait_for_decision} returns at the first.
36
+ SETTLED_STATUSES = %w[completed failed].freeze
33
37
  DECISION_POLL_FIRST_DELAY = 0.5
34
38
  # Each wait between decision checks is this many times the one before, up to DECISION_POLL_MAX_DELAY.
35
39
  DECISION_POLL_FACTOR = 1.5
36
40
  DECISION_POLL_MAX_DELAY = 2.0
37
41
  DEFAULT_DECISION_TIMEOUT = 60
38
- # The lifetime of a signed URL when none is given: 15 minutes.
39
- DEFAULT_SIGNED_URL_SECONDS = 15 * 60
40
- # The longest signed URL lifetime the API accepts: 24 hours.
41
- MAX_SIGNED_URL_SECONDS = 24 * 60 * 60
42
42
  # The operations {#batch} runs: `delete` deletes each asset with its child assets and stored
43
43
  # content; `remove-content` removes its stored content and keeps the record.
44
44
  BATCH_OPERATIONS = %w[delete remove-content].freeze
@@ -48,18 +48,20 @@ module CalmLens
48
48
  # Searches assets by meaning or keyword.
49
49
  # @option options [String] :query
50
50
  # @option options [String] :mode `hybrid`, `semantic` or `keyword`
51
- # @option options [Integer] :limit
51
+ # @option options [Integer] :page_size how many results, 1 to 20 (default 10)
52
52
  # @return [Hash] `{'query', 'items' => [{'asset', 'score', 'matchedFields'}], 'semanticStatus'}`
53
53
  def search(options = {})
54
54
  call(options, __method__) { |api, request| request('GET', project('assets/search'), query: api, request_options: request) }
55
55
  end
56
56
 
57
- # Lists one page of assets.
57
+ # Lists one page of assets, newest first unless `sort` says otherwise.
58
58
  # @option options [String] :cursor from the previous page
59
59
  # @option options [Integer] :page_size 1 to 100, default 50
60
- # @option options [String] :filter an RSQL filter such as `classification.moderation.status==allowed`
61
- # @option options [Array<Hash>] :ordering `[{field: 'createdAt', direction: 'desc'}]`
62
- # @return [Hash] a cursor page: items, nextCursor, hasMore, pageSize and ordering.
60
+ # @option options [String, nil] :filter an RSQL filter such as `classification.moderation.status==allowed`;
61
+ # `folderId==fldr_…` lists one folder's assets and `folderId==null` the project root's
62
+ # @option options [String] :sort `createdAt` and `id`, comma-separated, descending when prefixed
63
+ # with `-`, such as `-createdAt,id`; `-createdAt` by default
64
+ # @return [Hash] a cursor page: items, nextCursor and hasMore.
63
65
  def list(options = {})
64
66
  call(options, __method__) do |api, request|
65
67
  request('GET', project('assets'), query: api, request_options: request)
@@ -73,14 +75,14 @@ module CalmLens
73
75
  def each(options = {}, &block)
74
76
  return enum_for(:each, options) unless block
75
77
 
76
- api, request = call(options, __method__) { |*split| split }
77
- listing = pages(api['cursor']) { |index| list(api.merge('cursor' => index, 'request_options' => request)) }
78
- listing.each { |page| page['items'].each(&block) }
79
- nil
78
+ each_item(options, __method__, method(:list), &block)
80
79
  end
81
80
 
82
81
  # Lists one page of an asset's children, such as frames extracted from a video.
83
82
  # @option options [String] :asset_id
83
+ # @option options [String] :cursor
84
+ # @option options [Integer] :page_size
85
+ # @option options [String] :sort as for {#list}
84
86
  # @return [Hash] a page like {#list}
85
87
  def list_children(options)
86
88
  call(options, __method__) do |api, request|
@@ -89,6 +91,30 @@ module CalmLens
89
91
  end
90
92
  end
91
93
 
94
+ # Lists one page of an audio or video asset's findings timeline, in time order.
95
+ # `classification.moderation.timeline.segmentCount` says how many segments there are. Pass each
96
+ # page's `nextCursor` back unchanged; cursors are opaque.
97
+ # @option options [String] :asset_id
98
+ # @option options [String] :cursor
99
+ # @option options [Integer] :page_size
100
+ # @return [Hash] a cursor page of timeline segments: items, nextCursor and hasMore.
101
+ def list_timeline(options)
102
+ call(options, __method__) do |api, request|
103
+ query = without(api, 'assetId')
104
+ request('GET', project("assets/#{Internal.segment(api['assetId'])}/timeline"), query: query, request_options: request)
105
+ end
106
+ end
107
+
108
+ # Iterates every segment of an audio or video asset's findings timeline, in time order.
109
+ # @param options [Hash] the options of {#list_timeline}
110
+ # @yieldparam segment [Hash]
111
+ # @return [Enumerator, nil] an Enumerator when no block is given.
112
+ def each_timeline_segment(options, &block)
113
+ return enum_for(:each_timeline_segment, options) unless block
114
+
115
+ each_item(options, __method__, method(:list_timeline), &block)
116
+ end
117
+
92
118
  # Iterates every child of an asset, fetching pages as needed.
93
119
  # @param options [Hash] the options of {#list_children}
94
120
  # @yieldparam asset [Hash]
@@ -96,10 +122,7 @@ module CalmLens
96
122
  def each_child(options, &block)
97
123
  return enum_for(:each_child, options) unless block
98
124
 
99
- api, request = call(options, __method__) { |*split| split }
100
- listing = pages(api['cursor']) { |index| list_children(api.merge('cursor' => index, 'request_options' => request)) }
101
- listing.each { |page| page['items'].each(&block) }
102
- nil
125
+ each_item(options, __method__, method(:list_children), &block)
103
126
  end
104
127
 
105
128
  # @option options [String] :asset_id
@@ -110,18 +133,19 @@ module CalmLens
110
133
  end
111
134
  end
112
135
 
113
- # Waits until an asset's `classification.moderation.status` is no longer `pending` (`allowed`,
114
- # `limited`, `age-restricted`, `blocked` or `purged`), or its `classification.status` is `failed`, and returns
115
- # the asset. It retrieves the asset, waiting 0.5 seconds before the second check and 1.5 times
116
- # longer each time after that, up to 2 seconds.
136
+ # Waits until an asset is settled, its `classification.status` `completed` (with its decision in
137
+ # `classification.moderation.status`) or `failed`, and returns the asset. A `queued` or `processing`
138
+ # asset is still waiting, even when a reviewer has already decided it. It retrieves the asset,
139
+ # waiting 0.5 seconds before the second check and 1.5 times longer each time after that, up to 2
140
+ # seconds.
117
141
  # Webhooks report decisions without polling.
118
142
  #
119
143
  # `timeout:` is the whole wait in seconds; `request_options: {timeout:}` still limits each request.
120
144
  # To stop waiting early, run the call in a thread you can stop: it only sleeps and sends requests.
121
145
  # @option options [String] :asset_id
122
146
  # @option options [Numeric] :timeout seconds to wait for a decision (default 60)
123
- # @return [Hash] the decided asset
124
- # @raise [TimeoutError] `sdk/timeout` when no decision arrives in time; `details` holds
147
+ # @return [Hash] the settled asset
148
+ # @raise [TimeoutError] `sdk/timeout` when the asset isn't settled in time; `details` holds
125
149
  # `'assetId'` and the last `'classificationStatus'` and `'moderationStatus'`.
126
150
  def wait_for_decision(options)
127
151
  call(options, __method__) do |api, request|
@@ -155,7 +179,7 @@ module CalmLens
155
179
 
156
180
  # Records a reviewer's decision, replacing the policy's: `classification.moderation.status`
157
181
  # becomes the decision, the decision and note join `classification.moderation.manualReviews` and
158
- # the matching `asset.<status>` event is sent with `decided-by: reviewer`. An asset still pending
182
+ # the matching `asset.<status>` event is sent with the API key as its `actor`. An asset still pending
159
183
  # moderation can only be blocked or purged. Purging needs `confirm_purge: true` and the
160
184
  # `assets:delete` permission, and permanently deletes the stored content. Not retried: it carries
161
185
  # no `Idempotency-Key`.
@@ -173,8 +197,12 @@ module CalmLens
173
197
 
174
198
  # Completes the policy evidence behind a decided asset's moderation decision: the report's
175
199
  # `classification.moderation.explanationComplete` becomes `true` and every action stays as decided.
176
- # With `rerun: true` the asset is moderated again with the project's current policy and returns
177
- # in `processing`; use it when the policy changed since the decision (`asset/policy-changed`).
200
+ # With `rerun: true` the asset is moderated again with the project's current policy and an
201
+ # `asset.rerun-requested` event is recorded; use it when the policy changed since the decision
202
+ # (`asset/policy-changed`). The asset returns unchanged: it keeps its decision until the re-run's
203
+ # is stored, reported by `asset.classified` and `asset.<status>` events, and a failed re-run keeps
204
+ # it. A reviewed asset raises `asset/reviewed`, and one whose content is gone
205
+ # `asset/content-unavailable`.
178
206
  # @option options [String] :asset_id
179
207
  # @option options [Boolean] :rerun
180
208
  # @return [Hash] the asset
@@ -189,13 +217,14 @@ module CalmLens
189
217
  # `requested`); the record and classification results are kept.
190
218
  # @option options [String] :asset_id
191
219
  # @return [Hash] the asset
192
- def delete_content(options)
220
+ def remove_content(options)
193
221
  call(options, __method__) do |api, request|
194
222
  request('DELETE', "/assets/#{Internal.segment(api['assetId'])}/content", request_options: request)
195
223
  end
196
224
  end
197
225
 
198
- # Deletes the asset record and its stored content.
226
+ # Deletes the asset record and its stored content. An asset extracted from another (one with a
227
+ # `parentId`) raises `asset/child-not-deletable`: delete its parent instead.
199
228
  # @option options [String] :asset_id
200
229
  # @return [nil]
201
230
  def delete(options)
@@ -207,12 +236,14 @@ module CalmLens
207
236
 
208
237
  # Deletes up to 500 assets (`operation: 'delete'`) or removes their stored content
209
238
  # (`'remove-content'`) in one request. Every asset must exist in the project and the key needs
210
- # `assets:delete` for its folder, or nothing is queued. Returns once the operation is queued; the
239
+ # `assets:delete` for its folder, and a deletion can't name an asset extracted from another
240
+ # (`asset/child-not-deletable`), or nothing is queued. Returns once the operation is queued; the
211
241
  # work runs in the background and records `asset.deleted` or `asset.content-removed` for each
212
- # asset. Assets under a safety hold are skipped. Not retried: it carries no `Idempotency-Key`.
242
+ # asset, with a `batch-id` fact naming `batchId`. Assets under a safety hold are skipped. Not
243
+ # retried: it carries no `Idempotency-Key`.
213
244
  # @option options [Array<String>] :asset_ids 1 to 500 asset IDs
214
245
  # @option options [String] :operation one of {BATCH_OPERATIONS}
215
- # @return [Hash] `{'queued' => true}`
246
+ # @return [Hash] `{'batchId', 'status' => 'queued', 'assetCount'}`
216
247
  def batch(options)
217
248
  call(options, __method__) do |api, request|
218
249
  ids = api['assetIds']
@@ -238,27 +269,8 @@ module CalmLens
238
269
  end
239
270
  end
240
271
 
241
- # Creates a time-limited URL that delivers private content without an API key.
242
- # @option options [String] :asset_id
243
- # @option options [String] :resource `rendered` (the API's default) or `original`, which takes no lens or transform
244
- # @option options [Integer] :expires_in_seconds default 900
245
- # @option options [String] :lens a lens identifier
246
- # @option options [String, Hash] :transform a selector String or structured options, as for
247
- # {CDN#get_asset_url}; they replace the lens's options of the same name
248
- # @return [Hash] `{'url', 'expiresAt'}`
249
- def create_signed_url(options)
250
- call(options, __method__) do |api, request|
251
- path = "/assets/#{Internal.segment(api['assetId'])}/signed-url"
252
- body = { 'expiresInSeconds' => DEFAULT_SIGNED_URL_SECONDS }.merge(without(api, 'assetId'))
253
- check_signed_url(body)
254
- body['transform'] = Transforms.selector(body['transform']) if body.key?('transform')
255
- request('POST', path,
256
- body: body,
257
- request_options: request)
258
- end
259
- end
260
-
261
- # Uploads content and queues moderation; the call returns before classification finishes.
272
+ # Uploads content and returns without waiting for its decision (`wait=false`), with
273
+ # `classification.status` `queued`.
262
274
  # Watch for `asset.allowed`, `asset.limited`, `asset.age-restricted`, `asset.blocked` and
263
275
  # `asset.purged` events or retrieve the asset later.
264
276
  #
@@ -267,6 +279,11 @@ module CalmLens
267
279
  # `{type:, data:}` where `data` is a binary String or an IO. An IO streams without being read into
268
280
  # memory; pass `size_in_bytes:` when its length is unknown (a pipe or socket).
269
281
  #
282
+ # Text is sent as a JSON body. Other content is sent as `multipart/form-data`: the asset's fields
283
+ # (name, description, metadata, author ID, context and so on) in its `options` part, then the
284
+ # file. The URL carries only transport values (`wait`, the content type and sizes), never the
285
+ # asset's fields.
286
+ #
270
287
  # The request carries an `Idempotency-Key` (generated unless you pass
271
288
  # `request_options: {idempotency_key:}`), so a retried upload never creates a second asset.
272
289
  #
@@ -281,6 +298,7 @@ module CalmLens
281
298
  # @option options [Hash] :part `{type:, data:, is_final:}` for the first multipart part
282
299
  # @option options [Integer] :total_size_in_bytes the whole file's size, for multipart uploads
283
300
  # @option options [String] :url
301
+ # @option options [Array<String>] :tags tags for the asset, such as `['catalogue', 'backpacks']`
284
302
  # @option options [String] :author_id your identifier for who wrote the content; CalmLens only
285
303
  # compares it with the author IDs in `context`, to tell which posts share a speaker.
286
304
  # @option options [Hash] :context `{items: [{text:, author_id:}]}`, earlier posts in the
@@ -288,7 +306,7 @@ module CalmLens
288
306
  # {ModerationContext::USED_ITEMS} that fit in {ModerationContext::USED_CHARACTERS} characters.
289
307
  # @return [Hash] the asset
290
308
  def create(options)
291
- call(options, __method__) { |api, request| submit(api, request, true) }
309
+ call(options, __method__) { |api, request| submit(api, request, false) }
292
310
  end
293
311
 
294
312
  # Uploads content and waits for the moderation decision where the content type supports it.
@@ -303,15 +321,17 @@ module CalmLens
303
321
  raise Internal.invalid('Text to classify must be a non-empty string', ErrorCodes::INVALID_UPLOAD_PAYLOAD)
304
322
  end
305
323
 
306
- submit(api, request, false)
324
+ submit(api, request, true)
307
325
  end
308
326
  end
309
327
 
310
328
  # Uploads a later part of a multipart upload started with {#create} or {#classify} and `part:`.
311
- # The queue mode chosen at the start applies; pass `queue:` only to override it.
329
+ # Whether the final part waits for the decision was chosen when the upload started; pass `wait:`
330
+ # only to override it.
312
331
  # @option options [String] :asset_id
313
332
  # @option options [Hash] :part `{data:, part_number:, is_final:}`; part numbers start at 2.
314
- # @option options [Boolean] :queue
333
+ # @option options [Boolean] :wait `true` to wait for the decision, `false` to return with the
334
+ # asset `queued`
315
335
  # @return [Hash] the asset; `classification.status` stays `uploading` until the final part.
316
336
  def upload_part(options)
317
337
  call(options, __method__) do |api, request|
@@ -326,7 +346,7 @@ module CalmLens
326
346
  end
327
347
 
328
348
  body = UploadBody.from(part, multipart: true, part_size: !@session.defer_upload_checks)
329
- send_part(api['assetId'], number, part.fetch('isFinal', false), body, api.fetch('queue', nil), request)
349
+ send_part(api['assetId'], number, part.fetch('isFinal', false), body, api.fetch('wait', nil), request)
330
350
  end
331
351
  end
332
352
 
@@ -342,12 +362,12 @@ module CalmLens
342
362
  asset = retrieve('assetId' => asset_id, 'request_options' => request_options)
343
363
  processing = asset.dig('classification', 'status')
344
364
  moderation = asset.dig('classification', 'moderation', 'status')
345
- return asset if moderation != PENDING_MODERATION_STATUS || FINISHED_PROCESSING_STATUSES.include?(processing)
365
+ return asset if SETTLED_STATUSES.include?(processing)
346
366
 
347
367
  remaining = deadline - @session.monotonic
348
368
  unless remaining.positive?
349
369
  raise TimeoutError.new(SdkErrorCodes::TIMEOUT,
350
- "Asset #{asset['id']} had no moderation decision after #{timeout}s " \
370
+ "Asset #{asset['id']} was not settled after #{timeout}s " \
351
371
  "(processing #{processing}, moderation #{moderation})",
352
372
  details: { 'assetId' => asset['id'], 'classificationStatus' => processing,
353
373
  'moderationStatus' => moderation })
@@ -358,7 +378,7 @@ module CalmLens
358
378
  end
359
379
  end
360
380
 
361
- def submit(options, request_options, queue)
381
+ def submit(options, request_options, wait)
362
382
  raise Internal.invalid('Asset name is required', ErrorCodes::INVALID_UPLOAD_PAYLOAD) if options['name'].to_s.empty?
363
383
  unless %w[content part url].one? { |key| options.key?(key) } && !options.key?('isFinal')
364
384
  raise Internal.invalid('Supply exactly one of content, part, or url; isFinal belongs in part',
@@ -366,7 +386,7 @@ module CalmLens
366
386
  end
367
387
  options = with_used_context(options)
368
388
  if options.key?('url')
369
- return expect_asset(request('POST', project('assets'), query: { 'queue' => queue }, body: options,
389
+ return expect_asset(request('POST', project('assets'), query: { 'wait' => wait }, body: options,
370
390
  idempotent: true, request_options: request_options))
371
391
  end
372
392
 
@@ -380,16 +400,40 @@ module CalmLens
380
400
  end
381
401
 
382
402
  body = UploadBody.from(content, multipart: multipart, part_size: !@session.defer_upload_checks)
383
- return upload_in_parts(options, request_options, queue, mime, body) if !multipart && splits?(content, body)
403
+ return upload_in_parts(options, request_options, wait, mime, body) if !multipart && splits?(content, body)
404
+
405
+ check_upload_size(mime, body.size) unless multipart || @session.defer_upload_checks
406
+ part_query = first_part_query(options, content, mime, body) if multipart
407
+ if content.key?('plainText')
408
+ return expect_asset(request('POST', project('assets'), query: { 'wait' => wait },
409
+ body: text_body(options, content, multipart),
410
+ idempotent: true, request_options: request_options))
411
+ end
412
+
413
+ query = { 'wait' => wait, 'contentType' => content['type'], 'sizeInBytes' => body.size }
414
+ send_form(query.merge(part_query || {}), upload_fields(options), content['type'], body, multipart, request_options)
415
+ end
416
+
417
+ # The asset fields an upload sends, without unset values.
418
+ def upload_fields(options) = options.slice(*UPLOAD_OPTION_KEYS).compact
384
419
 
385
- check_direct_size(mime, body.size) unless multipart || @session.defer_upload_checks
386
- query = options.slice(*SUBMIT_QUERY_KEYS).merge('queue' => queue, 'contentType' => content['type'],
387
- 'sizeInBytes' => body.size)
388
- query.merge!(first_part_query(options, content, mime, body)) if multipart
420
+ # The JSON body of a text upload: its fields and `content`, or a first `part` with the sizes.
421
+ def text_body(options, content, multipart)
422
+ text = content.except('sizeInBytes')
423
+ return upload_fields(options).merge('content' => text) unless multipart
424
+
425
+ sizes = { 'sizeInBytes' => options['sizeInBytes'] || content['sizeInBytes'],
426
+ 'totalSizeInBytes' => options['totalSizeInBytes'] }.compact
427
+ upload_fields(options).merge('part' => text, **sizes)
428
+ end
429
+
430
+ # Sends file content, or a first part, as an upload form: the fields in its `options` part and
431
+ # the bytes in its `file` part. The query holds only transport values.
432
+ def send_form(query, fields, type, body, multipart, request_options)
433
+ form, form_type = body.to_form(JSON.generate(fields), type)
389
434
  expect_asset(request('POST', project('assets'),
390
- query: query, raw: body, idempotent: true, request_options: request_options,
391
- upload: !content.key?('plainText'),
392
- headers: { 'Content-Type' => content['type'], 'X-CalmLens-Upload' => multipart ? 'part' : 'content' }))
435
+ query: query, raw: form, idempotent: true, request_options: request_options, upload: true,
436
+ headers: { 'Content-Type' => form_type, 'X-CalmLens-Upload' => multipart ? 'part' : 'content' }))
393
437
  end
394
438
 
395
439
  # Whether content goes in parts: not text, of known size, readable again and larger than a part.
@@ -400,14 +444,14 @@ module CalmLens
400
444
  # Uploads `body` as a multipart upload of `part_size_bytes` parts and returns the final part's
401
445
  # asset. Part `n` after the first carries the key `<key>:part-<n>`, so repeating the call with
402
446
  # the same key replays the parts the API already has and sends the rest.
403
- def upload_in_parts(options, request_options, queue, mime, body)
447
+ def upload_in_parts(options, request_options, wait, mime, body)
404
448
  key = request_options[:idempotency_key] || SecureRandom.uuid
405
449
  if key.length > MAX_SPLIT_IDEMPOTENCY_KEY_LENGTH
406
450
  raise Internal.invalid("idempotency_key can have at most #{MAX_SPLIT_IDEMPOTENCY_KEY_LENGTH} characters " \
407
451
  'for content uploaded in parts')
408
452
  end
409
453
  unless @session.defer_upload_checks
410
- mime.start_with?('image/') ? check_image_size(body.size) : check_direct_size(mime, body.size)
454
+ mime.start_with?('image/') ? check_image_size(body.size) : check_upload_size(mime, body.size, in_parts: true)
411
455
  end
412
456
 
413
457
  total = body.size
@@ -416,12 +460,9 @@ module CalmLens
416
460
  piece = ->(number) { body.slice((number - 1) * part_size, [part_size, total - ((number - 1) * part_size)].min) }
417
461
  first = piece.call(1)
418
462
  type = options.dig('content', 'type')
419
- query = options.slice(*SUBMIT_QUERY_KEYS).merge('queue' => queue, 'contentType' => type, 'sizeInBytes' => first.size,
420
- 'partNumber' => 1, 'isFinal' => false, 'totalSizeInBytes' => total)
421
- asset = expect_asset(request('POST', project('assets'),
422
- query: query, raw: first, idempotent: true, upload: true,
423
- request_options: request_options.merge(idempotency_key: key),
424
- headers: { 'Content-Type' => type, 'X-CalmLens-Upload' => 'part' }))
463
+ query = { 'wait' => wait, 'contentType' => type, 'sizeInBytes' => first.size, 'partNumber' => 1, 'isFinal' => false,
464
+ 'totalSizeInBytes' => total }
465
+ asset = send_form(query, upload_fields(options), type, first, true, request_options.merge(idempotency_key: key))
425
466
  send_remaining_parts(asset, count, piece, key, request_options)
426
467
  end
427
468
 
@@ -452,16 +493,16 @@ module CalmLens
452
493
  asset
453
494
  end
454
495
 
455
- def send_part(asset_id, number, final, body, queue, request_options)
496
+ def send_part(asset_id, number, final, body, wait, request_options)
456
497
  query = { 'partNumber' => number, 'isFinal' => final, 'sizeInBytes' => body.size }
457
- query['queue'] = queue unless queue.nil?
498
+ query['wait'] = wait unless wait.nil?
458
499
  headers = { 'Content-Type' => 'application/octet-stream', 'X-CalmLens-Upload' => 'part' }
459
500
  expect_asset(request('POST', "/assets/#{Internal.segment(asset_id)}/part", query: query, raw: body, idempotent: true,
460
501
  request_options: request_options, upload: true,
461
502
  headers: headers))
462
503
  end
463
504
 
464
- # Keeps only the earlier posts CalmLens uses, which keeps a raw upload's query short.
505
+ # Keeps only the earlier posts CalmLens uses, which keeps an upload's body short.
465
506
  def with_used_context(options)
466
507
  return options unless options.key?('context')
467
508
 
@@ -471,7 +512,8 @@ module CalmLens
471
512
  options.merge('context' => { 'items' => ModerationContext.select_items(context['items']) })
472
513
  end
473
514
 
474
- # The query of a first multipart part: its number, whether it is the last and the declared total.
515
+ # The transport values of a first multipart part: its number, whether it is the last and the
516
+ # declared total.
475
517
  def first_part_query(options, content, mime, body)
476
518
  total = options['totalSizeInBytes']
477
519
  unless total.nil? || (total.is_a?(Integer) && total.positive?)
@@ -479,7 +521,10 @@ module CalmLens
479
521
  end
480
522
 
481
523
  final = content.fetch('isFinal', false)
482
- check_image_size(total || body.size) if mime.start_with?('image/') && !@session.defer_upload_checks
524
+ unless @session.defer_upload_checks
525
+ # A declared whole-file size over the limit for parts can never finish.
526
+ mime.start_with?('image/') ? check_image_size(total || body.size) : check_upload_size(mime, total, in_parts: true)
527
+ end
483
528
  { 'partNumber' => 1, 'isFinal' => final, 'totalSizeInBytes' => total }
484
529
  end
485
530
 
@@ -495,45 +540,34 @@ module CalmLens
495
540
  raise InvalidResponseError.new(SdkErrorCodes::INVALID_RESPONSE, 'Expected an asset response', status: 200)
496
541
  end
497
542
 
498
- def check_signed_url(body)
499
- resource = body.fetch('resource', 'rendered')
500
- raise Internal.invalid("A signed URL's resource must be original or rendered") unless %w[original rendered].include?(resource)
501
- if resource == 'original' && (body.key?('lens') || body.key?('transform'))
502
- raise Internal.invalid('Lenses and transforms apply only to the rendered resource')
503
- end
504
-
505
- expires = body['expiresInSeconds']
506
- return if expires.is_a?(Integer) && expires.between?(1, MAX_SIGNED_URL_SECONDS)
507
-
508
- raise Internal.invalid('expires_in_seconds must be a whole number from 1 to 86400')
509
- end
510
-
511
543
  def check_mime(type)
512
544
  mime = type.to_s.split(';').first.to_s.strip.downcase
513
545
  unless Internal.accepted_upload_mime_type?(mime)
514
- raise InvalidArgumentError.new(ErrorCodes::INVALID_UPLOAD_MIME_TYPE,
546
+ raise InvalidArgumentError.new(ErrorCodes::UNSUPPORTED_UPLOAD_TYPE,
515
547
  "#{mime} is not a valid upload file type. Uploads must use a concrete file MIME " \
516
548
  'type that CalmLens can store and process, such as image/png, image/jpeg, ' \
517
- 'application/pdf, or text/plain.', details: { 'mimeType' => type })
549
+ 'application/pdf, or text/plain.', details: { 'mimeType' => type, 'source' => 'calmlens' })
518
550
  end
519
551
  raise Internal.invalid('Content type cannot contain line breaks') unless type.match?(Internal::HEADER_VALUE)
520
552
 
521
553
  mime
522
554
  end
523
555
 
524
- # Refuses one-request content whose known size is over the API's limit for its type, before
525
- # any of it is sent.
526
- def check_direct_size(mime, size)
556
+ # Refuses content whose known size is over the API's limit for its type, before any of it is
557
+ # sent: `in_parts` selects the limits for content sent in parts. Images in parts are checked by
558
+ # {#check_image_size}.
559
+ def check_upload_size(mime, size, in_parts: false)
527
560
  return if size.nil?
528
561
 
529
562
  limit, message = if mime.start_with?('image/')
530
563
  [MAX_DIRECT_IMAGE_BYTES, 'Images uploaded in one request can be at most 10 MiB; upload larger images in parts']
531
564
  elsif mime.start_with?('audio/', 'video/')
532
- [MAX_MEDIA_BYTES, 'Audio and video files can be at most 50 MiB']
565
+ [in_parts ? MAX_LONG_MEDIA_BYTES : MAX_MEDIA_BYTES,
566
+ 'Audio and video files can be at most 50 MiB in one request, or 2 GiB in parts']
533
567
  else
534
568
  [MAX_FILE_BYTES, 'Files can be at most 5,000 MiB']
535
569
  end
536
- raise Internal.invalid(message, ErrorCodes::FILE_TOO_LARGE) if size > limit
570
+ raise too_large(message, limit) if size > limit
537
571
  end
538
572
 
539
573
  # Images in parts total at most 20 MiB, at any size below that; a declared total is checked
@@ -541,7 +575,12 @@ module CalmLens
541
575
  def check_image_size(size)
542
576
  return unless size > MAX_MULTIPART_IMAGE_BYTES
543
577
 
544
- raise Internal.invalid('Multipart image uploads cannot exceed 20MB.', ErrorCodes::MULTIPART_IMAGE_TOO_LARGE)
578
+ raise too_large('Multipart image uploads cannot exceed 20 MiB.', MAX_MULTIPART_IMAGE_BYTES)
579
+ end
580
+
581
+ # `upload/too-large` for content over the plan's `limit` bytes, as the API answers it.
582
+ def too_large(message, limit)
583
+ InvalidArgumentError.new(ErrorCodes::UPLOAD_TOO_LARGE, message, details: { 'limitBytes' => limit, 'source' => 'plan' })
545
584
  end
546
585
  end
547
586
  end