calmlens 0.1.0 → 0.17.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +134 -57
- data/lib/calmlens/assets.rb +169 -50
- data/lib/calmlens/cdn.rb +29 -7
- data/lib/calmlens/error_codes.rb +82 -4
- data/lib/calmlens/errors.rb +26 -17
- data/lib/calmlens/events.rb +15 -12
- data/lib/calmlens/folders.rb +13 -6
- data/lib/calmlens/hash_lists.rb +95 -0
- data/lib/calmlens/internal.rb +13 -7
- data/lib/calmlens/moderation_context.rb +55 -0
- data/lib/calmlens/resource.rb +23 -6
- data/lib/calmlens/server_client.rb +36 -2
- data/lib/calmlens/session.rb +170 -68
- data/lib/calmlens/testing/classification.rb +21 -0
- data/lib/calmlens/testing/fake_api.rb +946 -233
- data/lib/calmlens/testing/mock_decision.rb +158 -0
- data/lib/calmlens/testing/placeholder.rb +111 -0
- data/lib/calmlens/testing.rb +282 -39
- data/lib/calmlens/text.rb +7 -5
- data/lib/calmlens/transforms.rb +37 -18
- data/lib/calmlens/upload_body.rb +20 -9
- data/lib/calmlens/upload_tokens.rb +14 -6
- data/lib/calmlens/version.rb +1 -1
- data/lib/calmlens/webhooks.rb +127 -32
- data/lib/calmlens.rb +2 -0
- data/sig/calmlens.rbs +168 -39
- data/sig/error_codes.rbs +41 -2
- data/sig/qualifiers.rbs +141 -0
- data/sig/types.rbs +341 -163
- metadata +8 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e47521664056eafa1785563de5a6665872afae6a50ca85c54cce15fee09bffa7
|
|
4
|
+
data.tar.gz: 547e3ccba2fe2afa8bd41bd77784ac726d4710f4ce8c9b4125fa28925184edd5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 301769e50b4ca8e46c6de2d80d6c5b22408d2d7c50e74794870701da1193277069f3d9e6be63ec7bd0832b4e2c2ae5ee64021728faf5fbb7ce4194a02bfa3fac
|
|
7
|
+
data.tar.gz: 379df2e3c07e1af6b079dfe1d10c5cc8e6b8a2c38ce55dbdb63d52c4789bd269dcec07f6d387e3b46b2d01c726d3744987ad9deed69b373086d0e22768b12059
|
data/README.md
CHANGED
|
@@ -5,14 +5,14 @@ Ruby 3.1+ server-side client for CalmLens moderation, uploads, events, webhooks
|
|
|
5
5
|
```ruby
|
|
6
6
|
require 'calmlens'
|
|
7
7
|
client = CalmLens::ServerClient.new(api_key: ENV.fetch('CALMLENS_API_KEY'), project_id: ENV.fetch('CALMLENS_PROJECT_ID'))
|
|
8
|
-
result = client.assets.
|
|
8
|
+
result = client.assets.classify(name: 'text', content: {plain_text: 'Hello world', type: 'text/plain'})
|
|
9
9
|
asset = client.assets.create(name: 'note.txt', content: {data: 'Hello', type: 'text/plain'})
|
|
10
10
|
client.close
|
|
11
11
|
```
|
|
12
12
|
|
|
13
13
|
Install from RubyGems: `gem install calmlens`, or add `gem "calmlens"` to your Gemfile. The gem has no runtime dependencies and ships YARD documentation and RBS signatures in `sig/` (see [Types](#types)).
|
|
14
14
|
|
|
15
|
-
Resources cover every API operation: assets create/
|
|
15
|
+
Resources cover every API operation: assets create/classify/search/list/each/retrieve/wait_for_decision/update/retry/decide/explain/delete_content/delete/batch/move/list_children/each_child/upload_part/create_signed_url; folders list/create/retrieve/update/move/delete; hash lists list/retrieve/create/update/delete/list_entries/add_entry/remove_entry; text moderation; events list/each/each_page; CDN URL building and serving; and `ensure_lens`/`ensure_lenses`. `CalmLens::VERSION` holds the gem version.
|
|
16
16
|
|
|
17
17
|
## Options
|
|
18
18
|
|
|
@@ -20,50 +20,73 @@ Every option name may be snake_case or camelCase, at any depth, so options added
|
|
|
20
20
|
|
|
21
21
|
Any method also accepts `request_options:`:
|
|
22
22
|
|
|
23
|
-
| Key
|
|
24
|
-
|
|
25
|
-
| `idempotency_key`
|
|
26
|
-
| `timeout`
|
|
27
|
-
| `open_timeout`
|
|
28
|
-
| `
|
|
23
|
+
| Key | Meaning |
|
|
24
|
+
| -------------------- | ---------------------------------------------------------------------------- |
|
|
25
|
+
| `idempotency_key` | The `Idempotency-Key` for an upload. The SDK generates one when you omit it. |
|
|
26
|
+
| `timeout` | Seconds allowed for each socket read or write for this call. |
|
|
27
|
+
| `open_timeout` | Seconds allowed to open a connection for this call. |
|
|
28
|
+
| `upload_timeout` | Seconds allowed for each attempt of a binary upload for this call. |
|
|
29
|
+
| `max_retries` | Retries for this call (see below). |
|
|
30
|
+
| `max_response_bytes` | The largest response body this call reads. |
|
|
29
31
|
|
|
30
32
|
## Types
|
|
31
33
|
|
|
32
|
-
`sig/` holds RBS signatures for every public method, with record types for each option Hash and response (in `sig/types.rbs`): `CalmLens::asset` (with `
|
|
34
|
+
`sig/` holds RBS signatures for every public method, with record types for each option Hash and response (in `sig/types.rbs`): `CalmLens::asset` (with `asset_classification`, `asset_moderation` and `asset_content`), `classification_report` (categories, qualifiers, labels, media segments and `features` results), `processing_options_input`, `folder`, `lens`, `event`, `events_page`, `asset_page`, `asset_search_response`, `signed_url` and the rest. Steep and other RBS tools check your calls and the response keys you read:
|
|
33
35
|
|
|
34
36
|
```ruby
|
|
35
|
-
asset = client.
|
|
36
|
-
|
|
37
|
-
asset
|
|
37
|
+
asset = client.assets.classify(name: 'photo.jpg', content: {data: File.binread('photo.jpg'), type: 'image/jpeg'},
|
|
38
|
+
processing_options: {features: {face_detection: {enabled: true}}})
|
|
39
|
+
asset['classification']['moderation']['status'] # CalmLens::asset_moderation_status
|
|
40
|
+
asset.dig('classification', 'faceDetection', 'faces') # Array[CalmLens::detected_face]?
|
|
38
41
|
```
|
|
39
42
|
|
|
40
43
|
The signatures describe snake_case Symbol option keys; camelCase and String keys still work at runtime but are not typed. Results whose fields depend on `status` (category, qualifier and feature results) are typed as one record with those fields optional: check `status`, then read `score` or `reason`. `metadata` values and API error `details` are any JSON (`CalmLens::json_value`).
|
|
41
44
|
|
|
42
|
-
##
|
|
45
|
+
## Classification results
|
|
43
46
|
|
|
44
|
-
`assets.
|
|
47
|
+
`assets.classify` and `text.classify` return the asset, whose state is in three parts:
|
|
45
48
|
|
|
46
|
-
- `
|
|
47
|
-
- `
|
|
48
|
-
- `content`: `status` is `temporary` (held until processing finishes), `stored` (kept with `keep_content_after_processing: true`) or `removed`, with `removalReason` (`policy`, `retention`, `requested`, `parent-removed` or `disposed`). It also holds `format` (MIME type), `extension`, `sizeInBytes`, `checksums`, `image
|
|
49
|
+
- `upload`: receipt of source content, with `status` (`uploading` or `completed`), `updatedAt` and `partsReceived` for multipart uploads.
|
|
50
|
+
- `classification`: `status` (`pending`, `processing`, `completed` or `failed`), `failureReason` (why processing failed, while `status` is `failed`; otherwise `null`: `file-too-large`, `duration-limit`, `resolution-limit`, `unsupported-format`, `undecodable-content`, `multiple-tracks`, `type-mismatch`, `free-upload-limit`, `child-failed` or `processing-error`), `updatedAt`, `completedAt`, optional `labels`, and `moderation`. The moderation section contains `status` (`pending`, `allowed`, `limited`, `age-restricted`, `blocked` or `purged`), policy-adjusted `categories`, `automatedAction`, `flaggedCategories`, `explanationComplete`, and `raw` findings with an optional `timeline` and `manualReviews`, each with the reviewer's `decision`. A reviewer decision does not change `completedAt`.
|
|
51
|
+
- `content`: `status` is `temporary` (held until processing finishes), `stored` (kept with `keep_content_after_processing: true`) or `removed`, with `removalReason` (`policy`, `retention`, `requested`, `parent-removed` or `disposed`). It also holds `format` (MIME type), `extension`, `sizeInBytes`, `checksums`, `image` (the picture served at the rendered resource), `extractedText`, `fileMetadata` (metadata embedded in an uploaded image, such as its capture time, camera and software) and `safetyHold`.
|
|
49
52
|
|
|
50
|
-
|
|
53
|
+
The moderation status also decides delivery. Show content to everyone only when `asset['classification']['moderation']['status']` is `allowed` or `limited`. CalmLens delivers a `limited` asset like an `allowed` one; it is marked so your app can reduce its reach, such as keeping it out of recommendations, trending and notifications, ranking it lower or collapsing it. CalmLens delivers an `age-restricted` asset only with `viewerClaims.ageConfirmed=true`, so show it only behind your own age check. Don't show `blocked` assets (content kept, not delivered) or `purged` ones (content deleted, `removalReason` `policy`). A reviewer's decision takes precedence over the policy's; allowing a purged asset does not restore removed content. New statuses, visibilities and fields may appear in later API versions: the SDK returns them unchanged, so treat any value you do not recognise as not ready.
|
|
51
54
|
|
|
52
55
|
### Waiting for a decision
|
|
53
56
|
|
|
54
|
-
`assets.wait_for_decision` waits for a queued asset's decision and returns the asset once `moderation.status` is
|
|
57
|
+
`assets.wait_for_decision` waits for a queued asset's decision and returns the asset once `classification.moderation.status` is no longer `pending`, or `classification.status` is `failed`. It retrieves the asset every few seconds and raises `CalmLens::TimeoutError` with code `sdk/timeout` if there's no decision within `timeout:` seconds (default 60); `details` holds `'assetId'` and the last `'classificationStatus'` and `'moderationStatus'`. Use it after `assets.create`, or with an asset ID you stored earlier. Webhooks report decisions without polling.
|
|
55
58
|
|
|
56
59
|
```ruby
|
|
57
60
|
queued = client.assets.create(name: 'photo.jpg', content: {data: File.open('photo.jpg', 'rb'), type: 'image/jpeg'})
|
|
58
61
|
decided = client.assets.wait_for_decision(asset_id: queued['id'], timeout: 30)
|
|
59
|
-
|
|
62
|
+
show_photo(decided['id']) if %w[allowed limited].include?(decided['classification']['moderation']['status'])
|
|
60
63
|
```
|
|
61
64
|
|
|
62
65
|
`timeout:` covers the whole wait; `request_options: {timeout:}` still limits each request. The call only sleeps and sends requests, so you can stop it early by stopping its thread.
|
|
63
66
|
|
|
67
|
+
### Recording a reviewer's decision
|
|
68
|
+
|
|
69
|
+
`assets.decide` replaces the policy's decision with a reviewer's: `classification.moderation.status` becomes the decision, the decision and note join `manualReviews`, and the matching `asset.<status>` event is sent with `decided-by: reviewer`. An asset still pending moderation can only be blocked or purged. Purging needs `confirm_purge: true` and the `assets:delete` permission, and permanently deletes the stored content.
|
|
70
|
+
|
|
71
|
+
```ruby
|
|
72
|
+
client.assets.decide(asset_id: asset_id, decision: 'blocked', note: 'Spam reported by three members')
|
|
73
|
+
client.assets.decide(asset_id: asset_id, decision: 'purged', note: 'Illegal content', confirm_purge: true)
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### Replies in a conversation
|
|
77
|
+
|
|
78
|
+
`assets.create`, `assets.classify` and `text.classify` take `author_id:`, your identifier for who wrote the content, and `context:`, the earlier posts in the conversation, oldest first. CalmLens moderates only the content itself and uses the earlier posts to read who it addresses and what it refers to; it compares author IDs only with each other, to tell which posts share a speaker.
|
|
79
|
+
|
|
80
|
+
```ruby
|
|
81
|
+
client.text.classify(text: 'Same to you', author_id: 'user-9',
|
|
82
|
+
context: {items: [{text: 'Nice photo!', author_id: 'user-1'}, {text: 'Thanks', author_id: 'user-9'}]})
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The SDK sends only the posts CalmLens uses: the most recent 5 (`CalmLens::ModerationContext::USED_ITEMS`) that fit in 2,000 characters (`CalmLens::ModerationContext::USED_CHARACTERS`, counted as Unicode code points). The newest post is always sent. `CalmLens::ModerationContext.select_items(items)` returns the posts that would be sent. The asset returns `authorId` and the `context` used; `context` is `nil` once the content is removed.
|
|
86
|
+
|
|
64
87
|
## Uploads and delivery
|
|
65
88
|
|
|
66
|
-
Content is `{type:, plain_text:}`, `{type:, base64:}` or `{type:, data:}`, where `data` is a binary String or an IO. An IO is streamed
|
|
89
|
+
Content is `{type:, plain_text:}`, `{type:, base64:}` or `{type:, data:}`, where `data` is a binary String or an IO. An IO is streamed and never read into memory: a File, Tempfile or StringIO (or any IO with `size_in_bytes:`) is sent with a `Content-Length`, and a stream of unknown length, such as a pipe or socket, is sent with chunked transfer encoding and no `sizeInBytes`. A stream that ends early or holds more bytes than declared fails with `upload/invalid-payload`.
|
|
67
90
|
|
|
68
91
|
```ruby
|
|
69
92
|
File.open('video.mp4', 'rb') do |file|
|
|
@@ -71,7 +94,7 @@ File.open('video.mp4', 'rb') do |file|
|
|
|
71
94
|
end
|
|
72
95
|
```
|
|
73
96
|
|
|
74
|
-
Multipart uploads start with `part:` on `create`/`
|
|
97
|
+
Multipart uploads start with `part:` on `create`/`classify` and continue with `upload_part`. The queue mode chosen at the start is saved with the upload, so later parts can come from another process; pass `queue:` to `upload_part` only to override it. Pass `total_size_in_bytes:` at the start when you know the file size. See [Multipart uploads](https://calmlens.com/docs/guides/multipart-uploads).
|
|
75
98
|
|
|
76
99
|
`cdn.serve` fetches content with the API key. With a block it yields a `CalmLens::Response` whose body streams; without one it returns the response with the body read. Error statuses are returned, not raised.
|
|
77
100
|
|
|
@@ -86,21 +109,25 @@ end
|
|
|
86
109
|
|
|
87
110
|
Use `create_signed_url` to deliver private content to browsers without your key.
|
|
88
111
|
|
|
89
|
-
`cdn.get_asset_url`, `cdn.serve` and `create_signed_url` accept a `lens:`, a `transform:`, or both (the lens runs first). A transform is a selector string or structured steps (one Hash or an Array), serialized with sorted option keys:
|
|
112
|
+
`cdn.get_asset_url` and `cdn.serve` take a `resource:` of `rendered` (the default), `original` or `placeholder`; only `rendered` takes a lens or transform. `cdn.get_asset_url`, `cdn.serve` and `create_signed_url` accept a `lens:`, a `transform:`, or both (the lens runs first). A transform is a selector string or structured steps (one Hash or an Array), serialized with sorted option keys:
|
|
90
113
|
|
|
91
114
|
```ruby
|
|
92
|
-
client.cdn.get_asset_url(asset_identifier: asset_id, transform: '
|
|
115
|
+
client.cdn.get_asset_url(asset_identifier: asset_id, transform: 'format=png,width=1024')
|
|
93
116
|
client.cdn.get_asset_url(asset_identifier: asset_id, transform: {type: 'imageTransform', options: {width: 1024, format: 'png'}})
|
|
94
|
-
# => ".../
|
|
117
|
+
# => ".../rendered/format=png,width=1024" in both cases
|
|
95
118
|
```
|
|
96
119
|
|
|
120
|
+
A selector has one path segment per step, such as `fit=cover,width=640/format=webp`, and a lens and a transform are joined with `/` (`/rendered/@thumbnail/width=320`). Nested options are dotted (`border.color`) and hex colours drop the `#`.
|
|
121
|
+
|
|
122
|
+
A public asset whose moderation status is `age-restricted` is served from its public URL only with a JSON `viewerClaims` query parameter containing `{ "ageConfirmed": true }`; otherwise delivery is refused, even with a signed URL or API key. Pass `viewer_claims: { age_confirmed: true }` to `cdn.get_asset_url` (or `cdn.serve`) to add it after any lens or transform. Set it only where your app has already decided the viewer may see age-restricted content.
|
|
123
|
+
|
|
97
124
|
## Upload presets
|
|
98
125
|
|
|
99
|
-
Pass `upload_preset_id:` to `assets.create` or `assets.
|
|
126
|
+
Pass `upload_preset_id:` to `assets.create` or `assets.classify` to apply an upload preset's folder, visibility, file types and size limit. To let a browser upload through a preset that requires tokens, sign a single-use token on your server. Signing happens locally with your API key and makes no request.
|
|
100
127
|
|
|
101
128
|
```ruby
|
|
102
129
|
signed = client.upload_tokens.sign(
|
|
103
|
-
upload_preset_id: '
|
|
130
|
+
upload_preset_id: 'preset_4Lk8Qw2Xm9Rt5Vb3Nc7P',
|
|
104
131
|
external_id: 'post-42', # optional: the uploaded asset gets this ID
|
|
105
132
|
expires_in_seconds: 600 # optional: 1 to 3600, 600 by default
|
|
106
133
|
)
|
|
@@ -111,26 +138,48 @@ See [Upload presets](https://calmlens.com/docs/guides/upload-presets). `CalmLens
|
|
|
111
138
|
|
|
112
139
|
## Pagination
|
|
113
140
|
|
|
114
|
-
`
|
|
141
|
+
Every list returns a cursor page: pass its `nextCursor` as `cursor:` to fetch the next one (`page_size:` sets its length, 1 to 100, default 50). `ordering:` entries take a `field:` and a lowercase `direction:`, `'asc'` or `'desc'`. Folder pages also hold the `ancestors` of the listed folders, and `folders.list` takes one `ordering:` entry by `name` (the default, ascending) or `createdAt`.
|
|
142
|
+
|
|
143
|
+
```ruby
|
|
144
|
+
page = client.folders.list(parent_id: folder_id, page_size: 50) # items, ancestors, nextCursor, hasMore
|
|
145
|
+
page = client.folders.list(parent_id: folder_id, page_size: 50, cursor: page['nextCursor']) if page['hasMore']
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
`assets.each`, `assets.each_child`, `events.each` and `events.each_page` fetch pages as you iterate and return an Enumerator without a block, so `client.assets.each(filter: 'classification.moderation.status==allowed').lazy.first(5)` stops after the pages it needs.
|
|
149
|
+
|
|
150
|
+
## Hash lists
|
|
151
|
+
|
|
152
|
+
Image uploads are compared with every hash list in the project: a `flag` list reports matches, and a `block` list also blocks close matches. Add a 64-character PDQ hash, or an image asset whose hash CalmLens computed; adding a hash already in the list returns its existing entry.
|
|
153
|
+
|
|
154
|
+
```ruby
|
|
155
|
+
list = client.hash_lists.create(name: 'Removed images', action: 'block')
|
|
156
|
+
entry = client.hash_lists.add_entry(hash_list_id: list['id'], asset_id: asset_id, label: 'Report 1042')
|
|
157
|
+
page = client.hash_lists.list_entries(hash_list_id: list['id'], page_size: 50) # items, nextCursor, hasMore
|
|
158
|
+
client.hash_lists.update(hash_list_id: list['id'], action: 'flag', description: nil)
|
|
159
|
+
client.hash_lists.remove_entry(entry_id: entry['id'])
|
|
160
|
+
client.hash_lists.delete(hash_list_id: list['id']) # deletes its entries too
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
An asset without a perceptual hash fails with `hash-list/asset-not-hashed`.
|
|
115
164
|
|
|
116
165
|
## Events feed
|
|
117
166
|
|
|
118
|
-
`events.list` without `
|
|
167
|
+
`events.list` without `cursor:` returns **no events** and a `nextCursor` for the current time. Store it, then read on from it; persist each page's `nextCursor` once its events are handled. Unlike other lists, the events feed's `nextCursor` is never nil.
|
|
119
168
|
|
|
120
169
|
```ruby
|
|
121
|
-
cursor = client.events.list['
|
|
170
|
+
cursor = client.events.list['nextCursor'] # start from now
|
|
122
171
|
# ...later, and on every poll:
|
|
123
|
-
client.events.each_page(
|
|
124
|
-
page['
|
|
125
|
-
cursor = page['
|
|
172
|
+
client.events.each_page(cursor: cursor, page_size: 50) do |page|
|
|
173
|
+
page['items'].each { |event| handle(event) } # event['type'], event['facts'], event['assetId']
|
|
174
|
+
cursor = page['nextCursor'] # persist it
|
|
126
175
|
end
|
|
127
176
|
```
|
|
128
177
|
|
|
129
|
-
Each event
|
|
178
|
+
Each event has `id`, `projectId`, `type`, `severity`, `title`, `description`, `assetId`, `asset` (the asset's status when the event was created), `facts` and `createdAt`; `description`, `assetId` and `asset` are nil when the event has none. `facts` is an array of `{'key', 'value', 'url'?}` entries registered for its `type`. Webhook deliveries carry the same events in the same shape. An unknown cursor raises `validation/invalid-cursor`.
|
|
130
179
|
|
|
131
180
|
## Webhooks
|
|
132
181
|
|
|
133
|
-
Verify the **raw** request body, never re-serialized JSON. `CalmLens.parse_webhook_event` verifies the signature and timestamp, then parses the delivery into a `CalmLens::WebhookEvent`
|
|
182
|
+
Verify the **raw** request body, never re-serialized JSON. `CalmLens.parse_webhook_event` verifies the signature and timestamp, then parses the delivery into a `CalmLens::WebhookEvent` with the fields of an `events.list` item: `id`, `project_id`, `type`, `severity`, `title`, `description`, `asset_id`, `asset`, `facts` and `created_at`, plus `test?`. `description`, `asset_id` and `asset` are nil when the event has none. Fields CalmLens adds later are kept: `to_payload` returns the whole body as received, including unknown fields, asset fields and fact fields. `CalmLens.to_webhook_event_payload(event)` turns an `events.list` item into the same body. Instead of `signature:` and `timestamp:`, you can pass the request's `headers:`: a Hash with names in any case (repeated values as an Array are joined with `,`), `Net::HTTPHeader`, or a Rack env. It raises `CalmLens::WebhookVerificationError` with code `sdk/invalid-webhook-signature` or `sdk/invalid-webhook-payload`. Passing a parsed Hash instead of the body raises `CalmLens::InvalidArgumentError`.
|
|
134
183
|
|
|
135
184
|
Rails:
|
|
136
185
|
|
|
@@ -144,7 +193,7 @@ class CalmLensWebhooksController < ActionController::API
|
|
|
144
193
|
secret: ENV.fetch('CALMLENS_WEBHOOK_SECRET')
|
|
145
194
|
)
|
|
146
195
|
# Acknowledge first; do the work in a job. Skip a delivery ID you have already queued.
|
|
147
|
-
CalmLensEventJob.perform_later(request.headers[CalmLens::WEBHOOK_DELIVERY_HEADER], event.
|
|
196
|
+
CalmLensEventJob.perform_later(request.headers[CalmLens::WEBHOOK_DELIVERY_HEADER], event.to_payload)
|
|
148
197
|
head :no_content
|
|
149
198
|
rescue CalmLens::WebhookVerificationError
|
|
150
199
|
head :bad_request
|
|
@@ -170,53 +219,81 @@ rescue CalmLens::WebhookVerificationError
|
|
|
170
219
|
end
|
|
171
220
|
```
|
|
172
221
|
|
|
173
|
-
The header names are exported as `CalmLens::WEBHOOK_SIGNATURE_HEADER`, `WEBHOOK_TIMESTAMP_HEADER` and `WEBHOOK_DELIVERY_HEADER`. Retries of one delivery repeat its `x-calmlens-delivery` ID, so use it to skip duplicates. Deliveries signed more than `CalmLens::WEBHOOK_SIGNATURE_TOLERANCE_SECONDS` (300) from your clock fail; change it with `tolerance_seconds:`. `CalmLens.verify_webhook_signature` returns true or false instead of raising, and `CalmLens.
|
|
222
|
+
The header names are exported as `CalmLens::WEBHOOK_SIGNATURE_HEADER`, `WEBHOOK_TIMESTAMP_HEADER` and `WEBHOOK_DELIVERY_HEADER`. Retries of one delivery repeat its `x-calmlens-delivery` ID, so use it to skip duplicates. Deliveries signed more than `CalmLens::WEBHOOK_SIGNATURE_TOLERANCE_SECONDS` (300) from your clock fail; change it with `tolerance_seconds:`. `CalmLens.verify_webhook_signature` returns true or false instead of raising, and `CalmLens.verify_webhook_request(payload:, headers:, secret:)` does the same from the request headers. `CalmLens.test_webhook_event?(event)` is true for a test send from the portal or CLI (the `{key: 'test', value: true}` fact), like `event.test?`. To test your receiver, `CalmLens.create_webhook_signature_header(payload:, timestamp:, secret:)` signs a body and `CalmLens.create_webhook_headers(payload:, delivery_id:, secret:, timestamp:)` builds the headers of a delivery.
|
|
174
223
|
|
|
175
224
|
## Errors
|
|
176
225
|
|
|
177
|
-
Every error the SDK raises is a `CalmLens::Error` with `code`, `message`, `details`, `status` (nil without a response), `request_id` (the `X-Request-Id` header,
|
|
226
|
+
Every error the SDK raises is a `CalmLens::Error` with `code`, `message`, `details` (the API's details object, such as validation `issues`, or nil), `status` (nil without a response), `request_id` (the `X-Request-Id` header, or the error body's `requestId`; include it when you contact support), `headers`, `retry_after` (whole seconds from `Retry-After`) and the original exception in `cause`. Helpers: `not_found?` and `retryable?` (true for `sdk/transport-error`, `sdk/timeout`, `idempotency/request-in-progress`, `429`, `502`, `503` and `504`, whichever error class carries them). Compare codes with the constants in `CalmLens::ErrorCodes` (API codes, generated from the API's list, with their HTTP statuses in `ErrorCodes::STATUSES`) and `CalmLens::SdkErrorCodes`.
|
|
178
227
|
|
|
179
|
-
| Class
|
|
180
|
-
|
|
181
|
-
| `CalmLens::APIError` and its subclasses `BadRequestError`, `AuthenticationError`, `PermissionDeniedError`, `NotFoundError`, `ConflictError`, `RateLimitError`, `ServerError` | The API answered with an error status.
|
|
182
|
-
| `CalmLens::ConnectionError` (`sdk/transport-error`), `CalmLens::TimeoutError` (`sdk/timeout`)
|
|
183
|
-
| `CalmLens::InvalidResponseError` (`sdk/invalid-response`, `sdk/response-too-large`)
|
|
184
|
-
| `CalmLens::InvalidArgumentError` (`sdk/invalid-argument`, or an `upload/*` code for payload checks the API also makes)
|
|
185
|
-
| `CalmLens::ConfigurationError` (`sdk/invalid-config`)
|
|
186
|
-
| `CalmLens::WebhookVerificationError`
|
|
228
|
+
| Class | When |
|
|
229
|
+
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
230
|
+
| `CalmLens::APIError` and its subclasses `BadRequestError`, `AuthenticationError`, `PermissionDeniedError`, `NotFoundError`, `ConflictError`, `RateLimitError`, `ServerError` | The API answered with an error status, or with a redirect (`sdk/http-error` with the 3xx status), which is never followed. |
|
|
231
|
+
| `CalmLens::ConnectionError` (`sdk/transport-error`), `CalmLens::TimeoutError` (`sdk/timeout`) | No complete response arrived, or `assets.wait_for_decision` had no decision in time. |
|
|
232
|
+
| `CalmLens::InvalidResponseError` (`sdk/invalid-response`, `sdk/response-too-large`) | A successful response could not be read or was not what the API returns (such as an upload answered without an asset), or a response was larger than `max_response_bytes:` (default 32 MiB). |
|
|
233
|
+
| `CalmLens::InvalidArgumentError` (`sdk/invalid-argument`, or an `upload/*` code for payload checks the API also makes) | An argument was rejected before sending. |
|
|
234
|
+
| `CalmLens::ConfigurationError` (`sdk/invalid-config`) | `ServerClient.new` received an invalid setting, such as a malformed `base_url`. |
|
|
235
|
+
| `CalmLens::WebhookVerificationError` | A webhook delivery failed verification. |
|
|
187
236
|
|
|
188
237
|
## Retries and timeouts
|
|
189
238
|
|
|
190
|
-
Requests that are safe to repeat are retried up to `max_retries:` times (default 2): GET and HEAD, and uploads, which carry an `Idempotency-Key` reused on every retry so CalmLens runs them once and replays the saved response. Connection failures, timeouts, `429`, `502`, `503`, `504` and `idempotency/request-in-progress` are retried with
|
|
239
|
+
Requests that are safe to repeat are retried up to `max_retries:` times (default 2): GET and HEAD, and uploads, which carry an `Idempotency-Key` reused on every retry so CalmLens runs them once and replays the saved response. Connection failures, timeouts, `429`, `502`, `503`, `504` and `idempotency/request-in-progress` are retried with exponential backoff (`retry_base_delay:`, default 0.5 seconds, doubling up to 8), half of each wait fixed and half random. A `Retry-After` in seconds or as an HTTP date replaces the backoff (`0` retries at once); one over 60 seconds is returned to you instead of waited out. Other writes are never retried, and neither are redirects. A streamed upload is retried only when its IO can seek back. `cdn.serve` retries like a read and returns the last response.
|
|
191
240
|
|
|
192
|
-
`timeout:` (default 120 seconds) limits each socket read or write, not a whole transfer, so large
|
|
241
|
+
`timeout:` (default 120 seconds) limits each socket read or write, not a whole transfer, so large downloads are not cut off. `open_timeout:` (default 10 seconds) limits connecting. Binary uploads (file content and multipart parts) have no deadline unless you set `upload_timeout:`, which then limits each attempt as a whole; text content follows `timeout:`. `max_response_bytes:` (default 32 MiB) caps the response body read: a larger one fails with `sdk/response-too-large`, before it is read when its `Content-Length` says so. All of them can be set per call with `request_options:`.
|
|
193
242
|
|
|
194
243
|
## Connections
|
|
195
244
|
|
|
196
|
-
Reuse one client: it keeps up to 8 idle keep-alive connections per host, is safe to share between threads, never follows redirects, and after `fork` (Puma, Unicorn, Resque) the child opens its own connections instead of sharing the parent's. Call `close` when finished. Requests identify themselves as `calmlens-ruby/<version> (ruby
|
|
245
|
+
Reuse one client: it keeps up to 8 idle keep-alive connections per host, is safe to share between threads, never follows redirects, and after `fork` (Puma, Unicorn, Resque) the child opens its own connections instead of sharing the parent's. Call `close` when finished. Requests identify themselves as `calmlens-ruby/<version> (ruby/<version>)`. `inspect` never shows the API key.
|
|
197
246
|
|
|
198
247
|
## Offline testing
|
|
199
248
|
|
|
200
249
|
```ruby
|
|
201
250
|
require 'calmlens/testing'
|
|
202
|
-
|
|
203
|
-
|
|
251
|
+
key = CalmLens::MockServerClient.create_api_key([{actions: ['assets:download'], folder_id: nil, include_subfolders: true}])
|
|
252
|
+
moderate = lambda do |input| # decides each upload, retry and rerun
|
|
253
|
+
next 'pending' if input.queued && input.trigger == 'upload' # finish queued uploads with complete_processing
|
|
254
|
+
next 'allowed' unless input.content.text&.include?('kill')
|
|
255
|
+
|
|
256
|
+
CalmLens::MockDecision.of('blocked').flag('violence', 0.92, 'violence-threat' => 0.9)
|
|
257
|
+
end
|
|
258
|
+
mock = CalmLens::MockServerClient.new(api_key: key, moderate: moderate) # the default key cannot read private CDN content
|
|
259
|
+
asset = mock.assets.classify(name: 'note', content: {plain_text: 'hello', type: 'text/plain'},
|
|
204
260
|
processing_options: {keep_content_after_processing: true})
|
|
205
|
-
mock.assets.retrieve(asset_id: asset['id']).dig('moderation', 'status') # => "
|
|
206
|
-
mock.assets.list(filter: 'moderation.status==
|
|
261
|
+
mock.assets.retrieve(asset_id: asset['id']).dig('classification', 'moderation', 'status') # => "allowed"
|
|
262
|
+
mock.assets.list(filter: 'classification.moderation.status==allowed')['items'].length # => 1
|
|
207
263
|
mock.cdn.serve(asset_identifier: asset['id']).body # => "hello"
|
|
208
264
|
|
|
209
265
|
mock.enqueue(status: 503, body: {code: 'server/unavailable', message: 'Busy'}) # answers the next request
|
|
210
266
|
mock.enqueue_error(Net::ReadTimeout.new) # the next request raises CalmLens::TimeoutError
|
|
211
|
-
mock.enqueue_event(type: 'asset.
|
|
267
|
+
mock.enqueue_event(type: 'asset.allowed', asset_id: asset['id']) # joins the events feed
|
|
268
|
+
mock.decide(asset['id'], 'blocked') # a reviewer's decision, with its asset.blocked event
|
|
269
|
+
queued = mock.assets.create(name: 'later', content: {plain_text: 'hi', type: 'text/plain'}) # pending
|
|
270
|
+
mock.complete_processing(queued['id'], CalmLens::MockDecision.of('age-restricted').flag('sexual', 0.8))
|
|
271
|
+
mock.fail_next_request('upload/limit-exceeded', operation: 'assets.create') # the next create request fails with that API error
|
|
272
|
+
mock.fail_next_request('upload/website-rate-limit-exceeded', retry_after: 30, details: {'reason' => 'busy'})
|
|
273
|
+
mock.api_key_grants = [] # the same key, now without grants
|
|
274
|
+
CalmLens::MockServerClient.sign_upload_token(key, {upload_preset_id: 'preset_4Lk8Qw2Xm9Rt5Vb3Nc7P'}, now: 1_700_000_000_000, nonce: 'n')
|
|
212
275
|
|
|
213
276
|
request = mock.requests.last
|
|
214
277
|
request.method # => "GET"
|
|
215
|
-
request.path # => "/assets/project-1/<id>/
|
|
278
|
+
request.path # => "/assets/project-1/<id>/rendered" (relative to base_url or cdn_base_url, segments decoded)
|
|
279
|
+
request.status # => 200 (nil when the request failed without a response)
|
|
280
|
+
mock.clear_requests
|
|
216
281
|
```
|
|
217
282
|
|
|
218
|
-
`MockServerClient` runs the real client's validation, encoding, retries and error handling, then answers from an in-memory stand-in for the API and CDN. Uploads create assets you can retrieve, list, search, move and delete, and `assets.wait_for_decision` checks the stored asset's processing and moderation status; folders, multipart uploads, idempotency keys, lenses and the events feed behave like the API's; errors carry the API's codes, HTTP statuses and body shape; and `cdn.serve` returns stored bytes with real headers. Like the API, content is removed (`removalReason` `retention`) once
|
|
283
|
+
`MockServerClient` runs the real client's validation, encoding, retries and error handling, then answers from an in-memory stand-in for the API and CDN. Uploads create assets you can retrieve, list, search, move and delete, and `assets.wait_for_decision` checks the stored asset's processing and moderation status without real waiting; folders, hash lists, multipart uploads, idempotency keys, lenses and the events feed behave like the API's; errors carry the API's codes, HTTP statuses and body shape; and `cdn.serve` returns stored bytes with real headers, including `cache-control` as the CDN sets it (`public` only when anyone may see the asset, otherwise `private`; `max-age=31536000, immutable` for images and `max-age=3600` for other content). Like the API, content is removed (`removalReason` `retention`) once classification is `completed` unless the upload sets `keep_content_after_processing: true`; a `purged` outcome removes it with `removalReason` `policy`, and a reviewer's purge (`assets.decide` or `decide`) with `requested`. `assets.decide` validates like the API: a note of 1 to 1499 characters, `confirm_purge: true` to purge, `asset/decision-not-allowed` for anything but `blocked` or `purged` while moderation is `pending`, `asset/decision-unchanged` for the current status, `asset/content-purged` for anything but `blocked` once purged, and `asset/processing` while the asset is still uploading or processing; its review and event name the reviewer `An API key`. `mock.decide` is instead a test hook for a reviewer's decision, without a note. Removing content also removes a classified image's ThumbHash and average colour. Deletes, content removals and retries record `asset.deleted`, `asset.content-removed` and `asset.retried` events, as the API does. Unset fields are `null`. Hash lists keep their entries in memory until the list is deleted, `hash_lists.add_entry(asset_id:)` uses the asset's `content.image.perceptualHash` and `perceptualHashQuality` (set them through `mock.state` to arrange a test), and uploads are not compared with them. Requests are checked in the API's order with its codes: schema errors are `validation/<issue>` with HTTP 400 and the field in `details['path']` (page sizes 1 to 100, search limits 1 to 20), list cursors only continue the query they came from, multipart parts must arrive in order with 5 MiB until the last and image totals of 10 to 20 MiB, idempotency keys are 1 to 255 printable characters and a failed request frees its key, folder names are unique among siblings (case-sensitive) and listed by code point, and a lens change needs a greater version (`details` gives `currentVersion`, `minimumVersion` and `requestedVersion`). Like the CDN, `cdn.serve` answers `403` `asset/not-available` for blocked, purged and failed assets and removed content, and `403` `asset/api-key-missing-permission` for a private asset unless `api_key:` has an `assets:download` grant for the asset's folder. It sends the CDN's `cache-tag`, `x-content-type-options: nosniff` and `content-security-policy` headers, and for images an `etag` and `expires`. The `placeholder` resource is a classified image's ThumbHash as a PNG, public whatever the asset's visibility or moderation, and refused once purged or removed. `fail_next_request` makes the next request, or the next one an operation sends, fail with `transport-error`, `timeout` or an API error code (with optional `details:` and `retry_after:` seconds) without running; `api_key_grants=` changes the key's grants. List filters support `==` and `!=` (with `*` wildcards), `=in=` and `=out=` with a `(value,value)` list, and `=gt=`, `=ge=`, `=lt=` and `=le=`, joined by `;` (and), which binds tighter than `,` (or), and grouped with parentheses, on top-level and nested names such as `classification.moderation.status`, `classification.status`, `content.status` and `content.format`; other RSQL operators raise rather than being ignored. List cursors resume after the last item returned, so assets created or deleted between pages are neither repeated nor skipped. `assets.search` runs keyword search like the API: every word of the query (case-insensitive, up to 8) must appear in the name, ID, external ID, description, tags, type, content format or extension, or moderation status, and results carry the API's keyword `score` and `matchedFields`, ranked by score and then newest first. The default clock advances at least 1 ms per reading, so assets created in a row have increasing `createdAt`; a `now:` clock you pass is used as is. Like the API, new objects get IDs with their kind's prefix, `_` and 20 random base62 characters, such as `asset_5Gm2Rk8XqT3wLp9VbN4c`; for predictable IDs, pass `create_id:`, a callable that receives the prefix (`asset`, `fldr`, `hl`, `hle`, `lens`, `evt` or `rev`) and returns the ID. It does not classify content: `moderate:` decides each asset it moderates. It receives a `CalmLens::MockModerationInput` with the stored `asset` (still processing), its `content` (`bytes`, and `text` for a plain-text upload; `nil` without content, such as a website), the `trigger` (`upload`, `retry` after `assets.retry`, `rerun` after `assets.explain` with `rerun: true`, or `complete`) and `queued` (true for queued uploads, retries, reruns and `complete`), and returns a status (`pending`, `failed`, `allowed`, `limited`, `age-restricted`, `blocked` or `purged`) or a `CalmLens::MockDecision`. Without it, queued uploads, retries and reruns stay `pending` and everything else is `allowed`. `CalmLens::MockDecision.of(status)` starts a decision; `flag(category, score, qualifiers = {})` and `not_evaluated(category)` add findings and chain, and an unknown status, category or qualifier, a score outside 0 to 1, a repeated category or findings on `pending` or `failed` raise `sdk/invalid-argument`. A decided asset gets the full report CalmLens sends: every category, the highest `score`, and `policy` with each category's action and the `flaggedCategories` (scored 0.5 or more unless `allowed`). Each completed upload records `asset.uploaded`, then `asset.classified` and the automated `asset.<status>`, or `asset.failed`. `complete_processing(asset_id, decision = nil)` finishes a processing asset with `decision` or else `moderate:`'s answer for `complete`; it raises `asset/not-found` for an unknown asset and `sdk/invalid-argument` for one that isn't processing. `decide` refuses `pending` and `failed`. `enqueue_event` with the `asset_id:` of a stored asset attaches that asset's snapshot. Each `fail_next_request` failure fails one request, so a retry takes the next queued failure or reaches the stand-in, and an unknown `operation:` raises. The mock's `max_retries:` defaults to 0. Change records through `mock.state` to arrange a test. Queued responses and errors are used first, in order. Retries and decision checks wait no real time.
|
|
219
284
|
|
|
220
285
|
## Development
|
|
221
286
|
|
|
222
|
-
Native tests (Ruby 3.1+, Bundler 2.6+): `bundle install && bundle exec ruby -Ilib -e 'Dir["test/*_test.rb"].each { |file| require_relative file }'`, and lint with `bundle exec rubocop`. `lib/calmlens/error_codes.rb` and the files in `testdata/` are generated by `bun packages/sdk-contract/generate.ts` from the repository root. Container/backend checks: `bun run test:sdk:e2e`. Type-check the signatures with `gem install steep && steep check` (see `Steepfile`) and `rbs -I sig validate`. Release by setting `CalmLens::VERSION` in `lib/calmlens/version.rb`.
|
|
287
|
+
Native tests (Ruby 3.1+, Bundler 2.6+): `bundle install && bundle exec ruby -Ilib -e 'Dir["test/*_test.rb"].each { |file| require_relative file }'`, and lint with `bundle exec rubocop`. The contract tests replay every shared fixture in `testdata/` (packages/sdk-contract/README.md): the real client against scripted responses, the mock against the behaviour scenarios, webhooks, upload tokens and the surface. `lib/calmlens/error_codes.rb` and the files in `testdata/` are generated by `bun packages/sdk-contract/generate.ts` from the repository root. Container/backend checks: `bun run test:sdk:e2e`. Type-check the signatures with `gem install steep && steep check` (see `Steepfile`) and `rbs -I sig validate`. Release by setting `CalmLens::VERSION` in `lib/calmlens/version.rb`.
|
|
288
|
+
|
|
289
|
+
Image moderation considers readable text directly. Optional OCR (`processingOptions.features.ocr.enabled: true` on the wire) adds a text moderation pass and stores retained text in the image’s `content.extractedText`. No separate text child is created. Read the image’s combined `classification.moderation`; removing its content also removes the extracted text.
|
|
290
|
+
|
|
291
|
+
Folder listings return a cursor page like every other list: `items`, visible `ancestors`, `nextCursor`, `hasMore`, `pageSize` and `ordering`. Set `parent_id:` to open a branch (omit it for the project root), and pass `nextCursor` as `cursor:` to load the next page. `page_size:` is 1 to 100 (default 50). `search:` lists only the children whose name contains it.
|
|
292
|
+
|
|
293
|
+
### Viewer context for CDN delivery
|
|
294
|
+
|
|
295
|
+
CDN methods accept optional viewer claims (`viewerClaims`, or the language's `viewer_claims` / `ViewerClaims` field). The object contains `region` and `ageConfirmed`, using the SDK's normal field naming. Region defaults to the country of the connection reaching CalmLens; age confirmation defaults to false. Proxies should forward the incoming viewer country rather than rely on their outgoing connection's country. CalmLens accepts these as customer assertions without independently verifying them.
|
|
296
|
+
|
|
297
|
+
Age-restricted content returns `403` with `viewer/age-confirmation-required` unless age confirmation is true, including API-key and signed-URL requests. Private assets separately require access permission. Blocks and safety holds cannot be overridden by claims. Applications that never use age-restricted content should leave confirmation false, including processing jobs.
|
|
298
|
+
|
|
299
|
+
Raw HTTP clients can send a JSON `viewerClaims` query parameter or the JSON `X-CalmLens-Viewer-Claims` header; the query takes precedence. Signed URLs include claims when created and must be forwarded unchanged. Asset retrieval returns stored moderation results without a request-specific `delivery` object. Proxy caches must not bypass current viewer and moderation checks; honour `private, no-store` responses.
|