@giveitsmaller/contracts 0.1.2 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/asyncapi/events.yaml +2012 -0
- package/dist/asyncapi/OperationType.d.ts +4 -0
- package/dist/asyncapi/OperationType.js +4 -0
- package/dist/openapi/models/JobType.d.ts +8 -1
- package/dist/openapi/models/JobType.js +8 -1
- package/dist/openapi/models/MultipartCompleteRequest.d.ts +1 -1
- package/dist/openapi/models/MultipartCompleteRequest.js +3 -3
- package/dist/openapi/models/MultipartCompleteResponse.d.ts +50 -0
- package/dist/openapi/models/MultipartCompleteResponse.js +53 -0
- package/dist/openapi/models/MultipartCompleteSuccessEnvelope.d.ts +46 -0
- package/dist/openapi/models/MultipartCompleteSuccessEnvelope.js +54 -0
- package/dist/openapi/models/MultipartInitiateResponse.d.ts +1 -1
- package/dist/openapi/models/MultipartInitiateResponse.js +3 -3
- package/dist/openapi/models/OperationInputModel.d.ts +3 -1
- package/dist/openapi/models/OperationInputModel.js +3 -1
- package/dist/openapi/models/OperationType.d.ts +24 -2
- package/dist/openapi/models/OperationType.js +24 -2
- package/dist/openapi/models/index.d.ts +2 -0
- package/dist/openapi/models/index.js +2 -0
- package/dist/operations/index.d.ts +1 -0
- package/dist/operations/index.js +1 -0
- package/dist/operations/watermark.d.ts +28 -0
- package/dist/operations/watermark.js +19 -0
- package/openapi/api.yaml +2533 -0
- package/operations/schemas/archive.yaml +22 -0
- package/operations/schemas/compress.yaml +280 -0
- package/operations/schemas/convert.yaml +104 -0
- package/operations/schemas/merge.yaml +211 -0
- package/operations/schemas/thumbnail.yaml +120 -0
- package/operations/schemas/watermark.yaml +87 -0
- package/package.json +14 -3
|
@@ -0,0 +1,2012 @@
|
|
|
1
|
+
asyncapi: 3.0.0
|
|
2
|
+
info:
|
|
3
|
+
title: GISL Compression Events
|
|
4
|
+
version: 3.0.0
|
|
5
|
+
description: |
|
|
6
|
+
Asynchronous event contracts for the GISL (Give It Smaller) compression service.
|
|
7
|
+
|
|
8
|
+
**Architecture Overview — Dual-Family SNS Topology**
|
|
9
|
+
|
|
10
|
+
The API publishes operation requests to **two** SNS topics, split by semantic
|
|
11
|
+
family. Each topic uses a single-attribute filter policy; there is no
|
|
12
|
+
compound-filter routing anywhere in the live topology:
|
|
13
|
+
|
|
14
|
+
- **`gisl-{env}-{region}-job-requests`** — compression routing only.
|
|
15
|
+
Filter attribute: `job_type`. Subscriptions: four per-media-type queues
|
|
16
|
+
(`jobs-image`, `jobs-video`, `jobs-audio`, `jobs-document`). Each
|
|
17
|
+
compression Lambda handles `compress` for exactly one media type.
|
|
18
|
+
- **`gisl-{env}-{region}-operations`** — non-compression operations
|
|
19
|
+
(thumbnail, watermark, merge, archive, convert). Filter attribute:
|
|
20
|
+
`operation_type`. Subscriptions: one queue per operation type, plus four
|
|
21
|
+
thumbnail sub-type queues during the migration window.
|
|
22
|
+
|
|
23
|
+
**Publisher branching rule** (implemented in `compression_api`'s
|
|
24
|
+
`AwsSnsOperationPublisherAdapter` under Option A):
|
|
25
|
+
|
|
26
|
+
- For `operation_type == compress`: publish to the `job-requests` topic
|
|
27
|
+
with a single `job_type = {mediaGroup}` message attribute. Do **not**
|
|
28
|
+
set `operation_type` or `media_group` as attributes on this branch —
|
|
29
|
+
the media group is encoded as `job_type` and no other attribute is
|
|
30
|
+
used as a filter on this topic.
|
|
31
|
+
- For all other operation types: publish to the `operations` topic with
|
|
32
|
+
`operation_type = {type}` as the filter attribute. Additionally, set
|
|
33
|
+
`media_group = {mediaGroup}` as an informational message attribute
|
|
34
|
+
where a media group is meaningful (omit for archive). This second
|
|
35
|
+
attribute is **not** used as an SNS filter — it is preserved for
|
|
36
|
+
consumer observability and log correlation.
|
|
37
|
+
|
|
38
|
+
`media_group` is never a field in the message payload body. It exists
|
|
39
|
+
only as an SNS MessageAttribute on the `operations` topic branch.
|
|
40
|
+
|
|
41
|
+
**Thumbnail migration window.** The terraform topology has both the
|
|
42
|
+
legacy `ops-thumbnail` queue and four new sub-type queues
|
|
43
|
+
(`ops-thumbnail-{image,video,document,office}`) live simultaneously.
|
|
44
|
+
The API publisher currently emits the legacy `operation_type=thumbnail`
|
|
45
|
+
only; adoption of the four sub-type values (`thumbnail_image`,
|
|
46
|
+
`thumbnail_video`, `thumbnail_document`, `thumbnail_office`) is a
|
|
47
|
+
follow-up API PR. Both routing targets are documented in this contract
|
|
48
|
+
because both are valid during the migration.
|
|
49
|
+
|
|
50
|
+
**Notifications (Lambda -> API).** Lambda functions publish progress
|
|
51
|
+
and result notifications to the FIFO SNS topic
|
|
52
|
+
`gisl-{env}-{region}-notifications-operations.fifo`. The API consumes
|
|
53
|
+
from the corresponding FIFO SQS queue via a Raw Message Delivery
|
|
54
|
+
subscription. This notification path is unchanged from previous
|
|
55
|
+
releases.
|
|
56
|
+
|
|
57
|
+
**Lambda inventory** (post Option A):
|
|
58
|
+
|
|
59
|
+
- **Jobs family** (compression, 4 Lambdas), routed by `job_type`:
|
|
60
|
+
`compression-image`, `compression-video`, `compression-audio`,
|
|
61
|
+
`compression-document`. Each handles `compress` only for its media
|
|
62
|
+
type.
|
|
63
|
+
- **Operations family** (non-compression), routed by `operation_type`:
|
|
64
|
+
`operation-thumbnail` (legacy, retiring after migration),
|
|
65
|
+
`operation-thumbnail-image`, `operation-thumbnail-video`,
|
|
66
|
+
`operation-thumbnail-document`, `operation-thumbnail-office`,
|
|
67
|
+
`operation-watermark`, `operation-merge`, `operation-archive`,
|
|
68
|
+
`operation-convert`.
|
|
69
|
+
|
|
70
|
+
**Message Types**
|
|
71
|
+
|
|
72
|
+
- `JobRequest`: API -> compression Lambda (via `job-requests` topic).
|
|
73
|
+
- `OperationRequest`: API -> non-compression Lambda (via `operations`
|
|
74
|
+
topic).
|
|
75
|
+
- `OperationProgress`: Lambda -> API (progress updates, stored in Redis).
|
|
76
|
+
- `OperationResult`: Lambda -> API (terminal state, stored in DB).
|
|
77
|
+
|
|
78
|
+
`JobRequest` and `OperationRequest` share the same payload schema
|
|
79
|
+
(`OperationRequest`). They differ only in the SNS message attributes set
|
|
80
|
+
as headers, captured in two separate header schemas (`JobRequestAttributes`
|
|
81
|
+
and `OperationRequestAttributes`) so each topic's required attributes can
|
|
82
|
+
be enforced cleanly without conditional discriminators.
|
|
83
|
+
|
|
84
|
+
servers:
|
|
85
|
+
local:
|
|
86
|
+
host: localhost:4566
|
|
87
|
+
protocol: sqs
|
|
88
|
+
description: LocalStack for local development
|
|
89
|
+
staging:
|
|
90
|
+
host: sqs.eu-west-1.amazonaws.com
|
|
91
|
+
protocol: sqs
|
|
92
|
+
description: AWS SQS - Staging environment
|
|
93
|
+
|
|
94
|
+
defaultContentType: application/json
|
|
95
|
+
|
|
96
|
+
channels:
|
|
97
|
+
# ============================================
|
|
98
|
+
# SNS REQUEST TOPICS (API -> Lambda)
|
|
99
|
+
# ============================================
|
|
100
|
+
|
|
101
|
+
jobRequestsTopic:
|
|
102
|
+
address: gisl-{env}-{region}-job-requests
|
|
103
|
+
description: |
|
|
104
|
+
SNS topic where the API publishes **compression** operation requests.
|
|
105
|
+
Filter attribute: `job_type` (single-attribute filter policy, not
|
|
106
|
+
compound). Subscriptions fan out to four per-media-type compression
|
|
107
|
+
queues, keyed by `job_type` value.
|
|
108
|
+
|
|
109
|
+
**Filter vocabulary:**
|
|
110
|
+
- `job_type = image` -> `jobs-image` queue
|
|
111
|
+
- `job_type = video` -> `jobs-video` queue
|
|
112
|
+
- `job_type = audio` -> `jobs-audio` queue
|
|
113
|
+
- `job_type = document` -> `jobs-document` queue
|
|
114
|
+
|
|
115
|
+
The API's Option A publisher sets only `job_type` as a message
|
|
116
|
+
attribute on this branch. `operation_type` and `media_group` are
|
|
117
|
+
**not** set on this branch — the media group is encoded as
|
|
118
|
+
`job_type` and no other attribute is used as a filter on this topic.
|
|
119
|
+
See the `publishJobRequest` operation below for the publisher
|
|
120
|
+
branching rule.
|
|
121
|
+
parameters:
|
|
122
|
+
env:
|
|
123
|
+
description: Environment (local, stg, prod)
|
|
124
|
+
region:
|
|
125
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
126
|
+
messages:
|
|
127
|
+
jobRequest:
|
|
128
|
+
$ref: '#/components/messages/JobRequestMessage'
|
|
129
|
+
|
|
130
|
+
operationsTopic:
|
|
131
|
+
address: gisl-{env}-{region}-operations
|
|
132
|
+
description: |
|
|
133
|
+
SNS topic where the API publishes **non-compression** operation
|
|
134
|
+
requests (thumbnail, watermark, merge, archive, convert). Filter
|
|
135
|
+
attribute: `operation_type` (single-attribute filter policy).
|
|
136
|
+
Subscriptions fan out to per-operation-type queues, keyed by
|
|
137
|
+
`operation_type` value.
|
|
138
|
+
|
|
139
|
+
**Filter vocabulary:**
|
|
140
|
+
- `operation_type = thumbnail` -> `ops-thumbnail` queue (legacy)
|
|
141
|
+
- `operation_type = thumbnail_image` -> `ops-thumbnail-image` queue
|
|
142
|
+
- `operation_type = thumbnail_video` -> `ops-thumbnail-video` queue
|
|
143
|
+
- `operation_type = thumbnail_document` -> `ops-thumbnail-document` queue
|
|
144
|
+
- `operation_type = thumbnail_office` -> `ops-thumbnail-office` queue
|
|
145
|
+
- `operation_type = watermark` -> `ops-watermark` queue
|
|
146
|
+
- `operation_type = merge` -> `ops-merge` queue
|
|
147
|
+
- `operation_type = archive` -> `ops-archive` queue
|
|
148
|
+
- `operation_type = convert` -> `ops-convert` queue
|
|
149
|
+
|
|
150
|
+
The API's Option A publisher sets `operation_type` as the filter
|
|
151
|
+
attribute and additionally sets `media_group` as an **informational**
|
|
152
|
+
message attribute where a media group is meaningful (omitted for
|
|
153
|
+
archive). `media_group` is **not** used by SNS as a routing filter
|
|
154
|
+
on this topic — it is preserved for consumer observability and log
|
|
155
|
+
correlation. See the `publishOperationRequest` operation below for
|
|
156
|
+
the publisher branching rule.
|
|
157
|
+
|
|
158
|
+
**Thumbnail migration window.** The legacy `ops-thumbnail` queue and
|
|
159
|
+
the four sub-type queues are all live simultaneously. The API
|
|
160
|
+
publisher currently emits `operation_type=thumbnail` (legacy);
|
|
161
|
+
adoption of the four sub-type values is a follow-up API PR. Both
|
|
162
|
+
paths are valid routing targets today.
|
|
163
|
+
parameters:
|
|
164
|
+
env:
|
|
165
|
+
description: Environment (local, stg, prod)
|
|
166
|
+
region:
|
|
167
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
168
|
+
messages:
|
|
169
|
+
operationRequest:
|
|
170
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
171
|
+
|
|
172
|
+
# ============================================
|
|
173
|
+
# JOBS FAMILY - COMPRESSION QUEUES (per media type)
|
|
174
|
+
# ============================================
|
|
175
|
+
|
|
176
|
+
jobsImage:
|
|
177
|
+
address: gisl-{env}-{region}-jobs-image
|
|
178
|
+
description: |
|
|
179
|
+
SQS queue for image compression jobs.
|
|
180
|
+
Subscribed to the `job-requests` SNS topic with filter
|
|
181
|
+
`job_type = image`. Handles `compress` only for image inputs.
|
|
182
|
+
parameters:
|
|
183
|
+
env:
|
|
184
|
+
description: Environment (local, stg, prod)
|
|
185
|
+
region:
|
|
186
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
187
|
+
messages:
|
|
188
|
+
jobRequest:
|
|
189
|
+
$ref: '#/components/messages/JobRequestMessage'
|
|
190
|
+
|
|
191
|
+
jobsVideo:
|
|
192
|
+
address: gisl-{env}-{region}-jobs-video
|
|
193
|
+
description: |
|
|
194
|
+
SQS queue for video compression jobs.
|
|
195
|
+
Subscribed to the `job-requests` SNS topic with filter
|
|
196
|
+
`job_type = video`. Handles `compress` only for video inputs.
|
|
197
|
+
parameters:
|
|
198
|
+
env:
|
|
199
|
+
description: Environment (local, stg, prod)
|
|
200
|
+
region:
|
|
201
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
202
|
+
messages:
|
|
203
|
+
jobRequest:
|
|
204
|
+
$ref: '#/components/messages/JobRequestMessage'
|
|
205
|
+
|
|
206
|
+
jobsAudio:
|
|
207
|
+
address: gisl-{env}-{region}-jobs-audio
|
|
208
|
+
description: |
|
|
209
|
+
SQS queue for audio compression jobs.
|
|
210
|
+
Subscribed to the `job-requests` SNS topic with filter
|
|
211
|
+
`job_type = audio`. Handles `compress` only for audio inputs.
|
|
212
|
+
parameters:
|
|
213
|
+
env:
|
|
214
|
+
description: Environment (local, stg, prod)
|
|
215
|
+
region:
|
|
216
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
217
|
+
messages:
|
|
218
|
+
jobRequest:
|
|
219
|
+
$ref: '#/components/messages/JobRequestMessage'
|
|
220
|
+
|
|
221
|
+
jobsDocument:
|
|
222
|
+
address: gisl-{env}-{region}-jobs-document
|
|
223
|
+
description: |
|
|
224
|
+
SQS queue for document compression jobs.
|
|
225
|
+
Subscribed to the `job-requests` SNS topic with filter
|
|
226
|
+
`job_type = document`. Handles `compress` only for document inputs.
|
|
227
|
+
parameters:
|
|
228
|
+
env:
|
|
229
|
+
description: Environment (local, stg, prod)
|
|
230
|
+
region:
|
|
231
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
232
|
+
messages:
|
|
233
|
+
jobRequest:
|
|
234
|
+
$ref: '#/components/messages/JobRequestMessage'
|
|
235
|
+
|
|
236
|
+
# ============================================
|
|
237
|
+
# JOBS FAMILY - DEAD LETTER QUEUES
|
|
238
|
+
# ============================================
|
|
239
|
+
|
|
240
|
+
jobsImageDlq:
|
|
241
|
+
address: gisl-{env}-{region}-jobs-image-dlq
|
|
242
|
+
description: DLQ for failed image compression jobs. Messages land here after 5 failed processing attempts.
|
|
243
|
+
parameters:
|
|
244
|
+
env:
|
|
245
|
+
description: Environment (local, stg, prod)
|
|
246
|
+
region:
|
|
247
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
248
|
+
|
|
249
|
+
jobsVideoDlq:
|
|
250
|
+
address: gisl-{env}-{region}-jobs-video-dlq
|
|
251
|
+
description: DLQ for failed video compression jobs. Messages land here after 5 failed processing attempts.
|
|
252
|
+
parameters:
|
|
253
|
+
env:
|
|
254
|
+
description: Environment (local, stg, prod)
|
|
255
|
+
region:
|
|
256
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
257
|
+
|
|
258
|
+
jobsAudioDlq:
|
|
259
|
+
address: gisl-{env}-{region}-jobs-audio-dlq
|
|
260
|
+
description: DLQ for failed audio compression jobs. Messages land here after 5 failed processing attempts.
|
|
261
|
+
parameters:
|
|
262
|
+
env:
|
|
263
|
+
description: Environment (local, stg, prod)
|
|
264
|
+
region:
|
|
265
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
266
|
+
|
|
267
|
+
jobsDocumentDlq:
|
|
268
|
+
address: gisl-{env}-{region}-jobs-document-dlq
|
|
269
|
+
description: DLQ for failed document compression jobs. Messages land here after 5 failed processing attempts.
|
|
270
|
+
parameters:
|
|
271
|
+
env:
|
|
272
|
+
description: Environment (local, stg, prod)
|
|
273
|
+
region:
|
|
274
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
275
|
+
|
|
276
|
+
# ============================================
|
|
277
|
+
# OPS FAMILY - NON-COMPRESSION OPERATION QUEUES
|
|
278
|
+
# ============================================
|
|
279
|
+
|
|
280
|
+
opsThumbnail:
|
|
281
|
+
address: gisl-{env}-{region}-ops-thumbnail
|
|
282
|
+
description: |
|
|
283
|
+
SQS queue for legacy thumbnail operations.
|
|
284
|
+
Subscribed to the `operations` SNS topic with filter
|
|
285
|
+
`operation_type = thumbnail`.
|
|
286
|
+
|
|
287
|
+
**Migration window**: This is the legacy routing target and is
|
|
288
|
+
currently the only one the API publisher emits for (via
|
|
289
|
+
`operation_type=thumbnail`). It handles thumbnail requests for all
|
|
290
|
+
media types. Retirement is planned after the API publisher adopts
|
|
291
|
+
the four sub-type values (`thumbnail_image`, `thumbnail_video`,
|
|
292
|
+
`thumbnail_document`, `thumbnail_office`) in a follow-up API PR.
|
|
293
|
+
parameters:
|
|
294
|
+
env:
|
|
295
|
+
description: Environment (local, stg, prod)
|
|
296
|
+
region:
|
|
297
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
298
|
+
messages:
|
|
299
|
+
operationRequest:
|
|
300
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
301
|
+
|
|
302
|
+
opsThumbnailImage:
|
|
303
|
+
address: gisl-{env}-{region}-ops-thumbnail-image
|
|
304
|
+
description: |
|
|
305
|
+
SQS queue for image thumbnail operations.
|
|
306
|
+
Subscribed to the `operations` SNS topic with filter
|
|
307
|
+
`operation_type = thumbnail_image`. Backed by the Rust image
|
|
308
|
+
thumbnail Lambda (`operation-thumbnail-image`).
|
|
309
|
+
|
|
310
|
+
Not yet receiving traffic — the API publisher currently emits
|
|
311
|
+
`operation_type=thumbnail` (legacy) and will flip to
|
|
312
|
+
`thumbnail_image` after a follow-up API PR adds input-MIME dispatch.
|
|
313
|
+
parameters:
|
|
314
|
+
env:
|
|
315
|
+
description: Environment (local, stg, prod)
|
|
316
|
+
region:
|
|
317
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
318
|
+
messages:
|
|
319
|
+
operationRequest:
|
|
320
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
321
|
+
|
|
322
|
+
opsThumbnailVideo:
|
|
323
|
+
address: gisl-{env}-{region}-ops-thumbnail-video
|
|
324
|
+
description: |
|
|
325
|
+
SQS queue for video thumbnail operations.
|
|
326
|
+
Subscribed to the `operations` SNS topic with filter
|
|
327
|
+
`operation_type = thumbnail_video`. Backed by the FFmpeg video
|
|
328
|
+
thumbnail Lambda (`operation-thumbnail-video`).
|
|
329
|
+
|
|
330
|
+
Not yet receiving traffic — the API publisher currently emits
|
|
331
|
+
`operation_type=thumbnail` (legacy) and will flip to
|
|
332
|
+
`thumbnail_video` after a follow-up API PR adds input-MIME dispatch.
|
|
333
|
+
parameters:
|
|
334
|
+
env:
|
|
335
|
+
description: Environment (local, stg, prod)
|
|
336
|
+
region:
|
|
337
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
338
|
+
messages:
|
|
339
|
+
operationRequest:
|
|
340
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
341
|
+
|
|
342
|
+
opsThumbnailDocument:
|
|
343
|
+
address: gisl-{env}-{region}-ops-thumbnail-document
|
|
344
|
+
description: |
|
|
345
|
+
SQS queue for PDF/EPUB document thumbnail operations.
|
|
346
|
+
Subscribed to the `operations` SNS topic with filter
|
|
347
|
+
`operation_type = thumbnail_document`. Backed by the Ghostscript
|
|
348
|
+
document thumbnail Lambda (`operation-thumbnail-document`).
|
|
349
|
+
|
|
350
|
+
Not yet receiving traffic — the API publisher currently emits
|
|
351
|
+
`operation_type=thumbnail` (legacy) and will flip to
|
|
352
|
+
`thumbnail_document` after a follow-up API PR adds input-MIME
|
|
353
|
+
dispatch.
|
|
354
|
+
parameters:
|
|
355
|
+
env:
|
|
356
|
+
description: Environment (local, stg, prod)
|
|
357
|
+
region:
|
|
358
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
359
|
+
messages:
|
|
360
|
+
operationRequest:
|
|
361
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
362
|
+
|
|
363
|
+
opsThumbnailOffice:
|
|
364
|
+
address: gisl-{env}-{region}-ops-thumbnail-office
|
|
365
|
+
description: |
|
|
366
|
+
SQS queue for office document thumbnail operations
|
|
367
|
+
(DOCX, XLSX, PPTX, ODT, ODS, ODP).
|
|
368
|
+
Subscribed to the `operations` SNS topic with filter
|
|
369
|
+
`operation_type = thumbnail_office`. Backed by the LibreOffice
|
|
370
|
+
thumbnail Lambda (`operation-thumbnail-office`).
|
|
371
|
+
|
|
372
|
+
Not yet receiving traffic — the API publisher currently emits
|
|
373
|
+
`operation_type=thumbnail` (legacy) and will flip to
|
|
374
|
+
`thumbnail_office` after a follow-up API PR adds input-MIME
|
|
375
|
+
dispatch.
|
|
376
|
+
parameters:
|
|
377
|
+
env:
|
|
378
|
+
description: Environment (local, stg, prod)
|
|
379
|
+
region:
|
|
380
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
381
|
+
messages:
|
|
382
|
+
operationRequest:
|
|
383
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
384
|
+
|
|
385
|
+
opsWatermark:
|
|
386
|
+
address: gisl-{env}-{region}-ops-watermark
|
|
387
|
+
description: |
|
|
388
|
+
SQS queue for watermark operations (image-only).
|
|
389
|
+
Subscribed to the `operations` SNS topic with filter
|
|
390
|
+
`operation_type = watermark`. Handles image-overlay and text-overlay
|
|
391
|
+
watermark modes on image inputs. Backed by the watermark Lambda
|
|
392
|
+
(`operation-watermark`).
|
|
393
|
+
parameters:
|
|
394
|
+
env:
|
|
395
|
+
description: Environment (local, stg, prod)
|
|
396
|
+
region:
|
|
397
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
398
|
+
messages:
|
|
399
|
+
operationRequest:
|
|
400
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
401
|
+
|
|
402
|
+
opsMerge:
|
|
403
|
+
address: gisl-{env}-{region}-ops-merge
|
|
404
|
+
description: |
|
|
405
|
+
SQS queue for merge operations.
|
|
406
|
+
Subscribed to the `operations` SNS topic with filter
|
|
407
|
+
`operation_type = merge`. A single `operation-merge` Lambda handles
|
|
408
|
+
all merge output types (image collage/grid, animated GIF, video
|
|
409
|
+
slideshow/concat, audio concat, PDF concat) — the `output_type`
|
|
410
|
+
field on the request determines the encoding path internally, not
|
|
411
|
+
the routing.
|
|
412
|
+
parameters:
|
|
413
|
+
env:
|
|
414
|
+
description: Environment (local, stg, prod)
|
|
415
|
+
region:
|
|
416
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
417
|
+
messages:
|
|
418
|
+
operationRequest:
|
|
419
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
420
|
+
|
|
421
|
+
opsArchive:
|
|
422
|
+
address: gisl-{env}-{region}-ops-archive
|
|
423
|
+
description: |
|
|
424
|
+
SQS queue for archive operations (ZIP/tar.gz bundling).
|
|
425
|
+
Subscribed to the `operations` SNS topic with filter
|
|
426
|
+
`operation_type = archive`. Media-agnostic: accepts mixed input
|
|
427
|
+
types. Backed by the archive Lambda (`operation-archive`).
|
|
428
|
+
parameters:
|
|
429
|
+
env:
|
|
430
|
+
description: Environment (local, stg, prod)
|
|
431
|
+
region:
|
|
432
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
433
|
+
messages:
|
|
434
|
+
operationRequest:
|
|
435
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
436
|
+
|
|
437
|
+
opsConvert:
|
|
438
|
+
address: gisl-{env}-{region}-ops-convert
|
|
439
|
+
description: |
|
|
440
|
+
SQS queue for convert operations (format conversion).
|
|
441
|
+
Subscribed to the `operations` SNS topic with filter
|
|
442
|
+
`operation_type = convert`. Handles format conversion across all
|
|
443
|
+
media types (image, video, audio, document). Backed by the convert
|
|
444
|
+
Lambda (`operation-convert`).
|
|
445
|
+
parameters:
|
|
446
|
+
env:
|
|
447
|
+
description: Environment (local, stg, prod)
|
|
448
|
+
region:
|
|
449
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
450
|
+
messages:
|
|
451
|
+
operationRequest:
|
|
452
|
+
$ref: '#/components/messages/OperationRequestMessage'
|
|
453
|
+
|
|
454
|
+
# ============================================
|
|
455
|
+
# OPS FAMILY - DEAD LETTER QUEUES
|
|
456
|
+
# ============================================
|
|
457
|
+
|
|
458
|
+
opsThumbnailDlq:
|
|
459
|
+
address: gisl-{env}-{region}-ops-thumbnail-dlq
|
|
460
|
+
description: DLQ for failed legacy thumbnail operations. Messages land here after 5 failed processing attempts.
|
|
461
|
+
parameters:
|
|
462
|
+
env:
|
|
463
|
+
description: Environment (local, stg, prod)
|
|
464
|
+
region:
|
|
465
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
466
|
+
|
|
467
|
+
opsThumbnailImageDlq:
|
|
468
|
+
address: gisl-{env}-{region}-ops-thumbnail-image-dlq
|
|
469
|
+
description: DLQ for failed image thumbnail operations. Messages land here after 5 failed processing attempts.
|
|
470
|
+
parameters:
|
|
471
|
+
env:
|
|
472
|
+
description: Environment (local, stg, prod)
|
|
473
|
+
region:
|
|
474
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
475
|
+
|
|
476
|
+
opsThumbnailVideoDlq:
|
|
477
|
+
address: gisl-{env}-{region}-ops-thumbnail-video-dlq
|
|
478
|
+
description: DLQ for failed video thumbnail operations. Messages land here after 5 failed processing attempts.
|
|
479
|
+
parameters:
|
|
480
|
+
env:
|
|
481
|
+
description: Environment (local, stg, prod)
|
|
482
|
+
region:
|
|
483
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
484
|
+
|
|
485
|
+
opsThumbnailDocumentDlq:
|
|
486
|
+
address: gisl-{env}-{region}-ops-thumbnail-document-dlq
|
|
487
|
+
description: DLQ for failed document thumbnail operations. Messages land here after 5 failed processing attempts.
|
|
488
|
+
parameters:
|
|
489
|
+
env:
|
|
490
|
+
description: Environment (local, stg, prod)
|
|
491
|
+
region:
|
|
492
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
493
|
+
|
|
494
|
+
opsThumbnailOfficeDlq:
|
|
495
|
+
address: gisl-{env}-{region}-ops-thumbnail-office-dlq
|
|
496
|
+
description: DLQ for failed office thumbnail operations. Messages land here after 5 failed processing attempts.
|
|
497
|
+
parameters:
|
|
498
|
+
env:
|
|
499
|
+
description: Environment (local, stg, prod)
|
|
500
|
+
region:
|
|
501
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
502
|
+
|
|
503
|
+
opsWatermarkDlq:
|
|
504
|
+
address: gisl-{env}-{region}-ops-watermark-dlq
|
|
505
|
+
description: DLQ for failed watermark operations. Messages land here after 5 failed processing attempts.
|
|
506
|
+
parameters:
|
|
507
|
+
env:
|
|
508
|
+
description: Environment (local, stg, prod)
|
|
509
|
+
region:
|
|
510
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
511
|
+
|
|
512
|
+
opsMergeDlq:
|
|
513
|
+
address: gisl-{env}-{region}-ops-merge-dlq
|
|
514
|
+
description: DLQ for failed merge operations. Messages land here after 5 failed processing attempts.
|
|
515
|
+
parameters:
|
|
516
|
+
env:
|
|
517
|
+
description: Environment (local, stg, prod)
|
|
518
|
+
region:
|
|
519
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
520
|
+
|
|
521
|
+
opsArchiveDlq:
|
|
522
|
+
address: gisl-{env}-{region}-ops-archive-dlq
|
|
523
|
+
description: DLQ for failed archive operations. Messages land here after 5 failed processing attempts.
|
|
524
|
+
parameters:
|
|
525
|
+
env:
|
|
526
|
+
description: Environment (local, stg, prod)
|
|
527
|
+
region:
|
|
528
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
529
|
+
|
|
530
|
+
opsConvertDlq:
|
|
531
|
+
address: gisl-{env}-{region}-ops-convert-dlq
|
|
532
|
+
description: DLQ for failed convert operations. Messages land here after 5 failed processing attempts.
|
|
533
|
+
parameters:
|
|
534
|
+
env:
|
|
535
|
+
description: Environment (local, stg, prod)
|
|
536
|
+
region:
|
|
537
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
538
|
+
|
|
539
|
+
# ============================================
|
|
540
|
+
# NOTIFICATIONS (Lambda -> API)
|
|
541
|
+
# ============================================
|
|
542
|
+
|
|
543
|
+
notificationsOperationsTopic:
|
|
544
|
+
address: gisl-{env}-{region}-notifications-operations.fifo
|
|
545
|
+
description: |
|
|
546
|
+
FIFO SNS topic where Lambdas publish all operation notifications (progress and results).
|
|
547
|
+
API subscribes via FIFO SQS with Raw Message Delivery enabled.
|
|
548
|
+
|
|
549
|
+
**FIFO Publishing Requirements:**
|
|
550
|
+
- MessageGroupId: Use operation_id (ensures ordering per operation)
|
|
551
|
+
- MessageDeduplicationId: Not required (content-based deduplication enabled)
|
|
552
|
+
parameters:
|
|
553
|
+
env:
|
|
554
|
+
description: Environment (local, stg, prod)
|
|
555
|
+
region:
|
|
556
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
557
|
+
messages:
|
|
558
|
+
operationProgress:
|
|
559
|
+
$ref: '#/components/messages/OperationProgressMessage'
|
|
560
|
+
operationResult:
|
|
561
|
+
$ref: '#/components/messages/OperationResultMessage'
|
|
562
|
+
|
|
563
|
+
notificationsOperationsQueue:
|
|
564
|
+
address: gisl-{env}-{region}-notifications-operations.fifo
|
|
565
|
+
description: |
|
|
566
|
+
FIFO SQS queue subscribed to notifications-operations SNS topic.
|
|
567
|
+
Receives both OperationProgress and OperationResult messages.
|
|
568
|
+
Raw Message Delivery enabled (no SNS envelope).
|
|
569
|
+
|
|
570
|
+
FIFO ensures message ordering per operation (operation_id used as MessageGroupId).
|
|
571
|
+
This guarantees OperationProgress messages arrive before OperationResult.
|
|
572
|
+
parameters:
|
|
573
|
+
env:
|
|
574
|
+
description: Environment (local, stg, prod)
|
|
575
|
+
region:
|
|
576
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
577
|
+
messages:
|
|
578
|
+
operationProgress:
|
|
579
|
+
$ref: '#/components/messages/OperationProgressMessage'
|
|
580
|
+
operationResult:
|
|
581
|
+
$ref: '#/components/messages/OperationResultMessage'
|
|
582
|
+
|
|
583
|
+
notificationsOperationsDlq:
|
|
584
|
+
address: gisl-{env}-{region}-notifications-operations-dlq.fifo
|
|
585
|
+
description: DLQ for failed notification processing. Messages land here after 5 failed processing attempts.
|
|
586
|
+
parameters:
|
|
587
|
+
env:
|
|
588
|
+
description: Environment (local, stg, prod)
|
|
589
|
+
region:
|
|
590
|
+
description: AWS region abbreviation (euw1, use1, etc.)
|
|
591
|
+
|
|
592
|
+
operations:
|
|
593
|
+
# ============================================
|
|
594
|
+
# PUBLISH OPERATIONS (API -> SNS)
|
|
595
|
+
# ============================================
|
|
596
|
+
|
|
597
|
+
publishJobRequest:
|
|
598
|
+
action: send
|
|
599
|
+
channel:
|
|
600
|
+
$ref: '#/channels/jobRequestsTopic'
|
|
601
|
+
summary: Publish a compression job request to the job-requests SNS topic.
|
|
602
|
+
description: |
|
|
603
|
+
Compression-branch publisher path. Invoked by the API for any
|
|
604
|
+
operation with `operation_type == compress`, regardless of media
|
|
605
|
+
type. The target SNS topic filters subscriptions on `job_type`
|
|
606
|
+
alone.
|
|
607
|
+
|
|
608
|
+
**Message attribute set on this branch:** `{job_type}` only. The
|
|
609
|
+
publisher does **not** set `operation_type` or `media_group` on
|
|
610
|
+
this branch — the media group is encoded as `job_type` and no
|
|
611
|
+
other attribute is used as a filter on this topic.
|
|
612
|
+
|
|
613
|
+
**`job_type` value derivation:** the API derives `job_type` from
|
|
614
|
+
its `OperationMediaGroupResolver` output
|
|
615
|
+
(`image` | `video` | `audio` | `document`) at publish time.
|
|
616
|
+
|
|
617
|
+
See `info.description` above for the full publisher branching rule.
|
|
618
|
+
messages:
|
|
619
|
+
- $ref: '#/channels/jobRequestsTopic/messages/jobRequest'
|
|
620
|
+
|
|
621
|
+
publishOperationRequest:
|
|
622
|
+
action: send
|
|
623
|
+
channel:
|
|
624
|
+
$ref: '#/channels/operationsTopic'
|
|
625
|
+
summary: Publish a non-compression operation request to the operations SNS topic.
|
|
626
|
+
description: |
|
|
627
|
+
Non-compression-branch publisher path. Invoked by the API for any
|
|
628
|
+
operation where `operation_type != compress` — that is, thumbnail
|
|
629
|
+
(including the four sub-type values during the migration window),
|
|
630
|
+
watermark, merge, archive, convert. The target SNS topic filters
|
|
631
|
+
subscriptions on `operation_type` alone.
|
|
632
|
+
|
|
633
|
+
**Message attributes set on this branch:** `{operation_type,
|
|
634
|
+
media_group}`, with `media_group` omitted for `archive` (which is
|
|
635
|
+
media-agnostic). `operation_type` is the filter attribute;
|
|
636
|
+
`media_group` is informational metadata preserved for consumer
|
|
637
|
+
observability and log correlation and is **not** used by SNS as a
|
|
638
|
+
routing filter.
|
|
639
|
+
|
|
640
|
+
See `info.description` above for the full publisher branching rule.
|
|
641
|
+
messages:
|
|
642
|
+
- $ref: '#/channels/operationsTopic/messages/operationRequest'
|
|
643
|
+
|
|
644
|
+
# ============================================
|
|
645
|
+
# CONSUME OPERATIONS - JOBS FAMILY (COMPRESSION)
|
|
646
|
+
# ============================================
|
|
647
|
+
|
|
648
|
+
consumeImageOperation:
|
|
649
|
+
action: receive
|
|
650
|
+
channel:
|
|
651
|
+
$ref: '#/channels/jobsImage'
|
|
652
|
+
summary: Process image compression job
|
|
653
|
+
description: |
|
|
654
|
+
The image compression Lambda consumes jobs for image inputs.
|
|
655
|
+
Handles `compress` only. Non-compression image operations
|
|
656
|
+
(thumbnail, watermark, convert, merge) are routed to their
|
|
657
|
+
respective queues on the ops-family under the `operations` topic.
|
|
658
|
+
messages:
|
|
659
|
+
- $ref: '#/channels/jobsImage/messages/jobRequest'
|
|
660
|
+
|
|
661
|
+
consumeVideoOperation:
|
|
662
|
+
action: receive
|
|
663
|
+
channel:
|
|
664
|
+
$ref: '#/channels/jobsVideo'
|
|
665
|
+
summary: Process video compression job
|
|
666
|
+
description: |
|
|
667
|
+
The video compression Lambda consumes jobs for video inputs.
|
|
668
|
+
Handles `compress` only. Non-compression video operations
|
|
669
|
+
(thumbnail, convert, merge) are routed to their respective queues
|
|
670
|
+
on the ops-family under the `operations` topic.
|
|
671
|
+
messages:
|
|
672
|
+
- $ref: '#/channels/jobsVideo/messages/jobRequest'
|
|
673
|
+
|
|
674
|
+
consumeAudioOperation:
|
|
675
|
+
action: receive
|
|
676
|
+
channel:
|
|
677
|
+
$ref: '#/channels/jobsAudio'
|
|
678
|
+
summary: Process audio compression job
|
|
679
|
+
description: |
|
|
680
|
+
The audio compression Lambda consumes jobs for audio inputs.
|
|
681
|
+
Handles `compress` only. Non-compression audio operations
|
|
682
|
+
(convert, merge) are routed to their respective queues on the
|
|
683
|
+
ops-family under the `operations` topic.
|
|
684
|
+
messages:
|
|
685
|
+
- $ref: '#/channels/jobsAudio/messages/jobRequest'
|
|
686
|
+
|
|
687
|
+
consumeDocumentOperation:
|
|
688
|
+
action: receive
|
|
689
|
+
channel:
|
|
690
|
+
$ref: '#/channels/jobsDocument'
|
|
691
|
+
summary: Process document compression job
|
|
692
|
+
description: |
|
|
693
|
+
The document compression Lambda consumes jobs for document inputs.
|
|
694
|
+
Handles `compress` only. Non-compression document operations
|
|
695
|
+
(thumbnail, convert, merge) are routed to their respective queues
|
|
696
|
+
on the ops-family under the `operations` topic.
|
|
697
|
+
messages:
|
|
698
|
+
- $ref: '#/channels/jobsDocument/messages/jobRequest'
|
|
699
|
+
|
|
700
|
+
# ============================================
|
|
701
|
+
# CONSUME OPERATIONS - OPS FAMILY (NON-COMPRESSION)
|
|
702
|
+
# ============================================
|
|
703
|
+
|
|
704
|
+
consumeThumbnailOperation:
|
|
705
|
+
action: receive
|
|
706
|
+
channel:
|
|
707
|
+
$ref: '#/channels/opsThumbnail'
|
|
708
|
+
summary: Process legacy thumbnail operation
|
|
709
|
+
description: |
|
|
710
|
+
The legacy thumbnail Lambda consumes thumbnail requests routed
|
|
711
|
+
by `operation_type=thumbnail`. Handles thumbnail generation for
|
|
712
|
+
all media types in a single Lambda. This is the current live
|
|
713
|
+
routing target for the API publisher; it will be retired after
|
|
714
|
+
the publisher adopts the four sub-type values.
|
|
715
|
+
messages:
|
|
716
|
+
- $ref: '#/channels/opsThumbnail/messages/operationRequest'
|
|
717
|
+
|
|
718
|
+
consumeThumbnailImageOperation:
|
|
719
|
+
action: receive
|
|
720
|
+
channel:
|
|
721
|
+
$ref: '#/channels/opsThumbnailImage'
|
|
722
|
+
summary: Process image thumbnail operation
|
|
723
|
+
description: |
|
|
724
|
+
The image thumbnail Lambda consumes thumbnail requests routed by
|
|
725
|
+
`operation_type=thumbnail_image`. Backed by a Rust image crate.
|
|
726
|
+
Not yet receiving traffic — see channel description.
|
|
727
|
+
messages:
|
|
728
|
+
- $ref: '#/channels/opsThumbnailImage/messages/operationRequest'
|
|
729
|
+
|
|
730
|
+
consumeThumbnailVideoOperation:
|
|
731
|
+
action: receive
|
|
732
|
+
channel:
|
|
733
|
+
$ref: '#/channels/opsThumbnailVideo'
|
|
734
|
+
summary: Process video thumbnail operation
|
|
735
|
+
description: |
|
|
736
|
+
The video thumbnail Lambda consumes thumbnail requests routed by
|
|
737
|
+
`operation_type=thumbnail_video`. Backed by FFmpeg. Not yet
|
|
738
|
+
receiving traffic — see channel description.
|
|
739
|
+
messages:
|
|
740
|
+
- $ref: '#/channels/opsThumbnailVideo/messages/operationRequest'
|
|
741
|
+
|
|
742
|
+
consumeThumbnailDocumentOperation:
|
|
743
|
+
action: receive
|
|
744
|
+
channel:
|
|
745
|
+
$ref: '#/channels/opsThumbnailDocument'
|
|
746
|
+
summary: Process document thumbnail operation
|
|
747
|
+
description: |
|
|
748
|
+
The document (PDF/EPUB) thumbnail Lambda consumes thumbnail
|
|
749
|
+
requests routed by `operation_type=thumbnail_document`. Backed by
|
|
750
|
+
Ghostscript. Not yet receiving traffic — see channel description.
|
|
751
|
+
messages:
|
|
752
|
+
- $ref: '#/channels/opsThumbnailDocument/messages/operationRequest'
|
|
753
|
+
|
|
754
|
+
consumeThumbnailOfficeOperation:
|
|
755
|
+
action: receive
|
|
756
|
+
channel:
|
|
757
|
+
$ref: '#/channels/opsThumbnailOffice'
|
|
758
|
+
summary: Process office document thumbnail operation
|
|
759
|
+
description: |
|
|
760
|
+
The office document (DOCX/XLSX/PPTX/ODT/ODS/ODP) thumbnail Lambda
|
|
761
|
+
consumes thumbnail requests routed by
|
|
762
|
+
`operation_type=thumbnail_office`. Backed by LibreOffice. Not yet
|
|
763
|
+
receiving traffic — see channel description.
|
|
764
|
+
messages:
|
|
765
|
+
- $ref: '#/channels/opsThumbnailOffice/messages/operationRequest'
|
|
766
|
+
|
|
767
|
+
consumeWatermarkOperation:
|
|
768
|
+
action: receive
|
|
769
|
+
channel:
|
|
770
|
+
$ref: '#/channels/opsWatermark'
|
|
771
|
+
summary: Process watermark operation
|
|
772
|
+
description: |
|
|
773
|
+
The watermark Lambda consumes watermark requests routed by
|
|
774
|
+
`operation_type=watermark`. Image-only — handles both
|
|
775
|
+
image-overlay and text-overlay modes.
|
|
776
|
+
messages:
|
|
777
|
+
- $ref: '#/channels/opsWatermark/messages/operationRequest'
|
|
778
|
+
|
|
779
|
+
consumeMergeOperation:
|
|
780
|
+
action: receive
|
|
781
|
+
channel:
|
|
782
|
+
$ref: '#/channels/opsMerge'
|
|
783
|
+
summary: Process merge operation
|
|
784
|
+
description: |
|
|
785
|
+
The merge Lambda consumes merge requests routed by
|
|
786
|
+
`operation_type=merge`. A single Lambda handles all merge output
|
|
787
|
+
types (image collage/grid, animated GIF, video slideshow/concat,
|
|
788
|
+
audio concat, PDF concat). The `output_type` field on the request
|
|
789
|
+
determines the encoding path internally.
|
|
790
|
+
messages:
|
|
791
|
+
- $ref: '#/channels/opsMerge/messages/operationRequest'
|
|
792
|
+
|
|
793
|
+
consumeArchiveOperation:
|
|
794
|
+
action: receive
|
|
795
|
+
channel:
|
|
796
|
+
$ref: '#/channels/opsArchive'
|
|
797
|
+
summary: Process archive operation
|
|
798
|
+
description: |
|
|
799
|
+
The archive Lambda consumes archive requests routed by
|
|
800
|
+
`operation_type=archive`. Media-agnostic: bundles files of any
|
|
801
|
+
type into ZIP/tar.gz.
|
|
802
|
+
messages:
|
|
803
|
+
- $ref: '#/channels/opsArchive/messages/operationRequest'
|
|
804
|
+
|
|
805
|
+
consumeConvertOperation:
|
|
806
|
+
action: receive
|
|
807
|
+
channel:
|
|
808
|
+
$ref: '#/channels/opsConvert'
|
|
809
|
+
summary: Process convert operation
|
|
810
|
+
description: |
|
|
811
|
+
The convert Lambda consumes convert requests routed by
|
|
812
|
+
`operation_type=convert`. Handles format conversion across all
|
|
813
|
+
media types (image, video, audio, document).
|
|
814
|
+
messages:
|
|
815
|
+
- $ref: '#/channels/opsConvert/messages/operationRequest'
|
|
816
|
+
|
|
817
|
+
# ============================================
|
|
818
|
+
# NOTIFICATION OPERATIONS
|
|
819
|
+
# ============================================
|
|
820
|
+
|
|
821
|
+
publishOperationProgress:
|
|
822
|
+
action: send
|
|
823
|
+
channel:
|
|
824
|
+
$ref: '#/channels/notificationsOperationsTopic'
|
|
825
|
+
summary: Publish operation progress update
|
|
826
|
+
description: |
|
|
827
|
+
Lambda publishes progress updates during operation processing.
|
|
828
|
+
These are lightweight messages for real-time status tracking.
|
|
829
|
+
messages:
|
|
830
|
+
- $ref: '#/channels/notificationsOperationsTopic/messages/operationProgress'
|
|
831
|
+
|
|
832
|
+
publishOperationResult:
|
|
833
|
+
action: send
|
|
834
|
+
channel:
|
|
835
|
+
$ref: '#/channels/notificationsOperationsTopic'
|
|
836
|
+
summary: Publish operation result
|
|
837
|
+
description: |
|
|
838
|
+
Lambda publishes the final result when operation completes or fails.
|
|
839
|
+
This is a terminal message with full details.
|
|
840
|
+
messages:
|
|
841
|
+
- $ref: '#/channels/notificationsOperationsTopic/messages/operationResult'
|
|
842
|
+
|
|
843
|
+
consumeOperationNotifications:
|
|
844
|
+
action: receive
|
|
845
|
+
channel:
|
|
846
|
+
$ref: '#/channels/notificationsOperationsQueue'
|
|
847
|
+
summary: Consume operation notifications
|
|
848
|
+
description: |
|
|
849
|
+
API worker consumes both progress and result notifications from SQS.
|
|
850
|
+
- OperationProgress: Update Redis cache for real-time status (forwarded to frontend via SSE)
|
|
851
|
+
- OperationResult: Update database Operation entity, derive Job/Workflow status
|
|
852
|
+
messages:
|
|
853
|
+
- $ref: '#/channels/notificationsOperationsQueue/messages/operationProgress'
|
|
854
|
+
- $ref: '#/channels/notificationsOperationsQueue/messages/operationResult'
|
|
855
|
+
|
|
856
|
+
components:
|
|
857
|
+
messages:
|
|
858
|
+
# ============================================
|
|
859
|
+
# JOB REQUEST MESSAGE (compression branch)
|
|
860
|
+
# ============================================
|
|
861
|
+
|
|
862
|
+
JobRequestMessage:
|
|
863
|
+
name: JobRequest
|
|
864
|
+
title: Compression Job Request
|
|
865
|
+
summary: Request to process a compression operation
|
|
866
|
+
description: |
|
|
867
|
+
Message sent by the API to request processing of a **compression**
|
|
868
|
+
operation. Published to the `job-requests` SNS topic; routed by the
|
|
869
|
+
single-attribute filter `job_type` to one of the four
|
|
870
|
+
per-media-type compression queues.
|
|
871
|
+
|
|
872
|
+
**SNS Message Attributes (routing):**
|
|
873
|
+
- `job_type`: always present. Values: `image`, `video`, `audio`,
|
|
874
|
+
`document`. Used by SNS as the filter attribute on the
|
|
875
|
+
`job-requests` topic.
|
|
876
|
+
|
|
877
|
+
No other message attributes are set on this branch by the API
|
|
878
|
+
publisher. `operation_type` and `media_group` are **not** set —
|
|
879
|
+
the media group is encoded as `job_type`.
|
|
880
|
+
|
|
881
|
+
**Payload:** uses the shared `OperationRequest` schema. For
|
|
882
|
+
compression, `operation_type` inside the payload is always
|
|
883
|
+
`compress`, `file_type` is set to the input MIME, and the
|
|
884
|
+
`source_bucket` / `source_key` fields identify the input.
|
|
885
|
+
contentType: application/json
|
|
886
|
+
headers:
|
|
887
|
+
$ref: '#/components/schemas/JobRequestAttributes'
|
|
888
|
+
payload:
|
|
889
|
+
$ref: '#/components/schemas/OperationRequest'
|
|
890
|
+
examples:
|
|
891
|
+
- name: Image Compression
|
|
892
|
+
summary: Compress a PNG image with lossy mode
|
|
893
|
+
headers:
|
|
894
|
+
job_type: "image"
|
|
895
|
+
payload:
|
|
896
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
897
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
898
|
+
operation_type: "compress"
|
|
899
|
+
file_type: "image/png"
|
|
900
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
901
|
+
source_key: "uploads/018f9c42-aabb/photo.png"
|
|
902
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
903
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e112/018f9c42-5d6b-7481-b3df-9fd0a0a5e200/"
|
|
904
|
+
options:
|
|
905
|
+
mode: "lossy"
|
|
906
|
+
quality: 80
|
|
907
|
+
metadata: "copyright"
|
|
908
|
+
- name: Video Compression
|
|
909
|
+
summary: Compress an MP4 video
|
|
910
|
+
headers:
|
|
911
|
+
job_type: "video"
|
|
912
|
+
payload:
|
|
913
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e113"
|
|
914
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e201"
|
|
915
|
+
operation_type: "compress"
|
|
916
|
+
file_type: "video/mp4"
|
|
917
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
918
|
+
source_key: "uploads/018f9c42-ccdd/video.mp4"
|
|
919
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
920
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e113/018f9c42-5d6b-7481-b3df-9fd0a0a5e201/"
|
|
921
|
+
options:
|
|
922
|
+
codec: "h265"
|
|
923
|
+
crf: 28
|
|
924
|
+
preset: "medium"
|
|
925
|
+
|
|
926
|
+
# ============================================
|
|
927
|
+
# OPERATION REQUEST MESSAGE (non-compression branch)
|
|
928
|
+
# ============================================
|
|
929
|
+
|
|
930
|
+
OperationRequestMessage:
|
|
931
|
+
name: OperationRequest
|
|
932
|
+
title: Non-Compression Operation Request
|
|
933
|
+
summary: Request to process a non-compression operation
|
|
934
|
+
description: |
|
|
935
|
+
Message sent by the API to request processing of any
|
|
936
|
+
**non-compression** operation (thumbnail, watermark, merge,
|
|
937
|
+
archive, convert). Published to the `operations` SNS topic;
|
|
938
|
+
routed by the single-attribute filter `operation_type` to one
|
|
939
|
+
of the nine ops-family queues.
|
|
940
|
+
|
|
941
|
+
**SNS Message Attributes:**
|
|
942
|
+
- `operation_type`: always present. Values: `thumbnail`,
|
|
943
|
+
`thumbnail_image`, `thumbnail_video`, `thumbnail_document`,
|
|
944
|
+
`thumbnail_office`, `watermark`, `merge`, `archive`, `convert`.
|
|
945
|
+
Used by SNS as the filter attribute on the `operations` topic.
|
|
946
|
+
- `media_group`: informational metadata, present for every
|
|
947
|
+
operation except `archive` (which is media-agnostic). Values:
|
|
948
|
+
`image`, `video`, `audio`, `document`. **Not** used by SNS as
|
|
949
|
+
a routing filter — preserved for consumer observability and
|
|
950
|
+
log correlation.
|
|
951
|
+
|
|
952
|
+
**Input models:**
|
|
953
|
+
- Single-input operations (thumbnail, thumbnail_image,
|
|
954
|
+
thumbnail_video, thumbnail_document, thumbnail_office,
|
|
955
|
+
watermark, convert): use `source_bucket` + `source_key`.
|
|
956
|
+
`file_type` required.
|
|
957
|
+
- Multi-input operations (merge, archive): use `sources` array.
|
|
958
|
+
Merge additionally requires `output_type`.
|
|
959
|
+
|
|
960
|
+
**Migration window note.** During the thumbnail sub-type
|
|
961
|
+
migration, both `operation_type=thumbnail` (legacy) and
|
|
962
|
+
`operation_type=thumbnail_{image,video,document,office}` (new)
|
|
963
|
+
are valid routing targets. The API publisher currently emits
|
|
964
|
+
the legacy value only; sub-type adoption is a follow-up API PR.
|
|
965
|
+
contentType: application/json
|
|
966
|
+
headers:
|
|
967
|
+
$ref: '#/components/schemas/OperationRequestAttributes'
|
|
968
|
+
payload:
|
|
969
|
+
$ref: '#/components/schemas/OperationRequest'
|
|
970
|
+
examples:
|
|
971
|
+
- name: Thumbnail (legacy)
|
|
972
|
+
summary: Generate a thumbnail for an image using the legacy thumbnail routing target
|
|
973
|
+
headers:
|
|
974
|
+
operation_type: "thumbnail"
|
|
975
|
+
media_group: "image"
|
|
976
|
+
payload:
|
|
977
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e120"
|
|
978
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e220"
|
|
979
|
+
operation_type: "thumbnail"
|
|
980
|
+
file_type: "image/jpeg"
|
|
981
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
982
|
+
source_key: "uploads/018f9c42-iijj/photo.jpg"
|
|
983
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
984
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e120/018f9c42-5d6b-7481-b3df-9fd0a0a5e220/"
|
|
985
|
+
options:
|
|
986
|
+
width: 320
|
|
987
|
+
height: 240
|
|
988
|
+
fit: "crop"
|
|
989
|
+
format: "jpg"
|
|
990
|
+
- name: Thumbnail Image (sub-type)
|
|
991
|
+
summary: Generate a thumbnail for an image using the new thumbnail_image sub-type routing target
|
|
992
|
+
headers:
|
|
993
|
+
operation_type: "thumbnail_image"
|
|
994
|
+
media_group: "image"
|
|
995
|
+
payload:
|
|
996
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e121"
|
|
997
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e221"
|
|
998
|
+
operation_type: "thumbnail_image"
|
|
999
|
+
file_type: "image/png"
|
|
1000
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
1001
|
+
source_key: "uploads/018f9c42-kkll/screenshot.png"
|
|
1002
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1003
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e121/018f9c42-5d6b-7481-b3df-9fd0a0a5e221/"
|
|
1004
|
+
options:
|
|
1005
|
+
width: 512
|
|
1006
|
+
height: 512
|
|
1007
|
+
fit: "max"
|
|
1008
|
+
format: "webp"
|
|
1009
|
+
- name: Image Watermark (image overlay)
|
|
1010
|
+
summary: Apply a PNG logo overlay to a JPEG image
|
|
1011
|
+
headers:
|
|
1012
|
+
operation_type: "watermark"
|
|
1013
|
+
media_group: "image"
|
|
1014
|
+
payload:
|
|
1015
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e118"
|
|
1016
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e210"
|
|
1017
|
+
operation_type: "watermark"
|
|
1018
|
+
file_type: "image/jpeg"
|
|
1019
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
1020
|
+
source_key: "uploads/018f9c42-eeff/photo.jpg"
|
|
1021
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1022
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e118/018f9c42-5d6b-7481-b3df-9fd0a0a5e210/"
|
|
1023
|
+
options:
|
|
1024
|
+
watermark_type: "image"
|
|
1025
|
+
watermark_bucket: "gisl-stg-euw1-assets"
|
|
1026
|
+
watermark_key: "brand/logo.png"
|
|
1027
|
+
position: "bottom-right"
|
|
1028
|
+
opacity: 0.6
|
|
1029
|
+
- name: Image Watermark (tiled text overlay)
|
|
1030
|
+
summary: Render a rotated tiled text watermark across a PNG image
|
|
1031
|
+
headers:
|
|
1032
|
+
operation_type: "watermark"
|
|
1033
|
+
media_group: "image"
|
|
1034
|
+
payload:
|
|
1035
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e119"
|
|
1036
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e211"
|
|
1037
|
+
operation_type: "watermark"
|
|
1038
|
+
file_type: "image/png"
|
|
1039
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
1040
|
+
source_key: "uploads/018f9c42-gghh/screenshot.png"
|
|
1041
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1042
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e119/018f9c42-5d6b-7481-b3df-9fd0a0a5e211/"
|
|
1043
|
+
options:
|
|
1044
|
+
watermark_type: "text"
|
|
1045
|
+
watermark_mode: "tiled"
|
|
1046
|
+
text: "CONFIDENTIAL"
|
|
1047
|
+
font_size: 64
|
|
1048
|
+
color: "#FF000080"
|
|
1049
|
+
rotation: -45
|
|
1050
|
+
tile_spacing: 120
|
|
1051
|
+
opacity: 0.4
|
|
1052
|
+
- name: Image Convert
|
|
1053
|
+
summary: Convert a PNG image to WebP
|
|
1054
|
+
headers:
|
|
1055
|
+
operation_type: "convert"
|
|
1056
|
+
media_group: "image"
|
|
1057
|
+
payload:
|
|
1058
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e122"
|
|
1059
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e222"
|
|
1060
|
+
operation_type: "convert"
|
|
1061
|
+
file_type: "image/png"
|
|
1062
|
+
source_bucket: "gisl-stg-euw1-input"
|
|
1063
|
+
source_key: "uploads/018f9c42-mmnn/diagram.png"
|
|
1064
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1065
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e122/018f9c42-5d6b-7481-b3df-9fd0a0a5e222/"
|
|
1066
|
+
options:
|
|
1067
|
+
output_format: "webp"
|
|
1068
|
+
quality: 85
|
|
1069
|
+
- name: Video Merge
|
|
1070
|
+
summary: Concatenate multiple video files
|
|
1071
|
+
headers:
|
|
1072
|
+
operation_type: "merge"
|
|
1073
|
+
media_group: "video"
|
|
1074
|
+
payload:
|
|
1075
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e114"
|
|
1076
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e202"
|
|
1077
|
+
operation_type: "merge"
|
|
1078
|
+
file_type: "video/mp4"
|
|
1079
|
+
output_type: "video"
|
|
1080
|
+
sources:
|
|
1081
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1082
|
+
key: "jobs/018f9c42-aaa1/018f9c42-bbb1/intro.mp4"
|
|
1083
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1084
|
+
key: "jobs/018f9c42-aaa2/018f9c42-bbb2/content.mp4"
|
|
1085
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1086
|
+
key: "jobs/018f9c42-aaa3/018f9c42-bbb3/outro.mp4"
|
|
1087
|
+
transition: "none"
|
|
1088
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1089
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e114/018f9c42-5d6b-7481-b3df-9fd0a0a5e202/"
|
|
1090
|
+
options:
|
|
1091
|
+
transition: "crossfade"
|
|
1092
|
+
crossfade_duration: 1.0
|
|
1093
|
+
- name: Image Merge to GIF
|
|
1094
|
+
summary: Merge multiple images into an animated GIF
|
|
1095
|
+
headers:
|
|
1096
|
+
operation_type: "merge"
|
|
1097
|
+
media_group: "image"
|
|
1098
|
+
payload:
|
|
1099
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e115"
|
|
1100
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e203"
|
|
1101
|
+
operation_type: "merge"
|
|
1102
|
+
file_type: "image/png"
|
|
1103
|
+
output_type: "gif"
|
|
1104
|
+
sources:
|
|
1105
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1106
|
+
key: "jobs/018f9c42-aaa1/018f9c42-bbb1/frame1.png"
|
|
1107
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1108
|
+
key: "jobs/018f9c42-aaa2/018f9c42-bbb2/frame2.png"
|
|
1109
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1110
|
+
key: "jobs/018f9c42-aaa3/018f9c42-bbb3/frame3.png"
|
|
1111
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1112
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e115/018f9c42-5d6b-7481-b3df-9fd0a0a5e203/"
|
|
1113
|
+
options:
|
|
1114
|
+
frame_delay: 100
|
|
1115
|
+
loop: true
|
|
1116
|
+
- name: PDF Merge
|
|
1117
|
+
summary: Concatenate multiple PDFs into one
|
|
1118
|
+
headers:
|
|
1119
|
+
operation_type: "merge"
|
|
1120
|
+
media_group: "document"
|
|
1121
|
+
payload:
|
|
1122
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e116"
|
|
1123
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e204"
|
|
1124
|
+
operation_type: "merge"
|
|
1125
|
+
file_type: "application/pdf"
|
|
1126
|
+
output_type: "document"
|
|
1127
|
+
sources:
|
|
1128
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1129
|
+
key: "jobs/018f9c42-aaa1/018f9c42-bbb1/chapter1.pdf"
|
|
1130
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1131
|
+
key: "jobs/018f9c42-aaa2/018f9c42-bbb2/chapter2.pdf"
|
|
1132
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1133
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e116/018f9c42-5d6b-7481-b3df-9fd0a0a5e204/"
|
|
1134
|
+
options: {}
|
|
1135
|
+
- name: Archive
|
|
1136
|
+
summary: Bundle multiple files into a ZIP archive (no media_group attribute — archive is media-agnostic)
|
|
1137
|
+
headers:
|
|
1138
|
+
operation_type: "archive"
|
|
1139
|
+
payload:
|
|
1140
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e117"
|
|
1141
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e205"
|
|
1142
|
+
operation_type: "archive"
|
|
1143
|
+
sources:
|
|
1144
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1145
|
+
key: "jobs/018f9c42-aaa1/018f9c42-bbb1/photo_compressed.png"
|
|
1146
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1147
|
+
key: "jobs/018f9c42-aaa2/018f9c42-bbb2/video_compressed.mp4"
|
|
1148
|
+
- bucket: "gisl-stg-euw1-output"
|
|
1149
|
+
key: "jobs/018f9c42-aaa3/018f9c42-bbb3/report_compressed.pdf"
|
|
1150
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1151
|
+
output_key_prefix: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e117/018f9c42-5d6b-7481-b3df-9fd0a0a5e205/"
|
|
1152
|
+
options:
|
|
1153
|
+
format: "zip"
|
|
1154
|
+
|
|
1155
|
+
# ============================================
|
|
1156
|
+
# OPERATION PROGRESS MESSAGE
|
|
1157
|
+
# ============================================
|
|
1158
|
+
|
|
1159
|
+
OperationProgressMessage:
|
|
1160
|
+
name: OperationProgress
|
|
1161
|
+
title: Operation Progress
|
|
1162
|
+
summary: Progress update during operation processing
|
|
1163
|
+
description: |
|
|
1164
|
+
Lightweight message published by Lambda during operation processing.
|
|
1165
|
+
Used for real-time status tracking via SSE to the frontend.
|
|
1166
|
+
|
|
1167
|
+
**API Handling:**
|
|
1168
|
+
- First progress (started): Update Operation status to in_progress in DB
|
|
1169
|
+
- Subsequent progress: Update Redis cache only (not DB), forward via SSE
|
|
1170
|
+
|
|
1171
|
+
**Frontend Consumption:**
|
|
1172
|
+
- Via SSE endpoint: GET /api/workflows/{id}/events
|
|
1173
|
+
contentType: application/json
|
|
1174
|
+
headers:
|
|
1175
|
+
$ref: '#/components/schemas/FifoMessageHeaders'
|
|
1176
|
+
payload:
|
|
1177
|
+
$ref: '#/components/schemas/OperationProgress'
|
|
1178
|
+
examples:
|
|
1179
|
+
- name: Operation Started
|
|
1180
|
+
summary: Operation has been picked up by Lambda
|
|
1181
|
+
payload:
|
|
1182
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1183
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1184
|
+
operation_type: "compress"
|
|
1185
|
+
status: "started"
|
|
1186
|
+
progress: 0
|
|
1187
|
+
- name: Downloading
|
|
1188
|
+
summary: Lambda is downloading file from S3
|
|
1189
|
+
payload:
|
|
1190
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1191
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1192
|
+
operation_type: "compress"
|
|
1193
|
+
status: "downloading"
|
|
1194
|
+
progress: 10
|
|
1195
|
+
stage: "Fetching file from S3"
|
|
1196
|
+
- name: Processing
|
|
1197
|
+
summary: Lambda is actively processing the operation
|
|
1198
|
+
payload:
|
|
1199
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1200
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1201
|
+
operation_type: "compress"
|
|
1202
|
+
status: "processing"
|
|
1203
|
+
progress: 55
|
|
1204
|
+
stage: "Compressing image"
|
|
1205
|
+
- name: Uploading
|
|
1206
|
+
summary: Lambda is uploading result to S3
|
|
1207
|
+
payload:
|
|
1208
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1209
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1210
|
+
operation_type: "compress"
|
|
1211
|
+
status: "uploading"
|
|
1212
|
+
progress: 90
|
|
1213
|
+
stage: "Saving result to S3"
|
|
1214
|
+
|
|
1215
|
+
# ============================================
|
|
1216
|
+
# OPERATION RESULT MESSAGE
|
|
1217
|
+
# ============================================
|
|
1218
|
+
|
|
1219
|
+
OperationResultMessage:
|
|
1220
|
+
name: OperationResult
|
|
1221
|
+
title: Operation Result
|
|
1222
|
+
summary: Final result when operation completes or fails
|
|
1223
|
+
description: |
|
|
1224
|
+
Terminal message published by Lambda when operation finishes.
|
|
1225
|
+
Contains full details including output location, metrics, or error information.
|
|
1226
|
+
|
|
1227
|
+
**API Handling:**
|
|
1228
|
+
- Update Operation entity status in DB
|
|
1229
|
+
- Derive Job status from its operations (all completed -> job completed)
|
|
1230
|
+
- Derive Workflow status from its jobs
|
|
1231
|
+
- Clear Redis cache for this operation
|
|
1232
|
+
- Forward event via SSE to frontend
|
|
1233
|
+
|
|
1234
|
+
**Idempotency:**
|
|
1235
|
+
- Check if Operation already has a result before updating
|
|
1236
|
+
contentType: application/json
|
|
1237
|
+
headers:
|
|
1238
|
+
$ref: '#/components/schemas/FifoMessageHeaders'
|
|
1239
|
+
payload:
|
|
1240
|
+
$ref: '#/components/schemas/OperationResult'
|
|
1241
|
+
examples:
|
|
1242
|
+
- name: Successful Compression
|
|
1243
|
+
summary: Compress operation completed with metrics
|
|
1244
|
+
payload:
|
|
1245
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1246
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1247
|
+
operation_type: "compress"
|
|
1248
|
+
status: "completed"
|
|
1249
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1250
|
+
output_key: "jobs/018f9c42-5d6b/018f9c42-5d6b-e200/photo_compressed.png"
|
|
1251
|
+
output_size_bytes: 2097152
|
|
1252
|
+
metrics:
|
|
1253
|
+
original_size_bytes: 5242880
|
|
1254
|
+
output_size_bytes: 2097152
|
|
1255
|
+
compression_ratio: 0.40
|
|
1256
|
+
duration_ms: 1500
|
|
1257
|
+
- name: Successful Merge
|
|
1258
|
+
summary: Video merge completed
|
|
1259
|
+
payload:
|
|
1260
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e114"
|
|
1261
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e202"
|
|
1262
|
+
operation_type: "merge"
|
|
1263
|
+
status: "completed"
|
|
1264
|
+
output_bucket: "gisl-stg-euw1-output"
|
|
1265
|
+
output_key: "jobs/018f9c42-5d6b/018f9c42-5d6b-e202/merged.mp4"
|
|
1266
|
+
output_size_bytes: 157286400
|
|
1267
|
+
metrics:
|
|
1268
|
+
input_count: 3
|
|
1269
|
+
output_size_bytes: 157286400
|
|
1270
|
+
duration_ms: 8500
|
|
1271
|
+
- name: Failed - Invalid Format
|
|
1272
|
+
summary: Operation failed due to unsupported file format
|
|
1273
|
+
payload:
|
|
1274
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e113"
|
|
1275
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e201"
|
|
1276
|
+
operation_type: "compress"
|
|
1277
|
+
status: "failed"
|
|
1278
|
+
error_code: "invalid_format"
|
|
1279
|
+
error_message: "Unsupported file format: image/bmp"
|
|
1280
|
+
is_retryable: false
|
|
1281
|
+
last_progress: 10
|
|
1282
|
+
- name: Failed - Retryable
|
|
1283
|
+
summary: Operation failed with retryable S3 error
|
|
1284
|
+
payload:
|
|
1285
|
+
job_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1286
|
+
operation_id: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1287
|
+
operation_type: "compress"
|
|
1288
|
+
status: "failed"
|
|
1289
|
+
error_code: "s3_download_failed"
|
|
1290
|
+
error_message: "Failed to download source file from S3"
|
|
1291
|
+
is_retryable: true
|
|
1292
|
+
last_progress: 0
|
|
1293
|
+
|
|
1294
|
+
schemas:
|
|
1295
|
+
# ============================================
|
|
1296
|
+
# FIFO MESSAGE HEADERS
|
|
1297
|
+
# ============================================
|
|
1298
|
+
|
|
1299
|
+
FifoMessageHeaders:
|
|
1300
|
+
type: object
|
|
1301
|
+
description: |
|
|
1302
|
+
Required headers for FIFO SNS/SQS messaging.
|
|
1303
|
+
These ensure message ordering and deduplication.
|
|
1304
|
+
required:
|
|
1305
|
+
- MessageGroupId
|
|
1306
|
+
properties:
|
|
1307
|
+
MessageGroupId:
|
|
1308
|
+
type: string
|
|
1309
|
+
description: |
|
|
1310
|
+
Groups messages for FIFO ordering. Messages with the same
|
|
1311
|
+
MessageGroupId are processed in order. Use operation_id to ensure
|
|
1312
|
+
all messages for an operation are ordered correctly.
|
|
1313
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1314
|
+
|
|
1315
|
+
# ============================================
|
|
1316
|
+
# SNS MESSAGE ATTRIBUTES - COMPRESSION BRANCH
|
|
1317
|
+
# ============================================
|
|
1318
|
+
|
|
1319
|
+
JobRequestAttributes:
|
|
1320
|
+
type: object
|
|
1321
|
+
description: |
|
|
1322
|
+
SNS message attributes for the compression-branch publish path
|
|
1323
|
+
(`publishJobRequest`). These are set as SNS message attributes on
|
|
1324
|
+
messages sent to the `job-requests` topic, not in the payload.
|
|
1325
|
+
|
|
1326
|
+
Only `job_type` is set on this branch. The Option A publisher in
|
|
1327
|
+
`compression_api` does **not** set `operation_type` or
|
|
1328
|
+
`media_group` as attributes on compression messages — the media
|
|
1329
|
+
group is encoded as `job_type` and no other attribute is used as
|
|
1330
|
+
a filter on this topic. `job_type` is derived from the API's
|
|
1331
|
+
`OperationMediaGroupResolver` output at publish time.
|
|
1332
|
+
|
|
1333
|
+
SNS uses `job_type` as the filter attribute on the `job-requests`
|
|
1334
|
+
topic; subscriptions route by `job_type` value to the four
|
|
1335
|
+
per-media-type compression queues.
|
|
1336
|
+
required:
|
|
1337
|
+
- job_type
|
|
1338
|
+
properties:
|
|
1339
|
+
job_type:
|
|
1340
|
+
type: string
|
|
1341
|
+
description: |
|
|
1342
|
+
Media type category used as the SNS filter attribute on the
|
|
1343
|
+
`job-requests` topic. The sole routing key for compression
|
|
1344
|
+
traffic.
|
|
1345
|
+
enum:
|
|
1346
|
+
- image
|
|
1347
|
+
- video
|
|
1348
|
+
- audio
|
|
1349
|
+
- document
|
|
1350
|
+
|
|
1351
|
+
# ============================================
|
|
1352
|
+
# SNS MESSAGE ATTRIBUTES - NON-COMPRESSION BRANCH
|
|
1353
|
+
# ============================================
|
|
1354
|
+
|
|
1355
|
+
OperationRequestAttributes:
|
|
1356
|
+
type: object
|
|
1357
|
+
description: |
|
|
1358
|
+
SNS message attributes for the non-compression-branch publish path
|
|
1359
|
+
(`publishOperationRequest`). These are set as SNS message
|
|
1360
|
+
attributes on messages sent to the `operations` topic, not in the
|
|
1361
|
+
payload.
|
|
1362
|
+
|
|
1363
|
+
SNS uses `operation_type` as the filter attribute on the
|
|
1364
|
+
`operations` topic; subscriptions route by `operation_type` value
|
|
1365
|
+
to per-operation queues. `media_group` is a second attribute the
|
|
1366
|
+
Option A publisher always sets on non-archive operations, but it
|
|
1367
|
+
is **not** used by SNS as a routing filter — it is informational
|
|
1368
|
+
metadata preserved for consumer observability and log correlation.
|
|
1369
|
+
|
|
1370
|
+
**Required-attribute contract** (encoded in the `allOf`
|
|
1371
|
+
conditionals below, matching the runtime guarantee in
|
|
1372
|
+
`compression_api`'s `AwsSnsOperationPublisherAdapter.php:63-73`):
|
|
1373
|
+
|
|
1374
|
+
- `operation_type` is always present.
|
|
1375
|
+
- `media_group` is required iff `operation_type != archive`.
|
|
1376
|
+
Archive is explicitly omitted because it is media-agnostic.
|
|
1377
|
+
- For `operation_type == watermark`, `media_group` must equal
|
|
1378
|
+
`image` (watermark is image-only).
|
|
1379
|
+
- For the four thumbnail sub-types, `media_group` is constrained
|
|
1380
|
+
to the matching media:
|
|
1381
|
+
- `thumbnail_image` -> `image`
|
|
1382
|
+
- `thumbnail_video` -> `video`
|
|
1383
|
+
- `thumbnail_document` -> `document`
|
|
1384
|
+
- `thumbnail_office` -> `document` (office files are routed as
|
|
1385
|
+
documents at the media_group level)
|
|
1386
|
+
required:
|
|
1387
|
+
- operation_type
|
|
1388
|
+
properties:
|
|
1389
|
+
operation_type:
|
|
1390
|
+
type: string
|
|
1391
|
+
description: |
|
|
1392
|
+
Operation type used as the SNS filter attribute on the
|
|
1393
|
+
`operations` topic. The sole routing key for non-compression
|
|
1394
|
+
traffic.
|
|
1395
|
+
enum:
|
|
1396
|
+
- thumbnail
|
|
1397
|
+
- thumbnail_image
|
|
1398
|
+
- thumbnail_video
|
|
1399
|
+
- thumbnail_document
|
|
1400
|
+
- thumbnail_office
|
|
1401
|
+
- watermark
|
|
1402
|
+
- merge
|
|
1403
|
+
- archive
|
|
1404
|
+
- convert
|
|
1405
|
+
media_group:
|
|
1406
|
+
type: string
|
|
1407
|
+
description: |
|
|
1408
|
+
Media category. **Informational metadata, not an SNS filter
|
|
1409
|
+
attribute.** Set by the Option A publisher on every
|
|
1410
|
+
non-compress, non-archive operation for consumer observability
|
|
1411
|
+
and log correlation. Omitted for archive (media-agnostic).
|
|
1412
|
+
|
|
1413
|
+
For single-input operations, derived from `file_type` MIME
|
|
1414
|
+
prefix. For merge, derived from `output_type`.
|
|
1415
|
+
enum:
|
|
1416
|
+
- image
|
|
1417
|
+
- video
|
|
1418
|
+
- audio
|
|
1419
|
+
- document
|
|
1420
|
+
allOf:
|
|
1421
|
+
- if:
|
|
1422
|
+
properties:
|
|
1423
|
+
operation_type:
|
|
1424
|
+
not:
|
|
1425
|
+
const: archive
|
|
1426
|
+
then:
|
|
1427
|
+
required:
|
|
1428
|
+
- media_group
|
|
1429
|
+
- if:
|
|
1430
|
+
properties:
|
|
1431
|
+
operation_type:
|
|
1432
|
+
const: watermark
|
|
1433
|
+
then:
|
|
1434
|
+
properties:
|
|
1435
|
+
media_group:
|
|
1436
|
+
const: image
|
|
1437
|
+
- if:
|
|
1438
|
+
properties:
|
|
1439
|
+
operation_type:
|
|
1440
|
+
const: thumbnail_image
|
|
1441
|
+
then:
|
|
1442
|
+
properties:
|
|
1443
|
+
media_group:
|
|
1444
|
+
const: image
|
|
1445
|
+
- if:
|
|
1446
|
+
properties:
|
|
1447
|
+
operation_type:
|
|
1448
|
+
const: thumbnail_video
|
|
1449
|
+
then:
|
|
1450
|
+
properties:
|
|
1451
|
+
media_group:
|
|
1452
|
+
const: video
|
|
1453
|
+
- if:
|
|
1454
|
+
properties:
|
|
1455
|
+
operation_type:
|
|
1456
|
+
const: thumbnail_document
|
|
1457
|
+
then:
|
|
1458
|
+
properties:
|
|
1459
|
+
media_group:
|
|
1460
|
+
const: document
|
|
1461
|
+
- if:
|
|
1462
|
+
properties:
|
|
1463
|
+
operation_type:
|
|
1464
|
+
const: thumbnail_office
|
|
1465
|
+
then:
|
|
1466
|
+
properties:
|
|
1467
|
+
media_group:
|
|
1468
|
+
const: document
|
|
1469
|
+
|
|
1470
|
+
# ============================================
|
|
1471
|
+
# OPERATION REQUEST PAYLOAD SCHEMA (shared by both messages)
|
|
1472
|
+
# ============================================
|
|
1473
|
+
|
|
1474
|
+
OperationRequest:
|
|
1475
|
+
type: object
|
|
1476
|
+
description: |
|
|
1477
|
+
Message format for operation request payloads. Shared between
|
|
1478
|
+
`JobRequestMessage` (compression branch) and
|
|
1479
|
+
`OperationRequestMessage` (non-compression branch). Published by
|
|
1480
|
+
API to SNS, consumed by Lambda from SQS.
|
|
1481
|
+
|
|
1482
|
+
**Single-input operations** (compress, thumbnail, thumbnail_image,
|
|
1483
|
+
thumbnail_video, thumbnail_document, thumbnail_office, watermark,
|
|
1484
|
+
convert): use `source_bucket` + `source_key` for the input file.
|
|
1485
|
+
`file_type` is required.
|
|
1486
|
+
|
|
1487
|
+
**Multi-input operations** (merge, archive): use `sources` array.
|
|
1488
|
+
Each entry has `bucket` + `key`. Merge entries may include
|
|
1489
|
+
per-input overrides (e.g. transition type). Merge also requires
|
|
1490
|
+
`output_type`.
|
|
1491
|
+
|
|
1492
|
+
Note: `media_group` is **not** a field in this payload. It exists
|
|
1493
|
+
only as an SNS MessageAttribute on the non-compression branch; on
|
|
1494
|
+
the compression branch, the routing equivalent is the `job_type`
|
|
1495
|
+
MessageAttribute, also not in the payload.
|
|
1496
|
+
required:
|
|
1497
|
+
- job_id
|
|
1498
|
+
- operation_id
|
|
1499
|
+
- operation_type
|
|
1500
|
+
- output_bucket
|
|
1501
|
+
- output_key_prefix
|
|
1502
|
+
allOf:
|
|
1503
|
+
- if:
|
|
1504
|
+
properties:
|
|
1505
|
+
operation_type:
|
|
1506
|
+
enum:
|
|
1507
|
+
- compress
|
|
1508
|
+
- thumbnail
|
|
1509
|
+
- thumbnail_image
|
|
1510
|
+
- thumbnail_video
|
|
1511
|
+
- thumbnail_document
|
|
1512
|
+
- thumbnail_office
|
|
1513
|
+
- watermark
|
|
1514
|
+
- convert
|
|
1515
|
+
then:
|
|
1516
|
+
required:
|
|
1517
|
+
- file_type
|
|
1518
|
+
- source_bucket
|
|
1519
|
+
- source_key
|
|
1520
|
+
- if:
|
|
1521
|
+
properties:
|
|
1522
|
+
operation_type:
|
|
1523
|
+
enum: [merge, archive]
|
|
1524
|
+
then:
|
|
1525
|
+
required:
|
|
1526
|
+
- sources
|
|
1527
|
+
- if:
|
|
1528
|
+
properties:
|
|
1529
|
+
operation_type:
|
|
1530
|
+
const: merge
|
|
1531
|
+
then:
|
|
1532
|
+
required:
|
|
1533
|
+
- output_type
|
|
1534
|
+
properties:
|
|
1535
|
+
job_id:
|
|
1536
|
+
type: string
|
|
1537
|
+
format: uuid
|
|
1538
|
+
description: Parent job identifier (UUID v7)
|
|
1539
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1540
|
+
operation_id:
|
|
1541
|
+
type: string
|
|
1542
|
+
format: uuid
|
|
1543
|
+
description: Unique operation identifier (UUID v7)
|
|
1544
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1545
|
+
operation_type:
|
|
1546
|
+
$ref: '#/components/schemas/OperationType'
|
|
1547
|
+
workflow_id:
|
|
1548
|
+
type: string
|
|
1549
|
+
format: uuid
|
|
1550
|
+
description: Parent workflow identifier (UUID v7). Present when operation belongs to a workflow.
|
|
1551
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e100"
|
|
1552
|
+
file_type:
|
|
1553
|
+
type: string
|
|
1554
|
+
description: |
|
|
1555
|
+
MIME type of the input file. Required for single-input operations.
|
|
1556
|
+
For multi-input operations, represents the primary/expected input type.
|
|
1557
|
+
|
|
1558
|
+
Examples by media group:
|
|
1559
|
+
- image: image/png, image/jpeg, image/webp, image/avif, image/gif, image/svg+xml, image/tiff
|
|
1560
|
+
- video: video/mp4, video/webm
|
|
1561
|
+
- audio: audio/mpeg, audio/ogg, audio/wav, audio/flac, audio/aac
|
|
1562
|
+
- document:
|
|
1563
|
+
- application/pdf
|
|
1564
|
+
- application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX)
|
|
1565
|
+
- application/vnd.openxmlformats-officedocument.spreadsheetml.sheet (XLSX)
|
|
1566
|
+
- application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX)
|
|
1567
|
+
- application/vnd.oasis.opendocument.text (ODT)
|
|
1568
|
+
- application/vnd.oasis.opendocument.spreadsheet (ODS)
|
|
1569
|
+
- application/vnd.oasis.opendocument.presentation (ODP)
|
|
1570
|
+
- application/epub+zip (EPUB)
|
|
1571
|
+
example: "image/png"
|
|
1572
|
+
|
|
1573
|
+
# Single-input fields
|
|
1574
|
+
source_bucket:
|
|
1575
|
+
type: string
|
|
1576
|
+
description: S3 bucket containing the source file (single-input operations)
|
|
1577
|
+
example: "gisl-stg-euw1-input"
|
|
1578
|
+
source_key:
|
|
1579
|
+
type: string
|
|
1580
|
+
description: S3 key of the source file (single-input operations)
|
|
1581
|
+
example: "uploads/018f9c42-aabb/photo.png"
|
|
1582
|
+
|
|
1583
|
+
# Multi-input fields
|
|
1584
|
+
sources:
|
|
1585
|
+
type: array
|
|
1586
|
+
description: |
|
|
1587
|
+
Input files for multi-input operations (merge, archive).
|
|
1588
|
+
Each entry specifies an S3 location. Order matters for merge (concatenation order).
|
|
1589
|
+
Per-input overrides (e.g. transition) can be inlined on each entry.
|
|
1590
|
+
items:
|
|
1591
|
+
$ref: '#/components/schemas/SourceEntry'
|
|
1592
|
+
minItems: 2
|
|
1593
|
+
|
|
1594
|
+
# Output
|
|
1595
|
+
output_bucket:
|
|
1596
|
+
type: string
|
|
1597
|
+
description: S3 bucket for the operation output
|
|
1598
|
+
example: "gisl-stg-euw1-output"
|
|
1599
|
+
output_key_prefix:
|
|
1600
|
+
type: string
|
|
1601
|
+
description: |
|
|
1602
|
+
S3 key prefix for operation output. Lambda writes output files under this prefix.
|
|
1603
|
+
Convention: jobs/{job_id}/{operation_id}/
|
|
1604
|
+
example: "jobs/018f9c42-5d6b-7481-b3df-9fd0a0a5e112/018f9c42-5d6b-7481-b3df-9fd0a0a5e200/"
|
|
1605
|
+
|
|
1606
|
+
# Merge-specific
|
|
1607
|
+
output_type:
|
|
1608
|
+
$ref: '#/components/schemas/MergeOutputType'
|
|
1609
|
+
|
|
1610
|
+
# Operation options
|
|
1611
|
+
options:
|
|
1612
|
+
type: object
|
|
1613
|
+
description: |
|
|
1614
|
+
Operation-specific options. Schema varies by operation_type and media_group.
|
|
1615
|
+
Validated by the API before publishing. Lambda trusts these are valid.
|
|
1616
|
+
See the OpenAPI spec for per-operation-type option schemas.
|
|
1617
|
+
additionalProperties: true
|
|
1618
|
+
|
|
1619
|
+
# V2 placeholders
|
|
1620
|
+
source_credentials:
|
|
1621
|
+
type: object
|
|
1622
|
+
description: |
|
|
1623
|
+
Credentials for accessing source files in external storage (V2).
|
|
1624
|
+
Not used in V1 — all sources are in GISL-managed S3 buckets.
|
|
1625
|
+
additionalProperties: true
|
|
1626
|
+
export:
|
|
1627
|
+
type: object
|
|
1628
|
+
description: |
|
|
1629
|
+
Export configuration for writing output to external storage (V2).
|
|
1630
|
+
Not used in V1 — all outputs go to GISL-managed S3 buckets.
|
|
1631
|
+
additionalProperties: true
|
|
1632
|
+
|
|
1633
|
+
SourceEntry:
|
|
1634
|
+
type: object
|
|
1635
|
+
description: |
|
|
1636
|
+
A single source file for multi-input operations (merge, archive).
|
|
1637
|
+
Provides the S3 location and optional per-input overrides.
|
|
1638
|
+
required:
|
|
1639
|
+
- bucket
|
|
1640
|
+
- key
|
|
1641
|
+
properties:
|
|
1642
|
+
bucket:
|
|
1643
|
+
type: string
|
|
1644
|
+
description: S3 bucket containing the source file
|
|
1645
|
+
example: "gisl-stg-euw1-output"
|
|
1646
|
+
key:
|
|
1647
|
+
type: string
|
|
1648
|
+
description: S3 key of the source file
|
|
1649
|
+
example: "jobs/018f9c42-aaa1/018f9c42-bbb1/intro.mp4"
|
|
1650
|
+
transition:
|
|
1651
|
+
type: string
|
|
1652
|
+
description: |
|
|
1653
|
+
Per-input transition override for video merge.
|
|
1654
|
+
Overrides the global transition option for the join point before this input.
|
|
1655
|
+
example: "none"
|
|
1656
|
+
page_range:
|
|
1657
|
+
type: string
|
|
1658
|
+
description: |
|
|
1659
|
+
Page range for PDF merge. Selects specific pages from this input.
|
|
1660
|
+
Format: "1-5", "1,3,5-10". Omit for all pages.
|
|
1661
|
+
example: "1-5"
|
|
1662
|
+
|
|
1663
|
+
# ============================================
|
|
1664
|
+
# OPERATION PROGRESS SCHEMA
|
|
1665
|
+
# ============================================
|
|
1666
|
+
|
|
1667
|
+
OperationProgress:
|
|
1668
|
+
type: object
|
|
1669
|
+
description: |
|
|
1670
|
+
Lightweight progress update message.
|
|
1671
|
+
Published frequently during operation processing.
|
|
1672
|
+
Stored in Redis, not database (except first "started" status).
|
|
1673
|
+
|
|
1674
|
+
**Important**: Progress reaches 100 only on successful completion.
|
|
1675
|
+
On failure, progress stops at the last known value.
|
|
1676
|
+
required:
|
|
1677
|
+
- job_id
|
|
1678
|
+
- operation_id
|
|
1679
|
+
- operation_type
|
|
1680
|
+
- status
|
|
1681
|
+
- progress
|
|
1682
|
+
properties:
|
|
1683
|
+
job_id:
|
|
1684
|
+
type: string
|
|
1685
|
+
format: uuid
|
|
1686
|
+
description: Parent job identifier for correlation
|
|
1687
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1688
|
+
operation_id:
|
|
1689
|
+
type: string
|
|
1690
|
+
format: uuid
|
|
1691
|
+
description: Operation identifier matching the original request
|
|
1692
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1693
|
+
operation_type:
|
|
1694
|
+
$ref: '#/components/schemas/OperationType'
|
|
1695
|
+
status:
|
|
1696
|
+
$ref: '#/components/schemas/ProgressStatus'
|
|
1697
|
+
progress:
|
|
1698
|
+
type: integer
|
|
1699
|
+
minimum: 0
|
|
1700
|
+
maximum: 100
|
|
1701
|
+
description: Progress percentage (0-100)
|
|
1702
|
+
example: 55
|
|
1703
|
+
stage:
|
|
1704
|
+
type: string
|
|
1705
|
+
description: Human-readable description of current processing stage
|
|
1706
|
+
example: "Compressing image"
|
|
1707
|
+
|
|
1708
|
+
# ============================================
|
|
1709
|
+
# OPERATION RESULT SCHEMA
|
|
1710
|
+
# ============================================
|
|
1711
|
+
|
|
1712
|
+
OperationResult:
|
|
1713
|
+
type: object
|
|
1714
|
+
description: |
|
|
1715
|
+
Terminal message with full operation results.
|
|
1716
|
+
Stored in database as part of the Operation entity.
|
|
1717
|
+
|
|
1718
|
+
**Important**: This message MUST always be sent, whether the operation
|
|
1719
|
+
succeeds or fails. It is the definitive signal that processing is complete.
|
|
1720
|
+
required:
|
|
1721
|
+
- job_id
|
|
1722
|
+
- operation_id
|
|
1723
|
+
- operation_type
|
|
1724
|
+
- status
|
|
1725
|
+
if:
|
|
1726
|
+
properties:
|
|
1727
|
+
status:
|
|
1728
|
+
const: completed
|
|
1729
|
+
then:
|
|
1730
|
+
required:
|
|
1731
|
+
- output_bucket
|
|
1732
|
+
- output_key
|
|
1733
|
+
- output_size_bytes
|
|
1734
|
+
else:
|
|
1735
|
+
required:
|
|
1736
|
+
- error_code
|
|
1737
|
+
- error_message
|
|
1738
|
+
- is_retryable
|
|
1739
|
+
properties:
|
|
1740
|
+
job_id:
|
|
1741
|
+
type: string
|
|
1742
|
+
format: uuid
|
|
1743
|
+
description: Parent job identifier for correlation
|
|
1744
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e112"
|
|
1745
|
+
operation_id:
|
|
1746
|
+
type: string
|
|
1747
|
+
format: uuid
|
|
1748
|
+
description: Operation identifier matching the original request
|
|
1749
|
+
example: "018f9c42-5d6b-7481-b3df-9fd0a0a5e200"
|
|
1750
|
+
operation_type:
|
|
1751
|
+
$ref: '#/components/schemas/OperationType'
|
|
1752
|
+
status:
|
|
1753
|
+
$ref: '#/components/schemas/ResultStatus'
|
|
1754
|
+
|
|
1755
|
+
# Success fields
|
|
1756
|
+
output_bucket:
|
|
1757
|
+
type: string
|
|
1758
|
+
description: S3 bucket where output file is stored (completed only)
|
|
1759
|
+
example: "gisl-stg-euw1-output"
|
|
1760
|
+
output_key:
|
|
1761
|
+
type: string
|
|
1762
|
+
description: S3 key of the output file (completed only)
|
|
1763
|
+
example: "jobs/018f9c42-5d6b/018f9c42-5d6b-e200/photo_compressed.png"
|
|
1764
|
+
output_size_bytes:
|
|
1765
|
+
type: integer
|
|
1766
|
+
format: int64
|
|
1767
|
+
description: Output file size in bytes (completed only)
|
|
1768
|
+
example: 2097152
|
|
1769
|
+
metrics:
|
|
1770
|
+
$ref: '#/components/schemas/OperationMetrics'
|
|
1771
|
+
|
|
1772
|
+
# Failure fields
|
|
1773
|
+
error_code:
|
|
1774
|
+
$ref: '#/components/schemas/ErrorCode'
|
|
1775
|
+
error_message:
|
|
1776
|
+
type: string
|
|
1777
|
+
description: Human-readable error message (failed only)
|
|
1778
|
+
example: "Unsupported file format: image/bmp"
|
|
1779
|
+
is_retryable:
|
|
1780
|
+
type: boolean
|
|
1781
|
+
description: Whether the operation could be retried (failed only)
|
|
1782
|
+
default: false
|
|
1783
|
+
last_progress:
|
|
1784
|
+
type: integer
|
|
1785
|
+
minimum: 0
|
|
1786
|
+
maximum: 100
|
|
1787
|
+
description: Progress percentage when the operation failed (failed only)
|
|
1788
|
+
example: 10
|
|
1789
|
+
|
|
1790
|
+
# ============================================
|
|
1791
|
+
# METRICS
|
|
1792
|
+
# ============================================
|
|
1793
|
+
|
|
1794
|
+
OperationMetrics:
|
|
1795
|
+
type: object
|
|
1796
|
+
description: |
|
|
1797
|
+
Operation-specific metrics. Content varies by operation type.
|
|
1798
|
+
All metrics include duration_ms. Compression includes size/ratio.
|
|
1799
|
+
Merge includes input count. Archive includes file count and total size.
|
|
1800
|
+
properties:
|
|
1801
|
+
original_size_bytes:
|
|
1802
|
+
type: integer
|
|
1803
|
+
format: int64
|
|
1804
|
+
description: Original file size in bytes (compress, convert, thumbnail, watermark)
|
|
1805
|
+
example: 5242880
|
|
1806
|
+
output_size_bytes:
|
|
1807
|
+
type: integer
|
|
1808
|
+
format: int64
|
|
1809
|
+
description: Output file size in bytes
|
|
1810
|
+
example: 2097152
|
|
1811
|
+
compression_ratio:
|
|
1812
|
+
type: number
|
|
1813
|
+
format: float
|
|
1814
|
+
minimum: 0
|
|
1815
|
+
maximum: 1
|
|
1816
|
+
description: |
|
|
1817
|
+
Ratio of output to original size (0.0 to 1.0, lower = more compression).
|
|
1818
|
+
Only present for compress operations.
|
|
1819
|
+
example: 0.40
|
|
1820
|
+
duration_ms:
|
|
1821
|
+
type: integer
|
|
1822
|
+
format: int64
|
|
1823
|
+
description: Total processing time in milliseconds
|
|
1824
|
+
example: 1500
|
|
1825
|
+
input_count:
|
|
1826
|
+
type: integer
|
|
1827
|
+
description: Number of input files (merge, archive)
|
|
1828
|
+
example: 3
|
|
1829
|
+
file_count:
|
|
1830
|
+
type: integer
|
|
1831
|
+
description: Number of files in archive (archive only)
|
|
1832
|
+
example: 30
|
|
1833
|
+
total_input_size_bytes:
|
|
1834
|
+
type: integer
|
|
1835
|
+
format: int64
|
|
1836
|
+
description: Sum of all input file sizes (merge, archive)
|
|
1837
|
+
example: 52428800
|
|
1838
|
+
|
|
1839
|
+
# ============================================
|
|
1840
|
+
# ENUMS
|
|
1841
|
+
# ============================================
|
|
1842
|
+
|
|
1843
|
+
OperationType:
|
|
1844
|
+
type: string
|
|
1845
|
+
description: |
|
|
1846
|
+
Operation type. Appears both inside the message payload and, for
|
|
1847
|
+
non-compression operations, as the SNS `operation_type` filter
|
|
1848
|
+
attribute on the `operations` topic. Compression operations are
|
|
1849
|
+
routed by `job_type` on the separate `job-requests` topic — see
|
|
1850
|
+
`JobRequestAttributes` and the top-level `info.description`.
|
|
1851
|
+
|
|
1852
|
+
- `compress`: Reduce file size (image, video, audio, document).
|
|
1853
|
+
Routes via `job_type` on the `job-requests` topic.
|
|
1854
|
+
- `thumbnail`: Legacy thumbnail value. Routes via
|
|
1855
|
+
`operation_type=thumbnail` on the `operations` topic to the
|
|
1856
|
+
legacy `ops-thumbnail` queue. Currently the only thumbnail
|
|
1857
|
+
value the API publisher emits.
|
|
1858
|
+
- `thumbnail_image`: Image thumbnail sub-type. Routes via
|
|
1859
|
+
`operation_type=thumbnail_image` to `ops-thumbnail-image`.
|
|
1860
|
+
Not yet emitted by the API publisher — adoption is a
|
|
1861
|
+
follow-up API PR with input-MIME dispatch.
|
|
1862
|
+
- `thumbnail_video`: Video thumbnail sub-type. Routes via
|
|
1863
|
+
`operation_type=thumbnail_video` to `ops-thumbnail-video`.
|
|
1864
|
+
Not yet emitted.
|
|
1865
|
+
- `thumbnail_document`: PDF/EPUB thumbnail sub-type. Routes via
|
|
1866
|
+
`operation_type=thumbnail_document` to `ops-thumbnail-document`.
|
|
1867
|
+
Not yet emitted.
|
|
1868
|
+
- `thumbnail_office`: Office document (DOCX/XLSX/PPTX/ODT/ODS/ODP)
|
|
1869
|
+
thumbnail sub-type. Routes via `operation_type=thumbnail_office`
|
|
1870
|
+
to `ops-thumbnail-office`. Not yet emitted.
|
|
1871
|
+
- `watermark`: Apply branding/protection (image only). Routes
|
|
1872
|
+
via `operation_type=watermark` to `ops-watermark`.
|
|
1873
|
+
- `convert`: Change file format (image, video, audio, document).
|
|
1874
|
+
Routes via `operation_type=convert` to `ops-convert`.
|
|
1875
|
+
- `merge`: Concatenate/combine multiple files (image, video,
|
|
1876
|
+
audio, document/PDF). Routes via `operation_type=merge` to
|
|
1877
|
+
`ops-merge`. A single Lambda handles all merge output types.
|
|
1878
|
+
- `archive`: Bundle files into ZIP/tar.gz (media-agnostic).
|
|
1879
|
+
Routes via `operation_type=archive` to `ops-archive`. No
|
|
1880
|
+
`media_group` attribute is set for archive.
|
|
1881
|
+
enum:
|
|
1882
|
+
- compress
|
|
1883
|
+
- thumbnail
|
|
1884
|
+
- thumbnail_image
|
|
1885
|
+
- thumbnail_video
|
|
1886
|
+
- thumbnail_document
|
|
1887
|
+
- thumbnail_office
|
|
1888
|
+
- watermark
|
|
1889
|
+
- convert
|
|
1890
|
+
- merge
|
|
1891
|
+
- archive
|
|
1892
|
+
|
|
1893
|
+
MediaGroup:
|
|
1894
|
+
type: string
|
|
1895
|
+
description: |
|
|
1896
|
+
Media category. Used as an **informational** SNS message attribute
|
|
1897
|
+
on the `operations` topic only — preserved for consumer
|
|
1898
|
+
observability and log correlation, and **not** used by SNS as a
|
|
1899
|
+
routing filter. Routing on the `operations` topic is by
|
|
1900
|
+
`operation_type` alone; routing on the `job-requests` topic is by
|
|
1901
|
+
`job_type` (see `JobRequestAttributes`).
|
|
1902
|
+
|
|
1903
|
+
Derived by the publisher from MIME type prefix for single-input
|
|
1904
|
+
operations:
|
|
1905
|
+
- image/* -> image
|
|
1906
|
+
- video/* -> video
|
|
1907
|
+
- audio/* -> audio
|
|
1908
|
+
- document types -> document (PDF, EPUB, DOCX, XLSX, PPTX, ODT,
|
|
1909
|
+
ODS, ODP)
|
|
1910
|
+
|
|
1911
|
+
For merge, derived from `output_type`:
|
|
1912
|
+
- output_type=image or gif -> image
|
|
1913
|
+
- output_type=video -> video
|
|
1914
|
+
- output_type=audio -> audio
|
|
1915
|
+
- output_type=document -> document
|
|
1916
|
+
|
|
1917
|
+
Omitted from the SNS attributes for archive (media-agnostic). The
|
|
1918
|
+
four thumbnail sub-types map to media_group as follows:
|
|
1919
|
+
- thumbnail_image -> image
|
|
1920
|
+
- thumbnail_video -> video
|
|
1921
|
+
- thumbnail_document -> document
|
|
1922
|
+
- thumbnail_office -> document (office files folded under
|
|
1923
|
+
`document` at the media_group level)
|
|
1924
|
+
enum:
|
|
1925
|
+
- image
|
|
1926
|
+
- video
|
|
1927
|
+
- audio
|
|
1928
|
+
- document
|
|
1929
|
+
|
|
1930
|
+
MergeOutputType:
|
|
1931
|
+
type: string
|
|
1932
|
+
description: |
|
|
1933
|
+
Output format for merge operations. All merge operations route to
|
|
1934
|
+
the single `ops-merge` Lambda via `operation_type=merge`
|
|
1935
|
+
regardless of the `output_type` value — `output_type` is read by
|
|
1936
|
+
the merge Lambda to decide the internal encoding path and output
|
|
1937
|
+
format, not to influence SNS routing.
|
|
1938
|
+
- `image`: Collage/grid (ImageMagick montage inside the merge Lambda)
|
|
1939
|
+
- `gif`: Animated GIF (ImageMagick convert inside the merge Lambda)
|
|
1940
|
+
- `video`: Slideshow from images or video concatenation (FFmpeg inside the merge Lambda)
|
|
1941
|
+
- `audio`: Audio concatenation (FFmpeg inside the merge Lambda)
|
|
1942
|
+
- `document`: PDF concatenation (qpdf inside the merge Lambda)
|
|
1943
|
+
enum:
|
|
1944
|
+
- image
|
|
1945
|
+
- gif
|
|
1946
|
+
- video
|
|
1947
|
+
- audio
|
|
1948
|
+
- document
|
|
1949
|
+
|
|
1950
|
+
ProgressStatus:
|
|
1951
|
+
type: string
|
|
1952
|
+
description: |
|
|
1953
|
+
Status values for progress updates (non-terminal):
|
|
1954
|
+
- started: Lambda picked up the operation
|
|
1955
|
+
- downloading: Downloading source file(s) from S3
|
|
1956
|
+
- processing: Actively processing (compressing, merging, etc.)
|
|
1957
|
+
- uploading: Uploading result to S3
|
|
1958
|
+
enum:
|
|
1959
|
+
- started
|
|
1960
|
+
- downloading
|
|
1961
|
+
- processing
|
|
1962
|
+
- uploading
|
|
1963
|
+
|
|
1964
|
+
ResultStatus:
|
|
1965
|
+
type: string
|
|
1966
|
+
description: |
|
|
1967
|
+
Status values for operation results (terminal):
|
|
1968
|
+
- completed: Operation finished successfully
|
|
1969
|
+
- failed: Operation encountered an error
|
|
1970
|
+
enum:
|
|
1971
|
+
- completed
|
|
1972
|
+
- failed
|
|
1973
|
+
|
|
1974
|
+
ErrorCode:
|
|
1975
|
+
type: string
|
|
1976
|
+
description: |
|
|
1977
|
+
Machine-readable error code for categorization and retry logic.
|
|
1978
|
+
|
|
1979
|
+
**Retryable errors** (SQS will retry automatically):
|
|
1980
|
+
- s3_download_failed: Source file download failed
|
|
1981
|
+
- s3_upload_failed: Output file upload failed
|
|
1982
|
+
- s3_access_denied: Permission denied (might be temporary)
|
|
1983
|
+
- out_of_memory: Lambda ran out of memory
|
|
1984
|
+
- timeout: Lambda execution timed out
|
|
1985
|
+
|
|
1986
|
+
**Non-retryable errors** (message sent to DLQ):
|
|
1987
|
+
- invalid_format: Unsupported or corrupted file
|
|
1988
|
+
- format_mismatch: MIME type doesn't match content
|
|
1989
|
+
- decode_failed: Cannot decode/parse file
|
|
1990
|
+
- processing_failed: Processing library error
|
|
1991
|
+
- output_too_large: Output larger than original (compress only)
|
|
1992
|
+
- invalid_options: Invalid operation options
|
|
1993
|
+
- missing_source: Source file not found in S3
|
|
1994
|
+
- invalid_key: S3 key is malformed
|
|
1995
|
+
- invalid_request: Operation request validation failed
|
|
1996
|
+
- unknown: Unclassified error
|
|
1997
|
+
enum:
|
|
1998
|
+
- s3_download_failed
|
|
1999
|
+
- s3_upload_failed
|
|
2000
|
+
- s3_access_denied
|
|
2001
|
+
- out_of_memory
|
|
2002
|
+
- timeout
|
|
2003
|
+
- invalid_format
|
|
2004
|
+
- format_mismatch
|
|
2005
|
+
- decode_failed
|
|
2006
|
+
- processing_failed
|
|
2007
|
+
- output_too_large
|
|
2008
|
+
- invalid_options
|
|
2009
|
+
- missing_source
|
|
2010
|
+
- invalid_key
|
|
2011
|
+
- invalid_request
|
|
2012
|
+
- unknown
|