@smart-cloud/publisher-exporter 1.1.65 → 1.1.66
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -8
- package/dist/content-sync.js +15 -14
- package/dist/crawl.js +13 -12
- package/dist/deploy.js +6 -6
- package/dist/invalidate.js +2 -2
- package/dist/queue-runner.js +3 -3
- package/dist/remote-worker-health.js +1 -1
- package/dist/remote-worker.js +3 -3
- package/package.json +2 -1
- package/remote-workers.example.json +3 -0
package/README.md
CHANGED
|
@@ -181,7 +181,7 @@ small `resultKey` pointer. Reusing the same job/task ID is idempotent.
|
|
|
181
181
|
|
|
182
182
|
The companion CDK application is maintained separately at
|
|
183
183
|
[`smartcloudsol/static-publisher-lambda-workers`](https://github.com/smartcloudsol/static-publisher-lambda-workers).
|
|
184
|
-
|
|
184
|
+
Its deploy script emits a `remote-workers.json` containing the
|
|
185
185
|
versioned Lambda alias ARNs, workspace bucket/prefix, region, and generated
|
|
186
186
|
assumable role. The file contains no static AWS access keys.
|
|
187
187
|
|
|
@@ -194,10 +194,12 @@ publisher-exporter remote-worker-health \
|
|
|
194
194
|
--runtime-dir /srv/site/runtime
|
|
195
195
|
```
|
|
196
196
|
|
|
197
|
-
Protocol v1 accepts 1-20 same-origin URLs per render task.
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
197
|
+
Protocol v1 accepts 1-20 same-origin URLs per render task. The default render
|
|
198
|
+
batch size is 5, so a warm Chromium process can render several pages
|
|
199
|
+
sequentially in one invocation while Lambda concurrency supplies parallelism
|
|
200
|
+
between batches. Only failed or deferred URLs are retried. The Lambda response
|
|
201
|
+
is not the rendered body; read the returned S3 result object using the same
|
|
202
|
+
temporary role credentials.
|
|
201
203
|
|
|
202
204
|
Enable unified Lambda processing in the Static Publisher WordPress admin. The
|
|
203
205
|
single Processing concurrency value is dual-written to the legacy per-phase
|
|
@@ -206,6 +208,15 @@ remain in the workspace through rewrite and direct S3-to-S3 deployment; the
|
|
|
206
208
|
coordinator transfers only task/result metadata. Older mixed phase settings
|
|
207
209
|
remain supported until an administrator changes the common controls. A current
|
|
208
210
|
`remote-workers.json` must contain render, asset, rewrite, and deploy-copy aliases.
|
|
211
|
+
When it also contains `status.tableName`, the exporter reads strongly consistent
|
|
212
|
+
DynamoDB heartbeats while a synchronous Lambda invocation is running and
|
|
213
|
+
forwards them to the existing runtime progress snapshot. Workers write those
|
|
214
|
+
rows directly with conditionally increasing sequence numbers.
|
|
215
|
+
|
|
216
|
+
Rewrite requests carry the original object MIME type, and workers preserve it
|
|
217
|
+
when writing transformed HTML, CSS, JavaScript, or other text back to S3. After
|
|
218
|
+
upgrading from an earlier delegated rewrite implementation, republish affected
|
|
219
|
+
targets to replace already incorrect object metadata.
|
|
209
220
|
|
|
210
221
|
The saved runtime configuration remains compatible with older exporters:
|
|
211
222
|
|
|
@@ -217,6 +228,7 @@ The saved runtime configuration remains compatible with older exporters:
|
|
|
217
228
|
"render": {
|
|
218
229
|
"enabled": true,
|
|
219
230
|
"concurrency": 4,
|
|
231
|
+
"batchSize": 5,
|
|
220
232
|
"maxAttempts": 2
|
|
221
233
|
},
|
|
222
234
|
"rewrite": {
|
|
@@ -247,9 +259,9 @@ STATIC_PUBLISHER_RUNTIME_DIR=/srv/site/runtime \
|
|
|
247
259
|
Use `--local-render` only as a legacy render diagnostic override; it disables
|
|
248
260
|
the unified remote data path for that crawl. The coordinator keeps sitemap
|
|
249
261
|
discovery, incremental and safety decisions, compact manifests, and deploy
|
|
250
|
-
planning local.
|
|
251
|
-
stays in S3, while
|
|
252
|
-
reference. Asset workers fetch and inspect bounded URL batches. Final text
|
|
262
|
+
planning local. Admitted pages are grouped into bounded Lambda batches. Their
|
|
263
|
+
HTML stays in S3, while each result returns discovered pages/assets and an
|
|
264
|
+
object reference. Asset workers fetch and inspect bounded URL batches. Final text
|
|
253
265
|
objects are rewritten in place by Lambda, then deploy workers check or copy
|
|
254
266
|
them directly from the workspace to the allowlisted target. Only explicit
|
|
255
267
|
deploy-plan tombstones may be deleted. Each completed
|