@smart-cloud/publisher-exporter 1.1.64 → 1.1.66

package/README.md

@@ -174,14 +174,14 @@ Prune old archive directories with `publisher-exporter prune-logs --runtime-dir
 ## Optional Lambda workers
 
 The exporter includes protocol-v1 building blocks for delegating bounded
-render, text-rewrite, and S3 deploy-copy tasks to container-image Lambda
+render, asset-fetch, text-rewrite, and S3 deploy-copy tasks to container-image Lambda
 functions. Large payloads do not travel in the Lambda response: workers write
-HTML and detailed results below a configured S3 workspace prefix and return a
+HTML, assets, and detailed results below a configured S3 workspace prefix and return a
 small `resultKey` pointer. Reusing the same job/task ID is idempotent.
 
 The companion CDK application is maintained separately at
 [`smartcloudsol/static-publisher-lambda-workers`](https://github.com/smartcloudsol/static-publisher-lambda-workers).
-directory. Its deploy script emits a `remote-workers.json` containing the
+Its deploy script emits a `remote-workers.json` containing the
 versioned Lambda alias ARNs, workspace bucket/prefix, region, and generated
 assumable role. The file contains no static AWS access keys.
 
@@ -194,31 +194,52 @@ publisher-exporter remote-worker-health \
   --runtime-dir /srv/site/runtime
 ```
 
-Protocol v1 accepts 1-20 same-origin URLs per render task. Start with one URL
-per invocation so retries stay small and Lambda concurrency supplies the
-parallelism. The Lambda response is not the rendered body; read the returned
-S3 result object using the same temporary role credentials.
-
-Enable the desired Lambda phases in the Static Publisher WordPress admin. The saved
-runtime configuration contains only the operational settings:
+Protocol v1 accepts 1-20 same-origin URLs per render task. The default render
+batch size is 5, so a warm Chromium process can render several pages
+sequentially in one invocation while Lambda concurrency supplies parallelism
+between batches. Only failed or deferred URLs are retried. The Lambda response
+is not the rendered body; read the returned S3 result object using the same
+temporary role credentials.
+
+Enable unified Lambda processing in the Static Publisher WordPress admin. The
+single Processing concurrency value is dual-written to the legacy per-phase
+fields for mixed-version compatibility. In unified mode, HTML and asset bodies
+remain in the workspace through rewrite and direct S3-to-S3 deployment; the
+coordinator transfers only task/result metadata. Older mixed phase settings
+remain supported until an administrator changes the common controls. A current
+`remote-workers.json` must contain render, asset, rewrite, and deploy-copy aliases.
+When it also contains `status.tableName`, the exporter reads strongly consistent
+DynamoDB heartbeats while a synchronous Lambda invocation is running and
+forwards them to the existing runtime progress snapshot. Workers write those
+rows directly with conditionally increasing sequence numbers.
+
+Rewrite requests carry the original object MIME type, and workers preserve it
+when writing transformed HTML, CSS, JavaScript, or other text back to S3. After
+upgrading from an earlier delegated rewrite implementation, republish affected
+targets to replace already incorrect object metadata.
+
+The saved runtime configuration remains compatible with older exporters:
 
 ```json
 {
+  "lambdaDelegationEnabled": true,
+  "processingConcurrency": 4,
   "remoteWorkers": {
     "render": {
       "enabled": true,
       "concurrency": 4,
+      "batchSize": 5,
       "maxAttempts": 2
     },
     "rewrite": {
       "enabled": true,
-      "concurrency": 8,
+      "concurrency": 4,
       "batchSize": 25,
       "maxAttempts": 2
     },
     "deploy": {
       "enabled": true,
-      "concurrency": 8,
+      "concurrency": 4,
       "batchSize": 50,
       "maxAttempts": 2
     }
@@ -235,16 +256,15 @@ STATIC_PUBLISHER_RUNTIME_DIR=/srv/site/runtime \
   publisher-exporter crawl --remote-render
 ```
 
-Use `--local-render` to override an enabled configuration for a diagnostic run.
-The crawl coordinator keeps sitemap discovery, incremental decisions, asset
-downloads, manifests, and deploy planning local. Each admitted
-page is rendered by one Lambda task, and its HTML is read from S3 with length
-and SHA-256 verification. Failed/deferred tasks receive a new task ID when
-retried. Final text files are staged in the workspace, rewritten in bounded
-Lambda batches, checksum-verified, and restored to the local export tree.
-Deployment stages finalized objects with their cache and checksum metadata;
-deploy workers then check or copy them directly from S3 to the allowlisted
-target. Only explicit deploy-plan tombstones may be deleted. Each completed
+Use `--local-render` only as a legacy render diagnostic override; it disables
+the unified remote data path for that crawl. The coordinator keeps sitemap
+discovery, incremental and safety decisions, compact manifests, and deploy
+planning local. Admitted pages are grouped into bounded Lambda batches. Their
+HTML stays in S3, while each result returns discovered pages/assets and an
+object reference. Asset workers fetch and inspect bounded URL batches. Final text
+objects are rewritten in place by Lambda, then deploy workers check or copy
+them directly from the workspace to the allowlisted target. Only explicit
+deploy-plan tombstones may be deleted. Each completed
 batch updates phase-scoped runtime progress and adds a structured batch summary
 to `deploy.log.jsonl`; debug logging also records each returned object result.
 There is no silent local fallback for any