@smart-cloud/publisher-exporter 1.2.5 → 1.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +42 -3
- package/dist/cli.js +6 -1
- package/dist/content-sync.js +16 -16
- package/dist/crawl.js +14 -14
- package/dist/deploy.js +5 -5
- package/dist/invalidate.js +2 -2
- package/dist/queue-runner.js +3 -3
- package/dist/remote-worker.js +2 -2
- package/dist/rewrite-worker.js +2 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -41,6 +41,29 @@ manifests. The declaration does not require longer render readiness waits.
|
|
|
41
41
|
|
|
42
42
|
## Release notes
|
|
43
43
|
|
|
44
|
+
### 1.2.7
|
|
45
|
+
|
|
46
|
+
- Add `crawl --resume-rewrite --pending-only` to process only unfinished page
|
|
47
|
+
and text-asset outputs from the manifest, deduplicated by output path. It
|
|
48
|
+
does not enumerate or rewrite the rest of the export tree. Plain
|
|
49
|
+
`--resume-rewrite` retains its all-text-file behavior for changed rewrite rules.
|
|
50
|
+
- Add `--rewrite-concurrency <1..100>` as a per-run override for local rewrite
|
|
51
|
+
workers, applied after normal configuration defaults. The default continues
|
|
52
|
+
to use configured parallelism; lowering the worker count trades speed for
|
|
53
|
+
available CPU capacity for other work.
|
|
54
|
+
- Avoid constructing literal URL matchers when the source literal is absent,
|
|
55
|
+
and skip JSON-context URL escaping setup in files without script/meta tags.
|
|
56
|
+
Replacement order and URL boundary checks remain unchanged.
|
|
57
|
+
- Update only the host exporter for these options. No WordPress plugin or
|
|
58
|
+
Lambda stack update is required.
|
|
59
|
+
|
|
60
|
+
### 1.2.6
|
|
61
|
+
|
|
62
|
+
- Local rewrite resume tracks manifest completion and preserves pending deploy
|
|
63
|
+
changes. It rejects an S3-native export manifest before mutation; workspace
|
|
64
|
+
objects are authoritative and cannot use local rewrite completion flags.
|
|
65
|
+
This coordinator-only change requires no Lambda, plugin, or IAM change.
|
|
66
|
+
|
|
44
67
|
### 1.2.5
|
|
45
68
|
|
|
46
69
|
- Expand a queued URL job's `urls` array into repeated `--url` arguments for one
|
|
@@ -452,12 +475,28 @@ npx @smart-cloud/publisher-exporter queue-runner --runtime-dir /srv/site/runtime
|
|
|
452
475
|
- `crawl` without extra flags performs a normal full crawl, asset download, and final text rewrite.
|
|
453
476
|
- `crawl --crawl-mode incremental` reuses the existing crawl manifest when possible and rewrites only the files affected by re-crawled pages, text assets, or changed asset-map entries. After a successful unlimited discovery pass it also removes exporter-generated HTML that is no longer present in the current manifest and records exact deletion tombstones for the deploy step. Post-crawl copy-map destinations remain protected.
|
|
454
477
|
- An incremental crawl now fails closed when the remote WP Suite subscription configuration cannot be loaded. It never silently falls back to a full crawl that clears the existing output tree.
|
|
455
|
-
- `crawl --resume-rewrite` skips discovery, rendering, and asset download, then
|
|
478
|
+
- `crawl --resume-rewrite` skips discovery, rendering, and asset download, then rewrites existing nonempty text files using the current rewrite rules. It does not purge the page cache.
|
|
456
479
|
- `crawl --retry-timeouts` retries timed-out URLs from the latest archived full crawl or publish log snapshot.
|
|
457
480
|
|
|
458
481
|
Use `--resume-rewrite` when the already exported output is otherwise valid but the rewrite logic changed or a previous crawl stopped during the rewrite phase. This is the fastest repair path for stale output caused by rewrite-only bugs such as escaped JSON replacement issues, protocol-relative asset URLs like `//host/path.css`, or an interrupted final rewrite.
|
|
459
482
|
|
|
460
|
-
|
|
483
|
+
For an interrupted rewrite with otherwise unchanged rules, prefer
|
|
484
|
+
`crawl --resume-rewrite --pending-only`. It selects only manifest outputs whose
|
|
485
|
+
`rewriteComplete` is not true, including text assets; aliases sharing one path
|
|
486
|
+
are processed once. No pending files means no text files are scanned. A
|
|
487
|
+
`Checked unchanged` message means the selected file was read and its rewrite
|
|
488
|
+
result was byte-identical; its pending flag can still be completed safely.
|
|
489
|
+
|
|
490
|
+
The manifest is updated only after the whole rewrite phase succeeds, for paths
|
|
491
|
+
whose rewrite callbacks completed. Entries sharing a completed path are marked
|
|
492
|
+
complete together; missing or empty files remain incomplete with warnings. A
|
|
493
|
+
rewrite failure retains the incomplete marker and prior deploy plan. Successful
|
|
494
|
+
resume carries pending changes forward and includes previously incomplete files
|
|
495
|
+
that completed, even if their bytes did not change. The `rewrite-resume` plan
|
|
496
|
+
suppresses deletions and does not establish a content-sync baseline. It targets
|
|
497
|
+
changed files unless an earlier pending plan requires a full sync.
|
|
498
|
+
|
|
499
|
+
If you need to repair existing exported HTML after a rewrite bug, prefer `--resume-rewrite` over a new incremental crawl. Incremental crawl only rewrites targeted files; `--resume-rewrite` processes existing nonempty text files in the current output.
|
|
461
500
|
|
|
462
501
|
During `--resume-rewrite`, the terminal can be quieter than a full crawl. Progress still updates in `runtime/current-progress.json` and the live crawl event snapshot under the configured log directory.
|
|
463
502
|
|
|
@@ -712,7 +751,7 @@ enabled remote phase.
|
|
|
712
751
|
|
|
713
752
|
## Troubleshooting
|
|
714
753
|
|
|
715
|
-
- If exported HTML still contains old rewritten URLs after a rewrite fix, run `crawl --resume-rewrite`. This reprocesses
|
|
754
|
+
- If exported HTML still contains old rewritten URLs after a rewrite fix, run `crawl --resume-rewrite`. This reprocesses existing nonempty text files in the current output tree and does not depend on earlier `rewriteComplete: true` entries in `crawl-manifest.json`.
|
|
716
755
|
- If `--resume-rewrite` appears to stop after the initial resume message, check `runtime/current-progress.json` and `<logDir>/current-crawl-event.json` before assuming it is stuck. That phase can stay mostly quiet on stdout while it rewrites files.
|
|
717
756
|
- If direct CLI execution seems to ignore the expected runtime state, verify that `PUBLISHER_CONFIG` and `STATIC_PUBLISHER_RUNTIME_DIR` both point at the same runtime tree. A mismatched config path and runtime dir can make the run look idle or incomplete.
|
|
718
757
|
- Prefer incremental crawl when the existing manifest is trusted and you only need a fast recrawl of changed pages or assets. Prefer a full crawl when discovery rules changed, the manifest is missing or suspect, or you want a clean end-to-end rebuild.
|