@hraness/sponge 0.6.7 → 0.6.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/README.md +42 -0
  2. package/dist/capture/worker.js +154 -152
  3. package/dist/contracts/index.d.ts +2 -0
  4. package/dist/src/acquisition/capture.d.ts +4 -0
  5. package/dist/src/acquisition/capture.js +8 -2
  6. package/dist/src/acquisition/child-request.d.ts +5 -0
  7. package/dist/src/acquisition/normalize.js +3 -0
  8. package/dist/src/app/archive.d.ts +15 -0
  9. package/dist/src/app/archive.js +51 -0
  10. package/dist/src/app/article.d.ts +4 -0
  11. package/dist/src/app/article.js +24 -10
  12. package/dist/src/app/markdown.d.ts +2 -0
  13. package/dist/src/app/markdown.js +6 -0
  14. package/dist/src/browser/bridge.d.ts +48 -0
  15. package/dist/src/browser/bridge.js +90 -0
  16. package/dist/src/browser/host.d.ts +10 -0
  17. package/dist/src/browser/host.js +147 -0
  18. package/dist/src/browser/setup.d.ts +26 -0
  19. package/dist/src/browser/setup.js +103 -0
  20. package/dist/src/capture/network.d.ts +56 -0
  21. package/dist/src/capture/network.js +453 -0
  22. package/dist/src/capture/types.d.ts +26 -0
  23. package/dist/src/capture/types.js +1 -0
  24. package/dist/src/cli/commands.js +114 -12
  25. package/dist/src/cli/status.d.ts +7 -0
  26. package/dist/src/cli/status.js +33 -0
  27. package/dist/src/errors.d.ts +1 -0
  28. package/dist/src/errors.js +2 -0
  29. package/dist/src/pdf/handler.d.ts +2 -0
  30. package/dist/src/pdf/handler.js +7 -2
  31. package/dist/src/store/queue.d.ts +5 -1
  32. package/dist/src/store/queue.js +10 -4
  33. package/dist/src/store/types.d.ts +5 -1
  34. package/dist/src/worker/basic.js +15 -0
  35. package/extension/background.js +180 -0
  36. package/extension/manifest.json +25 -0
  37. package/extension/popup.html +85 -0
  38. package/extension/popup.js +39 -0
  39. package/native/runtime-manifest.json +5 -5
  40. package/package.json +2 -1
package/README.md CHANGED
@@ -41,6 +41,48 @@ websites.
41
41
  `doctor` reports runtime integrity and pending cleanup. Private-network capture
42
42
  requires `--allow-private-network` with `--wait` or `run --drain`.
43
43
 
44
+ ### Dead, blocked and paywalled pages
45
+
46
+ `sponge status` counts saved web articles under 200 words (social posts aside),
47
+ and `list --short` names them: usually a paywall stub, an error page or a page
48
+ that needed scripts. `refresh ITEM --archive` (or `add URL --archive`) captures
49
+ the newest Internet Archive copy instead, found through the Wayback Machine's CDX
50
+ index, and fetches the raw saved bytes (`id_`) without the replay toolbar. The
51
+ item keeps its original URL; the capture's `source_url` names the snapshot. Only
52
+ `--archive` sends the URL to archive.org. No copy fails with `unavailable` and
53
+ leaves the saved revision alone.
54
+
55
+ archive.today (archive.ph, archive.is) has no API, answers automated clients with
56
+ a CAPTCHA, and gives wrong DNS answers to some public resolvers, so Sponge does not
57
+ fetch from it. To use one of its copies, open it in a browser and save it as a PDF;
58
+ then `sponge add FILE.pdf` keeps it.
59
+
60
+ ### Signed-in pages through your browser
61
+
62
+ The Sponge extension lets captures use the sign-ins you already have in Chrome
63
+ (or Brave, Edge, Arc, Vivaldi, Chromium). Sponge asks the extension for a page;
64
+ the extension loads it inside your browser and hands back only the page. Sponge
65
+ never reads cookies, passwords or the browser's profile.
66
+
67
+ 1. `sponge browser setup` registers Sponge with each installed browser.
68
+ 2. In `chrome://extensions`, turn on Developer mode, choose "Load unpacked" and
69
+ pick the `extension/` folder that `setup` prints.
70
+ 3. `sponge browser status` (or `sponge doctor`) should say `connected`.
71
+
72
+ Once the extension is connected, a page that comes back thin from the public web
73
+ is tried again through the browser: first a plain fetch with your cookies, then a
74
+ muted background tab that runs the page's scripts and closes afterwards.
75
+ `refresh ITEM --browser` (or `add URL --browser`) skips the public web and reads
76
+ only through the browser. Its popup and right-click menu save the current page or
77
+ a link. If no browser is connected, a `--browser` job fails as `browser-offline`
78
+ and can be retried.
79
+
80
+ Pages captured through the browser start private. Any program running as you can
81
+ ask the extension for a page through Sponge's local socket, the same reach it
82
+ already has over your files. Turning off "Let Sponge load pages" in the popup
83
+ stops all background captures. `sponge browser remove` unregisters Sponge from
84
+ every browser.
85
+
44
86
  Images use `sponge-asset:SHA256` references. `asset ITEM SHA256 > image.png`
45
87
  retrieves verified bytes offline; `asset ITEM SHA256 --json` returns its metadata.
46
88
  Unresolved remote images become unavailable markers, and captures retain partial