@opencraw/core 0.1.2 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (28) hide show
  1. package/README.md +1 -1
  2. package/dist/index.esm.js +762 -71
  3. package/dist/src/access/access-profile.contract.d.ts +4 -0
  4. package/dist/src/access/index.d.ts +1 -1
  5. package/dist/src/api-steps/send-request.use-case.d.ts +3 -2
  6. package/dist/src/browser-session/browser-profile.store.d.ts +52 -0
  7. package/dist/src/browser-session/browser.client.d.ts +8 -0
  8. package/dist/src/browser-session/index.d.ts +1 -0
  9. package/dist/src/crawl-events/crawl-event.contract.d.ts +8 -0
  10. package/dist/src/crawl-execution/bootstrap-session.use-case.d.ts +23 -2
  11. package/dist/src/crawl-execution/crawl-options.config.d.ts +27 -0
  12. package/dist/src/crawl-execution/rotating-runner.use-case.d.ts +9 -0
  13. package/dist/src/crawl-execution/run-crawl.use-case.d.ts +6 -2
  14. package/dist/src/crawl-execution/run-input-recipe.use-case.d.ts +11 -3
  15. package/dist/src/index.d.ts +5 -4
  16. package/dist/src/recipe-schema/index.d.ts +2 -2
  17. package/dist/src/recipe-schema/input-recipe.contract.d.ts +24 -1
  18. package/dist/src/record-sink/dedupe.policy.d.ts +19 -7
  19. package/dist/src/record-sink/index.d.ts +1 -0
  20. package/dist/src/step-flow/for-each.use-case.d.ts +4 -2
  21. package/dist/src/step-flow/host-throttle.policy.d.ts +49 -0
  22. package/dist/src/step-flow/index.d.ts +5 -0
  23. package/dist/src/step-flow/run-gate.policy.d.ts +12 -1
  24. package/dist/src/step-flow/step-runner.contract.d.ts +13 -0
  25. package/dist/src/step-flow/transport-retry.policy.d.ts +76 -0
  26. package/dist/src/web-steps/navigate.use-case.d.ts +3 -2
  27. package/dist/src/web-steps/run-web-step.use-case.d.ts +8 -1
  28. package/package.json +1 -1
package/README.md CHANGED
@@ -16,7 +16,7 @@ npx playwright install chromium # web recipes and browser bootstraps only
16
16
  | `loadRecipes(source)` | Reads, validates and binds recipes from one source holding all of them, the output recipe found by its `kind`. The source is any of the forms below. Throws `RecipeValidationError` / `RecipeBindingError` with every problem and its JSON path. |
17
17
  | `loadRecipeSet({ output, inputs })` | The same, with the output recipe given apart; each part is any of the forms below. |
18
18
  | `readRecipeSource(source)` | Decodes recipes without validating them, each with where it came from, for tooling. |
19
- | `createCrawler(options)` | Builds an engine: `hooks`, `sink` (`memorySink()` default, `jsonLinesSink(path, { append? })`), `onEvent`, `browser` settings, `dedupe` (`run` / `recipe` / `off`), `onRecipeError` (`continue` / `stop`), `resume` (skip keys the sink already has), `access` + `accessPlugins` (proxy profiles, see [access.md](../../docs/recipes/access.md)), `captchaSolvers` (see [captcha.md](../../docs/recipes/captcha.md)). |
19
+ | `createCrawler(options)` | Builds an engine: `hooks`, `sink` (`memorySink()` default, `jsonLinesSink(path, { append? })`), `onEvent`, `browser` settings, `dedupe` (`run` / `recipe` / `off`), `onRecipeError` (`continue` / `stop`), `resume` (skip keys the sink already has), `access` + `accessPlugins` (proxy profiles, see [access.md](../../docs/recipes/access.md)), `captchaSolvers` (see [captcha.md](../../docs/recipes/captcha.md)), `throttle` (per-site politeness), `retry` (requests that fail in passing), `parallel` (input recipes at once), `profilesDir` (persistent browser profiles). |
20
20
  | `loadAccessConfig(path)`, `AccessBroker`, `ACCESS_PRESETS` | Access configs: load and validate one, lease a profile outside a crawl (the cli's `probe` does), list the provider presets. |
21
21
  | `crawler.run(set)` | Runs every input recipe in sequence; returns a `CrawlReport`. |
22
22
  | `crawler.close()` | Closes the browser, if one was launched. |