@opencraw/cli 0.1.30 → 0.1.31

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +9 -9
  2. package/package.json +5 -5
package/README.md CHANGED
@@ -1,10 +1,10 @@
1
1
  <p align="center">
2
- <img src="https://raw.githubusercontent.com/russoedu/open.craw/main/docs/assets/opencraw-logo.svg" alt="OpenCraw" width="360">
2
+ <img src="https://raw.githubusercontent.com/Open-Craw/OpenCraw/main/docs/assets/opencraw-logo.svg" alt="OpenCraw" width="360">
3
3
  </p>
4
4
 
5
5
  # @opencraw/cli
6
6
 
7
- Command-line tools for [`@opencraw/core`](https://github.com/russoedu/open.craw/blob/main/packages/core): probe a page for scrapable data, validate recipes,
7
+ Command-line tools for [`@opencraw/core`](https://github.com/Open-Craw/OpenCraw/blob/main/packages/core): probe a page for scrapable data, validate recipes,
8
8
  and run a crawl from the terminal.
9
9
 
10
10
  ## Install
@@ -48,11 +48,11 @@ input recipes) and reports what happened.
48
48
  | `--dry-run` | One record per input recipe, printed with the scope it was mapped from — for checking a recipe under construction without a full run. |
49
49
  | `--trace` | Print the crawl trace to stderr. |
50
50
  | `--headed` | Show the browser instead of running headless. |
51
- | `--parallel <n>` | Run this many input recipes at once (default 1). Iterations inside a recipe follow its `limits.concurrency`. See [§3.9 of the authoring guide](https://github.com/russoedu/open.craw/blob/main/docs/recipes/authoring.md#39-concurrency). |
52
- | `--retries <n>` | Tries per request that fails in passing (a dropped connection, a timeout, a 429, a 5xx), for recipes whose `limits.retry` says nothing. Default 3; `1` turns retrying off. See [§7 of the authoring guide](https://github.com/russoedu/open.craw/blob/main/docs/recipes/authoring.md#7-policies-what-happens-when-something-is-missing-or-fails). |
53
- | `--profiles <dir>` | Where the persistent browser profiles of `session.browserProfile` live (or `OPENCRAW_PROFILES`; default `.opencraw/profiles`). See [access.md](https://github.com/russoedu/open.craw/blob/main/docs/recipes/access.md#persistent-browser-profiles). |
54
- | `--host-delay <ms>`, `--host-concurrency <n>` | Per-site politeness across every recipe: at least `ms` between two requests to one site, at most `n` in flight. They override the access config's `throttle` defaults. See [access.md](https://github.com/russoedu/open.craw/blob/main/docs/recipes/access.md#throttling-per-site). |
55
- | `--plugins <file>` (or `--hooks`) | A plugins module: named exports `hooks` (the recipes' hook steps and transforms), `accessPlugins` (for `{ kind: "plugin" }` access profiles) and `captchaSolvers`. A module whose default export is `{ name: function }` is read as hooks alone. `OPENCRAW_PLUGINS` / `OPENCRAW_HOOKS` when not given; `probe` reads it too. It runs as your code; load only files you trust. See [§6 of the authoring guide](https://github.com/russoedu/open.craw/blob/main/docs/recipes/authoring.md#6-hooks). |
51
+ | `--parallel <n>` | Run this many input recipes at once (default 1). Iterations inside a recipe follow its `limits.concurrency`. See [§3.9 of the authoring guide](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/authoring.md#39-concurrency). |
52
+ | `--retries <n>` | Tries per request that fails in passing (a dropped connection, a timeout, a 429, a 5xx), for recipes whose `limits.retry` says nothing. Default 3; `1` turns retrying off. See [§7 of the authoring guide](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/authoring.md#7-policies-what-happens-when-something-is-missing-or-fails). |
53
+ | `--profiles <dir>` | Where the persistent browser profiles of `session.browserProfile` live (or `OPENCRAW_PROFILES`; default `.opencraw/profiles`). See [access.md](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/access.md#persistent-browser-profiles). |
54
+ | `--host-delay <ms>`, `--host-concurrency <n>` | Per-site politeness across every recipe: at least `ms` between two requests to one site, at most `n` in flight. They override the access config's `throttle` defaults. See [access.md](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/access.md#throttling-per-site). |
55
+ | `--plugins <file>` (or `--hooks`) | A plugins module: named exports `hooks` (the recipes' hook steps and transforms), `accessPlugins` (for `{ kind: "plugin" }` access profiles) and `captchaSolvers`. A module whose default export is `{ name: function }` is read as hooks alone. `OPENCRAW_PLUGINS` / `OPENCRAW_HOOKS` when not given; `probe` reads it too. It runs as your code; load only files you trust. See [§6 of the authoring guide](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/authoring.md#6-hooks). |
56
56
 
57
57
  ```sh
58
58
  opencraw run recipes/ --out out/products.jsonl --trace
@@ -118,7 +118,7 @@ opencraw probe ./decks/incentivi.pptx
118
118
  ```
119
119
 
120
120
  Use it before writing an input recipe, to find the shape a site's data actually takes (§9 of
121
- [the authoring guide](https://github.com/russoedu/open.craw/blob/main/docs/recipes/authoring.md)).
121
+ [the authoring guide](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/authoring.md)).
122
122
 
123
123
  ### Options common to `run` and `probe`
124
124
 
@@ -126,7 +126,7 @@ Use it before writing an input recipe, to find the shape a site's data actually
126
126
  |---|---|
127
127
  | `--browser-path <path>` | A browser binary other than the one Playwright installed (or `OPENCRAW_CHROMIUM`). |
128
128
  | `--insecure-tls` | Accept an intercepting proxy's certificate (or `OPENCRAW_INSECURE_TLS=1`). |
129
- | `--access <file>` | An access config: proxy profiles, with credentials as `{{env.NAME}}` (or `OPENCRAW_ACCESS`). See [access.md](https://github.com/russoedu/open.craw/blob/main/docs/recipes/access.md). |
129
+ | `--access <file>` | An access config: proxy profiles, with credentials as `{{env.NAME}}` (or `OPENCRAW_ACCESS`). See [access.md](https://github.com/Open-Craw/OpenCraw/blob/main/docs/recipes/access.md). |
130
130
  | `--access-profile <name>` | The profile used by recipes that name none, overriding the config's `default`. `probe` uses it for its fetch. |
131
131
  | `--user-agent <ua>` | The user agent to send. |
132
132
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@opencraw/cli",
3
- "version": "0.1.30",
3
+ "version": "0.1.31",
4
4
  "type": "module",
5
5
  "main": "./dist/index.esm.js",
6
6
  "module": "./dist/index.esm.js",
@@ -24,12 +24,12 @@
24
24
  "!**/*.js.map"
25
25
  ],
26
26
  "dependencies": {
27
- "@opencraw/core": "^0.1.29",
28
- "@opencraw/probe": "^0.0.12",
27
+ "@opencraw/core": "^0.1.30",
28
+ "@opencraw/probe": "^0.0.13",
29
29
  "playwright": "^1.63.0"
30
30
  },
31
31
  "peerDependencies": {
32
- "@opencraw/studio": "^0.0.19"
32
+ "@opencraw/studio": "^0.0.20"
33
33
  },
34
34
  "peerDependenciesMeta": {
35
35
  "@opencraw/studio": {
@@ -43,7 +43,7 @@
43
43
  "license": "MIT",
44
44
  "repository": {
45
45
  "type": "git",
46
- "url": "git+https://github.com/russoedu/open.craw.git",
46
+ "url": "git+https://github.com/Open-Craw/OpenCraw.git",
47
47
  "directory": "packages/cli"
48
48
  },
49
49
  "keywords": [