@scrapeatlas/cli 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +99 -0
- package/bin/scrapeatlas.mjs +7 -0
- package/catalog.json +4609 -0
- package/package.json +36 -0
- package/src/cli.mjs +320 -0
- package/src/config.mjs +91 -0
- package/src/http.mjs +104 -0
- package/src/schema.mjs +66 -0
package/README.md
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# ScrapeAtlas CLI
|
|
2
|
+
|
|
3
|
+
The official terminal client for the [ScrapeAtlas API](https://scrapeatlas.com/docs/).
|
|
4
|
+
Requires Node.js 22 or newer and a ScrapeAtlas customer API key.
|
|
5
|
+
|
|
6
|
+
```sh
|
|
7
|
+
npm install -g @scrapeatlas/cli
|
|
8
|
+
scrapeatlas auth login
|
|
9
|
+
scrapeatlas bluesky profile --handle bsky.app --pretty
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
Or run without a global installation:
|
|
13
|
+
|
|
14
|
+
```sh
|
|
15
|
+
npx @scrapeatlas/cli list
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
## Discover and call endpoints
|
|
19
|
+
|
|
20
|
+
```sh
|
|
21
|
+
scrapeatlas list
|
|
22
|
+
scrapeatlas list reddit
|
|
23
|
+
scrapeatlas bluesky profile --help
|
|
24
|
+
scrapeatlas list bluesky --json
|
|
25
|
+
scrapeatlas bluesky posts --handle bsky.app --limit 10
|
|
26
|
+
scrapeatlas call blueskyProfile --input '{"handle":"bsky.app"}'
|
|
27
|
+
scrapeatlas reddit search --input @request.json
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Commands and request schemas are generated from the API's gateway/MCP catalog.
|
|
31
|
+
`list <platform> --json` includes operation IDs, paths, JSON Schemas and availability.
|
|
32
|
+
Only cataloged operations are included. A catalog entry does not guarantee the
|
|
33
|
+
provider is available on a particular deployment; inspect its availability and the
|
|
34
|
+
actual response. API validation remains authoritative for cross-field rules and
|
|
35
|
+
URL refinements.
|
|
36
|
+
|
|
37
|
+
Use `scrapeatlas <platform> <action>`. Parameter flags accept the API's original
|
|
38
|
+
camelCase/snake_case name and its kebab-case spelling (`--user-id`). Boolean
|
|
39
|
+
parameters use `--flag` / `--no-flag`; explicit booleans, arrays and nested objects
|
|
40
|
+
can be passed through `--input '<JSON object>'` or `--input @file.json`. Required
|
|
41
|
+
parameters, types and structural bounds are checked locally before a request.
|
|
42
|
+
Duplicate parameters and unknown options are rejected.
|
|
43
|
+
|
|
44
|
+
## Authentication
|
|
45
|
+
|
|
46
|
+
Get a customer API key from [your dashboard](https://scrapeatlas.com/dashboard/keys/).
|
|
47
|
+
`auth login` reads a hidden key and verifies it through the gateway before saving.
|
|
48
|
+
For noninteractive login, pipe a key to `scrapeatlas auth login --stdin`.
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
scrapeatlas auth status
|
|
52
|
+
scrapeatlas auth logout
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Credential priority is `--api-key`, then `SCRAPEATLAS_API_KEY`, then stored login.
|
|
56
|
+
Prefer the environment variable in CI; a flag can appear in shell history and
|
|
57
|
+
process lists. `auth status` reports the credential source without printing the
|
|
58
|
+
key; it does not make a live verification request. Logout removes the stored key;
|
|
59
|
+
it does not revoke the key in your account or clear an environment variable.
|
|
60
|
+
|
|
61
|
+
Config is stored in `$XDG_CONFIG_HOME/scrapeatlas/config.json`, defaulting to
|
|
62
|
+
`~/.config/scrapeatlas/config.json`. Override with `SCRAPEATLAS_CONFIG_DIR`.
|
|
63
|
+
The config directory is mode 0700 and the file is mode 0600 on POSIX. The file
|
|
64
|
+
contains a plaintext key protected by those permissions; it is not a keychain.
|
|
65
|
+
|
|
66
|
+
The default API origin is `https://api.scrapeatlas.com`. `--base-url` overrides
|
|
67
|
+
`SCRAPEATLAS_BASE_URL`, the saved origin, then the default. HTTPS origins are
|
|
68
|
+
supported; HTTP is allowed only on loopback for local development. Stored keys
|
|
69
|
+
are used only for their saved origin. An explicit flag/environment key can be
|
|
70
|
+
used with a custom gateway. No cookies or provider worker secrets are required.
|
|
71
|
+
|
|
72
|
+
## Output and failures
|
|
73
|
+
|
|
74
|
+
```sh
|
|
75
|
+
scrapeatlas bluesky profile --handle bsky.app | jq '.followersCount'
|
|
76
|
+
scrapeatlas bluesky profile --handle bsky.app --pretty
|
|
77
|
+
scrapeatlas bluesky profile --handle bsky.app --output profile.json
|
|
78
|
+
scrapeatlas bluesky profile --handle bsky.app --envelope
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Default stdout is the complete compact gateway JSON. `--pretty` indents it.
|
|
82
|
+
`--envelope` wraps it as `{httpStatus, body, requestId?, retryAfter?}`.
|
|
83
|
+
`--output` writes a new file and prints only its absolute path; existing files
|
|
84
|
+
are never overwritten. Diagnostics are structured JSON on stderr and never
|
|
85
|
+
include request parameters, credentials, or retrieved content.
|
|
86
|
+
|
|
87
|
+
Nulls, failed lookups, partial coverage, ordering and pagination tokens are
|
|
88
|
+
preserved. A usable partial response exits 0; inspect `status` and `coverage`
|
|
89
|
+
before treating results as complete. HTTP errors or explicit `success: false`,
|
|
90
|
+
`status: failed` and `status: challenged` exit 1 while keeping the full API body
|
|
91
|
+
on stdout. Local argument/config/file errors exit 2. Ctrl-C during login exits 130.
|
|
92
|
+
|
|
93
|
+
Each call makes one gateway request, with no retries, automatic pagination,
|
|
94
|
+
redirect following or media downloads. Pass the returned cursor explicitly for
|
|
95
|
+
another page. Requests are capped at 64 KiB and responses at 16 MiB. The deadline
|
|
96
|
+
covers headers and body: 90 seconds by default, configurable using
|
|
97
|
+
`--timeout <milliseconds>` from 1 to 120000. Failures never masquerade as empty
|
|
98
|
+
successes. This version exports JSON only and does not install agent configs,
|
|
99
|
+
run an MCP server, perform browser signup, or supply a TypeScript SDK.
|