apify-cli 0.10.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +91 -57
- package/index.js +1 -1
- package/npm-shrinkwrap.json +1012 -992
- package/oclif.manifest.json +1 -1
- package/package.json +1 -1
- package/src/commands/call.js +3 -2
- package/src/commands/create.js +4 -2
- package/src/commands/edit-input-schema.js +29 -10
- package/src/commands/init.js +7 -5
- package/src/commands/push.js +12 -10
- package/src/commands/run.js +3 -3
- package/src/commands/secrets/index.js +4 -2
- package/src/commands/vis.js +23 -13
- package/src/lib/consts.js +11 -2
- package/src/lib/input_schema.js +70 -0
- package/src/lib/utils.js +79 -24
package/README.md
CHANGED
|
@@ -57,7 +57,7 @@ apify --version
|
|
|
57
57
|
|
|
58
58
|
which should print something like:
|
|
59
59
|
```
|
|
60
|
-
apify-cli/0.
|
|
60
|
+
apify-cli/0.10.0 darwin-x64 node-v16.14.2
|
|
61
61
|
```
|
|
62
62
|
|
|
63
63
|
## Basic usage
|
|
@@ -81,9 +81,9 @@ cd ./my/awesome/project
|
|
|
81
81
|
apify init
|
|
82
82
|
```
|
|
83
83
|
This command will only set up local actor development environment in an existing directory,
|
|
84
|
-
i.e. it will create the
|
|
84
|
+
i.e. it will create the `.actor/actor.json` file and `apify_storage` directory.
|
|
85
85
|
|
|
86
|
-
Before you can run your project using `apify run`, you have to set up the right start command in `package.json` under scripts.start. For example:
|
|
86
|
+
Before you can run your project locally using `apify run`, you have to set up the right start command in `package.json` under scripts.start. For example:
|
|
87
87
|
```text
|
|
88
88
|
{
|
|
89
89
|
...
|
|
@@ -123,7 +123,7 @@ Note that the command will store the API token and other sensitive information t
|
|
|
123
123
|
apify push
|
|
124
124
|
```
|
|
125
125
|
|
|
126
|
-
This command uploads your project to the Apify cloud and builds an actor from it.
|
|
126
|
+
This command uploads your project to the Apify cloud and builds an actor from it. On the platform, actor needs to be built before it can be run.
|
|
127
127
|
|
|
128
128
|
### Run an actor on the Apify cloud
|
|
129
129
|
|
|
@@ -139,73 +139,95 @@ This command can also be used to run other actors, for example:
|
|
|
139
139
|
apify call apify/hello-world
|
|
140
140
|
```
|
|
141
141
|
|
|
142
|
-
### So what's in this
|
|
142
|
+
### So what's in this `.actor/actor.json` file?
|
|
143
143
|
|
|
144
144
|
This file associates your local development project with an actor on the Apify platform.
|
|
145
145
|
It contains information such as actor name, version, build tag and environment variables.
|
|
146
146
|
Make sure you commit this file to the Git repository.
|
|
147
147
|
|
|
148
|
-
For example,
|
|
148
|
+
For example, `.actor/actor.json` file can look as follows:
|
|
149
|
+
|
|
149
150
|
|
|
150
151
|
```json
|
|
151
152
|
{
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
153
|
+
"actorSpecification": 1,
|
|
154
|
+
"name": "name-of-my-scraper",
|
|
155
|
+
"version": "0.0",
|
|
156
|
+
"buildTag": "latest",
|
|
157
|
+
"environmentVariables": {
|
|
156
158
|
"MYSQL_USER": "my_username",
|
|
157
159
|
"MYSQL_PASSWORD": "@mySecretPassword"
|
|
158
|
-
|
|
159
|
-
|
|
160
|
+
},
|
|
161
|
+
"dockerfile": "./Dockerfile",
|
|
162
|
+
"readme": "./ACTOR.md",
|
|
163
|
+
"input": "./input_schema.json",
|
|
164
|
+
"storages": {
|
|
165
|
+
"dataset": "./dataset_schema.json",
|
|
166
|
+
}
|
|
160
167
|
}
|
|
161
168
|
```
|
|
162
169
|
|
|
170
|
+
**`Dockerfile` field**\
|
|
171
|
+
If you specify the path to your Docker file under the `dockerfile` field, this file will be used for actor builds on the platform. If not specified, the system will look for Docker files at `.actor/Dockerfile` and `Dockerfile` in this order of preference.
|
|
172
|
+
|
|
173
|
+
**`Readme` field** \
|
|
174
|
+
If you specify the path to your readme file under the `readme` field, the readme at this path will be used on the platform. If not specified, readme at `.actor/README.md` and `README.md` will be used in this order of preference.
|
|
175
|
+
|
|
176
|
+
**`Input` field**\
|
|
177
|
+
You can embed your [input schema](https://docs.apify.com/actors/development/input-schema#specification-version-1) object directly in `actor.json` under `input` field. Alternatively, you can provide a path to a custom input schema. If not provided, the input schema at `.actor/INPUT_SCHEMA.json` and `INPUT_SCHEMA.json` is used in this order of preference.
|
|
178
|
+
|
|
179
|
+
**`Storages.dataset` field**\
|
|
180
|
+
You can define the schema of the items in your dataset under the `storages.dataset` field. This can be either an embedded object or a path to a JSON schema file. You can read more about the schema of your actor output [here](https://docs.apify.com/actors/development/output-schema#specification-version-1).
|
|
181
|
+
|
|
182
|
+
**Note on migration from deprecated config "apify.json"**\
|
|
183
|
+
*Note that previously, actor config was stored in the `apify.json` file that has been deprecated. You can find the (very slight) differences and migration info in [migration guidelines](https://github.com/apify/apify-cli/blob/master/MIGRATIONS.md).*
|
|
184
|
+
|
|
163
185
|
## Environment variables
|
|
164
186
|
|
|
165
187
|
There are two options how you can set up environment variables for actors.
|
|
166
188
|
|
|
167
|
-
### Set up environment variables in
|
|
189
|
+
### Set up environment variables in `.actor/actor.json`
|
|
168
190
|
All keys from `env` will be set as environment variables into Apify platform after you push actor to Apify. Current values on Apify will be overridden.
|
|
169
191
|
```json
|
|
170
192
|
{
|
|
193
|
+
"actorSpecification": 1,
|
|
171
194
|
"name": "dataset-to-mysql",
|
|
172
195
|
"version": "0.1",
|
|
173
196
|
"buildTag": "latest",
|
|
174
|
-
"
|
|
197
|
+
"environmentVariables": {
|
|
175
198
|
"MYSQL_USER": "my_username",
|
|
176
199
|
"MYSQL_PASSWORD": "@mySecretPassword"
|
|
177
|
-
}
|
|
178
|
-
"template": "basic"
|
|
200
|
+
}
|
|
179
201
|
}
|
|
180
202
|
```
|
|
181
203
|
|
|
182
204
|
### Set up environment variables in Apify Console
|
|
183
205
|
In [Apify Console](https://console.apify.com/actors) select your actor, you can set up variables into Source tab.
|
|
184
|
-
After setting up variables in the app,
|
|
206
|
+
After setting up variables in the app, remove the `environmentVariables` from `.actor/actor.json`. Otherwise, variables from `.actor/actor.json` will override variables in the app.
|
|
185
207
|
```json
|
|
186
208
|
{
|
|
209
|
+
"actorSpecification": 1,
|
|
187
210
|
"name": "dataset-to-mysql",
|
|
188
211
|
"version": "0.1",
|
|
189
|
-
"buildTag": "latest"
|
|
190
|
-
"env": null,
|
|
191
|
-
"template": "basic"
|
|
212
|
+
"buildTag": "latest"
|
|
192
213
|
}
|
|
193
214
|
```
|
|
194
215
|
|
|
195
216
|
|
|
196
|
-
#### How to set secret environment variables in
|
|
217
|
+
#### How to set secret environment variables in `.actor/actor.json`
|
|
197
218
|
|
|
198
|
-
CLI provides commands to manage secrets environment variables. Secrets are stored to the
|
|
199
|
-
|
|
219
|
+
CLI provides commands to manage secrets environment variables. Secrets are stored to the `~/.apify` directory.
|
|
220
|
+
You can add a new secret using the command:
|
|
200
221
|
```bash
|
|
201
222
|
apify secrets:add mySecretPassword pwd1234
|
|
202
223
|
```
|
|
203
|
-
After adding a new secret you can use the secret in
|
|
224
|
+
After adding a new secret you can use the secret in `.actor/actor.json`.
|
|
204
225
|
```text
|
|
205
226
|
{
|
|
227
|
+
"actorSpecification": 1,
|
|
206
228
|
"name": "dataset-to-mysql",
|
|
207
229
|
...
|
|
208
|
-
"
|
|
230
|
+
"environmentVariables": {
|
|
209
231
|
"MYSQL_PASSWORD": "@mySecretPassword"
|
|
210
232
|
},
|
|
211
233
|
...
|
|
@@ -262,7 +284,7 @@ USAGE
|
|
|
262
284
|
$ apify actor
|
|
263
285
|
```
|
|
264
286
|
|
|
265
|
-
_See code: [src/commands/actor/index.js](https://github.com/apify/apify-cli/blob/v0.
|
|
287
|
+
_See code: [src/commands/actor/index.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/actor/index.js)_
|
|
266
288
|
|
|
267
289
|
## `apify actor:get-input`
|
|
268
290
|
|
|
@@ -273,7 +295,7 @@ USAGE
|
|
|
273
295
|
$ apify actor:get-input
|
|
274
296
|
```
|
|
275
297
|
|
|
276
|
-
_See code: [src/commands/actor/get-input.js](https://github.com/apify/apify-cli/blob/v0.
|
|
298
|
+
_See code: [src/commands/actor/get-input.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/actor/get-input.js)_
|
|
277
299
|
|
|
278
300
|
## `apify actor:get-value KEY`
|
|
279
301
|
|
|
@@ -287,7 +309,7 @@ ARGUMENTS
|
|
|
287
309
|
KEY Key of the record in key-value store
|
|
288
310
|
```
|
|
289
311
|
|
|
290
|
-
_See code: [src/commands/actor/get-value.js](https://github.com/apify/apify-cli/blob/v0.
|
|
312
|
+
_See code: [src/commands/actor/get-value.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/actor/get-value.js)_
|
|
291
313
|
|
|
292
314
|
## `apify actor:push-data [ITEM]`
|
|
293
315
|
|
|
@@ -308,7 +330,7 @@ DESCRIPTION
|
|
|
308
330
|
$ cat ./test.json | apify actor:push-data
|
|
309
331
|
```
|
|
310
332
|
|
|
311
|
-
_See code: [src/commands/actor/push-data.js](https://github.com/apify/apify-cli/blob/v0.
|
|
333
|
+
_See code: [src/commands/actor/push-data.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/actor/push-data.js)_
|
|
312
334
|
|
|
313
335
|
## `apify actor:set-value KEY [VALUE]`
|
|
314
336
|
|
|
@@ -338,7 +360,7 @@ DESCRIPTION
|
|
|
338
360
|
$ cat ./my-text-file.txt | apify actor:set-value KEY --contentType text/plain
|
|
339
361
|
```
|
|
340
362
|
|
|
341
|
-
_See code: [src/commands/actor/set-value.js](https://github.com/apify/apify-cli/blob/v0.
|
|
363
|
+
_See code: [src/commands/actor/set-value.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/actor/set-value.js)_
|
|
342
364
|
|
|
343
365
|
## `apify call [ACTID]`
|
|
344
366
|
|
|
@@ -350,7 +372,7 @@ USAGE
|
|
|
350
372
|
|
|
351
373
|
ARGUMENTS
|
|
352
374
|
ACTID Name or ID of the actor to run (e.g. "apify/hello-world" or "E2jjCZBezvAZnX8Rb"). If not provided, the command
|
|
353
|
-
runs the remote actor specified in the "
|
|
375
|
+
runs the remote actor specified in the ".actor/actor.json" file.
|
|
354
376
|
|
|
355
377
|
OPTIONS
|
|
356
378
|
-b, --build=build Tag or number of the build to run (e.g. "latest" or "1.2.34").
|
|
@@ -359,11 +381,11 @@ OPTIONS
|
|
|
359
381
|
-w, --wait-for-finish=wait-for-finish Seconds for waiting to run to finish, if no value passed, it waits forever.
|
|
360
382
|
|
|
361
383
|
DESCRIPTION
|
|
362
|
-
The actor is run under your current Apify account
|
|
384
|
+
The actor is run under your current Apify account. Therefore you need to be logged in by calling "apify login". It
|
|
363
385
|
takes input for the actor from the default local key-value store by default.
|
|
364
386
|
```
|
|
365
387
|
|
|
366
|
-
_See code: [src/commands/call.js](https://github.com/apify/apify-cli/blob/v0.
|
|
388
|
+
_See code: [src/commands/call.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/call.js)_
|
|
367
389
|
|
|
368
390
|
## `apify create [ACTORNAME]`
|
|
369
391
|
|
|
@@ -382,7 +404,7 @@ OPTIONS
|
|
|
382
404
|
find available template names.
|
|
383
405
|
```
|
|
384
406
|
|
|
385
|
-
_See code: [src/commands/create.js](https://github.com/apify/apify-cli/blob/v0.
|
|
407
|
+
_See code: [src/commands/create.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/create.js)_
|
|
386
408
|
|
|
387
409
|
## `apify info`
|
|
388
410
|
|
|
@@ -396,7 +418,7 @@ DESCRIPTION
|
|
|
396
418
|
The information is printed to the console.
|
|
397
419
|
```
|
|
398
420
|
|
|
399
|
-
_See code: [src/commands/info.js](https://github.com/apify/apify-cli/blob/v0.
|
|
421
|
+
_See code: [src/commands/info.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/info.js)_
|
|
400
422
|
|
|
401
423
|
## `apify init [ACTORNAME]`
|
|
402
424
|
|
|
@@ -410,13 +432,13 @@ ARGUMENTS
|
|
|
410
432
|
ACTORNAME Name of the actor. If not provided, you will be prompted for it.
|
|
411
433
|
|
|
412
434
|
DESCRIPTION
|
|
413
|
-
The command only creates the "
|
|
414
|
-
touch anything else.
|
|
435
|
+
The command only creates the ".actor/actor.json" file and the "storage" directory in the current directory, but will
|
|
436
|
+
not touch anything else.
|
|
415
437
|
|
|
416
438
|
WARNING: The directory at "storage" will be overwritten if it already exists.
|
|
417
439
|
```
|
|
418
440
|
|
|
419
|
-
_See code: [src/commands/init.js](https://github.com/apify/apify-cli/blob/v0.
|
|
441
|
+
_See code: [src/commands/init.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/init.js)_
|
|
420
442
|
|
|
421
443
|
## `apify login`
|
|
422
444
|
|
|
@@ -434,7 +456,7 @@ DESCRIPTION
|
|
|
434
456
|
"apify" commands. To log out, call "apify logout".
|
|
435
457
|
```
|
|
436
458
|
|
|
437
|
-
_See code: [src/commands/login.js](https://github.com/apify/apify-cli/blob/v0.
|
|
459
|
+
_See code: [src/commands/login.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/login.js)_
|
|
438
460
|
|
|
439
461
|
## `apify logout`
|
|
440
462
|
|
|
@@ -449,7 +471,7 @@ DESCRIPTION
|
|
|
449
471
|
call "apify login".
|
|
450
472
|
```
|
|
451
473
|
|
|
452
|
-
_See code: [src/commands/logout.js](https://github.com/apify/apify-cli/blob/v0.
|
|
474
|
+
_See code: [src/commands/logout.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/logout.js)_
|
|
453
475
|
|
|
454
476
|
## `apify push [ACTORID]`
|
|
455
477
|
|
|
@@ -461,30 +483,30 @@ USAGE
|
|
|
461
483
|
|
|
462
484
|
ARGUMENTS
|
|
463
485
|
ACTORID ID of an existing actor on the Apify platform where the files will be pushed. If not provided, the command
|
|
464
|
-
will create or modify the actor with the name specified in "
|
|
486
|
+
will create or modify the actor with the name specified in ".actor/actor.json" file.
|
|
465
487
|
|
|
466
488
|
OPTIONS
|
|
467
489
|
-b, --build-tag=build-tag Build tag to be applied to the successful actor build. By default, it is taken
|
|
468
|
-
from the "
|
|
490
|
+
from the ".actor/actor.json" file
|
|
469
491
|
|
|
470
492
|
-v, --version=version Actor version number to which the files should be pushed. By default, it is
|
|
471
|
-
taken from the "
|
|
493
|
+
taken from the ".actor/actor.json" file.
|
|
472
494
|
|
|
473
495
|
-w, --wait-for-finish=wait-for-finish Seconds for waiting to build to finish, if no value passed, it waits forever.
|
|
474
496
|
|
|
475
497
|
--version-number=version-number DEPRECATED: Use flag version instead. Actor version number to which the files
|
|
476
|
-
should be pushed. By default, it is taken from the "
|
|
498
|
+
should be pushed. By default, it is taken from the ".actor/actor.json" file.
|
|
477
499
|
|
|
478
500
|
DESCRIPTION
|
|
479
|
-
The actor settings are read from the "
|
|
480
|
-
command-line options.
|
|
501
|
+
The actor settings are read from the ".actor/actor.json" file in the current directory, but they can be overridden
|
|
502
|
+
using command-line options.
|
|
481
503
|
NOTE: If the source files are smaller than 3 MB then they are uploaded as
|
|
482
504
|
"Multiple source files", otherwise they are uploaded as "Zip file".
|
|
483
505
|
|
|
484
506
|
WARNING: If the target actor already exists in your Apify account, it will be overwritten!
|
|
485
507
|
```
|
|
486
508
|
|
|
487
|
-
_See code: [src/commands/push.js](https://github.com/apify/apify-cli/blob/v0.
|
|
509
|
+
_See code: [src/commands/push.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/push.js)_
|
|
488
510
|
|
|
489
511
|
## `apify run`
|
|
490
512
|
|
|
@@ -510,11 +532,11 @@ DESCRIPTION
|
|
|
510
532
|
example, this causes the actor input, as well as all other data in key-value stores, datasets or request queues to be
|
|
511
533
|
stored in the "storage" directory, rather than on the Apify platform.
|
|
512
534
|
|
|
513
|
-
NOTE: You can override the default
|
|
514
|
-
|
|
535
|
+
NOTE: You can override the command's default behavior by overriding the npm start script value in a package.json file.
|
|
536
|
+
You can set up your own main file or environment variables by changing it.
|
|
515
537
|
```
|
|
516
538
|
|
|
517
|
-
_See code: [src/commands/run.js](https://github.com/apify/apify-cli/blob/v0.
|
|
539
|
+
_See code: [src/commands/run.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/run.js)_
|
|
518
540
|
|
|
519
541
|
## `apify secrets`
|
|
520
542
|
|
|
@@ -528,11 +550,13 @@ DESCRIPTION
|
|
|
528
550
|
Example:
|
|
529
551
|
$ apify secrets:add mySecret TopSecretValue123
|
|
530
552
|
|
|
531
|
-
Now the "mySecret" value can be used in an environment variable defined in "
|
|
553
|
+
Now the "mySecret" value can be used in an environment variable defined in ".actor/actor.json" file by adding the "@"
|
|
554
|
+
prefix:
|
|
532
555
|
|
|
533
556
|
{
|
|
557
|
+
"actorSpecification": 1,
|
|
534
558
|
"name": "my_actor",
|
|
535
|
-
"
|
|
559
|
+
"environmentVariables": { "SECRET_ENV_VAR": "@mySecret" },
|
|
536
560
|
"version": "0.1
|
|
537
561
|
}
|
|
538
562
|
|
|
@@ -540,7 +564,7 @@ DESCRIPTION
|
|
|
540
564
|
of the actor.
|
|
541
565
|
```
|
|
542
566
|
|
|
543
|
-
_See code: [src/commands/secrets/index.js](https://github.com/apify/apify-cli/blob/v0.
|
|
567
|
+
_See code: [src/commands/secrets/index.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/secrets/index.js)_
|
|
544
568
|
|
|
545
569
|
## `apify secrets:add NAME VALUE`
|
|
546
570
|
|
|
@@ -558,7 +582,7 @@ DESCRIPTION
|
|
|
558
582
|
The secrets are stored to a file at ~/.apify
|
|
559
583
|
```
|
|
560
584
|
|
|
561
|
-
_See code: [src/commands/secrets/add.js](https://github.com/apify/apify-cli/blob/v0.
|
|
585
|
+
_See code: [src/commands/secrets/add.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/secrets/add.js)_
|
|
562
586
|
|
|
563
587
|
## `apify secrets:rm NAME`
|
|
564
588
|
|
|
@@ -572,11 +596,11 @@ ARGUMENTS
|
|
|
572
596
|
NAME Name of the secret
|
|
573
597
|
```
|
|
574
598
|
|
|
575
|
-
_See code: [src/commands/secrets/rm.js](https://github.com/apify/apify-cli/blob/v0.
|
|
599
|
+
_See code: [src/commands/secrets/rm.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/secrets/rm.js)_
|
|
576
600
|
|
|
577
601
|
## `apify vis [PATH]`
|
|
578
602
|
|
|
579
|
-
Validates
|
|
603
|
+
Validates input schema and prints errors found.
|
|
580
604
|
|
|
581
605
|
```
|
|
582
606
|
USAGE
|
|
@@ -584,7 +608,17 @@ USAGE
|
|
|
584
608
|
|
|
585
609
|
ARGUMENTS
|
|
586
610
|
PATH Optional path to your INPUT_SCHEMA.json file. If not provided ./INPUT_SCHEMA.json is used.
|
|
611
|
+
|
|
612
|
+
DESCRIPTION
|
|
613
|
+
The input schema for the actor is used from these locations in order of preference.
|
|
614
|
+
The first one found is validated as it would be the one used on the Apify platform.
|
|
615
|
+
1. Directly embedded object in ".actor/actor.json" under 'input' key
|
|
616
|
+
2. Path to JSON file referenced in ".actor/actor.json" under 'input' key
|
|
617
|
+
3. JSON file at .actor/INPUT_SCHEMA.json
|
|
618
|
+
4. JSON file at INPUT_SCHEMA.json
|
|
619
|
+
|
|
620
|
+
You can also pass any custom path to your input schema to have it validated instead.
|
|
587
621
|
```
|
|
588
622
|
|
|
589
|
-
_See code: [src/commands/vis.js](https://github.com/apify/apify-cli/blob/v0.
|
|
623
|
+
_See code: [src/commands/vis.js](https://github.com/apify/apify-cli/blob/v0.11.0/src/commands/vis.js)_
|
|
590
624
|
<!-- commandsstop -->
|
package/index.js
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
module.exports = {};
|
|
1
|
+
module.exports = {};
|