apify-cli 1.0.0-beta.3 → 1.0.0-beta.31

package/CHANGELOG.md

@@ -0,0 +1,200 @@
+**Changelog file is no longer maintained, you can check all changes in [GitHub Releases](https://github.com/apify/apify-cli/releases).**
+
+0.6.1 / 2020-05-18
+==================
+- **BREAKING:** Templates are now fully decoupled from this project and
+  the [templates repository](https://github.com/apify/actor-templates)
+  serves as the single source of truth. Some templates were replaced
+  and others were renamed to better clarify their purpose.
+- **BREAKING:** Providing an invalid template in `apify.json` no longer
+  throws, but rather silently uses a reasonable default configuration.
+  This is to support regular changes to templates without breaking older
+  versions of the CLI.
+
+0.5.3 / 2020-03-03
+==================
+- Moved templates to separate repository
+- Fixed: creating `apify_storage` in root folder after `apify create` command
+
+0.5.2 / 2020-01-22
+==================
+- Added bot(dependabot.com) to check latest Apify SDK version in all templates
+- Updated apify package in all templates
+- Updated npm packages and fixed all npm audit issues
+
+0.5.1 / 2019-12-19
+==================
+- Added warning about outdated node.js version
+- Fixed infinite push, when the previous one was interrupted
+- Fixed calling public actors with `apify call`
+- `apify init` create empty INPUT.json file
+
+0.5.0 / 2019-11-27
+==================
+- Drop support for node 8 and 9
+- Fix: Pass the --max-http-header-size=80000 to the nodeJs process
+
+0.4.1 / 2019-10-02
+==================
+- New actor template for Apify projects, you can create it with `apify create --template apify_project`
+- `apify vis` - Using improved schema validator
+
+0.4.0 / 2019-09-23
+==================
+- Breaking Change - `apify push`: Pushes source code as a "Multiple source files" in case source code is less that 3 MB
+
+0.3.12 / 2019-09-18
+==================
+Bug fixes:
+- `apify create`: Added validation for actor name
+- `apify init` skips creation of apify.json if already exists
+- `apify run -p` runs actor, if apify_storage doesn't exist
+- Updated packages
+- Additional minor fixes
+
+0.3.11 / 2019-07-26
+==================
+- Updated packages
+- Updated Cheerio Crawler template
+- Updated Apify package version in all templates
+
+0.3.10 / 2019-06-03
+==================
+- Updated packages
+
+0.3.9 / 2019-05-15
+==================
+- Improved the templates and texts
+
+0.3.8 / 2019-03-29
+==================
+- Updated all templates regarding the last version of apify SDK.
+
+0.3.7 / 2019-03-18
+==================
+- Fixed templates to use Apify.getInput(), replaced deprecated function and options,
+added debug fields, added .idea to .gitignore
+- Updated packages
+- Fixed bug: Users without username can use push/call command
+
+0.3.6 / 2019-01-29
+==================
+- Added command `apify vis` that validates actor input schema.
+
+0.3.5 / 2019-01-25
+==================
+- Upgraded to apify@0.11 in templates
+
+0.3.3 / 2018-12-12
+==================
+- Omitted CMD command in all templates Dockerfile.
+
+0.3.2 / 2018-12-05
+==================
+- Updated apify-client package. It fixed bug, when user can not push actor, whe he changed version in apify.json.
+
+0.3.1 / 2018-11-29
+==================
+- :tada: New commands to manage secret environment variables: `apify secrets:add`, `apify secrets:rm`.
+- New documentation how to set environment variable in `apify.json`, see [documentation](https://github.com/apify/apify-cli/blob/master/README.md#environment-variables).
+- **BREAKING CHANGES**: Simplified `apify.json` structure. It will be updated automatically before execution apify run and push command.
+- Command `apify create` now shows progress bar of npm install.
+- Small bugs fixes
+
+0.2.7 / 2018-11-27
+==================
+- Updated all templates to latest apify packages
+
+0.2.6 / 2018-11-09
+==================
+- Added warning if `apify run` reuse old state in storage
+- Fixed issues #70 #65 #68
+
+0.2.5 / 2018-10-31
+==================
+- Updated NPM dependencies
+- Upgraded to apify-shared@0.1.6
+- Fixed templates to use apify/actor-node-chrome Docker image instead of outdated apify/actor-node-puppeteer
+
+0.2.3 / 2018-09-17
+==================
+- Updated all templates to apify version 0.8.*
+- Added template named hello_word
+
+0.2.1 / 2018-09-17
+==================
+- **BREAKING CHANGES**: The local storage directories have been renamed and package.json files needs a new `start` command.
+  See [migration guide](/MIGRATIONS.md) for existing projects if you are upgrading from 0.1.* to 0.2.*.
+- You can specified another file that main.js for `apify run` command using npm start script.
+
+0.2.0 / 2018-09-12
+==================
+- **BREAKING CHANGES**: Version 0.2.0 of Apify CLI supports only version 0.7.0 of API SDK or newer as management of environment variables
+  has been changed according to Apify SDK version 0.7.0.
+- Dropped support for Node 7
+
+0.1.18 / 2018-09-12
+===================
+- Updated NPM dependencies, npm-shrinkwrap.json replaced with package-lock.json
+- Updated NPM dependencies in code templates
+
+0.1.15 / 2018-07-23
+===================
+- Rename act to actor
+
+0.1.13 / 2018-07-12
+===================
+- Add environment variables for enable live view for local actors.
+
+0.1.12 / 2018-06-28
+===================
+- From now `apify call` and `apify push` commands stream live logs from run and build to your terminal
+- Add options -p, --purge, --purge-dataset, --purge-key-value-store, --purge-queue in `apify run` to clean stores before runs actor locally
+- Add option -w, --wait-for-finish=wait-for-finish in `apify push` and `apify call` - command waits x seconds to finish run or build on Apify
+- Fixes #26, #33, #34, #36, #38, #39, #37, #35
+
+0.1.11 / 2018-05-30
+===================
+- Use npm-shrinkwrap.json instead of package-lock.json for published module
+- Update template, where we using proxy
+- Fix #30
+
+0.1.9 / 2018-04-18
+==================
+- apify run takes APIFY_USER_ID and APIFY_TOKEN as environments variables, if client is logged locally
+- apify call takes input from default local key-value-store
+- Fix: duplicates new lines in log
+
+0.1.8 / 2018-04-17
+==================
+- Print warning if you have old version of cli
+- apify run - kills all sub processes for SIGINT signal (ctrl+c) - It kills all related browsers in apify run command, related issue:
+  https://github.com/apify/apify-js/issues/72
+
+0.1.7 / 2018-04-12
+==================
+- Readme and templates updates
+
+0.1.6 / 2018-04-11
+==================
+- Add support for request queue
+
+0.1.5 / 2018-04-09
+==================
+- Works for windows
+- New command apify info
+
+0.1.x / 2018-04-01
+==================
+- The first public release
+
+0.0.x / 2018-03-01
+==================
+- Initial development, lot of new stuff
+
+
+
+
+
+
+

package/README.md

@@ -51,7 +51,7 @@ If you receive an `EACCES` error, you might need to run the command as root:
 sudo npm -g install apify-cli
 ```
 
-Alternativaly, you can use [Node Version Manager (nvm)](https://github.com/nvm-sh/nvm) and install Apify CLI only into a selected user-level Node version without requiring root privileges:
+Alternatively, you can use [Node Version Manager (nvm)](https://github.com/nvm-sh/nvm) and install Apify CLI only into a selected user-level Node version without requiring root privileges:
 
 ```
 nvm install 16
@@ -276,13 +276,13 @@ This section contains printouts of `apify help` for all commands.
 
 <!-- commands -->
 * [`apify actor`](#apify-actor)
-* [`apify actor:get-input`](#apify-actorget-input)
-* [`apify actor:get-value KEY`](#apify-actorget-value-key)
-* [`apify actor:push-data [ITEM]`](#apify-actorpush-data-item)
-* [`apify actor:set-value KEY [VALUE]`](#apify-actorset-value-key-value)
+* [`apify actor get-input`](#apify-actor-get-input)
+* [`apify actor get-value KEY`](#apify-actor-get-value-key)
+* [`apify actor push-data [ITEM]`](#apify-actor-push-data-item)
+* [`apify actor set-value KEY [VALUE]`](#apify-actor-set-value-key-value)
 * [`apify call [ACTORID]`](#apify-call-actorid)
 * [`apify create [ACTORNAME]`](#apify-create-actorname)
-* [`apify help [COMMANDS]`](#apify-help-commands)
+* [`apify help [COMMAND]`](#apify-help-command)
 * [`apify info`](#apify-info)
 * [`apify init [ACTORNAME]`](#apify-init-actorname)
 * [`apify login`](#apify-login)
@@ -291,8 +291,8 @@ This section contains printouts of `apify help` for all commands.
 * [`apify push [ACTORID]`](#apify-push-actorid)
 * [`apify run`](#apify-run)
 * [`apify secrets`](#apify-secrets)
-* [`apify secrets:add NAME VALUE`](#apify-secretsadd-name-value)
-* [`apify secrets:rm NAME`](#apify-secretsrm-name)
+* [`apify secrets add NAME VALUE`](#apify-secrets-add-name-value)
+* [`apify secrets rm NAME`](#apify-secrets-rm-name)
 * [`apify vis [PATH]`](#apify-vis-path)
 
 ## `apify actor`
@@ -307,29 +307,29 @@ DESCRIPTION
   Commands are designed to be used in Actor runs. All commands are in PoC state, do not use in production environments.
 ```
 
-_See code: [src/commands/actor/index.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/actor/index.ts)_
+_See code: [src/commands/actor/index.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/actor/index.ts)_
 
-## `apify actor:get-input`
+## `apify actor get-input`
 
 Gets the Actor input value from the default key-value store associated with the Actor run.
 
 ```
 USAGE
-  $ apify actor:get-input
+  $ apify actor get-input
 
 DESCRIPTION
   Gets the Actor input value from the default key-value store associated with the Actor run.
 ```
 
-_See code: [src/commands/actor/get-input.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/actor/get-input.ts)_
+_See code: [src/commands/actor/get-input.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/actor/get-input.ts)_
 
-## `apify actor:get-value KEY`
+## `apify actor get-value KEY`
 
 Gets a value from the default key-value store associated with the Actor run.
 
 ```
 USAGE
-  $ apify actor:get-value KEY
+  $ apify actor get-value KEY
 
 ARGUMENTS
   KEY  Key of the record in key-value store
@@ -338,15 +338,15 @@ DESCRIPTION
   Gets a value from the default key-value store associated with the Actor run.
 ```
 
-_See code: [src/commands/actor/get-value.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/actor/get-value.ts)_
+_See code: [src/commands/actor/get-value.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/actor/get-value.ts)_
 
-## `apify actor:push-data [ITEM]`
+## `apify actor push-data [ITEM]`
 
 Stores an object or an array of objects to the default dataset of the Actor run.
 
 ```
 USAGE
-  $ apify actor:push-data [ITEM]
+  $ apify actor push-data [ITEM]
 
 ARGUMENTS
   ITEM  JSON string with one object or array of objects containing data to be stored in the default dataset.
@@ -360,15 +360,15 @@ DESCRIPTION
   $ cat ./test.json | apify actor:push-data
 ```
 
-_See code: [src/commands/actor/push-data.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/actor/push-data.ts)_
+_See code: [src/commands/actor/push-data.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/actor/push-data.ts)_
 
-## `apify actor:set-value KEY [VALUE]`
+## `apify actor set-value KEY [VALUE]`
 
 Sets or removes record into the default KeyValueStore associated with the Actor run.
 
 ```
 USAGE
-  $ apify actor:set-value KEY [VALUE] [-c <value>]
+  $ apify actor set-value KEY [VALUE] [-c <value>]
 
 ARGUMENTS
   KEY    Key of the record in key-value store.
@@ -389,7 +389,7 @@ DESCRIPTION
   $ cat ./my-text-file.txt | apify actor:set-value KEY --contentType text/plain
 ```
 
-_See code: [src/commands/actor/set-value.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/actor/set-value.ts)_
+_See code: [src/commands/actor/set-value.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/actor/set-value.ts)_
 
 ## `apify call [ACTORID]`
 
@@ -415,7 +415,7 @@ DESCRIPTION
   takes input for the Actor from the default local key-value store by default.
 ```
 
-_See code: [src/commands/call.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/call.ts)_
+_See code: [src/commands/call.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/call.ts)_
 
 ## `apify create [ACTORNAME]`
 
@@ -423,7 +423,7 @@ Creates a new Actor project directory from a selected boilerplate template.
 
 ```
 USAGE
-  $ apify create [ACTORNAME] [-t <value>] [--skip-dependency-install]
+  $ apify create [ACTORNAME] [-t <value>] [--skip-dependency-install] [--omit-optional-deps]
 
 ARGUMENTS
   ACTORNAME  Name of the Actor and its directory
@@ -433,24 +433,25 @@ FLAGS
                                  Visit
                                  https://raw.githubusercontent.com/apify/actor-templates/master/templates/manifest.json
                                  to find available template names.
+      --omit-optional-deps       Skip installing optional dependencies.
       --skip-dependency-install  Skip installing Actor dependencies.
 
 DESCRIPTION
   Creates a new Actor project directory from a selected boilerplate template.
 ```
 
-_See code: [src/commands/create.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/create.ts)_
+_See code: [src/commands/create.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/create.ts)_
 
-## `apify help [COMMANDS]`
+## `apify help [COMMAND]`
 
 Display help for apify.
 
 ```
 USAGE
-  $ apify help [COMMANDS] [-n]
+  $ apify help [COMMAND...] [-n]
 
 ARGUMENTS
-  COMMANDS  Command to show help for.
+  COMMAND...  Command to show help for.
 
 FLAGS
   -n, --nested-commands  Include all nested commands in the output.
@@ -459,7 +460,7 @@ DESCRIPTION
   Display help for apify.
 ```
 
-_See code: [@oclif/plugin-help](https://github.com/oclif/plugin-help/blob/v6.0.14/src/commands/help.ts)_
+_See code: [@oclif/plugin-help](https://github.com/oclif/plugin-help/blob/v6.0.20/src/commands/help.ts)_
 
 ## `apify info`
 
@@ -474,7 +475,7 @@ DESCRIPTION
   The information is printed to the console.
 ```
 
-_See code: [src/commands/info.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/info.ts)_
+_See code: [src/commands/info.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/info.ts)_
 
 ## `apify init [ACTORNAME]`
 
@@ -502,26 +503,28 @@ DESCRIPTION
   WARNING: The directory at "storage" will be overwritten if it already exists.
 ```
 
-_See code: [src/commands/init.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/init.ts)_
+_See code: [src/commands/init.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/init.ts)_
 
 ## `apify login`
 
-Logs in to your Apify account using a provided API token.
+Logs in to your Apify account.
 
 ```
 USAGE
-  $ apify login [-t <value>]
+  $ apify login [-t <value>] [-m console|manual]
 
 FLAGS
-  -t, --token=<value>  [Optional] Apify API token
+  -m, --method=<option>  [Optional] Method of logging in to Apify
+                         <options: console|manual>
+  -t, --token=<value>    [Optional] Apify API token
 
 DESCRIPTION
-  Logs in to your Apify account using a provided API token.
+  Logs in to your Apify account.
   The API token and other account information is stored in the ~/.apify directory, from where it is read by all other
   "apify" commands. To log out, call "apify logout".
 ```
 
-_See code: [src/commands/login.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/login.ts)_
+_See code: [src/commands/login.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/login.ts)_
 
 ## `apify logout`
 
@@ -537,7 +540,7 @@ DESCRIPTION
   call "apify login".
 ```
 
-_See code: [src/commands/logout.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/logout.ts)_
+_See code: [src/commands/logout.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/logout.ts)_
 
 ## `apify pull [ACTORID]`
 
@@ -559,7 +562,7 @@ DESCRIPTION
   cloned. If it is defined as Web IDE, it will fetch the files.
 ```
 
-_See code: [src/commands/pull.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/pull.ts)_
+_See code: [src/commands/pull.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/pull.ts)_
 
 ## `apify push [ACTORID]`
 
@@ -568,6 +571,7 @@ Uploads the Actor to the Apify platform and builds it there.
 ```
 USAGE
   $ apify push [ACTORID] [--version-number <value>] [-v <value>] [-b <value>] [-w <value>] [--no-prompt]
+    [--force]
 
 ARGUMENTS
   ACTORID  Name or ID of the Actor to push (e.g. "apify/hello-world" or "E2jjCZBezvAZnX8Rb"). If not provided, the
@@ -579,6 +583,7 @@ FLAGS
   -v, --version=<value>          Actor version number to which the files should be pushed. By default, it is taken from
                                  the ".actor/actor.json" file.
   -w, --wait-for-finish=<value>  Seconds for waiting to build to finish, if no value passed, it waits forever.
+      --force                    Push an Actor even when the local files are older than the Actor on the platform.
       --no-prompt                Do not prompt for opening the Actor details in a browser. This will also not open the
                                  browser automatically.
       --version-number=<value>   DEPRECATED: Use flag version instead. Actor version number to which the files should be
@@ -591,10 +596,11 @@ DESCRIPTION
   NOTE: If the source files are smaller than 3 MB then they are uploaded as
   "Multiple source files", otherwise they are uploaded as "Zip file".
 
-  WARNING: If the target Actor already exists in your Apify account, it will be overwritten!
+  When there's an attempt to push files that are older than the Actor on the platform, the command will fail. Can be
+  overwritten with --force flag.
 ```
 
-_See code: [src/commands/push.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/push.ts)_
+_See code: [src/commands/push.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/push.ts)_
 
 ## `apify run`
 
@@ -622,7 +628,7 @@ DESCRIPTION
   package.json file. You can set up your own main file or environment variables by changing it.
 ```
 
-_See code: [src/commands/run.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/run.ts)_
+_See code: [src/commands/run.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/run.ts)_
 
 ## `apify secrets`
 
@@ -636,7 +642,7 @@ DESCRIPTION
   Manages secret values for Actor environment variables.
 
   Example:
-  $ apify secrets:add mySecret TopSecretValue123
+  $ apify secrets add mySecret TopSecretValue123
 
   Now the "mySecret" value can be used in an environment variable defined in ".actor/actor.json" file by adding the "@"
   prefix:
@@ -652,15 +658,15 @@ DESCRIPTION
   of the Actor.
 ```
 
-_See code: [src/commands/secrets/index.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/secrets/index.ts)_
+_See code: [src/commands/secrets/index.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/secrets/index.ts)_
 
-## `apify secrets:add NAME VALUE`
+## `apify secrets add NAME VALUE`
 
 Adds a new secret value.
 
 ```
 USAGE
-  $ apify secrets:add NAME VALUE
+  $ apify secrets add NAME VALUE
 
 ARGUMENTS
   NAME   Name of the secret
@@ -671,15 +677,15 @@ DESCRIPTION
   The secrets are stored to a file at ~/.apify
 ```
 
-_See code: [src/commands/secrets/add.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/secrets/add.ts)_
+_See code: [src/commands/secrets/add.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/secrets/add.ts)_
 
-## `apify secrets:rm NAME`
+## `apify secrets rm NAME`
 
 Removes the secret.
 
 ```
 USAGE
-  $ apify secrets:rm NAME
+  $ apify secrets rm NAME
 
 ARGUMENTS
   NAME  Name of the secret
@@ -688,7 +694,7 @@ DESCRIPTION
   Removes the secret.
 ```
 
-_See code: [src/commands/secrets/rm.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/secrets/rm.ts)_
+_See code: [src/commands/secrets/rm.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/secrets/rm.ts)_
 
 ## `apify vis [PATH]`
 
@@ -713,5 +719,5 @@ DESCRIPTION
   You can also pass any custom path to your input schema to have it validated instead.
 ```
 
-_See code: [src/commands/vis.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.3/src/commands/vis.ts)_
+_See code: [src/commands/vis.ts](https://github.com/apify/apify-cli/blob/v1.0.0-beta.31/src/commands/vis.ts)_
 <!-- commandsstop -->

package/bin/dev.js

@@ -1,9 +1,14 @@
-#!/usr/bin/env node_modules/.bin/tsx
-
 import { execute } from '@oclif/core';
+import { satisfies } from 'semver';
 
+import { SUPPORTED_NODEJS_VERSION } from '../src/lib/consts.ts';
 import { error } from '../src/lib/outputs.ts';
 
+if (!satisfies(process.version, SUPPORTED_NODEJS_VERSION)) {
+    error(`Apify CLI requires Node.js version ${SUPPORTED_NODEJS_VERSION}. Your current version is ${process.version}.`);
+    process.exit(1);
+}
+
 try {
     await execute({ development: true, dir: import.meta.url });
 } catch (err) {

package/bin/dev.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+"${DIR}"/../node_modules/.bin/tsx "${DIR}"/dev.js "$@"

package/bin/run.js

@@ -1,9 +1,16 @@
 #!/usr/bin/env node
 
 import { execute } from '@oclif/core';
+import { satisfies } from 'semver';
 
+import { SUPPORTED_NODEJS_VERSION } from '../dist/lib/consts.js';
 import { error } from '../dist/lib/outputs.js';
 
+if (!satisfies(process.version, SUPPORTED_NODEJS_VERSION)) {
+    error(`Apify CLI requires Node.js version ${SUPPORTED_NODEJS_VERSION}. Your current version is ${process.version}.`);
+    process.exit(1);
+}
+
 try {
     await execute({ development: false, dir: import.meta.url });
 } catch (err) {