@openfn/cli 0.0.20 → 0.0.22

package/README.md

@@ -1,69 +1,101 @@
-# @openfn/cli (devtools)
+# @openfn/cli
 
-This package contains a new devtools CLI.
+This package contains a new devtools CLI for running openfn jobs.
 
-The CLI allows you to
+The new CLI includes:
 
-- Run a job (expression), writing output to disk or stdout
-- Compile a job
-- Install modules for jobs
-- Use local language adaptors to run the job
+* A new runtime for executing openfn jobs
+* A new compiler for making openfn jobs runnable
+* Improved, customisable logging output
+* Auto installation of language adaptors
+* Support for the adaptors monorepo
 
-The CLI reads a path as its main argument. That path should either point to a .js file or a folder.
+## Getting Started
 
-- If the path ends in .js, load as a job file and execute. State and output will be read/written relative to it.
-- If the path is a folder, the CLI will look for a job.js, state.json and write an output.json.
+To install:
 
-From this input it will infer a working directory, from which state will be read and output will be written.
-
-All inputs and outputs can be configured with arguments.
+```
+npm install -g @openfn/cli
+```
 
-Run `pnpm openfn -h` to print usage help (the best source of truth right now).
+Make sure everything works by running the built-in test job:
 
-## Usage from the global CLI
+```
+openfn test
+```
 
-See Getting Started for more setup steps.
+Check the version:
 
 ```
-$ npm install -g @openfn/cli`
-$ openfn --help
-$ openfn path/to/expression.js`
+openfn -v
 ```
 
-## Language Adaptors
+Get help:
 
-The old engine used to compile knowlede of langauge adaptors into the job file. We no longer do this.
+```
+openfn help
+```
 
-Generally the new runtime prefers to explictly import all dependencies - although the compiler will auto-insert imports from a given adaptor for you.
+## Basic Usage
 
-You need to pass the name of an adaptor for the runtime to use. It will auto-insert an import statement for you.
+You're probably here to run jobs (expressions), which the CLI makes easy:
 
 ```
-$ openfn job.js -a commmon
+openfn path/to/job.js -ia adaptor-name
 ```
 
-You can use a short-hand name, like above, but longform names also work:
-```
-$ openfn job.js -a @openfn/language-commmon
-```
+You MUST specify which adaptor to use. Pass the `-i` flag to auto-install that adaptor (it's safe to do this redundantly).
 
-You can pass an explicit version number too:
+If output.json and state.json are not passed, the CLI will look for them next to the job.js file. You can pass a path to state by adding `-s path/to/state.json`, and output by passing `-o path/to/output.json`. You can use `-S` and `-O` to pass state through stdin and return the output through stdout.
 
-```
-$ openfn job.js -a commmon@1.7.3
-```
+The CLI can auto-install language adaptors to its own privately maintained repo. Run `openfn repo list` to see where the repo is, and what's in it. Set the `OPENFN_REPO_DIR` env var to specify the repo folder. When autoinstalling, the CLI will check to see if a matching version is found in the repo.
 
-The adaptor also needs to be installed in the CLI's module repo. You can do this manually:
+You can specify adaptors with a shorthand (`http`) or use the full package name (`@openfn/language-http`). You can add a specific version like `http@2.0.0`. You can pass a path to a locally installed adaptor like `http=/repo/openfn/adaptors/my-http-build`. Set the OPENFN_ADAPTORS_REPO env var to load adaptors straight out of the monorepo (pass the `--no-adaptors-repo` flag to disable this for a single run).
 
-```
-$ openfn install commmon
-```
-If no version is provided, the latest will be installed. Again, long and short-form names can be used.
+You can pass `--log info` to get more feedback about what's happening, or `--log debug` for more details than you could ever use.
 
-Alternatively, pass the -i flag when running a job (it's safe to do this redundantly):
-```
-$ openfn job.js -i -a @openfn/language-commmon
-```
+## Advanced Usage
+
+The CLI has actually has a number of commands (the first argument after openfn)
+
+* execute - run a job
+* compile - compile a job to a .js file
+* doc - show documentation for an adaptor function
+* repo - manage the repo of installed modules
+* docgen - generate JSON documentation for an adaptor based on its typescript
+
+If no command is specified, execute will run.
+
+To get more information about a command, including usage examples, run `openfn <command> help`, ie, `openfn compile help`.
+
+## Compilation
+
+The CLI will attempt to compile your job code into normalized Javascript. It will do a number of things to make your code robust and portable:
+
+* The language adaptor will be imported into the file
+* The adaptor's execute function will be exported form the file
+* All top level operations will be added to an array
+* That array will be made the default export of the file
+
+The result of this is a lightweight, modern JS source file. It can be executed in any runtime environment: just execute each function in the exported array.
+
+The CLI uses openfn's own runtime to execute jobs in a safe environment.
+
+All jobs which work against `@openfn/core` will work in the new CLI and runtime environment (note: although this is a work in progress and we are actively looking for help to test this!).
+
+## New Runtime notes
+
+The new openfunction runtime basically does one thing: load a Javascript Module, find the default export, and execute the functions it holds.
+
+So long as your job has an array of functions as its default export, it will run in the new runtime.
+
+# Contributing
+
+First of all, thanks for helping! You're contributing to a digital public good that will always be free and open source and aimed at serving innovative NGOs, governments, and social impact organizations the world over! You rock. heart
+
+To get this started, you'll want to clone this repo.
+
+You also need to install `pnpm`.
 
 ## Usage from this repo
 
@@ -78,7 +110,7 @@ See test/execute.test.ts for more usage examples
 
 ## Installing globally
 
-To install the CLI globally from this repo (ie, to do `openfn job.js` instead of `pnpm openfn job.js`), run:
+To install the CLI globally from the build in repo:
 
 ```
 $ npm install -g .
@@ -94,24 +126,15 @@ You should set the OPENFN_REPO_DIR env var to something sensible.
 
 ```
 # In ~/.bashc or whatever
-export OPENFN_REPO_DIR=~/adaptors/@openfn
+export OPENFN_REPO_DIR=~/repo/openfn/cli-repo
 ```
 
-At the time of writing, teh env var name is about to change. Soon you will be able to pass the repo dir into the command line, but the env var is a much easier way to work.
-
-Monorepo support is coming soon.
+To run adaptors straight from the adaptors monorepo:
 
-## Automatic Imports
+export OPENFN_ADAPTORS_REPO=~/repo/openfn/adaptors
 
-The v2 runtime requires explicit imports to be in the job file, or else the job will fail.
-
-The v2 compiler can automatically insert import statements, but it needs to be told which adaptor to use.
-
-```
-$ openfn job.js --adaptors @openfn/language-http
-$ openfn job.js --adaptors @openfn/language-http=path/to/adaptor
-```
+## Contributing changes
 
-If a path is passed (relative to the working directory), that path will be used to load a local version of the adaptor (both at runtime and for import generation)
+Open a PR at https://github.com/openfn/kit. Include a changeset and a description of your change.
 
-If no path is passed, the currently deployed npm package will be used.
+See the root readme for more details about changests,

package/dist/index.js

@@ -4,6 +4,7 @@
 import path from "node:path";
 import * as url from "url";
 import { fork } from "node:child_process";
+import process2 from "node:process";
 function spawn_default(basePath, opts2) {
   const execArgv = [
     "--no-warnings",
@@ -12,15 +13,18 @@ function spawn_default(basePath, opts2) {
   ];
   const dirname = path.dirname(url.fileURLToPath(import.meta.url));
   const child = fork(`${dirname}/process/runner.js`, [], { execArgv });
-  child.on("message", ({ done, init }) => {
+  child.on("message", ({ done, init, exitCode }) => {
     if (init) {
       child.send({ init: true, basePath, opts: opts2 });
     }
     if (done) {
       child.kill();
-      process.exit(0);
+      process2.exit(exitCode);
     }
   });
+  child.on("close", (code) => {
+    process2.exitCode = code;
+  });
 }
 
 // src/cli.ts

package/dist/process/runner.js

@@ -384,10 +384,17 @@ var executeHandler = async (options, logger) => {
   }
   const state = await load_state_default(options, logger);
   const code = await compile_default(options, logger);
-  const result = await execute_default(code, state, options);
-  await serialize_output_default(options, result, logger);
-  const duration = printDuration(new Date().getTime() - start);
-  logger.success(`Done in ${duration}! \u2728`);
+  try {
+    const result = await execute_default(code, state, options);
+    await serialize_output_default(options, result, logger);
+    const duration = printDuration(new Date().getTime() - start);
+    logger.success(`Done in ${duration}! \u2728`);
+  } catch (error) {
+    logger.error(error);
+    const duration = printDuration(new Date().getTime() - start);
+    logger.error(`Took ${duration}.`);
+    process.exitCode = 1;
+  }
 };
 var handler_default = executeHandler;
 
@@ -625,7 +632,7 @@ var assertPath = (basePath) => {
 process.on("message", ({ init, basePath, opts }) => {
   if (init) {
     commands_default(basePath, opts).then(() => {
-      process.send({ done: true });
+      process.send({ done: true, exitCode: process.exitCode });
     });
   }
 });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@openfn/cli",
-  "version": "0.0.20",
+  "version": "0.0.22",
   "description": "CLI devtools for the openfn toolchain.",
   "engines": {
     "node": ">=16",
@@ -36,10 +36,10 @@
     "typescript": "^4.7.4"
   },
   "dependencies": {
-    "@openfn/compiler": "^0.0.19",
-    "@openfn/describe-package": "^0.0.11",
-    "@openfn/logger": "^0.0.8",
-    "@openfn/runtime": "^0.0.13",
+    "@openfn/compiler": "0.0.20",
+    "@openfn/describe-package": "0.0.13",
+    "@openfn/logger": "0.0.8",
+    "@openfn/runtime": "0.0.13",
     "fast-safe-stringify": "^2.1.1",
     "rimraf": "^3.0.2",
     "treeify": "^1.1.0",