@openfn/cli 0.0.11 → 0.0.14

package/README.md

@@ -3,15 +3,16 @@
 This package contains a new devtools CLI.
 
 The CLI allows you to
-* Run a job (expression), writing output to disk or stdout
-* ~~Compile a job~~ (coming soon)
-* ~~Validate a job~~ (coming soon)
-* Use local language adaptors to run the job
+
+- Run a job (expression), writing output to disk or stdout
+- Compile a job
+- Install modules for jobs
+- Use local language adaptors to run the job
 
 The CLI reads a path as its main argument. That path should either point to a .js file or a folder.
 
-* If the path ends in .js, load as a job file and execute. State and output will be read/written relative to it.
-* If the path is a folder, the CLI will look for a job.js, state.json and write an output.json.
+- If the path ends in .js, load as a job file and execute. State and output will be read/written relative to it.
+- If the path is a folder, the CLI will look for a job.js, state.json and write an output.json.
 
 From this input it will infer a working directory, from which state will be read and output will be written.
 
@@ -29,15 +30,40 @@ $ openfn --help
 $ openfn path/to/expression.js`
 ```
 
-## legacy Jobs failing?
+## Language Adaptors
+
+The old engine used to compile knowlede of langauge adaptors into the job file. We no longer do this.
+
+Generally the new runtime prefers to explictly import all dependencies - although the compiler will auto-insert imports from a given adaptor for you.
 
-If legacy jobs ar failing because adaptor functions aren't found, you need to tell the CLI which adaptor to use. It will then auto-insert import statements for you.
+You need to pass the name of an adaptor for the runtime to use. It will auto-insert an import statement for you.
 
+```
+$ openfn job.js -a commmon
+```
+
+You can use a short-hand name, like above, but longform names also work:
 ```
 $ openfn job.js -a @openfn/language-commmon
 ```
 
-There's more detail in this further down in the readme.
+You can pass an explicit version number too:
+
+```
+$ openfn job.js -a commmon@1.7.3
+```
+
+The adaptor also needs to be installed in the CLI's module repo. You can do this manually:
+
+```
+$ openfn install commmon
+```
+If no version is provided, the latest will be installed. Again, long and short-form names can be used.
+
+Alternatively, pass the -i flag when running a job (it's safe to do this redundantly):
+```
+$ openfn job.js -i -a @openfn/language-commmon
+```
 
 ## Usage from this repo
 
@@ -60,36 +86,20 @@ $ npm install -g .
 
 Note that this will install the built source from `dist`
 
-## Current state
-
-For legacy jobs (ie, jobs without explicit imports), the new runtime is only compatible with language adaptors with type definitions.
-
-Right now, that means @openfn/language-common@2.0.0-rc3.
+## Repo Directory
 
-## Getting Started
+The CLI will save and load adaptors from an arbitrary folder on your system.
 
-Here's how I recommend getting set up:
-
-* Create a folder for next-gen language adaptors somewhere on your machine
-
-```
-$ mkdir -p ~/adaptors/@openfn
-```
-
-* Clone `language-common` into that folder
-
-```
-git clone https://github.com/OpenFn/language-common.git ~/adaptors/@openfn --branch 2.0.0-pre
-```
-
-* Set your `OPENFN_MODULES_HOME` environment variable to point to the next-gen adaptors folder. This will tell the CLI to load adaptors from this folder by default.
+You should set the OPENFN_REPO_DIR env var to something sensible.
 
 ```
 # In ~/.bashc or whatever
-export OPENFN_MODULES_HOME=~/adaptors/@openfn
+export OPENFN_REPO_DIR=~/adaptors/@openfn
 ```
 
-This will improve in future, as we implement automatic module loading and add type definitions to the published adaptor packages.
+At the time of writing, teh env var name is about to change. Soon you will be able to pass the repo dir into the command line, but the env var is a much easier way to work.
+
+Monorepo support is coming soon.
 
 ## Automatic Imports
 
@@ -105,21 +115,3 @@ $ openfn job.js --adaptors @openfn/language-http=path/to/adaptor
 If a path is passed (relative to the working directory), that path will be used to load a local version of the adaptor (both at runtime and for import generation)
 
 If no path is passed, the currently deployed npm package will be used.
-
-## Notes on Module Resolution
-
-Any import statements inside a job have to resolve to a node module.
-
-A module can be resolved:
-
-* Relative to the env var OPENFN_MODULE_HOME
-* Relative to CLI's node_modules
-* Relative to global node_modules
-
-Basically, to work with adaptors, you should:
-
-* Save your adaptors globally
-
-Or
-
-* Save adaptors to a folder somewhere (~/openfn) and set OPENFN_MODULE_HOME=~/openfn

package/dist/index.d.ts

@@ -1,2 +1 @@
 #!/usr/bin/env node
-export {};

package/dist/index.js

@@ -1,102 +1,179 @@
 #!/usr/bin/env node
-import path from 'node:path';
-import * as url from 'url';
-import { fork } from 'node:child_process';
-import yargs from 'yargs';
-import { hideBin } from 'yargs/helpers';
 
-/**
- * Utility to run CLI commands inside a child process
- * This lets us hide the neccessary arguments needed to run our devtools
- */
-// The default export will create a new child process which calls itself
-function runInChildProcess (basePath, opts) {
-    const execArgv = [
-        // Suppress experimental argument warnings
-        '--no-warnings',
-        // Allows us to load an ESM module from a text string
-        '--experimental-vm-modules',
-        // Allows us to do import('path/to/language-common') in the linker
-        '--experimental-specifier-resolution=node',
-    ];
-    const dirname = path.dirname(url.fileURLToPath(import.meta.url));
-    const child = fork(`${dirname}/process/runner.js`, [], { execArgv });
-    child.on('message', ({ done }) => {
-        if (done) {
-            child.kill();
-            process.exit(0);
-        }
-    });
-    child.send({ init: true, basePath, opts });
+// src/process/spawn.ts
+import path from "node:path";
+import * as url from "url";
+import { fork } from "node:child_process";
+function spawn_default(basePath, opts2) {
+  const execArgv = [
+    "--no-warnings",
+    "--experimental-vm-modules",
+    "--experimental-specifier-resolution=node"
+  ];
+  const dirname = path.dirname(url.fileURLToPath(import.meta.url));
+  const child = fork(`${dirname}/process/runner.js`, [], { execArgv });
+  child.on("message", ({ done }) => {
+    if (done) {
+      child.kill();
+      process.exit(0);
+    }
+  });
+  child.send({ init: true, basePath, opts: opts2 });
 }
 
-const cmd = yargs(hideBin(process.argv))
-    .command('[path]', "Run the job at the path")
-    .command('--test', "Run a trivial test job with no disk I/O")
-    .example('openfn path/to/dir', 'Looks for job.js, state.json in path/to/dir')
-    .example('openfn foo/job.js', 'Reads foo/job.js, looks for state and output in foo')
-    .example('openfn job.js -adaptor @openfn/language-common', 'Run job.js with automatic imports from the commmon language adaptor')
-    .example('openfn job.js -adaptor @openfn/language-common=repo/openfn/language-common', 'Run job.js with a local implementation of the common language adaptor')
-    .example('openfn foo/job.js -c', 'Compile a job to foo/output/js')
-    .example('openfn foo/job.js -cO', 'Compile a job to stdout')
-    .example('openfn foo/job.js --log debug', 'Run a job with debug-level logging')
-    .example('openfn foo/job.js --log compiler=debug', 'Use debug logging in the compiler only')
-    .positional('path', {
-    describe: 'The path to load the job from (a .js file or a dir containing a job.js file)',
-    demandOption: true
-})
-    .option('test', {
-    description: 'Run a test job to exercise the installation. Pass a number via -S to multiply by 2.',
-    boolean: true,
-})
-    .option('output-path', {
-    alias: 'o',
-    description: 'Path to the output file',
-})
-    .option('output-stdout', {
-    alias: 'O',
-    boolean: true,
-    description: 'Print output to stdout (intead of a file)',
-})
-    .option('state-path', {
-    alias: 's',
-    description: 'Path to the state file'
-})
-    .option('state-stdin', {
-    alias: 'S',
-    description: 'Read state from stdin (instead of a file)'
-})
-    .option('no-validation', {
-    boolean: true,
-    description: 'Skip validation'
-})
-    .option('compile-only', {
-    alias: 'c',
-    boolean: true,
-    description: 'Compile the job but don\'t execute it. State is written to output.js or returned to console if -O is set.'
-})
-    .option('no-compile', {
-    boolean: true,
-    description: 'Skip compilation'
-})
-    .option('adaptors', {
-    alias: ['a', 'adaptor'],
-    description: 'Pass one or more adaptors in the form name=path/to/adaptor',
-    array: true
-})
-    // TODO this becomes log compiler=debug
-    .option('trace-linker', {
-    alias: ['t', 'trace'],
-    description: 'Trace module resolution output in the linker',
-    boolean: true,
-})
-    .option('log', {
-    alias: ['l'],
-    description: 'Set the default log level to none, trace, info or default',
-    array: true
-})
-    .alias('v', 'version');
+// src/cli.ts
+import yargs from "yargs";
+import { hideBin } from "yargs/helpers";
 
-const opts = cmd.parse();
-runInChildProcess(opts._[0], opts);
-//# sourceMappingURL=index.js.map
+// src/repo/command.ts
+var repo = {
+  command: "repo [subcommand]",
+  desc: "Run commands on the module repo (install|clean)",
+  builder: (yargs2) => yargs2.command(clean).command(install).command(pwd).command(list).example("repo install -a http", "Install @openfn/language-http").example("repo clean", "Remove everything from the repo working dir").example("repo pwd", "Print the current repo working dir")
+};
+var install = {
+  command: "install [packages...]",
+  desc: "install one or more packages to the runtime repo",
+  handler: (argv) => {
+    argv.command = "repo-install";
+  },
+  builder: (yargs2) => {
+    return yargs2.option("adaptor", {
+      alias: ["a"],
+      description: "Install an adaptor by passing a shortened version of the name",
+      boolean: true
+    }).example("install axios", "Install the axios npm package to the repo").example(
+      "install -a http",
+      "Install @openfn/language-http adaptor to the repo"
+    ).example(
+      "install @openfn/language-http",
+      "Install the language-http adaptor to the repo"
+    );
+  }
+};
+var clean = {
+  command: "clean",
+  desc: "Removes all modules from the runtime module repo",
+  handler: (argv) => {
+    argv.command = "repo-clean";
+  },
+  builder: (yargs2) => yargs2.option("force", {
+    alias: ["f"],
+    description: "Skip the prompt and force deletion",
+    boolean: true
+  })
+};
+var pwd = {
+  command: "pwd",
+  desc: "Print repo's current working directory",
+  handler: (argv) => {
+    argv.command = "repo-pwd";
+  }
+};
+var list = {
+  command: "list",
+  desc: "Show a report on what is installed in the repo",
+  handler: (argv) => {
+    argv.command = "repo-list";
+  }
+};
+
+// src/execute/command.ts
+var executeCommand = {
+  command: "execute [path]",
+  desc: "Run an openfn job",
+  aliases: ["$0"],
+  handler: (argv) => {
+    argv.command = "execute";
+  },
+  builder: (yargs2) => {
+    return applyExecuteOptions(yargs2).option("immutable", {
+      boolean: true,
+      description: "Treat state as immutable"
+    }).option("autoinstall", {
+      alias: "i",
+      boolean: true,
+      description: "Auto-install the language adaptor"
+    }).option("state-path", {
+      alias: "s",
+      description: "Path to the state file"
+    }).option("state-stdin", {
+      alias: "S",
+      description: "Read state from stdin (instead of a file)"
+    }).option("no-compile", {
+      boolean: true,
+      description: "Skip compilation"
+    }).example(
+      "openfn foo/job.js",
+      "Reads foo/job.js, looks for state and output in foo"
+    ).example(
+      "openfn job.js -a common",
+      "Run job.js using @openfn/language-common"
+    ).example(
+      "openfn install -a common",
+      "Install the latest version of language-common to the repo"
+    );
+  }
+};
+var applyExecuteOptions = (yargs2) => yargs2.positional("path", {
+  describe: "The path to load the job from (a .js file or a dir containing a job.js file)",
+  demandOption: true
+}).option("output-path", {
+  alias: "o",
+  description: "Path to the output file"
+}).option("output-stdout", {
+  alias: "O",
+  boolean: true,
+  description: "Print output to stdout (intead of a file)"
+}).option("adaptors", {
+  alias: ["a", "adaptor"],
+  description: "A language adaptor to use for the job. Short-form names are allowed. Can include an explicit path to a local adaptor build.",
+  array: true
+}).option("no-expand", {
+  description: "Don	 attempt to auto-expand adaptor shorthand names",
+  boolean: true
+});
+var command_default = executeCommand;
+
+// src/compile/command.ts
+var compileCommand = {
+  command: "compile [path]",
+  desc: "compile a openfn job and print or save the resulting js",
+  handler: (argv) => {
+    argv.command = "compile";
+  },
+  builder: (yargs2) => {
+    return applyExecuteOptions(yargs2).example(
+      "compile foo/job.js -O",
+      "Compiles foo/job.js and prints the result to stdout"
+    ).example(
+      "compile foo/job.js -o foo/job-compiled.js",
+      "Compiles foo/job.js and saves the result to foo/job-compiled.js"
+    );
+  }
+};
+var command_default2 = compileCommand;
+
+// src/test/command.ts
+var command_default3 = {
+  command: "test",
+  desc: "Compiles and runs a test job, printing the result to stdout",
+  handler: (argv) => {
+    argv.command = "test";
+  },
+  builder: (yargs2) => yargs2.option("state-stdin", {
+    alias: "S",
+    description: "Read state from stdin (instead of a file)"
+  }).example("test", "run the test script").example("test -S 42", "run the test script with state 42")
+};
+
+// src/cli.ts
+var cmd = yargs(hideBin(process.argv)).command(command_default).command(command_default2).command(install).command(repo).command(command_default3).option("log", {
+  alias: ["l"],
+  description: "Set the default log level to none, trace, info or default",
+  array: true
+}).alias("v", "version").help();
+
+// src/index.ts
+var opts = cmd.parse();
+spawn_default(opts.path, opts);

package/dist/process/runner.d.ts

@@ -1 +1 @@
-export {};
+