@openfn/cli 0.4.16 → 1.1.0

package/README.md

@@ -15,6 +15,7 @@ The CLI includes:
 
 - [Installation](#installation)
 - [Updating](#updating)
+- [Terminology](#terminology)
 - [Migrating from devtools](#migrating-from-devtools)
 - [Basic Usage](#basic-usage)
 - [Advanced Usage](#advanced-usage)
@@ -71,27 +72,36 @@ npm uninstall -g @openfn/cli
 
 And then re-installing.
 
-## Migrating from devtools
+## Terminology
 
-If you're coming to the CLI from the old openfn devtools, here are a couple of key points to be aware of:
+The CLI (and the wider OpenFn stack) has some very particular terminology
 
-- The CLI has a shorter, sleeker syntax, so your command should be much shorter
-- The CLI will automatically install adaptors for you (with full version control)
+- An **Expression** is a string of Javascript (or Javascript-like code) written to be run in the CLI or Lightning.
+- A **Job** is an expression plus some metadata required to run it - typically an adaptor and credentials.
+  The terms Job and Expression are often used interchangeably.
+- A **Workflow** is a series of steps to be executed in sequence. Steps are usually Jobs (and so job and step are often used
+  interchangeably), but can be Triggers.
+- An **Execution Plan** is a Workflow plus some options which inform how it should be executed (ie, start node, timeout).
+  The term "Execution plan" is mostly used internally and not exposed to users, and is usually interchangeable with Workflow.
+
+Note that an expression is not generally portable (ie, cannot run in other environments) unless it is compiled.
+A compiled expression has imports and exports and, so long as packages are available, can run in a simple
+JavaScript runtime.
 
 ## Basic Usage
 
-You're probably here to run jobs (expressions) or workflows, which the CLI makes easy:
+You're probably here to run Workflows (or individual jobs), which the CLI makes easy:
 
 ```
 openfn path/to/workflow.json
-openfn path/to/job.js -ia adaptor-name
+openfn path/to/job.js -a adaptor-name
 ```
 
 If running a single job, you MUST specify which adaptor to use.
 
-Pass the `-i` flag to auto-install any required adaptors (it's safe to do this redundantly, although the run will be a little slower).
+If the requested adaptor (or a matching version) is not already installed, it will be installed automatically. To disable this behaviour, pass the `--no-autoinstall` flag.
 
-When the finished, the CLI will write the resulting state to disk. By default the CLI will create an `output.json` next to the job file. You can pass a path to output by passing `-o path/to/output.json` and state by adding `-s path/to/state.json`. You can use `-S` and `-O` to pass state through stdin and return the output through stdout.
+When finished, the CLI will write the resulting state to disk. By default the CLI will create an `output.json` next to the job file. You can pass a path to output by passing `-o path/to/output.json` and state by adding `-s path/to/state.json`. You can use `-S` and `-O` to pass state through stdin and return the output through stdout.
 
 The CLI maintains a repo for auto-installed adaptors. Run `openfn repo list` to see where the repo is, and what's in it. Set the `OPENFN_REPO_DIR` env var to specify the repo folder. When autoinstalling, the CLI will check to see if a matching version is found in the repo. `openfn repo clean` will remove all adaptors from the repo. The repo also includes any documentation and metadata built with the CLI.
 
@@ -103,14 +113,16 @@ You can pass `--log info` to get more feedback about what's happening, or `--log
 
 ## Advanced Usage
 
-The CLI has a number of commands (the first argument after openfn)
+The CLI has a number of commands (the first argument after `openfn`):
 
 - execute - run a job
-- compile - compile a job to a .js file
+- compile - compile a job to a .js file (prints to stdout by default)
 - docs - show documentation for an adaptor function
 - repo - manage the repo of installed modules
 - docgen - generate JSON documentation for an adaptor based on its typescript
 
+For example, `openfn compile job.js -a common` will compile the code at `job.js` with the common adaptor.
+
 If no command is specified, execute will run.
 
 To get more information about a command, including usage examples, run `openfn <command> help`, ie, `openfn compile help`.
@@ -253,38 +265,43 @@ Pass `--log-json` to the CLI to do this. You can also set the OPENFN_LOG_JSON en
 
 ## Workflows
 
-As of v0.0.35 the CLI supports running workflows as well as jobs.
-
-A workflow is in execution plan for running several jobs in a sequence. It is defined as a JSON structure.
+A workflow is an execution plan for running several jobs in a sequence. It is defined as a JSON structure.
 
 To see an example workflow, run the test command with `openfn test`.
 
-A workflow has a structure like this (better documentation is coming soon):
+A workflow has a structure like this:
 
 ```json
 {
-  "start": "a", // optionally specify the start node (defaults to jobs[0])
-  "jobs": [
-    {
-      "id": "a",
-      "expression": "fn((state) => state)", // code or a path
-      "adaptor": "@openfn/language-common@1.75", // specifiy the adaptor to use (version optional)
-      "data": {}, // optionally pre-populate the data object (this will be overriden by keys in in previous state)
-      "configuration": {}, // Use this to pass credentials
-      "next": {
-        // This object defines which jobs to call next
-        // All edges returning true will run
-        // If there are no next edges, the workflow will end
-        "b": true,
-        "c": {
-          "condition": "!state.error" // Note that this is an expression, not a function
+  "workflow": {
+    "name": "my-workflow", // human readable name used in logging
+    "steps": [
+      {
+        "name": "a", // human readable name used in logging
+        "expression": "fn((state) => state)", // code or a path to an expression.js file
+        "adaptor": "@openfn/language-common@1.7.5", // specifiy the adaptor to use (version optional)
+        "data": {}, // optionally pre-populate the data object (this will be overriden by keys in in previous state)
+        "configuration": {}, // Use this to pass credentials
+        "next": {
+          // This object defines which jobs to call next
+          // All edges returning true will run
+          // If there are no next edges, the workflow will end
+          "b": true,
+          "c": {
+            "condition": "!state.error" // Note that this is a strict Javascript expression, not a function, and has no adaptor support
+          }
         }
       }
-    }
-  ]
+    ]
+  },
+  "options": {
+    "start": "a" // optionally specify the start node (defaults to steps[0])
+  }
 }
 ```
 
+See `packages/lexicon` for type definitions (the workflow format is covered by the `ExecutionPlan` type)/
+
 ## Compilation
 
 The CLI will compile your job code into regular Javascript. It does a number of things to make your code robust and portable:
@@ -298,8 +315,6 @@ The result of this is a lightweight, modern JS module. It can be executed in any
 
 The CLI uses openfn's own runtime to execute jobs in a safe environment.
 
-All jobs which work against `@openfn/core` will work in the new CLI and runtime environment (note: although this is a work in progress and we are actively looking for help to test this!).
-
 If you want to see how the compiler is changing your job, run `openfn compile path/to/job -a <adaptor>` to return the compiled code to stdout. Add `-o path/to/output.js` to save the result to disk.
 
 ## Contributing
@@ -355,10 +370,10 @@ export OPENFN_ADAPTORS_REPO=~/repo/openfn/adaptors
 
 ### Contributing changes
 
-Open a PR at https://github.com/openfn/kit. Include a changeset and a description of your change.
-
-See the root readme for more details about changests,
+Include a changeset and a description of your change. Run this command and follow the interactive prompt (it's really easy, promise!)
 
 ```
-
+pnpm changeset
 ```
+
+Commit the changeset files and open a PR at https://github.com/openfn/kit.

package/dist/index.js

@@ -49,19 +49,18 @@ var expand = (name) => {
   }
   return name;
 };
-var expand_adaptors_default = (opts2) => {
-  const { adaptors: adaptors2, workflow } = opts2;
-  if (adaptors2) {
-    opts2.adaptors = adaptors2?.map(expand);
-  }
-  if (workflow) {
-    Object.values(workflow.jobs).forEach((job) => {
-      if (job.adaptor) {
-        job.adaptor = expand(job.adaptor);
-      }
-    });
-  }
-  return opts2;
+var expand_adaptors_default = (input) => {
+  if (Array.isArray(input)) {
+    return input?.map(expand);
+  }
+  const plan = input;
+  Object.values(plan.workflow.steps).forEach((step) => {
+    const job = step;
+    if (job.adaptor) {
+      job.adaptor = expand(job.adaptor);
+    }
+  });
+  return plan;
 };
 
 // src/util/logger.ts
@@ -157,7 +156,7 @@ var adaptors = {
       opts2.adaptors = [];
     }
     if (opts2.expandAdaptors) {
-      expand_adaptors_default(opts2);
+      opts2.adaptors = expand_adaptors_default(opts2.adaptors);
     }
     delete opts2.adaptor;
     delete opts2.a;
@@ -168,8 +167,8 @@ var autoinstall = {
   yargs: {
     alias: ["i"],
     boolean: true,
-    description: "Auto-install the language adaptor",
-    default: false
+    description: "Auto-install the language adaptor(s)",
+    default: true
   }
 };
 var compile = {
@@ -263,12 +262,12 @@ var inputPath = {
   ensure: (opts2) => {
     const { path: basePath } = opts2;
     if (basePath?.endsWith(".json")) {
-      opts2.workflowPath = basePath;
+      opts2.planPath = basePath;
     } else if (basePath?.endsWith(".js")) {
-      opts2.jobPath = basePath;
+      opts2.expressionPath = basePath;
     } else {
       const base = getBaseDir(opts2);
-      setDefaultValue(opts2, "jobPath", path2.join(base, "job.js"));
+      setDefaultValue(opts2, "expressionPath", path2.join(base, "job.js"));
     }
   }
 };
@@ -345,33 +344,6 @@ var start = {
     description: "Specifiy the start node in a workflow"
   }
 };
-var strictOutput = {
-  name: "no-strict-output",
-  yargs: {
-    deprecated: true,
-    hidden: true,
-    boolean: true
-  },
-  ensure: (opts2) => {
-    if (!opts2.hasOwnProperty("strict")) {
-      opts2.strict = opts2.strictOutput;
-    }
-    delete opts2.strictOutput;
-  }
-};
-var strict = {
-  name: "strict",
-  yargs: {
-    default: false,
-    boolean: true,
-    description: "Enables strict state handling, meaning only state.data is returned from a job."
-  },
-  ensure: (opts2) => {
-    if (!opts2.hasOwnProperty("strictOutput")) {
-      setDefaultValue(opts2, "strict", false);
-    }
-  }
-};
 var skipAdaptorValidation = {
   name: "skip-adaptor-validation",
   yargs: {
@@ -573,18 +545,14 @@ var options4 = [
   start,
   statePath,
   stateStdin,
-  strict,
-  strictOutput,
   timeout,
   useAdaptorsMonorepo
 ];
 var executeCommand = {
   command: "execute [path]",
-  describe: `Run an openfn job or workflow. Get more help by running openfn <command> help.
-  
-Execute will run a job/workflow at the path and write the output state to disk (to ./state.json unless otherwise specified)
+  describe: `Run an openfn expression or workflow. Get more help by running openfn <command> help.
   
-By default only state.data will be returned fron a job. Include --no-strict to write the entire state object.
+Execute will run a expression/workflow at the path and write the output state to disk (to ./state.json unless otherwise specified)
   
 Remember to include the adaptor name with -a. Auto install adaptors with the -i flag.`,
   aliases: ["$0"],
@@ -603,7 +571,7 @@ Remember to include the adaptor name with -a. Auto install adaptors with the -i
     "Execute job.js with common adaptor and info-level logging"
   ).example(
     "openfn compile job.js -a http",
-    "Compile job.js with the http adaptor and print the code to stdout"
+    "Compile the expression at job.js with the http adaptor and print the code to stdout"
   )
 };
 var command_default5 = executeCommand;

package/dist/process/runner.js

@@ -32,18 +32,15 @@ var logger_default = createLogger;
 var createNullLogger = () => createLogger(void 0, { log: { default: "none" } });
 
 // src/execute/execute.ts
-var execute_default = async (input, state, opts) => {
+var execute_default = async (plan, input, opts) => {
   try {
-    const result = await run(input, state, {
-      strict: opts.strict,
-      start: opts.start,
-      timeout: opts.timeout,
+    const result = await run(plan, input, {
       immutableState: opts.immutable,
       logger: logger_default(RUNTIME, opts),
       jobLogger: logger_default(JOB, opts),
       linker: {
         repo: opts.repoDir,
-        modules: parseAdaptors(opts)
+        modules: parseAdaptors(plan)
       }
     });
     return result;
@@ -52,7 +49,7 @@ var execute_default = async (input, state, opts) => {
     throw e;
   }
 };
-function parseAdaptors(opts) {
+function parseAdaptors(plan) {
   const extractInfo = (specifier) => {
     const [module, path7] = specifier.split("=");
     const { name, version } = getNameAndVersion(module);
@@ -68,21 +65,13 @@ function parseAdaptors(opts) {
     return info;
   };
   const adaptors = {};
-  if (opts.adaptors) {
-    opts.adaptors.reduce((obj, exp) => {
-      const { name, ...maybeVersionAndPath } = extractInfo(exp);
-      obj[name] = { ...maybeVersionAndPath };
-      return obj;
-    }, adaptors);
-  }
-  if (opts.workflow) {
-    Object.values(opts.workflow.jobs).forEach((job) => {
-      if (job.adaptor) {
-        const { name, ...maybeVersionAndPath } = extractInfo(job.adaptor);
-        adaptors[name] = { ...maybeVersionAndPath };
-      }
-    });
-  }
+  Object.values(plan.workflow.steps).forEach((step) => {
+    const job = step;
+    if (job.adaptor) {
+      const { name, ...maybeVersionAndPath } = extractInfo(job.adaptor);
+      adaptors[name] = maybeVersionAndPath;
+    }
+  });
   return adaptors;
 }
 
@@ -91,15 +80,8 @@ import { writeFile } from "node:fs/promises";
 var serializeOutput = async (options, result, logger) => {
   let output = result;
   if (output && (output.configuration || output.data)) {
-    if (options.strict) {
-      output = { data: output.data };
-      if (result.errors) {
-        output.errors = result.errors;
-      }
-    } else {
-      const { configuration, ...rest } = result;
-      output = rest;
-    }
+    const { configuration, ...rest } = result;
+    output = rest;
   }
   if (output === void 0) {
     output = "";
@@ -119,20 +101,15 @@ var serializeOutput = async (options, result, logger) => {
 var serialize_output_default = serializeOutput;
 
 // src/execute/get-autoinstall-targets.ts
-var getAutoinstallTargets = (options) => {
-  if (options.workflow) {
-    const adaptors = {};
-    Object.values(options.workflow.jobs).forEach((job) => {
-      if (job.adaptor) {
-        adaptors[job.adaptor] = true;
-      }
-    });
-    return Object.keys(adaptors);
-  }
-  if (options.adaptors) {
-    return options.adaptors?.filter((a) => !/=/.test(a));
-  }
-  return [];
+var getAutoinstallTargets = (plan) => {
+  const adaptors = {};
+  Object.values(plan.workflow.steps).forEach((step) => {
+    const job = step;
+    if (job.adaptor && !/=/.test(job.adaptor)) {
+      adaptors[job.adaptor] = true;
+    }
+  });
+  return Object.keys(adaptors);
 };
 var get_autoinstall_targets_default = getAutoinstallTargets;
 
@@ -232,20 +209,15 @@ var abort_default = (logger, reason, error, help) => {
 };
 
 // src/compile/compile.ts
-var compile_default = async (opts, log) => {
-  log.debug("Compiling...");
-  let job;
-  if (opts.workflow) {
-    job = compileWorkflow(opts.workflow, opts, log);
-  } else {
-    job = await compileJob(opts.job || opts.jobPath, opts, log);
-  }
-  if (opts.jobPath) {
-    log.success(`Compiled from ${opts.jobPath}`);
-  } else {
-    log.success("Compilation complete");
+var compile_default = async (planOrPath, opts, log) => {
+  if (typeof planOrPath === "string") {
+    const result = await compileJob(planOrPath, opts, log);
+    log.success(`Compiled expression from ${opts.expressionPath}`);
+    return result;
   }
-  return job;
+  const compiledPlan = compileWorkflow(planOrPath, opts, log);
+  log.success("Compiled all expressions in workflow");
+  return compiledPlan;
 };
 var compileJob = async (job, opts, log, jobName) => {
   try {
@@ -258,10 +230,12 @@ var compileJob = async (job, opts, log, jobName) => {
       e,
       "Check the syntax of the job expression:\n\n" + job
     );
+    return "";
   }
 };
-var compileWorkflow = async (workflow, opts, log) => {
-  for (const job of workflow.jobs) {
+var compileWorkflow = async (plan, opts, log) => {
+  for (const step of plan.workflow.steps) {
+    const job = step;
     const jobOpts = {
       ...opts
     };
@@ -277,7 +251,7 @@ var compileWorkflow = async (workflow, opts, log) => {
       );
     }
   }
-  return workflow;
+  return plan;
 };
 var stripVersionSpecifier = (specifier) => {
   const idx = specifier.lastIndexOf("@");
@@ -379,15 +353,16 @@ var validateAdaptors = async (options, logger) => {
   if (options.skipAdaptorValidation) {
     return;
   }
+  const isPlan = options.planPath || options.workflowPath;
   const hasDeclaredAdaptors = options.adaptors && options.adaptors.length > 0;
-  if (options.workflowPath && hasDeclaredAdaptors) {
+  if (isPlan && hasDeclaredAdaptors) {
     logger.error("ERROR: adaptor and workflow provided");
     logger.error(
       "This is probably not what you meant to do. A workflow should declare an adaptor for each job."
     );
     throw new Error("adaptor and workflow provided");
   }
-  if (!options.workflowPath && !hasDeclaredAdaptors) {
+  if (!isPlan && !hasDeclaredAdaptors) {
     logger.warn("WARNING: No adaptor provided!");
     logger.warn(
       "This job will probably fail. Pass an adaptor with the -a flag, eg:"
@@ -399,118 +374,10 @@ var validateAdaptors = async (options, logger) => {
 };
 var validate_adaptors_default = validateAdaptors;
 
-// src/util/load-input.ts
-import path from "node:path";
+// src/util/load-plan.ts
 import fs2 from "node:fs/promises";
+import path2 from "node:path";
 import { isPath } from "@openfn/compiler";
-var load_input_default = async (opts, log) => {
-  const { job, workflow, jobPath, workflowPath } = opts;
-  if (workflow || workflowPath) {
-    return loadWorkflow(opts, log);
-  }
-  if (job) {
-    return job;
-  }
-  if (jobPath) {
-    try {
-      log.debug(`Loading job from ${jobPath}`);
-      opts.job = await fs2.readFile(jobPath, "utf8");
-      return opts.job;
-    } catch (e) {
-      abort_default(
-        log,
-        "Job not found",
-        void 0,
-        `Failed to load the job from ${jobPath}`
-      );
-    }
-  }
-};
-var loadWorkflow = async (opts, log) => {
-  const { workflowPath, workflow } = opts;
-  const readWorkflow = async () => {
-    try {
-      const text = await fs2.readFile(workflowPath, "utf8");
-      return text;
-    } catch (e) {
-      abort_default(
-        log,
-        "Workflow not found",
-        void 0,
-        `Failed to load a workflow from ${workflowPath}`
-      );
-    }
-  };
-  const parseWorkflow = (contents) => {
-    try {
-      return JSON.parse(contents);
-    } catch (e) {
-      abort_default(
-        log,
-        "Invalid JSON in workflow",
-        e,
-        `Check the syntax of the JSON at ${workflowPath}`
-      );
-    }
-  };
-  const fetchWorkflowFile = async (jobId, rootDir = "", filePath) => {
-    try {
-      const fullPath = filePath.startsWith("~") ? filePath : path.resolve(rootDir, filePath);
-      const result = await fs2.readFile(fullPath, "utf8");
-      return result;
-    } catch (e) {
-      abort_default(
-        log,
-        `File not found for job ${jobId}: ${filePath}`,
-        void 0,
-        `This workflow references a file which cannot be found at ${filePath}
-
-Paths inside the workflow are relative to the workflow.json`
-      );
-    }
-  };
-  log.debug(`Loading workflow from ${workflowPath}`);
-  try {
-    let wf;
-    let rootDir = opts.baseDir;
-    if (workflowPath) {
-      let workflowRaw = await readWorkflow();
-      wf = parseWorkflow(workflowRaw);
-      if (!rootDir) {
-        rootDir = path.dirname(workflowPath);
-      }
-    } else {
-      wf = workflow;
-    }
-    let idx = 0;
-    for (const job of wf.jobs) {
-      idx += 1;
-      const expressionStr = typeof job.expression === "string" && job.expression?.trim();
-      const configurationStr = typeof job.configuration === "string" && job.configuration?.trim();
-      if (expressionStr && isPath(expressionStr)) {
-        job.expression = await fetchWorkflowFile(
-          job.id || `${idx}`,
-          rootDir,
-          expressionStr
-        );
-      }
-      if (configurationStr && isPath(configurationStr)) {
-        const configString = await fetchWorkflowFile(
-          job.id || `${idx}`,
-          rootDir,
-          configurationStr
-        );
-        job.configuration = JSON.parse(configString);
-      }
-    }
-    opts.workflow = wf;
-    log.debug("Workflow loaded!");
-    return opts.workflow;
-  } catch (e) {
-    log.error(`Error loading workflow from ${workflowPath}`);
-    throw e;
-  }
-};
 
 // src/util/expand-adaptors.ts
 var expand = (name) => {
@@ -523,24 +390,23 @@ var expand = (name) => {
   }
   return name;
 };
-var expand_adaptors_default = (opts) => {
-  const { adaptors, workflow } = opts;
-  if (adaptors) {
-    opts.adaptors = adaptors?.map(expand);
+var expand_adaptors_default = (input) => {
+  if (Array.isArray(input)) {
+    return input?.map(expand);
   }
-  if (workflow) {
-    Object.values(workflow.jobs).forEach((job) => {
-      if (job.adaptor) {
-        job.adaptor = expand(job.adaptor);
-      }
-    });
-  }
-  return opts;
+  const plan = input;
+  Object.values(plan.workflow.steps).forEach((step) => {
+    const job = step;
+    if (job.adaptor) {
+      job.adaptor = expand(job.adaptor);
+    }
+  });
+  return plan;
 };
 
 // src/util/map-adaptors-to-monorepo.ts
 import { readFile } from "node:fs/promises";
-import path2 from "node:path";
+import path from "node:path";
 import assert from "node:assert";
 import { getNameAndVersion as getNameAndVersion2 } from "@openfn/runtime";
 var validateMonoRepo = async (repoPath, log) => {
@@ -564,33 +430,193 @@ var updatePath = (adaptor, repoPath, log) => {
     );
   }
   const shortName = name.replace("@openfn/language-", "");
-  const abspath = path2.resolve(repoPath, "packages", shortName);
+  const abspath = path.resolve(repoPath, "packages", shortName);
+  log.info(`Mapped adaptor ${name} to monorepo: ${abspath}`);
   return `${name}=${abspath}`;
 };
-var mapAdaptorsToMonorepo = async (options, log) => {
-  const { adaptors, monorepoPath, workflow } = options;
+var mapAdaptorsToMonorepo = (monorepoPath = "", input = [], log) => {
   if (monorepoPath) {
-    await validateMonoRepo(monorepoPath, log);
-    log.success(`Loading adaptors from monorepo at ${monorepoPath}`);
-    if (adaptors) {
-      options.adaptors = adaptors.map((a) => {
-        const p = updatePath(a, monorepoPath, log);
-        log.info(`Mapped adaptor ${a} to monorepo: ${p.split("=")[1]}`);
-        return p;
-      });
-    }
-    if (workflow) {
-      Object.values(workflow.jobs).forEach((job) => {
-        if (job.adaptor) {
-          job.adaptor = updatePath(job.adaptor, monorepoPath, log);
-        }
-      });
+    if (Array.isArray(input)) {
+      const adaptors = input;
+      return adaptors.map((a) => updatePath(a, monorepoPath, log));
     }
+    const plan = input;
+    Object.values(plan.workflow.steps).forEach((step) => {
+      const job = step;
+      if (job.adaptor) {
+        job.adaptor = updatePath(job.adaptor, monorepoPath, log);
+      }
+    });
+    return plan;
   }
-  return options;
+  return input;
 };
 var map_adaptors_to_monorepo_default = mapAdaptorsToMonorepo;
 
+// src/util/load-plan.ts
+var loadPlan = async (options, logger) => {
+  const { workflowPath, planPath, expressionPath } = options;
+  if (expressionPath) {
+    return loadExpression(options, logger);
+  }
+  const jsonPath = planPath || workflowPath;
+  if (!options.baseDir) {
+    options.baseDir = path2.dirname(jsonPath);
+  }
+  const json = await loadJson(jsonPath, logger);
+  const defaultName = path2.parse(jsonPath).name;
+  if (json.workflow) {
+    return loadXPlan(json, options, logger, defaultName);
+  } else {
+    return loadOldWorkflow(json, options, logger, defaultName);
+  }
+};
+var load_plan_default = loadPlan;
+var loadJson = async (workflowPath, logger) => {
+  let text;
+  try {
+    text = await fs2.readFile(workflowPath, "utf8");
+  } catch (e) {
+    return abort_default(
+      logger,
+      "Workflow not found",
+      void 0,
+      `Failed to load a workflow from ${workflowPath}`
+    );
+  }
+  let json;
+  try {
+    json = JSON.parse(text);
+  } catch (e) {
+    return abort_default(
+      logger,
+      "Invalid JSON in workflow",
+      e,
+      `Check the syntax of the JSON at ${workflowPath}`
+    );
+  }
+  return json;
+};
+var maybeAssign = (a, b, keys) => {
+  keys.forEach((key) => {
+    if (a.hasOwnProperty(key)) {
+      b[key] = a[key];
+    }
+  });
+};
+var loadExpression = async (options, logger) => {
+  const expressionPath = options.expressionPath;
+  logger.debug(`Loading expression from ${expressionPath}`);
+  try {
+    const expression = await fs2.readFile(expressionPath, "utf8");
+    const name = path2.parse(expressionPath).name;
+    const step = { expression };
+    if (options.adaptors) {
+      const [adaptor] = options.adaptors;
+      if (adaptor) {
+        step.adaptor = adaptor;
+      }
+    }
+    const wfOptions = {};
+    maybeAssign(options, wfOptions, ["timeout"]);
+    const plan = {
+      workflow: {
+        name,
+        steps: [step]
+      },
+      options: wfOptions
+    };
+    return loadXPlan(plan, options, logger);
+  } catch (e) {
+    abort_default(
+      logger,
+      "Expression not found",
+      void 0,
+      `Failed to load the expression from ${expressionPath}`
+    );
+    return {};
+  }
+};
+var loadOldWorkflow = async (workflow, options, logger, defaultName = "") => {
+  const plan = {
+    workflow: {
+      steps: workflow.jobs
+    },
+    options: {
+      start: workflow.start
+    }
+  };
+  if (workflow.id) {
+    plan.id = workflow.id;
+  }
+  const final = await loadXPlan(plan, options, logger, defaultName);
+  logger.warn("Converted workflow into new format:");
+  logger.warn(final);
+  return final;
+};
+var fetchFile = async (jobId, rootDir = "", filePath, log) => {
+  try {
+    const fullPath = filePath.startsWith("~") ? filePath : path2.resolve(rootDir, filePath);
+    const result = await fs2.readFile(fullPath, "utf8");
+    return result;
+  } catch (e) {
+    abort_default(
+      log,
+      `File not found for job ${jobId}: ${filePath}`,
+      void 0,
+      `This workflow references a file which cannot be found at ${filePath}
+
+Paths inside the workflow are relative to the workflow.json`
+    );
+    return ".";
+  }
+};
+var importExpressions = async (plan, rootDir, log) => {
+  let idx = 0;
+  for (const step of plan.workflow.steps) {
+    const job = step;
+    if (!job.expression) {
+      continue;
+    }
+    idx += 1;
+    const expressionStr = typeof job.expression === "string" && job.expression?.trim();
+    const configurationStr = typeof job.configuration === "string" && job.configuration?.trim();
+    if (expressionStr && isPath(expressionStr)) {
+      job.expression = await fetchFile(
+        job.id || `${idx}`,
+        rootDir,
+        expressionStr,
+        log
+      );
+    }
+    if (configurationStr && isPath(configurationStr)) {
+      const configString = await fetchFile(
+        job.id || `${idx}`,
+        rootDir,
+        configurationStr,
+        log
+      );
+      job.configuration = JSON.parse(configString);
+    }
+  }
+};
+var loadXPlan = async (plan, options, logger, defaultName = "") => {
+  if (!plan.options) {
+    plan.options = {};
+  }
+  if (!plan.workflow.name && defaultName) {
+    plan.workflow.name = defaultName;
+  }
+  await importExpressions(plan, options.baseDir, logger);
+  if (options.expandAdaptors) {
+    expand_adaptors_default(plan);
+  }
+  await map_adaptors_to_monorepo_default(options.monorepoPath, plan, logger);
+  maybeAssign(options, plan.options, ["timeout", "start"]);
+  logger.info(`Loaded workflow ${plan.workflow.name ?? ""}`);
+  return plan;
+};
+
 // src/util/assert-path.ts
 var assert_path_default = (path7) => {
   if (!path7) {
@@ -608,20 +634,13 @@ var executeHandler = async (options, logger) => {
   const start = (/* @__PURE__ */ new Date()).getTime();
   assert_path_default(options.path);
   await validate_adaptors_default(options, logger);
-  let input = await load_input_default(options, logger);
-  if (options.workflow) {
-    expand_adaptors_default(options);
-    await map_adaptors_to_monorepo_default(
-      options,
-      logger
-    );
-  }
+  let plan = await load_plan_default(options, logger);
   const { repoDir, monorepoPath, autoinstall } = options;
   if (autoinstall) {
     if (monorepoPath) {
       logger.warn("Skipping auto-install as monorepo is being used");
     } else {
-      const autoInstallTargets = get_autoinstall_targets_default(options);
+      const autoInstallTargets = get_autoinstall_targets_default(plan);
       if (autoInstallTargets.length) {
         logger.info("Auto-installing language adaptors");
         await install({ packages: autoInstallTargets, repoDir }, logger);
@@ -630,12 +649,12 @@ var executeHandler = async (options, logger) => {
   }
   const state = await load_state_default(options, logger);
   if (options.compile) {
-    input = await compile_default(options, logger);
+    plan = await compile_default(plan, options, logger);
   } else {
     logger.info("Skipping compilation as noCompile is set");
   }
   try {
-    const result = await execute_default(input, state, options);
+    const result = await execute_default(plan, state, options);
     await serialize_output_default(options, result, logger);
     const duration = printDuration((/* @__PURE__ */ new Date()).getTime() - start);
     if (result?.errors) {
@@ -661,21 +680,16 @@ var handler_default = executeHandler;
 import { writeFile as writeFile2 } from "node:fs/promises";
 var compileHandler = async (options, logger) => {
   assert_path_default(options.path);
-  await load_input_default(options, logger);
-  if (options.workflow) {
-    expand_adaptors_default(options);
-    await map_adaptors_to_monorepo_default(
-      options,
-      logger
-    );
-  }
-  let result = await compile_default(options, logger);
-  if (options.workflow) {
-    result = JSON.stringify(result);
+  let result;
+  if (options.expressionPath) {
+    result = await compile_default(options.expressionPath, options, logger);
+  } else {
+    const plan = await load_plan_default(options, logger);
+    result = await compile_default(plan, options, logger);
+    result = JSON.stringify(result, null, 2);
   }
   if (options.outputStdout) {
-    logger.success("Compiled code:");
-    logger.success("\n" + result);
+    logger.success("Result:\n\n" + result);
   } else {
     await writeFile2(options.outputPath, result);
     logger.success(`Compiled to ${options.outputPath}`);
@@ -685,37 +699,41 @@ var handler_default2 = compileHandler;
 
 // src/test/handler.ts
 var testHandler = async (options, logger) => {
-  logger.log("Running test job...");
+  logger.log("Running test workflow...");
   const opts = { ...options };
   opts.compile = true;
   opts.adaptors = [];
-  opts.workflow = {
-    start: "start",
-    jobs: [
-      {
-        id: "start",
-        state: { data: { defaultAnswer: 42 } },
-        expression: "const fn = () => (state) => { console.log('Starting computer...'); return state; }; fn()",
-        next: {
-          calculate: "!state.error"
-        }
-      },
-      {
-        id: "calculate",
-        expression: "const fn = () => (state) => { console.log('Calculating to life, the universe, and everything..'); return state }; fn()",
-        next: {
-          result: true
+  const plan = {
+    options: {
+      start: "start"
+    },
+    workflow: {
+      steps: [
+        {
+          id: "start",
+          state: { data: { defaultAnswer: 42 } },
+          expression: "const fn = () => (state) => { console.log('Starting computer...'); return state; }; fn()",
+          next: {
+            calculate: "!state.error"
+          }
+        },
+        {
+          id: "calculate",
+          expression: "const fn = () => (state) => { console.log('Calculating to life, the universe, and everything..'); return state }; fn()",
+          next: {
+            result: true
+          }
+        },
+        {
+          id: "result",
+          expression: "const fn = () => (state) => ({ data: { answer: state.data.answer || state.data.defaultAnswer } }); fn()"
         }
-      },
-      {
-        id: "result",
-        expression: "const fn = () => (state) => ({ data: { answer: state.data.answer || state.data.defaultAnswer } }); fn()"
-      }
-    ]
+      ]
+    }
   };
   logger.break();
-  logger.info("Workflow object:");
-  logger.info(JSON.stringify(opts.workflow, null, 2));
+  logger.info("Execution plan:");
+  logger.info(JSON.stringify(plan, null, 2));
   logger.break();
   if (!opts.stateStdin) {
     logger.debug(
@@ -724,8 +742,8 @@ var testHandler = async (options, logger) => {
     logger.debug('eg: -S "{ "data": { "answer": 33 }  }"');
   }
   const state = await load_state_default(opts, createNullLogger());
-  const code = await compile_default(opts, logger);
-  const result = await execute_default(code, state, opts);
+  const compiledPlan = await compile_default(plan, opts, logger);
+  const result = await execute_default(compiledPlan, state, opts);
   logger.success(`Result: ${result.data.answer}`);
   return result;
 };
@@ -909,7 +927,7 @@ ${data.functions.map((fn) => `  ${fn.name}(${fn.parameters.map((p) => p.name).jo
 `;
 var docsHandler = async (options, logger) => {
   const { adaptor, operation, repoDir } = options;
-  const { adaptors } = expand_adaptors_default({ adaptors: [adaptor] });
+  const adaptors = expand_adaptors_default([adaptor]);
   const [adaptorName] = adaptors;
   let { name, version } = getNameAndVersion4(adaptorName);
   if (!version) {
@@ -1177,7 +1195,7 @@ var loadVersionFromPath = (adaptorPath) => {
     return "unknown";
   }
 };
-var printVersions = async (logger, options = {}) => {
+var printVersions = async (logger, options = {}, includeComponents = false) => {
   const { adaptors, logJson } = options;
   let adaptor = "";
   if (adaptors && adaptors.length) {
@@ -1190,6 +1208,9 @@ var printVersions = async (logger, options = {}) => {
       const [namePart, pathPart] = adaptor.split("=");
       adaptorVersion = loadVersionFromPath(pathPart);
       adaptorName = getNameAndVersion5(namePart).name;
+    } else if (options.monorepoPath) {
+      adaptorName = getNameAndVersion5(adaptor).name;
+      adaptorVersion = "monorepo";
     } else {
       const { name, version: version2 } = getNameAndVersion5(adaptor);
       adaptorName = name;
@@ -1210,22 +1231,29 @@ var printVersions = async (logger, options = {}) => {
     output = {
       versions: {
         "node.js": process.version.substring(1),
-        cli: version,
-        runtime: runtimeVersion,
-        compiler: compilerVersion
+        cli: version
       }
     };
+    if (includeComponents) {
+      output.versions.runtime = runtimeVersion;
+      output.versions.compiler = compilerVersion;
+    }
     if (adaptorName) {
       output.versions[adaptorName] = adaptorVersion;
     }
   } else {
-    const adaptorVersionString = adaptorName ? `
-${prefix(adaptorName)}${adaptorVersion}` : "";
     output = `Versions:
 ${prefix(NODE)}${process.version.substring(1)}
-${prefix(CLI2)}${version}
+${prefix(CLI2)}${version}`;
+    if (includeComponents) {
+      output += `
 ${prefix(RUNTIME2)}${runtimeVersion}
-${prefix(COMPILER2)}${compilerVersion}${adaptorVersionString}`;
+${prefix(COMPILER2)}${compilerVersion}`;
+    }
+    if (adaptorName) {
+      output += `
+${prefix(adaptorName)}${adaptorVersion}`;
+    }
   }
   logger.always(output);
 };
@@ -1245,23 +1273,27 @@ var handlers = {
   ["repo-install"]: install,
   ["repo-pwd"]: pwd,
   ["repo-list"]: list,
-  version: async (opts, logger) => print_versions_default(logger, opts)
+  version: async (opts, logger) => print_versions_default(logger, opts, true)
 };
 var parse = async (options, log) => {
   const logger = log || logger_default(CLI, options);
   if (options.command === "execute" || options.command === "test") {
     await print_versions_default(logger, options);
   }
-  if (options.monorepoPath) {
-    if (options.monorepoPath === "ERR") {
+  const { monorepoPath } = options;
+  if (monorepoPath) {
+    if (monorepoPath === "ERR") {
       logger.error(
         "ERROR: --use-adaptors-monorepo was passed, but OPENFN_ADAPTORS_REPO env var is undefined"
       );
       logger.error("Set OPENFN_ADAPTORS_REPO to a path pointing to the repo");
       process.exit(9);
     }
-    await map_adaptors_to_monorepo_default(
-      options,
+    await validateMonoRepo(monorepoPath, logger);
+    logger.success(`Loading adaptors from monorepo at ${monorepoPath}`);
+    options.adaptors = map_adaptors_to_monorepo_default(
+      monorepoPath,
+      options.adaptors,
       logger
     );
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@openfn/cli",
-  "version": "0.4.16",
+  "version": "1.1.0",
   "description": "CLI devtools for the openfn toolchain.",
   "engines": {
     "node": ">=18",
@@ -35,7 +35,8 @@
     "ts-node": "^10.9.1",
     "tslib": "^2.4.0",
     "tsup": "^7.2.0",
-    "typescript": "^5.1.6"
+    "typescript": "^5.1.6",
+    "@openfn/lexicon": "^1.0.0"
   },
   "dependencies": {
     "@inquirer/prompts": "^1.1.4",
@@ -44,11 +45,11 @@
     "rimraf": "^3.0.2",
     "treeify": "^1.1.0",
     "yargs": "^17.7.2",
-    "@openfn/compiler": "0.0.39",
-    "@openfn/deploy": "0.4.1",
+    "@openfn/compiler": "0.0.40",
+    "@openfn/deploy": "0.4.2",
     "@openfn/describe-package": "0.0.18",
-    "@openfn/logger": "0.0.20",
-    "@openfn/runtime": "0.2.6"
+    "@openfn/logger": "1.0.0",
+    "@openfn/runtime": "1.1.0"
   },
   "files": [
     "dist",