agent-gauntlet 0.1.11 → 0.2.0

package/README.md

@@ -1,122 +1,90 @@
 # Agent Gauntlet
 
-Agent Gauntlet is a configurable “quality gate” runner for AI-assisted development workflows.
+> Don't just review the agent's code — put it through the gauntlet.
 
-You define:
-- **Entry points** (paths in your repo)
-- **Check gates** (shell commands: tests, linters, typecheck, etc.)
-- **Review gates** (AI CLI tools run on diffs, with regex-based pass/fail)
+Agent Gauntlet is a configurable “feedback loop” runner for AI-assisted development workflows.
 
-Then `agent-gauntlet` detects which parts of the repo changed and runs the relevant gates.
+You configure which paths in your repo should trigger which validations — shell commands like tests and linters, plus AI-powered code reviews. When files change, Gauntlet automatically runs the relevant validations and reports results.
 
-### AI CLI Integration
+For AI reviews, it uses the CLI tool of your choice: Gemini, Codex, Claude Code, GitHub Copilot, or Cursor. 
 
-Agent Gauntlet is designed to be "tool-agnostic" by leveraging the AI CLI tools you already have installed (such as `gemini`, `codex`, or `claude`). Instead of managing its own API keys or subscriptions, it invokes these CLIs directly. This allows you to:
-- **Leverage existing subscriptions**: Use the tools you are already paying for.
-- **Dynamic Context**: Agents are invoked in a non-interactive, read-only mode where they can use their own file-reading and search tools to pull additional context from your repository as needed.
-- **Security**: By using standard CLI tools with strict flags (like `--sandbox` or `--allowed-tools`), Agent Gauntlet ensures that agents can read your code to review it without being able to modify your files or escape the repository scope.
+## Features
 
-### Requirements
+- **Agent validation loop**: Keep your coding agent on track with automated feedback loops. Detect problems — deterministically and/or non-deterministically — and let your agent fix and Gauntlet verify.
+- **Multi-agent collaboration**: Enable one AI agent to automatically request code reviews from another. For example, if Claude made changes, Gauntlet can request a review from Codex or Gemini — spreading token usage across your subscriptions instead of burning through one.
+- **Leverage existing subscriptions**: Agent Gauntlet is *free* and tool-agnostic, leveraging the AI CLI tools you already have installed.
+- **Easy CI setup**: Define your checks once, run them locally and in GitHub.
 
-- **Bun** (Required runtime, v1.0.0+)
-- **git** (change detection and diffs)
-- For review gates: one or more supported AI CLIs installed (`gemini`, `codex`, `claude`, `github-copilot`, `cursor`). For the full list of tools and how they are used, see [CLI Invocation Details](docs/cli-invocation-details.md)
+## Usage Patterns
 
-### Installation
+Agent Gauntlet supports three primary usage patterns, each suited for different development workflows:
+1. Run CLI: `agent-gauntlet run`
+2. Run agent command: `/gauntlet`
+3. Automatically run after agent completes task
 
-You can install `agent-gauntlet` globally using `npm` or `bun` (Bun must be installed on the system in both cases):
+The use cases below illustrate when each of these patterns may be used.
 
-**Using Bun (Recommended):**
-```bash
-bun add -g agent-gauntlet
-```
+### 1. Planning Mode
 
-**Using npm:**
-```bash
-npm install -g agent-gauntlet
-```
+**Use case:** Generate and review high-level implementation plans before coding.
 
-### Quick start
+**Problem Gauntlet solves:** Catch architectural issues and requirement misunderstandings before coding to avoid costly rework.
 
-- **Initialize configuration**
+**Workflow:**
 
-```bash
-agent-gauntlet init
-```
+1. Create a plan document in your project directory
+2. Run `agent-gauntlet run` from the terminal
+3. Gauntlet detects the new or modified plan and invokes configured AI CLIs to review it
+4. *(Optional)* Ask your assistant to refine the plan based on review feedback
 
-- **Run gates**
+**Note:** Review configuration and prompts are fully customizable. Example prompt: *"Review this plan for completeness and potential issues."*
 
-```bash
-agent-gauntlet
-```
+### 2. AI-Assisted Development
 
-### Development
+**Use case:** Pair with an AI coding assistant to implement features with continuous quality checks.
 
-- **Install dependencies**
+**Problem Gauntlet solves:** Catch AI-introduced bugs and quality issues through automated checks and multi-LLM review.
 
-```bash
-bun install
-```
+**Workflow:**
 
-- **Build the CLI binary**
+1. Collaborate with your assistant to implement code changes
+2. Run `/gauntlet` from chat
+3. Gauntlet detects changed files and runs configured checks (linter, tests, type checking, etc.)
+4. Simultaneously, Gauntlet invokes AI CLIs for code review
+5. Assistant reviews results, fixes identified issues, and runs `agent-gauntlet rerun`
+6. Gauntlet verifies fixes and checks for new issues
+7. Process repeats automatically (up to 3 reruns) until all gates pass
 
-```bash
-bun run build
-```
+### 3. Agentic Implementation
 
-### Basic usage
+**Use case:** Delegate well-defined tasks to a coding agent for autonomous implementation.
 
-- **Run gates for detected changes**
+**Problem Gauntlet solves:** Enable autonomous agent development with built-in quality gates, eliminating the validation gap when humans aren't in the loop.
 
-```bash
-agent-gauntlet run
-```
+**Workflow:**
 
-- **Run only one gate name** (runs it across all applicable entry points)
+1. Configure your agent to automatically run `/gauntlet` after completing implementation:
+   - **Rules files:** Add to `.cursorrules`, `AGENT.md`, or similar
+   - **Custom commands:** Create a `/my-dev-workflow` that includes gauntlet
+   - **Git hooks:** Use pre-commit hooks to trigger gauntlet
+   - **Agent hooks:** Leverage platform features (e.g., Claude's Stop event)
+2. Assign the task to your agent and step away
+3. When you return: the task is complete, reviewed by a different LLM, all issues fixed, and CI checks passing
 
-```bash
-agent-gauntlet run --gate lint
-```
+**Benefit:** Fully autonomous quality assurance without manual intervention.
 
-- **List configured gates and entry points**
+## Quick Start
 
-```bash
-agent-gauntlet list
-```
+1. **Install**: `bun add -g agent-gauntlet`
+2. **Initialize**: `agent-gauntlet init`
+3. **Run**: `agent-gauntlet run`
 
-- **Check which AI CLIs are installed**
+For basic usage and configuration guide, see the [Quick Start Guide](docs/quick-start.md).
 
-```bash
-agent-gauntlet health
-```
-
-### Agent loop rules
-
-The `.gauntlet/run_gauntlet.md` file defines how AI agents should interact with the gauntlet. By default, agents will terminate after 4 runs (1 initial + 3 fix attempts). You can increase this limit by manually editing the termination conditions in that file.
-
-### Configuration layout
-
-Agent Gauntlet loads configuration from your repository:
-
-```text
-.gauntlet/
-  config.yml
-  checks/
-    *.yml
-  reviews/
-    *.md
-```
-
-- **Project config**: `.gauntlet/config.yml`
-- **Check definitions**: `.gauntlet/checks/*.yml`
-- **Review definitions**: `.gauntlet/reviews/*.md` (filename is the review gate name)
-
-### Logs
-
-Each job writes a log file under `log_dir` (default: `.gauntlet_logs/`). Filenames are derived from the job id (sanitized).
-
-### Documentation
+## Documentation
 
+- [Quick Start Guide](docs/quick-start.md) — installation, basic usage, and config layout
 - [User Guide](docs/user-guide.md) — full usage details
 - [Configuration Reference](docs/config-reference.md) — all configuration fields + defaults
 - [CLI Invocation Details](docs/cli-invocation-details.md) — how we securely invoke AI CLIs
+- [Development Guide](docs/development.md) — how to build and develop this project

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-gauntlet",
-  "version": "0.1.11",
+  "version": "0.2.0",
   "description": "A CLI tool for testing AI coding agents",
   "license": "Apache-2.0",
   "author": "Paul Caplan",

package/src/bun-plugins.d.ts

@@ -0,0 +1,4 @@
+declare module "*.yml" {
+	const content: string;
+	export default content;
+}

package/src/cli-adapters/github-copilot.ts

@@ -139,7 +139,7 @@ export class GitHubCopilotAdapter implements CLIAdapter {
 			// because copilot requires stdin input. The tmpFile path is system-controlled
 			// (os.tmpdir() + Date.now() + process.pid), not user-supplied, eliminating injection risk.
 			// Double quotes handle paths with spaces. This pattern matches claude.ts:131.
-			const cmd = `cat "${tmpFile}" | copilot --allow-tool shell(cat) --allow-tool shell(grep) --allow-tool shell(ls) --allow-tool shell(find) --allow-tool shell(head) --allow-tool shell(tail)`;
+			const cmd = `cat "${tmpFile}" | copilot --allow-tool "shell(cat)" --allow-tool "shell(grep)" --allow-tool "shell(ls)" --allow-tool "shell(find)" --allow-tool "shell(head)" --allow-tool "shell(tail)"`;
 			const { stdout } = await execAsync(cmd, {
 				timeout: opts.timeoutMs,
 				maxBuffer: MAX_BUFFER_BYTES,

package/src/commands/check.ts

@@ -13,6 +13,10 @@ export function registerCheckCommand(program: Command): void {
 	program
 		.command("check")
 		.description("Run only applicable checks for detected changes")
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific check gate only")
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
@@ -26,7 +30,17 @@ export function registerCheckCommand(program: Command): void {
 				// Rotate logs before starting
 				await rotateLogs(config.project.log_dir);
 
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});
@@ -65,7 +79,14 @@ export function registerCheckCommand(program: Command): void {
 
 				const logger = new Logger(config.project.log_dir);
 				const reporter = new ConsoleReporter();
-				const runner = new Runner(config, logger, reporter);
+				const runner = new Runner(
+					config,
+					logger,
+					reporter,
+					undefined,
+					undefined,
+					effectiveBaseBranch,
+				);
 
 				const success = await runner.run(jobs);
 				process.exit(success ? 0 : 1);

package/src/commands/ci/index.ts

@@ -0,0 +1,15 @@
+import type { Command } from "commander";
+import { initCI } from "./init.js";
+import { listJobs } from "./list-jobs.js";
+
+export function registerCICommand(program: Command): void {
+	const ci = program.command("ci").description("Manage CI integration");
+
+	ci.command("init")
+		.description("Initialize CI workflow and configuration")
+		.action(initCI);
+
+	ci.command("list-jobs")
+		.description("List CI jobs (used by workflow)")
+		.action(listJobs);
+}

package/src/commands/ci/init.ts

@@ -0,0 +1,96 @@
+import fs from "node:fs/promises";
+import path from "node:path";
+import chalk from "chalk";
+import YAML from "yaml";
+import { loadCIConfig } from "../../config/ci-loader.js";
+import type { CIConfig } from "../../config/types.js";
+import workflowTemplate from "../../templates/workflow.yml" with {
+	type: "text",
+};
+
+export async function initCI(): Promise<void> {
+	const workflowDir = path.join(process.cwd(), ".github", "workflows");
+	const workflowPath = path.join(workflowDir, "gauntlet.yml");
+	const gauntletDir = path.join(process.cwd(), ".gauntlet");
+	const ciConfigPath = path.join(gauntletDir, "ci.yml");
+
+	// 1. Ensure .gauntlet/ci.yml exists
+	if (!(await fileExists(ciConfigPath))) {
+		console.log(chalk.yellow("Creating starter .gauntlet/ci.yml..."));
+		await fs.mkdir(gauntletDir, { recursive: true });
+		const starterContent = `# CI Configuration for Agent Gauntlet
+# Define runtimes, services, and which checks to run in CI.
+
+runtimes:
+  # ruby:
+  #   version: "3.3"
+  #   bundler_cache: true
+
+services:
+  # postgres:
+  #   image: postgres:16
+  #   ports: ["5432:5432"]
+
+setup:
+  # - name: Global Setup
+  #   run: echo "Setting up..."
+
+checks:
+  # - name: linter
+  #   requires_runtimes: [ruby]
+`;
+		await fs.writeFile(ciConfigPath, starterContent);
+	} else {
+		console.log(chalk.dim("Found existing .gauntlet/ci.yml"));
+	}
+
+	// 2. Load CI config to get services
+	let ciConfig: CIConfig | undefined;
+	try {
+		ciConfig = await loadCIConfig();
+	} catch (_e) {
+		console.warn(
+			chalk.yellow(
+				"Could not load CI config to inject services. Workflow will have no services defined.",
+			),
+		);
+	}
+
+	// 3. Generate workflow file
+	console.log(chalk.dim(`Generating ${workflowPath}...`));
+	await fs.mkdir(workflowDir, { recursive: true });
+
+	let templateContent = workflowTemplate;
+
+	// Inject services
+	if (ciConfig?.services && Object.keys(ciConfig.services).length > 0) {
+		const servicesYaml = YAML.stringify({ services: ciConfig.services });
+		// Indent services
+		const indentedServices = servicesYaml
+			.split("\n")
+			.map((line) => (line.trim() ? `    ${line}` : line))
+			.join("\n");
+
+		templateContent = templateContent.replace(
+			"    # Services will be injected here by agent-gauntlet",
+			indentedServices,
+		);
+	} else {
+		templateContent = templateContent.replace(
+			"    # Services will be injected here by agent-gauntlet\n",
+			"",
+		);
+	}
+
+	await fs.writeFile(workflowPath, templateContent);
+	console.log(chalk.green("Successfully generated GitHub Actions workflow!"));
+}
+
+async function fileExists(path: string): Promise<boolean> {
+	try {
+		const stat = await fs.stat(path);
+		return stat.isFile();
+	} catch {
+		return false;
+	}
+}

package/src/commands/ci/list-jobs.ts

@@ -0,0 +1,89 @@
+import { loadCIConfig } from "../../config/ci-loader.js";
+import { loadConfig } from "../../config/loader.js";
+import type { CISetupStep } from "../../config/types.js";
+import { EntryPointExpander } from "../../core/entry-point.js";
+
+export async function listJobs(): Promise<void> {
+	try {
+		const config = await loadConfig();
+		const ciConfig = await loadCIConfig();
+		const expander = new EntryPointExpander();
+		const expandedEntryPoints = await expander.expandAll(
+			config.project.entry_points,
+		);
+
+		const matrixJobs = [];
+		const seenJobs = new Set<string>();
+
+		const globalSetup = formatSetup(ciConfig.setup || undefined);
+
+		if (ciConfig.checks) {
+			for (const ep of expandedEntryPoints) {
+				// Get checks enabled for this entry point
+				const allowedChecks = new Set(ep.config.checks || []);
+
+				for (const check of ciConfig.checks) {
+					if (allowedChecks.has(check.name)) {
+						// Check definition from .gauntlet/checks/*.yml
+						const checkDef = config.checks[check.name];
+						if (!checkDef) {
+							console.warn(
+								`Warning: Check '${check.name}' found in CI config but not defined in checks/*.yml`,
+							);
+							continue;
+						}
+
+						const workingDirectory = checkDef.working_directory || ep.path;
+						// Include entry point in key to ensure each entry point/check pair is distinct
+						const jobKey = `${ep.path}:${check.name}:${workingDirectory}`;
+
+						// Skip if we've already created a job for this exact entry point/check combination
+						if (seenJobs.has(jobKey)) {
+							continue;
+						}
+						seenJobs.add(jobKey);
+
+						const id = `${check.name}-${ep.path.replace(/\//g, "-")}`;
+
+						matrixJobs.push({
+							id,
+							name: check.name,
+							entry_point: ep.path,
+							working_directory: workingDirectory,
+							command: checkDef.command,
+							runtimes: check.requires_runtimes || [],
+							services: check.requires_services || [],
+							setup: formatSetup(check.setup || undefined),
+							global_setup: globalSetup,
+						});
+					}
+				}
+			}
+		}
+
+		const output = {
+			matrix: matrixJobs,
+			services: ciConfig.services || {},
+			runtimes: ciConfig.runtimes || {},
+		};
+
+		console.log(JSON.stringify(output));
+	} catch (e) {
+		console.error("Error generating CI jobs:", e);
+		process.exit(1);
+	}
+}
+
+const formatSetup = (steps: CISetupStep[] | null | undefined): string => {
+	if (!steps || steps.length === 0) return "";
+	return steps
+		.map((s) => {
+			const cmd = s.working_directory
+				? `(cd "${s.working_directory}" && ${s.run})`
+				: s.run;
+			return `echo "::group::${s.name}"
+${cmd}
+echo "::endgroup::"`;
+		})
+		.join("\n");
+};

package/src/commands/detect.ts

@@ -11,6 +11,10 @@ export function registerDetectCommand(program: Command): void {
 		.description(
 			"Show what gates would run for detected changes (without executing them)",
 		)
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
 			"-u, --uncommitted",
@@ -19,7 +23,18 @@ export function registerDetectCommand(program: Command): void {
 		.action(async (options) => {
 			try {
 				const config = await loadConfig();
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});

package/src/commands/health.test.ts

@@ -33,7 +33,7 @@ describe("Health Command", () => {
 			path.join(GAUNTLET_DIR, "config.yml"),
 			`
 base_branch: origin/main
-log_dir: .gauntlet_logs
+log_dir: gauntlet_logs
 cli:
   default_preference:
     - gemini

package/src/commands/help.ts

@@ -24,6 +24,7 @@ export function registerHelpCommand(program: Command): void {
 			console.log("  list     List configured gates");
 			console.log("  health   Check CLI tool availability");
 			console.log("  init     Initialize .gauntlet configuration");
+			console.log("  ci       CI integration commands (init, list-jobs)");
 			console.log("  help     Show this help message\n");
 			console.log(
 				"For more information, see: https://github.com/your-repo/agent-gauntlet",

package/src/commands/index.ts

@@ -1,4 +1,5 @@
 export { registerCheckCommand } from "./check.js";
+export { registerCICommand } from "./ci/index.js";
 export { registerDetectCommand } from "./detect.js";
 export { registerHealthCommand } from "./health.js";
 export { registerHelpCommand } from "./help.js";

package/src/commands/init.test.ts

@@ -40,6 +40,8 @@ mock.module("../cli-adapters/index.js", () => ({
 	getAllAdapters: () => mockAdapters,
 	getProjectCommandAdapters: () => mockAdapters,
 	getUserCommandAdapters: () => [],
+	getAdapter: (name: string) => mockAdapters.find((a) => a.name === name),
+	getValidCLITools: () => mockAdapters.map((a) => a.name),
 }));
 
 // Import after mocking

package/src/commands/init.ts

@@ -16,7 +16,7 @@ allowed-tools: Bash
 Execute the autonomous verification suite.
 
 1. Run \`agent-gauntlet run\`.
-2. If it fails, read the log files in \`.gauntlet_logs/\` to understand exactly what went wrong.
+2. If it fails, read the log files in \`gauntlet_logs/\` to understand exactly what went wrong.
 3. Fix any code or logic errors found by the tools or AI reviewers, prioritizing higher-priority violations (critical > high > medium > low).
 4. If you disagree with AI reviewer feedback, briefly explain your reasoning in the code comments rather than ignoring it silently.
 5. Do NOT commit your changes yet—keep them uncommitted so the rerun command can review them.
@@ -336,7 +336,7 @@ function generateConfigYml(config: InitConfig): string {
       - code-quality`;
 
 	return `base_branch: origin/main
-log_dir: .gauntlet_logs
+log_dir: gauntlet_logs
 
 # Run gates in parallel when possible (default: true)
 # allow_parallel: true

package/src/commands/list.test.ts

@@ -37,7 +37,7 @@ describe("List Command", () => {
 			path.join(GAUNTLET_DIR, "config.yml"),
 			`
 base_branch: origin/main
-log_dir: .gauntlet_logs
+log_dir: gauntlet_logs
 cli:
   default_preference:
     - gemini

package/src/commands/rerun.ts

@@ -19,6 +19,10 @@ export function registerRerunCommand(program: Command): void {
 		.description(
 			"Rerun gates (checks & reviews) with previous failures as context (defaults to uncommitted changes)",
 		)
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific gate only")
 		.option(
 			"-c, --commit <sha>",
@@ -71,6 +75,16 @@ export function registerRerunCommand(program: Command): void {
 				// Rotate logs before starting the new run
 				await rotateLogs(config.project.log_dir);
 
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
 				// Detect changes (default to uncommitted unless --commit is specified)
 				// Note: Rerun defaults to uncommitted changes for faster iteration loops,
 				// unlike 'run' which defaults to base_branch comparison.
@@ -80,7 +94,7 @@ export function registerRerunCommand(program: Command): void {
 				};
 
 				const changeDetector = new ChangeDetector(
-					config.project.base_branch,
+					effectiveBaseBranch,
 					changeOptions,
 				);
 				const expander = new EntryPointExpander();
@@ -132,6 +146,7 @@ export function registerRerunCommand(program: Command): void {
 					reporter,
 					failuresMap, // Pass previous failures map
 					changeOptions, // Pass change detection options
+					effectiveBaseBranch, // Pass effective base branch
 				);
 
 				const success = await runner.run(jobs);

package/src/commands/review.ts

@@ -13,6 +13,10 @@ export function registerReviewCommand(program: Command): void {
 	program
 		.command("review")
 		.description("Run only applicable reviews for detected changes")
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific review gate only")
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
@@ -26,7 +30,17 @@ export function registerReviewCommand(program: Command): void {
 				// Rotate logs before starting
 				await rotateLogs(config.project.log_dir);
 
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});
@@ -65,7 +79,14 @@ export function registerReviewCommand(program: Command): void {
 
 				const logger = new Logger(config.project.log_dir);
 				const reporter = new ConsoleReporter();
-				const runner = new Runner(config, logger, reporter);
+				const runner = new Runner(
+					config,
+					logger,
+					reporter,
+					undefined,
+					undefined,
+					effectiveBaseBranch,
+				);
 
 				const success = await runner.run(jobs);
 				process.exit(success ? 0 : 1);