agent-gauntlet 0.1.11 → 0.1.12

package/README.md

@@ -1,122 +1,90 @@
 # Agent Gauntlet
 
-Agent Gauntlet is a configurable “quality gate” runner for AI-assisted development workflows.
+> Don't just review the agent's code — put it through the gauntlet.
 
-You define:
-- **Entry points** (paths in your repo)
-- **Check gates** (shell commands: tests, linters, typecheck, etc.)
-- **Review gates** (AI CLI tools run on diffs, with regex-based pass/fail)
+Agent Gauntlet is a configurable “feedback loop” runner for AI-assisted development workflows.
 
-Then `agent-gauntlet` detects which parts of the repo changed and runs the relevant gates.
+You configure which paths in your repo should trigger which validations — shell commands like tests and linters, plus AI-powered code reviews. When files change, Gauntlet automatically runs the relevant validations and reports results.
 
-### AI CLI Integration
+For AI reviews, it uses the CLI tool of your choice: Gemini, Codex, Claude Code, GitHub Copilot, or Cursor. 
 
-Agent Gauntlet is designed to be "tool-agnostic" by leveraging the AI CLI tools you already have installed (such as `gemini`, `codex`, or `claude`). Instead of managing its own API keys or subscriptions, it invokes these CLIs directly. This allows you to:
-- **Leverage existing subscriptions**: Use the tools you are already paying for.
-- **Dynamic Context**: Agents are invoked in a non-interactive, read-only mode where they can use their own file-reading and search tools to pull additional context from your repository as needed.
-- **Security**: By using standard CLI tools with strict flags (like `--sandbox` or `--allowed-tools`), Agent Gauntlet ensures that agents can read your code to review it without being able to modify your files or escape the repository scope.
+## Features
 
-### Requirements
+- **Agent validation loop**: Keep your coding agent on track with automated feedback loops. Detect problems — deterministically and/or non-deterministically — and let your agent fix and Gauntlet verify.
+- **Multi-agent collaboration**: Enable one AI agent to automatically request code reviews from another. For example, if Claude made changes, Gauntlet can request a review from Codex or Gemini — spreading token usage across your subscriptions instead of burning through one.
+- **Leverage existing subscriptions**: Agent Gauntlet is *free* and tool-agnostic, leveraging the AI CLI tools you already have installed.
+- **Easy CI setup**: Define your checks once, run them locally and in GitHub.
 
-- **Bun** (Required runtime, v1.0.0+)
-- **git** (change detection and diffs)
-- For review gates: one or more supported AI CLIs installed (`gemini`, `codex`, `claude`, `github-copilot`, `cursor`). For the full list of tools and how they are used, see [CLI Invocation Details](docs/cli-invocation-details.md)
+## Usage Patterns
 
-### Installation
+Agent Gauntlet supports three primary usage patterns, each suited for different development workflows:
+1. Run CLI: `agent-gauntlet run`
+2. Run agent command: `/gauntlet`
+3. Automatically run after agent completes task
 
-You can install `agent-gauntlet` globally using `npm` or `bun` (Bun must be installed on the system in both cases):
+The use cases below illustrate when each of these patterns may be used.
 
-**Using Bun (Recommended):**
-```bash
-bun add -g agent-gauntlet
-```
+### 1. Planning Mode
 
-**Using npm:**
-```bash
-npm install -g agent-gauntlet
-```
+**Use case:** Generate and review high-level implementation plans before coding.
 
-### Quick start
+**Problem Gauntlet solves:** Catch architectural issues and requirement misunderstandings before coding to avoid costly rework.
 
-- **Initialize configuration**
+**Workflow:**
 
-```bash
-agent-gauntlet init
-```
+1. Create a plan document in your project directory
+2. Run `agent-gauntlet run` from the terminal
+3. Gauntlet detects the new or modified plan and invokes configured AI CLIs to review it
+4. *(Optional)* Ask your assistant to refine the plan based on review feedback
 
-- **Run gates**
+**Note:** Review configuration and prompts are fully customizable. Example prompt: *"Review this plan for completeness and potential issues."*
 
-```bash
-agent-gauntlet
-```
+### 2. AI-Assisted Development
 
-### Development
+**Use case:** Pair with an AI coding assistant to implement features with continuous quality checks.
 
-- **Install dependencies**
+**Problem Gauntlet solves:** Catch AI-introduced bugs and quality issues through automated checks and multi-LLM review.
 
-```bash
-bun install
-```
+**Workflow:**
 
-- **Build the CLI binary**
+1. Collaborate with your assistant to implement code changes
+2. Run `/gauntlet` from chat
+3. Gauntlet detects changed files and runs configured checks (linter, tests, type checking, etc.)
+4. Simultaneously, Gauntlet invokes AI CLIs for code review
+5. Assistant reviews results, fixes identified issues, and runs `agent-gauntlet rerun`
+6. Gauntlet verifies fixes and checks for new issues
+7. Process repeats automatically (up to 3 reruns) until all gates pass
 
-```bash
-bun run build
-```
+### 3. Agentic Implementation
 
-### Basic usage
+**Use case:** Delegate well-defined tasks to a coding agent for autonomous implementation.
 
-- **Run gates for detected changes**
+**Problem Gauntlet solves:** Enable autonomous agent development with built-in quality gates, eliminating the validation gap when humans aren't in the loop.
 
-```bash
-agent-gauntlet run
-```
+**Workflow:**
 
-- **Run only one gate name** (runs it across all applicable entry points)
+1. Configure your agent to automatically run `/gauntlet` after completing implementation:
+   - **Rules files:** Add to `.cursorrules`, `AGENT.md`, or similar
+   - **Custom commands:** Create a `/my-dev-workflow` that includes gauntlet
+   - **Git hooks:** Use pre-commit hooks to trigger gauntlet
+   - **Agent hooks:** Leverage platform features (e.g., Claude's Stop event)
+2. Assign the task to your agent and step away
+3. When you return: the task is complete, reviewed by a different LLM, all issues fixed, and CI checks passing
 
-```bash
-agent-gauntlet run --gate lint
-```
+**Benefit:** Fully autonomous quality assurance without manual intervention.
 
-- **List configured gates and entry points**
+## Quick Start
 
-```bash
-agent-gauntlet list
-```
+1. **Install**: `bun add -g agent-gauntlet`
+2. **Initialize**: `agent-gauntlet init`
+3. **Run**: `agent-gauntlet run`
 
-- **Check which AI CLIs are installed**
+For basic usage and configuration guide, see the [Quick Start Guide](docs/quick-start.md).
 
-```bash
-agent-gauntlet health
-```
-
-### Agent loop rules
-
-The `.gauntlet/run_gauntlet.md` file defines how AI agents should interact with the gauntlet. By default, agents will terminate after 4 runs (1 initial + 3 fix attempts). You can increase this limit by manually editing the termination conditions in that file.
-
-### Configuration layout
-
-Agent Gauntlet loads configuration from your repository:
-
-```text
-.gauntlet/
-  config.yml
-  checks/
-    *.yml
-  reviews/
-    *.md
-```
-
-- **Project config**: `.gauntlet/config.yml`
-- **Check definitions**: `.gauntlet/checks/*.yml`
-- **Review definitions**: `.gauntlet/reviews/*.md` (filename is the review gate name)
-
-### Logs
-
-Each job writes a log file under `log_dir` (default: `.gauntlet_logs/`). Filenames are derived from the job id (sanitized).
-
-### Documentation
+## Documentation
 
+- [Quick Start Guide](docs/quick-start.md) — installation, basic usage, and config layout
 - [User Guide](docs/user-guide.md) — full usage details
 - [Configuration Reference](docs/config-reference.md) — all configuration fields + defaults
 - [CLI Invocation Details](docs/cli-invocation-details.md) — how we securely invoke AI CLIs
+- [Development Guide](docs/development.md) — how to build and develop this project

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-gauntlet",
-  "version": "0.1.11",
+  "version": "0.1.12",
   "description": "A CLI tool for testing AI coding agents",
   "license": "Apache-2.0",
   "author": "Paul Caplan",

package/src/bun-plugins.d.ts

@@ -0,0 +1,4 @@
+declare module "*.yml" {
+	const content: string;
+	export default content;
+}

package/src/cli-adapters/github-copilot.ts

@@ -139,7 +139,7 @@ export class GitHubCopilotAdapter implements CLIAdapter {
 			// because copilot requires stdin input. The tmpFile path is system-controlled
 			// (os.tmpdir() + Date.now() + process.pid), not user-supplied, eliminating injection risk.
 			// Double quotes handle paths with spaces. This pattern matches claude.ts:131.
-			const cmd = `cat "${tmpFile}" | copilot --allow-tool shell(cat) --allow-tool shell(grep) --allow-tool shell(ls) --allow-tool shell(find) --allow-tool shell(head) --allow-tool shell(tail)`;
+			const cmd = `cat "${tmpFile}" | copilot --allow-tool "shell(cat)" --allow-tool "shell(grep)" --allow-tool "shell(ls)" --allow-tool "shell(find)" --allow-tool "shell(head)" --allow-tool "shell(tail)"`;
 			const { stdout } = await execAsync(cmd, {
 				timeout: opts.timeoutMs,
 				maxBuffer: MAX_BUFFER_BYTES,

package/src/commands/check.ts

@@ -13,6 +13,10 @@ export function registerCheckCommand(program: Command): void {
 	program
 		.command("check")
 		.description("Run only applicable checks for detected changes")
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific check gate only")
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
@@ -26,7 +30,17 @@ export function registerCheckCommand(program: Command): void {
 				// Rotate logs before starting
 				await rotateLogs(config.project.log_dir);
 
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});
@@ -65,7 +79,14 @@ export function registerCheckCommand(program: Command): void {
 
 				const logger = new Logger(config.project.log_dir);
 				const reporter = new ConsoleReporter();
-				const runner = new Runner(config, logger, reporter);
+				const runner = new Runner(
+					config,
+					logger,
+					reporter,
+					undefined,
+					undefined,
+					effectiveBaseBranch,
+				);
 
 				const success = await runner.run(jobs);
 				process.exit(success ? 0 : 1);

package/src/commands/ci/index.ts

@@ -0,0 +1,15 @@
+import type { Command } from "commander";
+import { initCI } from "./init.js";
+import { listJobs } from "./list-jobs.js";
+
+export function registerCICommand(program: Command): void {
+	const ci = program.command("ci").description("Manage CI integration");
+
+	ci.command("init")
+		.description("Initialize CI workflow and configuration")
+		.action(initCI);
+
+	ci.command("list-jobs")
+		.description("List CI jobs (used by workflow)")
+		.action(listJobs);
+}

package/src/commands/ci/init.ts

@@ -0,0 +1,96 @@
+import fs from "node:fs/promises";
+import path from "node:path";
+import chalk from "chalk";
+import YAML from "yaml";
+import { loadCIConfig } from "../../config/ci-loader.js";
+import type { CIConfig } from "../../config/types.js";
+import workflowTemplate from "../../templates/workflow.yml" with {
+	type: "text",
+};
+
+export async function initCI(): Promise<void> {
+	const workflowDir = path.join(process.cwd(), ".github", "workflows");
+	const workflowPath = path.join(workflowDir, "gauntlet.yml");
+	const gauntletDir = path.join(process.cwd(), ".gauntlet");
+	const ciConfigPath = path.join(gauntletDir, "ci.yml");
+
+	// 1. Ensure .gauntlet/ci.yml exists
+	if (!(await fileExists(ciConfigPath))) {
+		console.log(chalk.yellow("Creating starter .gauntlet/ci.yml..."));
+		await fs.mkdir(gauntletDir, { recursive: true });
+		const starterContent = `# CI Configuration for Agent Gauntlet
+# Define runtimes, services, and which checks to run in CI.
+
+runtimes:
+  # ruby:
+  #   version: "3.3"
+  #   bundler_cache: true
+
+services:
+  # postgres:
+  #   image: postgres:16
+  #   ports: ["5432:5432"]
+
+setup:
+  # - name: Global Setup
+  #   run: echo "Setting up..."
+
+checks:
+  # - name: linter
+  #   requires_runtimes: [ruby]
+`;
+		await fs.writeFile(ciConfigPath, starterContent);
+	} else {
+		console.log(chalk.dim("Found existing .gauntlet/ci.yml"));
+	}
+
+	// 2. Load CI config to get services
+	let ciConfig: CIConfig | undefined;
+	try {
+		ciConfig = await loadCIConfig();
+	} catch (_e) {
+		console.warn(
+			chalk.yellow(
+				"Could not load CI config to inject services. Workflow will have no services defined.",
+			),
+		);
+	}
+
+	// 3. Generate workflow file
+	console.log(chalk.dim(`Generating ${workflowPath}...`));
+	await fs.mkdir(workflowDir, { recursive: true });
+
+	let templateContent = workflowTemplate;
+
+	// Inject services
+	if (ciConfig?.services && Object.keys(ciConfig.services).length > 0) {
+		const servicesYaml = YAML.stringify({ services: ciConfig.services });
+		// Indent services
+		const indentedServices = servicesYaml
+			.split("\n")
+			.map((line) => (line.trim() ? `    ${line}` : line))
+			.join("\n");
+
+		templateContent = templateContent.replace(
+			"# Services will be injected here by agent-gauntlet",
+			indentedServices,
+		);
+	} else {
+		templateContent = templateContent.replace(
+			"    # Services will be injected here by agent-gauntlet\n",
+			"",
+		);
+	}
+
+	await fs.writeFile(workflowPath, templateContent);
+	console.log(chalk.green("Successfully generated GitHub Actions workflow!"));
+}
+
+async function fileExists(path: string): Promise<boolean> {
+	try {
+		const stat = await fs.stat(path);
+		return stat.isFile();
+	} catch {
+		return false;
+	}
+}

package/src/commands/ci/list-jobs.ts

@@ -0,0 +1,78 @@
+import { loadCIConfig } from "../../config/ci-loader.js";
+import { loadConfig } from "../../config/loader.js";
+import type { CISetupStep } from "../../config/types.js";
+import { EntryPointExpander } from "../../core/entry-point.js";
+
+export async function listJobs(): Promise<void> {
+	try {
+		const config = await loadConfig();
+		const ciConfig = await loadCIConfig();
+		const expander = new EntryPointExpander();
+		const expandedEntryPoints = await expander.expandAll(
+			config.project.entry_points,
+		);
+
+		const matrixJobs = [];
+
+		const globalSetup = formatSetup(ciConfig.setup || undefined);
+
+		if (ciConfig.checks) {
+			for (const ep of expandedEntryPoints) {
+				// Get checks enabled for this entry point
+				const allowedChecks = new Set(ep.config.checks || []);
+
+				for (const check of ciConfig.checks) {
+					if (allowedChecks.has(check.name)) {
+						// Check definition from .gauntlet/checks/*.yml
+						const checkDef = config.checks[check.name];
+						if (!checkDef) {
+							console.warn(
+								`Warning: Check '${check.name}' found in CI config but not defined in checks/*.yml`,
+							);
+							continue;
+						}
+
+						const id = `${check.name}-${ep.path.replace(/\//g, "-")}`;
+
+						matrixJobs.push({
+							id,
+							name: check.name,
+							entry_point: ep.path,
+							working_directory: checkDef.working_directory || ep.path,
+							command: checkDef.command,
+							runtimes: check.requires_runtimes || [],
+							services: check.requires_services || [],
+							setup: formatSetup(check.setup || undefined),
+							global_setup: globalSetup,
+						});
+					}
+				}
+			}
+		}
+
+		const output = {
+			matrix: matrixJobs,
+			services: ciConfig.services || {},
+			runtimes: ciConfig.runtimes || {},
+		};
+
+		console.log(JSON.stringify(output));
+	} catch (e) {
+		console.error("Error generating CI jobs:", e);
+		process.exit(1);
+	}
+}
+
+const formatSetup = (steps: CISetupStep[] | null | undefined): string => {
+	if (!steps || steps.length === 0) return "";
+	return steps
+		.map((s) => {
+			const cmd = s.working_directory
+				? `(cd "${s.working_directory}" && ${s.run})`
+				: s.run;
+			return `echo "::group::${s.name}"
+${cmd}
+echo "::endgroup::"`;
+		})
+		.join("\n");
+};

package/src/commands/detect.ts

@@ -11,6 +11,10 @@ export function registerDetectCommand(program: Command): void {
 		.description(
 			"Show what gates would run for detected changes (without executing them)",
 		)
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
 			"-u, --uncommitted",
@@ -19,7 +23,18 @@ export function registerDetectCommand(program: Command): void {
 		.action(async (options) => {
 			try {
 				const config = await loadConfig();
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});

package/src/commands/help.ts

@@ -24,6 +24,7 @@ export function registerHelpCommand(program: Command): void {
 			console.log("  list     List configured gates");
 			console.log("  health   Check CLI tool availability");
 			console.log("  init     Initialize .gauntlet configuration");
+			console.log("  ci       CI integration commands (init, list-jobs)");
 			console.log("  help     Show this help message\n");
 			console.log(
 				"For more information, see: https://github.com/your-repo/agent-gauntlet",

package/src/commands/index.ts

@@ -1,4 +1,5 @@
 export { registerCheckCommand } from "./check.js";
+export { registerCICommand } from "./ci/index.js";
 export { registerDetectCommand } from "./detect.js";
 export { registerHealthCommand } from "./health.js";
 export { registerHelpCommand } from "./help.js";

package/src/commands/init.test.ts

@@ -40,6 +40,8 @@ mock.module("../cli-adapters/index.js", () => ({
 	getAllAdapters: () => mockAdapters,
 	getProjectCommandAdapters: () => mockAdapters,
 	getUserCommandAdapters: () => [],
+	getAdapter: (name: string) => mockAdapters.find((a) => a.name === name),
+	getValidCLITools: () => mockAdapters.map((a) => a.name),
 }));
 
 // Import after mocking

package/src/commands/rerun.ts

@@ -19,6 +19,10 @@ export function registerRerunCommand(program: Command): void {
 		.description(
 			"Rerun gates (checks & reviews) with previous failures as context (defaults to uncommitted changes)",
 		)
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific gate only")
 		.option(
 			"-c, --commit <sha>",
@@ -71,6 +75,16 @@ export function registerRerunCommand(program: Command): void {
 				// Rotate logs before starting the new run
 				await rotateLogs(config.project.log_dir);
 
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
 				// Detect changes (default to uncommitted unless --commit is specified)
 				// Note: Rerun defaults to uncommitted changes for faster iteration loops,
 				// unlike 'run' which defaults to base_branch comparison.
@@ -80,7 +94,7 @@ export function registerRerunCommand(program: Command): void {
 				};
 
 				const changeDetector = new ChangeDetector(
-					config.project.base_branch,
+					effectiveBaseBranch,
 					changeOptions,
 				);
 				const expander = new EntryPointExpander();
@@ -132,6 +146,7 @@ export function registerRerunCommand(program: Command): void {
 					reporter,
 					failuresMap, // Pass previous failures map
 					changeOptions, // Pass change detection options
+					effectiveBaseBranch, // Pass effective base branch
 				);
 
 				const success = await runner.run(jobs);

package/src/commands/review.ts

@@ -13,6 +13,10 @@ export function registerReviewCommand(program: Command): void {
 	program
 		.command("review")
 		.description("Run only applicable reviews for detected changes")
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific review gate only")
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
@@ -26,7 +30,17 @@ export function registerReviewCommand(program: Command): void {
 				// Rotate logs before starting
 				await rotateLogs(config.project.log_dir);
 
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});
@@ -65,7 +79,14 @@ export function registerReviewCommand(program: Command): void {
 
 				const logger = new Logger(config.project.log_dir);
 				const reporter = new ConsoleReporter();
-				const runner = new Runner(config, logger, reporter);
+				const runner = new Runner(
+					config,
+					logger,
+					reporter,
+					undefined,
+					undefined,
+					effectiveBaseBranch,
+				);
 
 				const success = await runner.run(jobs);
 				process.exit(success ? 0 : 1);

package/src/commands/run.ts

@@ -13,6 +13,10 @@ export function registerRunCommand(program: Command): void {
 	program
 		.command("run")
 		.description("Run gates for detected changes")
+		.option(
+			"-b, --base-branch <branch>",
+			"Override base branch for change detection",
+		)
 		.option("-g, --gate <name>", "Run specific gate only")
 		.option("-c, --commit <sha>", "Use diff for a specific commit")
 		.option(
@@ -26,7 +30,17 @@ export function registerRunCommand(program: Command): void {
 				// Rotate logs before starting
 				await rotateLogs(config.project.log_dir);
 
-				const changeDetector = new ChangeDetector(config.project.base_branch, {
+				// Determine effective base branch
+				// Priority: CLI override > CI env var > config
+				const effectiveBaseBranch =
+					options.baseBranch ||
+					(process.env.GITHUB_BASE_REF &&
+					(process.env.CI === "true" || process.env.GITHUB_ACTIONS === "true")
+						? process.env.GITHUB_BASE_REF
+						: null) ||
+					config.project.base_branch;
+
+				const changeDetector = new ChangeDetector(effectiveBaseBranch, {
 					commit: options.commit,
 					uncommitted: options.uncommitted,
 				});
@@ -62,7 +76,14 @@ export function registerRunCommand(program: Command): void {
 
 				const logger = new Logger(config.project.log_dir);
 				const reporter = new ConsoleReporter();
-				const runner = new Runner(config, logger, reporter);
+				const runner = new Runner(
+					config,
+					logger,
+					reporter,
+					undefined,
+					undefined,
+					effectiveBaseBranch,
+				);
 
 				const success = await runner.run(jobs);
 				process.exit(success ? 0 : 1);

package/src/config/ci-loader.ts

@@ -0,0 +1,33 @@
+import fs from "node:fs/promises";
+import path from "node:path";
+import YAML from "yaml";
+import { ciConfigSchema } from "./ci-schema.js";
+import type { CIConfig } from "./types.js";
+
+const GAUNTLET_DIR = ".gauntlet";
+const CI_FILE = "ci.yml";
+
+export async function loadCIConfig(
+	rootDir: string = process.cwd(),
+): Promise<CIConfig> {
+	const ciPath = path.join(rootDir, GAUNTLET_DIR, CI_FILE);
+
+	if (!(await fileExists(ciPath))) {
+		throw new Error(
+			`CI configuration file not found at ${ciPath}. Run 'agent-gauntlet ci init' to create it.`,
+		);
+	}
+
+	const content = await fs.readFile(ciPath, "utf-8");
+	const raw = YAML.parse(content);
+	return ciConfigSchema.parse(raw);
+}
+
+async function fileExists(path: string): Promise<boolean> {
+	try {
+		const stat = await fs.stat(path);
+		return stat.isFile();
+	} catch {
+		return false;
+	}
+}

package/src/config/ci-schema.ts

@@ -0,0 +1,52 @@
+import { z } from "zod";
+
+export const runtimeConfigSchema = z.record(
+	z.string(),
+	z
+		.object({
+			version: z.string().min(1),
+			bundler_cache: z.boolean().optional(),
+		})
+		.passthrough(),
+);
+
+export const serviceConfigSchema = z.record(
+	z.string(),
+	z
+		.object({
+			image: z.string().min(1),
+			env: z.record(z.string()).optional(),
+			ports: z.array(z.string()).optional(),
+			options: z.string().optional(),
+			health_check: z
+				.object({
+					cmd: z.string().optional(),
+					interval: z.string().optional(),
+					timeout: z.string().optional(),
+					retries: z.number().optional(),
+				})
+				.optional(),
+		})
+		.passthrough(),
+);
+
+export const ciSetupStepSchema = z.object({
+	name: z.string().min(1),
+	run: z.string().min(1),
+	working_directory: z.string().optional(),
+	if: z.string().optional(),
+});
+
+export const ciCheckConfigSchema = z.object({
+	name: z.string().min(1),
+	requires_runtimes: z.array(z.string()).optional(),
+	requires_services: z.array(z.string()).optional(),
+	setup: z.array(ciSetupStepSchema).optional(),
+});
+
+export const ciConfigSchema = z.object({
+	runtimes: runtimeConfigSchema.nullable().optional(),
+	services: serviceConfigSchema.nullable().optional(),
+	setup: z.array(ciSetupStepSchema).nullable().optional(),
+	checks: z.array(ciCheckConfigSchema).nullable().optional(),
+});

package/src/config/schema.ts

@@ -11,7 +11,6 @@ export const checkGateSchema = z
 		command: z.string().min(1),
 		working_directory: z.string().optional(),
 		parallel: z.boolean().default(false),
-		run_in_ci: z.boolean().default(true),
 		run_locally: z.boolean().default(true),
 		timeout: z.number().optional(),
 		fail_fast: z.boolean().optional(),

package/src/config/types.ts

@@ -1,4 +1,11 @@
 import type { z } from "zod";
+import type {
+	ciCheckConfigSchema,
+	ciConfigSchema,
+	ciSetupStepSchema,
+	runtimeConfigSchema,
+	serviceConfigSchema,
+} from "./ci-schema.js";
 import type {
 	checkGateSchema,
 	cliConfigSchema,
@@ -17,6 +24,12 @@ export type EntryPointConfig = z.infer<typeof entryPointSchema>;
 export type GauntletConfig = z.infer<typeof gauntletConfigSchema>;
 export type CLIConfig = z.infer<typeof cliConfigSchema>;
 
+export type CIConfig = z.infer<typeof ciConfigSchema>;
+export type CICheckConfig = z.infer<typeof ciCheckConfigSchema>;
+export type CISetupStep = z.infer<typeof ciSetupStepSchema>;
+export type RuntimeConfig = z.infer<typeof runtimeConfigSchema>;
+export type ServiceConfig = z.infer<typeof serviceConfigSchema>;
+
 // Combined type for the fully loaded configuration
 export interface LoadedConfig {
 	project: GauntletConfig;

package/src/core/change-detector.ts

@@ -36,9 +36,9 @@ export class ChangeDetector {
 	}
 
 	private async getCIChangedFiles(): Promise<string[]> {
-		// In GitHub Actions, GITHUB_BASE_REF is the target branch (e.g., main)
-		// GITHUB_SHA is the commit being built
-		const baseRef = process.env.GITHUB_BASE_REF || this.baseBranch;
+		// In GitHub Actions, GITHUB_SHA is the commit being built
+		// Base branch priority is already resolved by caller
+		const baseRef = this.baseBranch;
 		const headRef = process.env.GITHUB_SHA || "HEAD";
 
 		// We might need to fetch first in some shallow clones, but assuming strictly for now

package/src/core/entry-point.ts

@@ -1,3 +1,4 @@
+import fs from "node:fs/promises";
 import path from "node:path";
 import type { EntryPointConfig } from "../config/types.js";
 
@@ -55,6 +56,31 @@ export class EntryPointExpander {
 		return results;
 	}
 
+	async expandAll(
+		entryPoints: EntryPointConfig[],
+	): Promise<ExpandedEntryPoint[]> {
+		const results: ExpandedEntryPoint[] = [];
+
+		for (const ep of entryPoints) {
+			if (ep.path === ".") {
+				results.push({ path: ".", config: ep });
+				continue;
+			}
+
+			if (ep.path.endsWith("*")) {
+				const parentDir = ep.path.slice(0, -2);
+				const subDirs = await this.listSubDirectories(parentDir);
+				for (const subDir of subDirs) {
+					results.push({ path: subDir, config: ep });
+				}
+			} else {
+				results.push({ path: ep.path, config: ep });
+			}
+		}
+
+		return results;
+	}
+
 	private async expandWildcard(
 		parentDir: string,
 		changedFiles: string[],
@@ -80,6 +106,17 @@ export class EntryPointExpander {
 		return Array.from(affectedSubDirs);
 	}
 
+	private async listSubDirectories(parentDir: string): Promise<string[]> {
+		try {
+			const dirents = await fs.readdir(parentDir, { withFileTypes: true });
+			return dirents
+				.filter((d) => d.isDirectory())
+				.map((d) => path.join(parentDir, d.name));
+		} catch {
+			return [];
+		}
+	}
+
 	private hasChangesInDir(dirPath: string, changedFiles: string[]): boolean {
 		// Check if any changed file starts with the dirPath
 		// Need to ensure exact match or subdirectory (e.g. "app" should not match "apple")

package/src/core/runner.ts

@@ -32,6 +32,7 @@ export class Runner {
 		private reporter: ConsoleReporter,
 		private previousFailuresMap?: Map<string, Map<string, PreviousViolation[]>>,
 		private changeOptions?: { commit?: string; uncommitted?: boolean },
+		private baseBranchOverride?: string,
 	) {}
 
 	async run(jobs: Job[]): Promise<boolean> {
@@ -89,12 +90,14 @@ export class Runner {
 			const safeJobId = sanitizeJobId(job.id);
 			const previousFailures = this.previousFailuresMap?.get(safeJobId);
 			const loggerFactory = this.logger.createLoggerFactory(job.id);
+			const effectiveBaseBranch =
+				this.baseBranchOverride || this.config.project.base_branch;
 			result = await this.reviewExecutor.execute(
 				job.id,
 				job.gateConfig as ReviewGateConfig & ReviewPromptFrontmatter,
 				job.entryPoint,
 				loggerFactory,
-				this.config.project.base_branch,
+				effectiveBaseBranch,
 				previousFailures,
 				this.changeOptions,
 				this.config.project.cli.check_usage_limit,

package/src/gates/review.test.ts

@@ -7,64 +7,112 @@ import type {
 	ReviewPromptFrontmatter,
 } from "../config/types.js";
 import { Logger } from "../output/logger.js";
-import { ReviewGateExecutor } from "./review.js";
+import type { ReviewGateExecutor } from "./review.js";
 
 const TEST_DIR = path.join(process.cwd(), `test-review-logs-${Date.now()}`);
-const LOG_DIR = path.join(TEST_DIR, "logs");
 
 describe("ReviewGateExecutor Logging", () => {
 	let logger: Logger;
 	let executor: ReviewGateExecutor;
+	let originalCI: string | undefined;
+	let originalGithubActions: string | undefined;
+	let originalCwd: string;
 
 	beforeEach(async () => {
 		await fs.mkdir(TEST_DIR, { recursive: true });
-		await fs.mkdir(LOG_DIR, { recursive: true });
-		logger = new Logger(LOG_DIR);
-		executor = new ReviewGateExecutor();
 
-		// Mock getAdapter
+		// Save and disable CI mode for this test to avoid complex git ref issues
+		originalCI = process.env.CI;
+		originalGithubActions = process.env.GITHUB_ACTIONS;
+		originalCwd = process.cwd();
+		delete process.env.CI;
+		delete process.env.GITHUB_ACTIONS;
+
+		// Change to test directory with its own git repo to avoid issues with the main repo
+		process.chdir(TEST_DIR);
+		// Initialize a minimal git repo for the test
+		const { exec } = await import("node:child_process");
+		const { promisify } = await import("node:util");
+		const execAsync = promisify(exec);
+		await execAsync("git init");
+		await execAsync('git config user.email "test@test.com"');
+		await execAsync('git config user.name "Test"');
+		// Create an initial commit so we have a history
+		await fs.writeFile("test.txt", "initial");
+		await execAsync("git add test.txt");
+		await execAsync('git commit -m "initial"');
+		// Create a "main" branch
+		await execAsync("git branch -M main");
+		// Create src directory for our test
+		await fs.mkdir("src", { recursive: true });
+		await fs.writeFile("src/test.ts", "test content");
+		await execAsync("git add src/test.ts");
+		await execAsync('git commit -m "add src"');
+
+		// Make uncommitted changes so the diff isn't empty
+		await fs.writeFile("src/test.ts", "modified test content");
+
+		// Now create the log directory and logger in the test directory
+		await fs.mkdir("logs", { recursive: true });
+		logger = new Logger(path.join(process.cwd(), "logs"));
+
+		// Create a factory function for mock adapters that returns the correct name
+		const createMockAdapter = (name: string): CLIAdapter =>
+			({
+				name,
+				isAvailable: async () => true,
+				checkHealth: async () => ({ status: "healthy" }),
+				// execute returns the raw string output from the LLM, which is then parsed by the executor.
+				// The real adapter returns a string. In this test, we return a JSON string to simulate
+				// the LLM returning structured data. This IS intentional and matches the expected contract
+				// where execute() -> Promise<string>.
+				execute: async () => {
+					await new Promise((r) => setTimeout(r, 1)); // Simulate async work
+					return JSON.stringify({ status: "pass", message: "OK" });
+				},
+				getProjectCommandDir: () => null,
+				getUserCommandDir: () => null,
+				getCommandExtension: () => "md",
+				canUseSymlink: () => false,
+				transformCommand: (c: string) => c,
+			}) as unknown as CLIAdapter;
+
+		// Mock getAdapter and other exports that may be imported by other modules
 		mock.module("../cli-adapters/index.js", () => ({
-			getAdapter: (name: string) =>
-				({
-					name,
-					isAvailable: async () => true,
-					checkHealth: async () => ({ status: "healthy" }),
-					// execute returns the raw string output from the LLM, which is then parsed by the executor.
-					// The real adapter returns a string. In this test, we return a JSON string to simulate
-					// the LLM returning structured data. This IS intentional and matches the expected contract
-					// where execute() -> Promise<string>.
-					execute: async () => {
-						await new Promise((r) => setTimeout(r, 1)); // Simulate async work
-						return JSON.stringify({ status: "pass", message: "OK" });
-					},
-					getProjectCommandDir: () => null,
-					getUserCommandDir: () => null,
-					getCommandExtension: () => "md",
-					canUseSymlink: () => false,
-					transformCommand: (c: string) => c,
-				}) as unknown as CLIAdapter,
+			getAdapter: (name: string) => createMockAdapter(name),
+			getAllAdapters: () => [
+				createMockAdapter("codex"),
+				createMockAdapter("claude"),
+			],
+			getProjectCommandAdapters: () => [
+				createMockAdapter("codex"),
+				createMockAdapter("claude"),
+			],
+			getUserCommandAdapters: () => [
+				createMockAdapter("codex"),
+				createMockAdapter("claude"),
+			],
+			getValidCLITools: () => ["codex", "claude", "gemini"],
 		}));
 
-		// Mock git commands via util.promisify(exec)
-		mock.module("node:util", () => ({
-			promisify: (fn: (...args: unknown[]) => unknown) => {
-				// Only mock exec, let others pass (though in this test env we likely only use exec)
-				if (fn.name === "exec") {
-					return async (cmd: string) => {
-						if (/^git diff/.test(cmd)) return "diff content";
-						if (/^git ls-files/.test(cmd)) return "file.ts";
-						return { stdout: "", stderr: "" };
-					};
-				}
-				// Fallback for other functions if needed
-				return async () => {};
-			},
-		}));
+		const { ReviewGateExecutor } = await import("./review.js");
+		executor = new ReviewGateExecutor();
 	});
 
 	afterEach(async () => {
+		// Restore working directory first
+		process.chdir(originalCwd);
+
 		await fs.rm(TEST_DIR, { recursive: true, force: true });
 		mock.restore();
+
+		// Restore CI env vars
+		if (originalCI !== undefined) {
+			process.env.CI = originalCI;
+		}
+		if (originalGithubActions !== undefined) {
+			process.env.GITHUB_ACTIONS = originalGithubActions;
+		}
 	});
 
 	it("should only create adapter-specific logs and no generic log", async () => {
@@ -89,39 +137,94 @@ describe("ReviewGateExecutor Logging", () => {
 			"main",
 		);
 
-		expect(result.status).toBe("pass");
-		expect(result.logPaths).toBeDefined();
-		expect(result.logPaths).toHaveLength(2);
-		expect(result.logPaths?.[0]).toContain("review_src_code-quality_codex.log");
-		expect(result.logPaths?.[1]).toContain(
-			"review_src_code-quality_claude.log",
-		);
+		// Enhanced error messages for better debugging
+		if (result.status !== "pass") {
+			throw new Error(
+				`Expected result.status to be "pass" but got "${result.status}". Message: ${result.message || "none"}. Duration: ${result.duration}ms`,
+			);
+		}
+
+		if (!result.logPaths) {
+			throw new Error(
+				`Expected result.logPaths to be defined but got ${JSON.stringify(result.logPaths)}`,
+			);
+		}
+
+		if (result.logPaths.length !== 2) {
+			throw new Error(
+				`Expected result.logPaths to have length 2 but got ${result.logPaths.length}. Paths: ${JSON.stringify(result.logPaths)}`,
+			);
+		}
+
+		if (!result.logPaths[0]?.includes("review_src_code-quality_codex.log")) {
+			throw new Error(
+				`Expected result.logPaths[0] to contain "review_src_code-quality_codex.log" but got "${result.logPaths[0]}"`,
+			);
+		}
+
+		if (!result.logPaths[1]?.includes("review_src_code-quality_claude.log")) {
+			throw new Error(
+				`Expected result.logPaths[1] to contain "review_src_code-quality_claude.log" but got "${result.logPaths[1]}"`,
+			);
+		}
+
+		const files = await fs.readdir("logs");
+		const filesList = files.join(", ");
+
+		if (!files.includes("review_src_code-quality_codex.log")) {
+			throw new Error(
+				`Expected log directory to contain "review_src_code-quality_codex.log" but only found: [${filesList}]`,
+			);
+		}
+
+		if (!files.includes("review_src_code-quality_claude.log")) {
+			throw new Error(
+				`Expected log directory to contain "review_src_code-quality_claude.log" but only found: [${filesList}]`,
+			);
+		}
 
-		const files = await fs.readdir(LOG_DIR);
-		expect(files).toContain("review_src_code-quality_codex.log");
-		expect(files).toContain("review_src_code-quality_claude.log");
-		expect(files).not.toContain("review_src_code-quality.log");
+		if (files.includes("review_src_code-quality.log")) {
+			throw new Error(
+				`Expected log directory NOT to contain generic log "review_src_code-quality.log" but it was found. All files: [${filesList}]`,
+			);
+		}
 
 		// Verify multiplexed content
 		const codexLog = await fs.readFile(
-			path.join(LOG_DIR, "review_src_code-quality_codex.log"),
+			"logs/review_src_code-quality_codex.log",
 			"utf-8",
 		);
-		expect(codexLog).toContain("Starting review: code-quality");
-		expect(codexLog).toContain("Review result (codex): pass");
+		if (!codexLog.includes("Starting review: code-quality")) {
+			throw new Error(
+				`Expected codex log to contain "Starting review: code-quality" but got: ${codexLog.substring(0, 200)}...`,
+			);
+		}
+		if (!codexLog.includes("Review result (codex): pass")) {
+			throw new Error(
+				`Expected codex log to contain "Review result (codex): pass" but got: ${codexLog.substring(0, 200)}...`,
+			);
+		}
 
 		const claudeLog = await fs.readFile(
-			path.join(LOG_DIR, "review_src_code-quality_claude.log"),
+			"logs/review_src_code-quality_claude.log",
 			"utf-8",
 		);
-		expect(claudeLog).toContain("Starting review: code-quality");
-		expect(claudeLog).toContain("Review result (claude): pass");
+		if (!claudeLog.includes("Starting review: code-quality")) {
+			throw new Error(
+				`Expected claude log to contain "Starting review: code-quality" but got: ${claudeLog.substring(0, 200)}...`,
+			);
+		}
+		if (!claudeLog.includes("Review result (claude): pass")) {
+			throw new Error(
+				`Expected claude log to contain "Review result (claude): pass" but got: ${claudeLog.substring(0, 200)}...`,
+			);
+		}
 	});
 
 	it("should be handled correctly by ConsoleReporter", async () => {
 		const jobId = "review:src:code-quality";
-		const codexPath = path.join(LOG_DIR, "review_src_code-quality_codex.log");
-		const claudePath = path.join(LOG_DIR, "review_src_code-quality_claude.log");
+		const codexPath = "logs/review_src_code-quality_codex.log";
+		const claudePath = "logs/review_src_code-quality_claude.log";
 
 		await fs.writeFile(
 			codexPath,

package/src/gates/review.ts

@@ -370,6 +370,13 @@ export class ReviewGateExecutor {
 		}
 
 		// Create per-adapter logger
+		// Defensive check: ensure adapter name is valid
+		if (!adapter.name || typeof adapter.name !== "string") {
+			await mainLogger(
+				`Error: Invalid adapter name: ${JSON.stringify(adapter.name)}\n`,
+			);
+			return null;
+		}
 		const adapterLogger = await getAdapterLogger(adapter.name);
 
 		try {
@@ -493,7 +500,8 @@ export class ReviewGateExecutor {
 		entryPointPath: string,
 		baseBranch: string,
 	): Promise<string> {
-		const baseRef = process.env.GITHUB_BASE_REF || baseBranch;
+		// Base branch priority is already resolved by caller
+		const baseRef = baseBranch;
 		const headRef = process.env.GITHUB_SHA || "HEAD";
 		const pathArg = this.pathArg(entryPointPath);
 

package/src/index.ts

@@ -3,6 +3,7 @@ import { Command } from "commander";
 import packageJson from "../package.json" with { type: "json" };
 import {
 	registerCheckCommand,
+	registerCICommand,
 	registerDetectCommand,
 	registerHealthCommand,
 	registerHelpCommand,
@@ -24,6 +25,7 @@ program
 registerRunCommand(program);
 registerRerunCommand(program);
 registerCheckCommand(program);
+registerCICommand(program);
 registerReviewCommand(program);
 registerDetectCommand(program);
 registerListCommand(program);

package/src/output/logger.ts

@@ -28,10 +28,12 @@ export class Logger {
 	}
 
 	private async initFile(logPath: string): Promise<void> {
-		if (!this.initializedFiles.has(logPath)) {
-			await fs.writeFile(logPath, "");
-			this.initializedFiles.add(logPath);
+		if (this.initializedFiles.has(logPath)) {
+			return;
 		}
+		// Add to set BEFORE writing to make this more atomic
+		this.initializedFiles.add(logPath);
+		await fs.writeFile(logPath, "");
 	}
 
 	async createJobLogger(

package/src/templates/workflow.yml

@@ -0,0 +1,77 @@
+name: Gauntlet CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  discover:
+    name: Discover Jobs
+    runs-on: ubuntu-latest
+    outputs:
+      matrix: ${{ steps.discover.outputs.matrix }}
+      runtimes: ${{ steps.discover.outputs.runtimes }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install agent-gauntlet
+        run: |
+          curl -fsSL https://bun.sh/install | bash
+          ~/.bun/bin/bun add -g agent-gauntlet
+
+      - name: Discover gauntlet jobs
+        id: discover
+        run: |
+          output=$(~/.bun/bin/agent-gauntlet ci list-jobs)
+          echo "matrix=$(echo "$output" | jq -c '.matrix')" >> $GITHUB_OUTPUT
+          echo "runtimes=$(echo "$output" | jq -c '.runtimes')" >> $GITHUB_OUTPUT
+
+  checks:
+    name: ${{ matrix.job.name }} (${{ matrix.job.entry_point }})
+    runs-on: ubuntu-latest
+    needs: discover
+    if: ${{ needs.discover.outputs.matrix != '[]' }}
+    strategy:
+      fail-fast: false
+      matrix:
+        job: ${{ fromJson(needs.discover.outputs.matrix) }}
+
+    # Services will be injected here by agent-gauntlet
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Ruby
+        if: contains(matrix.job.runtimes, 'ruby')
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: ${{ fromJson(needs.discover.outputs.runtimes).ruby.version }}
+          bundler-cache: ${{ fromJson(needs.discover.outputs.runtimes).ruby.bundler_cache }}
+          working-directory: ${{ matrix.job.working_directory }}
+
+      - name: Set up Node
+        if: contains(matrix.job.runtimes, 'node')
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ fromJson(needs.discover.outputs.runtimes).node.version }}
+
+      - name: Set up Bun
+        if: contains(matrix.job.runtimes, 'bun')
+        uses: oven-sh/setup-bun@v1
+        with:
+          bun-version: ${{ fromJson(needs.discover.outputs.runtimes).bun.version }}
+
+      - name: Run global setup
+        if: ${{ matrix.job.global_setup != '' }}
+        run: ${{ matrix.job.global_setup }}
+
+      - name: Run check setup
+        if: ${{ matrix.job.setup != '' }}
+        working-directory: ${{ matrix.job.working_directory }}
+        run: ${{ matrix.job.setup }}
+
+      - name: Run check
+        working-directory: ${{ matrix.job.working_directory }}
+        run: ${{ matrix.job.command }}