pipeline-sdk 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025-present Anshul
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,146 @@
+# pipeline-sdk
+
+YAML-driven pipeline orchestration for AI agents.
+
+[![npm version](https://img.shields.io/npm/v/pipeline-sdk)](https://www.npmjs.com/package/pipeline-sdk)
+[![CI](https://github.com/anshul-homelab/pipeline-sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/anshul-homelab/pipeline-sdk/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+
+## Why?
+
+AI coding agents (Claude Code, Cursor, Codex) execute multi-step workflows -- write code, run tests, deploy -- but lack a structured way to enforce stage gates, track evidence, or coordinate handoffs. pipeline-sdk gives your agent a state machine: define stages in YAML, gate transitions on passing tests, and keep a cryptographic audit trail of what happened.
+
+## What is this?
+
+Define multi-stage pipelines in YAML with gates, evidence chains, and cleanup handlers. The SDK provides a state machine engine, HTTP daemon, CLI, and MCP server to orchestrate AI agent workflows. Pipelines are declarative, auditable, and resumable.
+
+## Quick Start
+
+```bash
+# Install
+bun add pipeline-sdk
+
+# Initialize a pipeline from a template
+bunx pipeline init
+
+# Start the daemon
+bunx pipeline start
+
+# In another terminal -- check status
+bunx pipeline status
+
+# Advance to the next stage
+bunx pipeline advance STAGE_COMPLETE
+```
+
+## Architecture
+
+```
+pipeline.yaml --> StateMachine --> PipelineDaemon (HTTP localhost:<port>)
+                                        |
+                                  MCP Server <--> AI Agent
+                                        |
+                                  Evidence Chain
+```
+
+The YAML file defines stages and transitions. The state machine enforces the rules. The daemon exposes an HTTP API. The MCP server lets AI agents participate natively. Every transition is recorded in a tamper-evident evidence chain.
+
+## Key Concepts
+
+**Stages** -- Named phases of your pipeline (e.g., `dev`, `review`, `deploy`). Each stage can define entry/exit actions, gates, and cleanup handlers.
+
+**Events and Transitions** -- Events like `DEV_DONE` trigger transitions between stages. Transitions can be conditional, requiring gates to pass before proceeding.
+
+**Gates** -- Preconditions that must be satisfied before a transition completes:
+- *Builtin* -- automated checks (lint, test, typecheck)
+- *Custom* -- shell commands or scripts
+- *Async* -- external signals (human approval, CI completion)
+
+**Evidence Chain** -- A cryptographically linked log of every transition, gate result, and signal. Provides a verifiable audit trail for the entire pipeline run.
+
+**Adapters** -- Pluggable system for integrating with external tools (CI systems, deployment platforms, notification services).
+
+## CLI Reference
+
+| Command | Description |
+|---------|-------------|
+| `pipeline init` | Initialize a new pipeline from a template |
+| `pipeline start` | Start the pipeline daemon |
+| `pipeline status` | Show current stage and pipeline state |
+| `pipeline advance <event>` | Emit an event to trigger a transition |
+| `pipeline signal <gate>` | Signal an async gate as satisfied |
+| `pipeline verify` | Verify evidence chain integrity |
+| `pipeline resume` | Resume pipeline from saved state |
+| `pipeline cleanup` | Run cleanup handlers and stop the daemon |
+| `pipeline validate` | Validate pipeline.yaml against the schema |
+| `pipeline visualize` | Generate a Mermaid state diagram |
+| `pipeline template [name]` | List or apply a built-in template |
+
+## Built-in Templates
+
+| Template | Description |
+|----------|-------------|
+| `sdlc-full` | Complete software development lifecycle with review and deploy gates |
+| `static-site` | Web portfolio pipeline with accessibility gates |
+| `zship` | Dual-model development workflow (Claude + Codex) |
+| `infra-gitops` | Infrastructure deployment with approval gates and rollback |
+
+## MCP Integration
+
+The SDK exposes an MCP server so AI agents can participate in pipelines natively:
+
+- **4 tools** -- `check`, `advance`, `signal`, `status`
+- **3 resources** -- current state, stage instructions, gate status
+- **1 prompt** -- stage context for the active phase
+
+The MCP server is created programmatically via `createMcpServer()` and communicates over stdio. See [MCP Integration](docs/mcp-integration.md) for setup details. AI agents like Claude Code can then query pipeline state, advance stages, and signal gates through the standard MCP protocol.
+
+## API Usage
+
+```typescript
+import { loadPipeline, StateMachine } from "pipeline-sdk";
+
+const pipeline = await loadPipeline("pipeline.yaml");
+const sm = new StateMachine(pipeline);
+
+console.log(sm.currentStage); // "dev"
+
+const result = sm.transition("DEV_DONE");
+console.log(result.newStage); // "review"
+```
+
+### Evidence Store
+
+```typescript
+import { EvidenceStore } from "pipeline-sdk";
+
+const store = new EvidenceStore(".pipeline/evidence");
+const result = await store.verify();
+console.log(result.valid, result.recordCount);
+```
+
+## Documentation
+
+- [Getting Started](docs/getting-started.md)
+- [CLI Reference](docs/cli-reference.md)
+- [Pipeline YAML Reference](docs/pipeline-yaml-reference.md)
+- [Architecture](docs/architecture.md)
+- [MCP Integration](docs/mcp-integration.md)
+
+## Development
+
+```bash
+bun install
+bun test              # 284 tests
+bun run lint          # Biome
+bun run typecheck     # TypeScript
+bun run build         # Compile to binary
+```
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, commit conventions, and PR guidelines.
+
+## License
+
+[MIT](LICENSE)

package/package.json

@@ -0,0 +1,63 @@
+{
+	"name": "pipeline-sdk",
+	"version": "0.1.0",
+	"description": "YAML-driven pipeline orchestration SDK with CLI, daemon, MCP server, and adapter system",
+	"module": "index.ts",
+	"type": "module",
+	"author": "Anshul",
+	"license": "MIT",
+	"repository": {
+		"type": "git",
+		"url": "https://github.com/anshul-homelab/pipeline-sdk.git"
+	},
+	"homepage": "https://github.com/anshul-homelab/pipeline-sdk",
+	"bugs": {
+		"url": "https://github.com/anshul-homelab/pipeline-sdk/issues"
+	},
+	"keywords": [
+		"pipeline",
+		"orchestration",
+		"yaml",
+		"state-machine",
+		"mcp",
+		"ai-agent",
+		"cli",
+		"daemon",
+		"gates",
+		"evidence-chain"
+	],
+	"files": [
+		"src/",
+		"schemas/",
+		"templates/",
+		"README.md",
+		"LICENSE"
+	],
+	"bin": {
+		"pipeline": "./src/cli/index.ts"
+	},
+	"scripts": {
+		"dev": "bun run src/cli/index.ts",
+		"test": "bun test",
+		"lint": "bunx biome check src/",
+		"format": "bunx biome format --write src/",
+		"typecheck": "bunx tsc --noEmit",
+		"build": "bun build --compile src/cli/index.ts --outfile dist/pipeline"
+	},
+	"devDependencies": {
+		"@biomejs/biome": "^2.4.10",
+		"@commitlint/cli": "^20.5.0",
+		"@commitlint/config-conventional": "^20.5.0",
+		"@types/bun": "latest",
+		"lefthook": "^2.1.4"
+	},
+	"peerDependencies": {
+		"typescript": "^5"
+	},
+	"dependencies": {
+		"@modelcontextprotocol/sdk": "^1.29.0",
+		"ajv": "^8.18.0",
+		"commander": "^14.0.3",
+		"yaml": "^2.8.3"
+	}
+}

package/schemas/pipeline.schema.json

@@ -0,0 +1,158 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "title": "Pipeline Definition",
+  "type": "object",
+  "required": ["pipeline"],
+  "additionalProperties": false,
+  "properties": {
+    "pipeline": {
+      "type": "object",
+      "required": ["id", "version", "initial", "stages"],
+      "additionalProperties": false,
+      "properties": {
+        "id": { "type": "string", "minLength": 1 },
+        "version": { "type": "number" },
+        "description": { "type": "string" },
+        "initial": { "type": "string", "minLength": 1 },
+        "execution": {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "provider": { "type": "string" },
+            "mode": { "type": "string", "enum": ["daemon", "binary"] },
+            "timeout": { "type": "string" },
+            "evidence_dir": { "type": "string" },
+            "webhook_port": { "type": "number" },
+            "webhook_secret": { "type": "string" }
+          }
+        },
+        "stages": {
+          "type": "object",
+          "minProperties": 1,
+          "additionalProperties": {
+            "$ref": "#/definitions/stage"
+          }
+        },
+        "gates": {
+          "type": "object",
+          "additionalProperties": {
+            "$ref": "#/definitions/gate"
+          }
+        },
+        "conditions": {
+          "type": "object",
+          "additionalProperties": {
+            "$ref": "#/definitions/condition"
+          }
+        }
+      }
+    }
+  },
+  "definitions": {
+    "stage": {
+      "type": "object",
+      "additionalProperties": false,
+      "properties": {
+        "description": { "type": "string" },
+        "type": { "type": "string", "enum": ["terminal", "parallel-child"] },
+        "agent": {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "instructions": { "type": "string" },
+            "tools": { "type": "array", "items": { "type": "string" } },
+            "model": { "type": "string" },
+            "mode": { "type": "string" }
+          }
+        },
+        "gates": {
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "entry": { "type": "array", "items": { "type": "string" } },
+            "exit": { "type": "array", "items": { "type": "string" } }
+          }
+        },
+        "on": {
+          "type": "object",
+          "additionalProperties": {
+            "$ref": "#/definitions/transition"
+          }
+        },
+        "parallel": { "type": "array", "items": { "type": "string" } },
+        "when": { "type": "string" },
+        "max_retries": { "type": "number", "minimum": 0 },
+        "cleanup": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["script"],
+            "additionalProperties": false,
+            "properties": {
+              "script": { "type": "string" },
+              "inputs": { "type": "object" }
+            }
+          }
+        },
+        "on_enter": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["script"],
+            "additionalProperties": false,
+            "properties": {
+              "script": { "type": "string" },
+              "inputs": { "type": "object" }
+            }
+          }
+        },
+        "trigger": {
+          "type": "object",
+          "required": ["type", "cron"],
+          "additionalProperties": false,
+          "properties": {
+            "type": { "type": "string", "enum": ["schedule"] },
+            "cron": { "type": "string" }
+          }
+        }
+      }
+    },
+    "transition": {
+      "type": "object",
+      "required": ["target"],
+      "additionalProperties": false,
+      "properties": {
+        "target": { "type": "string", "minLength": 1 },
+        "guard": { "type": "string" }
+      }
+    },
+    "gate": {
+      "type": "object",
+      "required": ["type"],
+      "additionalProperties": false,
+      "properties": {
+        "type": { "type": "string", "enum": ["builtin", "custom", "async"] },
+        "command": { "type": "string" },
+        "script": { "type": "string" },
+        "protocol": { "type": "string" },
+        "signal": { "type": "string" },
+        "timeout": { "type": "string" },
+        "escalate_after": { "type": "string" },
+        "expect": { "type": "object" },
+        "inputs": { "type": "object" },
+        "when": { "type": "string" }
+      }
+    },
+    "condition": {
+      "type": "object",
+      "required": ["check"],
+      "additionalProperties": false,
+      "properties": {
+        "check": { "type": "string" },
+        "pattern": { "type": "string" },
+        "path": { "type": "string" },
+        "not": { "type": "string" }
+      }
+    }
+  }
+}

package/src/adapters/claude-code.ts

@@ -0,0 +1,112 @@
+import { execSync } from "node:child_process";
+import { mkdir, readFile, writeFile } from "node:fs/promises";
+import { dirname, join } from "node:path";
+import type { ProviderCapabilities, StageOutput, StageRequest } from "../types";
+import type { ProviderAdapter } from "./interface";
+
+export class ClaudeCodeAdapter implements ProviderAdapter {
+	getCapabilities(): ProviderCapabilities {
+		return {
+			name: "claude-code",
+			hooks: true,
+			parallel: true,
+			session: true,
+			headless: true,
+			mcp: true,
+		};
+	}
+
+	async invoke(request: StageRequest): Promise<StageOutput> {
+		const start = Date.now();
+		const prompt = [
+			`Stage: ${request.stage_id}`,
+			`Instructions: ${request.instructions}`,
+			`Tools: ${request.tools.join(", ")}`,
+			`Context: ${JSON.stringify(request.context)}`,
+		].join("\n");
+
+		let stdout = "";
+		let exit_code = 0;
+
+		try {
+			stdout = execSync(`claude --print ${JSON.stringify(prompt)}`, {
+				encoding: "utf-8",
+				timeout: request.timeout ?? 300_000,
+			});
+		} catch (err: unknown) {
+			exit_code = (err as NodeJS.ErrnoException & { status?: number }).status ?? 1;
+			stdout = (err as NodeJS.ErrnoException & { stdout?: string }).stdout ?? "";
+		}
+
+		return {
+			artifacts: [],
+			stdout,
+			exit_code,
+			duration_ms: Date.now() - start,
+		};
+	}
+
+	async installHook(config: { socketPath?: string; httpPort?: number }): Promise<() => void> {
+		const settingsPath = join(process.cwd(), ".claude", "settings.json");
+
+		let settings: Record<string, unknown> = {};
+		try {
+			const raw = await readFile(settingsPath, "utf-8");
+			settings = JSON.parse(raw) as Record<string, unknown>;
+		} catch {
+			// File may not exist yet — start fresh
+		}
+
+		const hookEntry: Record<string, unknown> = {
+			type: "pipeline-sdk",
+			...(config.socketPath ? { socketPath: config.socketPath } : {}),
+			...(config.httpPort !== undefined ? { httpPort: config.httpPort } : {}),
+		};
+
+		const existing = (settings.hooks as Record<string, unknown> | undefined) ?? {};
+		const preToolUse = (existing.PreToolUse as unknown[]) ?? [];
+
+		const alreadyInstalled = preToolUse.some(
+			(h) => (h as Record<string, unknown>).type === "pipeline-sdk",
+		);
+
+		if (!alreadyInstalled) {
+			const updated = {
+				...settings,
+				hooks: {
+					...existing,
+					PreToolUse: [...preToolUse, hookEntry],
+				},
+			};
+			await mkdir(dirname(settingsPath), { recursive: true });
+			await writeFile(settingsPath, JSON.stringify(updated, null, 2), "utf-8");
+		}
+
+		return async () => {
+			let current: Record<string, unknown> = {};
+			try {
+				const raw = await readFile(settingsPath, "utf-8");
+				current = JSON.parse(raw) as Record<string, unknown>;
+			} catch {
+				return;
+			}
+
+			const hooks = (current.hooks as Record<string, unknown> | undefined) ?? {};
+			const list = (hooks.PreToolUse as unknown[]) ?? [];
+			const filtered = list.filter((h) => (h as Record<string, unknown>).type !== "pipeline-sdk");
+
+			await writeFile(
+				settingsPath,
+				JSON.stringify(
+					{
+						...current,
+						hooks: { ...hooks, PreToolUse: filtered },
+					},
+					null,
+					2,
+				),
+				"utf-8",
+			);
+		};
+	}
+}

package/src/adapters/detector.ts

@@ -0,0 +1,26 @@
+import { access } from "node:fs/promises";
+import { join } from "node:path";
+
+const PROVIDER_DIRS: Array<{ dir: string; name: string }> = [
+	{ dir: ".claude", name: "claude-code" },
+	{ dir: ".cursor", name: "cursor" },
+	{ dir: ".gemini", name: "gemini-cli" },
+	{ dir: ".codex", name: "codex-cli" },
+	{ dir: ".windsurf", name: "windsurf" },
+	{ dir: ".vscode", name: "vscode" },
+];
+
+export async function detectProviders(cwd: string): Promise<string[]> {
+	const detected: string[] = [];
+
+	for (const { dir, name } of PROVIDER_DIRS) {
+		try {
+			await access(join(cwd, dir));
+			detected.push(name);
+		} catch {
+			// directory not present — skip
+		}
+	}
+
+	return detected;
+}

package/src/adapters/generic.ts

@@ -0,0 +1,30 @@
+import type { ProviderCapabilities, StageOutput, StageRequest } from "../types";
+import type { ProviderAdapter } from "./interface";
+
+export class GenericAdapter implements ProviderAdapter {
+	getCapabilities(): ProviderCapabilities {
+		return {
+			name: "generic",
+			hooks: false,
+			parallel: false,
+			session: false,
+			headless: true,
+			mcp: true,
+		};
+	}
+
+	async invoke(request: StageRequest): Promise<StageOutput> {
+		const start = Date.now();
+		return {
+			artifacts: [],
+			stdout: `Generic adapter: stage "${request.stage_id}" invoked. No provider-specific execution available.`,
+			exit_code: 0,
+			duration_ms: Date.now() - start,
+		};
+	}
+
+	async installHook(_config: { socketPath?: string; httpPort?: number }): Promise<() => void> {
+		// Generic adapter has no hook mechanism — no-op
+		return async () => {};
+	}
+}

package/src/adapters/interface.ts

@@ -0,0 +1,7 @@
+import type { ProviderCapabilities, StageOutput, StageRequest } from "../types";
+
+export interface ProviderAdapter {
+	getCapabilities(): ProviderCapabilities;
+	invoke(request: StageRequest): Promise<StageOutput>;
+	installHook(config: { socketPath?: string; httpPort?: number }): Promise<() => void>;
+}

package/src/cli/advance.ts

@@ -0,0 +1,27 @@
+import { fetchDaemon, readDaemonPid } from "./helpers";
+
+export async function advanceCommand(opts: { file: string; event: string }): Promise<void> {
+	const { port } = await readDaemonPid();
+	const res = await fetchDaemon(port, "/api/advance", {
+		method: "POST",
+		headers: { "Content-Type": "application/json" },
+		body: JSON.stringify({ event: opts.event }),
+	});
+	const result = (await res.json()) as {
+		transitioned: boolean;
+		new_stage?: string;
+		error?: string;
+	};
+
+	if (result.transitioned) {
+		console.log(`Advanced to stage: ${result.new_stage}`);
+		process.exit(0);
+	}
+
+	console.error(`Error: Transition failed for event "${opts.event}".`);
+	console.error("");
+	if (result.error) console.error(`  ${result.error}`);
+	console.error("");
+	console.error("  Check allowed events with: pipeline status");
+	process.exit(1);
+}

package/src/cli/cleanup.ts

@@ -0,0 +1,55 @@
+import { CleanupManager } from "../core/cleanup";
+import { loadPipeline } from "../core/loader";
+import { StateFile } from "../daemon/state-file";
+import type { PipelineDefinition } from "../types";
+
+export async function cleanupCommand(opts: { file: string }): Promise<void> {
+	let pipeline: PipelineDefinition;
+	try {
+		pipeline = await loadPipeline(opts.file);
+	} catch (err: unknown) {
+		console.error(`Error: Failed to load pipeline from ${opts.file}`);
+		console.error("");
+		if ((err as NodeJS.ErrnoException).code === "ENOENT") {
+			console.error("  File not found. Are you in the right directory?");
+		} else {
+			console.error(`  ${err instanceof Error ? err.message : String(err)}`);
+		}
+		process.exit(1);
+	}
+
+	const dir = ".pipeline";
+	const stateFile = new StateFile(`${dir}/state.json`);
+	const state = await stateFile.read();
+
+	const manager = new CleanupManager();
+	const completedStages = state?.stages_completed ?? [];
+	for (const stageId of completedStages) {
+		const stageDef = pipeline.stages[stageId];
+		if (stageDef?.cleanup) {
+			manager.register(stageId, stageDef.cleanup);
+		}
+	}
+
+	const results = await manager.runAll(process.cwd());
+	for (const r of results) {
+		const icon = r.success ? "OK" : "FAIL";
+		console.log(`  [${icon}] ${r.stage}: ${r.script}${r.error ? ` — ${r.error}` : ""}`);
+	}
+
+	// Kill daemon if running
+	const pidFile = Bun.file(`${dir}/daemon.pid`);
+	if (await pidFile.exists()) {
+		try {
+			const { pid } = JSON.parse(await pidFile.text()) as { port: number; pid: number };
+			process.kill(pid);
+		} catch {
+			// daemon already gone
+		}
+		const { unlink } = await import("node:fs/promises");
+		await unlink(`${dir}/daemon.pid`).catch(() => {});
+	}
+
+	console.log("Cleanup complete");
+	process.exit(0);
+}