@ryanfw/prompt-orchestration-pipeline 0.0.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Ryan Mahoney
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,290 @@
+# Pipeline Orchestrator (Prompt‑Orchestration Pipeline)
+
+A **Prompt‑orchestration pipeline (POP)** is a framework for building, running, and experimenting with complex chains of LLM tasks.
+
+Instead of relying on a single mega‑prompt, a pipeline decomposes work into stages, applies targeted transformations, validates outputs, and composes multiple model calls into a repeatable workflow.
+
+This repository provides a reference implementation of a prompt‑orchestration pipeline that can be consumed as an npm package by other Node.js projects. It is intentionally lightweight: just enough orchestration to run complex pipelines, inspect intermediate artifacts, and evolve new strategies.
+
+---
+
+## Why it matters
+
+Single‑prompt strategies are fragile:
+
+- Inputs must fit within a single context window.
+- Instructions and examples compete for limited space.
+- Quality control is all‑or‑nothing.
+
+A prompt‑orchestration pipeline changes the game:
+
+- **Chained reasoning** – break down complex problems into sequential tasks.
+- **Context compression & stacking** – condense outputs into artifacts that feed the next stage.
+- **Multi‑model strategies** – route subtasks to the most appropriate model (fast vs. large, cheap vs. accurate).
+- **Validation loops** – enforce structure, apply quality checks, and retry when needed.
+- **Experimentation** – swap tasks in and out to try new ideas without rewriting the whole system.
+
+The result: workflows that are **more robust, interpretable, and capable** than any single prompt.
+
+---
+
+## Architecture (conceptual)
+
+A prompt‑orchestration pipeline has **two layers**:
+
+### 1) Pipeline orchestration (outer layer)
+
+The outer pipeline manages runs, state, and isolation. It is responsible for:
+
+- Assigning a pipeline run ID for each new submission.
+- Creating predictable directories for pending seeds, active runs, and completed runs.
+- Spawning isolated processes for each task (so one failure doesn’t crash others).
+- Tracking progress in a run‑scoped status file.
+- Promoting completed runs into a repository of results with audit metadata.
+
+**Runtime directories (in the consuming project):**
+
+```
+my-project/
+└── pipeline-data/
+    ├── pending/           # queue seeds here (e.g., *.json)
+    ├── current/           # active run state (auto‑managed)
+    └── complete/          # archived runs (auto‑managed)
+```
+
+**High‑level flow**
+
+```mermaid
+flowchart TD
+  A["pipeline-data/pending/*-seed.json"] --> B[Orchestrator]
+  B --> C["create pipeline-data/current/<id>/seed.json"]
+  B --> D["init pipeline-data/current/<id>/tasks-status.json"]
+  B --> E[Read pipeline-config/pipeline.json]
+  E --> F[Spawn task runner]
+  F --> G["write tasks/<task>/letter.json"]
+  G --> H[Run task inner pipeline]
+  H --> I["write tasks/<task>/output.json"]
+  I --> J[Update tasks-status.json]
+  J --> K{More tasks?}
+  K -->|yes| F
+  K -->|no| L[Promote to complete]
+  L --> M["pipeline-data/complete/<id>/**"]
+  L --> N["append pipeline-data/complete/runs.jsonl"]
+```
+
+### 2) Task orchestration (inner layer)
+
+Each pipeline step runs through a **task runner** that executes canonical sub‑steps:
+
+1. **Ingestion** – retrieve existing data or context.
+2. **Pre‑processing** – compress or transform input to fit model constraints.
+3. **Prompt templating** – assemble the instruction.
+4. **Inference** – run the model call(s).
+5. **Parsing** – normalize outputs into structured form.
+6. **Validation** – check schema, quality, and semantic correctness.
+7. **Critique & refinement** – generate hints, re‑prompt, and retry if needed.
+8. **Finalization** – confirm valid output and persist artifacts.
+
+```mermaid
+flowchart TD
+  S[Start task] --> I1[Ingestion]
+  I1 --> P1[Pre‑processing]
+  P1 --> T1[Prompt templating]
+  T1 --> INF[Inference]
+  INF --> PAR[Parsing]
+  PAR --> VS[Validate structure]
+  VS -->|ok| VQ[Validate quality]
+  VS -->|fail| ERR[Fail task and log]
+  VQ -->|ok| FIN[Finalize & persist]
+  VQ -->|fail| HINTS[Critique & hints]
+  HINTS --> T1
+  FIN --> DONE[Done]
+  ERR --> DONE
+```
+
+---
+
+## Section A — Library (this package)
+
+### Repository layout
+
+```
+@ryan-fw/prompt-orchestration-pipeline/
+├── src/
+│   ├── core/
+│   │   ├── task-runner.js         # Core pipeline execution
+│   │   ├── pipeline-runner.js     # Pipeline management
+│   │   └── orchestrator.js        # Workflow orchestration
+│   ├── cli/
+│   │   └── index.js               # CLI entry point
+│   ├── api/
+│   │   └── index.js               # Programmatic API
+│   └── ui/
+│       └── server.js              # Optional UI server
+├── bin/
+│   └── pipeline-orchestrator      # CLI executable
+├── package.json
+└── README.md
+```
+
+### Package exports & CLI
+
+```json
+{
+  "name": "@ryan-fw/prompt-orchestration-pipeline",
+  "version": "1.0.0",
+  "type": "module",
+  "exports": {
+    ".": "./src/api/index.js",
+    "./cli": "./src/cli/index.js",
+    "./runner": "./src/core/task-runner.js"
+  },
+  "bin": {
+    "pipeline-orchestrator": "./bin/pipeline-orchestrator"
+  },
+  "dependencies": {
+    "chokidar": "^3.5.3",
+    "commander": "^11.0.0",
+    "express": "^4.18.0"
+  }
+}
+```
+
+- **CLI name:** `pipeline-orchestrator`
+- **Programmatic API:** import from `@ryan-fw/prompt-orchestration-pipeline` (see `src/api/index.js`).
+- **Task runner (advanced):** `@ryan-fw/prompt-orchestration-pipeline/runner`.
+
+---
+
+## Section B — Consuming project usage
+
+### Expected layout in a consumer project
+
+```
+my-project/
+├── pipeline-config/
+│   ├── pipeline.json              # Pipeline definition (ordered list of task IDs)
+│   └── tasks/                     # Task implementations
+│       ├── index.js               # Task registry (maps task IDs → modules)
+│       ├── task-a/
+│       │   └── index.js
+│       └── task-b/
+│           └── index.js
+├── pipeline-data/                 # Runtime directories (auto‑created/managed)
+│   ├── pending/
+│   ├── current/
+│   └── complete/
+├── package.json
+└── .pipelinerc.json              # Optional CLI config
+```
+
+**`pipeline.json` (example)**
+
+```json
+{
+  "tasks": ["task-a", "task-b"]
+}
+```
+
+**`pipeline-config/tasks/index.js` (example registry)**
+
+```js
+// ESM registry mapping task IDs to loader functions or modules
+export default {
+  "task-a": () => import("./task-a/index.js"),
+  "task-b": () => import("./task-b/index.js"),
+};
+```
+
+> The orchestrator resolves task IDs from `pipeline.json` using this registry.
+
+### Install & scripts
+
+Add the package and scripts to your consumer project:
+
+```json
+{
+  "scripts": {
+    "pipeline": "pipeline-orchestrator start",
+    "pipeline:ui": "pipeline-orchestrator start --ui",
+    "pipeline:init": "pipeline-orchestrator init",
+    "pipeline:submit": "pipeline-orchestrator submit"
+  },
+  "dependencies": {
+    "@ryan-fw/prompt-orchestration-pipeline": "^1.0.0"
+  }
+}
+```
+
+### CLI overview
+
+- **`pipeline-orchestrator init`** – scaffolds `pipeline-config/` and `pipeline-data/` if missing.
+- **`pipeline-orchestrator start`** – starts the orchestrator; watches `pipeline-data/pending/` for new seeds and processes them according to `pipeline-config/pipeline.json`.
+- **`pipeline-orchestrator start --ui`** – starts the orchestrator and the optional UI server.
+- **`pipeline-orchestrator submit [path]`** – submits a seed into `pipeline-data/pending/` (path can point to a JSON file).
+
+> Run `pipeline-orchestrator --help` in your project for the most current flags.
+
+### Optional configuration: `.pipelinerc.json`
+
+If present in the project root, this file can provide defaults for the CLI (e.g., custom locations). A minimal example:
+
+```json
+{
+  "configDir": "./pipeline-config",
+  "dataDir": "./pipeline-data"
+}
+```
+
+_(Keys and defaults may vary by version; prefer `--help` for authoritative options.)_
+
+### Example flow in a consumer project
+
+1. **Initialize**: `npm run pipeline:init` to ensure folders exist.
+2. **Define**: Edit `pipeline-config/pipeline.json` and implement tasks under `pipeline-config/tasks/`.
+3. **Run**: `npm run pipeline` (or `npm run pipeline:ui` for the UI).
+4. **Submit**: Add a seed JSON to `pipeline-data/pending/` or run `npm run pipeline:submit -- ./path/to/seed.json`.
+5. **Inspect**: Watch `pipeline-data/current/<runId>` for in‑progress artifacts and `pipeline-data/complete/<runId>` for results.
+
+---
+
+## Concepts & conventions (carry‑overs)
+
+- **Determinism** – each task persists its inputs/outputs; you can re‑run or debug any stage.
+- **Isolation** – tasks run in separate processes when appropriate.
+- **Artifacts** – tasks write structured artifacts (e.g., `letter.json`, `output.json`) to their run directory.
+- **Status** – a `tasks-status.json` file tracks progress and outcomes across the pipeline.
+
+---
+
+## Quick troubleshooting
+
+- **Nothing happens when I submit a seed** → Ensure the orchestrator is running and watching `pipeline-data/pending/`.
+- **Task not found** → Confirm the task ID exists in `pipeline-config/tasks/index.js` and matches `pipeline.json`.
+- **UI doesn’t load** → Try `pipeline-orchestrator start --ui` and check for port conflicts.
+
+---
+
+## Getting started (TL;DR)
+
+```bash
+# 1) Install
+npm i -S @ryan-fw/prompt-orchestration-pipeline
+
+# 2) Initialize scaffold
+npm run pipeline:init
+
+# 3) Start orchestrator (optionally with UI)
+npm run pipeline
+# or
+npm run pipeline:ui
+
+# 4) Submit a seed (JSON file)
+npm run pipeline:submit -- ./seeds/example-seed.json
+```
+
+---
+
+## Status
+
+This is an **experimental framework**. The goal is to explore and evolve best practices for orchestrating prompts, models, and validations into reliable workflows. Feedback, issues, and contributions are welcome.

package/package.json

@@ -0,0 +1,51 @@
+{
+  "name": "@ryanfw/prompt-orchestration-pipeline",
+  "version": "0.0.1",
+  "description": "A Prompt-orchestration pipeline (POP) is a framework for building, running, and experimenting with complex chains of LLM tasks.",
+  "type": "module",
+  "main": "src/ui/server.js",
+  "files": [
+    "src",
+    "README.md",
+    "LICENSE"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/ryan-mahoney/prompt-orchestration-pipeline.git"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "scripts": {
+    "test": "vitest run",
+    "ui": "nodemon src/ui/server.js",
+    "ui:prod": "node src/ui/server.js"
+  },
+  "dependencies": {
+    "ajv": "^8.17.1",
+    "chokidar": "^3.5.3",
+    "dotenv": "^17.2.2",
+    "openai": "^5.23.1"
+  },
+  "devDependencies": {
+    "@vitest/coverage-v8": "^3.2.4",
+    "nodemon": "^3.0.2",
+    "prettier": "^3.0.0",
+    "vitest": "^3.2.4"
+  },
+  "engines": {
+    "node": ">=20.0.0"
+  },
+  "keywords": [
+    "llm",
+    "prompt-engineering",
+    "pipeline",
+    "orchestration",
+    "chatgpt",
+    "ai",
+    "workflow",
+    "automation"
+  ],
+  "author": "Ryan Mahoney",
+  "license": "MIT"
+}

package/src/api/index.js

@@ -0,0 +1,220 @@
+import { Orchestrator } from "../core/orchestrator.js";
+import path from "node:path";
+import fs from "node:fs/promises";
+import { validateSeedOrThrow } from "../core/validation.js";
+
+// Pure functional utilities
+const createPaths = (config) => {
+  const {
+    rootDir,
+    dataDir = "pipeline-data",
+    configDir = "pipeline-config",
+  } = config;
+  return {
+    pending: path.join(rootDir, dataDir, "pending"),
+    current: path.join(rootDir, dataDir, "current"),
+    complete: path.join(rootDir, dataDir, "complete"),
+    pipeline: path.join(rootDir, configDir, "pipeline.json"),
+    tasks: path.join(rootDir, configDir, "tasks"),
+  };
+};
+
+const validateConfig = (options = {}) => ({
+  rootDir: options.rootDir || process.cwd(),
+  dataDir: options.dataDir || "pipeline-data",
+  configDir: options.configDir || "pipeline-config",
+  autoStart: options.autoStart ?? true,
+  ui: options.ui ?? false,
+  uiPort: options.uiPort || 3000,
+  ...options,
+});
+
+const ensureDirectories = async (paths) => {
+  for (const dir of Object.values(paths)) {
+    if (dir.endsWith(".json")) continue;
+    await fs.mkdir(dir, { recursive: true });
+  }
+};
+
+const loadPipelineDefinition = async (pipelinePath) => {
+  try {
+    const content = await fs.readFile(pipelinePath, "utf8");
+    const definition = JSON.parse(content);
+    definition.__path = pipelinePath;
+    return definition;
+  } catch (error) {
+    if (error.code === "ENOENT") {
+      throw new Error(`Pipeline definition not found at ${pipelinePath}`);
+    }
+    throw error;
+  }
+};
+
+const createOrchestrator = (paths, pipelineDefinition) =>
+  new Orchestrator({ paths, pipelineDefinition });
+
+// Main API functions
+export const createPipelineOrchestrator = async (options = {}) => {
+  const config = validateConfig(options);
+  const paths = createPaths(config);
+
+  await ensureDirectories(paths);
+  const pipelineDefinition = await loadPipelineDefinition(paths.pipeline);
+  const orchestrator = createOrchestrator(paths, pipelineDefinition);
+
+  let uiServer = null;
+
+  const state = {
+    config,
+    paths,
+    pipelineDefinition,
+    orchestrator,
+    uiServer,
+  };
+
+  // Auto-start if configured
+  if (config.autoStart) {
+    await orchestrator.start();
+  }
+
+  // Start UI if configured
+  if (config.ui) {
+    const { createUIServer } = await import("../ui/server.js");
+
+    // Create API object with state injection for UI server
+    const uiApi = {
+      submitJob: (seed) => submitJob(state, seed),
+      getStatus: (jobName) => getStatus(state, jobName),
+      listJobs: (status) => listJobs(state, status),
+    };
+
+    uiServer = createUIServer(uiApi);
+    uiServer.listen(config.uiPort, () => {
+      console.log(`Pipeline UI available at http://localhost:${config.uiPort}`);
+    });
+    state.uiServer = uiServer;
+  }
+
+  return state;
+};
+
+// Job management functions
+export const submitJob = async (state, seed) => {
+  // Validate seed structure before submitting
+  validateSeedOrThrow(seed);
+
+  const name = seed.name;
+  const seedPath = path.join(state.paths.pending, `${name}-seed.json`);
+  await fs.writeFile(seedPath, JSON.stringify(seed, null, 2));
+  return { name, seedPath };
+};
+
+export const getStatus = async (state, jobName) => {
+  try {
+    const statusPath = path.join(
+      state.paths.current,
+      jobName,
+      "tasks-status.json"
+    );
+    return JSON.parse(await fs.readFile(statusPath, "utf8"));
+  } catch {}
+  try {
+    const statusPath = path.join(
+      state.paths.complete,
+      jobName,
+      "tasks-status.json"
+    );
+    return JSON.parse(await fs.readFile(statusPath, "utf8"));
+  } catch {}
+  return null;
+};
+
+export const listJobs = async (state, status = "all") => {
+  const jobs = [];
+
+  const listDirectory = async (dir, suffix = "") => {
+    try {
+      const entries = await fs.readdir(dir);
+      if (suffix) {
+        return entries
+          .filter((e) => e.endsWith(suffix))
+          .map((e) => e.replace(suffix, ""));
+      }
+      return entries;
+    } catch {
+      return [];
+    }
+  };
+
+  if (status === "all" || status === "pending") {
+    const pending = await listDirectory(state.paths.pending, "-seed.json");
+    jobs.push(...pending.map((name) => ({ name, status: "pending" })));
+  }
+
+  if (status === "all" || status === "current") {
+    const current = await listDirectory(state.paths.current);
+    jobs.push(...current.map((name) => ({ name, status: "current" })));
+  }
+
+  if (status === "all" || status === "complete") {
+    const complete = await listDirectory(state.paths.complete);
+    jobs.push(...complete.map((name) => ({ name, status: "complete" })));
+  }
+
+  return jobs;
+};
+
+// Control functions
+export const start = async (state) => {
+  await state.orchestrator.start();
+  return state;
+};
+
+export const stop = async (state) => {
+  if (state.uiServer) {
+    await new Promise((resolve) => state.uiServer.close(resolve));
+  }
+  await state.orchestrator.stop();
+  return state;
+};
+
+// Backward compatibility - class-like API for easy migration
+export const PipelineOrchestrator = {
+  async create(options = {}) {
+    const state = await createPipelineOrchestrator(options);
+
+    // Return an object with methods that maintain the original API
+    return {
+      config: state.config,
+      paths: state.paths,
+
+      async start() {
+        await start(state);
+        return this;
+      },
+
+      async stop() {
+        await stop(state);
+        return this;
+      },
+
+      async submitJob(seed) {
+        return submitJob(state, seed);
+      },
+
+      async getStatus(jobName) {
+        return getStatus(state, jobName);
+      },
+
+      async listJobs(status = "all") {
+        return listJobs(state, status);
+      },
+    };
+  },
+};
+
+// Export the original functions for direct functional usage
+export { runPipeline } from "../core/task-runner.js";
+export { selectModel } from "../core/task-runner.js";
+
+export default PipelineOrchestrator;

package/src/cli/index.js

@@ -0,0 +1,70 @@
+#!/usr/bin/env node
+import { Command } from "commander";
+import { PipelineOrchestrator } from "../api/index.js";
+import fs from "node:fs/promises";
+
+const program = new Command();
+
+program
+  .name("pipeline-orchestrator")
+  .description("Pipeline orchestration system")
+  .version("1.0.0");
+
+program
+  .command("init")
+  .description("Initialize pipeline configuration")
+  .action(async () => {
+    const template = {
+      pipeline: { name: "my-pipeline", version: "1.0.0", tasks: ["example-task"] },
+      tasks: {
+        "example-task": {
+`ingestion`: `export async function ingestion(context) { return { data: "example" }; }`,
+`inference`: `export async function inference(context) { return { output: context.data }; }`
+      }
+    };
+    await fs.mkdir("pipeline-config/tasks/example-task", { recursive: true });
+    await fs.writeFile("pipeline-config/pipeline.json", JSON.stringify(template.pipeline, null, 2));
+    await fs.writeFile("pipeline-config/tasks/index.js", `export default {\n  'example-task': './example-task/index.js'\n};`);
+    await fs.writeFile("pipeline-config/tasks/example-task/index.js", `${template.tasks["example-task"].ingestion}\n\n${template.tasks["example-task"].inference}\n`);
+    console.log("Pipeline configuration initialized");
+  });
+
+program
+  .command("start")
+  .description("Start the pipeline orchestrator")
+  .option("-u, --ui", "Start with UI server")
+  .option("-p, --port <port>", "UI server port", "3000")
+  .action(async (options) => {
+    const orchestrator = new PipelineOrchestrator({ ui: options.ui, uiPort: parseInt(options.port) });
+    await orchestrator.initialize();
+    console.log("Pipeline orchestrator started");
+    process.on("SIGINT", async () => { await orchestrator.stop(); process.exit(0); });
+  });
+
+program
+  .command("submit <seed-file>")
+  .description("Submit a new job")
+  .action(async (seedFile) => {
+    const seed = JSON.parse(await fs.readFile(seedFile, "utf8"));
+    const orchestrator = new PipelineOrchestrator({ autoStart: false });
+    await orchestrator.initialize();
+    const job = await orchestrator.submitJob(seed);
+    console.log(`Job submitted: ${job.name}`);
+  });
+
+program
+  .command("status [job-name]")
+  .description("Get job status")
+  .action(async (jobName) => {
+    const orchestrator = new PipelineOrchestrator({ autoStart: false });
+    await orchestrator.initialize();
+    if (jobName) {
+      const status = await orchestrator.getStatus(jobName);
+      console.log(JSON.stringify(status, null, 2));
+    } else {
+      const jobs = await orchestrator.listJobs();
+      console.table(jobs);
+    }
+  });
+
+program.parse();