folpipe 0.1.0__py3-none-any.whl

folpipe-0.1.0.dist-info/METADATA

@@ -0,0 +1,235 @@
+Metadata-Version: 2.4
+Name: folpipe
+Version: 0.1.0
+Summary: Folder-based agentic AI pipeline framework
+Project-URL: Homepage, https://github.com/kroczynskikrzysztof/pipelinex
+Project-URL: Documentation, https://github.com/kroczynskikrzysztof/pipelinex/tree/main/docs
+Project-URL: Issues, https://github.com/kroczynskikrzysztof/pipelinex/issues
+Author-email: Krzysztof Roczynski <kroczynskikrzysztof.info@gmail.com>
+License: MIT
+License-File: LICENSE
+Keywords: agents,ai,automation,llm,pipeline
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: click>=8.0
+Requires-Dist: httpx>=0.24
+Requires-Dist: jinja2>=3.0
+Requires-Dist: litellm>=1.40
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: pyyaml>=6.0
+Description-Content-Type: text/markdown
+
+# pipelinex
+
+Folder-based agentic AI pipeline framework. No programming required to use.
+
+```bash
+pip install folpipe
+folpipe run ./my-pipeline --watch
+```
+
+---
+
+## The idea
+
+A pipeline is a folder. Instructions live in plain-English markdown files. Tools are drop-in folders. The model is swappable with one line.
+
+```
+my-pipeline/
+├── pipeline.yaml       ← steps, model, routing
+├── SKILL.md            ← global instructions (plain English)
+├── .env                ← API keys
+├── input/              ← drop files here
+├── output/             ← results land here
+├── tools/              ← drop-in custom tools
+├── step-01-ingest/
+│   └── SKILL.md        ← what this step does
+└── step-02-process/
+    ├── SKILL.md
+    └── sub-01-chunk/   ← sub-steps, recursively
+        └── SKILL.md
+```
+
+Zip the folder, share it, version it in git. No server, no database, no cloud account.
+
+---
+
+## Quick start
+
+**1. Install**
+
+```bash
+pip install pipelinex
+```
+
+**2. Create a pipeline**
+
+```bash
+pipelinex new my-pipeline
+cd my-pipeline
+```
+
+**3. Configure the model** — edit `.env`:
+
+```bash
+DEEPSEEK_API_KEY=sk-...
+```
+
+**4. Write your step** — edit `step-01-start/SKILL.md`:
+
+```markdown
+# Summarise input
+
+Read the file from input/ and write a 3-paragraph summary to output/summary.md.
+
+## Task
+
+Use read_file to read the input file.
+Use write_file to write output/summary.md.
+Leave a handoff note when done.
+```
+
+**5. Run it**
+
+```bash
+cp myfile.txt input/
+pipelinex run . --watch
+```
+
+---
+
+## What it can do
+
+**Linear pipelines** — steps run in order, each hands off to the next via shared state.
+
+**Agent branching** — the model decides which step comes next from a declared whitelist:
+
+```yaml
+- id: step-03-validate
+  can_goto: [step-04-output, step-05-partial-output]
+```
+
+**Parallel dispatch** — the model fires multiple `dispatch_task` calls in one response; the runner detects the batch and executes them concurrently:
+
+```markdown
+## Task
+Process all documents at once — send them all for chunking
+in the same breath rather than one at a time.
+```
+
+**Sub-steps** — steps contain named sub-step folders with their own SKILL.md and tools:
+
+```
+dispatch_task(task="chunk intro-to-ml.txt", substep="sub-01-chunk", context={...})
+```
+
+**Human-in-the-loop** — steps can pause for console input, file review, or a custom tool (Slack, email, webhook):
+
+```yaml
+- id: step-03-review
+  human_input:
+    mode: console
+    prompt: "Approve this output? (yes/no):"
+```
+
+**Model-driven context** — each step's `## Context` section tells the model what state to pay attention to and what to ignore. Skip signals are respected; the model never sees irrelevant data.
+
+**Any model** — DeepSeek, Anthropic, OpenAI, Ollama, Groq, or any OpenAI-compatible endpoint. Change one line in `pipeline.yaml`. Optional per-step overrides for cost optimisation.
+
+---
+
+## Custom tools
+
+Drop a folder into `tools/`:
+
+```
+tools/
+└── send_slack/
+    ├── tool.json    ← MCP-compatible schema + deps declaration
+    └── run.py       ← stdin JSON in, stdout JSON out
+```
+
+```json
+{
+  "name": "send_slack",
+  "description": "Post a message to a Slack channel. Use for approvals and failures that need human eyes.",
+  "inputSchema": { ... },
+  "deps": ["slack-sdk"]
+}
+```
+
+Dependencies declared in `deps` are installed automatically before the first run. No manual `pip install`.
+
+Any language works — `run.py`, `run.sh`, or `run.js`.
+
+---
+
+## Example pipelines
+
+### doc-pipeline
+
+Ingests documents → chunks → embeds → indexed output.
+
+```bash
+pipelinex run ./doc-pipeline --watch
+```
+
+Demonstrates: parallel dispatch, sub-steps, agent branching, partial-failure routing.
+
+### cv-pipeline
+
+Job URL → developer profile → tailored CV + styled PDF.
+
+```bash
+pipelinex run ./cv-pipeline --input "https://example.com/jobs/engineer" --watch
+```
+
+Demonstrates: custom tools (`fetch_page`, `render_pdf`), linear pipeline, HTML→PDF via Edge headless.
+
+---
+
+## CLI
+
+```bash
+pipelinex run ./my-pipeline               # run
+pipelinex run ./my-pipeline --watch       # run with live output
+pipelinex run ./my-pipeline --from step-03  # resume after failure
+pipelinex run ./my-pipeline --dry-run     # validate config without running
+pipelinex new my-pipeline                 # scaffold new pipeline
+pipelinex new step step-05-review --in ./my-pipeline
+pipelinex new tool send_email --in ./my-pipeline/tools
+pipelinex validate ./my-pipeline          # check config
+pipelinex tools list --pipeline ./my-pipeline
+pipelinex tools install ./my-pipeline     # pre-install tool deps
+pipelinex log ./my-pipeline               # show run log
+pipelinex log ./my-pipeline --errors      # show last error report
+```
+
+---
+
+## Documentation
+
+- [Getting Started](docs/getting-started.md)
+- [Pipeline Structure](docs/pipeline-structure.md) — `pipeline.yaml` and `SKILL.md` reference
+- [State & Handoffs](docs/state.md)
+- [Tools](docs/tools.md) — built-ins, custom tools, resolution order
+- [Execution Patterns](docs/execution-patterns.md) — linear, branching, loops, dispatch, sub-steps
+- [Context Management](docs/context-management.md)
+- [Human Input](docs/human-input.md)
+- [CLI Reference](docs/cli.md)
+- [Examples](docs/examples.md)
+
+---
+
+## Requirements
+
+- Python 3.10+
+- A model with tool calling support (DeepSeek, Claude, GPT-4, Llama 3.1+)
+- For PDF rendering in cv-pipeline: Edge or Chrome installed

folpipe-0.1.0.dist-info/RECORD

@@ -0,0 +1,18 @@
+pipelinex/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pipelinex/cli.py,sha256=VwuCsCffJbuhX5ydlEMjp3CgJwBW3Kuma9EohMA05rE,7416
+pipelinex/context_mgr.py,sha256=K9Z6lfB2g6xClSCxz_pPYxjdLlAIGo-8BaITmgvQgBo,2483
+pipelinex/loader.py,sha256=44VR7_nynmmXui3nY4yk4NDBhzUx_8SHXvskannPExI,3056
+pipelinex/logger.py,sha256=uhryTI-Puo4KOcaMi_TCD8vtRQLyeivkkCqL8ncRVr0,2835
+pipelinex/model.py,sha256=_usWH5_fLgfa5yd-AHRONI0PZNsIpnrT0bnr85Uc17E,2828
+pipelinex/runner.py,sha256=sQKdnI4vRUvmVfRzMtmcBbMdcEbB-hPKGL9XCCLeLHU,20816
+pipelinex/scaffold.py,sha256=46Ho6bLF-B2cwLo3ujA5tjlWdB0V7_f2CvsemCougNM,3485
+pipelinex/state.py,sha256=Dqw56Niaj7aU2JnbO-rZxZoJbXMeb3Qq7l3mHGxcxjA,2331
+pipelinex/tools/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+pipelinex/tools/builtin.py,sha256=31G0UVD77WzsNnw_v1Z2v8KSErU_Gpih_y4yrCvwbJM,12344
+pipelinex/tools/executor.py,sha256=AryHZ76JRhrDGF9daMO9OZHMtLT5kzUzJa34wIf0G3k,1770
+pipelinex/tools/resolver.py,sha256=FlU-WvNEsmjp1ns4b1n60qCZdulXNdaRrJNuNOXC5dk,2136
+folpipe-0.1.0.dist-info/METADATA,sha256=T-gvXCEI0Uahxjeg1ppLMMsbLkuNqbDsrb5gkJq_GBk,6636
+folpipe-0.1.0.dist-info/WHEEL,sha256=QccIxa26bgl1E6uMy58deGWi-0aeIkkangHcxk2kWfw,87
+folpipe-0.1.0.dist-info/entry_points.txt,sha256=BtJ2UJH9QDnQt6ybIATZlAmRVD-_yJIEbepW6UKIjC0,47
+folpipe-0.1.0.dist-info/licenses/LICENSE,sha256=OCwD3ACF1Br_JQbOE9Uy6W81ykoODdABXEmIGWsxZfA,1106
+folpipe-0.1.0.dist-info/RECORD,,

folpipe-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.29.0
+Root-Is-Purelib: true
+Tag: py3-none-any

folpipe-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+folpipe = pipelinex.cli:main

folpipe-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kroczyński Krzysztof s26192
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

pipelinex/cli.py

@@ -0,0 +1,219 @@
+import sys
+from pathlib import Path
+
+import click
+
+# Force UTF-8 stdout/stderr on Windows (avoids charmap errors with model output)
+if sys.stdout.encoding and sys.stdout.encoding.lower() not in ("utf-8", "utf8"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+if sys.stderr.encoding and sys.stderr.encoding.lower() not in ("utf-8", "utf8"):
+    sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+
+from .loader import load_pipeline, validate_pipeline
+from .runner import PipelineRunner
+from .scaffold import scaffold_pipeline, scaffold_step, scaffold_tool
+from .tools.builtin import BUILTIN_SCHEMAS
+from .tools.resolver import GLOBAL_TOOLS_PATH, install_tool_deps, resolve_tools
+
+
+@click.group()
+def main():
+    """pipelinex — folder-based agentic AI pipeline framework"""
+
+
+@main.command()
+@click.argument("pipeline_path", type=click.Path(exists=True))
+@click.option("--input", "input_data", default=None, help="Input text or path to input file")
+@click.option("--from", "from_step", default=None, metavar="STEP_ID", help="Resume from this step")
+@click.option("--dry-run", is_flag=True, help="Validate config and env, don't execute")
+@click.option("--watch", is_flag=True, help="Show live step/tool progress")
+@click.option("--model", "model_override", default=None, metavar="PROVIDER/NAME", help="Override model")
+def run(pipeline_path, input_data, from_step, dry_run, watch, model_override):
+    """Run a pipeline."""
+    path = Path(pipeline_path)
+
+    try:
+        config = load_pipeline(path)
+    except (FileNotFoundError, SystemExit) as e:
+        click.echo(f"ERROR: {e}", err=True)
+        sys.exit(1)
+
+    errors = validate_pipeline(config)
+    if errors:
+        click.echo("Validation errors:", err=True)
+        for e in errors:
+            click.echo(f"  - {e}", err=True)
+        sys.exit(1)
+
+    if dry_run:
+        click.echo("Pipeline config valid.")
+        click.echo(f"  Name:    {config.get('name')}")
+        click.echo(f"  Version: {config.get('version')}")
+        click.echo(f"  Model:   {config['model']['provider']}/{config['model']['name']}")
+        click.echo(f"  Steps:   {[s['id'] for s in config.get('steps', [])]}")
+        return
+
+    if input_data:
+        p = Path(input_data)
+        if p.exists():
+            input_data = p.read_text(encoding="utf-8")
+
+    runner = PipelineRunner(
+        pipeline_path=path,
+        watch=watch,
+        from_step=from_step,
+        model_override=model_override,
+        input_data=input_data,
+    )
+
+    try:
+        runner.run()
+    except SystemExit as e:
+        click.echo(str(e), err=True)
+        sys.exit(1)
+    except KeyboardInterrupt:
+        click.echo("\nInterrupted.", err=True)
+        sys.exit(1)
+    except Exception as e:
+        click.echo(f"ERROR: {e}", err=True)
+        sys.exit(1)
+
+
+@main.command()
+@click.argument("args", nargs=-1)
+@click.option("--in", "in_dir", default=None, help="Target directory")
+def new(args, in_dir):
+    """Scaffold a pipeline, step, or tool.
+
+    \b
+    pipelinex new my-pipeline
+    pipelinex new step step-05-review --in ./my-pipeline
+    pipelinex new tool send_email --in ./my-pipeline/tools
+    """
+    if not args:
+        click.echo("Usage:")
+        click.echo("  pipelinex new <pipeline-name>")
+        click.echo("  pipelinex new step <step-id> --in <pipeline-dir>")
+        click.echo("  pipelinex new tool <tool-name> --in <tools-dir>")
+        return
+
+    if args[0] == "step":
+        if len(args) < 2:
+            click.echo("ERROR: step name required. Usage: pipelinex new step <step-id> --in <pipeline-dir>", err=True)
+            sys.exit(1)
+        scaffold_step(args[1], Path(in_dir or "."))
+
+    elif args[0] == "tool":
+        if len(args) < 2:
+            click.echo("ERROR: tool name required. Usage: pipelinex new tool <name> --in <tools-dir>", err=True)
+            sys.exit(1)
+        scaffold_tool(args[1], Path(in_dir or "tools"))
+
+    else:
+        scaffold_pipeline(args[0], Path(in_dir or "."))
+
+
+@main.command()
+@click.argument("pipeline_path", type=click.Path(exists=True))
+def validate(pipeline_path):
+    """Validate pipeline config without running."""
+    path = Path(pipeline_path)
+    try:
+        config = load_pipeline(path)
+    except (FileNotFoundError, SystemExit) as e:
+        click.echo(f"ERROR: {e}", err=True)
+        sys.exit(1)
+
+    errors = validate_pipeline(config)
+    if errors:
+        click.echo("Validation failed:")
+        for e in errors:
+            click.echo(f"  x {e}")
+        sys.exit(1)
+
+    click.echo("OK Pipeline is valid")
+    click.echo(f"  Steps: {[s['id'] for s in config.get('steps', [])]}")
+    click.echo(f"  Model: {config['model']['provider']}/{config['model']['name']}")
+
+
+@main.group()
+def tools():
+    """Manage tools."""
+
+
+@tools.command("list")
+@click.option("--pipeline", "pipeline_path", default=None, help="Include pipeline-level tools")
+def tools_list(pipeline_path):
+    """List available tools."""
+    click.echo("Built-in tools:")
+    for t in BUILTIN_SCHEMAS:
+        click.echo(f"  {t['name']:<20} {t['description'][:60]}")
+
+    if pipeline_path:
+        path = Path(pipeline_path)
+        custom = [t for t in resolve_tools(path) if t["name"] not in {s["name"] for s in BUILTIN_SCHEMAS}]
+        if custom:
+            click.echo(f"\nPipeline tools ({path.name}):")
+            for t in custom:
+                click.echo(f"  {t['name']}")
+
+    if GLOBAL_TOOLS_PATH.exists():
+        global_tools = list(GLOBAL_TOOLS_PATH.iterdir())
+        if global_tools:
+            click.echo(f"\nGlobal tools (~/.pipelinex/tools/):")
+            for d in global_tools:
+                if d.is_dir():
+                    click.echo(f"  {d.name}")
+
+
+@tools.command("install")
+@click.argument("pipeline_path", type=click.Path(exists=True))
+def tools_install(pipeline_path):
+    """Install tool deps for a pipeline without running."""
+    path = Path(pipeline_path)
+    cache = Path.home() / ".pipelinex"
+    count = 0
+
+    for tools_dir in _all_tool_dirs(path):
+        if not tools_dir.exists():
+            continue
+        for tool_dir in tools_dir.iterdir():
+            if tool_dir.is_dir() and (tool_dir / "tool.json").exists():
+                try:
+                    install_tool_deps(tool_dir, cache)
+                    count += 1
+                except Exception as e:
+                    click.echo(f"  Failed {tool_dir.name}: {e}", err=True)
+
+    click.echo(f"Installed deps for {count} tool(s).")
+
+
+def _all_tool_dirs(pipeline_path: Path):
+    yield pipeline_path / "tools"
+    try:
+        config = load_pipeline(pipeline_path)
+        for step in config.get("steps", []):
+            yield pipeline_path / step["id"] / "tools"
+    except Exception:
+        pass
+
+
+@main.command()
+@click.argument("pipeline_path", type=click.Path(exists=True))
+@click.option("--errors", is_flag=True, help="Show error report from last failed run")
+def log(pipeline_path, errors):
+    """Show run log or error report."""
+    path = Path(pipeline_path)
+
+    if errors:
+        report = path / "output" / "errors" / "report.md"
+        if report.exists():
+            click.echo(report.read_text(encoding="utf-8"))
+        else:
+            click.echo("No error report found.")
+    else:
+        log_file = path / "output" / "run.log"
+        if log_file.exists():
+            click.echo(log_file.read_text(encoding="utf-8"))
+        else:
+            click.echo("No run log found.")

pipelinex/context_mgr.py

@@ -0,0 +1,78 @@
+import json
+import re
+from typing import Any
+
+
+def _extract_context_section(skill_md: str) -> str:
+    m = re.search(r'##\s+Context\s*\n(.*?)(?=\n##|\Z)', skill_md, re.DOTALL | re.IGNORECASE)
+    return m.group(1).strip() if m else ""
+
+
+def _classify_tiers(context_section: str, keys: list[str], model_cfg: dict) -> dict[str, str]:
+    from .model import call_llm, extract_text
+
+    prompt = (
+        f"A pipeline step has this context guidance:\n\n{context_section}\n\n"
+        f"Available state keys: {keys}\n\n"
+        "Classify each key as:\n"
+        "- 'essential': explicitly needed in full\n"
+        "- 'skip': explicitly not needed\n"
+        "- 'include': everything else\n\n"
+        "Reply with JSON only: {\"key\": \"essential|include|skip\", ...}"
+    )
+
+    try:
+        resp = call_llm(model_cfg, [{"role": "user", "content": prompt}], max_tokens=256)
+        text = extract_text(resp)
+        m = re.search(r'\{[^{}]+\}', text, re.DOTALL)
+        if m:
+            result = json.loads(m.group())
+            valid = {"essential", "include", "skip"}
+            return {k: (v if v in valid else "include") for k, v in result.items()}
+    except Exception:
+        pass
+
+    return {k: "include" for k in keys}
+
+
+def _fmt(value: Any) -> str:
+    if isinstance(value, str):
+        return value
+    return f"```json\n{json.dumps(value, indent=2, default=str)}\n```"
+
+
+def build_context_prompt(
+    state: dict,
+    skill_md: str,
+    handoff: str | None,
+    model_cfg: dict | None = None,
+) -> str:
+    context_section = _extract_context_section(skill_md)
+    parts = []
+
+    if handoff:
+        parts.append(f"## Handoff from previous step\n\n{handoff}")
+
+    meta_keys = {"_meta", "handoff", "_input_consumed"}
+    state_data = {k: v for k, v in state.items() if k not in meta_keys}
+
+    if state_data:
+        keys = list(state_data.keys())
+
+        if context_section and model_cfg and keys:
+            tiers = _classify_tiers(context_section, keys, model_cfg)
+        else:
+            tiers = {k: "include" for k in keys}
+
+        state_parts = []
+        for key, value in state_data.items():
+            tier = tiers.get(key, "include")
+            if tier == "skip":
+                continue
+            tag = " [ESSENTIAL]" if tier == "essential" else ""
+            state_parts.append(f"### {key}{tag}\n{_fmt(value)}")
+
+        if state_parts:
+            parts.append("## Pipeline State\n\n" + "\n\n".join(state_parts))
+
+    return "\n\n".join(parts)

pipelinex/loader.py

@@ -0,0 +1,101 @@
+import os
+import re
+from pathlib import Path
+
+import yaml
+from dotenv import dotenv_values
+
+
+def load_env(pipeline_path: Path) -> dict:
+    env = dict(os.environ)
+    env_file = pipeline_path / ".env"
+    if env_file.exists():
+        env.update(dotenv_values(env_file))
+    return env
+
+
+def _substitute(obj, env: dict):
+    if isinstance(obj, str):
+        def replace(m):
+            var = m.group(1)
+            if var not in env:
+                raise EnvironmentError(
+                    f"Missing environment variable: {var}\n"
+                    f"       Add it to .env or export it in your shell."
+                )
+            return env[var]
+        return re.sub(r'\$\{([^}]+)\}', replace, obj)
+    elif isinstance(obj, dict):
+        return {k: _substitute(v, env) for k, v in obj.items()}
+    elif isinstance(obj, list):
+        return [_substitute(i, env) for i in obj]
+    return obj
+
+
+def load_pipeline(pipeline_path: Path) -> dict:
+    pipeline_path = Path(pipeline_path)
+    yaml_file = pipeline_path / "pipeline.yaml"
+    if not yaml_file.exists():
+        raise FileNotFoundError(f"pipeline.yaml not found in {pipeline_path}")
+
+    env = load_env(pipeline_path)
+
+    with open(yaml_file) as f:
+        raw = yaml.safe_load(f)
+
+    try:
+        config = _substitute(raw, env)
+    except EnvironmentError as e:
+        raise SystemExit(f"ERROR: {e}")
+
+    config["_path"] = pipeline_path
+    config["_env"] = env
+    return config
+
+
+def load_skill_md(pipeline_path: Path, step_id: str | None = None, substep_id: str | None = None) -> str:
+    parts = []
+
+    global_skill = pipeline_path / "SKILL.md"
+    if global_skill.exists():
+        parts.append(global_skill.read_text(encoding="utf-8"))
+
+    if step_id:
+        step_skill = pipeline_path / step_id / "SKILL.md"
+        if step_skill.exists():
+            parts.append(step_skill.read_text(encoding="utf-8"))
+
+        if substep_id:
+            substep_skill = pipeline_path / step_id / substep_id / "SKILL.md"
+            if substep_skill.exists():
+                parts.append(substep_skill.read_text(encoding="utf-8"))
+
+    return "\n\n---\n\n".join(parts)
+
+
+def validate_pipeline(config: dict) -> list[str]:
+    errors = []
+
+    if "model" not in config:
+        errors.append("Missing 'model' section")
+    else:
+        if "provider" not in config["model"]:
+            errors.append("model.provider is required")
+        if "name" not in config["model"]:
+            errors.append("model.name is required")
+
+    if not config.get("steps"):
+        errors.append("No steps defined")
+    else:
+        step_ids = {s["id"] for s in config["steps"] if "id" in s}
+        for step in config["steps"]:
+            if "id" not in step:
+                errors.append(f"Step missing 'id': {step}")
+                continue
+            for target in step.get("can_goto", []):
+                if target not in step_ids:
+                    errors.append(
+                        f"Step '{step['id']}' can_goto '{target}' which doesn't exist"
+                    )
+
+    return errors