langflower 0.0.10 → 0.1.1

package/README.md

@@ -3,38 +3,63 @@
 > **Disclaimer:** Langflower is currently in internal testing. Anyone can
 > download and try it, but some functions may not be stable.
 
-**Turn a local folder into a repeatable AI workflow.**
+## Key concept: everything is a node
 
-Point Langflower at a workspace, choose or create a visual workflow, and let
-it read, create, review, and update files there. You see every stage on the
-canvas and decide what must happen before the next stage begins.
+Langflower is not built around the LLM as a first-class citizen. The core
+abstraction is a **reactive node**: several typed inputs, several typed
+outputs. Each port acts independently — a node can receive on one input
+and emit on one output at any time, without waiting for the rest. The
+runtime wires those nodes into a workflow.
 
-The workflow decides the process — not a model deciding it is “done.”
+That is enough for complex processing chains — including loops and
+conditions.
 
-![Langflower starter workflow](https://raw.githubusercontent.com/earthdmitriy/langflower/master/docs/img/starter.png)
+The trick is the same rule applied to models. Because ports fire on their
+own, an LLM agent is just another reactive node: prompt and tools in;
+response out; streaming tool log, reasoning, and draft as extra outputs
+that can update while the run continues. All LLM-specific logic stays as
+that node's internal state. It does not leak through the rest of the app.
+
+## Open possibilities
+
+Because every unit of work is a node, Langflower is not pinned to one
+product shape — coding harness, chat harness, or any other fixed loop.
+Workflows stay flexible. With the right wiring they fit the job.
+
+Need a chat? Wire user input, an agent, and HITL feedback. Agent-to-agent
+dialogue is two agents connected together. Coding is the same graph plus
+file-ops tools. Tired of “I’m done” while the code still does not compile?
+Write a custom review-gate node. The agent never gets a path around it.
+
+## Hard harness
 
-## What can you use it for?
+The sequence is the workflow topology. That is how Langflower orchestrates
+complex work: QA, review, and code checks sit on the graph, so agents
+cannot skip them. A well-configured workflow forces high-quality output —
+the model does not get to declare itself finished.
 
-### Ship a code change
+**Not another chat harness. A local node graph.**
 
-Turn a task into a visible pipeline: clarify the request, implement the
-change, review it, run QA, and accept the result. You can stop or redirect
-the work at the stages that matter.
+### Keep everything local
 
-### Write and refine files
+Langflower runs on your machine and does not expose your project as a
+hosted product. Files stay where they are. You can reproduce what mature
+cloud tools offer — at home, or on a closed network with internal LLM
+providers.
 
-Create an article or prompt in your workspace, review its tone and facts, then
-revise it in place. The useful output is a file you own, not only a chat
-transcript.
+### Scoped to a folder
 
-### Build a knowledge base
+You start from a folder. Data, custom nodes, and workflows are scoped to
+that workspace. Open the folder, and the graph, files, and run belong
+together.
 
-Ingest project documentation, resolve duplicates or contradictions, review the
-proposed changes, and maintain a project wiki or Obsidian vault.
+### Extensible even now
 
-Langflower also supports research fan-out and synthesis, reusable skills,
-prompt refinement, regression gates, and multi-agent review. Browse the
-[workflow ideas](https://github.com/earthdmitriy/langflower/blob/master/docs/public/workflows.md).
+This is still an early version, but Langflower already supports custom
+nodes and custom node packs. Share and reuse workflows and nodes the same
+way you share the rest of the project.
+
+![Langflower starter workflow](https://raw.githubusercontent.com/earthdmitriy/langflower/master/docs/img/starter.png)
 
 ## How it works
 
@@ -46,55 +71,61 @@ prompt refinement, regression gates, and multi-agent review. Browse the
    decision.
 5. Find the resulting files and data in the same workspace.
 
-The graph is a **hard harness**: it defines the order of work. If review, QA,
-or approval is part of the workflow, an agent cannot silently skip it because
-it is confident in its own answer. Bring your own review gate — Langflower
-enforces it, so LLMs cannot skip or bypass that step.
+## Under the hood
 
-## Why Langflower
+The core is a reactive runtime that wires nodes together. The **node SDK**
+is the public contract: any node that follows it can run on that runtime.
+**Common nodes** is the built-in catalog. It uses the same SDK as custom
+node packs.
 
-- **Your folder stays the centre of work.** Read and write files locally,
-  rather than leaving the result in a provider-hosted chat history.
-- **You stay in control.** Workflows can request approval before sensitive
-  file edits or shell commands run.
-- **Human review is built in.** Ask for a plan review, fact check, rewrite, or
-  approval exactly where it makes sense in the process.
-- **Long tasks do not own your browser tab.** Close the tab and return later;
-  the session continues and the UI catches up when you reconnect.
-- **Reuse a process, not just a prompt.** Build a workflow once, then apply it
-  to similar folders and tasks.
-- **Extend it when the defaults are not enough.** Add markdown skills under
-  `.langflower/skills/` or custom nodes under `.langflower/nodes/`.
-
-## Why it stays reliable while you work
-
-Langflower's runtime treats every workflow step as a live exchange of data.
-That means a step can naturally wait for your input and continue when you
-reply — human review is part of the workflow, not a special interruption
-mode.
-
-The server owns the active session, while the browser is a view of it. Closing
-or reopening a tab does not create a competing workflow state or cancel a
-long-running job; the UI reconnects to the current progress.
+The **server** composes those pieces, compiles user-defined nodes, and owns
+the live run. The **UI** is a thin browser client. It listens to WebSocket
+events from the server; it does not own the workflow. Close the tab while a
+long run continues. Reopen it, and the server catches the UI up, so the
+canvas stays in sync.
 
 For a short builder-oriented picture, see
 [How it works](https://github.com/earthdmitriy/langflower/blob/master/docs/public/how-it-works.md).
 
+## Why Langflower
+
+- **You stay in control.** Workflows can request approval before sensitive
+  file edits or shell commands run.
+- **Use the browser you already have.** Other harnesses pack a web UI into
+  a built-in browser such as Electron. Langflower uses your existing
+  browser, so you can close the tab, free those resources, and let the
+  server keep the run; reopen it and the UI catches up.
+- **Extend it when the defaults are not enough.** Langflower uses common
+  agent primitives — **MCP** and **skills** — and lets you define **custom
+  nodes** on top: processing in the graph, or custom tools for agents via
+  the same **ToolHandle** contract as built-ins.
+
 ## How it compares
 
-Langflower gives you graph-based chaining like mature orchestration tools —
-stages, branches, and review gates — but it is aimed at **local daily use**,
-not at shipping a backend service or wiring a cloud automation platform.
+Versus chat-style harnesses (often an Electron shell around a model loop):
+Langflower is a local node graph. The LLM is a node, not the product.
+Order comes from topology, not from the model deciding it is done.
+The UI is your existing browser, not a bundled one.
 
-Build a reusable workflow once for coding, documentation, article writing, or
-any similar desk work, then run it against the folders and tasks you already
-touch every day. The canvas keeps the process visible and repeatable; the
-project folder stays where the work happens.
+Versus cloud graph tools: the same idea of wiring nodes, but aimed at a
+folder on your machine — home or a closed network with internal
+providers — not at hosting a service or cloning ETL in the cloud.
+
+## What it lacks
+
+- **Chat sessions.** Node-internal state is the current architecture, so
+  serializable chat-session mechanics are hard. Maybe later.
+- **Image and video.** No asset management for multimodal models. Not yet.
+- **No built-in IDE or git UI.** We are not reinventing those wheels. Use
+  the editor and git tools you already have.
 
 ## Quick start
 
 Requires **Node.js ≥ 22**.
 
+One-shot OS installers (Node LTS if needed + global `langflower`):
+[install/](install/) (`windows.ps1`, `linux.sh`, `macos.sh`).
+
 ```bash
 npm install -g langflower
 langflower
@@ -122,17 +153,12 @@ Maintainers (monorepo only): [docs/RELEASE.md](https://github.com/earthdmitriy/l
 
 Shipped with the npm package under [`docs/public/`](https://github.com/earthdmitriy/langflower/tree/master/docs/public):
 
-| Want to…                     | Start here                                                                                                 |
-| ---------------------------- | ---------------------------------------------------------------------------------------------------------- |
-| Install and first run        | [Getting started](https://github.com/earthdmitriy/langflower/blob/master/docs/public/getting-started.md)   |
-| Understand the product       | [Product overview](https://github.com/earthdmitriy/langflower/blob/master/docs/public/product.md)          |
-| Use the canvas and runs      | [Using the editor](https://github.com/earthdmitriy/langflower/blob/master/docs/public/using-the-editor.md) |
-| Browse workflow ideas        | [Workflow ideas](https://github.com/earthdmitriy/langflower/blob/master/docs/public/workflows.md)          |
-| Configure providers and keys | [Configuration](https://github.com/earthdmitriy/langflower/blob/master/docs/public/configuration.md)       |
-| Add skills or custom nodes   | [Extending](https://github.com/earthdmitriy/langflower/blob/master/docs/public/extending.md)               |
-| Builder runtime picture      | [How it works](https://github.com/earthdmitriy/langflower/blob/master/docs/public/how-it-works.md)         |
-
-## What Langflower is not
-
-Langflower is local and project-scoped. It is not a hosted multi-tenant cloud
-product or a generic cloud ETL clone.
+| Want to…                         | Start here                                                                                                 |
+| -------------------------------- | ---------------------------------------------------------------------------------------------------------- |
+| Install and first run            | [Getting started](https://github.com/earthdmitriy/langflower/blob/master/docs/public/getting-started.md)   |
+| Understand the product           | [Product overview](https://github.com/earthdmitriy/langflower/blob/master/docs/public/product.md)          |
+| Use the canvas and runs          | [Using the editor](https://github.com/earthdmitriy/langflower/blob/master/docs/public/using-the-editor.md) |
+| Browse workflow ideas            | [Workflow ideas](https://github.com/earthdmitriy/langflower/blob/master/docs/public/workflows.md)          |
+| Configure providers and keys     | [Configuration](https://github.com/earthdmitriy/langflower/blob/master/docs/public/configuration.md)       |
+| Add MCP, skills, or custom nodes | [Extending](https://github.com/earthdmitriy/langflower/blob/master/docs/public/extending.md)               |
+| Builder runtime picture          | [How it works](https://github.com/earthdmitriy/langflower/blob/master/docs/public/how-it-works.md)         |

package/bin/langflower.js

@@ -1,3 +1,17 @@
 #!/usr/bin/env node
 
-import '../dist/index.js';
+const args = process.argv.slice(2);
+const first = args[0];
+const skipHeartbeat =
+	first === 'eval' ||
+	first === 'help' ||
+	args.includes('--help') ||
+	args.includes('-h') ||
+	args.includes('--version') ||
+	args.includes('-V');
+
+if (!skipHeartbeat) {
+	process.stdout.write('Starting Langflower...\n');
+}
+
+await import('../dist/index.js');

package/dist/chunk-526FJL62.js

@@ -0,0 +1,307 @@
+import { createRequire as __lfCreateRequire } from 'node:module';
+const require = __lfCreateRequire(import.meta.url);
+
+import {
+  createProjectHarness
+} from "./chunk-WQZHA5FM.js";
+import "./chunk-EAZMCX37.js";
+
+// packages/cli/dist/eval-command.js
+import path3 from "node:path";
+
+// packages/eval/dist/load-pack.js
+import fs from "node:fs/promises";
+import path from "node:path";
+var isScorerKind = (value) => value === "exact" || value === "includes";
+var parseCase = (raw, index) => {
+  if (raw === null || typeof raw !== "object") {
+    throw new Error(`eval pack case[${index}] must be an object`);
+  }
+  const row = raw;
+  const id = typeof row.id === "string" ? row.id.trim() : "";
+  const input = typeof row.input === "string" ? row.input : "";
+  const expected = typeof row.expected === "string" ? row.expected.trim() : "";
+  if (id.length === 0) {
+    throw new Error(`eval pack case[${index}] needs a non-empty id`);
+  }
+  if (expected.length === 0) {
+    throw new Error(`eval pack case "${id}" needs a non-empty expected`);
+  }
+  const scorer = row.scorer;
+  if (scorer !== void 0 && !isScorerKind(scorer)) {
+    throw new Error(`eval pack case "${id}" scorer must be "exact" or "includes"`);
+  }
+  return scorer === void 0 ? { id, input, expected } : { id, input, expected, scorer };
+};
+var parsePack = (raw) => {
+  if (raw === null || typeof raw !== "object") {
+    throw new Error("eval pack.json must be an object");
+  }
+  const doc = raw;
+  const id = typeof doc.id === "string" ? doc.id.trim() : "";
+  if (id.length === 0) {
+    throw new Error("eval pack.json needs a non-empty id");
+  }
+  const threshold = typeof doc.threshold === "number" && Number.isFinite(doc.threshold) ? doc.threshold : NaN;
+  if (!(threshold >= 0 && threshold <= 1)) {
+    throw new Error(`eval pack "${id}" threshold must be a number in [0, 1]`);
+  }
+  const scorer = doc.scorer;
+  if (!isScorerKind(scorer)) {
+    throw new Error(`eval pack "${id}" scorer must be "exact" or "includes"`);
+  }
+  if (!Array.isArray(doc.cases) || doc.cases.length === 0) {
+    throw new Error(`eval pack "${id}" needs a non-empty cases array`);
+  }
+  const cases = doc.cases.map((c, i) => parseCase(c, i));
+  const caseIds = cases.map((c) => c.id);
+  const duplicateId = caseIds.find((caseId, i) => caseIds.indexOf(caseId) !== i);
+  if (duplicateId !== void 0) {
+    throw new Error(`eval pack "${id}" has duplicate case id "${duplicateId}"`);
+  }
+  const skillPath = typeof doc.skillPath === "string" && doc.skillPath.trim().length > 0 ? doc.skillPath.trim() : void 0;
+  return skillPath === void 0 ? { id, threshold, scorer, cases } : { id, threshold, scorer, skillPath, cases };
+};
+var loadEvalPack = async (packDir) => {
+  const packPath = path.join(packDir, "pack.json");
+  const text = await fs.readFile(packPath, "utf8");
+  let parsed;
+  try {
+    parsed = JSON.parse(text);
+  } catch {
+    throw new Error(`invalid JSON in ${packPath}`);
+  }
+  return parsePack(parsed);
+};
+var loadReplayMap = async (replayPath) => {
+  const text = await fs.readFile(replayPath, "utf8");
+  let parsed;
+  try {
+    parsed = JSON.parse(text);
+  } catch {
+    throw new Error(`invalid JSON in ${replayPath}`);
+  }
+  if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
+    throw new Error(`replay map must be a JSON object: ${replayPath}`);
+  }
+  const out = {};
+  for (const [key, value] of Object.entries(parsed)) {
+    if (typeof value !== "string") {
+      throw new Error(`replay map entry "${key}" must be a string (${replayPath})`);
+    }
+    out[key] = value;
+  }
+  return out;
+};
+
+// packages/eval/dist/run-eval-suite.js
+import path2 from "node:path";
+
+// packages/eval/dist/load-skill-via-read.js
+var loadSkillViaRead = async (harness, skillPath) => {
+  const result = await harness.invoke({
+    toolId: "read",
+    args: { path: skillPath }
+  });
+  if (!result.ok) {
+    throw new Error(`failed to read skill via harness read (${skillPath}): ${result.text}`);
+  }
+  return result.text;
+};
+
+// packages/eval/dist/score-case.js
+var scoreCase = (actual, expected, scorer) => {
+  const a = actual.trim();
+  const e = expected.trim();
+  switch (scorer) {
+    case "exact":
+      return a === e ? 1 : 0;
+    case "includes":
+      return a.includes(e) ? 1 : 0;
+  }
+};
+
+// packages/eval/dist/run-eval-suite.js
+var allowReadPermission = {
+  read: { "*": "allow" },
+  glob: { "*": "allow" },
+  grep: { "*": "allow" },
+  edit: { "*": "deny" },
+  write: { "*": "deny" },
+  create: { "*": "deny" },
+  delete: { "*": "deny" },
+  bash: { "*": "deny" }
+};
+var meanScore = (scores) => {
+  if (scores.length === 0) {
+    return 0;
+  }
+  const sum = scores.reduce((acc, n) => acc + n, 0);
+  return sum / scores.length;
+};
+var runEvalSuite = async (options) => {
+  const packDir = path2.resolve(options.packDir);
+  const projectRoot = path2.resolve(options.projectRoot ?? packDir);
+  const pack = await loadEvalPack(packDir);
+  const harness = options.harness ?? (pack.skillPath === void 0 ? null : createProjectHarness({
+    projectRoot,
+    permission: allowReadPermission
+  }));
+  const skillMarkdown = pack.skillPath === void 0 || harness === null ? null : await loadSkillViaRead(harness, pack.skillPath);
+  const caseResults = [];
+  for (const evalCase of pack.cases) {
+    const actual = await options.runCase({
+      pack,
+      case: evalCase,
+      skillMarkdown,
+      projectRoot,
+      harness
+    });
+    const scorer = evalCase.scorer ?? pack.scorer;
+    const score = scoreCase(actual, evalCase.expected, scorer);
+    caseResults.push({
+      caseId: evalCase.id,
+      score,
+      passed: score >= 1,
+      actual,
+      expected: evalCase.expected,
+      scorer
+    });
+  }
+  const suiteScore = meanScore(caseResults.map((c) => c.score));
+  const passed = suiteScore >= pack.threshold;
+  const failedCaseIds = caseResults.filter((c) => !c.passed).map((c) => c.caseId);
+  return {
+    packId: pack.id,
+    threshold: pack.threshold,
+    suiteScore,
+    passed,
+    skillMarkdown,
+    cases: caseResults,
+    failedCaseIds
+  };
+};
+var createReplayCaseRunner = (replay) => {
+  return async ({ case: evalCase }) => {
+    const actual = replay[evalCase.id];
+    if (actual === void 0) {
+      throw new Error(`replay map missing output for case "${evalCase.id}"`);
+    }
+    return actual;
+  };
+};
+
+// packages/cli/dist/create-fake-skill-case-runner.js
+var LINE_TOKEN_RULE = /^(.*)answer\s+exactly:\s*`([^`]+)`/i;
+var STOP_WORDS = /* @__PURE__ */ new Set([
+  "a",
+  "an",
+  "and",
+  "answer",
+  "asked",
+  "do",
+  "exactly",
+  "for",
+  "from",
+  "not",
+  "of",
+  "or",
+  "say",
+  "the",
+  "to",
+  "token",
+  "when"
+]);
+var tokenize = (text) => text.toLowerCase().split(/[^a-z0-9_]+/).filter((part) => part.length > 0 && !STOP_WORDS.has(part));
+var parseSkillTokenRules = (skillMarkdown) => skillMarkdown.split(/\r?\n/).flatMap((line) => {
+  const match = line.match(LINE_TOKEN_RULE);
+  if (match === null) {
+    return [];
+  }
+  const context = match[1]?.trim() ?? "";
+  const token = match[2]?.trim() ?? "";
+  if (token.length === 0) {
+    return [];
+  }
+  return [{ context, token }];
+});
+var scoreOverlap = (inputTokens, contextTokens) => {
+  if (contextTokens.length === 0) {
+    return 0;
+  }
+  const inputSet = new Set(inputTokens);
+  return contextTokens.filter((token) => inputSet.has(token)).length;
+};
+var createFakeSkillCaseRunner = () => {
+  return async ({ case: evalCase, skillMarkdown }) => {
+    if (skillMarkdown === null || skillMarkdown.trim().length === 0) {
+      throw new Error("Fake skill agent requires pack skillPath markdown; use --replay for packs without a skill");
+    }
+    const rules = parseSkillTokenRules(skillMarkdown);
+    if (rules.length === 0) {
+      throw new Error('Fake skill agent found no "answer exactly: `TOKEN`" rules in skill markdown');
+    }
+    const inputTokens = tokenize(evalCase.input);
+    const ranked = rules.map((rule) => ({
+      rule,
+      score: scoreOverlap(inputTokens, tokenize(rule.context))
+    })).sort((a, b) => b.score - a.score);
+    const best = ranked[0];
+    if (best === void 0 || best.score <= 0) {
+      throw new Error(`Fake skill agent could not match case "${evalCase.id}" input to a skill token rule`);
+    }
+    return best.rule.token;
+  };
+};
+
+// packages/cli/dist/eval-command.js
+var printSuiteSummary = (result) => {
+  console.log(`Pack: ${result.packId}`);
+  console.log(`Suite score: ${result.suiteScore.toFixed(3)} (threshold ${result.threshold})`);
+  for (const row of result.cases) {
+    const mark = row.passed ? "PASS" : "FAIL";
+    console.log(`  [${mark}] ${row.caseId} score=${row.score}`);
+  }
+  if (result.passed) {
+    console.log("Gate passed.");
+    return;
+  }
+  console.error(`Gate failed (regression). Failed cases: ${result.failedCaseIds.join(", ") || "(none \u2014 aggregate below threshold)"}`);
+};
+var resolveCaseRunner = async (opts) => {
+  if (opts.replay !== void 0) {
+    const replay = await loadReplayMap(path3.resolve(opts.replay));
+    return {
+      label: "replay map",
+      runCase: createReplayCaseRunner(replay)
+    };
+  }
+  return {
+    label: "Fake skill-token agent",
+    runCase: createFakeSkillCaseRunner()
+  };
+};
+var runEvalCommand = async (packDirArg, opts) => {
+  try {
+    const packDir = path3.resolve(packDirArg);
+    const projectRoot = path3.resolve(opts.project ?? packDir);
+    const { label, runCase } = await resolveCaseRunner(opts);
+    console.log(`Agent under test: ${label}`);
+    const result = await runEvalSuite({
+      packDir,
+      projectRoot,
+      runCase
+    });
+    printSuiteSummary(result);
+    if (!result.passed) {
+      process.exitCode = 1;
+    }
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    console.error(`langflower eval failed: ${message}`);
+    process.exitCode = 1;
+  }
+};
+export {
+  runEvalCommand
+};