langflower 0.1.0 → 0.1.2

package/README.md

@@ -3,125 +3,63 @@
 > **Disclaimer:** Langflower is currently in internal testing. Anyone can
 > download and try it, but some functions may not be stable.
 
-## Key concept: everything is a node
-
-Langflower is not built around the LLM as a first-class citizen. The core
-abstraction is a **reactive node**: several typed inputs, several typed
-outputs. Each port acts independently — a node can receive on one input
-and emit on one output at any time, without waiting for the rest. The
-runtime wires those nodes into a workflow.
-
-That is enough for complex processing chains — including loops and
-conditions.
-
-The trick is the same rule applied to models. Because ports fire on their
-own, an LLM agent is just another reactive node: prompt and tools in;
-response out; streaming tool log, reasoning, and draft as extra outputs
-that can update while the run continues. All LLM-specific logic stays as
-that node's internal state. It does not leak through the rest of the app.
-
-## Open possibilities
-
-Because every unit of work is a node, Langflower is not pinned to one
-product shape — coding harness, chat harness, or any other fixed loop.
-Workflows stay flexible. With the right wiring they fit the job.
-
-Need a chat? Wire user input, an agent, and HITL feedback. Agent-to-agent
-dialogue is two agents connected together. Coding is the same graph plus
-file-ops tools. Tired of “I’m done” while the code still does not compile?
-Write a custom review-gate node. The agent never gets a path around it.
-
-## Hard harness
-
-The sequence is the workflow topology. That is how Langflower orchestrates
-complex work: QA, review, and code checks sit on the graph, so agents
-cannot skip them. A well-configured workflow forces high-quality output —
-the model does not get to declare itself finished.
+Langflower is a local visual workflow for a project folder: a node graph
+that runs agents, tools, checks, and human gates on your machine. You
+author the pipeline on a canvas in the browser you already have. The LLM
+is a node in that graph, not the product — the topology you wire decides
+what happens next. Data, custom nodes, and workflows stay in that folder.
 
 **Not another chat harness. A local node graph.**
 
-### Keep everything local
-
-Langflower runs on your machine and does not expose your project as a
-hosted product. Files stay where they are. You can reproduce what mature
-cloud tools offer — at home, or on a closed network with internal LLM
-providers.
-
-### Scoped to a folder
-
-You start from a folder. Data, custom nodes, and workflows are scoped to
-that workspace. Open the folder, and the graph, files, and run belong
-together.
-
-### Extensible even now
-
-This is still an early version, but Langflower already supports custom
-nodes and custom node packs. Share and reuse workflows and nodes the same
-way you share the rest of the project.
+The unit of work is a **reactive node**: typed inputs, typed outputs.
+Each port fires on its own — a node can emit on one output without
+waiting for the rest, so the node chooses the path. That is enough for
+cycles and conditional branches, not only a straight chain. An LLM agent
+is the same kind of node — prompt and tools in, response out. Coding,
+chat, agent-to-agent dialogue, and custom gates are the same idea: pick
+the graph, not a fixed loop.
 
 ![Langflower starter workflow](https://raw.githubusercontent.com/earthdmitriy/langflower/master/docs/img/starter.png)
 
-## How it works
-
-1. Start Langflower with a project folder.
-2. Open a workflow on the canvas, or create one for the task.
-3. Run it and watch agents, tools, checks, and file changes move through
-   visible stages.
-4. Approve, reject, or add feedback when the workflow asks for a human
-   decision.
-5. Find the resulting files and data in the same workspace.
+The default **starter** workflow: an onboarding helper plus a Writer
+sub-agent for workflows and custom nodes.
 
-## Under the hood
+![Langflower dev workflow](https://raw.githubusercontent.com/earthdmitriy/langflower/master/docs/img/lf_dev.png)
 
-The core is a reactive runtime that wires nodes together. The **node SDK**
-is the public contract: any node that follows it can run on that runtime.
-**Common nodes** is the built-in catalog. It uses the same SDK as custom
-node packs.
-
-The **server** composes those pieces, compiles user-defined nodes, and owns
-the live run. The **UI** is a thin browser client. It listens to WebSocket
-events from the server; it does not own the workflow. Close the tab while a
-long run continues. Reopen it, and the server catches the UI up, so the
-canvas stays in sync.
-
-For a short builder-oriented picture, see
-[How it works](https://github.com/earthdmitriy/langflower/blob/master/docs/public/how-it-works.md).
+**Langflower dev** — the workflow used to develop Langflower itself.
+Memory tools, ts-scan (MCP), and custom **LF Dev Tools** merge in a tool
+collection and fan out to the agents. The prompt passes **lf-review-gate**
+first (format, typecheck, tests) so Plan starts from a green tree. A
+human review gate accepts the plan, then Coder works; the result must
+pass lf-review-gate again, then a human review gate.
 
 ## Why Langflower
 
-- **You stay in control.** Workflows can request approval before sensitive
-  file edits or shell commands run.
-- **Use the browser you already have.** Other harnesses pack a web UI into
-  a built-in browser such as Electron. Langflower uses your existing
-  browser, so you can close the tab, free those resources, and let the
-  server keep the run; reopen it and the UI catches up.
-- **Extend it when the defaults are not enough.** Langflower uses common
-  agent primitives — **MCP** and **skills** — and lets you define **custom
-  nodes** on top: processing in the graph, or custom tools for agents via
-  the same **ToolHandle** contract as built-ins.
-
-## How it compares
-
-Versus chat-style harnesses (often an Electron shell around a model loop):
-Langflower is a local node graph. The LLM is a node, not the product.
-Order comes from topology, not from the model deciding it is done.
-The UI is your existing browser, not a bundled one.
-
-Versus cloud graph tools: the same idea of wiring nodes, but aimed at a
-folder on your machine — home or a closed network with internal
-providers — not at hosting a service or cloning ETL in the cloud.
-
-## What it lacks
-
-- **Chat sessions.** Node-internal state is the current architecture, so
-  serializable chat-session mechanics are hard. Maybe later.
-- **Image and video.** No asset management for multimodal models. Not yet.
-- **No built-in IDE or git UI.** We are not reinventing those wheels. Use
-  the editor and git tools you already have.
+- **The graph is the harness.** QA, review, build, and tests sit on the
+  topology, so the agent cannot skip them or declare itself finished. A
+  review-gate has no path around it.
+- **Safe tools, not a general shell.** Wrap format, build, and unit tests
+  as custom nodes that expose a **ToolHandle**. You do not need a general
+  bash tool that could accidentally wipe all data from your disk. If a
+  command must stay open-ended, the workflow can still ask for approval
+  first.
+- **Your browser, not a bundled one.** Close the tab and free those
+  resources; the server keeps the run. Reopen it, and the UI catches up.
+
+Versus chat-style harnesses (OpenCode-like): order comes from the graph,
+not from the model deciding it is done. Versus cloud graph tools
+(Langflow, n8n): the same idea of wiring nodes, aimed at a folder on
+your machine — home or a closed network with internal providers — not at
+hosting a service.
+
+**MCP** and **skills** work as usual. Custom nodes sit on top of the same
+**ToolHandle** contract as built-ins.
 
 ## Quick start
 
-Requires **Node.js ≥ 22**.
+Requires **Node.js ≥ 22**. Live agent runs need an OpenAI-compatible
+provider in Settings (API keys via `{env:VAR_NAME}`). Simple nodes and
+the Fake LLM work without one.
 
 One-shot OS installers (Node LTS if needed + global `langflower`):
 [install/](install/) (`windows.ps1`, `linux.sh`, `macos.sh`).
@@ -146,6 +84,37 @@ instances from different folders at once.
 
 Full walkthrough: [Getting started](https://github.com/earthdmitriy/langflower/blob/master/docs/public/getting-started.md).
 
+## How it works
+
+1. Start Langflower with a project folder.
+2. Open a workflow on the canvas, or create one for the task.
+3. Run it and watch agents, tools, checks, and file changes move through
+   visible stages.
+4. Approve, reject, or add feedback when the workflow asks for a human
+   decision.
+5. Find the resulting files and data in the same workspace.
+
+## What it lacks
+
+- **Chat sessions.** You cannot save a chat and reopen it tomorrow.
+  Agent state lives inside the node for now. Maybe later.
+- **Image and video.** No asset management for multimodal models. Not yet.
+- **No built-in IDE or git UI.** We are not reinventing those wheels. Use
+  the editor and git tools you already have.
+
+## Under the hood
+
+The core is a reactive runtime. The **node SDK** is the public contract:
+any node that follows it can run on that runtime. **Common nodes** is the
+built-in catalog. It uses the same SDK as custom node packs.
+
+The **server** compiles user-defined nodes and owns the live run. The
+**UI** is a thin browser client over WebSocket; it does not own the
+workflow.
+
+For a short builder-oriented picture, see
+[How it works](https://github.com/earthdmitriy/langflower/blob/master/docs/public/how-it-works.md).
+
 Maintainers (monorepo only): [docs/RELEASE.md](https://github.com/earthdmitriy/langflower/blob/master/docs/RELEASE.md),
 [packages/cli/README.md](https://github.com/earthdmitriy/langflower/blob/master/packages/cli/README.md).
 

package/bin/langflower.js

@@ -1,3 +1,17 @@
 #!/usr/bin/env node
 
-import '../dist/index.js';
+const args = process.argv.slice(2);
+const first = args[0];
+const skipHeartbeat =
+	first === 'eval' ||
+	first === 'help' ||
+	args.includes('--help') ||
+	args.includes('-h') ||
+	args.includes('--version') ||
+	args.includes('-V');
+
+if (!skipHeartbeat) {
+	process.stdout.write('Starting Langflower...\n');
+}
+
+await import('../dist/index.js');

package/dist/chunk-AO33IM2J.js

@@ -0,0 +1,307 @@
+import { createRequire as __lfCreateRequire } from 'node:module';
+const require = __lfCreateRequire(import.meta.url);
+
+import {
+  createProjectHarness
+} from "./chunk-NIRPHJLO.js";
+import "./chunk-EAZMCX37.js";
+
+// packages/cli/dist/eval-command.js
+import path3 from "node:path";
+
+// packages/eval/dist/load-pack.js
+import fs from "node:fs/promises";
+import path from "node:path";
+var isScorerKind = (value) => value === "exact" || value === "includes";
+var parseCase = (raw, index) => {
+  if (raw === null || typeof raw !== "object") {
+    throw new Error(`eval pack case[${index}] must be an object`);
+  }
+  const row = raw;
+  const id = typeof row.id === "string" ? row.id.trim() : "";
+  const input = typeof row.input === "string" ? row.input : "";
+  const expected = typeof row.expected === "string" ? row.expected.trim() : "";
+  if (id.length === 0) {
+    throw new Error(`eval pack case[${index}] needs a non-empty id`);
+  }
+  if (expected.length === 0) {
+    throw new Error(`eval pack case "${id}" needs a non-empty expected`);
+  }
+  const scorer = row.scorer;
+  if (scorer !== void 0 && !isScorerKind(scorer)) {
+    throw new Error(`eval pack case "${id}" scorer must be "exact" or "includes"`);
+  }
+  return scorer === void 0 ? { id, input, expected } : { id, input, expected, scorer };
+};
+var parsePack = (raw) => {
+  if (raw === null || typeof raw !== "object") {
+    throw new Error("eval pack.json must be an object");
+  }
+  const doc = raw;
+  const id = typeof doc.id === "string" ? doc.id.trim() : "";
+  if (id.length === 0) {
+    throw new Error("eval pack.json needs a non-empty id");
+  }
+  const threshold = typeof doc.threshold === "number" && Number.isFinite(doc.threshold) ? doc.threshold : NaN;
+  if (!(threshold >= 0 && threshold <= 1)) {
+    throw new Error(`eval pack "${id}" threshold must be a number in [0, 1]`);
+  }
+  const scorer = doc.scorer;
+  if (!isScorerKind(scorer)) {
+    throw new Error(`eval pack "${id}" scorer must be "exact" or "includes"`);
+  }
+  if (!Array.isArray(doc.cases) || doc.cases.length === 0) {
+    throw new Error(`eval pack "${id}" needs a non-empty cases array`);
+  }
+  const cases = doc.cases.map((c, i) => parseCase(c, i));
+  const caseIds = cases.map((c) => c.id);
+  const duplicateId = caseIds.find((caseId, i) => caseIds.indexOf(caseId) !== i);
+  if (duplicateId !== void 0) {
+    throw new Error(`eval pack "${id}" has duplicate case id "${duplicateId}"`);
+  }
+  const skillPath = typeof doc.skillPath === "string" && doc.skillPath.trim().length > 0 ? doc.skillPath.trim() : void 0;
+  return skillPath === void 0 ? { id, threshold, scorer, cases } : { id, threshold, scorer, skillPath, cases };
+};
+var loadEvalPack = async (packDir) => {
+  const packPath = path.join(packDir, "pack.json");
+  const text = await fs.readFile(packPath, "utf8");
+  let parsed;
+  try {
+    parsed = JSON.parse(text);
+  } catch {
+    throw new Error(`invalid JSON in ${packPath}`);
+  }
+  return parsePack(parsed);
+};
+var loadReplayMap = async (replayPath) => {
+  const text = await fs.readFile(replayPath, "utf8");
+  let parsed;
+  try {
+    parsed = JSON.parse(text);
+  } catch {
+    throw new Error(`invalid JSON in ${replayPath}`);
+  }
+  if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
+    throw new Error(`replay map must be a JSON object: ${replayPath}`);
+  }
+  const out = {};
+  for (const [key, value] of Object.entries(parsed)) {
+    if (typeof value !== "string") {
+      throw new Error(`replay map entry "${key}" must be a string (${replayPath})`);
+    }
+    out[key] = value;
+  }
+  return out;
+};
+
+// packages/eval/dist/run-eval-suite.js
+import path2 from "node:path";
+
+// packages/eval/dist/load-skill-via-read.js
+var loadSkillViaRead = async (harness, skillPath) => {
+  const result = await harness.invoke({
+    toolId: "read",
+    args: { path: skillPath }
+  });
+  if (!result.ok) {
+    throw new Error(`failed to read skill via harness read (${skillPath}): ${result.text}`);
+  }
+  return result.text;
+};
+
+// packages/eval/dist/score-case.js
+var scoreCase = (actual, expected, scorer) => {
+  const a = actual.trim();
+  const e = expected.trim();
+  switch (scorer) {
+    case "exact":
+      return a === e ? 1 : 0;
+    case "includes":
+      return a.includes(e) ? 1 : 0;
+  }
+};
+
+// packages/eval/dist/run-eval-suite.js
+var allowReadPermission = {
+  read: { "*": "allow" },
+  glob: { "*": "allow" },
+  grep: { "*": "allow" },
+  edit: { "*": "deny" },
+  write: { "*": "deny" },
+  create: { "*": "deny" },
+  delete: { "*": "deny" },
+  bash: { "*": "deny" }
+};
+var meanScore = (scores) => {
+  if (scores.length === 0) {
+    return 0;
+  }
+  const sum = scores.reduce((acc, n) => acc + n, 0);
+  return sum / scores.length;
+};
+var runEvalSuite = async (options) => {
+  const packDir = path2.resolve(options.packDir);
+  const projectRoot = path2.resolve(options.projectRoot ?? packDir);
+  const pack = await loadEvalPack(packDir);
+  const harness = options.harness ?? (pack.skillPath === void 0 ? null : createProjectHarness({
+    projectRoot,
+    permission: allowReadPermission
+  }));
+  const skillMarkdown = pack.skillPath === void 0 || harness === null ? null : await loadSkillViaRead(harness, pack.skillPath);
+  const caseResults = [];
+  for (const evalCase of pack.cases) {
+    const actual = await options.runCase({
+      pack,
+      case: evalCase,
+      skillMarkdown,
+      projectRoot,
+      harness
+    });
+    const scorer = evalCase.scorer ?? pack.scorer;
+    const score = scoreCase(actual, evalCase.expected, scorer);
+    caseResults.push({
+      caseId: evalCase.id,
+      score,
+      passed: score >= 1,
+      actual,
+      expected: evalCase.expected,
+      scorer
+    });
+  }
+  const suiteScore = meanScore(caseResults.map((c) => c.score));
+  const passed = suiteScore >= pack.threshold;
+  const failedCaseIds = caseResults.filter((c) => !c.passed).map((c) => c.caseId);
+  return {
+    packId: pack.id,
+    threshold: pack.threshold,
+    suiteScore,
+    passed,
+    skillMarkdown,
+    cases: caseResults,
+    failedCaseIds
+  };
+};
+var createReplayCaseRunner = (replay) => {
+  return async ({ case: evalCase }) => {
+    const actual = replay[evalCase.id];
+    if (actual === void 0) {
+      throw new Error(`replay map missing output for case "${evalCase.id}"`);
+    }
+    return actual;
+  };
+};
+
+// packages/cli/dist/create-fake-skill-case-runner.js
+var LINE_TOKEN_RULE = /^(.*)answer\s+exactly:\s*`([^`]+)`/i;
+var STOP_WORDS = /* @__PURE__ */ new Set([
+  "a",
+  "an",
+  "and",
+  "answer",
+  "asked",
+  "do",
+  "exactly",
+  "for",
+  "from",
+  "not",
+  "of",
+  "or",
+  "say",
+  "the",
+  "to",
+  "token",
+  "when"
+]);
+var tokenize = (text) => text.toLowerCase().split(/[^a-z0-9_]+/).filter((part) => part.length > 0 && !STOP_WORDS.has(part));
+var parseSkillTokenRules = (skillMarkdown) => skillMarkdown.split(/\r?\n/).flatMap((line) => {
+  const match = line.match(LINE_TOKEN_RULE);
+  if (match === null) {
+    return [];
+  }
+  const context = match[1]?.trim() ?? "";
+  const token = match[2]?.trim() ?? "";
+  if (token.length === 0) {
+    return [];
+  }
+  return [{ context, token }];
+});
+var scoreOverlap = (inputTokens, contextTokens) => {
+  if (contextTokens.length === 0) {
+    return 0;
+  }
+  const inputSet = new Set(inputTokens);
+  return contextTokens.filter((token) => inputSet.has(token)).length;
+};
+var createFakeSkillCaseRunner = () => {
+  return async ({ case: evalCase, skillMarkdown }) => {
+    if (skillMarkdown === null || skillMarkdown.trim().length === 0) {
+      throw new Error("Fake skill agent requires pack skillPath markdown; use --replay for packs without a skill");
+    }
+    const rules = parseSkillTokenRules(skillMarkdown);
+    if (rules.length === 0) {
+      throw new Error('Fake skill agent found no "answer exactly: `TOKEN`" rules in skill markdown');
+    }
+    const inputTokens = tokenize(evalCase.input);
+    const ranked = rules.map((rule) => ({
+      rule,
+      score: scoreOverlap(inputTokens, tokenize(rule.context))
+    })).sort((a, b) => b.score - a.score);
+    const best = ranked[0];
+    if (best === void 0 || best.score <= 0) {
+      throw new Error(`Fake skill agent could not match case "${evalCase.id}" input to a skill token rule`);
+    }
+    return best.rule.token;
+  };
+};
+
+// packages/cli/dist/eval-command.js
+var printSuiteSummary = (result) => {
+  console.log(`Pack: ${result.packId}`);
+  console.log(`Suite score: ${result.suiteScore.toFixed(3)} (threshold ${result.threshold})`);
+  for (const row of result.cases) {
+    const mark = row.passed ? "PASS" : "FAIL";
+    console.log(`  [${mark}] ${row.caseId} score=${row.score}`);
+  }
+  if (result.passed) {
+    console.log("Gate passed.");
+    return;
+  }
+  console.error(`Gate failed (regression). Failed cases: ${result.failedCaseIds.join(", ") || "(none \u2014 aggregate below threshold)"}`);
+};
+var resolveCaseRunner = async (opts) => {
+  if (opts.replay !== void 0) {
+    const replay = await loadReplayMap(path3.resolve(opts.replay));
+    return {
+      label: "replay map",
+      runCase: createReplayCaseRunner(replay)
+    };
+  }
+  return {
+    label: "Fake skill-token agent",
+    runCase: createFakeSkillCaseRunner()
+  };
+};
+var runEvalCommand = async (packDirArg, opts) => {
+  try {
+    const packDir = path3.resolve(packDirArg);
+    const projectRoot = path3.resolve(opts.project ?? packDir);
+    const { label, runCase } = await resolveCaseRunner(opts);
+    console.log(`Agent under test: ${label}`);
+    const result = await runEvalSuite({
+      packDir,
+      projectRoot,
+      runCase
+    });
+    printSuiteSummary(result);
+    if (!result.passed) {
+      process.exitCode = 1;
+    }
+  } catch (error) {
+    const message = error instanceof Error ? error.message : String(error);
+    console.error(`langflower eval failed: ${message}`);
+    process.exitCode = 1;
+  }
+};
+export {
+  runEvalCommand
+};