fyflow-plugin-sdk 0.0.0-stage → 0.3.0

package/README.md

@@ -1,3 +1,138 @@
-# Temporary Holding Version
+# fyflow-plugin-sdk
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Types and `definePlugin` for writing **Plugins** for fyflow-app, the visual builder and Host for [fyflow-orchestration](https://www.npmjs.com/package/fyflow-orchestration) flows. A Plugin contributes Node Types (with the Workers that run their Tasks), gate types, Resource Group types and Resource Groups.
+
+```sh
+npm install --save-peer fyflow-plugin-sdk
+npm install --save-dev fyflow-orchestration   # its types, for type checking; the Host brings its own at run time
+```
+
+## The contract
+
+A Plugin is a module whose default export is a plain object. `definePlugin` returns its argument unchanged; it only gives you type checking. Depend on the SDK as a peer and do not bundle it: the Host provides it at run time (see below), and a copy of your own would only report its own `SDK_VERSION`, not the Host's.
+
+```ts
+import { definePlugin } from "fyflow-plugin-sdk";
+
+export default definePlugin({
+  id: "acme.signer", // stable and unique across Plugins
+  version: "1.0.0",
+  base: import.meta.url, // what Worker `url`s resolve against
+  workers: {
+    sign: {
+      url: "./sign.worker.js",
+      configSchema: {
+        type: "object",
+        properties: { key: { type: "string", writeOnly: true } }, // a Profile secret
+      },
+      settingsSchema: { type: "object", properties: { field: { type: "string" } } },
+    },
+  },
+  nodeTypes: {
+    Sign: {
+      worker: "sign",
+      inline: true,
+      groups: ["acme.signing"],
+      icon: "signature", // from the app's icon set, below
+      description: "Signs a field with HMAC.",
+    },
+  },
+  groups: { "acme.signing": { type: "ConcurrentLimitGroup", config: { limit: 2 } } },
+  gateTypes: {
+    threshold: {
+      routes: ["below", "at-or-above"],
+      defaultRoute: "below",
+      fn: ({ key, settings }) => (Number(key) >= Number(settings.at) ? "at-or-above" : "below"),
+    },
+  },
+});
+```
+
+| Field | What it is |
+| --- | --- |
+| `workers` | Worker modules by key. A Worker module's default export is a class: its constructor gets the Profile's config, and `run({ settings, input }, context)` does one Task (see [Workers](#workers)). Schemas are JSON Schema in fyflow's subset. A Worker's `outputSchema`, what `run` returns (accepted by every version of the SDK, as fyflow's own Worker entry), is the form a person fills in to mark a failed Task done, and the paths the editor suggests for conditions on its Node's Edges and gates before any Run. |
+| `nodeTypes` | Node Types by name, each naming one of this Plugin's `workers`, with fyflow's declaration fields (`inline`, `pool`, `config`, `settings` defaults, `retryPolicy`, `groups`), and, since 0.2.0, how the app shows it: an optional `icon` and a one-line `description`. |
+| `gateTypes` | Gate types by name: their routes, default route, settings schema, and the decision `fn`. |
+| `groupTypes` | Resource Group types by name: a `create(config, key)` factory for the scheduler's limit, and a `configSchema`. |
+| `groups` | Resource Groups by key, of fyflow's built-in types (`ConcurrentLimitGroup`, `RateLimitGroup`, `KeyedRateLimitGroup`) or this Plugin's own. A Node Type may use its own Plugin's groups, and the built-in Plugin's `disk` group (below). |
+
+Do not name a setting like a member of every JavaScript object (`constructor`, `toString`, `valueOf`, `hasOwnProperty`, `__proto__`). The Host hands Agents a Flow Definition's shape as a JSON Schema that includes your `settingsSchema`, and some JSON Schema validators misread such a property: they refuse, or throw, on a Flow that is valid.
+
+Every name a Plugin contributes (Node Type, gate type, Resource Group type, Resource Group) must be unique across the Plugins a Host loads: the Plugin the Host saw first keeps a name, and a later one that repeats it is skipped with the reason.
+
+## The Plugin directory
+
+A Host loads Plugins from its Plugin directory (`<dataDir>/plugins` unless `pluginDir`, `FYFLOW_PLUGIN_DIR` or `--plugin-dir` says otherwise):
+
+- a **folder** holding `plugin.ts` or `plugin.js`, or a package (`package.json` and its entry), as the Host lays out a Plugin installed from the app;
+- a **package** named `fyflow-plugin-*` or `@scope/fyflow-plugin-*` under the directory's `node_modules`, installed with npm. A package must ship JavaScript: Node does not strip TypeScript under `node_modules`.
+
+A Plugin's own imports resolve from the Plugin directory's `node_modules`, so `npm install` your other dependencies there (or bundle them). The SDK is the Host's to provide: on a server, `npm install fyflow-plugin-sdk` there (npm also installs it as your package's peer); the desktop app puts its own there by itself. Changes take effect when the Host is told to **Reload**.
+
+## Peer range
+
+Declare the SDK versions your Plugin works with as a peer dependency:
+
+```json
+{ "peerDependencies": { "fyflow-plugin-sdk": "^0.2.0" } }
+```
+
+A Host implements its SDK version (`SDK_VERSION`, exported here) and the earlier ones it is compatible with (`SDK_COMPATIBLE`): a 0.3 Host still loads a Plugin declaring `^0.2.0` or `^0.1.0`. It skips, with the reason, a Plugin whose range excludes them all. Raise the range only for what a Host must do at run time: declare `^0.2.0` when you use `icon` or `description`. The Worker types of 0.3.0 vanish at run time (`import type`), so take 0.3 as a devDependency for them and leave the peer range as it was.
+
+## Icons
+
+A Node Type's `icon` names one of the app's icons, from [Lucide](https://lucide.dev/icons/): `activity`, `archive`, `bell`, `bot`, `box`, `braces`, `brain`, `calculator`, `calendar`, `camera`, `chart-bar`, `check`, `clock`, `cloud`, `code`, `cpu`, `credit-card`, `database`, `download`, `file`, `file-text`, `filter`, `flag`, `folder`, `git-branch`, `globe`, `hash`, `image`, `key`, `link`, `list`, `lock`, `mail`, `map-pin`, `message-square`, `package`, `pen-tool`, `play`, `plug`, `repeat`, `search`, `send`, `server`, `settings`, `shield`, `shopping-cart`, `signature`, `sparkles`, `table`, `tag`, `terminal`, `timer`, `trash`, `upload`, `user`, `users`, `wand-sparkles`, `webhook`, `zap`. A name outside the set is not an error: the Node Type shows its Kind's icon, and the Plugins page shows a warning.
+
+## Workers
+
+A Worker module's default export is a class. The Host builds each instance (one per worker of the pool: a thread, or on an inline pool an object on the Host's own thread) with the config of the Profile the pool is for, and reuses it across Tasks, so keep per-Task state out of fields. `run` gets the Task's envelope (the Node's `settings` and the Task input) and a context; what it returns is the Task's result, and throwing fails the Task with the error's message. Since 0.3.0 the SDK types all of it:
+
+```ts
+import type { TaskEnvelope, Worker, WorkerContext } from "fyflow-plugin-sdk";
+
+type Config = { key?: string };
+type Settings = { field?: string };
+
+export default class Sign implements Worker<Settings> {
+  readonly #config: Config;
+
+  constructor(config: Config) {
+    this.#config = config;
+  }
+
+  async run({ settings, input }: TaskEnvelope<Settings>, context: WorkerContext) {
+    if (this.#config.key === undefined) throw new Error("No key: set `key` on this Node's Profile");
+    context.sendProgress(0, "signing"); // a fraction from 0 to 1, not a percentage
+    context.signal.throwIfAborted(); // aborted when the Task is Cancelled
+    const value = settings.field === undefined ? input : (input as Record<string, unknown>)[settings.field];
+    return { signed: await hmac(this.#config.key, value) }; // hmac: your own signing function
+  }
+}
+```
+
+`WorkerClass<Config, Settings, Input, Output>` types the class itself, constructor included. Check `context.signal` between steps, or pass it to anything that takes one (`fetch`, streams): a Worker that stops promptly frees its slot sooner, and what it returns after a Cancel is discarded. Use `import type`, so a Worker module needs nothing from the SDK at run time. The context offers what the Host shows (progress) and Cancel; fyflow's scheduler passes more, but the SDK promises only these.
+
+## The `disk` group
+
+The Host's built-in Plugin declares one Resource Group any Plugin's Node Types may name: `disk` (exported as `DISK_GROUP` since 0.3.0), a concurrency limit on Tasks that read or write the Host's disks. It is 4 unless the Host overrides it on its Resource Groups page. Name it when your Tasks do heavy disk work (copying, transcoding, indexing), so they share one budget with the Host's own file system Node Types instead of thrashing the disk beside them:
+
+```ts
+import { DISK_GROUP, definePlugin } from "fyflow-plugin-sdk";
+
+export default definePlugin({
+  // …
+  nodeTypes: { Transcode: { worker: "transcode", groups: [DISK_GROUP] } },
+});
+```
+
+Other Plugins' groups stay out of reach, so Disabling one Plugin never breaks another. `DISK_GROUP` is a value, so a Plugin that imports it needs the SDK at run time; writing `"disk"` instead works on any Host that has the group.
+
+## Profiles and secrets
+
+A Node Type's Worker config lives on the Host, not in Flows: a **Profile** is a named configuration of a Node Type, edited in the Host's UI with a form built from the Worker's `configSchema`, and a task Node chooses one. Every Node Type has a default Profile seeded from its declaration's `config`. A Profile's config is merged over the default's, which is merged over yours.
+
+Mark a config property `writeOnly: true` to make it a **secret**: the Host never shows it again, only whether it is set, and redacts its value from errors and logs. Only properties of the config object can be secrets.
+
+## Versioning
+
+This package follows semver. `SDK_VERSION` always equals the package version.

package/dist/index.d.ts

@@ -0,0 +1,80 @@
+import type { GateType, GroupInstance, GroupType, NodeTypeDeclaration, WorkerEntry } from "fyflow-orchestration";
+export type { GateContext, GateType, GroupInstance, GroupType, JsonSchema, NodeTypeDeclaration, RetryPolicy, WorkerEntry, } from "fyflow-orchestration";
+/**
+ * The SDK version this package is, and so the one a Host built with it implements. A Plugin whose
+ * `fyflow-plugin-sdk` peer range excludes it, and every one in `SDK_COMPATIBLE`, is not loaded. Kept equal to
+ * package.json's `version` (tested).
+ */
+export declare const SDK_VERSION = "0.3.0";
+/**
+ * The earlier SDK versions this one is a superset of: nothing a Plugin written for them says has changed meaning,
+ * so a Host implements them too (ADR-0021). 0.2.0 only added optional fields, and 0.3.0 only types.
+ */
+export declare const SDK_COMPATIBLE: readonly string[];
+/**
+ * The key of the `disk` Resource Group the Host's built-in Plugin declares (exported since 0.3.0): a concurrency
+ * limit (4 unless the Host overrides it) on Tasks that read or write the Host's disks. Any Plugin's Node Type may
+ * name it in `groups`, so a Plugin doing heavy disk work shares one budget with every other. Other Plugins' groups
+ * are out of reach.
+ */
+export declare const DISK_GROUP = "disk";
+/** A Node Type as a Plugin declares it: fyflow's declaration, and how the app shows it (since 0.2.0). */
+export interface PluginNodeType extends NodeTypeDeclaration {
+    /** An icon from the app's icon set (Lucide names, listed in the README); an unknown one shows its Kind's icon. */
+    icon?: string;
+    /** One line on what it does, shown in the palette. */
+    description?: string;
+}
+export interface Plugin {
+    /** Stable, unique id, e.g. `acme.images`. */
+    id: string;
+    version: string;
+    /** What worker `url`s resolve against: the plugin module's own `import.meta.url`. */
+    base: string;
+    /**
+     * Worker modules by key; `url` is relative to `base`. A property of a `configSchema` marked `writeOnly: true` is
+     * a Profile secret: a Host never shows it (ADR-0016).
+     */
+    workers?: Readonly<Record<string, WorkerEntry>>;
+    /**
+     * Node Types by name, each naming one of this plugin's `workers`; `groups` names Resource Groups (from this
+     * plugin's `groups`) every Task of the type acquires.
+     */
+    nodeTypes?: Readonly<Record<string, PluginNodeType>>;
+    /** Gate types by name: the routes a gate of the type takes, and its decision. */
+    gateTypes?: Readonly<Record<string, GateType>>;
+    /** Resource Group types by name: a factory for the scheduler's limit, from a Resource Group's `config`. */
+    groupTypes?: Readonly<Record<string, GroupType>>;
+    /** Resource Groups by key: an instance of a group type (fyflow's built-ins or this plugin's), with its config. */
+    groups?: Readonly<Record<string, GroupInstance>>;
+}
+/**
+ * What a Worker's `run` is handed for one Task: the Node's `settings` (checked by the Worker's `settingsSchema`)
+ * and the Task input, after the Node's transform if it has one (since 0.3.0).
+ */
+export interface TaskEnvelope<Settings = Record<string, unknown>, Input = unknown> {
+    settings: Settings;
+    input: Input;
+}
+/** The second argument to a Worker's `run`: what it may do while its Task runs (since 0.3.0). */
+export interface WorkerContext {
+    /** Reports how far the Task is, as a fraction from 0 to 1 (not a percentage), with an optional message. */
+    sendProgress(progress: number, message?: string): void;
+    /**
+     * Aborted when the Task is Cancelled. Pass it on to anything that takes one, or check it between steps: a Worker
+     * that stops promptly frees its slot sooner. What `run` returns after it aborts is discarded.
+     */
+    readonly signal: AbortSignal;
+}
+/** One instance of a Worker, reused across the Tasks of its pool (since 0.3.0). */
+export interface Worker<Settings = Record<string, unknown>, Input = unknown, Output = unknown> {
+    /** Does one Task. Throwing fails it; the error's message is what the Host shows. */
+    run(envelope: TaskEnvelope<Settings, Input>, context: WorkerContext): Output | Promise<Output>;
+}
+/**
+ * A Worker module's default export: a class whose constructor gets the config of the Profile its pool was built
+ * for (the Worker's `configSchema` checks it), once per instance (since 0.3.0).
+ */
+export type WorkerClass<Config = Record<string, unknown>, Settings = Record<string, unknown>, Input = unknown, Output = unknown> = new (config: Config) => Worker<Settings, Input, Output>;
+/** Identity at runtime; gives plugin authors type checking and a single import. */
+export declare function definePlugin(plugin: Plugin): Plugin;

package/dist/index.js

@@ -0,0 +1,116 @@
+// The contract between fyflow-app and the code that extends it (ADR-0007, ADR-0015, ADR-0016). Published as
+// `fyflow-plugin-sdk`: everything here is types but `definePlugin`, `SDK_VERSION`, `SDK_COMPATIBLE` and
+// `DISK_GROUP`.
+             
+           
+                
+            
+                      
+              
+                              
+
+             
+              
+           
+                
+            
+             
+                      
+              
+              
+                              
+
+/**
+ * The SDK version this package is, and so the one a Host built with it implements. A Plugin whose
+ * `fyflow-plugin-sdk` peer range excludes it, and every one in `SDK_COMPATIBLE`, is not loaded. Kept equal to
+ * package.json's `version` (tested).
+ */
+export const SDK_VERSION = "0.3.0";
+
+/**
+ * The earlier SDK versions this one is a superset of: nothing a Plugin written for them says has changed meaning,
+ * so a Host implements them too (ADR-0021). 0.2.0 only added optional fields, and 0.3.0 only types.
+ */
+export const SDK_COMPATIBLE                    = ["0.2.0", "0.1.0"];
+
+/**
+ * The key of the `disk` Resource Group the Host's built-in Plugin declares (exported since 0.3.0): a concurrency
+ * limit (4 unless the Host overrides it) on Tasks that read or write the Host's disks. Any Plugin's Node Type may
+ * name it in `groups`, so a Plugin doing heavy disk work shares one budget with every other. Other Plugins' groups
+ * are out of reach.
+ */
+export const DISK_GROUP = "disk";
+
+/** A Node Type as a Plugin declares it: fyflow's declaration, and how the app shows it (since 0.2.0). */
+                                                             
+                                                                                                                    
+                
+                                                        
+                       
+ 
+
+                         
+                                               
+             
+                  
+                                                                                       
+               
+     
+                                                                                                                   
+                                                        
+     
+                                                  
+     
+                                                                                                              
+                                                        
+     
+                                                       
+                                                                                   
+                                                 
+                                                                                                             
+                                                   
+                                                                                                                    
+                                                   
+ 
+
+/**
+ * What a Worker's `run` is handed for one Task: the Node's `settings` (checked by the Worker's `settingsSchema`)
+ * and the Task input, after the Node's transform if it has one (since 0.3.0).
+ */
+                                                                                    
+                     
+               
+ 
+
+/** The second argument to a Worker's `run`: what it may do while its Task runs (since 0.3.0). */
+                                
+                                                                                                             
+                                                         
+     
+                                                                                                                   
+                                                                                                
+     
+                               
+ 
+
+/** One instance of a Worker, reused across the Tasks of its pool (since 0.3.0). */
+                                                                                                
+                                                                                      
+                                                                                                 
+ 
+
+/**
+ * A Worker module's default export: a class whose constructor gets the config of the Profile its pool was built
+ * for (the Worker's `configSchema` checks it), once per instance (since 0.3.0).
+ */
+                        
+                                   
+                                     
+                  
+                   
+                                                            
+
+/** Identity at runtime; gives plugin authors type checking and a single import. */
+export function definePlugin(plugin        )         {
+  return plugin;
+}

package/package.json

@@ -1,6 +1,40 @@
 {
   "name": "fyflow-plugin-sdk",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
+  "version": "0.3.0",
+  "description": "Types and definePlugin for fyflow-app Plugins: Node Types, Workers, gate types and Resource Groups",
+  "keywords": [
+    "fyflow",
+    "fyflow-plugin",
+    "plugin",
+    "workflow"
+  ],
+  "type": "module",
+  "license": "MIT",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    }
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "peerDependencies": {
+    "fyflow-orchestration": "^0.12.0"
+  },
+  "peerDependenciesMeta": {
+    "fyflow-orchestration": {
+      "optional": true
+    }
+  },
+  "devDependencies": {
+    "fyflow-orchestration": "0.12.0"
+  },
+  "files": [
+    "dist",
+    "src",
+    "!src/**/*.test.ts"
+  ],
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts"
 }

package/src/index.ts

@@ -0,0 +1,116 @@
+// The contract between fyflow-app and the code that extends it (ADR-0007, ADR-0015, ADR-0016). Published as
+// `fyflow-plugin-sdk`: everything here is types but `definePlugin`, `SDK_VERSION`, `SDK_COMPATIBLE` and
+// `DISK_GROUP`.
+import type {
+  GateType,
+  GroupInstance,
+  GroupType,
+  NodeTypeDeclaration,
+  WorkerEntry,
+} from "fyflow-orchestration";
+
+export type {
+  GateContext,
+  GateType,
+  GroupInstance,
+  GroupType,
+  JsonSchema,
+  NodeTypeDeclaration,
+  RetryPolicy,
+  WorkerEntry,
+} from "fyflow-orchestration";
+
+/**
+ * The SDK version this package is, and so the one a Host built with it implements. A Plugin whose
+ * `fyflow-plugin-sdk` peer range excludes it, and every one in `SDK_COMPATIBLE`, is not loaded. Kept equal to
+ * package.json's `version` (tested).
+ */
+export const SDK_VERSION = "0.3.0";
+
+/**
+ * The earlier SDK versions this one is a superset of: nothing a Plugin written for them says has changed meaning,
+ * so a Host implements them too (ADR-0021). 0.2.0 only added optional fields, and 0.3.0 only types.
+ */
+export const SDK_COMPATIBLE: readonly string[] = ["0.2.0", "0.1.0"];
+
+/**
+ * The key of the `disk` Resource Group the Host's built-in Plugin declares (exported since 0.3.0): a concurrency
+ * limit (4 unless the Host overrides it) on Tasks that read or write the Host's disks. Any Plugin's Node Type may
+ * name it in `groups`, so a Plugin doing heavy disk work shares one budget with every other. Other Plugins' groups
+ * are out of reach.
+ */
+export const DISK_GROUP = "disk";
+
+/** A Node Type as a Plugin declares it: fyflow's declaration, and how the app shows it (since 0.2.0). */
+export interface PluginNodeType extends NodeTypeDeclaration {
+  /** An icon from the app's icon set (Lucide names, listed in the README); an unknown one shows its Kind's icon. */
+  icon?: string;
+  /** One line on what it does, shown in the palette. */
+  description?: string;
+}
+
+export interface Plugin {
+  /** Stable, unique id, e.g. `acme.images`. */
+  id: string;
+  version: string;
+  /** What worker `url`s resolve against: the plugin module's own `import.meta.url`. */
+  base: string;
+  /**
+   * Worker modules by key; `url` is relative to `base`. A property of a `configSchema` marked `writeOnly: true` is
+   * a Profile secret: a Host never shows it (ADR-0016).
+   */
+  workers?: Readonly<Record<string, WorkerEntry>>;
+  /**
+   * Node Types by name, each naming one of this plugin's `workers`; `groups` names Resource Groups (from this
+   * plugin's `groups`) every Task of the type acquires.
+   */
+  nodeTypes?: Readonly<Record<string, PluginNodeType>>;
+  /** Gate types by name: the routes a gate of the type takes, and its decision. */
+  gateTypes?: Readonly<Record<string, GateType>>;
+  /** Resource Group types by name: a factory for the scheduler's limit, from a Resource Group's `config`. */
+  groupTypes?: Readonly<Record<string, GroupType>>;
+  /** Resource Groups by key: an instance of a group type (fyflow's built-ins or this plugin's), with its config. */
+  groups?: Readonly<Record<string, GroupInstance>>;
+}
+
+/**
+ * What a Worker's `run` is handed for one Task: the Node's `settings` (checked by the Worker's `settingsSchema`)
+ * and the Task input, after the Node's transform if it has one (since 0.3.0).
+ */
+export interface TaskEnvelope<Settings = Record<string, unknown>, Input = unknown> {
+  settings: Settings;
+  input: Input;
+}
+
+/** The second argument to a Worker's `run`: what it may do while its Task runs (since 0.3.0). */
+export interface WorkerContext {
+  /** Reports how far the Task is, as a fraction from 0 to 1 (not a percentage), with an optional message. */
+  sendProgress(progress: number, message?: string): void;
+  /**
+   * Aborted when the Task is Cancelled. Pass it on to anything that takes one, or check it between steps: a Worker
+   * that stops promptly frees its slot sooner. What `run` returns after it aborts is discarded.
+   */
+  readonly signal: AbortSignal;
+}
+
+/** One instance of a Worker, reused across the Tasks of its pool (since 0.3.0). */
+export interface Worker<Settings = Record<string, unknown>, Input = unknown, Output = unknown> {
+  /** Does one Task. Throwing fails it; the error's message is what the Host shows. */
+  run(envelope: TaskEnvelope<Settings, Input>, context: WorkerContext): Output | Promise<Output>;
+}
+
+/**
+ * A Worker module's default export: a class whose constructor gets the config of the Profile its pool was built
+ * for (the Worker's `configSchema` checks it), once per instance (since 0.3.0).
+ */
+export type WorkerClass<
+  Config = Record<string, unknown>,
+  Settings = Record<string, unknown>,
+  Input = unknown,
+  Output = unknown,
+> = new (config: Config) => Worker<Settings, Input, Output>;
+
+/** Identity at runtime; gives plugin authors type checking and a single import. */
+export function definePlugin(plugin: Plugin): Plugin {
+  return plugin;
+}