@flowsint/plugin-sdk 2.6.1 → 2.6.4

package/README.md

@@ -1,374 +1,71 @@
-# Flowsint enricher plugins (`@flowsint/plugin-sdk`)
+# @flowsint/plugin-sdk
 
-The SDK for Flowsint local enricher plugins: the plugin contract and one
-TypeScript interface per Flowsint entity type. Types only.
+⚠️ This is a package used for later versions of Flowsint (>v2.0). This exposes Flowsint types through a typescript interface. This will later be useful for creating local (desktop version) enrichers, and extend the application.
 
-```sh
-npm i -D @flowsint/plugin-sdk@<Flowsint version you target>
-```
-
-The package version is the Flowsint version. A plugin built against
-`@flowsint/plugin-sdk@X.Y.Z` declares `"engines": { "flowsint": "^X.Y.Z" }`:
-removing or renaming a spec field only ships in a major release. This
-guarantee covers the published types only, not the app's runtime
-handling of what a plugin returns.
-
-A plugin is a directory dropped into Flowsint's plugins folder
-(`~/Library/Application Support/Flowsint/plugins/` on macOS,
-`%APPDATA%/Flowsint/plugins/` on Windows,
-`~/.config/Flowsint/plugins/` on Linux), containing:
-
-- `package.json` - see shape below.
-- `README.md` - **required**. Shown in Settings -> Plugins ("README"
-  button, rendered as markdown). Cover: what the dependency is, how to
-  install it, and your inputs/outputs contract. The section below
-  shows the layout Flowsint's own bundled plugins use.
-- Whatever `package.json`'s `"main"` points to. Your bundled entry
-  file, exporting `run` per the signature below.
-- `node_modules/`. Only if your bundler couldn't inline a dependency
-  (e.g. a native addon).
-
-Flowsint never runs `npm install` for you. Bundle your dependencies
-(esbuild, tsup, whatever) before distributing - your plugin must work
-by `require()` alone, on a machine with no package manager.
-
-## package.json
-
-```json
-{
-  "name": "my-plugin",
-  "version": "1.0.0",
-  "main": "dist/index.js",
-  "author": "you",
-  "engines": { "flowsint": "^2.6.1" },
-  "flowsint": {
-    "category": "Domain",
-    "description": "What this does, one sentence.",
-    "inputs": { "type": "domain", "properties": [{ "name": "domain", "type": "string" }] },
-    "outputs": { "type": "ip", "properties": [{ "name": "address", "type": "string" }] },
-    "icon": "Globe",
-    "installUrl": "https://example.com/install-docs",
-    "params": [
-      {
-        "name": "LIMIT",
-        "type": "string",
-        "description": "Max results",
-        "default": "50",
-        "required": false
-      },
-      {
-        "name": "API_KEY",
-        "type": "vaultSecret",
-        "description": "Your API key",
-        "default": "",
-        "required": true
-      }
-    ]
-  },
-  "dependencies": {}
-}
-```
-
-- `name` is your plugin's id - must be unique across every installed
-  plugin. Two plugins with the same `name`: the one Flowsint finds
-  first (alphabetical by directory name) loads, the other is skipped
-  with an error visible in Settings → Plugins.
-- `engines.flowsint` is a normal semver range, checked against the
-  running app version. Omit it to skip the check (not recommended).
-- `flowsint.category` and `flowsint.inputs` are required. Everything
-  else in the `flowsint` key is optional.
-- `flowsint.icon` is a [lucide-react](https://lucide.dev/icons/) icon
-  name.
-- `flowsint.installUrl` - link to your dependency's install docs, shown
-  as a clickable link in Settings → Plugins. Omit it if your plugin has
-  no external dependency (pure Node/OS APIs only). This is static
-  metadata, unrelated to `init()`'s `fix` message below - see "Checking
-  a dependency."
-- `flowsint.params` - optional. An array of
-  `{ name, type, description, default, required }` entries, shown
-  in the launch settings drawer before your plugin runs. A
-  `type: "vaultSecret"` entry is resolved against Flowsint's local
-  vault (the same store backing the chat feature's provider keys)
-  before your `run()` is called - you receive the decrypted secret
-  as a plain string in `params[name]` (or `null` if it's optional and
-  the user never set one), never a key id or reference.
-  `type: "select"` renders a dropdown of its
-  `options: [{ "label": "...", "value": "..." }]` (malformed entries
-  are dropped) and passes the picked `value`. `type: "date"` renders a
-  calendar and passes a `Date` at midnight UTC of the picked day
-  (`undefined` if left empty). `type: "number"` renders a numeric
-  input and passes a `number` (`undefined` if left empty).
-  `type: "boolean"` renders a switch and passes a `boolean` (`false`
-  if never touched). `type: "list"` renders a tags input and passes a
-  `string[]` (`[]` if empty). An invalid value fails the run before
-  `run()` is called. Any other `type` is passed through as the raw string the
-  user typed.
-
-## `main`'s export
-
-Two things your entry file can export - `run` is required, `init` is
-optional, nothing else is read. The contract lives in
-this package's `plugin.d.ts`, the one definition the app's plugin
-runner also imports. Shown here without
-its doc comments:
-
-```ts
-interface EnricherEntityInput {
-  type: string
-  properties: Record<string, unknown>
-}
+## Install
 
-interface LocalEnricherNode {
-  id: string
-  nodeType: string
-  nodeProperties: Record<string, unknown> // typed per nodeType, see below
-  nodeLabel?: string // falls back to `nodeType`
-  nodeSize?: number // falls back to the app's default size
-  sources?: string[] // overrides the finding's `source`
-}
-
-interface LocalEnricherEdge {
-  id: string
-  source: string
-  target: string
-  label: string
-  sources?: string[]
-}
-
-interface LocalEnricherEvent {
-  timestamp: number
-  endTimestamp?: number | null
-  title: string
-  description?: string | null
-  type?: string
-  confidence?: number | null
-  nodeIds?: string[] // ids of nodes in this same chunk
-  tags?: string[]
-  properties?: Record<string, unknown>
-}
-
-interface LocalEnricherFinding {
-  nodes: LocalEnricherNode[]
-  edges: LocalEnricherEdge[]
-  events?: LocalEnricherEvent[]
-}
-
-interface LocalEnricherLogger {
-  info(message: string): void
-  warn(message: string): void
-  error(message: string): void
-  success(message: string): void
-}
-
-// An async generator (`async function* run`): each yielded finding is a
-// chunk, merged into the graph as soon as it is yielded.
-function run(
-  entities: EnricherEntityInput[],
-  logger: LocalEnricherLogger,
-  params: Record<string, unknown>,
-  signal: AbortSignal
-): AsyncIterable<LocalEnricherFinding>
-```
-
-Yield one chunk per entity: the entity's node, what you found for it
-and the edges between them. **An edge's `source` and `target` (and an
-event's `nodeIds`) must be nodes of the same chunk.** When results are
-shared across entities, yield everything once at the end instead.
-Merging a chunk twice is a no-op.
-
-`signal` aborts after one hour: Flowsint stops reading your chunks
-and fails the run, keeping the chunks already merged. Pass it to
-`spawn(cmd, args, { signal })` and `fetch(url, { signal })`, and call
-`signal.throwIfAborted()` in a per-entity `catch` so an abort ends the
-run instead of being logged as one failed entity.
-
-`entities` are not narrowed per type: an input can be a user-defined
-custom type. Check `entity.type` before reading `entity.properties`.
-
-`params` mirrors your `flowsint.params` declaration by name - a
-plugin with no `params` field in `package.json` still receives an
-(empty) object, never `undefined`. See the `flowsint.params` bullet
-above for what a `vaultSecret` entry resolves to.
-
-```ts
-type InitResult = { ok: true } | { ok: false; message: string; fix: string }
-
-// Optional - omit entirely if your plugin has no runtime precondition
-// to check. Called once at app boot and again whenever the user clicks
-// "Reload plugins" in Settings → Plugins - never on a timer, never
-// before a run. A missing/failing dependency keeps your plugin out of
-// the enricher picker until init() resolves { ok: true }.
-function init(): Promise<InitResult>
+```sh
+# pnpm
+pnpm add @flowsint/plugin-sdk
+# yarn
+yarn add @flowsint/plugin-sdk
+# npm
+npm i @flowsint/plugin-sdk
 ```
 
-`logger` calls show up live in the sketch's Logs panel; anything you
-`throw` surfaces as a failed run there too (chunks already yielded stay).
-
-## Typing your plugin with `@flowsint/plugin-sdk`
-
-Import the contract instead of copying it. Types only: use
-`import type`, your bundler erases it and nothing ships at runtime.
+## Quick start
 
 ```ts
+import { randomUUID } from "node:crypto";
+import { lookup } from "node:dns/promises";
 import type {
-  EnricherEntityInput,
-  LocalEnricherFinding,
-  LocalEnricherLogger,
-  LocalEnricherNode
-} from '@flowsint/plugin-sdk'
-
-// Compile error: Ip has no `ip` field, its primary field is `address`.
-const node: LocalEnricherNode = { id, nodeType: 'ip', nodeProperties: { ip: '8.8.8.8' } }
-```
-
-`LocalEnricherNode` is discriminated by `nodeType`: `nodeProperties`
-is checked against that type's spec (the interfaces are generated from
-Flowsint's own type specs, so they never drift from them). Narrow on
-`nodeType` before reading properties
-(`nodes.filter((n) => n.nodeType === 'dnsrecord')`).
-
-`LocalEnricherRun` types `run` as a whole (`export const run:
-LocalEnricherRun = async function* (entities, logger, params, signal) {...}`).
-`PluginPackageJson` is the shape of your `package.json` and
-`LocalEnricherPlugin` the module the host loads from `main`.
-
-## Validating params before you spawn anything
-
-If your plugin shells out to a CLI (most do), the params you declared
-in `flowsint.params` will usually become argv values for that
-`spawn()` call. Calling `spawn(cmd, args)` with an argv array, never
-with `shell: true` and never by building a single command string
-yourself, already rules out classic shell-metacharacter injection
-(`;`, `|`, `` ` ``, `$()`) by construction: no shell ever parses the
-strings you pass, the OS execs your binary directly with each array
-entry as one literal argv token. Do not build commands by
-concatenating strings into `exec()`; use `spawn()` with an argv array
-every time.
-
-That guarantee alone is not a reason to pass a param straight through
-unchecked, though. A malformed value can still confuse your CLI's own
-argument parser (a stray value swallowed as the wrong flag's argument,
-a numeric flag rejecting a non-numeric string with a cryptic error) or
-trigger something your plugin didn't intend (a flag with a very
-different meaning than the one the param name suggests). Validate each
-param before using it, and fail loud - `throw` before calling `spawn`
-at all, not after - so the whole run aborts with a clear error instead
-of silently doing the wrong thing:
-
-```ts
-const INT_RE = /^\d+$/
-const NAME_LIST_RE = /^[A-Za-z0-9_-]+(,[A-Za-z0-9_-]+)*$/
-
-function requireInt(name: string, v: unknown): string | undefined {
-  if (typeof v !== 'string' || v.trim() === '') return undefined
-  const s = v.trim()
-  if (!INT_RE.test(s))
-    throw new Error(`Invalid ${name}: expected a non-negative integer, got "${s}"`)
-  return s
-}
-
-function requireList(name: string, v: unknown, pattern: RegExp): string | undefined {
-  if (typeof v !== 'string' || v.trim() === '') return undefined
-  const s = v.trim()
-  if (!pattern.test(s))
-    throw new Error(`Invalid ${name}: "${s}" contains characters outside the allowed set`)
-  return s
-}
-
-// usage: requireList('SOURCES', params.SOURCES, NAME_LIST_RE)
+  Domain,
+  Ip,
+  LocalEnricherRun,
+  LocalEnricherEdge,
+} from "@flowsint/plugin-sdk";
+
+export const run: LocalEnricherRun = async function* (entities, logger) {
+  for (const entity of entities) {
+    if (entity.type !== "domain") continue;
+    const domain: Domain = { domain: String(entity.properties.domain) };
+    const ip: Ip = { address: (await lookup(domain.domain)).address };
+    logger.info(`${domain.domain} resolves to ${ip.address}`);
+    const [domainId, ipId] = [randomUUID(), randomUUID()];
+    const relation: LocalEnricherEdge = {
+      id: randomUUID(),
+      source: domainId,
+      target: ipId,
+      label: "RESOLVES_TO",
+    };
+    yield {
+      nodes: [
+        { id: domainId, nodeType: "domain", nodeProperties: domain },
+        { id: ipId, nodeType: "ip", nodeProperties: ip },
+      ],
+      edges: [relation],
+    };
+  }
+};
 ```
 
-A URL-shaped param (a proxy, a webhook) is simplest to validate with
-`new URL(value)` - it throws on anything that isn't a real URL, which
-is exactly the check you want, and it's already in Node's stdlib.
-
-Build the whole argv array **once, before your entity loop**, not
-inside the per-entity `try`/`catch` most plugins use around their
-`spawn()` call. If a param is invalid, that's a configuration
-mistake, not a search failure for one entity - it should abort the
-whole run with a thrown error, not get silently caught and logged as
-a per-entity warning while the rest of the batch runs on stale/wrong
-settings.
+Build it (`npm i -D @flowsint/plugin-sdk esbuild` first), then install it:
 
-Logging the exact command you're about to run (`logger.info(...)`,
-built from the same array you pass to `spawn`, never a re-derived
-string) costs one line and pays for itself the first time a param
-doesn't do what someone expected - the sketch's Logs panel then shows
-precisely what ran, not just "started" or "failed."
+## Entity types
 
-Flowsint's own bundled plugins (seeded into your plugins folder by
-the app) all
-follow this pattern - `domain-to-dns-records`, `ip-to-ports`, and
-`username-to-social-accounts` in particular are worth reading if
-you want a fuller example with several param types (integers,
-comma-separated allowlisted lists, an exact-match set, a boolean
-flag) in one file.
+| Export                                     | What it is                                                             |
+| ------------------------------------------ | ---------------------------------------------------------------------- |
+| `Domain`, `Ip`, `Email`, `Individual`, ... | One interface per Flowsint entity type: the shape of `nodeProperties`. |
+| `FlowsintTypeMap`                          | Type key (`"domain"`, `"ip"`, ...) to its interface.                   |
+| `FlowsintTypeKey`                          | Union of every type key.                                               |
 
-## Checking a dependency (optional `init`)
-
-If your plugin shells out to an external tool - a CLI, a binary that
-must already be on the user's `PATH` - check for it in `init()` instead
-of letting `run()` fail with a bare "spawn ENOENT". There's no shared
-helper to import (see "nothing else is read" above) - copy this
-pattern into your own `src/index.ts`:
+A node is checked against its `nodeType`:
 
 ```ts
-import { spawn } from 'node:child_process'
-
-async function checkBinaryOnPath(bin: string, fix: string): Promise<InitResult> {
-  return new Promise((resolve) => {
-    const probe = spawn(process.platform === 'win32' ? 'where' : 'which', [bin])
-    probe.on('close', (code) =>
-      resolve(code === 0 ? { ok: true } : { ok: false, message: `${bin} not found on PATH`, fix })
-    )
-    probe.on('error', () => resolve({ ok: false, message: `${bin} not found on PATH`, fix }))
-  })
-}
-
-export async function init(): Promise<InitResult> {
-  return checkBinaryOnPath(
-    'subfinder',
-    "install with `brew install subfinder` or see the plugin's README"
-  )
-}
+// Compile error: Ip has no `ip` field, its primary field is `address`.
+const node: LocalEnricherNode = {
+  id: "i",
+  nodeType: "ip",
+  nodeProperties: { ip: "8.8.8.8" },
+};
 ```
-
-`message` is what's wrong, shown next to your plugin in Settings →
-Plugins. `fix` is free text rendered as-is, no link-parsing - a short
-human instruction, not necessarily a URL. The clickable link in
-Settings → Plugins comes from `flowsint.installUrl` in `package.json`
-instead (see above) - a separate, static piece of metadata that has
-nothing to do with `init()`.
-
-## Install / uninstall
-
-Develop wherever you like. The plugins directory above is an install
-target, not a workspace. Copy your plugin's folder into it, then
-either restart Flowsint or click "Reload plugins" in Settings ->
-Plugins. Both re-scan the plugins directory and re-run every plugin's
-`init()` check. To remove a plugin, delete its folder and do the same.
-There is no enable/disable toggle in v1. Settings -> Plugins is
-read-only, showing what's loaded, what's degraded and why, and what's
-failed to load entirely.
-
-Symlinking your dev folder into the plugins directory (instead of
-copying on every change) saves the copy step - rebuild, then just
-click "Reload plugins".
-
-## Testing your plugin
-
-No test harness ships in the app. "does it work" is checked by
-running it for real:
-
-1. Settings → Plugins - your plugin should show `loaded`. `degraded`
-   or `failed` shows the `init()` message/fix or load error right
-   there; fix that before anything else.
-2. Open a sketch, add an entity matching your `flowsint.inputs.type`.
-3. Pick your plugin from that entity's enricher list and run it.
-4. Bottom panel → "terminal" tab (`Cmd+Shift+T` to toggle) shows your
-   `logger.info/warn/error/success` calls live, plus anything your
-   `run`/`init` throws.
-5. Check the sketch's graph for the `nodes`/`edges` you returned.
-
-Changed the code? Rebuild, click "Reload plugins" in Settings →
-Plugins, re-run step 3 - no restart needed.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@flowsint/plugin-sdk",
-  "version": "2.6.1",
+  "version": "2.6.4",
   "description": "SDK for Flowsint local enricher plugins: the plugin contract and the entity property types. Types only: consume with `import type`.",
   "license": "Apache-2.0",
   "author": "dextmorgn <contact@flowsint.io>",