@fkws/klonk 0.0.4

package/README.md

@@ -0,0 +1,276 @@
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset=".github/assets/logo_dark_mode.png">
+  <source media="(prefers-color-scheme: light)" srcset=".github/assets/logo_bright_mode.png">
+  <img alt="Klonk Logo" src=".github/assets/logo_bright_mode.png">
+</picture>
+
+# Klonk
+
+Klonk is a **code-first, self-hosted automation engine** for TypeScript developers who demand true end-to-end type safety. It enables you to build complex, event-driven workflows with a fluent, declarative API.
+
+Stop wrestling with `any` types and stringly-typed data blobs between your automation steps. With Klonk, your entire workflow is a single, statically-checked TypeScript application, from the initial trigger to the final task.
+
+## Core Principles
+
+Klonk is designed to provide a developer experience focused on type safety and a code-first approach.
+
+### End-to-End Type Safety
+
+Klonk's fluent API leverages TypeScript's type system to provide compile-time safety across your entire workflow. As you add tasks to a `Playlist`, the `outputs` object available to subsequent tasks is **automatically and cumulatively typed**.
+
+```typescript
+.setPlaylist(p => p
+  .addTask(new A_Task("task-a", client), /* ... */)
+  // In the next step, `outputs["task-a"]` is fully typed! No `any`, no manual casting.
+  .addTask(new B_Task("task-b", client), (source, outputs) => ({
+      inputForB: outputs['task-a'].someProperty // Easily leverage auto-completion in your IDE
+  }))
+);
+```
+
+This means you get **flawless autocompletion** and can **prevent runtime errors** before your code ever runs, providing a level of safety and productivity not found in many GUI-based tools or other code-based orchestrators.
+
+### A Code-First Approach
+
+Klonk is built for developers who prefer to work with code. You use the full power and expressiveness of TypeScript to:
+-   Implement complex logic and data transformations with ease.
+-   Version control your workflows with Git.
+-   Write unit and integration tests.
+-   Integrate with your existing CI/CD pipelines.
+
+### Lightweight & Simple to Self-Host
+
+Klonk is incredibly lightweight, running as a simple Node.js or Bun process with no complex dependencies or external infrastructure. This makes it a simpler alternative to heavy-duty orchestrators like Airflow or Temporal. You can get a workflow up and running in minutes on your own server with no subscription fees.
+
+### Powerful Built-Ins
+`@fkws-npm/klonk/tasks` and `@fkws-npm/klonk/triggers` implement powerful built-ins. Klonk provides a rich set of triggers and tasks that cover a wide range of common automation scenarios, from file system events to webhook listeners. These components are designed to be robust and ready for production use. When combined with the official integrations for popular services like Dropbox, Notion, and OpenRouter, you have all the tools you need to build versatile and powerful workflows right out of the box, without having to write custom components for every step.
+
+## Features
+
+- **Statically-Checked Workflows**: Build complex automations with a fluent API where data passed between steps is fully type-safe.
+- **Modular by Design**: Easily extend the engine with your own custom triggers and tasks.
+- **Modern Integration Support**: Built-in, async-ready integrations for services like Notion, Dropbox, and OpenRouter.
+- **AI-Powered**: Leverage powerful AI models for intelligent document and data processing right out of the box.
+- **Code-First & Declarative**: Define workflows in pure TypeScript for maximum flexibility and clarity.
+- **Self-Hosted**: Run on your own servers with no recurring fees.
+
+## Installation
+
+```bash
+bun add @fkws-npm/klonk
+```
+
+Or with npm:
+
+```bash
+npm install @fkws-npm/klonk
+```
+
+## Core Concepts
+
+Klonk is built around a few key concepts that work together to create powerful automations:
+
+- **Workflow**: The main orchestrator. You build a workflow by adding triggers and then defining a single playlist of tasks to execute.
+- **Triggers**: Event sources that start a workflow's playlist. A trigger could be a file being added to a Dropbox folder, a webhook being called, or a scheduled timer. Each time a trigger fires, it produces an event that is passed to the playlist.
+- **Playlist**: An ordered sequence of tasks. The playlist is run once for every event produced by a trigger. It intelligently carries the typed outputs of each task to the ones that follow.
+- **Tasks**: The individual, atomic actions that make up a playlist, such as calling an API, downloading a file, or processing data. The output of one task is available by its unique ID to all subsequent tasks.
+
+## Getting Started: A Simple Workflow
+
+Here's a basic example that demonstrates the core concepts. This workflow triggers when a file is added to a Dropbox folder, downloads it, and then logs its contents.
+
+```typescript
+import { Workflow } from '@fkws-npm/klonk';
+import { TRDropboxFileAdded } from '@fkws-npm/klonk/triggers';
+import { TADropboxDownloadFile, TALogToConsole } from '@fkws-npm/klonk/tasks';
+import { IDropbox } from '@fkws-npm/klonk/integrations';
+
+// 1. Initialize your service clients
+const dropboxClient = new IDropbox({
+    appKey: process.env.DROPBOX_APP_KEY!,
+    appSecret: process.env.DROPBOX_APP_SECRET!,
+    refreshToken: process.env.DROPBOX_REFRESH_TOKEN!,
+});
+
+// 2. Define the workflow using the fluent API
+const simpleWorkflow = Workflow.create()
+  // Add a trigger. The type of `source.data` in the playlist will be inferred from this.
+  .addTrigger(
+    new TRDropboxFileAdded("new-file-in-dropbox", {
+      client: dropboxClient,
+      folderPath: "/MyKlonkFiles"
+    })
+  )
+  // Define the sequence of tasks to run when the trigger fires.
+  .setPlaylist(p =>
+    p.addTask(
+      // Each task has a unique identifier.
+      new TADropboxDownloadFile("download-the-file", dropboxClient),
+      // The builder function defines the task's input.
+      // `source` is the event from the trigger. `outputs` contains results from previous tasks.
+      (source, outputs) => ({
+        file_metadata: source.data // Fully typed, autocomplete-ready and will yell if wrong.
+      })
+    ) // -> Produces { file: Buffer }
+    .addTask(
+      // The second task logs the content of the downloaded file.
+      new TALogToConsole("log-the-content"),
+      (source, outputs) => ({
+        // The `file` property from the previous task's output is accessed in a type-safe way so you can safely call `.toString()`.
+        message: outputs["download-the-file"].file.toString(),
+      })
+    )
+  );
+
+// 3. Start the workflow
+simpleWorkflow.start();
+console.log("Simple workflow started! Waiting for new files in Dropbox...");
+```
+
+## Advanced Example: AI-Powered Invoice Processing
+
+This example shows a real-world workflow that:
+1. Triggers when a new invoice PDF is added to a Dropbox folder.
+2. Fetches the PDF content.
+3. Uses an AI model to extract structured data from the invoice.
+4. Creates a new record in a Notion database with the extracted data.
+
+```typescript
+import { Workflow } from '@fkws-npm/klonk';
+import { z } from 'zod';
+import { TRDropboxFileAdded } from '@fkws-npm/klonk/triggers';
+import {
+  TADropboxDownloadFile,
+  TAParsePdfAi,
+  TACreateNotionDatabaseItem
+} from '@fkws-npm/klonk/tasks';
+import { IDropbox, IOpenRouter, INotion } from '@fkws-npm/klonk/integrations';
+
+// --- Client & Schema Setup ---
+const dropbox = new IDropbox({ /* ... */ });
+const openRouter = new IOpenRouter({ apiKeyEnvVar: "OPENROUTER_API_KEY" });
+const notion = new INotion({ apiKeyEnvVar: "NOTION_API_KEY" });
+
+const InvoiceSchema = z.object({
+  invoice_number: z.string(),
+  total_amount: z.number(),
+  is_paid: z.boolean(),
+});
+
+// --- Workflow Definition ---
+const invoiceWorkflow = Workflow.create()
+  .addTrigger(new TRDropboxFileAdded("new-invoice", { client: dropbox, folderPath: "/Invoices" }))
+  .setPlaylist(p => p
+    // Step 1: Download the file from Dropbox
+    .addTask(new TADropboxDownloadFile("download-pdf", dropbox),
+      (source, outputs) => ({
+        file_metadata: source.data
+      })
+    )
+    // Step 2: Use AI to parse the downloaded PDF
+    .addTask(new TAParsePdfAi("parse-with-ai", openRouter),
+      (source, outputs) => ({
+        pdf: outputs['download-pdf'].file, // Type-safe access to previous output by its ID
+        instructions: "Extract the invoice number, total amount, and payment status from the PDF.",
+        zodSchema: InvoiceSchema,
+      })
+    )
+    // Step 3: Create a new item in a Notion database
+    .addTask(new TACreateNotionDatabaseItem("create-notion-record", notion),
+      (source, outputs) => {
+        const invoiceData = outputs['parse-with-ai']; // Type-safe access to the parsed data
+        return {
+          database_id: 'YOUR_INVOICES_DATABASE_ID',
+          properties: {
+            'Invoice Number': { title: [{ text: { content: invoiceData.invoice_number } }] },
+            'Total': { number: invoiceData.total_amount },
+            'Status': { select: { name: invoiceData.is_paid ? "Paid" : "Unpaid" } }
+          }
+        };
+      }
+    )
+  );
+
+// --- Start the workflow ---
+invoiceWorkflow.start();
+```
+
+## Creating Custom Components
+
+If your use case can't be served with Klonk's built-ins, you can easily extend Klonk by implementing the `Task` and `Trigger` abstract classes.
+
+### Custom Task Example
+
+A task must define its Input and Output types and implement `validateInput` and `run` methods.
+
+```typescript
+import { Task } from '@fkws-npm/klonk';
+import type { IEmailService } from './my-email-service'; // Your custom service
+
+// 1. Define the Input and Output types
+type EmailSenderInput = { to: string; subject: string; body: string; };
+type EmailSenderOutput = { messageId: string; };
+
+// 2. Create the class, extending the generic Task
+export class TAEmailSender<IdentType extends string> extends Task<
+  EmailSenderInput,
+  EmailSenderOutput,
+  IdentType
+> {
+  // 3. The constructor receives its ident and any clients it needs
+  constructor(ident: IdentType, private emailClient: IEmailService) {
+    super(ident);
+  }
+
+  // 4. For runtime validation
+  validateInput(input: EmailSenderInput): boolean {
+    return !!input.to && !!input.subject && !!input.body;
+  }
+
+  async run(input: EmailSenderInput): Promise<EmailSenderOutput> {
+    const result = await this.emailClient.send(input);
+    return { messageId: result.id };
+  }
+}
+```
+
+### Custom Trigger Example
+
+A trigger must define the type of event data it produces and can implement a `start` method for background polling.
+
+```typescript
+import { Trigger, TriggerEvent } from '@fkws-npm/klonk';
+
+// 1. Define the shape of the data this trigger produces
+type NewUserEventData = { userId: string; signupDate: Date; };
+
+// 2. Create the class, extending the generic Trigger
+export class TRNewUserSignedUp<IdentType extends string> extends Trigger<
+  IdentType,
+  TriggerEvent<IdentType, NewUserEventData>
+> {
+  constructor(ident: IdentType) {
+    super(ident);
+  }
+
+  // 3. The `start` method is called by the workflow to begin polling
+  async start(): Promise<void> {
+    // Example: check for new users every 10 seconds
+    setInterval(async () => {
+      const newUser = await this.checkForNewUserInDb();
+      if (newUser) {
+        // 4. Add new events to the `this.queue` property
+        this.queue.push({
+          triggerIdent: this.ident,
+          data: newUser
+        });
+      }
+    }, 10000);
+  }
+
+  private async checkForNewUserInDb(): Promise<NewUserEventData | null> {
+    // Your database logic here...
+    return null;
+  }
+}
+```

package/dist/cli.cjs

@@ -0,0 +1,132 @@
+#!/usr/bin/env node
+var import_node_module = require("node:module");
+var __create = Object.create;
+var __getProtoOf = Object.getPrototypeOf;
+var __defProp = Object.defineProperty;
+var __getOwnPropNames = Object.getOwnPropertyNames;
+var __hasOwnProp = Object.prototype.hasOwnProperty;
+var __toESM = (mod, isNodeMode, target) => {
+  target = mod != null ? __create(__getProtoOf(mod)) : {};
+  const to = isNodeMode || !mod || !mod.__esModule ? __defProp(target, "default", { value: mod, enumerable: true }) : target;
+  for (let key of __getOwnPropNames(mod))
+    if (!__hasOwnProp.call(to, key))
+      __defProp(to, key, {
+        get: () => mod[key],
+        enumerable: true
+      });
+  return to;
+};
+
+// src/cli.ts
+var import_commander = require("commander");
+var import_express = __toESM(require("express"));
+var import_json_to_zod = require("json-to-zod");
+var import_cli_highlight = require("cli-highlight");
+var readline = __toESM(require("node:readline/promises"));
+var import_node_process = require("node:process");
+var program = new import_commander.Command;
+program.name("klonk").description("CLI for Klonk workflow automation engine").version("0.1.18");
+program.command("test").description("Run a test to check if the CLI is working.").action(() => {
+  console.log("TEST SUCCESSFUL");
+});
+var setupCommand = program.command("setup").description("Setup integrations");
+var setupIntegrationCommand = setupCommand.command("integration").description("Setup a specific integration");
+setupIntegrationCommand.command("dropbox").description("Get a refresh token for Dropbox").action(async () => {
+  const rl = readline.createInterface({ input: import_node_process.stdin, output: import_node_process.stdout });
+  const appKey = await rl.question("Enter your Dropbox App Key: ");
+  const appSecret = await rl.question("Enter your Dropbox App Secret: ");
+  console.log(`
+Please go to this URL to authorize the app:
+https://www.dropbox.com/oauth2/authorize?client_id=${appKey}&response_type=code&token_access_type=offline
+`);
+  const authCode = await rl.question("Paste the authorization code here: ");
+  rl.close();
+  try {
+    const response = await fetch("https://api.dropbox.com/oauth2/token", {
+      method: "POST",
+      headers: {
+        "Content-Type": "application/x-www-form-urlencoded",
+        Authorization: `Basic ${Buffer.from(`${appKey}:${appSecret}`).toString("base64")}`
+      },
+      body: new URLSearchParams({
+        code: authCode,
+        grant_type: "authorization_code"
+      })
+    });
+    const data = await response.json();
+    if (!response.ok) {
+      console.error(`
+Error getting refresh token:`);
+      console.error(import_cli_highlight.highlight(JSON.stringify(data, null, 2), { language: "json" }));
+      process.exit(1);
+    }
+    console.log(`
+Success! Here are your tokens:`);
+    console.log(import_cli_highlight.highlight(JSON.stringify(data, null, 2), { language: "json" }));
+    console.log("\nStore the `refresh_token` securely. You will use it, along with your app key and secret, to configure the Dropbox integration.");
+  } catch (error) {
+    console.error(`
+An error occurred while fetching the refresh token:`, error);
+  }
+});
+var introspectCommand = program.command("introspect").description("Introspection utilities");
+introspectCommand.command("webhook-payload").description("Introspect a webhook payload").option("--gt, --generate-type", "Generate a TypeScript type definition for the payload").option("-p, --port <port>", "Port to listen on", "2021").option("--base-url <url>", "Base URL for the webhook endpoint").action(async (options) => {
+  const app = import_express.default();
+  app.use(import_express.default.json());
+  const port = parseInt(options.port, 10);
+  const displayUrl = options.baseUrl ? options.baseUrl : `http://localhost:${port}`;
+  let server = null;
+  const closeServer = () => {
+    if (server) {
+      server.close(() => {
+        console.log(`
+Server closed.`);
+        process.exit(0);
+      });
+    }
+  };
+  app.get("/", (req, res) => {
+    const challenge = req.query.challenge;
+    if (challenge && typeof challenge === "string") {
+      console.log(`
+Received verification challenge. Responding with: ${challenge}`);
+      res.setHeader("Content-Type", "text/plain");
+      res.status(200).send(challenge);
+      console.log("Challenge response sent. Waiting for POST payload...");
+      return;
+    }
+    console.warn(`
+Received GET request to / without a 'challenge' parameter.`);
+    res.status(400).send('Bad Request: Missing "challenge" query parameter.');
+  });
+  app.post("/", (req, res) => {
+    const payload = req.body;
+    console.log(`
+Webhook Payload:`);
+    console.log(import_cli_highlight.highlight(JSON.stringify(payload, null, 2), { language: "json", ignoreIllegals: true }));
+    if (options.generateType) {
+      try {
+        const zodSchemaString = import_json_to_zod.jsonToZod(payload, "PayloadSchema", true);
+        const typeAliasString = `
+type Payload = z.infer<typeof PayloadSchema>;`;
+        const combinedOutput = zodSchemaString + typeAliasString;
+        console.log(`
+Generated Zod Schema & TypeScript Type:`);
+        console.log(import_cli_highlight.highlight(combinedOutput, { language: "typescript", ignoreIllegals: true }));
+      } catch (error) {
+        console.error(`
+Error generating Zod schema:`, error);
+      }
+    }
+    res.status(200).send("Payload received");
+    closeServer();
+  });
+  server = app.listen(port, () => {
+    console.log(`Listening for webhook payload on ${displayUrl}`);
+  });
+  server.on("error", (error) => {
+    console.error("Server error:", error);
+    process.exit(1);
+  });
+});
+program.parse(process.argv);

package/dist/cli.js

@@ -0,0 +1,115 @@
+#!/usr/bin/env node
+
+// src/cli.ts
+import { Command } from "commander";
+import express from "express";
+import { jsonToZod } from "json-to-zod";
+import { highlight } from "cli-highlight";
+import * as readline from "node:readline/promises";
+import { stdin as input, stdout as output } from "node:process";
+var program = new Command;
+program.name("klonk").description("CLI for Klonk workflow automation engine").version("0.1.18");
+program.command("test").description("Run a test to check if the CLI is working.").action(() => {
+  console.log("TEST SUCCESSFUL");
+});
+var setupCommand = program.command("setup").description("Setup integrations");
+var setupIntegrationCommand = setupCommand.command("integration").description("Setup a specific integration");
+setupIntegrationCommand.command("dropbox").description("Get a refresh token for Dropbox").action(async () => {
+  const rl = readline.createInterface({ input, output });
+  const appKey = await rl.question("Enter your Dropbox App Key: ");
+  const appSecret = await rl.question("Enter your Dropbox App Secret: ");
+  console.log(`
+Please go to this URL to authorize the app:
+https://www.dropbox.com/oauth2/authorize?client_id=${appKey}&response_type=code&token_access_type=offline
+`);
+  const authCode = await rl.question("Paste the authorization code here: ");
+  rl.close();
+  try {
+    const response = await fetch("https://api.dropbox.com/oauth2/token", {
+      method: "POST",
+      headers: {
+        "Content-Type": "application/x-www-form-urlencoded",
+        Authorization: `Basic ${Buffer.from(`${appKey}:${appSecret}`).toString("base64")}`
+      },
+      body: new URLSearchParams({
+        code: authCode,
+        grant_type: "authorization_code"
+      })
+    });
+    const data = await response.json();
+    if (!response.ok) {
+      console.error(`
+Error getting refresh token:`);
+      console.error(highlight(JSON.stringify(data, null, 2), { language: "json" }));
+      process.exit(1);
+    }
+    console.log(`
+Success! Here are your tokens:`);
+    console.log(highlight(JSON.stringify(data, null, 2), { language: "json" }));
+    console.log("\nStore the `refresh_token` securely. You will use it, along with your app key and secret, to configure the Dropbox integration.");
+  } catch (error) {
+    console.error(`
+An error occurred while fetching the refresh token:`, error);
+  }
+});
+var introspectCommand = program.command("introspect").description("Introspection utilities");
+introspectCommand.command("webhook-payload").description("Introspect a webhook payload").option("--gt, --generate-type", "Generate a TypeScript type definition for the payload").option("-p, --port <port>", "Port to listen on", "2021").option("--base-url <url>", "Base URL for the webhook endpoint").action(async (options) => {
+  const app = express();
+  app.use(express.json());
+  const port = parseInt(options.port, 10);
+  const displayUrl = options.baseUrl ? options.baseUrl : `http://localhost:${port}`;
+  let server = null;
+  const closeServer = () => {
+    if (server) {
+      server.close(() => {
+        console.log(`
+Server closed.`);
+        process.exit(0);
+      });
+    }
+  };
+  app.get("/", (req, res) => {
+    const challenge = req.query.challenge;
+    if (challenge && typeof challenge === "string") {
+      console.log(`
+Received verification challenge. Responding with: ${challenge}`);
+      res.setHeader("Content-Type", "text/plain");
+      res.status(200).send(challenge);
+      console.log("Challenge response sent. Waiting for POST payload...");
+      return;
+    }
+    console.warn(`
+Received GET request to / without a 'challenge' parameter.`);
+    res.status(400).send('Bad Request: Missing "challenge" query parameter.');
+  });
+  app.post("/", (req, res) => {
+    const payload = req.body;
+    console.log(`
+Webhook Payload:`);
+    console.log(highlight(JSON.stringify(payload, null, 2), { language: "json", ignoreIllegals: true }));
+    if (options.generateType) {
+      try {
+        const zodSchemaString = jsonToZod(payload, "PayloadSchema", true);
+        const typeAliasString = `
+type Payload = z.infer<typeof PayloadSchema>;`;
+        const combinedOutput = zodSchemaString + typeAliasString;
+        console.log(`
+Generated Zod Schema & TypeScript Type:`);
+        console.log(highlight(combinedOutput, { language: "typescript", ignoreIllegals: true }));
+      } catch (error) {
+        console.error(`
+Error generating Zod schema:`, error);
+      }
+    }
+    res.status(200).send("Payload received");
+    closeServer();
+  });
+  server = app.listen(port, () => {
+    console.log(`Listening for webhook payload on ${displayUrl}`);
+  });
+  server.on("error", (error) => {
+    console.error("Server error:", error);
+    process.exit(1);
+  });
+});
+program.parse(process.argv);