@hogsend/cli 0.7.0 → 0.9.0

package/skills/hogsend-webhooks-and-workflows/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: hogsend-webhooks-and-workflows
-description: Use when adding an inbound webhook source in src/webhook-sources/ (defineWebhookSource — auth header + envKey, optional Zod schema, transform(payload, ctx) -> IngestEvent | null, served at POST /v1/webhooks/:id) or a custom Hatchet task in src/workflows/ passed as extraWorkflows (NOT workflows) to createWorker, including the idempotent batched expand→migrate→contract backfill pattern.
+description: Use when adding an inbound webhook source in src/webhook-sources/ (defineWebhookSource — auth as a match|signature discriminated union, optional Zod schema, transform(payload, ctx) -> IngestEvent | null, served at POST /v1/webhooks/:id), reaching for a built-in integration preset (Clerk/Supabase/Stripe/Segment), or a custom Hatchet task in src/workflows/ passed as extraWorkflows (NOT workflows) to createWorker, including the idempotent batched expand→migrate→contract backfill pattern. Outbound signed webhooks are managed separately (hogsend webhooks CLI / hs.webhooks).
 license: MIT
 metadata:
   author: withSeismic
@@ -25,10 +25,20 @@ Relative imports use the ESM `.js` extension.
 
 - **`defineWebhookSource({ meta, auth, schema?, transform })`** (from
   `@hogsend/engine`) — declares one source served at `POST /v1/webhooks/:id`.
-  `auth` matches a request header against an env secret; `schema` is an optional
-  Zod validator; `transform(payload, ctx)` returns an `IngestEvent | null`
-  (`null` = accept-and-skip). Register sources in `src/webhook-sources/index.ts`
-  and pass them to `createApp(client, { webhookSources })` in `src/index.ts`.
+  `auth` is a **discriminated union on `type`**: `"match"` (shared-secret
+  equality against a header/`Authorization: Bearer`; OPEN when the secret is
+  unset) or `"signature"` (`scheme: "svix" | "stripe" | "hmac-hex"`, with an
+  `envKey`, optional `header`/`fallbackMatchHeader`; FAILS CLOSED with 401 when
+  the secret is unset). `schema` is an optional Zod validator; `transform(payload,
+  ctx)` returns an `IngestEvent | null` (`null` = accept-and-skip). Register
+  sources in `src/webhook-sources/index.ts` and pass them to
+  `createApp(client, { webhookSources })` in `src/index.ts`.
+- **Built-in integration presets** — the engine ships four ready-made inbound
+  sources (Clerk, Supabase, Stripe, Segment) served at
+  `POST /v1/webhooks/{clerk,supabase,stripe,segment}` with no code to write. Each
+  mounts only when its secret env var is set AND `ENABLED_WEBHOOK_PRESETS`
+  allows it (`"*"`/absent = auto, a csv of ids = exactly those, `"none"` = off).
+  Defining your own source with the SAME id overrides the preset (you win).
 - **`IngestEvent`** — the shape `transform` must return:
   `{ event, userId, userEmail, properties, idempotencyKey? }`. The route feeds it
   straight into `ingestEvent()`, so a webhook can enroll users into journeys.
@@ -66,3 +76,20 @@ Relative imports use the ESM `.js` extension.
   helpers.
 - To verify a webhook or task against a running instance (events landing,
   contacts upserted, journeys firing), see the **hogsend-cli** skill.
+- **Inbound vs outbound:** this skill is about *inbound* sources (HTTP → engine).
+  The engine also emits an *outbound* event stream (`contact.*`, `email.*`,
+  `journey.completed`, `bucket.*`). Two halves:
+  - **Subscriber endpoints** — manage the `webhook_endpoints` rows with
+    `hogsend webhooks …` (hogsend-cli skill) or `hs.webhooks.*` (hogsend-client-sdk
+    skill); verify signed deliveries with `verifyHogsendWebhook`. `create`/`update`
+    now take a `kind` + `config`: the default `kind="webhook"` is the byte-identical
+    signed `whsec_` POST (returns a one-time `secret`); a keyed destination
+    (`kind="posthog"|"segment"|"slack"|…`) carries its credentials in `config`
+    (e.g. `{ apiKey }`) and returns NO secret. Same durable retry/backoff/DLQ spine
+    either way — `kind` just selects the delivery-time transform.
+  - **Code-defined destinations** — author a delivery-time transform for a new
+    fan-out TARGET (a CRM, a warehouse, a custom shape) with `defineDestination()`
+    in `src/destinations/`. This is the symmetric twin of `defineWebhookSource`
+    on the OUTBOUND side. → the **hogsend-authoring-destinations** skill. NOTE:
+    outbound is no longer "just code in a journey" — for event fan-out, reach for
+    a destination, not a per-step integration call.

package/skills/hogsend-webhooks-and-workflows/references/webhook-source.md

@@ -18,19 +18,40 @@ You write the source files; the engine owns the route. Edit only
 | `meta.id` | `string` | The `:sourceId` segment in the URL. Keep it URL-safe. |
 | `meta.name` | `string` | Human label. |
 | `meta.description?` | `string` | Optional. |
-| `auth.header` | `string` | Request header carrying the shared secret. |
-| `auth.envKey` | `string` | Env var holding the expected secret value. |
-| `auth.type` | `"match"` | Only mode today: header value must equal the env value. |
+| `auth` | discriminated union on `type` | `"match"` or `"signature"` — see below. |
 | `schema?` | `z.ZodSchema<T>` | Optional Zod validator; on success `payload` is typed `T`. |
 | `transform(payload, ctx)` | `=> Promise<IngestEvent \| null>` | Map payload → event. Return `null` to accept-and-skip. |
 
-### Auth behaviour (important)
+### Auth: a discriminated union on `type`
 
-The route enforces auth **only when the env secret is set**. If
-`process.env[auth.envKey]` is empty/undefined the source is treated as **open**
-(no auth). When the secret is present, the request must send it either in
-`auth.header` or as `Authorization: Bearer <secret>`; otherwise the route returns
-`401`. Always set the env secret in any non-local environment.
+`auth` is a discriminated union — pick the variant that matches your provider:
+
+```ts
+// "match" — plain shared-secret equality (the PostHog scaffold source uses this)
+auth: { type: "match"; header: string; envKey: string }
+
+// "signature" — provider HMAC verification over the EXACT raw body bytes
+auth: {
+  type: "signature";
+  scheme: "svix" | "stripe" | "hmac-hex";
+  envKey: string;
+  header?: string;             // the signature header to read
+  fallbackMatchHeader?: string; // e.g. Supabase's plain x-supabase-webhook-secret
+  verify?(args): boolean | Promise<boolean>; // optional override of the scheme
+}
+```
+
+**Auth behaviour (important — the two variants differ on the unset-secret case):**
+
+- **`"match"`** enforces auth **only when the env secret is set**. If
+  `process.env[envKey]` is empty/undefined the source is treated as **open** (no
+  auth). When the secret is present, the request must send it in `header` or as
+  `Authorization: Bearer <secret>`, else the route returns `401`. Always set the
+  secret in any non-local environment.
+- **`"signature"`** **FAILS CLOSED** — when its secret is unset the route returns
+  `401` and never reaches `transform`. The route reads the EXACT raw body once
+  and verifies the HMAC (Svix / Stripe `t=…,v1=…` with 5-min tolerance / generic
+  hex HMAC) over those bytes. The raw bytes are also exposed as `ctx.rawBody`.
 
 ### Validation
 
@@ -52,9 +73,11 @@ interface IngestEvent {
 }
 ```
 
-`ctx` is `{ db, logger }` — a Drizzle `Database` and the engine logger — for
-lookups/diagnostics inside the transform. It does **not** carry `hatchet` or the
-registry; those are applied by the route when it calls `ingestEvent`.
+`ctx` is `{ db, logger, rawBody?, headers? }` — a Drizzle `Database`, the engine
+logger, and (populated by the route) the EXACT raw request body bytes and the
+lowercased request headers, for lookups/diagnostics or provider-specific raw
+access inside the transform. It does **not** carry `hatchet` or the registry;
+those are applied by the route when it calls `ingestEvent`.
 
 Notes that match the engine's behaviour:
 
@@ -160,6 +183,28 @@ const app = createApp(client, { webhookSources });
 
 That's it. Your source is now live at `POST /v1/webhooks/stripe`.
 
+## Built-in integration presets (no code)
+
+Before writing a source, check whether the engine already ships one. Four
+presets are built into `@hogsend/engine` and served with no consumer code:
+
+| Preset | Route | Scheme | Secret env var |
+|--------|-------|--------|----------------|
+| `clerk` | `POST /v1/webhooks/clerk` | `svix` | `CLERK_WEBHOOK_SECRET` |
+| `supabase` | `POST /v1/webhooks/supabase` | `svix` (+ plain `x-supabase-webhook-secret` fallback) | `SUPABASE_WEBHOOK_SECRET` |
+| `stripe` | `POST /v1/webhooks/stripe` | `stripe` | `STRIPE_WEBHOOK_SECRET` |
+| `segment` | `POST /v1/webhooks/segment` | `hmac-hex` | `SEGMENT_WEBHOOK_SECRET` |
+
+A preset mounts only when **both** its secret env var is set **and**
+`ENABLED_WEBHOOK_PRESETS` allows it: `"*"`/absent = auto (every preset whose
+secret is set), a comma-separated list of ids = exactly those (still requires the
+secret), `"none"` = all off. A preset with no secret is never mounted (signature
+sources fail closed). Defining your own `defineWebhookSource` with the SAME id
+**overrides** the preset — the consumer always wins. Each preset's exact
+provider-event → Hogsend-event mapping (and the `contactProperties` vs
+`eventProperties` split) lives in its source file under
+`packages/engine/src/webhook-sources/presets/`.
+
 ## Authoring a new source — checklist
 
 1. Create `src/webhook-sources/<id>.ts` exporting a `defineWebhookSource({...})`.

package/src/commands/index.ts

@@ -12,6 +12,7 @@ import { statsCommand } from "./stats.js";
 import { studioCommand } from "./studio.js";
 import type { Command } from "./types.js";
 import { upgradeCommand } from "./upgrade.js";
+import { webhooksCommand } from "./webhooks.js";
 
 /**
  * The command registry. The router (src/bin.ts) matches the leading argv token
@@ -29,6 +30,7 @@ export const commands: Command[] = [
   eventsCommand,
   emailsCommand,
   campaignsCommand,
+  webhooksCommand,
   studioCommand,
   setupCommand,
   skillsCommand,