@bettercms-ai/mcp 0.80.1 → 0.81.0

package/dist/skills/bettercms/references/integrations.md

@@ -1,6 +1,6 @@
 ---
 title: "Custom integrations"
-description: "Connect any vendor with a public API (Greenhouse, JobDiva, Shopify, HubSpot, Zoom, Apollo...) to a BetterCMS site: declare the integration, name its secrets and where they go, have an admin approve it, call the vendor through the server, and recipes per vendor."
+description: "Connect any vendor with a public API (Greenhouse, JobDiva, Shopify, HubSpot, Zoom, Apollo...) to a BetterCMS site: declare the integration, name its secrets and where they go, have an admin approve it, call the vendor through the server, sync lists both ways (two-way write-back), receive the vendor's webhooks, run your own server code, serve live reads to the site, reference vendor items, sign in with OAuth, reuse it as a template, and recipes per vendor."
 ---
 
 # Custom integrations
@@ -103,7 +103,10 @@ list is replaced whole) and run it with `run_integration_sync { integration, syn
   items: "/jobs", externalKey: "/id", slug: "/title",
   map: { title: "/title", location: "/location/name", department: { path: "/departments", transform: "join" },
          applyUrl: "/absolute_url", body: "/content" },
-  publish: false, schedule: "hourly" }
+  publish: false, schedule: "hourly",
+  // For webhooks (section 9): how to fetch ONE item by its id. {{item}} is the id.
+  item: { request: { method: "GET", url: "https://boards-api.greenhouse.io/v1/boards/acme/jobs/{{item}}" },
+          pointer: "", onMissing: "delete" } }
 ```
 
 - `map` keys are the collection's field keys; values are JSON Pointers into one item, with an
@@ -113,8 +116,9 @@ list is replaced whole) and run it with `run_integration_sync { integration, syn
   A sync that publishes is a publish: adding one, changing one, or running one needs a connection
   allowed to publish (`PUBLISH_NOT_GRANTED` otherwise). Ask the user to turn it on from the dashboard.
 - The run reports `created`, `updated`, `unchanged`, `failed`, `pages` and `items`. An item with no
-  external key, or a row a field rejects (a required field the vendor left empty), is `failed`;
-  adjust the map or the field and run again. A developer-locked field is never written by a sync.
+  external key, or a row a field rejects (a required field the vendor left empty), is `failed`, and
+  a done run says why in `error` (the first few, one per line); adjust the map or the field and run
+  again. Two items whose `slug` pointer gives one slug: the second takes its external key as a suffix. A developer-locked field is never written by a sync.
 - Entries remember where they came from (`provenance.source` = `integration:<slug>`), and
   `list_content_entries` with `externalKey` finds the one for a vendor id.
 
@@ -162,6 +166,8 @@ person's rights:
 | `syncs.set` | `{ syncs }` → syncs (schedules, mappings) |
 | `models.list` | → the collections a sync can write into |
 | `components.list` / `editor.insertSection` | the project's section components / `{ componentId }` adds one to the open page (dock only) |
+| `function.run` / `function.status` | `{ name, input? }` → the run / `{ runId }` → the run (output, logs, error) |
+| `reference.search` | `{ reference, query }` → up to 20 `{ id, title, image }` |
 
 ## 8. Recipes
 
@@ -177,3 +183,182 @@ person's rights:
 Form submissions to a CRM do not need an integration: attach the form to HubSpot or a webhook
 (forms-and-interactions.md). Client-side widgets (a Shopify cart) use the vendor's PUBLIC
 storefront token in site code; an Admin token never goes into the site.
+
+## 9. Webhooks: the vendor tells the site something changed
+
+Polling hourly is the fallback. Most vendors sign a webhook on every change; declare one per event
+and the site follows within seconds. **The payload is never trusted**: only the item's id is read
+from it, and the item is re-fetched from the vendor through the sync's `item` request (section 6).
+A 404 there means the vendor deleted it (`onMissing: "delete"` removes the entry).
+
+```
+update_integration { hooks: [{
+  key: "products",
+  verify: { scheme: "hmac", header: "X-Shopify-Hmac-Sha256", secret: "WEBHOOK_SECRET",
+            algorithm: "sha256", encoding: "base64" },
+  deliveryIdHeader: "X-Shopify-Webhook-Id",
+  action: { kind: "sync", sync: "products", mode: "item", itemKey: "/id" } }] }
+```
+
+- `secret` is a template NAME declared in `secrets` (an env var the user sets). Hooks are on the
+  approval surface: adding or changing one needs an admin again.
+- `get_integration` → `hookUrls.<key>` is the URL to paste at the vendor (Shopify: Settings →
+  Notifications → Webhooks, JSON). Recent deliveries are in `hookEvents`.
+- `mode: "full"` runs the whole sync instead (a vendor that sends no id). `action: { kind:
+  "function", function: "<name>" }` hands the verified payload to a function (section 10).
+- Signature schemes (`verify`):
+
+| Vendor | header | algorithm / encoding | prefix | message |
+|---|---|---|---|---|
+| Shopify | `X-Shopify-Hmac-Sha256` | sha256 / base64 | | `{body}` |
+| GitHub | `X-Hub-Signature-256` | sha256 / hex | `sha256=` | `{body}` |
+| Greenhouse | `Signature` | sha256 / hex | `sha256 ` | `{body}` |
+| Stripe | `Stripe-Signature` | sha256 / hex, `parse: "stripe"` | | `{timestamp}.{body}` |
+| Slack | `X-Slack-Signature` | sha256 / hex, `timestampHeader: "X-Slack-Request-Timestamp"` | `v0=` | `v0:{timestamp}:{body}` |
+| HubSpot v3 | `X-HubSpot-Signature-v3` | sha256 / base64, `timestampHeader: "X-HubSpot-Request-Timestamp"` | | `{method}{url}{body}{timestamp}` |
+| a shared token | `scheme: "token"`, the header it arrives in | | `prefix` if any | |
+
+  A signed timestamp outside `toleranceSeconds` (300 by default) is refused as a replay, and a
+  redelivery (same delivery id, or the same body) is answered without running twice. Rotating a
+  webhook secret in Settings keeps the old value valid for 24 hours by default.
+
+## 10. Functions: your own code, never your secrets
+
+For what a declarative sync cannot say (send a form to HubSpot, reshape a webhook payload, a nightly
+report), write `bettercms/functions/<name>.ts` in the site repo and declare it:
+
+```ts
+// bettercms/functions/to-hubspot.ts
+export default async (event: { formId: string; submission: { data: Record<string, string> } }, ctx) => {
+  const { email, name } = event.submission.data;
+  const res = await fetch("https://api.hubapi.com/crm/v3/objects/contacts", {
+    method: "POST", headers: { "content-type": "application/json" },
+    body: JSON.stringify({ properties: { email, firstname: name } }),
+  });
+  console.log("hubspot", res.status);
+  return { status: res.status };
+};
+```
+```
+update_integration { functions: [{ name: "to-hubspot", triggers: ["form:<formId>"], cpuMs: 1000 }] }
+```
+
+- It runs on Cloudflare Workers when the site's build includes it. Its `fetch()` reaches ONLY this
+  integration's approved hosts, JSON bodies only, and the server adds the `inject` secrets: the code
+  never sees a key, so never write one into it (there is no env to read).
+- Triggers: `manual` (run_integration_function, the panel's `function.run`), `form:<formId>`
+  (each accepted submission: `{ formId, submission: { id, data } }`), `schedule:hourly|daily`, and
+  any hook whose action names it (`{ hook, payload }`). `ctx` = `{ integration, function, trigger,
+  runId, settings }`.
+- Return `{ upsert: [{ sync, externalKey, data, slug? }] }` to write entries into a sync's
+  collection (drafts, like a sync). `run_integration_function { waitSeconds: 20 }` answers with the
+  output, the captured `console` lines and any error; runs are in `get_integration.functionRuns`.
+- If the server answers `functions_not_enabled`, functions are not available yet on this
+  BetterCMS server: use a sync or an endpoint instead.
+
+## 11. Site endpoints: live reads for visitors, no key in the page
+
+A page that must show something live (stock, a search, today's webinars) calls the integration's
+endpoint, never the vendor: the key stays on the server and only the fields you pick leave.
+
+```
+update_integration { endpoints: [{
+  key: "stock",
+  request: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/variants/{{param.variant}}.json" },
+  params: { variant: { pattern: "[0-9]{1,20}", required: true } },
+  pick: { available: "/variant/inventory_quantity", price: "/variant/price" },
+  cacheSeconds: 60 }] }
+```
+
+- The site calls `GET <endpointUrls.stock>?variant=123` (public, CORS open, no credentials) and gets
+  `{ data: { available, price } }`. With `items: "/products"` each list item is picked instead.
+- Every param must be declared with an anchored `pattern`; an unknown or non-matching param is a
+  400. Patterns with a repeated group inside a repeat (`(a+)+`) are refused: they can hang.
+- Answers are cached for `cacheSeconds` per params; visitors are rate-limited per IP.
+- Endpoints are on the approval surface (what anyone on the internet can read).
+
+## 12. Live references: point at a vendor item without copying it
+
+```
+update_integration { references: [{
+  key: "product", label: "Shopify product",
+  search: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products.json", query: { title: "{{query}}", limit: "20" } },
+  lookup: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{id}}.json" },
+  items: "/products", item: "/product", id: "/id", title: "/title", image: "/image/src",
+  pick: { price: "/variants/0/price" } }] }
+```
+
+- The field that holds one: type `json` with `ui.reference: { integration: "<integration slug>",
+  key: "<reference key>" }`; editors get a search picker. Its value is `{ integration, reference, id, title, image? }` (title
+  and image are a snapshot for editors).
+- `search_integration_reference` finds ids for an agent. The site resolves the live item at
+  `<refs URL>/<id>` (`GET /api/v1/public/integrations/<projectId>/<slug>/refs/<key>/<id>`):
+  `{ id, title, image?, ...pick }`, cached `cacheSeconds` (300 by default).
+
+## 13. Sign in with OAuth (the customer's own OAuth app)
+
+For vendors that only offer OAuth (Google, HubSpot public apps, Zoom user-level): the user creates an
+OAuth app at the vendor, sets its client id and secret as env vars, and declares:
+
+```
+update_integration { secrets: { CLIENT_ID: "GOOGLE_CLIENT_ID", CLIENT_SECRET: "GOOGLE_CLIENT_SECRET" },
+  allowedHosts: ["oauth2.googleapis.com", "www.googleapis.com"],
+  oauth: { authorizeUrl: "https://accounts.google.com/o/oauth2/v2/auth",
+           tokenUrl: "https://oauth2.googleapis.com/token",
+           scopes: ["https://www.googleapis.com/auth/calendar.readonly"],
+           clientId: "CLIENT_ID", clientSecret: "CLIENT_SECRET",
+           extraAuthorizeParams: { access_type: "offline", prompt: "consent" } },
+  inject: { headers: { authorization: "Bearer {{token}}" } } }
+```
+
+- After approval, `start_integration_signin` gives a `url` for the USER to open (and the
+  `redirectUri` they register in their OAuth app first). They sign in at the vendor; the tokens are
+  stored encrypted and refreshed by the server. `{{token}}` in `inject` is the access token.
+- `oauth` and `auth` (a token exchange) do not mix. The token URL's host must be an allowed host.
+
+## 14. Templates, the call log, alerts
+
+- `save_integration_template` stores an integration's shape for the workspace (no value, no
+  settings; the collections its syncs write into are copied). `install_integration_template` puts
+  it on another project: missing collections are created, the integration waits for that project's
+  admin, and `secretsNeeded` lists the env vars to set there.
+- `get_integration.calls` is the call log: every vendor call (source, host, path, status, time),
+  never a query string or body. When runs fail `alertAfter` times in a row (3 by default; 0 = never)
+  the workspace's admins are notified once; the next success clears it.
+
+## 15. Two-way sync: BetterCMS edits go back to the vendor
+
+A sync pulls (section 6) and follows the vendor's webhooks (section 9). Add `writeBack` and edits
+made in BetterCMS (dashboard, Visual editor, API, you) go back to the vendor too:
+
+```
+{ key: "products", ..., map: { title: "/title", description: "/body_html", price: "/variants/0/price" },
+  item: { request: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json" }, pointer: "/product" },
+  writeBack: {
+    fields: ["description"],                      // the fields BetterCMS owns
+    on: "save",                                   // or "publish": only what goes live is pushed
+    update: { method: "PUT", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json",
+              json: { product: { body_html: "{{field.description}}" } } },
+    create: { method: "POST", url: "https://acme.myshopify.com/admin/api/2024-10/products.json",
+              json: { product: { title: "{{field.title}}", body_html: "{{field.description}}" } }, idPointer: "/product/id" },
+    delete: { method: "DELETE", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json" } } }
+```
+
+- `{{item}}` is the vendor id (the entry's externalKey). `{{field.KEY}}` is the entry's value; a
+  JSON string that is exactly one placeholder keeps the value's type (a number stays a number).
+  `update` may use only `fields`; `create` may use any field of the new entry.
+- Needs `allowPost: true`, and `writeBack` is on the approval surface (it writes, and may delete,
+  at the vendor). `delete` is never done unless declared.
+- **Who wins.** Vendor-owned fields (`map` only) always follow the vendor. For `fields`, a BetterCMS
+  edit is pushed within about a minute; until it has reached the vendor, a pull or webhook does not
+  overwrite it. After that, the newest change from either side wins.
+- **No loops.** Both sides' agreed values are remembered per entry; our own push coming back
+  through a webhook, or a pull's own write, is recognised and never pushed again.
+- **New and deleted entries.** An entry made in BetterCMS is created at the vendor (with `create`)
+  and keeps the vendor's id as its externalKey, so the next pull updates it instead of duplicating.
+  Without `create` it stays BetterCMS-only. Deleting an entry deletes the vendor item only with `delete`.
+- **Failures** retry with backoff (1, 2, 4, 8, 16 minutes), then count toward `alertAfter`. Pushes
+  appear in the call log with source `push`.
+- **Known windows** (seconds to a minute, rare): an edit saved while a pull is writing that same entry,
+  or within a minute of changing `writeBack.fields`, can be replaced by the vendor's value. Change the
+  field list when editors are idle, and re-save an edit if a pull overwrote it.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bettercms-ai/mcp",
-  "version": "0.80.1",
+  "version": "0.81.0",
   "type": "module",
   "description": "BetterCMS MCP server — schema, content, forms + components authoring tools over the Management API, authed via OAuth device-code.",
   "bin": {