@bettercms-ai/mcp 0.83.0 → 0.83.2

package/dist/skills/bettercms/references/integrations.md

@@ -1,364 +1,364 @@
----
-title: "Custom integrations"
-description: "Connect any vendor with a public API (Greenhouse, JobDiva, Shopify, HubSpot, Zoom, Apollo...) to a BetterCMS site: declare the integration, name its secrets and where they go, have an admin approve it, call the vendor through the server, sync lists both ways (two-way write-back), receive the vendor's webhooks, run your own server code, serve live reads to the site, reference vendor items, sign in with OAuth, reuse it as a template, and recipes per vendor."
----
-
-# Custom integrations
-
-**Outcome.** The site shows the user's own data from another tool (jobs from their ATS, products
-from their store, webinars from their video platform) through an integration you declared, that
-an admin approved, and that never exposes a secret. There is no native per-vendor integration:
-any vendor with a public HTTPS API and a key works the same way.
-
-Everything here can also be done by a person in the dashboard (Site settings → Integrations), with the
-same records. What you declare, they see and can edit.
-
-## 1. The order of work
-
-1. **Model where the data lands.** A collection per vendor list (Jobs, Products), with fields for
-   what the site shows (content-modeling.md). Nothing vendor-specific in the names.
-2. **Name the secrets, never handle them.** Pick env var names (`GREENHOUSE_API_KEY`,
-   `SHOPIFY_ADMIN_TOKEN`). The user sets the VALUES under Settings → Environment variables
-   (production) or with `bettercms env push`. `list_env_vars` shows which names exist. Never ask
-   the user to paste a key into the chat, and never put one in a file you write.
-3. **Declare the integration** with `create_integration` (section 2).
-4. **Have an admin approve it** (section 3).
-5. **Discover the shape** with `test_integration_fetch` (section 4).
-6. **Sync the list into the collection** (section 6) and check the entries.
-7. **Show it on the site**: register components for the data (components.md); they appear in the
-   section library under "Add section".
-
-## 2. Declaring one
-
-```
-create_integration {
-  slug: "shopify", name: "Shopify products", vendor: "Shopify",
-  allowedHosts: ["acme.myshopify.com"],
-  secrets: { TOKEN: "SHOPIFY_ADMIN_TOKEN" },                          // template name -> env var KEY
-  inject: { headers: { "X-Shopify-Access-Token": "{{secret.TOKEN}}" } },  // where it goes
-  settings: { shop: "acme" }                                           // non-secret, the panel reads it
-}
-```
-
-- `allowedHosts` are exact hostnames, called over https on the default port only. A redirect must
-  stay on the same origin. List the API host and, for a token exchange, the auth host.
-- `secrets` maps a template name to an env var KEY. `inject` is the ONLY place a secret goes: the
-  headers and query parameters added to every request to the integration's hosts. A request you
-  send (a sync's, `test_integration_fetch`'s, a panel's) carries no `{{...}}` at all.
-- Vendor calls are GET only. `allowPost: true` lets other requests through: vendor writes, and
-  search APIs that take POST (Apollo). Ask for it only when the integration needs it.
-- The answer has `secretsPresent` per name: false means the user has not set that env var yet.
-
-## 3. Approval
-
-Until an admin approves the integration, every vendor call and sync is refused (409
-`integration_not_approved`). You cannot approve it: no tool and no header does. The create and
-update answers carry `approval.changes` (in plain words: which hosts it will call, which env vars
-it will read, where it sends them) and `approval.approveUrl`. Give the user both, and say why the
-integration needs each host and secret. They (or an admin) press Approve on that page.
-
-Changing `allowedHosts`, `secrets`, `inject`, `allowPost` or `auth` later withdraws the approval until an admin
-approves again. Name, settings, panel and syncs never need approval.
-
-## 4. Calling the vendor
-
-`test_integration_fetch { integration, url, method?, headers?, query?, json? }` runs one request
-through the server and returns `{ status, headers, body }`. The `inject` secrets are added
-server-side and every secret value is replaced by `[secret]` in what comes back. Bodies over 1 MB
-are truncated. Use it to read the response shape before you map anything: list paths and ids are
-JSON Pointers (`/jobs`, `/id`, `/data/0/attributes/name`).
-
-## 5. Token exchanges
-
-Some vendors trade credentials for a short-lived token (Zoom Server-to-Server OAuth, JobDiva).
-Declare the exchange once and inject `{{token}}`:
-
-```
-auth: {
-  request: { method: "POST", url: "https://zoom.us/oauth/token",
-             query: { grant_type: "account_credentials", account_id: "{{secret.ACCOUNT_ID}}" },
-             headers: { authorization: "Basic {{secret.BASIC}}" } },
-  tokenPath: "/access_token", expiresInPath: "/expires_in"
-},
-inject: { headers: { authorization: "Bearer {{token}}" } }
-```
-
-The token is cached until shortly before it expires, and the token endpoint itself can never be
-called with `test_integration_fetch`. A vendor that wants `base64(id:secret)` in a Basic header
-gets it as ONE env var holding the encoded string (the user can make it with
-`printf 'id:secret' | base64`). `tokenPath: ""` reads a plain-text response as the token.
-
-## 6. Syncing a list into a collection
-
-A sync reads every page of a vendor list and writes each item as a DRAFT entry of one collection,
-matched on its `externalKey`: a re-run updates what changed and leaves the rest `unchanged`, so a
-sync is safe to run any number of times. Add it with `update_integration { syncs: [...] }` (the
-list is replaced whole) and run it with `run_integration_sync { integration, sync, waitSeconds: 20 }`.
-
-```
-{ key: "jobs", label: "Open roles", contentModelId: "<Jobs collection id>",
-  request: { method: "GET", url: "https://boards-api.greenhouse.io/v1/boards/acme/jobs",
-             query: { content: "true" } },
-  pagination: { style: "none" },          // page: param=1,2,3 | cursor: param=<cursorPath value> | link: rel="next"
-  items: "/jobs", externalKey: "/id", slug: "/title",
-  map: { title: "/title", location: "/location/name", department: { path: "/departments", transform: "join" },
-         applyUrl: "/absolute_url", body: "/content" },
-  publish: false, schedule: "hourly",
-  // For webhooks (section 9): how to fetch ONE item by its id. {{item}} is the id.
-  item: { request: { method: "GET", url: "https://boards-api.greenhouse.io/v1/boards/acme/jobs/{{item}}" },
-          pointer: "", onMissing: "delete" } }
-```
-
-- `map` keys are the collection's field keys; values are JSON Pointers into one item, with an
-  optional `transform` (`string`, `number`, `boolean`, `join` for a list of names).
-- Run it once with `publish: false`, read a few entries, then turn on `publish` and a `schedule`
-  (`hourly` or `daily`) if the user wants the site to follow the vendor on its own.
-  A sync that publishes is a publish: adding one, changing one, or running one needs a connection
-  allowed to publish (`PUBLISH_NOT_GRANTED` otherwise). Ask the user to turn it on from the dashboard.
-- The run reports `created`, `updated`, `unchanged`, `failed`, `pages` and `items`. An item with no
-  external key, or a row a field rejects (a required field the vendor left empty), is `failed`, and
-  a done run says why in `error` (the first few, one per line); adjust the map or the field and run
-  again. Two items whose `slug` pointer gives one slug: the second takes its external key as a suffix. A developer-locked field is never written by a sync.
-- Entries remember where they came from (`provenance.source` = `integration:<slug>`), and
-  `list_content_entries` with `externalKey` finds the one for a vendor id.
-
-## 7. A custom panel
-
-The stock integration page (Site settings → Integrations → the integration) already shows the
-approval, secrets, syncs with Import now and Auto-sync, a request tester and the runs. Build a
-panel only when the user wants their own screen for it (a store picker, a mapping matrix, an
-"Add component" drawer), the way Storesynk draws one for Shopify in Webflow.
-
-1. Install `@bettercms-ai/dock-input` (0.2.0 or later) and React in the site repo if they are not there.
-2. Write `bettercms/panels/<name>.tsx` (lowercase, hyphens), default-exporting a React component.
-   It is built with the site, served from the project's own isolated origin and drawn in a
-   sandboxed frame on the integration's page and in the Visual editor's dock.
-3. Set `update_integration { integration, panel: "<name>" }`. A panel never needs approval.
-
-```tsx
-import type { PanelProps } from "@bettercms-ai/dock-input";
-import { Button, Switch } from "@bettercms-ai/kit"; // when the site has the kit
-
-export default function ShopifyPanel({ integration, call, place }: PanelProps) {
-  const sync = integration.syncs[0] as { key: string; schedule?: string | null };
-  return (
-    <div>
-      <Switch checked={!!sync?.schedule} onCheckedChange={(on) =>
-        call("syncs.set", { syncs: integration.syncs.map((s) => (s === sync ? { ...s, schedule: on ? "hourly" : null } : s)) })} />
-      <Button onClick={() => call("sync.run", { sync: sync.key })}>Import Shopify data</Button>
-      {place === "dock" ? <Button onClick={() => call("editor.insertSection", { componentId: "<buy-button component id>" })}>Add buy button</Button> : null}
-    </div>
-  );
-}
-```
-
-The panel can make no request of its own (its CSP is `connect-src 'none'`), and it never sees a token
-or a secret; navigating its frame away ends it. A `syncs.set` that would make a sync publish, run on a
-schedule, write into another collection, or add or remove a sync asks the person first. Everything goes through `call(method, params)`, run by the dashboard with the signed-in
-person's rights:
-
-| Method | Params → result |
-|---|---|
-| `vendor.fetch` | `{ method?, url, query?, headers?, json? }` → `{ status, headers, body }`, as test_integration_fetch |
-| `sync.run` / `sync.status` | `{ sync }` → the run / `{ runId }` → the run |
-| `runs.list` | → the last runs |
-| `settings.set` | `{ settings }` → settings (non-secret, ≤ 16 KB; store ids, toggles) |
-| `syncs.set` | `{ syncs }` → syncs (schedules, mappings) |
-| `models.list` | → the collections a sync can write into |
-| `components.list` / `editor.insertSection` | the project's section components / `{ componentId }` adds one to the open page (dock only) |
-| `function.run` / `function.status` | `{ name, input? }` → the run / `{ runId }` → the run (output, logs, error) |
-| `reference.search` | `{ reference, query }` → up to 20 `{ id, title, image }` |
-
-## 8. Recipes
-
-| Vendor | Hosts | Secrets and inject | List request | Items / key |
-|---|---|---|---|---|
-| Greenhouse job board | `boards-api.greenhouse.io` | none (public board) | `GET /v1/boards/{board}/jobs?content=true` | `/jobs` / `/id` |
-| Shopify Admin | `{shop}.myshopify.com` | `TOKEN` → header `X-Shopify-Access-Token` | `GET /admin/api/2024-10/products.json?limit=250` | `/products` / `/id`, Link-header pages |
-| HubSpot CRM | `api.hubapi.com` | `TOKEN` (private app) → header `authorization: Bearer {{secret.TOKEN}}` | `GET /crm/v3/objects/contacts?limit=100` | `/results` / `/id`, cursor `/paging/next/after` |
-| Zoom webinars | `zoom.us`, `api.zoom.us` | `ACCOUNT_ID`, `BASIC` + token exchange → `authorization: Bearer {{token}}` | `GET https://api.zoom.us/v2/users/me/webinars` | `/webinars` / `/id`, cursor `/next_page_token` |
-| JobDiva | `api.jobdiva.com` | `CLIENT_ID`, `USERNAME`, `PASSWORD` + token exchange → `authorization: Bearer {{token}}` | the job search endpoint of their API | per their docs |
-| Apollo | `api.apollo.io` | `KEY` → header `x-api-key`; `allowPost: true` | `POST /api/v1/mixed_people/search` | `/people` / `/id` |
-
-Form submissions to a CRM do not need an integration: attach the form to HubSpot or a webhook
-(forms-and-interactions.md). Client-side widgets (a Shopify cart) use the vendor's PUBLIC
-storefront token in site code; an Admin token never goes into the site.
-
-## 9. Webhooks: the vendor tells the site something changed
-
-Polling hourly is the fallback. Most vendors sign a webhook on every change; declare one per event
-and the site follows within seconds. **The payload is never trusted**: only the item's id is read
-from it, and the item is re-fetched from the vendor through the sync's `item` request (section 6).
-A 404 there means the vendor deleted it (`onMissing: "delete"` removes the entry).
-
-```
-update_integration { hooks: [{
-  key: "products",
-  verify: { scheme: "hmac", header: "X-Shopify-Hmac-Sha256", secret: "WEBHOOK_SECRET",
-            algorithm: "sha256", encoding: "base64" },
-  deliveryIdHeader: "X-Shopify-Webhook-Id",
-  action: { kind: "sync", sync: "products", mode: "item", itemKey: "/id" } }] }
-```
-
-- `secret` is a template NAME declared in `secrets` (an env var the user sets). Hooks are on the
-  approval surface: adding or changing one needs an admin again.
-- `get_integration` → `hookUrls.<key>` is the URL to paste at the vendor (Shopify: Settings →
-  Notifications → Webhooks, JSON). Recent deliveries are in `hookEvents`.
-- `mode: "full"` runs the whole sync instead (a vendor that sends no id). `action: { kind:
-  "function", function: "<name>" }` hands the verified payload to a function (section 10).
-- Signature schemes (`verify`):
-
-| Vendor | header | algorithm / encoding | prefix | message |
-|---|---|---|---|---|
-| Shopify | `X-Shopify-Hmac-Sha256` | sha256 / base64 | | `{body}` |
-| GitHub | `X-Hub-Signature-256` | sha256 / hex | `sha256=` | `{body}` |
-| Greenhouse | `Signature` | sha256 / hex | `sha256 ` | `{body}` |
-| Stripe | `Stripe-Signature` | sha256 / hex, `parse: "stripe"` | | `{timestamp}.{body}` |
-| Slack | `X-Slack-Signature` | sha256 / hex, `timestampHeader: "X-Slack-Request-Timestamp"` | `v0=` | `v0:{timestamp}:{body}` |
-| HubSpot v3 | `X-HubSpot-Signature-v3` | sha256 / base64, `timestampHeader: "X-HubSpot-Request-Timestamp"` | | `{method}{url}{body}{timestamp}` |
-| a shared token | `scheme: "token"`, the header it arrives in | | `prefix` if any | |
-
-  A signed timestamp outside `toleranceSeconds` (300 by default) is refused as a replay, and a
-  redelivery (same delivery id, or the same body) is answered without running twice. Rotating a
-  webhook secret in Settings keeps the old value valid for 24 hours by default.
-
-## 10. Functions: your own code, never your secrets
-
-For what a declarative sync cannot say (send a form to HubSpot, reshape a webhook payload, a nightly
-report), write `bettercms/functions/<name>.ts` in the site repo and declare it:
-
-```ts
-// bettercms/functions/to-hubspot.ts
-export default async (event: { formId: string; submission: { data: Record<string, string> } }, ctx) => {
-  const { email, name } = event.submission.data;
-  const res = await fetch("https://api.hubapi.com/crm/v3/objects/contacts", {
-    method: "POST", headers: { "content-type": "application/json" },
-    body: JSON.stringify({ properties: { email, firstname: name } }),
-  });
-  console.log("hubspot", res.status);
-  return { status: res.status };
-};
-```
-```
-update_integration { functions: [{ name: "to-hubspot", triggers: ["form:<formId>"], cpuMs: 1000 }] }
-```
-
-- It runs on Cloudflare Workers when the site's build includes it. Its `fetch()` reaches ONLY this
-  integration's approved hosts, JSON bodies only, and the server adds the `inject` secrets: the code
-  never sees a key, so never write one into it (there is no env to read).
-- Triggers: `manual` (run_integration_function, the panel's `function.run`), `form:<formId>`
-  (each accepted submission: `{ formId, submission: { id, data } }`), `schedule:hourly|daily`, and
-  any hook whose action names it (`{ hook, payload }`). `ctx` = `{ integration, function, trigger,
-  runId, settings }`.
-- Return `{ upsert: [{ sync, externalKey, data, slug? }] }` to write entries into a sync's
-  collection (drafts, like a sync). `run_integration_function { waitSeconds: 20 }` answers with the
-  output, the captured `console` lines and any error; runs are in `get_integration.functionRuns`.
-- If the server answers `functions_not_enabled`, functions are not available yet on this
-  BetterCMS server: use a sync or an endpoint instead.
-
-## 11. Site endpoints: live reads for visitors, no key in the page
-
-A page that must show something live (stock, a search, today's webinars) calls the integration's
-endpoint, never the vendor: the key stays on the server and only the fields you pick leave.
-
-```
-update_integration { endpoints: [{
-  key: "stock",
-  request: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/variants/{{param.variant}}.json" },
-  params: { variant: { pattern: "[0-9]{1,20}", required: true } },
-  pick: { available: "/variant/inventory_quantity", price: "/variant/price" },
-  cacheSeconds: 60 }] }
-```
-
-- The site calls `GET <endpointUrls.stock>?variant=123` (public, CORS open, no credentials) and gets
-  `{ data: { available, price } }`. With `items: "/products"` each list item is picked instead.
-- Every param must be declared with an anchored `pattern`; an unknown or non-matching param is a
-  400. Patterns with a repeated group inside a repeat (`(a+)+`) are refused: they can hang.
-- Answers are cached for `cacheSeconds` per params; visitors are rate-limited per IP.
-- Endpoints are on the approval surface (what anyone on the internet can read).
-
-## 12. Live references: point at a vendor item without copying it
-
-```
-update_integration { references: [{
-  key: "product", label: "Shopify product",
-  search: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products.json", query: { title: "{{query}}", limit: "20" } },
-  lookup: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{id}}.json" },
-  items: "/products", item: "/product", id: "/id", title: "/title", image: "/image/src",
-  pick: { price: "/variants/0/price" } }] }
-```
-
-- The field that holds one: type `json` with `ui.reference: { integration: "<integration slug>",
-  key: "<reference key>" }`; editors get a search picker. Its value is `{ integration, reference, id, title, image? }` (title
-  and image are a snapshot for editors).
-- `search_integration_reference` finds ids for an agent. The site resolves the live item at
-  `<refs URL>/<id>` (`GET /api/v1/public/integrations/<projectId>/<slug>/refs/<key>/<id>`):
-  `{ id, title, image?, ...pick }`, cached `cacheSeconds` (300 by default).
-
-## 13. Sign in with OAuth (the customer's own OAuth app)
-
-For vendors that only offer OAuth (Google, HubSpot public apps, Zoom user-level): the user creates an
-OAuth app at the vendor, sets its client id and secret as env vars, and declares:
-
-```
-update_integration { secrets: { CLIENT_ID: "GOOGLE_CLIENT_ID", CLIENT_SECRET: "GOOGLE_CLIENT_SECRET" },
-  allowedHosts: ["oauth2.googleapis.com", "www.googleapis.com"],
-  oauth: { authorizeUrl: "https://accounts.google.com/o/oauth2/v2/auth",
-           tokenUrl: "https://oauth2.googleapis.com/token",
-           scopes: ["https://www.googleapis.com/auth/calendar.readonly"],
-           clientId: "CLIENT_ID", clientSecret: "CLIENT_SECRET",
-           extraAuthorizeParams: { access_type: "offline", prompt: "consent" } },
-  inject: { headers: { authorization: "Bearer {{token}}" } } }
-```
-
-- After approval, `start_integration_signin` gives a `url` for the USER to open (and the
-  `redirectUri` they register in their OAuth app first). They sign in at the vendor; the tokens are
-  stored encrypted and refreshed by the server. `{{token}}` in `inject` is the access token.
-- `oauth` and `auth` (a token exchange) do not mix. The token URL's host must be an allowed host.
-
-## 14. Templates, the call log, alerts
-
-- `save_integration_template` stores an integration's shape for the workspace (no value, no
-  settings; the collections its syncs write into are copied). `install_integration_template` puts
-  it on another project: missing collections are created, the integration waits for that project's
-  admin, and `secretsNeeded` lists the env vars to set there.
-- `get_integration.calls` is the call log: every vendor call (source, host, path, status, time),
-  never a query string or body. When runs fail `alertAfter` times in a row (3 by default; 0 = never)
-  the workspace's admins are notified once; the next success clears it.
-
-## 15. Two-way sync: BetterCMS edits go back to the vendor
-
-A sync pulls (section 6) and follows the vendor's webhooks (section 9). Add `writeBack` and edits
-made in BetterCMS (dashboard, Visual editor, API, you) go back to the vendor too:
-
-```
-{ key: "products", ..., map: { title: "/title", description: "/body_html", price: "/variants/0/price" },
-  item: { request: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json" }, pointer: "/product" },
-  writeBack: {
-    fields: ["description"],                      // the fields BetterCMS owns
-    on: "save",                                   // or "publish": only what goes live is pushed
-    update: { method: "PUT", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json",
-              json: { product: { body_html: "{{field.description}}" } } },
-    create: { method: "POST", url: "https://acme.myshopify.com/admin/api/2024-10/products.json",
-              json: { product: { title: "{{field.title}}", body_html: "{{field.description}}" } }, idPointer: "/product/id" },
-    delete: { method: "DELETE", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json" } } }
-```
-
-- `{{item}}` is the vendor id (the entry's externalKey). `{{field.KEY}}` is the entry's value; a
-  JSON string that is exactly one placeholder keeps the value's type (a number stays a number).
-  `update` may use only `fields`; `create` may use any field of the new entry.
-- Needs `allowPost: true`, and `writeBack` is on the approval surface (it writes, and may delete,
-  at the vendor). `delete` is never done unless declared.
-- **Who wins.** Vendor-owned fields (`map` only) always follow the vendor. For `fields`, a BetterCMS
-  edit is pushed within about a minute; until it has reached the vendor, a pull or webhook does not
-  overwrite it. After that, the newest change from either side wins.
-- **No loops.** Both sides' agreed values are remembered per entry; our own push coming back
-  through a webhook, or a pull's own write, is recognised and never pushed again.
-- **New and deleted entries.** An entry made in BetterCMS is created at the vendor (with `create`)
-  and keeps the vendor's id as its externalKey, so the next pull updates it instead of duplicating.
-  Without `create` it stays BetterCMS-only. Deleting an entry deletes the vendor item only with `delete`.
-- **Failures** retry with backoff (1, 2, 4, 8, 16 minutes), then count toward `alertAfter`. Pushes
-  appear in the call log with source `push`.
-- **Known windows** (seconds to a minute, rare): an edit saved while a pull is writing that same entry,
-  or within a minute of changing `writeBack.fields`, can be replaced by the vendor's value. Change the
-  field list when editors are idle, and re-save an edit if a pull overwrote it.
+---
+title: "Custom integrations"
+description: "Connect any vendor with a public API (Greenhouse, JobDiva, Shopify, HubSpot, Zoom, Apollo...) to a BetterCMS site: declare the integration, name its secrets and where they go, have an admin approve it, call the vendor through the server, sync lists both ways (two-way write-back), receive the vendor's webhooks, run your own server code, serve live reads to the site, reference vendor items, sign in with OAuth, reuse it as a template, and recipes per vendor."
+---
+
+# Custom integrations
+
+**Outcome.** The site shows the user's own data from another tool (jobs from their ATS, products
+from their store, webinars from their video platform) through an integration you declared, that
+an admin approved, and that never exposes a secret. There is no native per-vendor integration:
+any vendor with a public HTTPS API and a key works the same way.
+
+Everything here can also be done by a person in the dashboard (Site settings → Integrations), with the
+same records. What you declare, they see and can edit.
+
+## 1. The order of work
+
+1. **Model where the data lands.** A collection per vendor list (Jobs, Products), with fields for
+   what the site shows (content-modeling.md). Nothing vendor-specific in the names.
+2. **Name the secrets, never handle them.** Pick env var names (`GREENHOUSE_API_KEY`,
+   `SHOPIFY_ADMIN_TOKEN`). The user sets the VALUES under Settings → Environment variables
+   (production) or with `bettercms env push`. `list_env_vars` shows which names exist. Never ask
+   the user to paste a key into the chat, and never put one in a file you write.
+3. **Declare the integration** with `create_integration` (section 2).
+4. **Have an admin approve it** (section 3).
+5. **Discover the shape** with `test_integration_fetch` (section 4).
+6. **Sync the list into the collection** (section 6) and check the entries.
+7. **Show it on the site**: register components for the data (components.md); they appear in the
+   section library under "Add section".
+
+## 2. Declaring one
+
+```
+create_integration {
+  slug: "shopify", name: "Shopify products", vendor: "Shopify",
+  allowedHosts: ["acme.myshopify.com"],
+  secrets: { TOKEN: "SHOPIFY_ADMIN_TOKEN" },                          // template name -> env var KEY
+  inject: { headers: { "X-Shopify-Access-Token": "{{secret.TOKEN}}" } },  // where it goes
+  settings: { shop: "acme" }                                           // non-secret, the panel reads it
+}
+```
+
+- `allowedHosts` are exact hostnames, called over https on the default port only. A redirect must
+  stay on the same origin. List the API host and, for a token exchange, the auth host.
+- `secrets` maps a template name to an env var KEY. `inject` is the ONLY place a secret goes: the
+  headers and query parameters added to every request to the integration's hosts. A request you
+  send (a sync's, `test_integration_fetch`'s, a panel's) carries no `{{...}}` at all.
+- Vendor calls are GET only. `allowPost: true` lets other requests through: vendor writes, and
+  search APIs that take POST (Apollo). Ask for it only when the integration needs it.
+- The answer has `secretsPresent` per name: false means the user has not set that env var yet.
+
+## 3. Approval
+
+Until an admin approves the integration, every vendor call and sync is refused (409
+`integration_not_approved`). You cannot approve it: no tool and no header does. The create and
+update answers carry `approval.changes` (in plain words: which hosts it will call, which env vars
+it will read, where it sends them) and `approval.approveUrl`. Give the user both, and say why the
+integration needs each host and secret. They (or an admin) press Approve on that page.
+
+Changing `allowedHosts`, `secrets`, `inject`, `allowPost` or `auth` later withdraws the approval until an admin
+approves again. Name, settings, panel and syncs never need approval.
+
+## 4. Calling the vendor
+
+`test_integration_fetch { integration, url, method?, headers?, query?, json? }` runs one request
+through the server and returns `{ status, headers, body }`. The `inject` secrets are added
+server-side and every secret value is replaced by `[secret]` in what comes back. Bodies over 1 MB
+are truncated. Use it to read the response shape before you map anything: list paths and ids are
+JSON Pointers (`/jobs`, `/id`, `/data/0/attributes/name`).
+
+## 5. Token exchanges
+
+Some vendors trade credentials for a short-lived token (Zoom Server-to-Server OAuth, JobDiva).
+Declare the exchange once and inject `{{token}}`:
+
+```
+auth: {
+  request: { method: "POST", url: "https://zoom.us/oauth/token",
+             query: { grant_type: "account_credentials", account_id: "{{secret.ACCOUNT_ID}}" },
+             headers: { authorization: "Basic {{secret.BASIC}}" } },
+  tokenPath: "/access_token", expiresInPath: "/expires_in"
+},
+inject: { headers: { authorization: "Bearer {{token}}" } }
+```
+
+The token is cached until shortly before it expires, and the token endpoint itself can never be
+called with `test_integration_fetch`. A vendor that wants `base64(id:secret)` in a Basic header
+gets it as ONE env var holding the encoded string (the user can make it with
+`printf 'id:secret' | base64`). `tokenPath: ""` reads a plain-text response as the token.
+
+## 6. Syncing a list into a collection
+
+A sync reads every page of a vendor list and writes each item as a DRAFT entry of one collection,
+matched on its `externalKey`: a re-run updates what changed and leaves the rest `unchanged`, so a
+sync is safe to run any number of times. Add it with `update_integration { syncs: [...] }` (the
+list is replaced whole) and run it with `run_integration_sync { integration, sync, waitSeconds: 20 }`.
+
+```
+{ key: "jobs", label: "Open roles", contentModelId: "<Jobs collection id>",
+  request: { method: "GET", url: "https://boards-api.greenhouse.io/v1/boards/acme/jobs",
+             query: { content: "true" } },
+  pagination: { style: "none" },          // page: param=1,2,3 | cursor: param=<cursorPath value> | link: rel="next"
+  items: "/jobs", externalKey: "/id", slug: "/title",
+  map: { title: "/title", location: "/location/name", department: { path: "/departments", transform: "join" },
+         applyUrl: "/absolute_url", body: "/content" },
+  publish: false, schedule: "hourly",
+  // For webhooks (section 9): how to fetch ONE item by its id. {{item}} is the id.
+  item: { request: { method: "GET", url: "https://boards-api.greenhouse.io/v1/boards/acme/jobs/{{item}}" },
+          pointer: "", onMissing: "delete" } }
+```
+
+- `map` keys are the collection's field keys; values are JSON Pointers into one item, with an
+  optional `transform` (`string`, `number`, `boolean`, `join` for a list of names).
+- Run it once with `publish: false`, read a few entries, then turn on `publish` and a `schedule`
+  (`hourly` or `daily`) if the user wants the site to follow the vendor on its own.
+  A sync that publishes is a publish: adding one, changing one, or running one needs a connection
+  allowed to publish (`PUBLISH_NOT_GRANTED` otherwise). Ask the user to turn it on from the dashboard.
+- The run reports `created`, `updated`, `unchanged`, `failed`, `pages` and `items`. An item with no
+  external key, or a row a field rejects (a required field the vendor left empty), is `failed`, and
+  a done run says why in `error` (the first few, one per line); adjust the map or the field and run
+  again. Two items whose `slug` pointer gives one slug: the second takes its external key as a suffix. A developer-locked field is never written by a sync.
+- Entries remember where they came from (`provenance.source` = `integration:<slug>`), and
+  `list_content_entries` with `externalKey` finds the one for a vendor id.
+
+## 7. A custom panel
+
+The stock integration page (Site settings → Integrations → the integration) already shows the
+approval, secrets, syncs with Import now and Auto-sync, a request tester and the runs. Build a
+panel only when the user wants their own screen for it (a store picker, a mapping matrix, an
+"Add component" drawer), the way Storesynk draws one for Shopify in Webflow.
+
+1. Install `@bettercms-ai/dock-input` (0.2.0 or later) and React in the site repo if they are not there.
+2. Write `bettercms/panels/<name>.tsx` (lowercase, hyphens), default-exporting a React component.
+   It is built with the site, served from the project's own isolated origin and drawn in a
+   sandboxed frame on the integration's page and in the Visual editor's dock.
+3. Set `update_integration { integration, panel: "<name>" }`. A panel never needs approval.
+
+```tsx
+import type { PanelProps } from "@bettercms-ai/dock-input";
+import { Button, Switch } from "@bettercms-ai/kit"; // when the site has the kit
+
+export default function ShopifyPanel({ integration, call, place }: PanelProps) {
+  const sync = integration.syncs[0] as { key: string; schedule?: string | null };
+  return (
+    <div>
+      <Switch checked={!!sync?.schedule} onCheckedChange={(on) =>
+        call("syncs.set", { syncs: integration.syncs.map((s) => (s === sync ? { ...s, schedule: on ? "hourly" : null } : s)) })} />
+      <Button onClick={() => call("sync.run", { sync: sync.key })}>Import Shopify data</Button>
+      {place === "dock" ? <Button onClick={() => call("editor.insertSection", { componentId: "<buy-button component id>" })}>Add buy button</Button> : null}
+    </div>
+  );
+}
+```
+
+The panel can make no request of its own (its CSP is `connect-src 'none'`), and it never sees a token
+or a secret; navigating its frame away ends it. A `syncs.set` that would make a sync publish, run on a
+schedule, write into another collection, or add or remove a sync asks the person first. Everything goes through `call(method, params)`, run by the dashboard with the signed-in
+person's rights:
+
+| Method | Params → result |
+|---|---|
+| `vendor.fetch` | `{ method?, url, query?, headers?, json? }` → `{ status, headers, body }`, as test_integration_fetch |
+| `sync.run` / `sync.status` | `{ sync }` → the run / `{ runId }` → the run |
+| `runs.list` | → the last runs |
+| `settings.set` | `{ settings }` → settings (non-secret, ≤ 16 KB; store ids, toggles) |
+| `syncs.set` | `{ syncs }` → syncs (schedules, mappings) |
+| `models.list` | → the collections a sync can write into |
+| `components.list` / `editor.insertSection` | the project's section components / `{ componentId }` adds one to the open page (dock only) |
+| `function.run` / `function.status` | `{ name, input? }` → the run / `{ runId }` → the run (output, logs, error) |
+| `reference.search` | `{ reference, query }` → up to 20 `{ id, title, image }` |
+
+## 8. Recipes
+
+| Vendor | Hosts | Secrets and inject | List request | Items / key |
+|---|---|---|---|---|
+| Greenhouse job board | `boards-api.greenhouse.io` | none (public board) | `GET /v1/boards/{board}/jobs?content=true` | `/jobs` / `/id` |
+| Shopify Admin | `{shop}.myshopify.com` | `TOKEN` → header `X-Shopify-Access-Token` | `GET /admin/api/2024-10/products.json?limit=250` | `/products` / `/id`, Link-header pages |
+| HubSpot CRM | `api.hubapi.com` | `TOKEN` (private app) → header `authorization: Bearer {{secret.TOKEN}}` | `GET /crm/v3/objects/contacts?limit=100` | `/results` / `/id`, cursor `/paging/next/after` |
+| Zoom webinars | `zoom.us`, `api.zoom.us` | `ACCOUNT_ID`, `BASIC` + token exchange → `authorization: Bearer {{token}}` | `GET https://api.zoom.us/v2/users/me/webinars` | `/webinars` / `/id`, cursor `/next_page_token` |
+| JobDiva | `api.jobdiva.com` | `CLIENT_ID`, `USERNAME`, `PASSWORD` + token exchange → `authorization: Bearer {{token}}` | the job search endpoint of their API | per their docs |
+| Apollo | `api.apollo.io` | `KEY` → header `x-api-key`; `allowPost: true` | `POST /api/v1/mixed_people/search` | `/people` / `/id` |
+
+Form submissions to a CRM do not need an integration: attach the form to HubSpot or a webhook
+(forms-and-interactions.md). Client-side widgets (a Shopify cart) use the vendor's PUBLIC
+storefront token in site code; an Admin token never goes into the site.
+
+## 9. Webhooks: the vendor tells the site something changed
+
+Polling hourly is the fallback. Most vendors sign a webhook on every change; declare one per event
+and the site follows within seconds. **The payload is never trusted**: only the item's id is read
+from it, and the item is re-fetched from the vendor through the sync's `item` request (section 6).
+A 404 there means the vendor deleted it (`onMissing: "delete"` removes the entry).
+
+```
+update_integration { hooks: [{
+  key: "products",
+  verify: { scheme: "hmac", header: "X-Shopify-Hmac-Sha256", secret: "WEBHOOK_SECRET",
+            algorithm: "sha256", encoding: "base64" },
+  deliveryIdHeader: "X-Shopify-Webhook-Id",
+  action: { kind: "sync", sync: "products", mode: "item", itemKey: "/id" } }] }
+```
+
+- `secret` is a template NAME declared in `secrets` (an env var the user sets). Hooks are on the
+  approval surface: adding or changing one needs an admin again.
+- `get_integration` → `hookUrls.<key>` is the URL to paste at the vendor (Shopify: Settings →
+  Notifications → Webhooks, JSON). Recent deliveries are in `hookEvents`.
+- `mode: "full"` runs the whole sync instead (a vendor that sends no id). `action: { kind:
+  "function", function: "<name>" }` hands the verified payload to a function (section 10).
+- Signature schemes (`verify`):
+
+| Vendor | header | algorithm / encoding | prefix | message |
+|---|---|---|---|---|
+| Shopify | `X-Shopify-Hmac-Sha256` | sha256 / base64 | | `{body}` |
+| GitHub | `X-Hub-Signature-256` | sha256 / hex | `sha256=` | `{body}` |
+| Greenhouse | `Signature` | sha256 / hex | `sha256 ` | `{body}` |
+| Stripe | `Stripe-Signature` | sha256 / hex, `parse: "stripe"` | | `{timestamp}.{body}` |
+| Slack | `X-Slack-Signature` | sha256 / hex, `timestampHeader: "X-Slack-Request-Timestamp"` | `v0=` | `v0:{timestamp}:{body}` |
+| HubSpot v3 | `X-HubSpot-Signature-v3` | sha256 / base64, `timestampHeader: "X-HubSpot-Request-Timestamp"` | | `{method}{url}{body}{timestamp}` |
+| a shared token | `scheme: "token"`, the header it arrives in | | `prefix` if any | |
+
+  A signed timestamp outside `toleranceSeconds` (300 by default) is refused as a replay, and a
+  redelivery (same delivery id, or the same body) is answered without running twice. Rotating a
+  webhook secret in Settings keeps the old value valid for 24 hours by default.
+
+## 10. Functions: your own code, never your secrets
+
+For what a declarative sync cannot say (send a form to HubSpot, reshape a webhook payload, a nightly
+report), write `bettercms/functions/<name>.ts` in the site repo and declare it:
+
+```ts
+// bettercms/functions/to-hubspot.ts
+export default async (event: { formId: string; submission: { data: Record<string, string> } }, ctx) => {
+  const { email, name } = event.submission.data;
+  const res = await fetch("https://api.hubapi.com/crm/v3/objects/contacts", {
+    method: "POST", headers: { "content-type": "application/json" },
+    body: JSON.stringify({ properties: { email, firstname: name } }),
+  });
+  console.log("hubspot", res.status);
+  return { status: res.status };
+};
+```
+```
+update_integration { functions: [{ name: "to-hubspot", triggers: ["form:<formId>"], cpuMs: 1000 }] }
+```
+
+- It runs on Cloudflare Workers when the site's build includes it. Its `fetch()` reaches ONLY this
+  integration's approved hosts, JSON bodies only, and the server adds the `inject` secrets: the code
+  never sees a key, so never write one into it (there is no env to read).
+- Triggers: `manual` (run_integration_function, the panel's `function.run`), `form:<formId>`
+  (each accepted submission: `{ formId, submission: { id, data } }`), `schedule:hourly|daily`, and
+  any hook whose action names it (`{ hook, payload }`). `ctx` = `{ integration, function, trigger,
+  runId, settings }`.
+- Return `{ upsert: [{ sync, externalKey, data, slug? }] }` to write entries into a sync's
+  collection (drafts, like a sync). `run_integration_function { waitSeconds: 20 }` answers with the
+  output, the captured `console` lines and any error; runs are in `get_integration.functionRuns`.
+- If the server answers `functions_not_enabled`, functions are not available yet on this
+  BetterCMS server: use a sync or an endpoint instead.
+
+## 11. Site endpoints: live reads for visitors, no key in the page
+
+A page that must show something live (stock, a search, today's webinars) calls the integration's
+endpoint, never the vendor: the key stays on the server and only the fields you pick leave.
+
+```
+update_integration { endpoints: [{
+  key: "stock",
+  request: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/variants/{{param.variant}}.json" },
+  params: { variant: { pattern: "[0-9]{1,20}", required: true } },
+  pick: { available: "/variant/inventory_quantity", price: "/variant/price" },
+  cacheSeconds: 60 }] }
+```
+
+- The site calls `GET <endpointUrls.stock>?variant=123` (public, CORS open, no credentials) and gets
+  `{ data: { available, price } }`. With `items: "/products"` each list item is picked instead.
+- Every param must be declared with an anchored `pattern`; an unknown or non-matching param is a
+  400. Patterns with a repeated group inside a repeat (`(a+)+`) are refused: they can hang.
+- Answers are cached for `cacheSeconds` per params; visitors are rate-limited per IP.
+- Endpoints are on the approval surface (what anyone on the internet can read).
+
+## 12. Live references: point at a vendor item without copying it
+
+```
+update_integration { references: [{
+  key: "product", label: "Shopify product",
+  search: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products.json", query: { title: "{{query}}", limit: "20" } },
+  lookup: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{id}}.json" },
+  items: "/products", item: "/product", id: "/id", title: "/title", image: "/image/src",
+  pick: { price: "/variants/0/price" } }] }
+```
+
+- The field that holds one: type `json` with `ui.reference: { integration: "<integration slug>",
+  key: "<reference key>" }`; editors get a search picker. Its value is `{ integration, reference, id, title, image? }` (title
+  and image are a snapshot for editors).
+- `search_integration_reference` finds ids for an agent. The site resolves the live item at
+  `<refs URL>/<id>` (`GET /api/v1/public/integrations/<projectId>/<slug>/refs/<key>/<id>`):
+  `{ id, title, image?, ...pick }`, cached `cacheSeconds` (300 by default).
+
+## 13. Sign in with OAuth (the customer's own OAuth app)
+
+For vendors that only offer OAuth (Google, HubSpot public apps, Zoom user-level): the user creates an
+OAuth app at the vendor, sets its client id and secret as env vars, and declares:
+
+```
+update_integration { secrets: { CLIENT_ID: "GOOGLE_CLIENT_ID", CLIENT_SECRET: "GOOGLE_CLIENT_SECRET" },
+  allowedHosts: ["oauth2.googleapis.com", "www.googleapis.com"],
+  oauth: { authorizeUrl: "https://accounts.google.com/o/oauth2/v2/auth",
+           tokenUrl: "https://oauth2.googleapis.com/token",
+           scopes: ["https://www.googleapis.com/auth/calendar.readonly"],
+           clientId: "CLIENT_ID", clientSecret: "CLIENT_SECRET",
+           extraAuthorizeParams: { access_type: "offline", prompt: "consent" } },
+  inject: { headers: { authorization: "Bearer {{token}}" } } }
+```
+
+- After approval, `start_integration_signin` gives a `url` for the USER to open (and the
+  `redirectUri` they register in their OAuth app first). They sign in at the vendor; the tokens are
+  stored encrypted and refreshed by the server. `{{token}}` in `inject` is the access token.
+- `oauth` and `auth` (a token exchange) do not mix. The token URL's host must be an allowed host.
+
+## 14. Templates, the call log, alerts
+
+- `save_integration_template` stores an integration's shape for the workspace (no value, no
+  settings; the collections its syncs write into are copied). `install_integration_template` puts
+  it on another project: missing collections are created, the integration waits for that project's
+  admin, and `secretsNeeded` lists the env vars to set there.
+- `get_integration.calls` is the call log: every vendor call (source, host, path, status, time),
+  never a query string or body. When runs fail `alertAfter` times in a row (3 by default; 0 = never)
+  the workspace's admins are notified once; the next success clears it.
+
+## 15. Two-way sync: BetterCMS edits go back to the vendor
+
+A sync pulls (section 6) and follows the vendor's webhooks (section 9). Add `writeBack` and edits
+made in BetterCMS (dashboard, Visual editor, API, you) go back to the vendor too:
+
+```
+{ key: "products", ..., map: { title: "/title", description: "/body_html", price: "/variants/0/price" },
+  item: { request: { method: "GET", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json" }, pointer: "/product" },
+  writeBack: {
+    fields: ["description"],                      // the fields BetterCMS owns
+    on: "save",                                   // or "publish": only what goes live is pushed
+    update: { method: "PUT", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json",
+              json: { product: { body_html: "{{field.description}}" } } },
+    create: { method: "POST", url: "https://acme.myshopify.com/admin/api/2024-10/products.json",
+              json: { product: { title: "{{field.title}}", body_html: "{{field.description}}" } }, idPointer: "/product/id" },
+    delete: { method: "DELETE", url: "https://acme.myshopify.com/admin/api/2024-10/products/{{item}}.json" } } }
+```
+
+- `{{item}}` is the vendor id (the entry's externalKey). `{{field.KEY}}` is the entry's value; a
+  JSON string that is exactly one placeholder keeps the value's type (a number stays a number).
+  `update` may use only `fields`; `create` may use any field of the new entry.
+- Needs `allowPost: true`, and `writeBack` is on the approval surface (it writes, and may delete,
+  at the vendor). `delete` is never done unless declared.
+- **Who wins.** Vendor-owned fields (`map` only) always follow the vendor. For `fields`, a BetterCMS
+  edit is pushed within about a minute; until it has reached the vendor, a pull or webhook does not
+  overwrite it. After that, the newest change from either side wins.
+- **No loops.** Both sides' agreed values are remembered per entry; our own push coming back
+  through a webhook, or a pull's own write, is recognised and never pushed again.
+- **New and deleted entries.** An entry made in BetterCMS is created at the vendor (with `create`)
+  and keeps the vendor's id as its externalKey, so the next pull updates it instead of duplicating.
+  Without `create` it stays BetterCMS-only. Deleting an entry deletes the vendor item only with `delete`.
+- **Failures** retry with backoff (1, 2, 4, 8, 16 minutes), then count toward `alertAfter`. Pushes
+  appear in the call log with source `push`.
+- **Known windows** (seconds to a minute, rare): an edit saved while a pull is writing that same entry,
+  or within a minute of changing `writeBack.fields`, can be replaced by the vendor's value. Change the
+  field list when editors are idle, and re-save an edit if a pull overwrote it.