@drawbridge/drawbridge-agents 0.1.59 → 0.1.60

package/.claude-template/skills/drawbridge-add-connection/SKILL.md

@@ -1,102 +1,235 @@
 ---
 name: drawbridge-add-connection
-description: Use when adding a vendor connection (a new manifest in drawbridge-utils), changing one's auth type or scopes, or adding a hook, link, or step to an existing manifest — and when asked to check an existing connection against the connection checklist.
+description: Use for ANY work touching connections, vendors or hooks in drawbridge-utils — adding or checking a manifest, adding a vendor or its credentials, changing auth shape or scopes, adding a hook/link/step, or moving a vendor's API calls out of another repo. Also use before answering questions about how connections work.
 ---
 
-# Add a connection
+# Connections
 
-A connection is one manifest in `drawbridge-utils/lib/connections/providers/`.
-The manifest is the only place vendor facts live, and it carries its own
-proof: a `review` key the contract test refuses to accept incomplete. This
-skill is the order of work that fills it honestly.
+**Invoke this before touching a manifest, a vendor file, or any code that calls a
+vendor's API. Not after. Darren, 2026-09-13: "never make a connection without
+using that skill."**
 
-**A manifest ships complete: every hook the vendor can serve, every link the
-dashboard exposes, every scope those need, copy humanized, walked on dev.
-Nothing is deferred for a deadline. Darren, 2026-09-11: "Don't skip shit, do
-this right, our production launch deadline is somewhat flexible." If the work
-does not fit the time, the time moves.**
+## Which job is this?
+
+| What you are doing | Go to |
+|---|---|
+| adding a whole new vendor connection | **A. New connection** |
+| adding credentials for a vendor | **B. Vendor** |
+| a repo calls a vendor's API directly | **C. Vendor code outside a manifest** |
+| checking an existing connection | **D. Check** |
+
+They share one law, and it is the reason this file exists.
+
+## THE LAW
+
+**A vendor's facts live on its manifest. Nothing else.** Not a constant in
+drawbridge-sync, not a client in drawbridge-api, not an env var read at a call
+site. If a line of code knows a vendor's endpoint, header, payload shape, error
+string or id format, it belongs in a hook.
+
+**A vendor is NAMED by the connection that spends it.** `vendors : [ … ]` on the
+manifest. That is what makes the provider screen able to say what a credential is
+for, and what makes `vendorSettings()` hand a hook its fields.
+
+**Grep before you invent.** A key, a field, a hook or a collection that already
+has a shape must reuse it. Read the contract and the scaffold first, every time.
+
+## STOP — the three mistakes that have actually been made
+
+Each of these was made by someone who had read this file. They are here as
+gates, not trivia.
+
+**1. Inventing a key instead of using the mechanism.** A vendor card said nothing
+about what its credentials were for, so a `usage` key was added to the vendor
+scaffold and to all eight vendor files — when the answer was that the manifest
+that spends it had not named it in `vendors`. The screen derives that. One list
+entry, no new key. *(2026-09-13. Earlier: a flat `provider` string was invented
+when `provider : { slug, id }` was already in five collections.)*
+
+> **Before adding any key, answer out loud: what already answers this? If the
+> answer is "a manifest would, if it declared the vendor / hook / field it
+> already can", do that instead.**
 
-## Steps
+**2. "This vendor has no connection, so it has no manifest."** Bright Data was
+written as a vendor nothing named, on the grounds that no merchant links a Bright
+Data account. Neither does anyone link our SendGrid account. **Who a credential
+belongs to is not who spends it.** Drawbridge's own vendors are spent by the
+`drawbridge` manifest and named in its `vendors` list — SendGrid, Twilio, HubSpot,
+Bright Data. A vendor no manifest names is a bug, not a category.
+
+**3. Leaving the vendor's calls behind.** Bright Data became a provider row and
+its API calls stayed in `drawbridge-sync/lib/scrape.js` for another day — an
+endpoint constant, a bearer header and a request body inline in a 3,798-line
+file. Moving the credential is half the job. **The calls are the job.**
+
+## A. New connection
+
+**Copy `_scaffold.js`. Answer every key. Delete nothing.**
+`drawbridge-utils/lib/connections/_scaffold.js` is three things at once: the file
+you copy, the reference for what each key may hold, and **the schema `build()`
+reads**. A manifest that omits a key or invents one cannot be imported — it fails
+before any test runs, in every repo that pulls the package, naming the key.
+
+So "did I miss something" is answered by the suite. What follows is judgement,
+which a schema cannot check.
+
+**A manifest ships complete: every hook the vendor can serve, every link its
+dashboard exposes, every scope those need, copy humanized, walked on dev.
+Nothing is deferred for a deadline. Darren, 2026-09-11: "Don't skip shit, do this
+right, our production launch deadline is somewhat flexible." If the work does not
+fit the time, the time moves.**
 
 1. **Read the vendor twice, and write down where.** Fetch, never recall — a
-   consent url once written from memory pointed at the api host and stalled
-   every merchant on a blank page.
+   consent url written from memory pointed at the api host and stalled every
+   merchant on a blank page.
    - **API reference**: auth mechanisms and who the vendor says each is for
      (your own account vs accounts outside your organization — Drawbridge is
-     always the second), partner or public-app program, OAuth urls and
+     always the second), partner or public-app program, OAuth endpoints and
      transport, token lifetime, refresh, every endpoint a hook will call.
-   - **Dashboard docs** (help centre, app UI): the page url for each object
-     we will hold an id for (list, segment, tag, contact, order), where the
-     vendor says the id appears, what the vendor calls each thing, and what a
-     disconnect actually does on their side. This read fills `urls.*` and the
-     merchant copy. A vendor whose UI urls are not documented gets an
-     observed form, marked observed, confirmed on dev before promote.
-   `review.api` and `review.dashboard` take the urls read.
-
-2. **Scope for the whole manifest, not today's hooks.** List every scope the
-   vendor offers with its url in `review.scopes`, then request every scope any
-   hook in the contract could need for this vendor — contacts, segments,
-   tags, custom fields, lists, webhooks — whether or not that hook ships
-   now. Scopes are app-level at Klaviyo and Attentive: adding one later costs
-   every connected merchant a re-consent. "Minimum for what I built today" is
-   the wrong minimum.
-
-3. **Pick `auth.type` from the vendor's own framing:**
-
-   | vendor's shape | type |
+   - **Dashboard docs**: the page url for each object we hold an id for, where
+     the vendor says that id appears, the vendor's own names for things, and what
+     a disconnect really does on their side. This read fills `urls.*` and the
+     merchant copy.
+
+   Both urls go on **the vendor file's `urls`**, beside its `scopes` link.
+   `build()` throws if a `review` key reappears on a manifest.
+
+2. **Scope for the whole manifest, not today's hooks.** Request every scope any
+   hook in the contract could need for this vendor, whether or not it ships now.
+   Scopes are app-level at Klaviyo and Attentive: adding one later costs every
+   connected merchant a re-consent. Declare them in `auth.oauth.scopes` —
+   **without it a vendor cannot detect drift**, which is the bug this whole
+   rework came out of.
+
+3. **The auth SHAPE says which kind it is. There is no `auth.type` to pick.**
+   All five options are present and exactly one is non-false:
+
+   | the vendor's shape | the truthy key |
    |---|---|
-   | installed from the vendor's app store, never redirects back | `install` |
-   | registered app, merchant consents on the vendor's screen | `oauth` |
-   | the vendor offers ONLY per-account credentials | `keys` |
-   | no third party behind it at all | `generated` |
-
-   `keys` is only for a vendor that offers nothing else to third parties.
-   Where OAuth exists, build OAuth; a registration that has not landed ships
-   the manifest dormant (`requires` names the client vars), never as keys.
-
-4. **Implement every hook the vendor can serve.** The only `false` is "the
-   vendor has no endpoint for this", with the one-line reason and the page
-   that shows the absence. Time is never a reason. "Flip later" is how segment
-   labels shipped as tags with no id kept. Every `urls` entry the dashboard
-   read found is implemented too.
-
-5. **Write the merchant copy from the two reads, then run the humanizer.**
-   Excerpt, description, guide, confirm, error copy: the vendor's own names
-   for things, the guide's navigation path from the help centre, and what
-   disconnect really does. Then invoke `humanizer:humanizer` on the copy
-   before commit and put the date in `review.content`. Not "if time".
-
-6. **Build against the checklist** in
-   `drawbridge-docs/reference/connection-hooks.md` → "Adding a connection".
-   Its guards are the enforcement: directory registration, feature grant,
-   `build()`, the `review` contract test. Trust a named failure over your
-   memory of the steps.
-
-7. **Walk it on dev, then date `review.verified`.** Connect a real account,
-   pick the resource, sync one contact and one opted-out contact, open each
-   `urls.*` link. `verified` is `null` on the publish that first carries the
-   manifest (the walk needs it deployed) and dated on the next; a manifest
-   still `null` at promote does not promote. A green suite is not the walk.
-
-## Checking an existing connection
-
-Same steps, read as questions: is each `review` entry a real url or date, do
-the requested scopes cover every hook the contract lists, does each `urls.*`
-entry exist and open, was the copy humanized. A gap is a manifest fix, not a
-note.
+   | installed from the vendor's app store, never redirects back | `auth.install` |
+   | registered app, merchant consents on the vendor's screen | `auth.oauth` |
+   | the vendor offers ONLY per-account credentials | `auth.keys` |
+   | Drawbridge mints the credential, no third party | `auth.generated` |
+   | nothing to authenticate | `auth.none` |
+
+   The other four are `false` **carrying the reason**. `type` is INJECTED by
+   `build()` from the shape; leave it out — a declared `type` that disagrees
+   with the shape throws.
+
+4. **Implement every hook the vendor can serve.** The only `false` is "the vendor
+   has no endpoint for this", with the reason and the page that shows the absence.
+   Time is never a reason.
+
+   **`lifecycle.health` is REQUIRED of every `oauth` and `install` connection** —
+   both end in a token somebody else can revoke, expire or narrow, and none of
+   those tells us.
+
+   **A hook describes its effects; it never performs them.** It receives
+   `( props, options )` and returns the answer envelope; `perform()` does the
+   writing. **Read `lib/connections/_hooks.js` before writing your first one** —
+   it carries a complete annotated hook and runs it.
+
+5. **Write the merchant copy from the two reads, then run `humanizer:humanizer`
+   on it before commit.** Excerpt, description, guide, confirm, error copy: the
+   vendor's own names, the help centre's navigation path, what disconnect really
+   does. `content.errors` names the vendor in both arms — a merchant connecting
+   two accounts in one sitting needs to know which one failed.
+
+6. **Let the enforcement check you.** `build()` refuses, at import: a missing or
+   invented top-level key, a missing hook, a hook prop its slot does not carry, an
+   unknown trigger, an oauth vendor declining `lifecycle.health`, two truthy auth
+   kinds, a step whose type disagrees with its position. Trust a named failure
+   over your memory of these steps.
+
+7. **Walk it on dev.** Connect a real account, pick the resource, sync one contact
+   and one opted-out contact, break the grant and confirm the error reaches the
+   card, the page and the checklist, reconnect and confirm it clears *immediately*,
+   then open each `urls.*` link. **A green suite is not the walk.** Record it in
+   the release doc for the change that carries the manifest — there is no
+   `review.verified` any more.
+
+## B. Vendor
+
+One vendor, one file: `lib/connections/vendors/<vendor>.js`, carrying `fields`,
+`icon`, `name`, `slug` and `urls`. However many connections spend it.
+
+**Then name it.** Add the slug to the `vendors` list of every manifest that
+spends it. **This is not optional bookkeeping** — it is what makes the provider
+screen say who spends the credential, and what makes `vendorSettings({ slug })`
+return the fields to that connection's hooks.
+
+- A vendor merchants connect → named by that connection (`klaviyo`, `shopify`).
+- **A vendor DRAWBRIDGE spends → named by the `drawbridge` manifest**, beside
+  SendGrid, Twilio, HubSpot and Bright Data. There is no such thing as a vendor
+  with no connection; there are vendors no MERCHANT connects.
+- Each field's `credential` names the env var it falls back to. That is the only
+  place that name is written — `requires` derives from it, and so does the test
+  env fixture.
+
+A vendor that is named by nothing renders a credential form saying nothing about
+what it powers. If you find yourself wanting a key to explain a vendor, you have
+found an unnamed vendor instead.
+
+## C. Vendor code outside a manifest
+
+You found an endpoint, a token header, a payload shape or a vendor's error string
+in drawbridge-sync, drawbridge-api, drawbridge-webhooks or anywhere else.
+
+**It becomes a hook.** Not "later", not "it has no connection", not "it needs a
+dependency the package cannot have".
+
+1. **Which manifest spends this vendor?** If a merchant connects it, theirs. If
+   Drawbridge pays for it, `drawbridge`. Name it in `vendors` if it is not there.
+2. **Which domain?** Read `HOOKS` in `contract.js` first. If nothing fits, add a
+   domain — with the reasoning for why it is not one of the existing ones, and
+   slots named for what every vendor of that kind does rather than for what this
+   one calls it. Every other manifest then answers the new domain `false` with a
+   reason; `build()` will not let you forget one.
+3. **Split vendor from Drawbridge, explicitly.** This is the judgement call and
+   it is the whole design. The vendor's facts move; our policy stays. From the
+   Bright Data move: the endpoint and the unblocking request moved; the tier
+   ladder that chooses between renderers, and the extraction that runs on whatever
+   comes back, did not.
+4. **A dependency the package must not carry is not an exemption.** Published
+   utils is imported by app-web and share, so it holds no browser, no BullMQ, no
+   socket server. The hook answers the part the vendor knows and the shell does
+   the rest — `scrape.browser` answers an ENDPOINT and the shell drives it. An
+   injected `fetcher` covers plain HTTP.
+5. **Sweep, and pin the sweep.** A test that greps this repo for the vendor's
+   endpoint and env names, failing on any that survive. Both
+   `connection-slug-neutral.test.js` (api) and `commerce-slug-neutral.test.js`
+   (sync) are the pattern, exemptions named with reasons and each one re-checked
+   for whether it is still earning its place.
+
+## D. Check
+
+Same steps read as questions: do the requested scopes cover every hook the
+contract lists, does each `urls.*` entry open, does the vendor file carry its
+api/dashboard/scopes links, **is the vendor named by the connection that spends
+it**, **is any of its wire detail still outside the manifest**, was the copy
+humanized, has it been walked. A gap is a fix, not a note.
 
 ## Red flags — stop
 
 - "Minimum scopes for now" — re-consent for every merchant later.
-- "Help centre only for the guide" — `urls.*` will be empty.
-- "Humanizer if time" — it is a step with a date, not a polish.
-- "Verified in the header comment" — `review.verified` is the record.
-- "Flip that hook later", "skipped because of the deadline" — nothing is
-  skipped. Say the time is short and keep going.
+- "Humanizer if time" — it is a step, not a polish.
+- "Flip that hook later", "skipped because of the deadline" — nothing is skipped.
 - "Keys as an interim, OAuth after the demo" — the OAuth build is the work.
+- "I'll add `review.verified`" — the key is gone and `build()` throws on it.
+- "The hook can just write it" — a hook holds no controller. Describe the write.
+- "`false` because nobody has asked for it yet" — `false` is a decision about the
+  VENDOR, with a reason.
+- **"This vendor has no connection"** — it has no MERCHANT. Name it on
+  `drawbridge`.
+- **"I'll add a key for that"** — what already answers it? Almost always a
+  manifest declaring something it already can.
+- **"The credentials moved, that's the migration"** — the calls are the job.
 
 ## Notes
 
 - A fact that must hold across repos goes through `drawbridge-record-contract`.
-- If the vendor's list endpoint filters by name server-side, the field must
-  NOT declare `search : false`; a test pins that for Attentive.
+- If the vendor's list endpoint filters by name server-side, the field must NOT
+  declare `search : false`; a test pins that for Attentive.
+- The reference doc's tables are GENERATED from the registry
+  (`lib/connections/reference.js`) and a test fails if they drift. Regenerate,
+  never hand-edit.

package/conventions/cross-repo-contracts.md

@@ -519,6 +519,12 @@ api's identity/organization routes); the merged keys keep the old names. Reading
 `providerSettings({ slug : 'drawbridge' })` anywhere finds no row — it was split on dev
 2026-09-08 and prod is seeded per-vendor. Detail: `drawbridge-docs/reference/providers.md`.
 
+- **The embedded app's install paths ARE `INSTALL_ENDPOINTS`** (utils `lib/connections/contract.js`
+  ↔ drawbridge-shopify-app `app/drawbridge.server.ts`). The api's `POST /connection/:vendor/*`
+  allows exactly those thirteen and 404s anything else; shopify-app does not depend on utils, so
+  `app/drawbridge.server.test.ts` carries the list verbatim and fails on drift. Rename one side,
+  rename both and the test — a mismatch is an embedded app whose organization list silently 404s.
+
 ## Import surfaces
 
 - Import names against a package's **actual exports** — a missing export resolves to `undefined`
@@ -527,10 +533,11 @@ api's identity/organization routes); the merged keys keep the old names. Reading
 
 ## Shopify sales channel
 
-- `@drawbridge/shopify` storefront `getProduct`/`getProductVariantsPage` throw the **exact
-  message `'Shopify product not found'`** when the Storefront API can't see a product
+- The Shopify storefront client (`@drawbridge/drawbridge-utils/shopify/storefront`, formerly its
+  own `@drawbridge/shopify` package) throws the **exact message `'Shopify product not found'`**
+  from `getProduct`/`getProductVariantsPage` when the Storefront API can't see a product
   (unpublished from the Drawbridge channel / deleted); transport and GraphQL failures throw
-  `'Shopify storefront error: …'`. drawbridge-sync's product worker (`queue/product.js`)
+  `'Shopify storefront error: …'`. The Shopify manifest's `commerce.product` hook
   string-matches the not-found message to deactivate the product and send ResourceFeedback
   instead of retrying. Reword the message → unpublished products retry-loop forever and
   merchants get no feedback; drawbridge-api's shopifyProduct route relies on the not-found
@@ -546,7 +553,7 @@ api's identity/organization routes); the merged keys keep the old names. Reading
 - **Feedback must forward the product's own `updatedAt`, never a fresh timestamp.** Shopify
   versions feedback on `productUpdatedAt` and refuses a payload older than the version it holds,
   so stamping `new Date()` records a version ahead of the product and every later send comes back
-  `'Feedback for a later version of this resource was already accepted'`. `@drawbridge/shopify`
+  `'Feedback for a later version of this resource was already accepted'`. The admin client
   reads it (costing one extra `read_products` query per send) so callers cannot get this wrong.
 - **ResourceFeedback messages are format-validated by Shopify, not by us**: one message, ≤ 100
   characters, leading capital, trailing period, not all caps. A breach fails the whole send.
@@ -570,7 +577,7 @@ api's identity/organization routes); the merged keys keep the old names. Reading
   throttling, or a field missing on the API version land in a **top-level `errors`** — a bare
   string for a 401, an array of objects otherwise — with a null `data`. Reading only `userErrors`
   turns an auth failure into a silent success.
-- `SHOPIFY_REQUIRED_SCOPES` (`@drawbridge/shopify` `lib/constants.js`) must stay in lockstep with
+- `SHOPIFY_REQUIRED_SCOPES` (`@drawbridge/drawbridge-utils` `lib/shopify/constants.js`) must stay in lockstep with
   `[access_scopes]` in drawbridge-shopify-app's `shopify.app.*.toml` (both files). drawbridge-api
   diffs each store's granted scopes against it to flag "update permissions" on the connections
   page. A scope added to the toml but not the constant silently escapes the check; one added to

package/conventions/drawbridge-packages.md

@@ -27,8 +27,9 @@ worked fine.
 ## Coordinating peer dependencies on bump
 
 When bumping any `@drawbridge/*` package, audit every sibling package's
-`peerDependencies` for a stale pin. Peer pins are version-exact (e.g.
-`"@drawbridge/shopify": "0.0.3"`), so bumping the package in consumer
+`peerDependencies` for a stale pin. Consumer pins are version-exact (e.g.
+`"@drawbridge/drawbridge-utils": "0.0.183"`; components' peer on utils is the
+one range, `>=0.0.118`), so bumping the package in consumer
 apps while leaving a sibling's peer dep on the old version triggers
 `ERESOLVE` at install time.
 

package/conventions/migrations.md

@@ -37,9 +37,32 @@ quietly that dev served stale code the whole time.
 - Add it to the release document in `drawbridge-docs/releases/` for the launch it
   belongs to, with the ordering constraint stated: what has to be deployed before
   it can run, and what breaks if it runs early.
-- If the work needs a worker, enqueue the job the live path enqueues, with the
-  same job id shape, so a migration-queued run and a live run coalesce rather
-  than doubling.
+- If the work needs a worker, enqueue the job the live path enqueues, through the
+  same helper the live path uses rather than a copy of it, so the payload and the
+  job id shape cannot drift.
+
+## Do not reach for a coalescing job id
+
+A static job id looks like coalescing and is usually suppression. BullMQ drops an
+add whose id still exists, and a queue that retains completed jobs will refuse a
+re-add for as long as it keeps them — a week on the connection queue, a month for
+failed ones. Worse, an id can be reused a layer down: `drawbridge-sync`'s
+`queue/workflow.js` derives each step's job id from the orchestrator job's id, so
+a static orchestrator id produced a static step id even though the orchestrator
+job itself carried `removeOnComplete`.
+
+That is not theoretical. It is what made a renamed segment never reach Mailchimp
+or Klaviyo: the first register ran, and every later one for that segment was
+dropped in silence for seven days.
+
+**Make the id unique per dispatch and make the work idempotent instead.** A
+register that writes the same row twice costs nothing. A register that is dropped
+loses a change the merchant can see. Where a timestamp is the unique part, add a
+counter too, because a loop dispatches many jobs inside one millisecond.
+
+Reserve a static id for the case where running twice is genuinely harmful AND the
+queue clears the id on both outcomes, and say in a comment which of the two you
+are relying on.
 
 ## Media migrations, the one exception
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@drawbridge/drawbridge-agents",
-  "version": "0.1.59",
+  "version": "0.1.60",
   "description": "Shared agent-instruction content (rules, code style, conventions) for the drawbridge-* monorepo.",
   "license": "UNLICENSED",
   "publishConfig": {