toga-ai 1.0.360 → 1.0.361

package/knowledge/sessions/2026-07-16-bdr-hscontactid-rate-limit-tcox.md

@@ -0,0 +1,81 @@
+---
+type: session
+slug: bdr-hscontactid-rate-limit
+title: BDR ?hsContactId= enumeration rate-limit mitigation (security-reviewed SHIP)
+author: tcox
+repos: [bdr, ai-bdr]
+framework: "2.0"
+client: shared
+status: active
+created: 2026-07-16
+updated: 2026-07-16
+---
+
+# Session: bdr-hscontactid-rate-limit
+**Date:** 2026-07-16
+**Project/Repo:** bdr (2.0) — local `C:\WWW\BDR`, GitHub `agilantsolutions/BDR`, branch `BDR-Phase-2`
+**Task:** Build an interim app-layer rate-limit mitigation for the public funnel's `?hsContactId=` enumeration risk (PLAN §10, HIGH), driven to a security-reviewed SHIP; along the way, wire local dev env and diagnose why a live test call never fired.
+
+---
+
+## What WORKED
+- **Interim rate-limit mitigation BUILT and SHIP-verified (two CSO review rounds).** New `src/proxy.ts` (Next 16 proxy), `src/lib/rateLimit.ts` (token bucket), `src/lib/clientIp.ts` (trusted IP resolution). Policy: entry pages with `?hsContactId=` **soft-degrade** over limit (strip id, `NextResponse.rewrite` to render anonymously — no PII, no Toga upsert; hsCampaignId preserved); `/api/contact`, `/api/call-now`, `/api/call-later` return a `429` `{isSuccess,status,error,messages,data}` envelope with `Retry-After`. Evidence: tsc clean, eslint clean, **104/104 vitest**, `next build` compiles it (logs `ƒ Proxy (Middleware)`).
+- **C1 (Critical) FIXED — trusted client IP.** `resolveClientIp()` parses `X-Forwarded-For` **right-to-left** at `parts.length - 1 - RATE_LIMIT_TRUSTED_PROXY_HOPS` (default 0 => rightmost = CloudFront-appended viewer), so a forged/leftmost XFF can't shift the selected index. `x-real-ip` no longer trusted. `CloudFront-Viewer-Address` trusted ONLY when `RATE_LIMIT_TRUST_CF_VIEWER_HEADER=true` (env, default OFF). `normalizeIp` strips port, `[ipv6]:port`, `::ffff:` mapping. Verified by `test/clientIp.test.ts` incl. a 20-entry spoof-flood that still resolves to the trusted tail.
+- **H1 (High) FIXED — limiter map hard-bounded.** `MAX_TRACKED_KEYS=10_000`; `check()` re-inserts existing keys (Map order = LRU); new-key insert at cap runs `makeRoom()` = throttled `sweepIdle` (≤ once/window) + O(1) LRU eviction loop. Verified: 50k unique keys → size ≤ 10k; a key touched every 50 iters survives a 40k flood and stays drained/denied.
+- **M FIXED — `?hsCampaignId=` validated.** `page.tsx` now `isTogaUuid`-validates it before it flows into `togaLeadSink.upsertLead` as `togaCampaignUuid`; malformed drops to `undefined`.
+- **CSO re-review verdict: SHIP** (SAFE TO SHIP) — all of C1/H1/M genuinely closed, no new defects.
+- **Local dev env wired.** Created `C:\WWW\BDR\.env.local` (gitignored) by copying the 5 server-side vars (`HUBSPOT_ACCESS_TOKEN` + 4 `TOGA_*`) from `C:\WWW\info\.env.local` (PRODUCTION values, base `https://api.togahub.com/v2`). GA left unset locally (analytics.ts no-ops). Verified auth + contact writes hit prod.
+- **Diagnosed the BDR call flow end to end.** A `200 POST /api/call-now` in the browser is success from BDR's side; Toga calls are server-side (never in the browser network tab). BDR only sets `dtNextContactRequested`/`isOkayToCall`/call-type on the contact; the worker2 dialer cron places the actual Vapi call. Confirmed the test contact `88800869-acc7-632b-2520-bbd601224ff5` got `dtNextContactRequested` written in prod.
+- **Two `/capture` pushes to team KB (PUSHED).** `web-funnel-app.md` (rate-limit mitigation, server-side call seam, phone-less bug, `/api/contact` scrape vector, Next 16 proxy gotcha, dev-setup), `worker2/vapi-webhook-handler.md` (new "Outbound dialer eligibility" section), `ai-bdr/call-orchestration.md` (cross-ref). Second capture superseded the mid-state (test count → 104, clientIp.ts, SHIP verdict, 3 new security gotchas).
+
+## What did NOT work — DO NOT RETRY THESE
+- **Live test call never fired despite a correct 200 submit — TWO production-data blockers, not a code bug.**
+  1. The test contact's `Contacts.primaryContactPhoneNumberId` is **null**, so the dialer's eligibility SQL (`getEligibleContactsForVapi`, INNER JOIN `ContactPhoneNumbers ON Contacts.primaryContactPhoneNumberId`) can't see it. Root cause: the contact was created WITHOUT a phone (HubSpot record has none); the phone arrives via the later Call Now `PUT`, and **api2 creates the ContactPhoneNumber row but does not set the primary pointer on update**. Re-PUTting does NOT fix it (verified: primary stayed null while a phone row `(630) 803-7765` existed). This is a real product bug — any phone-less HubSpot contact can never be called; the funnel still shows success.
+  2. The active campaign `26.05 - AI BDR - Ryan Nitti` (`15d0c02f-501e-e081-2ead-61b78c854223`) reads `campaignAssistantId = null`; the dialer INNER JOINs `CampaignAssistants`, so a null-assistant campaign is never dialed. Could not confirm whether truly null or hidden from the API credential.
+- **`node check-contact.mjs` with a `with=ContactCallType,PrimaryContactPhoneNumber,CampaignsContacts` clause → HTTP 404.** The `with=` relation names were wrong for this endpoint; the plain `join=…&where=(…)` query returns 200. Use the join form, not `with=`.
+- **Toga `/contact-attempts` and `/campaigns-contacts` (some join variants) → HTTP 403/500 for this API credential.** Can't read ContactAttempts or all campaign-contact links directly; infer from contact + campaign state instead.
+- **Running `npx tsc`/`npx vitest`/`npx eslint`/`npx next build` without `cd /c/WWW/BDR &&` first.** The Bash tool cwd resets to `c:\WWW`, so tsc resolves the wrong binary and vitest runs sibling repos (saw 189 failing toga-blox-npm tests). ALWAYS prefix `cd /c/WWW/BDR &&`.
+- **`middleware.ts` on Next 16 is DEPRECATED.** Next 16.2.3 warns "use proxy instead". Correct convention is `src/proxy.ts` exporting a `proxy` function + `export const config: ProxyConfig`; import from `next/server`. (Confirmed from `node_modules/next/dist/lib/constants.js` + build source; did not ship a middleware.ts.)
+- **`info/.env.beta` is a DEAD file** — empty URLs, obsolete var names (`TOGA_CLIENT_SECRET` vs the current `TOGA_CLIENT_API_UUID`/`SECRET` pair). Production (`info/.env.local`) is the only working credential set `info` has. Do not source dev env from `.env.beta`.
+
+## Not tried yet (candidates for next session)
+- **Commit the working-tree changes** on `BDR-Phase-2` (developer has not asked yet). Suggested split: `feat: rate-limit hsContactId enumeration surfaces` (proxy.ts, rateLimit.ts, clientIp.ts, tests, PLAN §10) + `fix: validate hsCampaignId before toga upsert` (page.tsx).
+- **Unblock the live call test** (needs AI-BDR owner or a production write): set the test contact's `primaryContactPhoneNumberId`, and confirm/fix the campaign's `campaignAssistantId`.
+- **Decide the phone-less-contact bug fix**: api2-side (set primary on phone-row create during update) vs. a BDR workaround (PUT linking the existing phone-row uuid as primary).
+- **Non-gating security follow-ups (PLAN §10 / Phase 8):** lock the Amplify origin so nothing bypasses CloudFront; confirm the distribution injects+strips `CloudFront-Viewer-Address` before enabling `RATE_LIMIT_TRUST_CF_VIEWER_HEADER`; harden `readJsonObject` (body buffered before size check; measures UTF-16 length not bytes); re-verify no un-normalized path variant reaches a handler off the proxy `matcher`.
+- **Phase 8 authoritative controls:** edge WAF per-IP rate rule (supersedes the interim in-app limiter); opaque/signed CRM link tokens instead of raw `hsContactId` (raise with HubSpot/CRM owner).
+- Remaining PLAN §9 items (unchanged from prior sessions): §9.3 agent choice, §9.4 Landing service-picker scope, §9.5 campaign catalog, §9.6 togatech integration mechanism (gates deploy), §9.7 Tweaks panel in prod, §9.8 first-campaign copy, §9.12 Share button (still a mock), §9.13 connect-time copy. Phase 7 polish (pixel/motion/a11y; GA4 done).
+
+## Current file state
+| File | Status | Notes |
+|------|--------|-------|
+| `src/proxy.ts` | ADDED, uncommitted | Next 16 proxy; per-IP limits, soft-degrade entry + 429 API routes; env `RATE_LIMIT_TRUSTED_PROXY_HOPS`, `RATE_LIMIT_TRUST_CF_VIEWER_HEADER` |
+| `src/lib/rateLimit.ts` | ADDED, uncommitted | Pure token-bucket, clock-injected, hard-bounded 10k-key LRU map |
+| `src/lib/clientIp.ts` | ADDED, uncommitted | Pure trusted-IP resolution (right-to-left XFF; CF header opt-in; normalizeIp) |
+| `src/app/page.tsx` | MODIFIED, uncommitted | `isTogaUuid`-validate `hsCampaignId` before Toga upsert |
+| `test/rateLimit.test.ts` | ADDED, uncommitted | limiter unit tests incl. hard-bound-under-flood + LRU survival |
+| `test/clientIp.test.ts` | ADDED, uncommitted | IP-resolution tests incl. spoofed-leftmost-XFF bypass attempt |
+| `PLAN.md` | MODIFIED, uncommitted | §10 mitigation + SHIP + residuals; §11 Phase-8 checklist items |
+| `.env.local` | ADDED, uncommitted (gitignored) | 5 server-side vars from info prod; NEVER committed |
+| Repo otherwise | clean at `bb40b15` on `BDR-Phase-2` | all above are working-tree only; no commit/PR yet |
+| Team KB (`toga-tech`) | PUSHED | web-funnel-app.md, worker2/vapi-webhook-handler.md, ai-bdr/call-orchestration.md updated via 2× /capture |
+
+## Decisions made
+- **App-layer rate limiting now, as defense-in-depth** — not the authoritative fix. Per-instance memory on Amplify SSR is honestly documented; the WAF per-IP rule + opaque/signed CRM tokens remain the real controls (Phase 8). Chosen because it's the only layer buildable pre-deploy.
+- **Entry page soft-degrades; API routes hard-429.** Stripping `hsContactId` keeps a real (shared-NAT) user in the funnel minus prefill while denying a scraper any PII/upsert; the call/contact routes trigger real actions or leak full PII, so a hard stop is correct.
+- **Trust IP from the rightmost XFF, not leftmost** (C1 fix) — the leftmost is client-prependable behind CloudFront; the rightmost is CloudFront-appended. `CloudFront-Viewer-Address` made **opt-in (default off)** so an unverified distribution can't be bypassed via a forged copy — safe-by-default, ops enables after verifying.
+- **Hard-bound the limiter map with LRU eviction** (H1 fix) — a refill-only sweep is itself a unique-key-flood DoS; a hard ceiling + O(1) LRU + throttled reclaim guarantees bounded memory and O(1) amortized inserts.
+- **Used Next 16 `proxy.ts`, not `middleware.ts`** — the latter is deprecated in 16.
+- **Deferred `readJsonObject` + matcher-normalization fixes** as non-gating follow-ups — pre-existing, bounded by the Lambda payload cap + the call limiter; keeping the change focused and low-risk was the better call than widening scope into shared validated code.
+- **Phone-less-contact bug recorded as a gotcha, not its own doc** — has an api2-side root cause but surfaces on the funnel + worker2 dialer docs per KB conventions.
+
+## Blockers
+- **Live call test blocked on production data** (owner: AI-BDR owner, e.g. Rohan/Akash): test contact `primaryContactPhoneNumberId` is null, and campaign `campaignAssistantId` appears null. Both need a production write or owner action; developer's standing rule is read-only DB access for Claude.
+- **Committing to `BDR-Phase-2`** awaits developer go-ahead (standing "ask before git/remote" policy).
+- Pre-existing PLAN blockers unchanged: §9.6 togatech integration (gates Phase 8 deploy), §9.8 first-campaign copy from leadership.
+
+## Exact next step
+> Decide whether to (a) commit the working-tree changes on `BDR-Phase-2` as the two commits noted above, or (b) tackle the phone-less-contact bug fix — whichever the developer wants first. The security work itself is complete (SHIP-verified) and fully captured; nothing there is pending.
+
+---
+_Saved by /session-save on 2026-07-16_

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.360",
+  "version": "1.0.361",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",