toga-ai 1.0.358 → 1.0.360
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -6,6 +6,6 @@
|
|
|
6
6
|
| [BDR Web Funnel — Full Implementation Plan](features/bdr-web-funnel-plan.md) | > **Execution status (2026-07-16):** PLAN **Phases 0–5 are BUILT and QA'd**, Phase 6 > was **mostly already built** (the `info` contract was implemented at comm | bdr/PLAN.md, bdr/mockup/app.jsx, bdr/mockup/screens.jsx, bdr/mockup/components.jsx |
|
|
7
7
|
| [Call Orchestration — PHP Worker ↔ Vapi (the integration seam)](features/call-orchestration.md) | The **PHP worker** is the orchestrator; **Vapi** is the actor. | ai-bdr/docs/client-onboarding-sop.md, ai-bdr/docs/vapi-firstmessage-timing-fix.md, ai-bdr/docs/vapi-inbound-callback-assistant-request.md, ai-bdr/docs/vapi-voicemail-iphone-screening.md, ai-bdr/scripts/update_structured_output.py |
|
|
8
8
|
| [Vapi Integration — Assistants, Tools, Structured Output](features/vapi-integration.md) | Everything inside Vapi: the three assistants, the three shared tools, the shared structured-output schema, the Liquid-templated system prompt, and the Python sc | ai-bdr/vapi/templates/assistant.template.json, ai-bdr/vapi/templates/system-prompt.template.md, ai-bdr/prompts/archive/prompt-may-15.txt, ai-bdr/prompts/campaigns/healthcare-2026-03-18.md, ai-bdr/prompts/campaigns/healthcare-v2-2026-03-25.md, ai-bdr/prompts/templates/campaign-template.md, ai-bdr/scripts/update_assistant.py, ai-bdr/scripts/update_system_prompt.py, ai-bdr/scripts/update_structured_output.py, ai-bdr/scripts/update_call_summary_context.py, ai-bdr/scripts/fix_booking_guardrails.py, ai-bdr/scripts/get_assistant.py, ai-bdr/docs/vapi-firstmessage-timing-fix.md, ai-bdr/docs/vapi-voicemail-iphone-screening.md |
|
|
9
|
-
| [Web Funnel — Built Next.js App (structure, stack, how to run)](features/web-funnel-app.md) | The **built** state of the BDR web funnel — the Next.js UI / lead-capture front door of the existing AI-BDR product (see `../architecture.md`). | bdr/PLAN.md, bdr/.gitignore, bdr/eslint.config.mjs, bdr/amplify.yml, bdr/src/app, bdr/src/app/layout.tsx, bdr/src/app/page.tsx, bdr/src/app/studio.css, bdr/src/lib/resolveCampaignId.ts, bdr/src/lib/scheduleSlots.ts, bdr/src/lib/formatPhone.ts, bdr/src/lib/analytics.ts, bdr/src/components/GoogleAnalytics.tsx, bdr/src/components/RichText.tsx, bdr/src/content, bdr/src/components, bdr/src/flow, bdr/src/server, bdr/src/app/api, bdr/test, bdr/public, bdr/mockup/styles.css |
|
|
9
|
+
| [Web Funnel — Built Next.js App (structure, stack, how to run)](features/web-funnel-app.md) | The **built** state of the BDR web funnel — the Next.js UI / lead-capture front door of the existing AI-BDR product (see `../architecture.md`). | bdr/PLAN.md, bdr/.gitignore, bdr/eslint.config.mjs, bdr/amplify.yml, bdr/src/app, bdr/src/app/layout.tsx, bdr/src/app/page.tsx, bdr/src/app/studio.css, bdr/src/lib/resolveCampaignId.ts, bdr/src/lib/scheduleSlots.ts, bdr/src/lib/formatPhone.ts, bdr/src/lib/analytics.ts, bdr/src/lib/rateLimit.ts, bdr/src/lib/clientIp.ts, bdr/src/proxy.ts, bdr/.env.example, bdr/src/components/GoogleAnalytics.tsx, bdr/src/components/RichText.tsx, bdr/src/content, bdr/src/components, bdr/src/flow, bdr/src/server, bdr/src/app/api, bdr/test, bdr/public, bdr/mockup/styles.css |
|
|
10
10
|
| [Web Funnel — Next.js UI + Config-Driven Campaign Content](features/web-funnel-content-model.md) | > **Status note (RESOLVED 2026-07-16):** the content **contract** below (schema, > provider seam, adapters) is BUILT and QA'd (PLAN Phases 0–5). | bdr/src/content/schema.ts, bdr/src/content/provider.ts, bdr/src/content/useCampaign.ts, bdr/src/content/default.ts, bdr/src/lib/resolveCampaignId.ts, bdr/src/server/leadSink.ts, bdr/src/server/callbackService.ts, bdr/src/app/api/call-now/route.ts, bdr/src/app/api/call-later/route.ts, bdr/src/app/api/contact/route.ts |
|
|
11
11
|
| [New-Campaign Onboarding (6-phase runbook)](workflows/new-campaign-onboarding.md) | The end-to-end procedure for launching a new outbound BDR campaign (industry, offer, target persona, geography). | ai-bdr/docs/client-onboarding-sop.md, ai-bdr/docs/calcom-new-campaign-event-guide.md, ai-bdr/prompts/templates/campaign-template.md, ai-bdr/scripts/update_assistant.py, ai-bdr/scripts/update_system_prompt.py |
|
|
@@ -6,8 +6,8 @@ project: AI-BDR
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [akhokhani]
|
|
9
|
+
updated: 2026-07-16
|
|
10
|
+
owners: [akhokhani, tcox]
|
|
11
11
|
files:
|
|
12
12
|
- ai-bdr/docs/client-onboarding-sop.md
|
|
13
13
|
- ai-bdr/docs/vapi-firstmessage-timing-fix.md
|
|
@@ -208,7 +208,11 @@ gracefully handles `"Information not available"`.
|
|
|
208
208
|
**Now committed in `worker2`** as `_Worker_Vapi` (`worker2/Worker/Vapi.php`,
|
|
209
209
|
the webhook) alongside the outbound dialer `_Worker_Ai_Bdr_Vapi`
|
|
210
210
|
(`worker2/Worker/Ai/Bdr/Vapi.php`). Full detail:
|
|
211
|
-
**`2.0/apps/worker2/features/vapi-webhook-handler.md
|
|
211
|
+
**`2.0/apps/worker2/features/vapi-webhook-handler.md`** — which now also documents the
|
|
212
|
+
**dialer eligibility rules** (`getEligibleContactsForVapi()`: the exact ALL-of criteria that
|
|
213
|
+
decide whether a contact is dialed, the `dtNextContactRequested <= NOW()` call-window/cadence
|
|
214
|
+
bypass used by the website "call now/scheduled" path, and the phone-less-contact gotcha where a
|
|
215
|
+
missing `primaryContactPhoneNumberId` silently excludes a contact).
|
|
212
216
|
|
|
213
217
|
> **Corrections confirmed against the live `_Worker_Vapi` code (2026-06-24):**
|
|
214
218
|
> 1. **Structured output is in `artifact.structuredOutputs`** (keyed by a random
|
|
@@ -263,4 +267,7 @@ feature doc in that repo's `features/` and link it from this file's
|
|
|
263
267
|
- **`attemptNumber` is 1-indexed**, not 0-indexed. First call = 1.
|
|
264
268
|
|
|
265
269
|
## Change history
|
|
270
|
+
- 2026-07-16 — Cross-referenced the newly documented dialer eligibility rules
|
|
271
|
+
(`getEligibleContactsForVapi`) in `worker2/features/vapi-webhook-handler.md` from the
|
|
272
|
+
"Where the PHP code lives" section. (tcox)
|
|
266
273
|
- 2026-06-16 — Initial call-orchestration feature doc (PHP ↔ Vapi seam). (akhokhani)
|
|
@@ -21,6 +21,10 @@ files:
|
|
|
21
21
|
- bdr/src/lib/scheduleSlots.ts
|
|
22
22
|
- bdr/src/lib/formatPhone.ts
|
|
23
23
|
- bdr/src/lib/analytics.ts
|
|
24
|
+
- bdr/src/lib/rateLimit.ts
|
|
25
|
+
- bdr/src/lib/clientIp.ts
|
|
26
|
+
- bdr/src/proxy.ts
|
|
27
|
+
- bdr/.env.example
|
|
24
28
|
- bdr/src/components/GoogleAnalytics.tsx
|
|
25
29
|
- bdr/src/components/RichText.tsx
|
|
26
30
|
- bdr/src/content
|
|
@@ -53,8 +57,17 @@ fully implemented at commit `34a3910`: `page.tsx primeLead` reads `?hsContactId=
|
|
|
53
57
|
campaign attribution → only `togaUuid` crosses to the client). The 2026-07-16 session
|
|
54
58
|
added the deltas below (phone prefill + live input mask, `submitOk` on Success, GA4
|
|
55
59
|
wiring) and resolved §9.10 (content mechanism — see `web-funnel-content-model.md`) and
|
|
56
|
-
§9.11 (consent links).
|
|
57
|
-
|
|
60
|
+
§9.11 (consent links). A later 2026-07-16 session added an **app-layer rate-limit
|
|
61
|
+
mitigation** for the `?hsContactId=` enumeration risk (`src/proxy.ts` + `src/lib/rateLimit.ts`
|
|
62
|
+
+ `src/lib/clientIp.ts`),
|
|
63
|
+
documented the **fully server-side request-a-call seam**, the **outbound dialer eligibility
|
|
64
|
+
rules** (see `../../worker2/features/vapi-webhook-handler.md`), the **phone-less-contact
|
|
65
|
+
dialer bug**, and the **local dev env wiring**. Developer QA verdict: the UI is "very close,
|
|
66
|
+
minimal adjustments to be near exact" vs. the mockup. The rate-limit mitigation then passed an
|
|
67
|
+
**independent `cso` security review — final verdict SHIP after two rounds** — and was HARDENED
|
|
68
|
+
to its ship state (trusted-IP position for X-Forwarded-For; O(1) LRU-capped limiter map; the
|
|
69
|
+
`?hsCampaignId=` page-prime param now `isTogaUuid`-validated). All checks green: tsc, eslint,
|
|
70
|
+
**104/104 Vitest**, `next build` (build output labels the proxy `ƒ Proxy (Middleware)`).
|
|
58
71
|
|
|
59
72
|
- **Repo:** `agilantsolutions/BDR` on GitHub, working branch `BDR-Development`. The
|
|
60
73
|
machine-local `.claude/` harness is **gitignored** — committing it would snapshot
|
|
@@ -75,9 +88,19 @@ to be near exact" vs. the mockup. All checks green: tsc, eslint, 79/79 Vitest, `
|
|
|
75
88
|
- **ESLint flat config** carries a **no-em-dash rule** on shipped strings
|
|
76
89
|
(`Literal` / `TemplateElement` / `JSXText` selectors) — enforces the mockup copy
|
|
77
90
|
rule on in-repo strings (`eslint.config.mjs`).
|
|
78
|
-
- **Vitest** test suite: **
|
|
79
|
-
phone format/prefill/mask, live-anchor consent links, `submitOk` race
|
|
80
|
-
|
|
91
|
+
- **Vitest** test suite: **104 tests, all green** (13 added in the first 2026-07-16 session:
|
|
92
|
+
phone format/prefill/mask, live-anchor consent links, `submitOk` race; the later 2026-07-16
|
|
93
|
+
session added the rate-limiter + trusted-IP + LRU-eviction + `hsCampaignId` validation tests
|
|
94
|
+
to reach 104). Every phase was verified with `tsc` + lint + build + a live SSR `curl` before
|
|
95
|
+
commit.
|
|
96
|
+
- **Edge proxy (Next 16 `proxy.ts`) — 3-file rate-limit mitigation.** `src/proxy.ts` is the
|
|
97
|
+
Next 16 request proxy (the successor to the deprecated `middleware.ts` convention — see the
|
|
98
|
+
Next 16 gotcha) carrying the rate-limit mitigation. It composes two libs: `src/lib/rateLimit.ts`,
|
|
99
|
+
a pure clock-injected in-memory token-bucket limiter with an O(1) LRU-capped map (no
|
|
100
|
+
`server-only`, so it runs in the edge proxy and is unit-testable in Node); and
|
|
101
|
+
`src/lib/clientIp.ts`, the pure client-IP resolution logic (no `next` import, so it is
|
|
102
|
+
independently testable) that parses `X-Forwarded-For` from a **trusted** position. See the two
|
|
103
|
+
rate-limiter security gotchas below.
|
|
81
104
|
- **Deploy target:** AWS Amplify (SSR), `amplify.yml` present (Phase 8 not done).
|
|
82
105
|
|
|
83
106
|
## Directory map (what got built)
|
|
@@ -99,12 +122,16 @@ to be near exact" vs. the mockup. All checks green: tsc, eslint, 79/79 Vitest, `
|
|
|
99
122
|
- `src/server/*` — the real backend, ported from `info`'s `toga.ts` / `hubspot.ts`
|
|
100
123
|
behind `LeadSink` + `CallbackService` adapters (detail in
|
|
101
124
|
`web-funnel-content-model.md`).
|
|
102
|
-
- `src/app/api/*` — route handlers `call-now`, `call-later`, `contact`.
|
|
125
|
+
- `src/app/api/*` — route handlers `call-now`, `call-later`, `contact`. **These do NOT
|
|
126
|
+
place a call themselves** — see "Request-a-call is fully server-side" below.
|
|
127
|
+
- `src/proxy.ts` — Next 16 edge proxy (rate limiting; see gotchas). `src/lib/rateLimit.ts` —
|
|
128
|
+
the LRU-capped token-bucket limiter it uses. `src/lib/clientIp.ts` — pure client-IP
|
|
129
|
+
resolution (trusted `X-Forwarded-For` position; no `next` import).
|
|
103
130
|
- `public/*` — assets pruned to the **default campaign only**, **plus AutoBuild's
|
|
104
131
|
FLOURISH characters** (Owl 1 `owl-02` + Robot 1 `robot-02`, art + talk videos) which
|
|
105
132
|
the assembly cycle needs even though they are not the campaign agent. See the
|
|
106
133
|
asset-pruning gotcha below.
|
|
107
|
-
- `test/*` — the
|
|
134
|
+
- `test/*` — the Vitest suite (104 tests).
|
|
108
135
|
|
|
109
136
|
## How the built app behaves (key runtime facts)
|
|
110
137
|
|
|
@@ -156,6 +183,67 @@ to be near exact" vs. the mockup. All checks green: tsc, eslint, 79/79 Vitest, `
|
|
|
156
183
|
`http(s)` hrefs as live anchors (`target=_blank`, `rel="noopener noreferrer"`, aria-label
|
|
157
184
|
"... (opens in new tab)"); bare `#hash` hrefs keep the mockup's inert `preventDefault`.
|
|
158
185
|
|
|
186
|
+
## Request-a-call is fully server-side ("no API call in my console" is expected)
|
|
187
|
+
|
|
188
|
+
The "request a call" flow (Call Now / Schedule) is **entirely server-side** — this is by
|
|
189
|
+
design, and repeatedly causes "there's no Toga API call in my network tab" confusion.
|
|
190
|
+
|
|
191
|
+
- The **browser only POSTs to the Next route handler** (`/api/call-now`, `/api/call-later`).
|
|
192
|
+
A `200 POST /api/call-now` in the console **is** the success signal.
|
|
193
|
+
- The **Toga calls happen inside that route handler**, server-side: `POST /auth/api` to get a
|
|
194
|
+
token, then `PUT /contacts/{uuid}` against production `api.togahub.com/v2`
|
|
195
|
+
(`src/server/toga.ts`, `src/server/callbackService.ts`, `src/flow/leadActions.ts`). So
|
|
196
|
+
Toga/api2 traffic **never appears in the browser network tab** — only the internal
|
|
197
|
+
`/api/*` hop does.
|
|
198
|
+
- **BDR itself never places a call.** The `PUT /contacts/{uuid}` only flips flags on the
|
|
199
|
+
contact: `dtNextContactRequested` (literal `NOW()` for immediate, a CST-formatted datetime
|
|
200
|
+
for scheduled), `isOkayToCall=1`, `isActive=1`, and the `contactCallType {uuid}` (immediate
|
|
201
|
+
vs. scheduled `CONTACT_CALL_TYPE_UUID`). The actual Vapi call is placed later by the
|
|
202
|
+
**worker2 dialer cron** (`_Worker_Ai_Bdr_Vapi`) — see the eligibility rules below.
|
|
203
|
+
|
|
204
|
+
## The outbound dialer decides whether the call ever happens (worker2)
|
|
205
|
+
|
|
206
|
+
Setting `dtNextContactRequested` does **not** guarantee a call — the worker2 dialer's
|
|
207
|
+
eligibility SQL (`getEligibleContactsForVapi()` in `worker2/Worker/Ai/Bdr/Vapi.php`) is the
|
|
208
|
+
source of truth for "why isn't my contact being called." Full rules are documented in
|
|
209
|
+
**`../../worker2/features/vapi-webhook-handler.md`**. Two consequences matter to the funnel:
|
|
210
|
+
|
|
211
|
+
- The website "call me now / scheduled" path (`dtNextContactRequested <= NOW()`) **bypasses**
|
|
212
|
+
call windows and weekly cadence limits, but still requires an active campaign **with an
|
|
213
|
+
assigned assistant**, `isActive=1`, `isOkayToCall=1`, a live **primary** phone, and
|
|
214
|
+
`attemptCount < maxAttemptsPerContact`.
|
|
215
|
+
- **BUG — a phone-less HubSpot contact can never receive a call (silent).** When the HubSpot
|
|
216
|
+
contact has no phone, BDR creates the Toga contact **without** a phone at page-prime; the
|
|
217
|
+
phone the visitor types arrives later via the Call Now `PUT`. api2 creates a
|
|
218
|
+
`ContactPhoneNumber` row on that update but does **not** set
|
|
219
|
+
`Contacts.primaryContactPhoneNumberId`. The dialer's eligibility SQL `INNER JOIN`s on
|
|
220
|
+
`primaryContactPhoneNumberId`, so the contact stays invisible to the dialer. **Re-submitting
|
|
221
|
+
does not fix it** (verified in production: the phone row exists but the primary pointer
|
|
222
|
+
stays null). The funnel still shows success and returns `200` — a silent failure. The old
|
|
223
|
+
`info` site never hit this because CRM-sourced contacts already had a phone at CREATE time
|
|
224
|
+
(api2 links the primary on create, not on a later update). Fix candidates: api2-side (set
|
|
225
|
+
the primary when a phone row is created during an update) or a BDR workaround (a follow-up
|
|
226
|
+
`PUT` linking the existing phone-row uuid as primary). Affects **any** phone-less HubSpot
|
|
227
|
+
contact and blocks live call testing.
|
|
228
|
+
|
|
229
|
+
## Local dev setup (BDR)
|
|
230
|
+
|
|
231
|
+
BDR local dev needs **5 server-side env vars** in `BDR/.env.local` (gitignored, never
|
|
232
|
+
committed): `HUBSPOT_ACCESS_TOKEN`, `TOGA_API_BASE_URL`, `TOGA_CLIENT_ID`,
|
|
233
|
+
`TOGA_CLIENT_API_UUID`, `TOGA_CLIENT_API_SECRET`. **Where the values live (not the values
|
|
234
|
+
themselves):** source them from the **`info` repo's `.env.local`**, which carries the
|
|
235
|
+
PRODUCTION credential set (`TOGA_API_BASE_URL = https://api.togahub.com/v2`).
|
|
236
|
+
|
|
237
|
+
- **`info/.env.beta` is a DEAD file** — empty URLs and obsolete var names (e.g.
|
|
238
|
+
`TOGA_CLIENT_SECRET` vs. the current `TOGA_CLIENT_API_UUID` / `TOGA_CLIENT_API_SECRET`
|
|
239
|
+
pair). Production is the only working credential set `info` has.
|
|
240
|
+
- **Consequence: local dev hits LIVE Toga** and can trigger **real calls** — treat local
|
|
241
|
+
submits accordingly.
|
|
242
|
+
- **Leave `NEXT_PUBLIC_GA_MEASUREMENT_ID` unset locally** so dev traffic does not pollute the
|
|
243
|
+
production GA4 property (`analytics.ts` no-ops without it).
|
|
244
|
+
- **Restart the dev server after creating `.env.local`** — Next reads env files only at
|
|
245
|
+
server startup. `.env.*` is gitignored in BDR.
|
|
246
|
+
|
|
159
247
|
## What is parked / not built
|
|
160
248
|
|
|
161
249
|
- **Creator screen + `TweaksPanel` — deliberately NOT ported** (parked per plan
|
|
@@ -170,6 +258,16 @@ to be near exact" vs. the mockup. All checks green: tsc, eslint, 79/79 Vitest, `
|
|
|
170
258
|
- **Remaining work is decision-gated, not code:** leadership's first-campaign copy;
|
|
171
259
|
Phase 8 Amplify SSR (incl. a **per-IP WAF rate rule** — see the enumeration gotcha) +
|
|
172
260
|
`togatech` integration. GA4 (Phase 7) is now wired; §9.10 and §9.11 are resolved.
|
|
261
|
+
- **Non-gating security residuals (recorded in `BDR/PLAN.md` §10 / Phase 8 checklist — do not
|
|
262
|
+
re-litigate; the `cso` review flagged these as follow-ups, not ship-blockers):** (a) **lock
|
|
263
|
+
the Amplify origin** so nothing reaches it without transiting CloudFront — any
|
|
264
|
+
direct-to-origin path lets an attacker own the whole XFF chain, the standing caveat behind
|
|
265
|
+
every header-based IP-trust decision above; (b) confirm the distribution **injects + strips**
|
|
266
|
+
`CloudFront-Viewer-Address` before setting `RATE_LIMIT_TRUST_CF_VIEWER_HEADER=true`;
|
|
267
|
+
(c) `readJsonObject` buffers the body **before** the size check and measures UTF-16 length,
|
|
268
|
+
not bytes (bounded in practice by the Lambda payload cap + the call limiter); (d) the proxy
|
|
269
|
+
matcher is an **exact-path allowlist** — re-verify no un-normalized path variant reaches a
|
|
270
|
+
handler off-matcher.
|
|
173
271
|
|
|
174
272
|
## Gotchas
|
|
175
273
|
|
|
@@ -179,13 +277,68 @@ to be near exact" vs. the mockup. All checks green: tsc, eslint, 79/79 Vitest, `
|
|
|
179
277
|
entry identifiers would ship to Google on every hit. Mitigation (in place): initialize
|
|
180
278
|
with `gtagOptions {send_page_view:false}` and pin a **query-scrubbed** `page_location`
|
|
181
279
|
via `ReactGA.set` before each pageview. Never send raw `location.href` to analytics here.
|
|
182
|
-
- **HIGH
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
**
|
|
188
|
-
|
|
280
|
+
- **HIGH (security) — `?hsContactId=` is an enumerable capability token (interim mitigation
|
|
281
|
+
in place, authoritative fix still deferred).** The entry page is public and HubSpot contact
|
|
282
|
+
ids are small integers; a bot walking ids can (a) scrape each contact's phone from the
|
|
283
|
+
server-rendered prefill, (b) scrape the **full** contact record via `GET /api/contact`
|
|
284
|
+
(see next gotcha), and (c) drive the page-prime Toga upsert, polluting contacts/attribution.
|
|
285
|
+
**Inherited from `info`'s CRM-link URL contract — cannot be changed unilaterally.**
|
|
286
|
+
**App-layer mitigation (BUILT + security-reviewed SHIP, 2026-07-16):** `src/proxy.ts` +
|
|
287
|
+
`src/lib/rateLimit.ts` + `src/lib/clientIp.ts` rate-limit per client (keyed off a **trusted**
|
|
288
|
+
client IP — see the two rate-limiter gotchas below). Entry pages carrying `?hsContactId=`
|
|
289
|
+
**soft-degrade** over the limit (strip `hsContactId`, `NextResponse.rewrite` to render
|
|
290
|
+
anonymously — a real shared-NAT user keeps the funnel minus prefill; a scraper gets no PII and
|
|
291
|
+
triggers no Toga upsert; `hsCampaignId` is preserved). `/api/contact`, `/api/call-now`,
|
|
292
|
+
`/api/call-later` return a `429 {isSuccess:false,…}` envelope with `Retry-After` over the
|
|
293
|
+
limit. **This is defense-in-depth ONLY** — the limiter is per-runtime-instance in-memory
|
|
294
|
+
(Amplify SSR may run several instances), so it is not authoritative. The authoritative
|
|
295
|
+
controls — a **per-IP edge WAF rate rule** (Phase 8) and **opaque/signed CRM link tokens**
|
|
296
|
+
(raise with the CRM owner) — remain deferred. Tracked in `BDR/PLAN.md` §10/§11. An independent
|
|
297
|
+
`cso` review of this mitigation returned a **final verdict of SHIP after two rounds**; the
|
|
298
|
+
hardening from that review (trusted-IP position, LRU-capped map, `hsCampaignId` validation) is
|
|
299
|
+
documented in the gotchas below.
|
|
300
|
+
- **HIGH (security) — `GET /api/contact?hsContactId=` returns the FULL contact record.** Name,
|
|
301
|
+
email, phone, company, and job title by id — a **more direct** PII scrape vector than the SSR
|
|
302
|
+
phone prefill. Now covered by the same rate limiter; **any change to
|
|
303
|
+
`src/app/api/contact/route.ts` must keep it rate-limited.**
|
|
304
|
+
- **HIGH (security) — rate limiting behind CloudFront MUST key off a TRUSTED IP position, never
|
|
305
|
+
the leftmost `X-Forwarded-For`.** The leftmost XFF entry is client-prependable, so keying the
|
|
306
|
+
limiter off it is trivially bypassable (a scraper spoofs a fresh left-most IP per request and
|
|
307
|
+
never hits the limit). Correct approach (in `src/lib/clientIp.ts`): parse XFF
|
|
308
|
+
**right-to-left** and take index `parts.length - 1 - RATE_LIMIT_TRUSTED_PROXY_HOPS` (default
|
|
309
|
+
`0` ⇒ the **rightmost**, CloudFront-appended, viewer IP). `x-real-ip` is **NOT** trusted.
|
|
310
|
+
`CloudFront-Viewer-Address` is trusted **only** when env
|
|
311
|
+
`RATE_LIMIT_TRUST_CF_VIEWER_HEADER=true` (default **OFF**) — because an unverified
|
|
312
|
+
distribution that forwards a client-supplied copy of that header reintroduces the bypass; ops
|
|
313
|
+
enables it only after confirming CloudFront both injects **and** strips it. `normalizeIp()`
|
|
314
|
+
handles port strip, `[ipv6]:port`, and `::ffff:` IPv4-mapped forms so one client = one key.
|
|
315
|
+
This whole approach is a **reusable pattern for any Next app behind CloudFront/Amplify** — do
|
|
316
|
+
not copy a naive leftmost-XFF limiter into another repo.
|
|
317
|
+
- **HIGH (security) — a naive in-memory limiter sweep is itself a DoS.** A sweep that only
|
|
318
|
+
reclaims fully-refilled buckets leaves an **unbounded map** plus an O(n) sweep per insert;
|
|
319
|
+
under a unique-key flood (e.g. rotated IPv6 sources) that is a self-inflicted DoS. Fix (in
|
|
320
|
+
`src/lib/rateLimit.ts`): a **HARD ceiling** `MAX_TRACKED_KEYS = 10_000` enforced by **O(1) LRU
|
|
321
|
+
eviction** (Map insertion order; every check re-inserts the key to move it to MRU), **plus**
|
|
322
|
+
the O(n) reclaim throttled to at most **once per window**. This guarantees the map never
|
|
323
|
+
exceeds the cap and per-insert work is O(1) amortized. Do not "simplify" the LRU re-insert or
|
|
324
|
+
the throttle away — both are load-bearing against the flood case.
|
|
325
|
+
- **`?hsCampaignId=` page-prime param must be UUID-validated before it crosses to the Toga
|
|
326
|
+
upsert.** It flows into the upsert as `togaCampaignUuid`; it previously had only a
|
|
327
|
+
`typeof === 'string'` guard. `page.tsx` now `isTogaUuid`-validates it before use, dropping a
|
|
328
|
+
malformed value to `undefined`. Never pass an unvalidated inbound query param into the Toga
|
|
329
|
+
upsert.
|
|
330
|
+
- **Next 16 — `middleware.ts` is DEPRECATED; use `proxy.ts`.** Next 16.2.3 warns "The
|
|
331
|
+
middleware file convention is deprecated. Please use proxy instead"
|
|
332
|
+
(nextjs.org/docs/messages/middleware-to-proxy). Correct convention: `src/proxy.ts` exporting
|
|
333
|
+
a function named `proxy` (or default) plus `export const config: ProxyConfig = { matcher:
|
|
334
|
+
[...] }`; import `NextRequest` / `NextResponse` / `ProxyConfig` from `"next/server"`;
|
|
335
|
+
`NextResponse.next()` / `.rewrite()` are unchanged. Build output labels it
|
|
336
|
+
`ƒ Proxy (Middleware)`. Write `proxy.ts` from the start on Next 16 — do not use `middleware.ts`.
|
|
337
|
+
- **Request-a-call is server-side — expect NO Toga call in the browser network tab.** A
|
|
338
|
+
`200 POST /api/call-now` is the success signal; the `POST /auth/api` + `PUT /contacts/{uuid}`
|
|
339
|
+
happen inside the route handler. See "Request-a-call is fully server-side" above. And note
|
|
340
|
+
setting `dtNextContactRequested` does not guarantee a call — the worker2 dialer eligibility
|
|
341
|
+
SQL decides (a phone-less contact is silently never dialed; see that section).
|
|
189
342
|
- **Standing rule: never add more PII fields to `ClientLeadRef`.** Phone is the one
|
|
190
343
|
deliberate exception (visitor's own number, shown back to them, never logged). Every field
|
|
191
344
|
on `ClientLeadRef` crosses to the public client and is exposed to the enumeration risk
|
|
@@ -218,6 +371,41 @@ to be near exact" vs. the mockup. All checks green: tsc, eslint, 79/79 Vitest, `
|
|
|
218
371
|
harness; pull it with `npx toga-ai`.
|
|
219
372
|
|
|
220
373
|
## Change history
|
|
374
|
+
- 2026-07-16 — Rate-limit mitigation HARDENED to ship state after an independent `cso` security
|
|
375
|
+
review returned **SHIP after two rounds** (supersedes the mid-state recorded in the entry
|
|
376
|
+
below: test count is **104**, not 89, and the review is complete, not "still running"). Added
|
|
377
|
+
`src/lib/clientIp.ts` (pure, `next`-free, testable) → the mitigation is now **3 source files**
|
|
378
|
+
(`proxy.ts` + `rateLimit.ts` + `clientIp.ts`). Security hardening: (1) rate limiting behind
|
|
379
|
+
CloudFront keys off a **trusted** XFF position (parse right-to-left, index
|
|
380
|
+
`len-1-RATE_LIMIT_TRUSTED_PROXY_HOPS`, default rightmost/viewer IP; `x-real-ip` untrusted;
|
|
381
|
+
`CloudFront-Viewer-Address` trusted only when `RATE_LIMIT_TRUST_CF_VIEWER_HEADER=true`,
|
|
382
|
+
default OFF) — never the client-prependable leftmost IP; `normalizeIp` collapses port/
|
|
383
|
+
`[ipv6]:port`/`::ffff:` so one client = one key; (2) the limiter map is bounded by a HARD
|
|
384
|
+
`MAX_TRACKED_KEYS=10_000` via O(1) LRU eviction + reclaim throttled to once/window, closing a
|
|
385
|
+
unique-key-flood DoS in the naive sweep; (3) `?hsCampaignId=` is now `isTogaUuid`-validated in
|
|
386
|
+
`page.tsx` before it crosses to the Toga upsert (was a bare `typeof`-string guard). Recorded
|
|
387
|
+
the non-gating `cso` follow-ups (lock Amplify origin to CloudFront; verify CF-Viewer-Address
|
|
388
|
+
inject+strip before trusting it; `readJsonObject` buffers-then-measures UTF-16 not bytes;
|
|
389
|
+
proxy matcher exact-path allowlist). All green: tsc, eslint, **104/104** Vitest, `next build`.
|
|
390
|
+
(tcox)
|
|
391
|
+
- 2026-07-16 — Later same-day session. BUILT an interim app-layer rate-limit mitigation for the
|
|
392
|
+
`?hsContactId=` enumeration risk: `src/lib/rateLimit.ts` (pure clock-injected in-memory
|
|
393
|
+
token-bucket, idle-key eviction, no `server-only`) + `src/proxy.ts` (Next 16 proxy). Policy:
|
|
394
|
+
entry pages soft-degrade (strip `hsContactId`, rewrite anonymous, keep `hsCampaignId`);
|
|
395
|
+
`/api/{contact,call-now,call-later}` return `429` over the limit. Defense-in-depth only
|
|
396
|
+
(per-instance memory; edge WAF + signed CRM tokens still deferred to Phase 8); a `cso` review
|
|
397
|
+
was still running at capture. 10 new tests (89 total). DISCOVERED + documented: (1) the
|
|
398
|
+
request-a-call flow is fully server-side (`POST /auth/api` + `PUT /contacts/{uuid}` happen in
|
|
399
|
+
the route handler; browser only sees `/api/*`, so "no Toga call in my console" is expected;
|
|
400
|
+
the `PUT` only flips `dtNextContactRequested`/`isOkayToCall`/`isActive`/`contactCallType`, the
|
|
401
|
+
worker2 dialer places the call); (2) BUG — a phone-less HubSpot contact is silently never
|
|
402
|
+
dialed because api2 doesn't set `primaryContactPhoneNumberId` on a later update and the dialer
|
|
403
|
+
INNER JOINs on it (re-submit does not fix; verified in prod); (3) `GET /api/contact` returns
|
|
404
|
+
the full contact record (name/email/phone/company/title) — a direct PII scrape vector, now
|
|
405
|
+
rate-limited; (4) Next 16 `middleware.ts` is deprecated in favor of `proxy.ts`; (5) local dev
|
|
406
|
+
env wiring (5 server vars in `.env.local` sourced from `info`'s PRODUCTION `.env.local`;
|
|
407
|
+
`.env.beta` is dead; local dev hits LIVE Toga; leave GA id unset; restart to pick up env).
|
|
408
|
+
Dialer eligibility rules recorded in `../../worker2/features/vapi-webhook-handler.md`. (tcox)
|
|
221
409
|
- 2026-07-16 — Phase 6 delta + §9.10/§9.11 resolutions. DISCOVERED the `info` URL/backend
|
|
222
410
|
contract was already fully built at commit `34a3910` (primeLead: `?hsContactId=` +
|
|
223
411
|
`?hsCampaignId=` → HubSpot `getContact` → Toga upsert + attribution → only `togaUuid` to
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [snaredla]
|
|
9
|
+
updated: 2026-07-16
|
|
10
|
+
owners: [snaredla, tcox]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Vapi.php
|
|
13
13
|
- worker2/Worker/Ai/Bdr/Vapi.php
|
|
@@ -125,6 +125,46 @@ Therefore an `end-of-call-report` can legitimately arrive **without** a `contact
|
|
|
125
125
|
job. `Contact Attempt not found` should be reserved for a report that *has* a
|
|
126
126
|
`contactAttemptUuid` but no matching row (a real data-integrity anomaly).
|
|
127
127
|
|
|
128
|
+
## Outbound dialer eligibility (`getEligibleContactsForVapi`)
|
|
129
|
+
|
|
130
|
+
The outbound dialer `_Worker_Ai_Bdr_Vapi` (`worker2/Worker/Ai/Bdr/Vapi.php`) is the source of
|
|
131
|
+
truth for **"why isn't my contact being called."** Confirmed against source
|
|
132
|
+
`getEligibleContactsForVapi()` (~L197–306). A contact is dialed **only when ALL** hold:
|
|
133
|
+
|
|
134
|
+
1. It is linked to an **active campaign that has an assigned assistant** —
|
|
135
|
+
`INNER JOIN CampaignAssistants ON Campaigns.campaignAssistantId`. A campaign with
|
|
136
|
+
`campaignAssistantId = null` is **never** dialed.
|
|
137
|
+
2. `Contacts.isActive = 1`.
|
|
138
|
+
3. `Contacts.isOkayToCall = 1`.
|
|
139
|
+
4. A non-empty **PRIMARY** phone — `INNER JOIN ContactPhoneNumbers ON
|
|
140
|
+
Contacts.primaryContactPhoneNumberId` (the primary pointer, **not** just any phone row).
|
|
141
|
+
5. `attemptCount < Campaigns.maxAttemptsPerContact`.
|
|
142
|
+
6. **EITHER** `Contacts.dtNextContactRequested <= NOW()` — this path **BYPASSES call windows
|
|
143
|
+
AND weekly cadence limits**; it is the website "call me now / scheduled" path — **OR**
|
|
144
|
+
(`dtNextContactRequested IS NULL` AND within `CampaignCallWindows` for the campaign timezone
|
|
145
|
+
AND not a `CampaignCallWindowException` date AND under
|
|
146
|
+
`CampaignCadences.maxCallsPerContactPerWeek`).
|
|
147
|
+
|
|
148
|
+
**Cron entry point:** `ProcessAllActiveCampaigns` (~L42) → `ProcessCampaignCalls` →
|
|
149
|
+
`getEligibleContactsForVapi` → `processOutboundCalls`.
|
|
150
|
+
|
|
151
|
+
**Single "Call Now" path:** `InitiateOutboundCall(contactPhoneNumberUuid)` (~L151–170) →
|
|
152
|
+
`initiateCallForContact(…, CALL_TYPE__IMMEDIATE)`. It errors **"not assigned to any active
|
|
153
|
+
campaign"** if no ACTIVE/PENDING campaign-contact link exists.
|
|
154
|
+
|
|
155
|
+
### Gotcha — a phone-less contact is silently never dialed (BDR web funnel)
|
|
156
|
+
|
|
157
|
+
Criterion 4's `INNER JOIN` on `primaryContactPhoneNumberId` is why a contact whose phone was
|
|
158
|
+
added **after** creation can be invisible to the dialer: **api2 creates the
|
|
159
|
+
`ContactPhoneNumber` row but does not set `Contacts.primaryContactPhoneNumberId` on an UPDATE**
|
|
160
|
+
(it only links the primary on CREATE). The BDR web funnel hits this for any phone-less HubSpot
|
|
161
|
+
contact — the visitor's typed phone arrives via a later `PUT /contacts/{uuid}`, so the row
|
|
162
|
+
exists but the primary pointer stays null and the eligibility join excludes it. Re-submitting
|
|
163
|
+
does not fix it (verified in production). See
|
|
164
|
+
`../../ai-bdr/features/web-funnel-app.md` for the funnel side. Fix candidates: api2 sets the
|
|
165
|
+
primary on phone-row create-during-update, or a caller-side `PUT` linking the existing
|
|
166
|
+
phone-row uuid as primary.
|
|
167
|
+
|
|
128
168
|
## Data model
|
|
129
169
|
|
|
130
170
|
Writes to `ContactAttempts` (via Toga 2.0 API) — `dtStarted`, `dtEnded`, `transcript`,
|
|
@@ -152,5 +192,11 @@ status. See the `ai-bdr` skill / `_Model_True_ContactAttempt` for the full field
|
|
|
152
192
|
- [VAPI Integration — assistants, tools, structured output](../../ai-bdr/features/vapi-integration.md)
|
|
153
193
|
|
|
154
194
|
## Change history
|
|
195
|
+
- 2026-07-16 — Documented the outbound dialer eligibility rules from source
|
|
196
|
+
`getEligibleContactsForVapi()` (the 6 ALL-of criteria, the `dtNextContactRequested <= NOW()`
|
|
197
|
+
window/cadence bypass = the website call-now/scheduled path, cron entry chain, and the single
|
|
198
|
+
"Call Now" `InitiateOutboundCall` path). Added the phone-less-contact gotcha: api2 does not
|
|
199
|
+
set `primaryContactPhoneNumberId` on an UPDATE, so the `INNER JOIN` on it silently excludes a
|
|
200
|
+
contact whose phone was added after create (blocks the BDR web funnel; verified in prod). (tcox)
|
|
155
201
|
- 2026-06-24 — Initial doc: webhook handler, contactAttemptUuid write-back, message-type
|
|
156
202
|
filtering bug, inbound/test vs outbound metadata. (snaredla)
|
package/package.json
CHANGED