toga-ai 1.0.110 → 1.0.111

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -7,5 +7,5 @@
 | [Creating Worker Actions](features/creating-worker-actions.md) | How to add a new callable Worker action — a PHP class whose `public static` methods are invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). | worker2/Worker/, worker2/Controller/Index.php, _underscore/Worker.php |
 | [Elite Freshservice Sync (worker2)](features/elite-freshservice-sync.md) | `_Worker_Elite` processes Freshservice webhook events and syncs them into TOGA 2. | worker2/Worker/Elite.php, worker2/Config/dev-kmaramreddy-laptop.ini |
 | [Monitoring Framework (Orchestrator + Child Monitors)](features/monitoring-framework.md) | A unified, DB-driven monitoring framework for business-critical data flows (Compass POs, Prudential asset imports, AIG closed claims, …). | worker2/Worker/Monitor.php, worker2/Worker/Monitors/, worker2/Worker/Notification/Email.php, dbchanges2/Core/2026-05-21 - Monitors.sql |
-| [NetSuite → TOGA Opportunity Sync (API Message Queue + worker2 webhook)](features/netsuite-opportunity-sync.md) | Outbound sync from NetSuite to TOGA for the record types the Forecast2 importer pulls (opportunities first; sales/items/etc. | worker2/Worker/Netsuite.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Controller/Index.php, _underscore/Worker.php, test/@dave/NetSuite/api-message-queue/lib_amq_queue.js, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/ue_amq_drain.js, test/@dave/NetSuite/api-message-queue/ss_amq_drain.js, test/@dave/NetSuite/api-message-queue/DEPLOY_RUNBOOK.md, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
+| [NetSuite → TOGA Opportunity Sync (API Message Queue + worker2 webhook)](features/netsuite-opportunity-sync.md) | Outbound sync from NetSuite to TOGA for the record types the Forecast2 importer pulls (opportunities first; sales/items/etc. | worker2/Worker/Netsuite.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Controller/Index.php, _underscore/Worker.php, test/@dave/NetSuite/api-message-queue/lib_amq_queue.js, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/ue_amq_drain.js, test/@dave/NetSuite/api-message-queue/ss_amq_drain.js, test/@dave/NetSuite/api-message-queue/DEPLOY_RUNBOOK.md, test/@dave/clickup/backfill_opportunity_numbers.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
 | [Teams Meeting Transcript Export](features/teams-transcript-export.md) | `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for Teams meeting transcripts produced by a set of organizers, classifies ea | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini |

package/knowledge/2.0/apps/worker2/features/netsuite-opportunity-sync.md

@@ -18,6 +18,7 @@ files:
   - test/@dave/NetSuite/api-message-queue/ue_amq_drain.js
   - test/@dave/NetSuite/api-message-queue/ss_amq_drain.js
   - test/@dave/NetSuite/api-message-queue/DEPLOY_RUNBOOK.md
+  - test/@dave/clickup/backfill_opportunity_numbers.php
   - worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php
 related:
   - ../architecture.md
@@ -80,19 +81,30 @@ second, independently-gated concern in the same handler.
 
 ## ClickUp opportunity task
 
-Created by `maybeCreateClickupTask()` → `createTask()` (ported verbatim from the legacy
-`webhook/` handler), gated on a **Presales Lead** (`custbody_ctc_solution_architect` employee →
-email; empty email = skip). Posts to list `901111987449` with `custom_item_id` = `1009`.
+`maybeCreateClickupTask()` runs on both POST and PUT, gated on a **Presales Lead**
+(`custbody_ctc_solution_architect` employee → email; empty email = skip). It is **find-or-update**
+(dedup, since 2026-06-16):
+
+1. `findClickupTaskByOpportunityNumber($tranId)` — GET the list filtered by the `Opportunity #`
+   custom field (`include_closed=true&include_archived=true&custom_fields=[{field_id,operator:'=',value}]`).
+2. **Match → `updateTask()`**: PUT name/description, then POST each custom field individually to
+   `/task/{id}/field/{fieldId}` (ClickUp has no bulk custom-field set on an existing task).
+3. **No match → `createTask()`**: POST to list `901111987449`, `custom_item_id` = `1009`.
+
+Shared helpers: `buildCustomFields()` (the field array, used by both create and update),
+`resolvePresalesUserId()` (cached `Client_True.Users.c_clickupUserIdentifier` lookup → else a
+`GET /team` roster scan that back-fills the cache via `cacheClickupUser()`).
 
 - **Task name** = `{opportunityNumber} — {customer} — {title}`, e.g.
   `60533 — 560 Endeavor Health — Data Center Facility Consolidation Services` (empty parts dropped).
 - **Custom-field value mapping** (the non-obvious part — these were swapped before 2026-06-16):
   `Opportunity #` (`a5529cdc-…`) ← `tranId`; `Customer #` (`170dc118-…`) ← `entity->refName`
   (the customer **name**, deliberately — not the NetSuite customer number). Plus `Sales Rep`,
-  `Presales Lead Email`, and `Presales Lead` (users field, resolved via a cached
-  `Client_True.Users.c_clickupUserIdentifier` lookup, falling back to a `GET /team` scan).
+  `Presales Lead Email`, and `Presales Lead` (users field).
 - The ClickUp token comes from `[clickup] token` in the worker2 config (legacy webhook received
   it on the payload; the new contract drops it).
+- Every ClickUp `_ApiRequest` calls `setLogging(self::CLICKUP_API_LOGGING_ENABLED=false)` — the
+  `_ApiRequest` Logs-DB default otherwise surfaces a misleading "connection refused" on dev laptops.
 
 ## Data model
 
@@ -158,13 +170,29 @@ None — platform-wide Forecast sync.
   (`https://webhook.togahub.com/netsuite`) with `debugWrap` off. Also flip the enqueuer deployment
   `Testing → Released` (Testing fires only for the deploying user). Full checklist in
   `DEPLOY_RUNBOOK.md` §6.
-- **No ClickUp dedup (deferred).** `maybeCreateClickupTask()` only gates on the Presales Lead — it
-  does **not** check whether a task already exists. Since both POST and PUT now create, **every edit
-  of an opportunity with a Presales Lead creates a NEW ClickUp task** (and the NetSuite-side
-  enqueuer fires on every save). Before this hits production traffic, add dedup — recommended:
-  store the created ClickUp task id on the `Forecast.Opportunities` row and skip when present.
-  Until 2026-06-16 PUT skipped ClickUp entirely, which masked this (an edit produced no duplicate
-  only because the edit path didn't create at all).
+- **ClickUp dedup keys on the `Opportunity #` field, not a DB column.** We deliberately did NOT add
+  a `clickupTaskId` column — the dedup lookup queries ClickUp itself by `Opportunity #` (= tranId).
+  Consequences to know:
+  - **Legacy tasks had `Opportunity #` empty** (the old `webhook/` handler never set it), so they
+    can't be matched until backfilled — see the backfill tool below. Without backfill, the first
+    edit of a pre-existing opp creates a *second* task.
+  - **The find-then-create is not atomic.** Two near-simultaneous events for one opp (e.g. the
+    NetSuite hourly `OPP` fallback firing alongside a real edit) can both miss and both create. A
+    committed local task id would close most of that window; the field-lookup approach can't.
+    ClickUp custom fields aren't unique-constrained, so some race exists regardless.
+  - **Update is per-field** (`/task/{id}/field/{id}` — no bulk set), so a mid-loop failure leaves a
+    partially-updated task; the thrown error names how far it got. The Presales Lead users field is
+    **add-only** on update (we don't read the prior assignee to remove it).
+  - **Null guard:** the Presales Lead users field is omitted when the email has no matching ClickUp
+    user — a `null` in the `add` array is a 400 from ClickUp.
+- **Backfill legacy `Opportunity #` with `test/@dave/clickup/backfill_opportunity_numbers.php`.**
+  Parses the opportunity number from the task title (the leading integer before the separator) and
+  writes it to the `Opportunity #` field, **never overwriting** a set value. Two legacy title
+  formats exist and both must be handled: em-dash (`74253 — 5780 Cdw — …`) **and** plain hyphen
+  (`70539 - 450 Living Well - …`). Safe by default: dry-run unless `--commit`, one task unless
+  `--all`; `--task=<id>` targets one. Token from `CLICKUP_API_KEY`. Run 2026-06-16: 146/148 tasks
+  backfilled; the 2 skipped are genuine test tasks (`Test Opportunity 2`, `TEST TEST 71437 …` —
+  number not leading).
 - **NetSuite doubled-prefix ids.** The queue record and its fields were created with the full
   custom prefix typed into the ID box, so the real ids are doubled:
   type `customrecordcustomrecord_api_msg_queue`, fields `custrecordcustrecord_amq_*`. The
@@ -178,6 +206,12 @@ None — platform-wide Forecast sync.
   (`Created`,`Pending`,`Retry`,`Sending`) — omitting `Created` silently matches nothing (`candidates:0`).
 
 ## Change history
+- 2026-06-16 — ClickUp dedup implemented (find-or-update by `Opportunity #`, no DB column): edits
+  now update the existing task instead of duplicating. Added `findClickupTaskByOpportunityNumber()`,
+  `updateTask()`, and shared `buildCustomFields()`/`resolvePresalesUserId()`/`cacheClickupUser()`
+  helpers; `setLogging(false)` on all ClickUp calls; null-user guard on the users field;
+  `include_archived=true` in the lookup. Built `backfill_opportunity_numbers.php` and backfilled
+  146/148 legacy tasks (two title formats: em-dash + hyphen). (dfranks)
 - 2026-06-16 — ClickUp parity hardening (TRUE-79182): verified the `Opportunity #`/`Customer #`
   custom fields exist on list 901111987449 and **fixed their swapped values**; `PUT` now also
   creates the ClickUp task (was POST-only); task name reformatted to

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.110",
+  "version": "1.0.111",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",