toga-ai 1.0.232 → 1.0.233

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -12,7 +12,7 @@
 | [Elite Freshservice Sync (worker2)](features/elite-freshservice-sync.md) | `_Worker_Elite` processes Freshservice webhook events and syncs them into TOGA 2. | worker2/Worker/Elite.php, worker2/Config/dev-kmaramreddy-laptop.ini |
 | [Etilize Catalog Item Import & Refresh](features/etilize-catalog-item-import.md) | Client-generic catalog onboarding from an S3 CSV plus an Etilize re-pull. | worker2/Worker/Etilize/Items.php |
 | [Etilize Item Translation Import](features/etilize-item-translation-import.md) | The abstract worker class `_Worker_Etilize_ItemTranslations` imports **non-English** item text from Etilize into the client's `ItemTranslations` table. | worker2/Worker/Etilize/ItemTranslations.php |
-| [Monitoring Framework (Orchestrator + Child Monitors)](features/monitoring-framework.md) | A unified, DB-driven monitoring framework for business-critical data flows (Compass POs, Prudential asset imports, AIG closed claims, …). | worker2/Worker/Monitor.php, worker2/Worker/Monitors/, worker2/Worker/Notification/Email.php, dbchanges2/Core/2026-05-21 - Monitors.sql |
+| [Monitoring Framework (Orchestrator + Child Monitors)](features/monitoring-framework.md) | A unified, DB-driven monitoring framework for business-critical data flows (Compass POs, Prudential asset imports, AIG closed claims, …). | worker2/Worker/Monitor.php, worker2/Worker/Monitors/, worker2/Worker/Monitors/RateEntitlement.php, worker2/Worker/Notification/Email.php, worker2/Worker/Rate.php, dbchanges2/Core/2026-05-21 - Monitors.sql, dbchanges2/Core/2026-06-29a - Rate Entitlement Contract Monitor.sql |
 | [NetSuite → TOGA Opportunity Sync (API Message Queue + worker2 webhook)](features/netsuite-opportunity-sync.md) | Outbound sync from NetSuite to TOGA for the record types the Forecast2 importer pulls (opportunities first; sales/items/etc. | worker2/Worker/Netsuite.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Controller/Index.php, _underscore/Worker.php, test/@dave/NetSuite/api-message-queue/lib_amq_queue.js, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/ue_amq_drain.js, test/@dave/NetSuite/api-message-queue/ss_amq_drain.js, test/@dave/NetSuite/api-message-queue/DEPLOY_RUNBOOK.md, test/@dave/clickup/backfill_opportunity_numbers.php, test/@dave/clickup/probe_opportunity_fields.php, test/@dave/probe_clickup_desc_match.php, test/@dave/test_model_load_behavior.php, dbchanges2/Forecast/2026-06-25a - Add unique index on Opportunities netsuiteOpportunityInternalId.sql, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
 | [NetSuite → Forecast Open-Orders Sync (salesOrder webhook → OpenOrderItems)](features/netsuite-salesorder-open-orders-sync.md) | Webhook-driven, single-record port of the legacy open-orders importer (TRUE-79142). | worker2/Worker/Netsuite/SalesOrder.php, worker2/Worker/Netsuite.php, test/@dave/probe_salesorder_rest_shape.php, test/@dave/probe_open_order_lines.php, test/@dave/check_so_status.php, test/@dave/check_so_history.php, test/@dave/probe_so_rest_lines.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_open_order_gating.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
 | [DB-Driven Notification (Internal) Email](features/notification-email.md) | Internal/notification emails (merge-conflict alerts, ops notices — anything system-generated, not client-facing transactional mail) are sent through one worker | worker2/Worker/Notification/Email.php, _underscore/Model/Client/EmailTemplate.php, dbchanges2/Client/2026-06-23a - EmailTemplateWrapper.sql, dbchanges2/Client_True/2026-06-23a - EmailTemplateWrapper.sql |

package/knowledge/2.0/apps/worker2/features/monitoring-framework.md

@@ -6,17 +6,21 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-06-10
+updated: 2026-06-29
 owners: [mhammontree]
 files:
   - worker2/Worker/Monitor.php
   - worker2/Worker/Monitors/
+  - worker2/Worker/Monitors/RateEntitlement.php
   - worker2/Worker/Notification/Email.php
+  - worker2/Worker/Rate.php
   - dbchanges2/Core/2026-05-21 - Monitors.sql
+  - dbchanges2/Core/2026-06-29a - Rate Entitlement Contract Monitor.sql
 related:
   - ../architecture.md
   - ./creating-worker-actions.md
   - ../../dbchanges2/architecture.md
+  - ../../../clients/rate/features/aig-contract-creation.md
 ---
 
 ## Summary
@@ -26,10 +30,11 @@ Prudential asset imports, AIG closed claims, …). One orchestrator applies cons
 anti-flap logic and notification rules to any registered "is X healthy?" check. Replaces
 scattered/ad-hoc monitoring where failures surfaced only when a client complained.
 
-**Status (2026-06-10):** v1.0 framework + orchestrator landed; **first child monitor
-pending**. Migration applied to local Core_2 only — **not yet on staging/production Core**
-(coordinate before merge). Dashboard/acknowledgment layer under discussion (see Pending
-scope).
+**Status (2026-06-29):** v1.0 framework + orchestrator landed; **first child monitor built**
+— `_Worker_Monitors_RateEntitlement` (TRUE-79129, see [worked example](#worked-example--rateentitlement-the-first-child-monitor)).
+`Core.Monitors` migration applied to local Core_2 only — **still not on staging/production
+Core** (coordinate before merge; the new Rate change-set hard-fails where the table is
+absent). Dashboard/acknowledgment layer under discussion (see Pending scope).
 
 Design principles: async isolation (one cron per monitor — a slow monitor can't block
 others) · anti-flap on the recovery side only (N consecutive OKs before declaring
@@ -131,6 +136,19 @@ Index `Monitors_isActive_IDX (isActive)`. `Core.CronJobs` reused as-is — one r
   (recipients see it verbatim); register any non-Core DB connection inline at the top of
   `Run()` — the framework does **not** call `initialize()` on child monitors.
 
+**Confirmed against the orchestrator (`Worker/Monitor.php`):**
+- The orchestrator calls `$phpClass::Run()` with **no args** and does **not** call
+  `initialize()` on children. A child needing a non-Core connection (e.g. a `Logs_<Client>`
+  DB) must self-register it inside `Run()`, guarding against a double-register:
+  `if (!isset(_Database::$_registers[$alias])) { _Database::register(db, host, user, pass,
+  null, null, $alias); }`, with credentials from `_Config::databaseLogs(...)`. The
+  registration pattern mirrors `worker2/Worker/Rate.php::initialize()`.
+- The orchestrator wraps the call in its own try/catch and turns any `Throwable` into an
+  alert — children must **not** catch (re-confirms the "fail loudly" rule above).
+- A read-only `Run()` (e.g. a log scan) needs **no** `transactionCommit()`.
+- **PHP heredoc gotcha:** `self::CONST` does **not** interpolate inside a heredoc. Bind
+  class constants to local variables before building a heredoc SQL string.
+
 ### Adding a new monitor (runbook)
 
 1. Write the child class (one check, returns `{isOk, message}`).
@@ -150,17 +168,59 @@ No Lambda, SQS, or orchestrator changes needed per monitor.
 - Pause a monitor: `isActive = 0` (cron row can stay active). Pause email only: `peopleToNotify = JSON_ARRAY()`.
 - Manual state reset: clear `state`/`consecutiveOkCount`/`lastNotificationDt`/`lastNotificationType` — sparingly; the state machine self-corrects.
 
+### Worked example — RateEntitlement (the first child monitor)
+
+`_Worker_Monitors_RateEntitlement` (`worker2/Worker/Monitors/RateEntitlement.php`, the
+**first** file in `Worker/Monitors/`) watches Rate's AIG warranty-contract creation
+(TRUE-79129). It is the reference implementation for the **log-scan** monitor pattern:
+
+1. `Run()` self-registers the `Logs_Rate` connection (the guard + `_Config::databaseLogs`
+   pattern above), because the orchestrator never calls `initialize()`.
+2. Scans `Logs_Rate.Api` for **failed outbound** AIG contract POSTs in the last 30 minutes
+   — `direction = 'OUT'`, `method = 'POST'`, `route LIKE '%/contract'`, and
+   `responseCode` non-2xx **or NULL**. Returns `(object){isOk, message}`; read-only, so no
+   commit.
+3. The detection signal is the `_underscore` **outbound API log**, not the entitlement
+   itself — see [why a monitor is needed](#why-an-external-monitor) below.
+
+The change-set `dbchanges2/Core/2026-06-29a - Rate Entitlement Contract Monitor.sql` does
+the registration: one `Core.Monitors` row (`phpClass = _Worker_Monitors_RateEntitlement`,
+`peopleToNotify = ['devteam@goagilant.com']`, `requiredConsecutiveOks = 2`,
+`reminderFrequencyMinutes = 60`) + one `Core.CronJobs` row (`action = 'Monitor/Run'`,
+`parameters = JSON_OBJECT('monitorId', LAST_INSERT_ID())`, schedule `*/15`,
+`maxExecutionTime = 60`). LAST_INSERT_ID() chains the cron row to the just-inserted monitor.
+
+### Detection via the _underscore outbound API log
+
+`_underscore/ApiRequest.php` logs **every** outbound call to `_Model_Client_Logs_Api`
+(table `Logs_<Client>.Api`) and **commits it immediately**, independent of whether the
+caller later swallows the exception — which is exactly what makes log-scan monitoring
+possible for silent-failure interceptors. Columns to scan: `dtStamp`
+(`FIELD_DATETIME_CREATED`), `direction` (`OUT`), `method`, `hostname` (scheme://host),
+`route` (path after host), `responseCode`. `setUrl()` splits a full URL into
+`hostname` + `route`, so a call to `.../contract` is logged with `route` ending `/contract`
+(and cancellation as `/contract/cancel`). **Pattern for new monitors:** scan the client's
+`Logs_<Client>.Api` rather than trying to detect failure on the business object.
+
+### Why an external monitor
+
+Some interceptors **silently swallow** their own failures (no persistent flag on the
+business record), so the only durable failure signal is the committed outbound API log.
+The Rate case is documented in
+[Rate AIG contract creation](../../../clients/rate/features/aig-contract-creation.md).
+
 ## Client variations
 
 None — the framework is shared Core infrastructure. Individual monitors target specific
-clients' data flows (Compass, Prudential, AIG, …) but live as separate child classes.
+clients' data flows (Compass, Prudential, AIG, Rate, …) but live as separate child classes.
 
 ## Gotchas / known issues
 
-- **First child monitor not yet built** — `worker2/Worker/Monitors/` does not exist yet
-  (as of 2026-06-10).
-- **Migration not applied to staging/production Core** — only local Core_2. Coordinate
-  before merge.
+- **Migration not applied to staging/production Core** — `Core.Monitors`
+  (`2026-05-21 - Monitors.sql`) is on local Core_2 only (as of 2026-06-10, re-confirmed
+  2026-06-29). The new `2026-06-29a` Rate change-set will **hard-fail (table not found)** on
+  any environment where `2026-05-21 - Monitors.sql` has not yet been promoted — confirm the
+  Monitors table exists before deploying.
 - `worker2/MONITORING_PLAN.md` is referenced by the design doc but **missing on disk** —
   stale reference to resolve.
 - Child return value must be an **object** with both `isOk` and `message`; a bare array
@@ -177,6 +237,7 @@ clients' data flows (Compass, Prudential, AIG, …) but live as separate child c
   HTML email + dashboard deep-links · anti-flap on the alarm side.
 
 ## Change history
+- 2026-06-29 — Built the first child monitor, `_Worker_Monitors_RateEntitlement` (TRUE-79129) + change-set `2026-06-29a`; added the log-scan worked example, the `_underscore` outbound-API-log detection pattern, and confirmed child details (orchestrator never calls `initialize()` so children self-register non-Core connections; no commit on read-only `Run()`; heredoc cannot interpolate `self::CONST`). Re-flagged that `Core.Monitors` is still local-only — the new change-set hard-fails where the table is absent. (mhammontree)
 - 2026-06-10 — Documented the v1.0 monitoring framework (orchestrator, `Core.Monitors` table, recovery-side anti-flap state machine, child contract). First child monitor + staging/prod migration still pending. (mhammontree)
 
 ## Related docs

package/knowledge/INDEX.md

@@ -16,7 +16,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 17 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 18 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 22 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 7 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)

package/knowledge/clients/rate/INDEX.md

@@ -2,6 +2,7 @@
 
 | Doc | Framework | Summary | Files |
 |-----|-----------|---------|-------|
+| [Rate AIG Warranty Contract Creation (silent-failure interceptor)](features/aig-contract-creation.md) | 2.0 | When a Rate entitlement is created, `_Model_Rate_Entitlement::postPost` (`_underscore/Model/Rate/Entitlement.php`) creates an **AIG warranty contract** as a non | _underscore/Model/Rate/Entitlement.php, _underscore/ApiRequest.php, worker2/Worker/Monitors/RateEntitlement.php |
 | [Rate Monthly Reconciliation Report](features/monthly-reconciliation-report.md) | 1.0 | A monthly cron that emails an Excel reconciliation report covering all Rate subscription sales orders and their linked PayPal payments for the prior calendar mo | worker/crons/notifications/reports/rate/send_monthly_rate_purchases_report.php, worker/schedules/cron.worker.notification.json |
 | [Rate SalesOrder → NetSuite CashSale Export (postPost)](features/netsuite-cashsale-export.md) | 2.0 | Rate sells home-warranty / home-tech-support products. | _underscore/Model/Rate/SalesOrder.php, _underscore/Model/Rate/Item.php |
 | [Rate SAML SSO](features/saml-sso.md) | 2.0 | Rate uses Azure AD as its IdP (`login.rate.com`). | _underscore/Model/Rate/ClientAuthentication.php, saml/Controller/Index.php, toga2-view/src/hooks/useAuthenticationFlow.ts |

package/knowledge/clients/rate/features/aig-contract-creation.md

@@ -0,0 +1,97 @@
+---
+title: "Rate AIG Warranty Contract Creation (silent-failure interceptor)"
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: rate
+type: client-feature
+status: active
+updated: 2026-06-29
+owners: [mhammontree]
+files:
+  - _underscore/Model/Rate/Entitlement.php
+  - _underscore/ApiRequest.php
+  - worker2/Worker/Monitors/RateEntitlement.php
+related:
+  - clients/rate/profile.md
+  - ../../../2.0/apps/worker2/features/monitoring-framework.md
+---
+
+## Summary
+
+When a Rate entitlement is created, `_Model_Rate_Entitlement::postPost`
+(`_underscore/Model/Rate/Entitlement.php`) creates an **AIG warranty contract** as a
+non-critical side effect: it authenticates against AIG, POSTs to `.../contract`, and on
+success stamps `payload->c_aigContractNumber` and `payload->c_aigContractId` back onto the
+entitlement. **It only attempts this when the entitlement payload has a primary contact
+address.**
+
+The catch: **every failure path is silent.** There is no persistent failure flag on the
+entitlement, so a contract that never got created looks identical to one that did until a
+customer or AIG reports it missing. That is why this flow is watched externally by the
+`_Worker_Monitors_RateEntitlement` monitor rather than by any state on the entitlement.
+
+This is a base-`_underscore` interceptor specific to Rate's product (it lives under
+`Model/Rate/`); documented here as a Rate client-feature because the behavior and its
+monitoring are Rate-scoped.
+
+## How it works
+
+1. `postPost` runs after an entitlement is saved. It proceeds **only if** the payload has a
+   primary contact address; otherwise it does nothing.
+2. Authenticate against AIG (POST `/authentication/login`) → obtain an access token.
+3. POST the contract payload to `.../contract`.
+4. On success: read `aigContractNumber` from the response and set
+   `payload->c_aigContractNumber` + `payload->c_aigContractId`.
+
+## Silent-failure paths (the gotcha)
+
+All of the following fail **silently** — no exception surfaces to the caller and nothing is
+persisted on the entitlement:
+
+- A `try/catch` that only `error_log()`s with the note *"AIG contract creation is
+  non-critical"* and swallows the exception.
+- **Auth response non-2xx** (no throw).
+- **No access token** returned (no throw).
+- **Contract POST non-2xx** (no throw).
+- **Success response but missing `aigContractNumber`** (no throw).
+
+Net effect: there is no durable signal on the entitlement that contract creation failed.
+
+## Detection / monitoring
+
+Because the interceptor swallows failures, detection rides the **committed outbound API
+log** instead. `_underscore/ApiRequest.php` logs every outbound call to
+`_Model_Client_Logs_Api` (table `Logs_Rate.Api`) and commits it immediately, regardless of
+whether the caller later swallows the exception. The AIG contract call lands with
+`direction = OUT`, `method = POST`, and `route LIKE '%/contract'` (cancellation is
+`/contract/cancel`).
+
+`_Worker_Monitors_RateEntitlement` (worker2) scans `Logs_Rate.Api` for failed AIG contract
+POSTs (`route LIKE '%/contract'`, `responseCode` non-2xx **or NULL**) in the last 30
+minutes and alerts the dev team. See the
+[Monitoring Framework worked example](../../../2.0/apps/worker2/features/monitoring-framework.md#worked-example--rateentitlement-the-first-child-monitor).
+
+### Known detection limitation (decided)
+
+Detection matches **only** `POST .../contract` (`route LIKE '%/contract'`), which
+deliberately **excludes** `/contract/cancel` and the shared `/authentication/login`.
+Consequence: a creation failure caused by an **auth failure** is **not** caught, because
+`/authentication/login` is shared with the cancellation flow and so cannot be attributed to
+creation vs. cancellation. This is an accepted, documented limitation — not an oversight.
+
+## Gotchas / known issues
+
+- **No persistent failure flag** — the only durable evidence of a failed contract creation
+  is the outbound API log row, not the entitlement.
+- **Address-gated** — `postPost` does nothing when the payload lacks a primary contact
+  address, so an entitlement with no address never even attempts contract creation (and
+  never logs a `/contract` call).
+- **Auth-induced failures are blind to the monitor** — see the detection limitation above.
+
+## Change history
+
+- 2026-06-29 — Documented the silent-failure AIG warranty-contract interceptor and the
+  external log-scan monitor (`_Worker_Monitors_RateEntitlement`, TRUE-79129), including the
+  accepted detection limitation that auth-induced failures are not attributable (shared
+  `/authentication/login`). (mhammontree)

package/knowledge/clients/rate/profile.md

@@ -6,17 +6,21 @@ apps:
   - saml
   - toga2-view
   - worker
+  - worker2
+  - dbchanges2
 project: SAML SSO Gateway
 client: rate
 type: profile
 status: active
-updated: 2026-06-25
-owners: ["rgirish", "bala"]
+updated: 2026-06-29
+owners: ["rgirish", "bala", "mhammontree"]
 files: []
 related:
   - clients/rate/features/saml-sso.md
   - clients/rate/features/monthly-reconciliation-report.md
   - clients/rate/features/netsuite-cashsale-export.md
+  - clients/rate/features/service-card-entitlements.md
+  - clients/rate/features/aig-contract-creation.md
 ---
 
 ## Summary

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.232",
+  "version": "1.0.233",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",