toga-ai 1.0.312 → 1.0.313
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -11,5 +11,6 @@
|
|
|
11
11
|
| [1.0 MVC Page Pattern & New-App Skeleton](features/mvc-page-pattern-and-app-skeleton.md) | This is the **reusable recipe for standing up a new 1.0 (`App_`) application** and for adding pages to one — the folder-based MVC routing, the page lifecycle, t | library/app/framework.php, library/app/frameworkindex.php, library/app/mvc.php, library/app/database.php, library/app/model.php, library/app/config.php |
|
|
12
12
|
| [NetSuite SuiteQL/REST API Reference](features/netsuite-suiteql-api-reference.md) | General working reference for the Agilant NetSuite integration: how to authenticate, how SuiteQL behaves, and the confirmed schema of the tables/columns/codes w | library/app/api/netsuite/rest.php, library/ssl/netsuite_ec_key.pem, test/@dave/Junk Drawer/nsq.php |
|
|
13
13
|
| [NetSuite SuiteQL/REST Shim — Field Semantics](features/netsuite-suiteql-rest-shim.md) | `App_Api_Netsuite_Rest` is the REST/SuiteQL replacement for the deprecated NetSuite SOAP toolkit. | library/app/api/netsuite/rest.php |
|
|
14
|
+
| [NetSuite Sync Alert Monitor (App_SystemMonitor_NetSuiteIntegration)](features/netsuite-sync-alert-monitor.md) | `App_SystemMonitor_NetSuiteIntegration` (`library/app/systemmonitor/netsuiteintegration.php`, title **"NetSuite Sync Alert"**) is a 1.0 system monitor that watc | library/app/systemmonitor/netsuiteintegration.php, worker/crons/infrastructure/system_monitors.php |
|
|
14
15
|
| [Startech PC Matic B2B Sync (library)](features/startech-pcmaticb2b-sync.md) | `library/app/api/toga2.php` handles bidirectional ticket sync for PC Matic B2B between TOGaDesk 1.0 and TOGA 2.0. | library/app/api/toga2.php, library/app/api/startechticket.php, worker/crons/toga2/startech/common_import_supporting_records.php |
|
|
15
16
|
| [App_Api_Toga2 — TOGa2 API Client & 1.0↔2.0 Sync Bridge](features/toga2-api-client-and-bridge.md) | `App_Api_Toga2` (`library/app/api/toga2.php`, ~8400 lines) is the **1.0-side client for the TOGa 2 (`_underscore`/api2) public API** *and* the home of the cross | library/app/api/toga2.php, worker/crons/toga2/aig/sync_togasupply_aig.php, worker/crons/toga2/wje/sync_togasupply_wje.php |
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: NetSuite Sync Alert Monitor (App_SystemMonitor_NetSuiteIntegration)
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: library
|
|
5
|
+
project: Library
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-07-10
|
|
10
|
+
owners: ["bala"]
|
|
11
|
+
files:
|
|
12
|
+
- library/app/systemmonitor/netsuiteintegration.php
|
|
13
|
+
- worker/crons/infrastructure/system_monitors.php
|
|
14
|
+
related:
|
|
15
|
+
- ../../worker/features/netsuite-togasupply-per-client-sync.md
|
|
16
|
+
- cron-execution-monitoring.md
|
|
17
|
+
- toga2-api-client-and-bridge.md
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Summary
|
|
21
|
+
|
|
22
|
+
`App_SystemMonitor_NetSuiteIntegration` (`library/app/systemmonitor/netsuiteintegration.php`,
|
|
23
|
+
title **"NetSuite Sync Alert"**) is a 1.0 system monitor that watches every TOGa Supply
|
|
24
|
+
client's NetSuite sync checkpoints and opens/updates a ClickUp task when any checkpoint goes
|
|
25
|
+
stale. It reads each client's six `NETSUITE_LAST_SYNC_DATETIME_*` parameters over the TOGa2
|
|
26
|
+
REST API and flags any older than **48h** (`STALE_THRESHOLD_SECONDS`). It is the alerting
|
|
27
|
+
surface sitting on top of the [NetSuite → TOGa Supply per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md);
|
|
28
|
+
that worker job does the syncing, this library class notices when it stops.
|
|
29
|
+
|
|
30
|
+
## Key files / entry points
|
|
31
|
+
|
|
32
|
+
- **`library/app/systemmonitor/netsuiteintegration.php`** — the monitor class. Extends
|
|
33
|
+
`App_SystemMonitor`. Holds the hardcoded per-client allowlist (`CLIENT_CONFIGURATIONS`),
|
|
34
|
+
the checkpoint key list (`NETSUITE_SYNC_PARAMETER_KEYS`), the 48h threshold, and all
|
|
35
|
+
ClickUp task create/update logic.
|
|
36
|
+
- **`worker/crons/infrastructure/system_monitors.php`** — the shared dispatcher (runs once a
|
|
37
|
+
minute). It instantiates every active monitor and calls `getStatus()`. This monitor is
|
|
38
|
+
registered there as `new App_SystemMonitor_NetsuiteIntegration`.
|
|
39
|
+
|
|
40
|
+
## How it works
|
|
41
|
+
|
|
42
|
+
- **Schedule.** `minuteFrequency = 60`, `minHourToRun = 10`, `maxHourToRun = 17` — so its
|
|
43
|
+
only tick is the **top of each hour**, 10:00–17:00. There is exactly one run per hour; if
|
|
44
|
+
that run dies, the monitor produces nothing until the next hour.
|
|
45
|
+
- **`getStatus()`** iterates `CLIENT_CONFIGURATIONS`, calls `getClientStatus()` per client,
|
|
46
|
+
and returns OK only if **every** client is OK (messages joined with `<br>`).
|
|
47
|
+
- **`getClientStatus()`** reads all six `NETSUITE_SYNC_PARAMETER_KEYS` for the client in a
|
|
48
|
+
**single** TOGa2 `GET /parameters` call using the options `where..in` DSL
|
|
49
|
+
(`['where' => ['AND' => [['key' => ['in' => KEYS]]]]]`), then compares each `value`
|
|
50
|
+
(`strtotime`) against `time() - STALE_THRESHOLD_SECONDS`. Any checkpoint older than 48h is
|
|
51
|
+
collected as stale.
|
|
52
|
+
- **`extractParameterRows()`** normalizes the api2 response: a single match comes back as an
|
|
53
|
+
**object**, many matches as an **array** — this helper always returns an array to iterate
|
|
54
|
+
(guarding on `isSuccess` + `data->parameters`).
|
|
55
|
+
- **`reportStaleParametersToClickUp()`** resolves the current True Dev sprint list, then for
|
|
56
|
+
each stale checkpoint creates or updates one ClickUp task titled
|
|
57
|
+
`"<client name> - <PARAMETER_KEY> Error"`. Existing tasks are matched by that exact title
|
|
58
|
+
and updated in place (description + Last Occurrence + incremented Occurrences), so repeated
|
|
59
|
+
stalls don't duplicate tasks. One task per client + sync type.
|
|
60
|
+
|
|
61
|
+
## Data model
|
|
62
|
+
|
|
63
|
+
- **Monitored clients** — hardcoded in `CLIENT_CONFIGURATIONS` (each entry = plain-English
|
|
64
|
+
`name` + the client/api/secret consts from `App_Api_Toga2`). Only clients in this array are
|
|
65
|
+
ever checked; anything omitted is invisible to the monitor. As of 2026-07-10 the list holds
|
|
66
|
+
15 clients: Canon, Endeavor Health, NYCHH, Trividia, Prudential-EDI, Broward Sheriff's
|
|
67
|
+
Office, Miami-Dade, Compass Group, S&P Global, ERAU, Masonite, NCCI, SHRSS, SBA, Quad,
|
|
68
|
+
Growrk. **AIG is deliberately excluded** (see the sync doc's AIG gotcha — a held account
|
|
69
|
+
whose checkpoints never advance, so it would alert forever).
|
|
70
|
+
- **Checkpoints** — the six `NETSUITE_LAST_SYNC_DATETIME_{SALES_ORDERS,PURCHASE_ORDERS,INVOICES,ITEM_RECEIPTS,ITEM_FULFILLMENTS,INVENTORY_ADJUSTMENTS}`
|
|
71
|
+
parameters, written per client by `common_sync_togasupply.php` and read here over the API.
|
|
72
|
+
- **Reporting state** — the dispatcher records `lastReported` / `lastStatus` /
|
|
73
|
+
`lastMessage` per monitor in the **`SystemMonitors`** table in **`db_log`** (title
|
|
74
|
+
"NetSuite Sync Alert"). ClickUp tasks land in the current True Dev sprint list.
|
|
75
|
+
|
|
76
|
+
## Gotchas / known issues
|
|
77
|
+
|
|
78
|
+
- **A monitor that can't finish inside the cron budget silently never runs.** The dispatcher
|
|
79
|
+
calls `App_Framework::cronInitialization(300, true)` — a hard **300s** budget shared across
|
|
80
|
+
**all** monitors in one process. When each of the six checkpoints was read in its own
|
|
81
|
+
`/parameters` call (14 clients × 6 = 84 serialized ~2.7s calls ≈ 230s), the whole
|
|
82
|
+
`system_monitors.php` run blew the budget and was killed mid-execution. Because this
|
|
83
|
+
monitor's only tick is the top of the hour — the exact run that died — it never completed
|
|
84
|
+
and **never reported**. Fixed 2026-07-10 by reading all six keys per client in one
|
|
85
|
+
`where..in` call (~230s → ~40s). If you add more clients/keys or more per-client calls,
|
|
86
|
+
watch the total against 300s.
|
|
87
|
+
- **Two silent-failure traps hide a dead monitor.** (1) The dispatcher wraps `getStatus()` in
|
|
88
|
+
`catch (Exception)` and on a throw sets `$status = array(true, 'Unexpected error occurred')`
|
|
89
|
+
— a **thrown monitor is recorded OK**, not ALERT. (2) A run killed by the time limit records
|
|
90
|
+
nothing at all (no `cronFinished()`). Either way the monitor looks fine while doing nothing.
|
|
91
|
+
- **"Seeded/listed but not reporting = dead monitor."** To catch this: check
|
|
92
|
+
`SystemMonitors.lastReported` in `db_log` — if this monitor's row is stale while every other
|
|
93
|
+
monitor is current, its hourly run is dying. Cross-check the cron log
|
|
94
|
+
(`Vision_Log.Log`, `sourceJob LIKE '%system_monitors%'`) for a **Started without a Finished**
|
|
95
|
+
on the heavy runs. This is exactly how the 2026-06-29 → 07-10 blind period was found:
|
|
96
|
+
`lastReported` was frozen at 2026-06-29 14:19 while everything else advanced.
|
|
97
|
+
- **Hardcoded ClickUp API token in source.** `CLICK_UP_API_ACCESS_TOKEN` is a literal constant
|
|
98
|
+
in this file (pre-existing). It should be moved to config/secrets rather than living in the
|
|
99
|
+
class — treat the in-repo token as compromised if the repo leaks. (Documenting that it
|
|
100
|
+
exists and where; the value itself must never be copied into the KB.)
|
|
101
|
+
- **Adding a client here is not the same as onboarding it to the sync.** This list only
|
|
102
|
+
controls *alerting*. A client must actually be provisioned on the sync engine (wrapper +
|
|
103
|
+
schedule + Parameters seed + customer link + custom fields — see the sync doc) before its
|
|
104
|
+
checkpoints mean anything; and a client that is held / never transacts supply (AIG) must be
|
|
105
|
+
left **off** this list or it produces permanent false-positive tickets.
|
|
106
|
+
|
|
107
|
+
## Change history
|
|
108
|
+
|
|
109
|
+
- 2026-07-10 — **Fixed the monitor timing out and silently not running since 2026-06-29.**
|
|
110
|
+
Rewrote `getClientStatus()` to read all six checkpoints per client in one TOGa2
|
|
111
|
+
`where..in` `/parameters` call (was 84 serialized calls ≈ 230s, over the 300s
|
|
112
|
+
`system_monitors.php` budget → killed every hourly run); added `extractParameterRows()` to
|
|
113
|
+
normalize api2's single-object-vs-array response. Run dropped ~230s → ~40s (verified
|
|
114
|
+
read-only against prod, 41.2s). Added **Quad** and **Growrk** to `CLIENT_CONFIGURATIONS`
|
|
115
|
+
(previously unmonitored); **AIG deliberately left off** (held account, would alert forever).
|
|
116
|
+
Documented the dead-monitor detection path and the two silent-failure traps. (bala)
|
|
@@ -153,11 +153,27 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
|
|
|
153
153
|
(e.g. this parse error), not per-client data. When the fix deploys, MODE flips to RUNNING and the
|
|
154
154
|
watermark jumps to ~now on completion (a positive live signal); the sync self-recovers and
|
|
155
155
|
backfills the gap.
|
|
156
|
-
- **
|
|
156
|
+
- **The NetSuite sync monitor's coverage is a per-client allowlist.**
|
|
157
157
|
`App_SystemMonitor_NetSuiteIntegration` (`library/app/systemmonitor/netsuiteintegration.php`)
|
|
158
|
-
alerts ClickUp when any client's checkpoint is >48h stale, but
|
|
159
|
-
|
|
160
|
-
|
|
158
|
+
alerts ClickUp when any client's checkpoint is >48h stale, but it only checks the clients in
|
|
159
|
+
its hardcoded `CLIENT_CONFIGURATIONS` list — an unlisted client's stall raises no alert.
|
|
160
|
+
Quad and Growrk were added 2026-07-10 (previously omitted); **AIG is deliberately left off**
|
|
161
|
+
because it's a held account whose checkpoints never advance (see the AIG gotcha) and would
|
|
162
|
+
alert forever. Full monitor mechanics + its own silent-failure traps live in the
|
|
163
|
+
[NetSuite Sync Alert Monitor doc](../../library/features/netsuite-sync-alert-monitor.md).
|
|
164
|
+
- **Item Fulfillments + Inventory Adjustments syncs are stalled across most togasupply clients
|
|
165
|
+
(real stalls, not disabled/false positives) as of 2026-07-10.**
|
|
166
|
+
`NETSUITE_LAST_SYNC_DATETIME_ITEM_FULFILLMENTS` has been frozen since ~2026-04-28/05-05 and
|
|
167
|
+
`NETSUITE_LAST_SYNC_DATETIME_INVENTORY_ADJUSTMENTS` since ~2026-06-12/20 on **Canon,
|
|
168
|
+
Endeavor, Trividia, Miami-Dade, S&P, ERAU, NCCI, SHRSS** and **Growrk (IF/InvAdj)**, while
|
|
169
|
+
**Prudential-EDI** is fully frozen (all six checkpoints) since ~2026-06-20. The other four
|
|
170
|
+
sync types (sales orders, POs, invoices, item receipts) advance normally for these clients.
|
|
171
|
+
All wrappers have `IS_ENABLED_INTEGRATION_ITEM_FULFILLMENTS = true` and
|
|
172
|
+
`IS_ENABLED_INTEGRATION_INVENTORY_ADJUSTMENTS = true`, so the IF/InvAdj sections are
|
|
173
|
+
**erroring while the others succeed** — a real data issue, not a disabled integration type.
|
|
174
|
+
Healthy across all six: **NYCHH, Broward, Compass, Masonite, SBA.** When the fixed monitor
|
|
175
|
+
runs, expect it to surface these as ~24 legitimate ClickUp tickets on its first successful
|
|
176
|
+
pass.
|
|
161
177
|
- **Seeded-but-stale = stalled sync.** Verify health by reading `LAST_SYNC_DATETIME_*` in each
|
|
162
178
|
`Client_<Id>.Parameters`: a healthy client advances to ~now every 5 min. As of 2026-06-17,
|
|
163
179
|
Quad was seeded but frozen at 2025-04/05 (not advancing) while the other 12 deployed clients
|
|
@@ -175,9 +191,19 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
|
|
|
175
191
|
opportunity/entitlement account, not a procurement/supply customer. Lesson: before onboarding a
|
|
176
192
|
client, confirm it actually transacts supply orders in NetSuite (SuiteQL the `customer`'s
|
|
177
193
|
transaction history) — being in the array doesn't mean it belongs on this engine.
|
|
194
|
+
Confirmed again 2026-07-10: `Client_Aig`'s six checkpoints are all still frozen at exactly
|
|
195
|
+
the seed date `2026-04-28 00:00:00` (never advanced), which is why AIG is **deliberately
|
|
196
|
+
excluded** from the NetSuite Sync Alert monitor — including it would generate permanent
|
|
197
|
+
false-positive stale tickets.
|
|
178
198
|
|
|
179
199
|
## Change history
|
|
180
200
|
|
|
201
|
+
- 2026-07-10 — Added **Quad + Growrk** to the NetSuite Sync Alert monitor's
|
|
202
|
+
`CLIENT_CONFIGURATIONS` (previously unmonitored); **AIG deliberately left off** (held
|
|
203
|
+
account, checkpoints frozen at the 2026-04-28 seed). Recorded a real, ongoing IF/InvAdj
|
|
204
|
+
stall across most togasupply clients (Prudential-EDI fully frozen) surfaced during the
|
|
205
|
+
monitor fix — see the new gotchas. Monitor mechanics moved to their own
|
|
206
|
+
[library feature doc](../../library/features/netsuite-sync-alert-monitor.md). (bala)
|
|
181
207
|
- 2026-07-09 — **Tier-wide outage (2026-07-06 → 07-09): a dropped brace in shared `rest.php`
|
|
182
208
|
killed every client's sync for 3 days.** A bad merge (`c3caa11a`, "Merge branch '_production'
|
|
183
209
|
into TRUE-79078", 2026-06-24) silently dropped two tokens at the `listLocations()` /
|
package/knowledge/INDEX.md
CHANGED
|
@@ -4,7 +4,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
4
4
|
|
|
5
5
|
## 1.0 framework
|
|
6
6
|
|
|
7
|
-
- **library** (Library) _(framework core)_ —
|
|
7
|
+
- **library** (Library) _(framework core)_ — 12 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
|
|
8
8
|
- **worker** (Worker) — 14 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
|
|
9
9
|
- **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
|
|
10
10
|
- **togadesk** (TOGa Desk) — 9 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)
|
package/package.json
CHANGED