toga-ai 1.0.879 → 1.0.881

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -20,4 +20,5 @@
20
20
  | [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | Tools (1.0) admin UI to browse/fix Talos KB documents, provision new Bedrock VECTOR KBs (S3 Vectors), and manage transcript-cleanup vocabulary — the human front |
21
21
  | [Talos Pricing UI (Contracts, Pricing Dashboard, Usage, Settings + Estimator)](features/talos-pricing-ui.md) | The 1.0 (tools) face of the Talos Pricing Platform — Contracts / Pricing Dashboard / Usage / Pricing Settings pages + the estimate engine, reading the Team DB v |
22
22
  | [App-Wide Colour Theme (light / dark / auto)](features/theme-light-dark.md) | App-wide light/dark/auto colour theme for the Tools app via a single semantic-token layer (`theme.css`), localStorage-persisted, applied before first paint; ope |
23
+ | [Tools → worker2 Job Bridge (App_Worker::enqueue + reading WorkerJobs.output)](features/worker2-job-bridge.md) | How a 1.0 `tools` page runs 2.0 worker2 work and reads the result back; open before adding any Tools page that triggers a worker action. |
23
24
  | [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the Tools 1.0 app deploys/boots on Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023) — AL2023 package migration, INI/fpm session gotchas, git-clone of library |
@@ -6,8 +6,8 @@ project: Tools
6
6
  client: shared
7
7
  type: architecture
8
8
  status: active
9
- updated: 2026-09-16
10
- owners: [jcardinal, kyalamarthi, ajean]
9
+ updated: 2026-09-23
10
+ owners: [jcardinal, kyalamarthi, ajean, mhammontree]
11
11
  files:
12
12
  - tools/index.php
13
13
  - tools/.ebextensions/020_setup_git_libraries.config
@@ -79,6 +79,7 @@ Create `mvc/<folder>/<tool>/get.php` (copy `mvc/_TEMPLATE/get.php`), add one ent
79
79
 
80
80
  ## Gotchas
81
81
 
82
+ - **⚠ `tools` runs PHP 8, not 7.2 — check the vhost, not the framework label.** Despite being a 1.0 `App_` app, its vhost lives in the PHP 8 Apache (`DocumentRoot "C:/WWW/tools"`), and `App_Compass::tenantKey(mixed $key)` uses the `mixed` type hint (PHP 8.0+). The trap: `mixed` **passes** `php -l` on 7.2 (parsed as a class name) and only fatals at runtime — "Argument 1 passed to f() must be an instance of mixed, string given". So linting `tools` on 7.2 fails in the dangerous direction: it accepts code that would die.
82
83
  - **Plaintext secrets were publicly web-readable (fixed 2026-07-24; rotation still owed).** `config.*.ini` (incl. `config.prod.ini`) holds plaintext production secrets (RDS master password, SMTP, Payeezy, NetSuite). The front controller routed through `index.php` on a `RewriteCond !-f` rule **only**, which is not access control — a direct `GET /config.production.ini` served the file **verbatim**. Fixed 2026-07-24 by adding `<FilesMatch>` `Require all denied` blocks (sensitive extensions incl. `.ini`, and any `config.*`) — see the [1.0 back-end security standard](../../standards/backend-php.md). **Action still owed:** rotate every secret in `config.production.ini` — the file was reachable, so treat all of it as compromised. (Location only — no values recorded here.)
83
84
  - **Hardcoded GitHub Personal Access Token committed in plaintext (rotation owed).** `.ebextensions/020_setup_git_libraries.config` contains a GitHub PAT in plaintext, committed to the repo and therefore exposed in git history. **Action owed:** rotate the token and move it to an EB environment property (read via `getenv()`/`$_SERVER`/`$_ENV` per the deploy workflow's clear_env gotcha); a rotation does not un-expose git history. (Location only — no value recorded here.)
84
85
  - **SSO initiation + replay defense are open items** — see `features/saml-sso-auth.md`.
@@ -0,0 +1,68 @@
1
+ ---
2
+ title: Tools → worker2 Job Bridge (App_Worker::enqueue + reading WorkerJobs.output)
3
+ framework: "1.0"
4
+ repo: tools
5
+ project: Tools
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-09-23
10
+ owners: [mhammontree]
11
+ files:
12
+ - tools/_/app/worker.php
13
+ related:
14
+ - ../../../clients/compass-usa/features/people-file-import-admin-page.md
15
+ - ./mvc-data-access-patterns.md
16
+ - ../../../2.0/apps/worker2/architecture.md
17
+ - ../../../2.0/apps/worker2/features/stranded-worker-jobs.md
18
+ ---
19
+
20
+ How a 1.0 `tools` page runs 2.0 worker2 work and reads the result back; open before adding any Tools page that triggers a worker action.
21
+
22
+ ## Summary
23
+
24
+ `tools` never talks to a third-party system that worker2 already owns. It **enqueues a worker2
25
+ action** and reads the answer out of `Core.WorkerJobs.output`. `App_Worker::enqueue()`
26
+ (`tools/_/app/worker.php`) inserts the `Core.WorkerJobs` row and sends the SQS message.
27
+
28
+ ## How it works
29
+
30
+ 1. Page calls `App_Worker::enqueue($action, $parameters)`.
31
+ 2. worker2 picks the job up, runs it, writes the result into `Core.WorkerJobs.output`.
32
+ 3. Page polls the row by `workerJobId` and decodes `output`.
33
+
34
+ **`output` is a LOG LINE wrapping the JSON**, written by `worker2/Controller/Index.php`:
35
+
36
+ ```
37
+ [requestId] Successfully Executed (Ns): {"...json..."}
38
+ ```
39
+
40
+ So find the first `{` and decode from there. If no `{` is present, show a clear error — never an
41
+ empty result — because the wrapper format is worker2's, not a contract.
42
+
43
+ ## Gotchas
44
+
45
+ - **`enqueue()` has THREE outcomes, not two.**
46
+ (a) insert fails → `ok=false`, `workerJobId=null`.
47
+ (b) insert succeeded but the **SQS send failed** → `ok=false` with a **real** `workerJobId` and
48
+ error "Job row created but SQS dispatch failed".
49
+ (c) both succeeded → `ok=true`.
50
+ A caller that tests only `workerJobId === null` reads (b) as success and then shows "still
51
+ running" forever, because no worker will ever pick that row up. **Test `ok`, not the id.**
52
+ - **`isSuccess IS NULL` means two different things** — running, or abandoned. Same trap as
53
+ [stranded worker jobs](../../../2.0/apps/worker2/features/stranded-worker-jobs.md).
54
+ - **Tools must not speak SFTP (or any integration worker2 owns).** Decided on CTO review
55
+ (TRUE-82250). Giving `tools` its own SFTP client would copy a production credential into a
56
+ second repo (rotation becomes a two-repo change forever), and **`library/phpseclib` is
57
+ phpseclib 1.x**: it offers only `diffie-hellman-group1-sha1`, `group14-sha1`,
58
+ `group-exchange-sha1`, `group-exchange-sha256` — no curve25519, no ecdh-sha2. Modern SFTP
59
+ servers commonly refuse sha1 key exchange, so 1.0 may not negotiate at all where worker2's
60
+ phpseclib3 can.
61
+ - **Timeouts do not match.** Tools' PHP `max_execution_time` is **120s**. A job enqueued outside
62
+ the CronScheduler has `cronJobId NULL` and therefore the **300s** action cap, not the cron's
63
+ `maxExecutionTime`. Never make a page wait for long work — poll, don't block.
64
+ - **Deploy worker2 first.** A Tools page that calls a new action is broken until the action
65
+ exists.
66
+
67
+ ## Change history
68
+ - 2026-09-23 — Created from TRUE-82250: documented the three `enqueue()` outcomes, the log-line-wrapped `output`, and the CTO decision that tools never speaks SFTP (phpseclib 1.x sha1-only KEX + credential duplication). (mhammontree)
@@ -6,9 +6,10 @@ project: API
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-16
10
- owners: ["bala", "mhammontree", "tcox", "apeterson", "rgirish"]
9
+ updated: 2026-09-23
10
+ owners: ["bala", "mhammontree", "tcox", "apeterson", "rgirish", "jcardinal"]
11
11
  files:
12
+ - toga25-supply/src/pages/TransferOrders/view/CreateTransferOrderModal/hooks/useCreateTransferOrder.ts
12
13
  - api2/Component/Api/V2/V2.php
13
14
  - _underscore/Model/Client/ContactEmailAddress.php
14
15
  - _underscore/Model/Client/TransferOrderStage.php
@@ -156,6 +157,13 @@ WHERE recordId = 351 AND field = 'c_netsuiteInternalTransferOrderStatus';
156
157
 
157
158
  `recordId 351` is `transfer-order-stages`; the equivalent row exists per tenant with a **different `id`** but often the **same `uuid`** (both Elite and NYCHH carry `b273f4d1-dac0-4093-ad1d-7c3a526ab894`), the tell that the row came from one shared seed and only one tenant's copy has the wrong flag.
158
159
 
160
+ ### Nested inherent-child create — parent FK is auto-set, any depth
161
+
162
+ - V2 auto-sets the child's FK to the parent: first field whose `FIELDOPT_FOREIGNKEY_MODEL` equals the parent class (exact normalized `_Model_Client_` string), then break (`V2.php` ~5211-5221). Payload supplies the OTHER FK only (e.g. bridge row `{ purchaseOrder: { uuid } }` under a transfer order).
163
+ - `processRoutePairs` recurses with `parentModel` (~5284-5300); `lookupInherentChildByFieldName` is rebuilt per record (~3407), so grandchildren work (`transferOrderItems[].purchaseOrderItemsTransferOrderItems[]`).
164
+ - Each level's key must be an `InherentRecordChildren` child of THAT level's record, else it lands in `unknownFields` (~5047).
165
+ - Adding an inherent-child row also inlines that child in the parent's default GET for every client — check deep readers (1.0 NetSuite sync reads depth 5) first.
166
+
159
167
  ## Gotchas
160
168
 
161
169
  - **Nested write with only a non-identifier field silently creates duplicates.** A duplicate-generating footgun: each write inserts a new empty child and the parent links to a fresh shell (or is left with a NULL FK). It compounds on any cron/integration that writes the same entity repeatedly. Root-caused in production on the Prudential device-sync cron, which sent `contact: {c_employeeXid}` and generated ~893k empty Contact rows (~5,600/day). See [Prudential device import + contact linking](../../../clients/prudential/features/device-information-import-and-contact-linking.md).
@@ -6,8 +6,8 @@ project: TOGa 2.5 Supply
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-16
10
- owners: [apeterson, bala]
9
+ updated: 2026-09-23
10
+ owners: [apeterson, bala, "jcardinal"]
11
11
  files:
12
12
  - toga25-supply/src/pages/TransferOrders/view/CreateTransferOrderModal/hooks/useCreateTransferOrder.ts
13
13
  - toga25-supply/src/pages/TransferOrders/view/CreateTransferOrderModal/view/PanelSelect.tsx
@@ -199,7 +199,7 @@ One `SurfaceOverrides` row, `IS_VISIBLE = '1'`, **client-wide** (`personaId`/`ro
199
199
 
200
200
  **Two record requests, not one.** The modal reads the lines from the nested route `GET /v2/transfer-orders/{uuid}/transfer-order-items`, with the child route read from the items surface's `config.childRoute` so the FE never hardcodes it.
201
201
 
202
- > ⚠ **The reason for the second request is GONE — the read path was not revisited.** This build declined to add the `InherentRecordChildren` row (it changes the default response shape for *every* existing `/transfer-orders` caller, including the 1.0 NetSuite importer). **That decision was reversed later:** `Core.InherentRecordChildren` row **152** now declares `transfer-orders` (312) → `transfer-order-items` (313), so `/transfer-orders` **does** inline `transferOrderItems` today. The modal still makes two calls; collapsing it to one is available but unclaimed work. See [v2-rest-query-contract](../../api2/features/v2-rest-query-contract.md).
202
+ > ⚠ **The reason for the second request is GONE — the read path was not revisited.** This build declined to add the `InherentRecordChildren` row (it changes the default response shape for *every* existing `/transfer-orders` caller, including the 1.0 NetSuite importer). **That decision was reversed later:** `Core.InherentRecordChildren` row **150** now declares `transfer-orders` (312) → `transfer-order-items` (313), so `/transfer-orders` **does** inline `transferOrderItems` today. The modal still makes two calls; collapsing it to one is available but unclaimed work. See [v2-rest-query-contract](../../api2/features/v2-rest-query-contract.md).
203
203
 
204
204
  **The record fetch raises `calcDepth` to 2.** `_Model_Client_Location` exposes `_addressLine1/_addressLine2/_addressCity/_addressState/_addressZip` as `FIELD_SQL` calculated fields. One FK hop out from the order they need `calcDepth 2`; at the default of 1 they come back **NULL silently** rather than erroring.
205
205
 
@@ -321,7 +321,7 @@ Related data-shape facts found the same day:
321
321
 
322
322
  #### The whole transfer is ONE POST — lines nested under `transferOrderItems`
323
323
 
324
- `hooks/useCreateTransferOrder.ts`. Header **and** every line go out in a **single `POST /transfer-orders`**, with the lines nested under the key **`transferOrderItems`**. This is possible because `Core.InherentRecordChildren` row **152** now makes `transfer-order-items` (313) an inherent child of `transfer-orders` (312).
324
+ `hooks/useCreateTransferOrder.ts`. Header **and** every line go out in a **single `POST /transfer-orders`**, with the lines nested under the key **`transferOrderItems`**. This is possible because `Core.InherentRecordChildren` row **150** now makes `transfer-order-items` (313) an inherent child of `transfer-orders` (312).
325
325
 
326
326
  **Why one request, not three.** An earlier draft posted the header and then each line separately and could leave a **half-written transfer** behind on a mid-sequence failure. One API request is one API transaction — the front controller commits the umbrella connection on `isSuccess` and rolls the data back on failure — so the nested POST is all-or-nothing **without** a scripted endpoint.
327
327
 
@@ -331,6 +331,13 @@ Related data-shape facts found the same day:
331
331
 
332
332
  This closes both **required POST changes** recorded on 2026-09-02: `number` is no longer sent (so `FIELD_AUTOINCREMENT` mints the `TA100000` sequence) and the lines ride nested in the same request (so `_Model_Client_TransferOrder::postPost` sees them when it reads the order back for the NetSuite push) — see [Toga TransferOrder → NetSuite Push](../../worker2/features/netsuite-transferorder-outbound-push.md).
333
333
 
334
+ **PO links ride the same POST (2026-09-23).** `CreateTransferOrderModal/hooks/useCreateTransferOrder.ts`:
335
+ - header: `purchaseOrdersTransferOrders: [{ purchaseOrder: { uuid } }]` → bridge `PurchaseOrders_TransferOrders` (record **325**; inherent child of 312 via row 151).
336
+ - each `transferOrderItems[]` line: `purchaseOrderItemsTransferOrderItems: [{ purchaseOrderItem: { uuid } }]` → bridge `PurchaseOrderItems_TransferOrderItems` (record **329**; inherent child of 313 via `dbchanges2/Core/2026-09-23d`). Source: `TransferOrderDraftLine.purchaseOrderItemUuid` ← `PurchaseOrderItemRow.purchaseOrderItemUuid`.
337
+ - V2 auto-sets each bridge's parent-side FK; payload supplies only the other side. Engine: [nested-relationship-writes](../../api2/features/nested-relationship-writes.md).
338
+ - Direct `purchaseOrder: { uuid }` (`TransferOrders.purchaseOrderId`) is **kept** — the record modal's PO deep link reads `transferOrder.purchaseOrder.uuid`.
339
+ - Use 325/329, never the reverse-named twins 326 `TransferOrders_PurchaseOrders` / 330 `TransferOrderItems_PurchaseOrderItems` (0 rows in prod; dead).
340
+
334
341
  **FKs go out as nested `{ uuid }` objects under the field name WITHOUT the `Id` suffix** — `contact: { uuid }`, `createdByUser: { uuid }`. Each is omitted entirely when absent rather than sent as `null`: a location can have no contact, and an SSO session can lack a user uuid.
335
342
 
336
343
  #### The picker fetch — 13.3 s → 0.39 s by filtering SERVER-side (2026-09-14)
@@ -516,7 +523,7 @@ Money (Unit Value / Total / Total Value) and Target Contact are **dropped for v1
516
523
 
517
524
  ~~⚠ `TransferOrders` carries **no contact FK at all** today, so the Target Contact element's `valueKey` is a **placeholder**~~ — **closed 2026-09-01.** `TransferOrders.contactId` (FK `Contacts`) exists and RecordField **2561** is granted to roles 1 and 3, so the element has a real binding and enabling it is now genuinely a one-row client override.
518
525
 
519
- The items table's **PO column is order-level**: `TransferOrderItems` has no line-level PO, so the column binds the parent order's `purchaseOrder.number` and repeats on every row. For NYCHH it resolves NULL because their PO links are **upstream** — see [NYCHH PO direction](../../../../clients/nychh/features/po-number-upstream-direction.md). It is left visible in Core (neutral default; other tenants populate `TransferOrders.purchaseOrderId` normally). **The header's PO value is now a link into Inventory** (above); for NYCHH it is the same null, so nothing renders there for this tenant yet.
526
+ The items table's **PO column is order-level**: `TransferOrderItems` has no line-level PO column (line PO link lives in bridge 329 since 2026-09-23, not bound to the column), so the column binds the parent order's `purchaseOrder.number` and repeats on every row. For NYCHH it resolves NULL because their PO links are **upstream** — see [NYCHH PO direction](../../../../clients/nychh/features/po-number-upstream-direction.md). It is left visible in Core (neutral default; other tenants populate `TransferOrders.purchaseOrderId` normally). **The header's PO value is now a link into Inventory** (above); for NYCHH it is the same null, so nothing renders there for this tenant yet.
520
527
 
521
528
  ### Entry point #2 — the Inventory page's "Create Transfer Order" button (2026-09-03)
522
529
 
@@ -53,6 +53,7 @@
53
53
  | [Service Request → Sales Order → Purchase Order generation (Sync/ServiceRequest)](features/service-request-sales-order-generation.md) | Worker `_Worker_Sync_ServiceRequest` that turns a Service Request → Sales Order → one PO per vendor for any tenant; open for the SR→SO→PO chain, its interceptor |
54
54
  | [SSO Stability Monitor (Monitor/Operations/SsoStability)](features/sso-stability-monitor.md) | Reporter cron that GETs the saml `/healthcheck` and pushes an SSO heartbeat to a OneUptime monitor; open when working on the SSO uptime probe, its tokens, or it |
55
55
  | [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Inbound Startech (Easeedesk) webhook handler that creates/updates the matching TOGA 2.0 ticket; open for the webhook flow, its payload, or ticket-type/stage res |
56
+ | [Stranded WorkerJobs (deploy-window job loss) and how to repair them](features/stranded-worker-jobs.md) | Jobs queued while worker2 restarts are dropped silently with no retry and no alert; open when a cron "never ran" or a webhook vanished. |
56
57
  | [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | Consumer side of Talos (TOGa IQ): the endpoints, DevCore assistant, and JWT auto-refresh a worker2 dev tool needs to query the meeting-notes corpus; open when a |
57
58
  | [Talos Pricing Automation (worker2 Cron — Usage Import, AWS Actuals, Margins, Profiles)](features/talos-pricing-automation.md) | Monthly worker2 crons that turn raw usage + AWS cost into a measured platform unit price, per-client cost, and margins; open for the pricing pipeline, its AWS c |
58
59
  | [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` cron pipeline ingesting Teams VTT transcripts into Talos/TOGa IQ Bedrock KBs + emailing organizer recaps; open for Export/Process/Ret |
@@ -6,8 +6,8 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-16
10
- owners: ["bala"]
9
+ updated: 2026-09-23
10
+ owners: ["bala", "jcardinal"]
11
11
  files:
12
12
  - worker2/Worker/Netsuite/SalesOrder.php
13
13
  - _underscore/Model/Client/TransferOrder.php
@@ -84,7 +84,7 @@ Nothing beyond this list. A transfer order has no shipping cost, ship method, te
84
84
  | line `rate` | **0** — the zero-value part |
85
85
  | line `location` | the **origin** warehouse. **`location` is NOT a ship-to** — NetSuite never uses a line's location for delivery. |
86
86
 
87
- **Missing data ABORTS the order — never sends a partial one.** Each throws with a named message: **no origin location, no destination location, no lines, no PO number**. Mirrors the sales-order push's missing-mapping rule. **PO # is mandatory on the NetSuite order form**, which is why a missing PO number is fatal rather than omitted. `resolvePurchaseOrderNumber()` reads the PO number from the **`purchaseOrdersTransferOrders` header bridge first** (what the Create Transfer Order modal writes), falling back to the direct `purchaseOrderId` column. See [SO↔PO bridge direction](../../_underscore/features/sales-order-purchase-order-bridge-direction.md).
87
+ **Missing data ABORTS the order — never sends a partial one.** Each throws with a named message: **no origin location, no destination location, no lines, no PO number**. Mirrors the sales-order push's missing-mapping rule. **PO # is mandatory on the NetSuite order form**, which is why a missing PO number is fatal rather than omitted. `resolvePurchaseOrderNumber()` reads the PO number from the **`purchaseOrdersTransferOrders` header bridge first** (written by the Create Transfer Order modal since 2026-09-23), falling back to the direct `purchaseOrderId` column. See [SO↔PO bridge direction](../../_underscore/features/sales-order-purchase-order-bridge-direction.md).
88
88
 
89
89
  **`description` → `memo` cannot leak into a plain sales order.** `memo` is sent only from the transfer-order shape. There is **no `memo` RecordField and no `memo` column anywhere** in the sales-order path.
90
90
 
@@ -0,0 +1,56 @@
1
+ ---
2
+ title: Stranded WorkerJobs (deploy-window job loss) and how to repair them
3
+ framework: "2.0"
4
+ repo: worker2
5
+ project: Worker
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-09-23
10
+ owners: [mhammontree]
11
+ files:
12
+ - worker2/Controller/Index.php
13
+ - worker2/LambdaFunctions/JobScheduler/lambda_function.py
14
+ - worker2/Worker/Infrastructure/Worker.php
15
+ related:
16
+ - ../architecture.md
17
+ - ./creating-worker-actions.md
18
+ - ../../../1.0/apps/tools/features/worker2-job-bridge.md
19
+ ---
20
+
21
+ Jobs queued while worker2 restarts are dropped silently with no retry and no alert; open when a cron "never ran" or a webhook vanished.
22
+
23
+ ## Summary
24
+
25
+ A **worker2 deploy silently drops every job queued during the restart window.** The row stays
26
+ `dtQueued` set, `dtStarted NULL`, `isSuccess NULL` — forever. Nothing recovers it:
27
+
28
+ - JobScheduler Part A only re-queues rows where `dtQueued IS NULL`.
29
+ - The watchdog (Part B) only fails rows that already have `dtStarted`.
30
+
31
+ So there is **no failure, no alert, no `Logs.Issue`** — the job just never happened.
32
+
33
+ Measured in prod 2026-09-23: **95 rows** stranded between 09:15:45 and 11:05:22. Not only crons —
34
+ Clickup/Webhook 9, Netsuite/Webhook 6, Github/Webhook 5, Infrastructure/Email/Send/Run 4,
35
+ Infrastructure/Errors/Escalate 4, plus 2 each for eight `Monitor/Compass/*` actions.
36
+
37
+ ## How it works — repairing them (crons and webhooks need OPPOSITE treatment)
38
+
39
+ **CRON row** — mark it `isSuccess = 0` with an explanatory `output`.
40
+ - Do **not** delete it: that destroys the audit record.
41
+ - Do **not** re-queue by nulling `dtQueued`: that re-runs stale time-based work.
42
+ - If the run is genuinely still needed, fire it deliberately as a fresh job.
43
+
44
+ **WEBHOOK row** — the payload is still in `parameters`, so
45
+ `_Worker_Infrastructure_Worker::Retry($workerJobId)` genuinely recovers it. Retry it.
46
+
47
+ ## Gotchas
48
+
49
+ - **`isSuccess IS NULL` is ambiguous** — running, or abandoned. Check `dtStarted`: NULL plus a
50
+ `dtQueued` older than a few minutes = stranded.
51
+ - **After any worker2 deploy, sweep for stranded rows.** Nothing else will tell you.
52
+ - Same ambiguity on the enqueue side in `tools` — see
53
+ [Tools → worker2 job bridge](../../../1.0/apps/tools/features/worker2-job-bridge.md).
54
+
55
+ ## Change history
56
+ - 2026-09-23 — Created: 95 prod jobs lost in a deploy window (2026-09-23), with the cron-vs-webhook repair rule. (mhammontree)
@@ -131,6 +131,7 @@ NetSuite or to `Forecast`.
131
131
 
132
132
  - **⚠ Locally, worker2's `Core` resolves to the 1.0 `Core` database, not the 2.0 one.** `worker2/_.php:20` has `const DB_CORE = 'Core';` and `Core` is never explicitly registered, so the schema name is used literally. On a dev machine the 2.0 tables live in **`Core_2`** (48 tables) while `Core` is the 1.0 schema (10 tables) — symptom: `Table 'core.workerjobs' doesn't exist`. Workaround: flip `DB_CORE` to `'Core_2'` locally and **revert before committing**.
133
133
  - **That workaround is incomplete — `_underscore/Email.php:435` hardcodes the literal prefix `FROM Core.Clients`.** So the summary-email step still throws `Table 'core.clients' doesn't exist` no matter what `DB_CORE` says. In the Compass PEOPLE cron that step runs **last**, after both tenants commit, so the data work is unaffected — but do not read the trailing exception as a failed run.
134
+ - **Locally, YOU are the queue — non-prod worker2 has no SQS consumer.** Nothing picks a `WorkerJobs` row up on its own; kick each job by POSTing `{"workerJobId": N}` to the worker2 host (Postman).
134
135
  - **⚠ A stale local client database silently invalidates the test.** A local tenant that predates a migration can make broken code **pass** locally — e.g. local `Client_Compass` still had `contactId` NULLABLE, so a missing FK value was accepted where prod's `NOT NULL` turns it into `0` and a constraint error. Before testing anything schema-sensitive, drop and reload the tenant from production and **re-verify the constraint/index actually survived the copy**.
135
136
 
136
137
  - **The ad-hoc SuiteQL runner `test/@dave/Junk Drawer/NetSuite/nsq.php` cannot start — stale bootstrap.**
@@ -181,4 +182,5 @@ NetSuite or to `Forecast`.
181
182
  — the api2 sibling of steps 2–3.
182
183
 
183
184
  ## Change history
185
+ - 2026-09-23 — Noted that non-prod worker2 has no SQS consumer: kick jobs locally by POSTing `{"workerJobId": N}`. (mhammontree)
184
186
  - 2026-09-23 — Added three local-run traps found on TRUE-82230: worker2's literal `DB_CORE = 'Core'` hits the 1.0 schema locally (2.0 lives in `Core_2`); `_underscore/Email.php:435` hardcodes `FROM Core.Clients` so the summary-email step throws regardless; and a stale local client DB can make broken code pass (reload from prod and re-verify constraints). (mhammontree)
@@ -15,12 +15,12 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
15
15
  - **test** (Test) — 16 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
16
16
  - **toga** (TOGa) — 3 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
17
17
  - **store** (Store) — 3 doc(s) → [1.0/apps/store/INDEX.md](1.0/apps/store/INDEX.md)
18
- - **tools** (Tools) — 19 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
18
+ - **tools** (Tools) — 21 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
19
19
 
20
20
  ## 2.0 framework
21
21
 
22
22
  - **_underscore** (_Underscore) _(framework core)_ — 90 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
23
- - **worker2** (Worker) — 74 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
23
+ - **worker2** (Worker) — 75 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
24
24
  - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
25
25
  - **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
26
26
  - **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
@@ -21,6 +21,7 @@
21
21
  | [Compass: ODP / Strategic Systems ASN CSV email import (and the scientific-notation tracking-number loss)](features/odp-strategic-systems-asn-csv-import.md) | 1.0 | The 1.0 worker cron that reads Office Depot / Strategic Systems ASN CSVs from email, why it routes by column count, and why a tracking number can arrive as `8.7 |
22
22
  | [Compass: MA refresh-order monitor (Monitor/Compass/MaRefreshOrderExceptions)](features/oneuptime-ma-refresh-order-monitor.md) | 2.0 | worker2 OneUptime push monitor counting Compass MA employee-refresh orders whose ODP SO has no NetSuite order yet; open when working on Compass MA monitoring or |
23
23
  | [Compass USA — Fulfilled vs Partially Fulfilled (order total kept; per-line rule built then rejected)](features/order-fulfillment-status-per-line.md) | 2.0 | Why Compass USA shows Fulfilled on partly-shipped orders and why the per-line fix was built, measured, then reverted; open before touching Compass USA `_status` |
24
+ | [People File Import admin page (tools /compass/people-file-import → worker2)](features/people-file-import-admin-page.md) | 1.0 | Browser page that lists the Compass PEOPLE files on SFTP and runs the importer on demand; open before changing the page or the importer's listing/guard code. |
24
25
  | [Compass PEOPLE-File User Lifecycle (duplicate accounts, reactivation grace window, raw-SQL deactivation)](features/people-file-user-lifecycle.md) | 2.0 | The nightly Compass PEOPLE-file cron's Users-row lifecycle (insert/reactivate/deactivate), why it creates duplicate accounts, and its raw-SQL/index traps; open |
25
26
  | [Persona Model & Levy-Sector Gating (worker2 PEOPLE cron)](features/persona-model-and-levy-gating.md) | 2.0 | Compass USA catalogue visibility is driven by **personas** in `Client_Compass`. |
26
27
  | [Compass Sales-Order Line-Number Renumbering (and why it hangs off the SO hooks)](features/sales-order-line-renumbering.md) | 2.0 | How Compass keeps SO lines numbered 1..N with no gaps, why the renumber rides the parent SO hooks, and the two-phase SQL that avoids a duplicate-key fatal; open |
@@ -0,0 +1,62 @@
1
+ ---
2
+ title: People File Import admin page (tools /compass/people-file-import → worker2)
3
+ framework: "1.0"
4
+ repo: tools
5
+ project: Tools
6
+ client: compass-usa
7
+ type: client-feature
8
+ status: active
9
+ updated: 2026-09-23
10
+ owners: [mhammontree]
11
+ files:
12
+ - tools/_/app/compass/peoplefile.php
13
+ - tools/mvc/compass/people-file-import/get.php
14
+ - tools/mvc/compass/people-file-import/post.php
15
+ - worker2/Worker/Client/Compass/PeopleFile.php
16
+ related:
17
+ - ./people-file-user-lifecycle.md
18
+ - ../../../1.0/apps/tools/features/worker2-job-bridge.md
19
+ - ../../../1.0/apps/tools/features/compass-user-persona-admin.md
20
+ - ../../../2.0/apps/worker2/features/stranded-worker-jobs.md
21
+ ---
22
+
23
+ Browser page that lists the Compass PEOPLE files on SFTP and runs the importer on demand; open before changing the page or the importer's listing/guard code.
24
+
25
+ ## Summary
26
+
27
+ `tools` page **/compass/people-file-import** (`App_Compass_PeopleFile`, Development Team persona
28
+ only) lists the PEOPLE files waiting on the SFTP server, with a **Refresh** button and an
29
+ **Import** button shown only when files exist. It exists so a stranded or failed nightly cron can
30
+ be re-run without a developer.
31
+
32
+ `tools` does **not** touch SFTP — it enqueues worker2 actions and reads `Core.WorkerJobs.output`.
33
+ See [Tools → worker2 job bridge](../../../1.0/apps/tools/features/worker2-job-bridge.md) for the
34
+ enqueue/decode rules and why SFTP stays in worker2.
35
+
36
+ ## How it works
37
+
38
+ worker2 side (`Worker/Client/Compass/PeopleFile.php`):
39
+
40
+ - `gatherPeopleFiles(SFTP): array` — oldest first, **shared by both** the listing and the import,
41
+ so the page can never show a different "next file" than the importer takes.
42
+ - `Client/Compass/PeopleFile/ListFiles` — new read-only action returning JSON, used by the page.
43
+ - `isAnotherRunInFlight()` guard on `ProcessPeopleFiles`.
44
+ - `ProcessPeopleFiles` now **returns a JSON summary on every path**. It previously returned
45
+ nothing, so every run recorded "<no return message>".
46
+
47
+ **Deploy order: worker2 FIRST, then tools.** The page calls `ListFiles`, which does not exist
48
+ until worker2 ships.
49
+
50
+ Reference: prod `Core.CronJobs` id **8** = "Compass - People File Import", schedule `15 9 * * *`,
51
+ `maxExecutionTime` 3600, action `Client/Compass/PeopleFile/ProcessPeopleFiles`. A job enqueued
52
+ from this page is **not** scheduler-fired, so it gets the 300s action cap, not 3600.
53
+
54
+ ## Gotchas
55
+
56
+ - **A failed run does not mean nothing happened.** The importer is not all-or-nothing — see
57
+ [PEOPLE-file user lifecycle](./people-file-user-lifecycle.md). Check the data before importing
58
+ the same file again.
59
+ - **Persona-gated to Development Team.** It runs a production import; do not widen it casually.
60
+
61
+ ## Change history
62
+ - 2026-09-23 — Created with the page (TRUE-82250, tools 0cc6f99 / worker2 a81ad6c, branch TRUE-82250): shared `gatherPeopleFiles`, new `ListFiles` action, in-flight guard, and a JSON summary return on every path. (mhammontree)
@@ -91,6 +91,7 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
91
91
  **Open product question, still open:** when a manager has genuinely left and HR supplied no replacement, what should the order do — block, escalate to the supervisor's supervisor, or route to an admin queue? The engine deliberately **skips** these as ambiguous rather than guessing.
92
92
 
93
93
  ## Gotchas
94
+ - **⚠ The import is NOT all-or-nothing — a failed job can still have written everything.** Each tenant's import is transactional, but everything **after** the tenant transactions commit runs outside it: `sendProcessingSummaryEmail()` (~line 264) and the SFTP delete. A run that fails at the email step records `isSuccess = 0` while all the data stays written, **and the file stays on SFTP**, so the same file can be imported a second time. Proven locally 2026-09-23: job 1076884 failed in `sendProcessingSummaryEmail` after 25.4s with 460 US + 26 CA users (and the same number of Contacts) committed and kept. **Check the data before re-running.**
94
95
  - **⚠ Deactivation is raw bulk SQL — model hooks never fire.** User inactivation runs as `UPDATE Users SET isActive = 0 WHERE id IN (...)` through `_Query`, **not** the model layer. A `postPut` / `postPost` interceptor on the User model will **never** run on deactivation. Any deactivation-time side effect must be written **inside the importer itself** — adding a model hook and expecting it to fire is a silent no-op.
95
96
  - **The same raw SQL is invisible to the change-audit log.** It produces no `RecordField` rows in `Logs_Compass`, so anything the importer should be accountable for must write its own `Record` **note** explicitly. See [Record Change Audit Log](../../../2.0/apps/_underscore/features/record-change-audit-log.md).
96
97
  - **A duplicate account strands approvals, but not email.** The approval gate matches on user id, so a decision assigned to the old row is invisible to the same human's new account — while the notification email still arrives, because both rows carry the same address. Visible symptom: a person chased to approve an order they cannot see. Detail: [Approval-Decision Flow](./approval-decision-flow.md).
@@ -118,8 +119,11 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
118
119
  - [Compass Approval-Decision Flow](./approval-decision-flow.md) — what a duplicate account breaks downstream.
119
120
  - [Stranded Approval Reassignment](./stranded-approval-reassignment.md) — the engine that repairs the approvals a duplicate account stranded.
120
121
  - [Record Change Audit Log](../../../2.0/apps/_underscore/features/record-change-audit-log.md) — why the importer's raw writes leave no audit trail.
122
+ - [People File Import admin page](./people-file-import-admin-page.md) — the on-demand re-run page in `tools`.
123
+ - [Stranded WorkerJobs](../../../2.0/apps/worker2/features/stranded-worker-jobs.md) — why the nightly cron row can never run at all.
121
124
  - [Compass USA profile](../profile.md)
122
125
  - [Users.email as login identity](../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md)
123
126
 
124
127
  ## Change history
128
+ - 2026-09-23 — Recorded that the import is **not** all-or-nothing (post-commit email + SFTP delete run outside the tenant transaction; job 1076884 failed yet kept 460 US / 26 CA users). TRUE-82230 confirmed in prod: 543 `Client_Compass` users created, **zero** null-or-0 `contactId`, two manual ACTION runs ~18s each — needed because the 09:15 cron row (job 1784367) was stranded. (mhammontree)
125
129
  - 2026-09-23 — **Fixed the `Users.contactId` FK write order (TRUE-82230).** Jeff's `NOT NULL` + `Users_ibfk_1` on both tenants broke the importer from 2026-09-22 09:15 (last good run 2026-09-21); the whole run rolled back and Canada never executed. New `insertContactsForNewUsers()` creates the Contact before the `Users` batch and puts `contactId` in the INSERT column list; existing users missing a contact keep the create-then-UPDATE path. Also recorded: `LAST_INSERT_ID()` must be read on the write connection (not `_Query`); `TEST_MODE` vs `DRY_RUN` neither being safe alone; the FIFO/one-file/empty-is-silent-success facts; and that a stale local DB hides this class of bug. Verified against prod-reloaded local schemas: 80 new US / 17 new CA users, 0 with null-or-0 `contactId`. (mhammontree)
@@ -6,8 +6,8 @@ project: _Underscore
6
6
  client: nychh
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-09-17
10
- owners: ["bala"]
9
+ updated: 2026-09-23
10
+ owners: ["bala", "jcardinal"]
11
11
  files:
12
12
  - _underscore/Model/Client/TransferOrder.php
13
13
  - worker2/Worker/Netsuite/SalesOrder.php
@@ -203,11 +203,18 @@ email, which must fire *after* this push writes `number` back. The `POST` row ab
203
203
 
204
204
  ## Data + metadata state (2026-09-02)
205
205
 
206
- - **`Client_Nychh.PurchaseOrders_TransferOrders` has 0 rows in production**, and
207
- `TransferOrders.purchaseOrderId` is set on **0** of the 1,930 rows — **no transfer order has a PO
208
- link yet.** New orders from the Create Transfer Order modal are what will populate the bridge.
209
- Until then `resolvePurchaseOrderNumber()` returns null and **the push aborts by design** (PO # is
210
- mandatory on the NetSuite order form).
206
+ - **Prod 2026-09-23:** `PurchaseOrders_TransferOrders` (325) = **1,592 rows**; `TransferOrders.purchaseOrderId`
207
+ set on 10 rows. The Create Transfer Order modal now writes 325 (header) + 329 (lines) in its single POST
208
+ ([transfer-orders-page](../../../2.0/apps/toga25-supply/features/transfer-orders-page.md)). An order
209
+ with no PO link still makes `resolvePurchaseOrderNumber()` return null → push aborts by design.
210
+ - Reverse-named twins 326 / 330 have 0 rows — dead; never read or write them.
211
+ - Prod `InherentRecordChildren` ids: 150 = 312→313, 151 = 312→325, 152 = 312→357; added 2026-09-23
212
+ (`Core/2026-09-23d`): 17→325, 18→329, 313→329.
213
+ - RecordFields: 325 = 2209 id / 2210 uuid / 2211 purchaseOrderId / 2212 transferOrderId; 329 = 2225 / 2226 /
214
+ 2227 purchaseOrderItemId / 2228 transferOrderItemId.
215
+ - Base (role 1) + API (role 3) ACL for 325/329 already complete (record CRUD, expressions 302/303, field
216
+ write on uuid + both FKs). Duplicate Base `AclRecordPermissions` (325/331, 328/334) — harmless.
217
+ - `TransferOrderStages` prod: 1 Pending, 2 In Transit, 3 Received, 4 Canceled, 5 Closed.
211
218
  - **`Core.InherentRecordChildren` for record 312** — the depth-4 read needs both the lines and the
212
219
  header PO bridge:
213
220
  - **prod** — was 312→313 and 312→**329** (the *item-level* bridge); repointed by the developer, now
@@ -18,7 +18,7 @@ project: _Underscore
18
18
  client: nychh
19
19
  type: profile
20
20
  status: active
21
- updated: 2026-09-17
21
+ updated: 2026-09-23
22
22
  owners: ["jcardinal", "apeterson", "bala", "akhokhani"]
23
23
  files:
24
24
  - dbchanges2/Client_Nychh/2026-09-02a - TransferOrderNetsuitePushInterceptor.sql
@@ -102,6 +102,7 @@ TOGA 2.0 client on `_underscore`, prod schema `Client_Nychh`. Supply-chain / ful
102
102
  - 🚨 **`Client_Nychh.TransferOrders` HAS production data — 1,927 rows (verified 2026-08-31 on `reader1.client.database.togahub.com`).** The transfer-order *tables and data* have been live in prod for a while (the 1.0 NetSuite importer writes them); only the **2026-08-28 UI vertical** (TableView, Core surfaces, nav override) is new. An earlier version said transfer orders were "local only … not yet run on sandbox-client or production", and that stale line produced a wrong recommendation — that a prod-to-local restore would *wipe* the transfer-orders feature. **Never infer prod state from this repo or from a migration's run status; read prod.** Restoring or overwriting `Client_Nychh` destroys real transfer-order rows.
103
103
  - **The Transfer Orders SCREEN is NYCHH-only and new (2026-08-28).** NYCHH is the first — and today the only — tenant with a `transfer-orders` TableView (`Client_Nychh/2026-08-28a`, which also seeds the ACL chain for record 351 that the Status column silently depends on) and the one `SurfaceOverrides` row that reveals the nav item (`2026-08-28b`, client-wide, matching how this tenant grants Sales Orders and Inventory — NYCHH does not scope navigation by role). The Core surfaces and stage colour tokens are **shared/neutral**, hidden for every other client until they opt in. The seven migration files were **committed 2026-08-31** (`dbchanges2` commit `1c3fe2e`, branch `TRUE-80852`, pushed); run order is **`Core/` → `Client/` fan-out → `Client_Nychh/`**, because the client files reference Core record ids and the NYCHH TableView/ACL need the columns the fan-out files add. Run state per environment must be **verified against that environment**, never assumed from the repo. **beta is blocked** because it lacks the `TransferOrderStages` table. The **Create Transfer Order** modal was reworked 2026-09-01 and its data layer is wired, but 🚨 **the item picker shows zero selectable lines for NYCHH** — see the availability blocker below. See [Transfer Orders page](../../2.0/apps/toga25-supply/features/transfer-orders-page.md).
104
104
  - 🚨 **The transfer-order ACL + schema on prod/sandbox are UNTRACKED (2026-08-28).** Record and field ACL for records 312/313, their logic-group/expression chain, and the `TransferOrderItems.dtCreated`/`dtUpdated` columns were all hand-applied when the NetSuite TransferOrder POST was enabled — **no `dbchanges2` file creates any of it**, so a fresh environment can reproduce neither the UI nor the importer. Reconstructed in `Client_Nychh/2026-08-28c` (ACL, faithful — role 2 deliberately excluded) and `Client/2026-08-28b` (columns; root cause is the blank client template, which affects **every** tenant). See [transfer-order inventory & quantities](./features/transfer-order-inventory-quantities.md).
105
+ - **Create Transfer Order links POs (2026-09-23)** — one POST writes header bridge 325 + line bridge 329 (Core `2026-09-23d` inherent children). Prod 325 has 1,592 rows; 326/330 twins are dead. See [transfer-order push](./features/transfer-order-netsuite-push.md).
105
106
  - **Two-branch inventory quantities + TransferOrder V2 enablement** — NYCHH-only on-hand/available/fulfilled/backordered model on `_Model_Nychh_Item`, plus the client-override models + ACL rows that let a NetSuite TransferOrder POST succeed. See [transfer-order inventory & quantities](./features/transfer-order-inventory-quantities.md).
106
107
  - 🚨 **OPEN BLOCKER — every NYCHH PO line reports availability 0, in sandbox AND production (verified 2026-09-01).** The deployed `_Model_Nychh_PurchaseOrderItem` sources stock-in from inventory adjustments and `INNER JOIN`s `InventoryAdjustmentItems.itemFulfillmentItemId`, which is **NULL on 100% of rows** (292/292 sandbox, 317/317 prod). So `_qtyReceived` / `_qtyOnHand` / `_qtyAvailable` are all `0.00` for every purchase-order line, and the **Create Transfer Order item picker has nothing to show**. Needs an **integration / data-owner decision** — backfill the FK, or change the override's stock-in source (base PO item receipts return real numbers today). Not a frontend or ACL defect, and **not** fixed by `Client/2026-09-01a`. See [transfer-order inventory & quantities](./features/transfer-order-inventory-quantities.md).
107
108
  - 🚨 **`_Model_Nychh_PurchaseOrderItem` is NOT on every `_underscore` branch (verified 2026-09-01).** `Model/Nychh/PurchaseOrderItem.php` exists on **`_production`** and **`_sandbox-client`** but is **absent from `_beta`** and **`_sandbox-dev`**. Because `_underscore` is pulled at deploy by branch, NYCHH PO-line `_qty*` values are **different numbers per tier**: prod/sandbox-client run the inventory-adjustment formula (0.00 until the NetSuite link is backfilled), beta/sandbox-dev fall back to the base item-receipt formula and report real quantities. **Never eliminate a tenant override by `ls`-ing your working tree** — use `git ls-tree -r --name-only origin/<branch>`. That mistake cost a full session chasing a non-existent "API returns 0" bug. See [transfer-order inventory & quantities](./features/transfer-order-inventory-quantities.md).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.879",
3
+ "version": "1.0.881",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",