toga-ai 1.0.36 → 1.0.38

package/knowledge/1.0/apps/library/INDEX.md

@@ -3,4 +3,4 @@
 | Doc | Summary | Files |
 |-----|---------|-------|
 | [Library (1.0 Framework) Architecture](architecture.md) | `library` is the shared library repository for **all 1.0 (legacy) applications** — the `App_` framework. | library/_.php, library/app/, library/browser/ |
-| [Elite Freshservice Sync (library)](features/elite-freshservice-sync.md) | `App_Api_Toga2` in `library/app/api/toga2.php` contains three private helpers that handle bidirectional file-attachment sync between TOGA 2 and other systems fo | library/app/api/toga2.php |
+| [Elite Freshservice Sync (library)](features/elite-freshservice-sync.md) | `App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between TOGA 2 and TOGaDesk. | library/app/api/toga2.php |

package/knowledge/1.0/apps/library/features/elite-freshservice-sync.md

@@ -17,77 +17,150 @@ related:
 
 ## Summary
 
-`App_Api_Toga2` in `library/app/api/toga2.php` contains three private helpers that handle
-bidirectional file-attachment sync between TOGA 2 and other systems for Elite:
+`App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between
+TOGA 2 and TOGaDesk. The top-level entry point is `syncWithTogadesk()`. Three private
+helper methods handle file-attachment sync within that orchestration.
 
-| Method | Direction | Purpose |
-|---|---|---|
-| `syncFreshserviceConversationAttachmentsIntoToga2` | Freshservice → TOGA 2 | Download attachments from Freshservice presigned S3 URLs and store in TOGA 2 |
-| `syncToga2NoteFilesIntoTogadesk1` | TOGA 2 → TOGaDesk | Copy TOGA 2 note files to TOGaDesk reply attachments |
-| `syncTogadesk1NoteFilesIntoToga2` | TOGaDesk → TOGA 2 | Copy TOGaDesk reply attachments to TOGA 2 note files |
-
-## Key files / entry points
-
-- `library/app/api/toga2.php` — all three methods live here, called from larger
-  sync orchestration methods in the same class
-- `App_Api_Toga2::send()` — the HTTP client used by all three
+## Top-level entry point — `syncWithTogadesk()`
 
-## How dedup works (all three methods)
+14-parameter static method. Called from a cron script for each client.
 
-All three methods avoid re-uploading files by checking what is already linked before
-uploading. The pattern uses `depth=5` on the parent ticket-note:
+```php
+App_Api_Toga2::syncWithTogadesk(
+    int    $togadeskClientId,
+    int    $togadeskTicketDepartmentId,
+    string $togaClientUuid,
+    string $togaApiKey,
+    string $togaApiSecret,
+    string $integrationType,                                          // 'REPAIR_ORDERS' or 'TICKETS'
+    bool   $isEnabledToga2ToTogadeskIntegration,
+    bool   $isEnabledTogadeskToToga2Integration,
+    bool   $isEnabledToga2ContactsToTogadeskPeopleIntegration,
+    bool   $isEnabledToga2ItemsUnitsToTogadeskAssetsIntegration,
+    bool   $isEnabledToga2PredefinedRepliesToTogadeskTicketsPrIntegration,
+    bool   $isEnabledToga2TicketTeamsToTogadeskGroupsIntegration,
+    bool   $isEnabledToga2TicketCategoriesToTogadeskCategorySubcategoryItemsIntegration,
+    array  $monitorTogadeskDepartmentIds = []
+);
+```
 
+Constants for `$integrationType`:
 ```php
-$togaNote = App_Api_Toga2::send($clientUuid, $apiKey, $apiSecret,
-    'GET', '/ticket-notes/' . $ticketNoteUuid, null, ['depth' => 5], true, $endpoint);
-
-$existingFileNames = [];
-foreach (($togaNote->data->ticketNotes->ticketNoteFiles ?? []) as $link) {
-    if (!empty($link->file->name)) {
-        $existingFileNames[] = $link->file->name;
-    }
-}
+App_Api_Toga2::SYNC_WITH_TOGADESK_INTEGRATION_TYPE__REPAIR_ORDERS  // = 'REPAIR_ORDERS'
+App_Api_Toga2::SYNC_WITH_TOGADESK_INTEGRATION_TYPE__TICKETS         // = 'TICKETS'
 ```
 
-Dedup is by **filename** — if `$file->name` is already in `$existingFileNames`, skip.
+## Watermark timestamps
 
-## TOGA 2 API — critical gotchas
+The sync uses two watermarks stored as TOGA 2 `/parameters` records. These are fetched
+at the start of each run and updated at the end:
 
-**Response key is `ticketNoteFiles`, not `ticketNotesFiles`.**
-The extra `s` before `Files` causes a silent empty array — dedup never fires and files
-get re-uploaded on every sync run. Always use `ticketNoteFiles`.
+| TOGA 2 parameter key | Meaning | Used for |
+|---|---|---|
+| `TOGADESK_LAST_TICKET_INTEGRATION_DATETIME` | Newest `_updated` timestamp of tickets synced from TOGA 2 | Drives TOGA 2 → TOGaDesk query |
+| `TOGA_LAST_TICKET_INTEGRATION_DATETIME` | Newest `timestamp` of tickets/replies synced from TOGaDesk | Drives TOGaDesk → TOGA 2 query |
+
+Both default to `2000-01-01 00:00:00` if not yet set (full initial sync).
+
+## TOGA 2 → TOGaDesk sync (`isEnabledToga2ToTogadeskIntegration`)
+
+Runs when this flag is `true`. Processes 200 records per page.
+
+**Step 1 — Changed tickets:**
+- `GET /tickets?where[_updated > TOGADESK_LAST_TICKET_INTEGRATION_DATETIME]` (paged)
+- Each ticket re-fetched at `depth=5` for full details
+- Calls `syncToga2TicketIntoTogadesk1Ticket($togaTicket)`
+
+**Step 2 — Changed ticket-notes (catches notes on already-processed tickets):**
+- `GET /ticket-notes?where[_updated > TOGADESK_LAST_TICKET_INTEGRATION_DATETIME]` (paged)
+- Skips tickets already processed in Step 1 (tracked in `$alreadyProcessedToga2TicketUuids`)
+- Re-fetches the parent ticket at `depth=5`, calls `syncToga2TicketIntoTogadesk1Ticket`
+
+**Inside `syncToga2TicketIntoTogadesk1Ticket`:**
+- `isExternal = true` note → creates `tickets_replies` record with `referenceid = ticketNote->uuid`
+- `isExternal = false` note → creates `comments` record with `referenceid = ticketNote->uuid`
+- After creating a new reply: calls `syncToga2NoteFilesIntoTogadesk1($ticketNote, $togadeskReplyId)`
+- Also writes a `tickets_history` row for each new reply
+
+## TOGaDesk → TOGA 2 sync (`isEnabledTogadeskToToga2Integration`)
+
+Runs when this flag is `true`.
+
+**Ticket selection:**
+```sql
+SELECT tickets.id,
+    GREATEST(
+        COALESCE(latestReply.timestamp, tickets.timestamp),
+        COALESCE(latestComment.timestamp, tickets.timestamp)
+    ) AS ticketIntegrationTimestamp
+FROM tickets
+LEFT JOIN (SELECT ticketid, MAX(timestamp) FROM tickets_replies GROUP BY ticketid) AS latestReply ...
+LEFT JOIN (SELECT ticketid, MAX(timestamp) FROM comments GROUP BY ticketid) AS latestComment ...
+WHERE tickets.clientid = ? AND tickets.departmentId IN (?)
+AND GREATEST(...) > '$dtTogaLastTicketIntegration'
+```
 
-**`GET /ticket-notes/{uuid}/ticket-note-files` is NOT a list endpoint.**
-This nested route uses `childPolicy = MATCH`:
-- 0 bridge records → 404 EV-6
-- 1 record → 200 single object
-- 2+ records → 404 EV-14 ("multiple found, expected one")
+**Per ticket:** calls `syncTogaDesk1TicketIntoToga2Ticket($togadeskTicketId)`
 
-Always use `depth=5` on the parent note for listing — never the nested route.
+**Inside that method (replies loop):**
+- For each `tickets_replies` row without a `referenceId`: POST to `/ticket-notes` in TOGA 2
+  with `c_eliteConversationId` (for Elite) or `c_wjeConversationId` (for WJE)
+- Stores returned `ticketNotes.uuid` back into `tickets_replies.referenceId`
+- Calls `syncTogadesk1NoteFilesIntoToga2($togadeskReplyId, $ticketNoteUuid)`
 
-**`throwExceptionOnApiError` (8th param of `App_Api_Toga2::send()`).**
-Pass `true` for the dedup GET — a valid note UUID should always return 200. Passing
-`false` silently swallows errors and the dedup array stays empty.
+**`referenceid` field** in TOGaDesk (`tickets_replies.referenceId`, `comments.referenceId`)
+is the foreign key to TOGA 2 — it holds the TOGA 2 `ticketNotes.uuid`. When `referenceId`
+is already populated the row was previously synced and is skipped.
 
-## Bridge route for linking files to notes
+## Other sync operations within `syncWithTogadesk`
 
-```php
-// POST /ticket-notes/{uuid}/ticket-note-files
-App_Api_Toga2::send($clientUuid, $apiKey, $apiSecret, 'POST',
-    '/ticket-notes/' . $ticketNoteUuid . '/ticket-note-files',
-    ['file' => ['uuid' => $fileUuid]],
-    null, true, $endpoint);
-```
+| Flag | What it syncs | TOGA 2 source → TOGaDesk target |
+|---|---|---|
+| `isEnabledToga2ContactsToTogadeskPeopleIntegration` | Contacts → People | `/contacts` → `people` table |
+| `isEnabledToga2ItemsUnitsToTogadeskAssetsIntegration` | Units → Assets | `/units` → `assets` table |
+| `isEnabledToga2PredefinedRepliesToTogadeskTicketsPrIntegration` | Predefined replies → TicketsPR | `/predefined-replies` → `tickets_pr` table |
+| `isEnabledToga2TicketTeamsToTogadeskGroupsIntegration` | Ticket teams → Groups | `/ticket-teams` → `groups` table |
+| `isEnabledToga2TicketCategoriesToTogadeskCategorySubcategoryItemsIntegration` | Categories → custom field | Three-level tree flattened to comma-separated string, stored in TOGaDesk `tickets_customfields` |
+
+## File sync helpers
+
+### `syncToga2NoteFilesIntoTogadesk1(object $ticketNote, int $togadeskReplyId)`
+Direction: **TOGA 2 → TOGaDesk**
+
+1. `GET /ticket-notes/{uuid}?depth=5` → reads `ticketNoteFiles` array
+2. `SELECT name FROM files WHERE ticketreplyid = $togadeskReplyId` — existing filename dedup
+3. For each new file:
+   - `GET /files/{uuid}?depth=1` → `base64_decode($fileData->data->files->data)`
+   - INSERT `App_Model_TogaDesk_File` (ticketreplyid, name) → get `$fileId`
+   - Write to disk: `/var/www/html/ontrack/desk/uploads/{fileId}-{name}`
+   - `UPDATE files SET file='{fileId}-{name}', filetype=?, filesize=? WHERE id=?`
+
+### `syncTogadesk1NoteFilesIntoToga2(int $togadeskReplyId, string $ticketNoteUuid)`
+Direction: **TOGaDesk → TOGA 2**
+
+1. `SELECT id, name, file FROM files WHERE ticketreplyid = $togadeskReplyId`
+2. `GET /ticket-notes/{uuid}?depth=5` → reads `ticketNoteFiles` — existing filename dedup
+3. For each new file:
+   - Read from disk: `/var/www/html/ontrack/desk/uploads/{file}`
+   - `POST /files` with base64-encoded data → get `$fileUuid`
+   - `POST /ticket-notes/{uuid}/ticket-note-files` with `{file: {uuid: $fileUuid}}`
+
+### `syncFreshserviceConversationAttachmentsIntoToga2(...)`
+Direction: **Freshservice → TOGA 2** — see worker2 doc.
 
-## Freshservice attachment handling
+## TOGA 2 API — critical gotchas
 
-Freshservice attachment URLs are **presigned S3 URLs that expire**. The sync must run
-immediately at webhook time — do not defer or re-queue the download step.
+**Response key is `ticketNoteFiles`, not `ticketNotesFiles`.**
+The extra `s` causes a silent empty array — dedup never fires, files re-upload every run.
 
-## TOGaDesk file storage
+**`GET /ticket-notes/{uuid}/ticket-note-files` is NOT a list endpoint.**
+`childPolicy = MATCH`: 0 records → 404 EV-6, 2+ records → 404 EV-14.
+Always use `depth=5` on the parent note for listing.
 
-`syncToga2NoteFilesIntoTogadesk1` writes files to disk at:
-```
-/var/www/html/ontrack/desk/uploads/{id}-{filename}
-```
-Stored as `{id}-{filename}` in the TOGaDesk `files.file` column.
+**`throwExceptionOnApiError` (8th param of `App_Api_Toga2::send()`).**
+Pass `true` for the dedup GET — errors must not silently empty the dedup array.
+
+## Error handling
+
+Each ticket sync is wrapped in `try/catch` — exceptions are sent to Sentry and the run
+continues with the next ticket. A single bad ticket does not abort the whole sync.

package/knowledge/INDEX.md

@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 1.0 framework
 
 - **library** (Library) _(framework core)_ — 2 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
-- **worker** (Worker) — 1 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
+- **worker** (Worker) — 2 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 
 ## 2.0 framework
 
@@ -20,4 +20,5 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 - **compass-canada** → [clients/compass-canada/INDEX.md](clients/compass-canada/INDEX.md)
 - **compass-usa** → [clients/compass-usa/INDEX.md](clients/compass-usa/INDEX.md)
 - **elite** → [clients/elite/INDEX.md](clients/elite/INDEX.md)
+- **nycdoe** → [clients/nycdoe/INDEX.md](clients/nycdoe/INDEX.md)
 

package/knowledge/clients/nycdoe/INDEX.md

@@ -0,0 +1,6 @@
+# Client: nycdoe
+
+| Doc | Framework | Summary | Files |
+|-----|-----------|---------|-------|
+| [NYCDOE ServiceNow / ASN Integration](features/servicenow-integration.md) | 1.0 | The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents + RITMs) into local tables, turns vendor shipment notices into NetSuite Sales Orde | worker/crons/sync/nycdoe/import_asn.php, worker/crons/sync/nycdoe/import_inc.php, worker/crons/sync/nycdoe/legacy_import_asn.php, worker/crons/sync/nycdoe/legacy_process_asn_queue.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/1_send_asn_to_netsuite.php, worker/crons/sync/nycdoe/2_send_serials_to_netsuite.php, worker/crons/sync/nycdoe/3_create_installation_ticket.php, worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/send_request_item_updates.php, worker/crons/sync/nycdoe/send_nycdoe_proof_of_delivery.php, worker/crons/sync/nycdoe/sync_nycdoe_locations.php, worker/crons/sync/nycdoe/receive_edi_purchase_orders.php, worker/crons/sync/nycdoe/send_edi_open_invoices.php, worker/crons/notifications/nycdoe/, worker/schedules/cron.worker.sync.json, worker/schedules/cron.worker.notification.json, library/app/api/nycdoe.php, library/app/api/nycdoev2.php, library/app/asnprocessor/manufacturer.php, library/app/edi.php |
+| [NYC DOE (New York City Department of Education)](profile.md) | 1.0 | NYC DOE (New York City Department of Education) is a TOGA client whose entire integration runs in the **1.0 worker tier** (~30 cron scripts under `worker/crons/ |  |

package/knowledge/clients/nycdoe/features/servicenow-integration.md

@@ -0,0 +1,196 @@
+---
+title: NYCDOE ServiceNow / ASN Integration
+framework: "1.0"
+repo: worker
+project: Worker
+client: nycdoe
+type: client-feature
+status: active
+updated: 2026-06-10
+owners: [mhammontree]
+files:
+  - worker/crons/sync/nycdoe/import_asn.php
+  - worker/crons/sync/nycdoe/import_inc.php
+  - worker/crons/sync/nycdoe/legacy_import_asn.php
+  - worker/crons/sync/nycdoe/legacy_process_asn_queue.php
+  - worker/crons/sync/nycdoe/process_tickets.php
+  - worker/crons/sync/nycdoe/1_send_asn_to_netsuite.php
+  - worker/crons/sync/nycdoe/2_send_serials_to_netsuite.php
+  - worker/crons/sync/nycdoe/3_create_installation_ticket.php
+  - worker/crons/sync/nycdoe/send_ticket_updates.php
+  - worker/crons/sync/nycdoe/send_request_item_updates.php
+  - worker/crons/sync/nycdoe/send_nycdoe_proof_of_delivery.php
+  - worker/crons/sync/nycdoe/sync_nycdoe_locations.php
+  - worker/crons/sync/nycdoe/receive_edi_purchase_orders.php
+  - worker/crons/sync/nycdoe/send_edi_open_invoices.php
+  - worker/crons/notifications/nycdoe/
+  - worker/schedules/cron.worker.sync.json
+  - worker/schedules/cron.worker.notification.json
+  - library/app/api/nycdoe.php
+  - library/app/api/nycdoev2.php
+  - library/app/asnprocessor/manufacturer.php
+  - library/app/edi.php
+related:
+  - ../profile.md
+  - ../../../1.0/apps/worker/architecture.md
+---
+
+## Summary
+
+The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents + RITMs) into
+local tables, turns vendor shipment notices into NetSuite Sales Orders and TogaDesk
+installation repair orders (the **ASN pipeline** — the heart of the integration), and pushes
+ticket state / ETA / proof-of-delivery back to ServiceNow. A parallel EDI surface exchanges
+850 POs and invoices over DOE's SFTP.
+
+> **Schedule files are the source of truth for what runs** (`worker/schedules/
+> cron.worker.sync.json`, `cron.worker.notification.json`). Header comments inside the
+> scripts are frequently stale about cadence. A script on disk but absent from the schedule
+> never runs.
+
+## Core local tables (`db_common` unless noted)
+
+| Table | Purpose |
+|---|---|
+| `NYCDOETickets` | Local ticket mirror (keyed `referenceSysId`; `type` = INCIDENT / RITM) |
+| `NYCDOETicketsDepartments` | Allowlist of valid assignment groups — others are not processed |
+| `NYCDOELocations` | Site-id → address mirror (backs ASN ship-to resolution) |
+| `AdvanceShippingNoticeQueue` | Stage-1 landing zone; **UNIQUE index `idx_dedupeKey`** |
+| `AdvanceShippingNotices` / `…Items` / `…Units` | ASN header (vendor+PO) → per-part items (`qtyOrder`) → individual units |
+| `managed_service_orders`, `repair_orders` (`db_togadesk`) | TogaDesk-side fulfillment (one MSO per ASN; `nycDoeTicketId` links back to the RITM) |
+
+## Integration surfaces (live schedule)
+
+| Surface | Script | Schedule |
+|---|---|---|
+| Incidents in | `import_inc.php` (500/batch into `NYCDOETickets`) | every 15 min |
+| RITMs in (ASN Stage 1a) | `import_asn.php` via `App_Api_NYCDOEV2::listRequestItems(500)` | every 15 min |
+| Vendor SFTP in (ASN Stage 1b) | `legacy_import_asn.php` — "Will NOT be turned off" | every 5 min |
+| Incident → TogaDesk ticket | `process_tickets.php` (client 16; throws if >200 unprocessed — flood valve) | every 15 min |
+| Queue → ASN records (Stage 2) | `legacy_process_asn_queue.php` | every 15 min |
+| NetSuite SO create (Stage 3, **freeze**) | `1_send_asn_to_netsuite.php` | every 2 h at :07 |
+| NetSuite PO/serials (Stage 4) | `2_send_serials_to_netsuite.php` | every 2 h at :27 |
+| TogaDesk repair orders (Stage 5) | `3_create_installation_ticket.php` | every 5 min |
+| NetSuite Item Fulfillment (Stage 6) | `4_create_ns_item_fulfillment.php` (**paused**, `active: 0`); `5_DOA_…` unscheduled | not running |
+| ETA → ServiceNow | `send_ticket_updates.php` | every 6 min |
+| RITM state → ServiceNow | `send_request_item_updates.php` | every 5 min |
+| Proof of delivery → ServiceNow | `send_nycdoe_proof_of_delivery.php` | every 5 min |
+| Locations sync | `sync_nycdoe_locations.php` | daily 3:00 AM |
+| Site-id monitor | `email_notification_siteid.php` | daily 7:01 AM |
+| EDI 850 POs in → NetSuite SO + 997 ack | `receive_edi_purchase_orders.php` (`senderId == 'NYCDOE'` only) | 9:30 AM & 3:30 PM |
+| EDI invoices out (NetSuite saved search 2108) | `send_edi_open_invoices.php` | every 2 h at :13 |
+| Installation schedule reports | `notifications/nycdoe/send_installation_schedules_doe.php` (3 business days ahead, weekends skipped, **holidays NOT skipped**), daily + current-week variants, weekly part-orders | 5–8 AM daily/Mondays |
+
+**Unscheduled / dormant:** `download_tickets*.php` (superseded by `import_inc.php` /
+`import_asn.php`), `DOA_*` variants, `doe_install_fix.php`,
+`send_installation_schedules.php` (base variant). Note `sync/syncro/download_tickets.php`
+in the schedule is the **Syncro** integration, not NYCDOE. A second
+`crons/infrastructure/import_asn.php` also exists — confirm which variant before editing.
+
+## The ASN pipeline
+
+```
+ServiceNow API ──(import_asn.php, SERIALIZED ONLY)──┐
+                                                    ├─► AdvanceShippingNoticeQueue ──(legacy_process_asn_queue.php)──►
+Vendor SFTP ───(legacy_import_asn.php, ser+non-ser)─┘      [UNIQUE dedupeKey]
+
+    AdvanceShippingNotices ─► Items (qtyOrder) ─► Units (1/serial; 1 per non-serial item row)
+          │                        │                    │
+          │             1_send_asn_to_netsuite.php      3_create_installation_ticket.php
+          │                        ▼                    ▼
+          │             NetSuite Sales Order        TogaDesk repair orders (1 per Unit)
+          │             ★ FREEZE qtyOrder ★
+          │                        │
+          ▼             2_send_serials_to_netsuite.php
+   TogaDesk "Receiving Summary" → "#received / #ordered" (denominator = SUM(qtyOrder))
+```
+
+- **Stage 1a (API):** vendor detected by string match on `u_asset_mfg` (apple=1, acer=2,
+  lenovo=3, lexmark=4; unknown → Lexmark). Serial 'S'-prefix stripped for Lexmark/Lenovo.
+  Ship-to resolved via `App_Api_NYCDOEV2::resolveShippingAddress()`; on failure a tracking
+  `NYCDOETicket` is recorded but **ASN creation is skipped**.
+  `dedupeKey = vendorId|PO|DOE-part|serialNumber`. Dedup is SELECT-before-INSERT with
+  per-row try/catch (unique index is the race backstop).
+- **Stage 1b (SFTP):** the only source of **non-serialized** items (in practice the Lexmark
+  feed). Serial-less lines of the same PO+part are summed per file into one queue row with
+  `dedupeKey = vendorId|PO|DOE-part|<file token>`. Dedup is INSERT + catch "Duplicate entry"
+  (race-safe).
+- **Stage 2:** queue → ASN/Items/Units/TrackingNumbers in a transaction. Serialized: one
+  Unit per serial. Non-serialized: the file's quantity is a **cumulative** total —
+  `unsent item qtyOrder = cumulative − SUM(committed qtyOrder)`; post-freeze growth becomes
+  a **fresh item row**, a decrease fires a discrepancy email and changes nothing. Exactly
+  one Unit per serial-less item row regardless of quantity. Failed rows retry 5× then alert
+  `devteam@goagilant.com`.
+- **Stage 3 (THE FREEZE):** creates the NetSuite SO and stamps
+  `AdvanceShippingNoticeItems.netSuiteInternalSalesOrderId`. **Once stamped, that item's
+  `qtyOrder` is immutable** — there is no update path to a sent SO line; growth must be a
+  new item row → new SO line.
+- **Stage 4:** reconciles the NetSuite **PO** (`netSuiteInternalPurchaseOrderId` — not the
+  SO id) and sends serial inventory assignments.
+- **Stage 5:** for units `WHERE togadeskRepairOrderId IS NULL`, reads serials from NetSuite
+  item receipts (so TogaDesk RO serials are **driven by NetSuite**, not the ASN unit),
+  creates/reuses TogaDesk manufacturers/models/assets/serial-numbers/MSO and creates **a
+  new repair_orders row unconditionally per loop iteration** — one RO per Unit (a 50-cable
+  line = 1 unit = 1 RO). Existing-serial lookup is scoped **per item row** (`$asnItemId`).
+- **"#/N received" UI** (`togadesk/desk/template/pages/managedserviceorders/view.php`):
+  denominator = `SUM(qtyOrder)` per part (committed + unsent); numerator = Units with
+  `togadeskRepairOrderId IS NOT NULL`.
+
+## Critical business rules (SME-confirmed — re-confirm, don't infer)
+
+1. **An ASN file is a CUMULATIVE restatement** of a PO+part total, not a delta (50 grows to
+   75; the file says "75").
+2. **Match non-serialized items on customer PO + part number** — tracking/waybill numbers
+   are NOT reliable.
+3. **ServiceNow only delivers serialized items.** Non-serialized comes only via vendor SFTP
+   — this is why both ingestion paths exist.
+4. **The NetSuite freeze:** `netSuiteInternalSalesOrderId IS NOT NULL` ⇒ `qtyOrder` locked.
+   Growth = new item row; decrease = discrepancy alert, never a reduction.
+5. **One unit per non-serialized item row** — a 50-cable line is 1 unit and 1 repair order.
+6. **The `-RPL` PO suffix** forces a brand-new ASN/MSO/RO chain for replacements.
+
+## Gotchas / known issues
+
+- **The queue `dedupeKey` format is load-bearing.** A 2026 incident (WR260236464 duplicate
+  repair order) was caused by dedupeKey format drift (4-part keys with serial vs 3-part
+  without) letting old rows re-import; Stage 2 then created a sibling item row (frozen
+  original not reusable), and Stage 5's per-item-row serial scoping duplicated the unit →
+  duplicate RO. Any change to the key format needs a backfill or transitional matcher.
+  Forensic fingerprint: a Unit with `trackingNumberId IS NULL` was added by Stage 5 from
+  NetSuite, not by the importer.
+- **Two dedupe strategies coexist:** SFTP path = INSERT+catch (race-safe); API path =
+  check-then-insert (unique index backstop).
+- **Timezone traps:** worker runs `America/Chicago`; `repair_orders.dtEta` is stored
+  **Eastern**; ServiceNow `u_eta` is sent as **UTC** `YYYY-MM-DD HH:MM:SS` (converted via
+  `App_Date::convertTimezone()`, only sent when first 19 chars differ from remote).
+- **EDI PO104 leading-zero suppression** (go-live 2026-06-07): DOE's IBM-based translator
+  sends `.94` instead of `0.94` — exposure is in `App_Edi::translatePurchaseOrder850()`,
+  not the cron.
+- **Installation schedule reports skip weekends but not holidays** — empty output near a
+  holiday is expected, not a bug. Empty also just means no ROs with `dtEta` on the exact
+  target date.
+- **Stage cadences differ** (5 min / 15 min / 6 min / 2 h) — "missing" data downstream is
+  often just a stage that hasn't ticked yet.
+- **Stage 6 is not currently running** (`4_` paused with `active: 0`, `5_` unscheduled).
+- TRUE-77354: Stage 5 lookups changed `LIKE` → `=` to stop `_`/`%` wildcard matches in
+  serials.
+
+## Operating rules when changing this integration
+
+1. **Trace the full pipeline downstream before fixing** — the constraint that makes an edit
+   unsafe usually lives downstream (the freeze point and the consumer/display query).
+2. **Ask the SME for business-data semantics — never guess** (cumulative vs delta, identity
+   keys, what counts as one unit, whether totals can decrease).
+3. All ServiceNow calls go through `App_Api_NYCDOEV2` public static methods.
+4. Never mutate a frozen item's quantity; carry deltas forward as new rows.
+5. With no staging environment, verify with a standalone arithmetic simulation plus a read
+   of the consumer query; `php -l` every touched file.
+
+## Related docs
+
+- [NYC DOE client profile](../profile.md)
+- [Worker (1.0) architecture](../../../1.0/apps/worker/architecture.md) — cron dispatch,
+  role election, schedule files
+- Local deep-dive (full WR260236464 case study with record IDs):
+  `doe_service_now_integration.md` in Mark's WWW workspace root; also the
+  `service-now-integration` skill (`asn-import-flow.md`)

package/knowledge/clients/nycdoe/profile.md

@@ -0,0 +1,50 @@
+---
+title: NYC DOE (New York City Department of Education)
+framework: "1.0"
+project: Worker
+client: nycdoe
+type: profile
+status: active
+updated: 2026-06-10
+owners: [mhammontree]
+files: []
+related:
+  - features/servicenow-integration.md
+---
+
+## Summary
+NYC DOE (New York City Department of Education) is a TOGA client whose entire integration
+runs in the **1.0 worker tier** (~30 cron scripts under `worker/crons/sync/nycdoe/` and
+`worker/crons/notifications/nycdoe/`, on the `sync` and `notification` worker roles).
+ServiceNow is DOE's system of record for tickets (Incidents and RITMs); TOGA mirrors those
+locally, drives fulfillment through TogaDesk and NetSuite, and pushes status / ETA /
+proof-of-delivery back to ServiceNow. A parallel EDI relationship (850 POs in, invoices out)
+runs over DOE's SFTP.
+
+## Platforms & data
+- **1.0 worker tier only** — no 2.0 footprint. Local mirror tables live in `db_common`
+  (`NYCDOETickets`, `NYCDOELocations`, `AdvanceShippingNotice*`).
+- **TogaDesk:** DOE is client id **16** (`db_togadesk`) — repair orders, managed service
+  orders, and the "#received / #ordered" Receiving Summary UI.
+- **NetSuite:** Sales Orders, PO reconciliation, item receipts, Item Fulfillments, invoices
+  (SuiteTalk toolkit in `library`).
+
+## Endpoints
+| System | Endpoint | Notes |
+|---|---|---|
+| ServiceNow prod | `https://nycdoeprod.service-now.com` | Incidents + RITMs |
+| ServiceNow stage | `https://nycdoedev2.service-now.com` | used when `App_Registry::inTestMode()` |
+| ServiceNow disposals | `https://supportadmin.schools.nyc` (stage `nycdoedev3`) | disposal service requests |
+| DOE EDI SFTP | `transfer.schools.nyc` (SSH key `doedimedu.pem`) | EDI 850 in, invoices out |
+| Vendor SFTP feeds | Apple / Lexmark / Lenovo (Lenovo: `ftp.asisystem.com`) | legacy ASN file ingestion |
+
+## API clients
+- `App_Api_NYCDOE` (`library/app/api/nycdoe.php`) — V1 client (legacy ticket downloaders).
+- `App_Api_NYCDOEV2` (`library/app/api/nycdoev2.php`) — centralized V2 client. **All
+  ServiceNow calls must go through its public static methods** — never call endpoints
+  directly from a cron.
+
+## Key features (this client)
+- [ServiceNow / ASN Integration](features/servicenow-integration.md) — the full integration
+  map: ticket sync, the ASN pipeline (the core flow), outbound status sync, EDI, and the
+  SME-confirmed business rules.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.36",
+  "version": "1.0.38",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",