toga-ai 1.0.36 → 1.0.37

package/knowledge/1.0/apps/library/INDEX.md

@@ -3,4 +3,4 @@
 | Doc | Summary | Files |
 |-----|---------|-------|
 | [Library (1.0 Framework) Architecture](architecture.md) | `library` is the shared library repository for **all 1.0 (legacy) applications** — the `App_` framework. | library/_.php, library/app/, library/browser/ |
-| [Elite Freshservice Sync (library)](features/elite-freshservice-sync.md) | `App_Api_Toga2` in `library/app/api/toga2.php` contains three private helpers that handle bidirectional file-attachment sync between TOGA 2 and other systems fo | library/app/api/toga2.php |
+| [Elite Freshservice Sync (library)](features/elite-freshservice-sync.md) | `App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between TOGA 2 and TOGaDesk. | library/app/api/toga2.php |

package/knowledge/1.0/apps/library/features/elite-freshservice-sync.md

@@ -17,77 +17,150 @@ related:
 
 ## Summary
 
-`App_Api_Toga2` in `library/app/api/toga2.php` contains three private helpers that handle
-bidirectional file-attachment sync between TOGA 2 and other systems for Elite:
+`App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between
+TOGA 2 and TOGaDesk. The top-level entry point is `syncWithTogadesk()`. Three private
+helper methods handle file-attachment sync within that orchestration.
 
-| Method | Direction | Purpose |
-|---|---|---|
-| `syncFreshserviceConversationAttachmentsIntoToga2` | Freshservice → TOGA 2 | Download attachments from Freshservice presigned S3 URLs and store in TOGA 2 |
-| `syncToga2NoteFilesIntoTogadesk1` | TOGA 2 → TOGaDesk | Copy TOGA 2 note files to TOGaDesk reply attachments |
-| `syncTogadesk1NoteFilesIntoToga2` | TOGaDesk → TOGA 2 | Copy TOGaDesk reply attachments to TOGA 2 note files |
-
-## Key files / entry points
-
-- `library/app/api/toga2.php` — all three methods live here, called from larger
-  sync orchestration methods in the same class
-- `App_Api_Toga2::send()` — the HTTP client used by all three
+## Top-level entry point — `syncWithTogadesk()`
 
-## How dedup works (all three methods)
+14-parameter static method. Called from a cron script for each client.
 
-All three methods avoid re-uploading files by checking what is already linked before
-uploading. The pattern uses `depth=5` on the parent ticket-note:
+```php
+App_Api_Toga2::syncWithTogadesk(
+    int    $togadeskClientId,
+    int    $togadeskTicketDepartmentId,
+    string $togaClientUuid,
+    string $togaApiKey,
+    string $togaApiSecret,
+    string $integrationType,                                          // 'REPAIR_ORDERS' or 'TICKETS'
+    bool   $isEnabledToga2ToTogadeskIntegration,
+    bool   $isEnabledTogadeskToToga2Integration,
+    bool   $isEnabledToga2ContactsToTogadeskPeopleIntegration,
+    bool   $isEnabledToga2ItemsUnitsToTogadeskAssetsIntegration,
+    bool   $isEnabledToga2PredefinedRepliesToTogadeskTicketsPrIntegration,
+    bool   $isEnabledToga2TicketTeamsToTogadeskGroupsIntegration,
+    bool   $isEnabledToga2TicketCategoriesToTogadeskCategorySubcategoryItemsIntegration,
+    array  $monitorTogadeskDepartmentIds = []
+);
+```
 
+Constants for `$integrationType`:
 ```php
-$togaNote = App_Api_Toga2::send($clientUuid, $apiKey, $apiSecret,
-    'GET', '/ticket-notes/' . $ticketNoteUuid, null, ['depth' => 5], true, $endpoint);
-
-$existingFileNames = [];
-foreach (($togaNote->data->ticketNotes->ticketNoteFiles ?? []) as $link) {
-    if (!empty($link->file->name)) {
-        $existingFileNames[] = $link->file->name;
-    }
-}
+App_Api_Toga2::SYNC_WITH_TOGADESK_INTEGRATION_TYPE__REPAIR_ORDERS  // = 'REPAIR_ORDERS'
+App_Api_Toga2::SYNC_WITH_TOGADESK_INTEGRATION_TYPE__TICKETS         // = 'TICKETS'
 ```
 
-Dedup is by **filename** — if `$file->name` is already in `$existingFileNames`, skip.
+## Watermark timestamps
 
-## TOGA 2 API — critical gotchas
+The sync uses two watermarks stored as TOGA 2 `/parameters` records. These are fetched
+at the start of each run and updated at the end:
 
-**Response key is `ticketNoteFiles`, not `ticketNotesFiles`.**
-The extra `s` before `Files` causes a silent empty array — dedup never fires and files
-get re-uploaded on every sync run. Always use `ticketNoteFiles`.
+| TOGA 2 parameter key | Meaning | Used for |
+|---|---|---|
+| `TOGADESK_LAST_TICKET_INTEGRATION_DATETIME` | Newest `_updated` timestamp of tickets synced from TOGA 2 | Drives TOGA 2 → TOGaDesk query |
+| `TOGA_LAST_TICKET_INTEGRATION_DATETIME` | Newest `timestamp` of tickets/replies synced from TOGaDesk | Drives TOGaDesk → TOGA 2 query |
+
+Both default to `2000-01-01 00:00:00` if not yet set (full initial sync).
+
+## TOGA 2 → TOGaDesk sync (`isEnabledToga2ToTogadeskIntegration`)
+
+Runs when this flag is `true`. Processes 200 records per page.
+
+**Step 1 — Changed tickets:**
+- `GET /tickets?where[_updated > TOGADESK_LAST_TICKET_INTEGRATION_DATETIME]` (paged)
+- Each ticket re-fetched at `depth=5` for full details
+- Calls `syncToga2TicketIntoTogadesk1Ticket($togaTicket)`
+
+**Step 2 — Changed ticket-notes (catches notes on already-processed tickets):**
+- `GET /ticket-notes?where[_updated > TOGADESK_LAST_TICKET_INTEGRATION_DATETIME]` (paged)
+- Skips tickets already processed in Step 1 (tracked in `$alreadyProcessedToga2TicketUuids`)
+- Re-fetches the parent ticket at `depth=5`, calls `syncToga2TicketIntoTogadesk1Ticket`
+
+**Inside `syncToga2TicketIntoTogadesk1Ticket`:**
+- `isExternal = true` note → creates `tickets_replies` record with `referenceid = ticketNote->uuid`
+- `isExternal = false` note → creates `comments` record with `referenceid = ticketNote->uuid`
+- After creating a new reply: calls `syncToga2NoteFilesIntoTogadesk1($ticketNote, $togadeskReplyId)`
+- Also writes a `tickets_history` row for each new reply
+
+## TOGaDesk → TOGA 2 sync (`isEnabledTogadeskToToga2Integration`)
+
+Runs when this flag is `true`.
+
+**Ticket selection:**
+```sql
+SELECT tickets.id,
+    GREATEST(
+        COALESCE(latestReply.timestamp, tickets.timestamp),
+        COALESCE(latestComment.timestamp, tickets.timestamp)
+    ) AS ticketIntegrationTimestamp
+FROM tickets
+LEFT JOIN (SELECT ticketid, MAX(timestamp) FROM tickets_replies GROUP BY ticketid) AS latestReply ...
+LEFT JOIN (SELECT ticketid, MAX(timestamp) FROM comments GROUP BY ticketid) AS latestComment ...
+WHERE tickets.clientid = ? AND tickets.departmentId IN (?)
+AND GREATEST(...) > '$dtTogaLastTicketIntegration'
+```
 
-**`GET /ticket-notes/{uuid}/ticket-note-files` is NOT a list endpoint.**
-This nested route uses `childPolicy = MATCH`:
-- 0 bridge records → 404 EV-6
-- 1 record → 200 single object
-- 2+ records → 404 EV-14 ("multiple found, expected one")
+**Per ticket:** calls `syncTogaDesk1TicketIntoToga2Ticket($togadeskTicketId)`
 
-Always use `depth=5` on the parent note for listing — never the nested route.
+**Inside that method (replies loop):**
+- For each `tickets_replies` row without a `referenceId`: POST to `/ticket-notes` in TOGA 2
+  with `c_eliteConversationId` (for Elite) or `c_wjeConversationId` (for WJE)
+- Stores returned `ticketNotes.uuid` back into `tickets_replies.referenceId`
+- Calls `syncTogadesk1NoteFilesIntoToga2($togadeskReplyId, $ticketNoteUuid)`
 
-**`throwExceptionOnApiError` (8th param of `App_Api_Toga2::send()`).**
-Pass `true` for the dedup GET — a valid note UUID should always return 200. Passing
-`false` silently swallows errors and the dedup array stays empty.
+**`referenceid` field** in TOGaDesk (`tickets_replies.referenceId`, `comments.referenceId`)
+is the foreign key to TOGA 2 — it holds the TOGA 2 `ticketNotes.uuid`. When `referenceId`
+is already populated the row was previously synced and is skipped.
 
-## Bridge route for linking files to notes
+## Other sync operations within `syncWithTogadesk`
 
-```php
-// POST /ticket-notes/{uuid}/ticket-note-files
-App_Api_Toga2::send($clientUuid, $apiKey, $apiSecret, 'POST',
-    '/ticket-notes/' . $ticketNoteUuid . '/ticket-note-files',
-    ['file' => ['uuid' => $fileUuid]],
-    null, true, $endpoint);
-```
+| Flag | What it syncs | TOGA 2 source → TOGaDesk target |
+|---|---|---|
+| `isEnabledToga2ContactsToTogadeskPeopleIntegration` | Contacts → People | `/contacts` → `people` table |
+| `isEnabledToga2ItemsUnitsToTogadeskAssetsIntegration` | Units → Assets | `/units` → `assets` table |
+| `isEnabledToga2PredefinedRepliesToTogadeskTicketsPrIntegration` | Predefined replies → TicketsPR | `/predefined-replies` → `tickets_pr` table |
+| `isEnabledToga2TicketTeamsToTogadeskGroupsIntegration` | Ticket teams → Groups | `/ticket-teams` → `groups` table |
+| `isEnabledToga2TicketCategoriesToTogadeskCategorySubcategoryItemsIntegration` | Categories → custom field | Three-level tree flattened to comma-separated string, stored in TOGaDesk `tickets_customfields` |
+
+## File sync helpers
+
+### `syncToga2NoteFilesIntoTogadesk1(object $ticketNote, int $togadeskReplyId)`
+Direction: **TOGA 2 → TOGaDesk**
+
+1. `GET /ticket-notes/{uuid}?depth=5` → reads `ticketNoteFiles` array
+2. `SELECT name FROM files WHERE ticketreplyid = $togadeskReplyId` — existing filename dedup
+3. For each new file:
+   - `GET /files/{uuid}?depth=1` → `base64_decode($fileData->data->files->data)`
+   - INSERT `App_Model_TogaDesk_File` (ticketreplyid, name) → get `$fileId`
+   - Write to disk: `/var/www/html/ontrack/desk/uploads/{fileId}-{name}`
+   - `UPDATE files SET file='{fileId}-{name}', filetype=?, filesize=? WHERE id=?`
+
+### `syncTogadesk1NoteFilesIntoToga2(int $togadeskReplyId, string $ticketNoteUuid)`
+Direction: **TOGaDesk → TOGA 2**
+
+1. `SELECT id, name, file FROM files WHERE ticketreplyid = $togadeskReplyId`
+2. `GET /ticket-notes/{uuid}?depth=5` → reads `ticketNoteFiles` — existing filename dedup
+3. For each new file:
+   - Read from disk: `/var/www/html/ontrack/desk/uploads/{file}`
+   - `POST /files` with base64-encoded data → get `$fileUuid`
+   - `POST /ticket-notes/{uuid}/ticket-note-files` with `{file: {uuid: $fileUuid}}`
+
+### `syncFreshserviceConversationAttachmentsIntoToga2(...)`
+Direction: **Freshservice → TOGA 2** — see worker2 doc.
 
-## Freshservice attachment handling
+## TOGA 2 API — critical gotchas
 
-Freshservice attachment URLs are **presigned S3 URLs that expire**. The sync must run
-immediately at webhook time — do not defer or re-queue the download step.
+**Response key is `ticketNoteFiles`, not `ticketNotesFiles`.**
+The extra `s` causes a silent empty array — dedup never fires, files re-upload every run.
 
-## TOGaDesk file storage
+**`GET /ticket-notes/{uuid}/ticket-note-files` is NOT a list endpoint.**
+`childPolicy = MATCH`: 0 records → 404 EV-6, 2+ records → 404 EV-14.
+Always use `depth=5` on the parent note for listing.
 
-`syncToga2NoteFilesIntoTogadesk1` writes files to disk at:
-```
-/var/www/html/ontrack/desk/uploads/{id}-{filename}
-```
-Stored as `{id}-{filename}` in the TOGaDesk `files.file` column.
+**`throwExceptionOnApiError` (8th param of `App_Api_Toga2::send()`).**
+Pass `true` for the dedup GET — errors must not silently empty the dedup array.
+
+## Error handling
+
+Each ticket sync is wrapped in `try/catch` — exceptions are sent to Sentry and the run
+continues with the next ticket. A single bad ticket does not abort the whole sync.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.36",
+  "version": "1.0.37",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",