toga-ai 1.0.35 → 1.0.37

package/knowledge/1.0/apps/library/INDEX.md

@@ -3,3 +3,4 @@
 | Doc | Summary | Files |
 |-----|---------|-------|
 | [Library (1.0 Framework) Architecture](architecture.md) | `library` is the shared library repository for **all 1.0 (legacy) applications** — the `App_` framework. | library/_.php, library/app/, library/browser/ |
+| [Elite Freshservice Sync (library)](features/elite-freshservice-sync.md) | `App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between TOGA 2 and TOGaDesk. | library/app/api/toga2.php |

package/knowledge/1.0/apps/library/features/elite-freshservice-sync.md

@@ -0,0 +1,166 @@
+---
+title: Elite Freshservice Sync (library)
+framework: "1.0"
+repo: library
+project: Library
+client: elite
+type: client-feature
+status: active
+updated: 2026-06-10
+owners: []
+files:
+  - library/app/api/toga2.php
+related:
+  - clients/elite/profile.md
+  - 2.0/apps/worker2/features/elite-freshservice-sync.md
+---
+
+## Summary
+
+`App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between
+TOGA 2 and TOGaDesk. The top-level entry point is `syncWithTogadesk()`. Three private
+helper methods handle file-attachment sync within that orchestration.
+
+## Top-level entry point — `syncWithTogadesk()`
+
+14-parameter static method. Called from a cron script for each client.
+
+```php
+App_Api_Toga2::syncWithTogadesk(
+    int    $togadeskClientId,
+    int    $togadeskTicketDepartmentId,
+    string $togaClientUuid,
+    string $togaApiKey,
+    string $togaApiSecret,
+    string $integrationType,                                          // 'REPAIR_ORDERS' or 'TICKETS'
+    bool   $isEnabledToga2ToTogadeskIntegration,
+    bool   $isEnabledTogadeskToToga2Integration,
+    bool   $isEnabledToga2ContactsToTogadeskPeopleIntegration,
+    bool   $isEnabledToga2ItemsUnitsToTogadeskAssetsIntegration,
+    bool   $isEnabledToga2PredefinedRepliesToTogadeskTicketsPrIntegration,
+    bool   $isEnabledToga2TicketTeamsToTogadeskGroupsIntegration,
+    bool   $isEnabledToga2TicketCategoriesToTogadeskCategorySubcategoryItemsIntegration,
+    array  $monitorTogadeskDepartmentIds = []
+);
+```
+
+Constants for `$integrationType`:
+```php
+App_Api_Toga2::SYNC_WITH_TOGADESK_INTEGRATION_TYPE__REPAIR_ORDERS  // = 'REPAIR_ORDERS'
+App_Api_Toga2::SYNC_WITH_TOGADESK_INTEGRATION_TYPE__TICKETS         // = 'TICKETS'
+```
+
+## Watermark timestamps
+
+The sync uses two watermarks stored as TOGA 2 `/parameters` records. These are fetched
+at the start of each run and updated at the end:
+
+| TOGA 2 parameter key | Meaning | Used for |
+|---|---|---|
+| `TOGADESK_LAST_TICKET_INTEGRATION_DATETIME` | Newest `_updated` timestamp of tickets synced from TOGA 2 | Drives TOGA 2 → TOGaDesk query |
+| `TOGA_LAST_TICKET_INTEGRATION_DATETIME` | Newest `timestamp` of tickets/replies synced from TOGaDesk | Drives TOGaDesk → TOGA 2 query |
+
+Both default to `2000-01-01 00:00:00` if not yet set (full initial sync).
+
+## TOGA 2 → TOGaDesk sync (`isEnabledToga2ToTogadeskIntegration`)
+
+Runs when this flag is `true`. Processes 200 records per page.
+
+**Step 1 — Changed tickets:**
+- `GET /tickets?where[_updated > TOGADESK_LAST_TICKET_INTEGRATION_DATETIME]` (paged)
+- Each ticket re-fetched at `depth=5` for full details
+- Calls `syncToga2TicketIntoTogadesk1Ticket($togaTicket)`
+
+**Step 2 — Changed ticket-notes (catches notes on already-processed tickets):**
+- `GET /ticket-notes?where[_updated > TOGADESK_LAST_TICKET_INTEGRATION_DATETIME]` (paged)
+- Skips tickets already processed in Step 1 (tracked in `$alreadyProcessedToga2TicketUuids`)
+- Re-fetches the parent ticket at `depth=5`, calls `syncToga2TicketIntoTogadesk1Ticket`
+
+**Inside `syncToga2TicketIntoTogadesk1Ticket`:**
+- `isExternal = true` note → creates `tickets_replies` record with `referenceid = ticketNote->uuid`
+- `isExternal = false` note → creates `comments` record with `referenceid = ticketNote->uuid`
+- After creating a new reply: calls `syncToga2NoteFilesIntoTogadesk1($ticketNote, $togadeskReplyId)`
+- Also writes a `tickets_history` row for each new reply
+
+## TOGaDesk → TOGA 2 sync (`isEnabledTogadeskToToga2Integration`)
+
+Runs when this flag is `true`.
+
+**Ticket selection:**
+```sql
+SELECT tickets.id,
+    GREATEST(
+        COALESCE(latestReply.timestamp, tickets.timestamp),
+        COALESCE(latestComment.timestamp, tickets.timestamp)
+    ) AS ticketIntegrationTimestamp
+FROM tickets
+LEFT JOIN (SELECT ticketid, MAX(timestamp) FROM tickets_replies GROUP BY ticketid) AS latestReply ...
+LEFT JOIN (SELECT ticketid, MAX(timestamp) FROM comments GROUP BY ticketid) AS latestComment ...
+WHERE tickets.clientid = ? AND tickets.departmentId IN (?)
+AND GREATEST(...) > '$dtTogaLastTicketIntegration'
+```
+
+**Per ticket:** calls `syncTogaDesk1TicketIntoToga2Ticket($togadeskTicketId)`
+
+**Inside that method (replies loop):**
+- For each `tickets_replies` row without a `referenceId`: POST to `/ticket-notes` in TOGA 2
+  with `c_eliteConversationId` (for Elite) or `c_wjeConversationId` (for WJE)
+- Stores returned `ticketNotes.uuid` back into `tickets_replies.referenceId`
+- Calls `syncTogadesk1NoteFilesIntoToga2($togadeskReplyId, $ticketNoteUuid)`
+
+**`referenceid` field** in TOGaDesk (`tickets_replies.referenceId`, `comments.referenceId`)
+is the foreign key to TOGA 2 — it holds the TOGA 2 `ticketNotes.uuid`. When `referenceId`
+is already populated the row was previously synced and is skipped.
+
+## Other sync operations within `syncWithTogadesk`
+
+| Flag | What it syncs | TOGA 2 source → TOGaDesk target |
+|---|---|---|
+| `isEnabledToga2ContactsToTogadeskPeopleIntegration` | Contacts → People | `/contacts` → `people` table |
+| `isEnabledToga2ItemsUnitsToTogadeskAssetsIntegration` | Units → Assets | `/units` → `assets` table |
+| `isEnabledToga2PredefinedRepliesToTogadeskTicketsPrIntegration` | Predefined replies → TicketsPR | `/predefined-replies` → `tickets_pr` table |
+| `isEnabledToga2TicketTeamsToTogadeskGroupsIntegration` | Ticket teams → Groups | `/ticket-teams` → `groups` table |
+| `isEnabledToga2TicketCategoriesToTogadeskCategorySubcategoryItemsIntegration` | Categories → custom field | Three-level tree flattened to comma-separated string, stored in TOGaDesk `tickets_customfields` |
+
+## File sync helpers
+
+### `syncToga2NoteFilesIntoTogadesk1(object $ticketNote, int $togadeskReplyId)`
+Direction: **TOGA 2 → TOGaDesk**
+
+1. `GET /ticket-notes/{uuid}?depth=5` → reads `ticketNoteFiles` array
+2. `SELECT name FROM files WHERE ticketreplyid = $togadeskReplyId` — existing filename dedup
+3. For each new file:
+   - `GET /files/{uuid}?depth=1` → `base64_decode($fileData->data->files->data)`
+   - INSERT `App_Model_TogaDesk_File` (ticketreplyid, name) → get `$fileId`
+   - Write to disk: `/var/www/html/ontrack/desk/uploads/{fileId}-{name}`
+   - `UPDATE files SET file='{fileId}-{name}', filetype=?, filesize=? WHERE id=?`
+
+### `syncTogadesk1NoteFilesIntoToga2(int $togadeskReplyId, string $ticketNoteUuid)`
+Direction: **TOGaDesk → TOGA 2**
+
+1. `SELECT id, name, file FROM files WHERE ticketreplyid = $togadeskReplyId`
+2. `GET /ticket-notes/{uuid}?depth=5` → reads `ticketNoteFiles` — existing filename dedup
+3. For each new file:
+   - Read from disk: `/var/www/html/ontrack/desk/uploads/{file}`
+   - `POST /files` with base64-encoded data → get `$fileUuid`
+   - `POST /ticket-notes/{uuid}/ticket-note-files` with `{file: {uuid: $fileUuid}}`
+
+### `syncFreshserviceConversationAttachmentsIntoToga2(...)`
+Direction: **Freshservice → TOGA 2** — see worker2 doc.
+
+## TOGA 2 API — critical gotchas
+
+**Response key is `ticketNoteFiles`, not `ticketNotesFiles`.**
+The extra `s` causes a silent empty array — dedup never fires, files re-upload every run.
+
+**`GET /ticket-notes/{uuid}/ticket-note-files` is NOT a list endpoint.**
+`childPolicy = MATCH`: 0 records → 404 EV-6, 2+ records → 404 EV-14.
+Always use `depth=5` on the parent note for listing.
+
+**`throwExceptionOnApiError` (8th param of `App_Api_Toga2::send()`).**
+Pass `true` for the dedup GET — errors must not silently empty the dedup array.
+
+## Error handling
+
+Each ticket sync is wrapped in `try/catch` — exceptions are sent to Sentry and the run
+continues with the next ticket. A single bad ticket does not abort the whole sync.

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -5,3 +5,4 @@
 | [Worker (worker2) Architecture](architecture.md) | Worker (repo `worker2`) is an AWS Elastic Beanstalk **Worker Tier** application that processes background jobs. | worker2/Controller/Index.php, worker2/Worker/, worker2/LambdaFunctions/, _underscore/Worker.php |
 | [ClickUp Project & Opportunity Multi-List Routing](features/clickup-project-routing.md) | Routes ClickUp tasks into the correct **secondary multi-list memberships** based on their custom-field values, via the `clickup` webhook. | worker2/Worker/Clickup/Project.php, worker2/Worker/Clickup.php |
 | [Creating Worker Actions](features/creating-worker-actions.md) | How to add a new callable Worker action — a PHP class whose `public static` methods are invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). | worker2/Worker/, worker2/Controller/Index.php, _underscore/Worker.php |
+| [Elite Freshservice Sync (worker2)](features/elite-freshservice-sync.md) | `_Worker_Elite` processes Freshservice webhook events and syncs them into TOGA 2. | worker2/Worker/Elite.php, worker2/Config/dev-kmaramreddy-laptop.ini |

package/knowledge/2.0/apps/worker2/features/elite-freshservice-sync.md

@@ -0,0 +1,142 @@
+---
+title: Elite Freshservice Sync (worker2)
+framework: "2.0"
+repo: worker2
+project: Worker
+client: elite
+type: client-feature
+status: active
+updated: 2026-06-10
+owners: []
+files:
+  - worker2/Worker/Elite.php
+  - worker2/Config/dev-kmaramreddy-laptop.ini
+related:
+  - clients/elite/profile.md
+  - 1.0/apps/library/features/elite-freshservice-sync.md
+---
+
+## Summary
+
+`_Worker_Elite` processes Freshservice webhook events and syncs them into TOGA 2.
+Dispatched via action `Elite/Webhook` on the worker queue.
+
+## Key files / entry points
+
+- `Worker/Elite.php` — `_Worker_Elite` class, all sync logic lives here
+- `_Component_Api_Elite` (`_underscore`) — authenticated Freshservice HTTP client
+- `_Component_Api_Toga` (`_underscore`) — authenticated TOGA 2 HTTP client
+
+## How the webhook flow works
+
+1. Freshservice fires a webhook to `webhook.togahub.com/elite`
+2. Lambda inserts a `WorkerJobs` row with action `Elite/Webhook`
+3. Worker picks up the job → calls `_Worker_Elite::Webhook($payload, $headers)`
+4. Worker syncs the ticket and all conversations into TOGA 2
+
+## Constants
+
+```php
+const CLIENT_UUID_ELITE = '97627e9d-605f-3ded-9e83-97a4ec7a0c15';
+const API_UUID_ELITE    = 'ee08cf3d-caee-462d-adb5-acc5c36b1f01';
+const API_SECRET_ELITE  = 'c2701d21-1908-4895-aaed-2e6456710e8d';
+```
+
+## Freshservice API — gotchas
+
+**No single-conversation GET endpoint.**
+`GET /api/v2/conversations/{id}` only supports PATCH/PUT/DELETE — it returns 405 on GET.
+To fetch a specific conversation, retrieve the full list for the ticket:
+
+```php
+$conversations = _Component_Api_Elite::send('GET', '/tickets/' . $ticketId . '/conversations', null);
+foreach ($conversations->conversations as $conv) {
+    if ((string) $conv->id === $targetId) { ... }
+}
+```
+
+**Attachment URLs are presigned S3 URLs — they expire.**
+Call the attachment sync method immediately at webhook time. Do not queue the download.
+
+**Attachment shape on a conversation object:**
+```
+$attachment->name
+$attachment->content_type
+$attachment->size
+$attachment->attachment_url   // presigned S3 URL
+```
+
+## TOGA 2 API — gotchas
+
+**`childPolicy = MATCH` nested routes are get-ONE, not a list.**
+`GET /ticket-notes/{uuid}/ticket-note-files` returns:
+- 404 EV-6 when 0 bridge records exist
+- 200 object when exactly 1 exists
+- 404 EV-14 ("multiple found, expected one") when 2+ exist
+
+**Never use this route for listing or dedup.** Use `depth=5` on the parent note instead:
+
+```php
+$togaNote = _Component_Api_Toga::send(..., 'GET', '/ticket-notes/' . $uuid, null, ['depth' => 5], true, $endpoint);
+$existingFileNames = [];
+foreach (($togaNote->data->ticketNotes->ticketNoteFiles ?? []) as $link) {
+    if (!empty($link->file->name)) {
+        $existingFileNames[] = $link->file->name;
+    }
+}
+```
+
+Response key is `ticketNoteFiles` (not `ticketNotesFiles` — no `s` before `Files`).
+
+## Local dev setup
+
+To test worker2 Elite code locally against production TOGA 2 and Freshservice:
+
+1. **Dev config** — `[api]` section needs both keys (`endpoint` for some callers,
+   `_` for `_Config::api('_')`):
+   ```ini
+   [api]
+   _ = "https://api.togahub.com/v2"
+   endpoint = "https://api.togahub.com/v2"
+   ```
+
+2. **`ClientLogs` database** must exist locally — the framework logs every API call there:
+   ```sql
+   CREATE DATABASE IF NOT EXISTS ClientLogs CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci;
+   USE ClientLogs;
+   CREATE TABLE `Api` (
+     `id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
+     `uuid` CHAR(36) NOT NULL,
+     `dtStamp` DATETIME NOT NULL,
+     `transactionId` VARCHAR(64) NULL,
+     `apiId` INT UNSIGNED NULL,
+     `userId` INT UNSIGNED NULL,
+     `isAuthRequest` TINYINT UNSIGNED NOT NULL DEFAULT 0,
+     `direction` ENUM('IN','OUT') NOT NULL,
+     `source` VARCHAR(255) NULL,
+     `sourceIp` VARCHAR(45) NULL,
+     `instanceId` VARCHAR(128) NULL,
+     `executionTimeSeconds` DECIMAL(10,4) NULL,
+     `method` VARCHAR(10) NOT NULL,
+     `hostname` VARCHAR(255) NOT NULL,
+     `route` VARCHAR(1024) NOT NULL,
+     `responseCode` SMALLINT UNSIGNED NULL,
+     `queryString` TEXT NULL,
+     `requestHeaders` TEXT NULL,
+     `requestPayload` MEDIUMTEXT NULL,
+     `responseHeaders` TEXT NULL,
+     `responsePayload` MEDIUMTEXT NULL
+   ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
+   ```
+
+3. **Bootstrap** for a standalone test script:
+   ```php
+   chdir(__DIR__);
+   putenv('ENVIRONMENT=dev-kmaramreddy-laptop');
+   $_ENV['ENVIRONMENT'] = 'dev-kmaramreddy-laptop';
+   require_once 'vendor/autoload.php';
+   require '_underscore.php';
+   ```
+
+4. **PHP version** — worker2 vendor targets PHP >= 8.5. Local PHP 8.1 requires a
+   temporary bypass in `vendor/composer/platform_check.php` — revert before committing.

package/knowledge/INDEX.md

@@ -4,13 +4,13 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 1.0 framework
 
-- **library** (Library) _(framework core)_ — 1 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
+- **library** (Library) _(framework core)_ — 2 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
 - **worker** (Worker) — 1 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 
 ## 2.0 framework
 
 - **_underscore** (_Underscore) _(framework core)_ — 3 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
-- **worker2** (Worker) — 3 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
+- **worker2** (Worker) — 4 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 1 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 1 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 2 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
@@ -19,4 +19,5 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 - **compass-canada** → [clients/compass-canada/INDEX.md](clients/compass-canada/INDEX.md)
 - **compass-usa** → [clients/compass-usa/INDEX.md](clients/compass-usa/INDEX.md)
+- **elite** → [clients/elite/INDEX.md](clients/elite/INDEX.md)
 

package/knowledge/clients/elite/INDEX.md

@@ -0,0 +1,5 @@
+# Client: elite
+
+| Doc | Framework | Summary | Files |
+|-----|-----------|---------|-------|
+| [Elite Client Profile](profile.md) | 2.0 | Elite is a managed-services client that uses **Freshservice** as their helpdesk platform. | worker2/Worker/Elite.php, library/app/api/toga2.php |

package/knowledge/clients/elite/profile.md

@@ -0,0 +1,61 @@
+---
+title: Elite Client Profile
+framework: "2.0"
+project: Worker
+client: elite
+type: client-feature
+status: active
+updated: 2026-06-10
+owners: []
+files:
+  - worker2/Worker/Elite.php
+  - library/app/api/toga2.php
+related:
+  - 2.0/apps/worker2/features/elite-freshservice-sync.md
+  - 1.0/apps/library/features/elite-freshservice-sync.md
+---
+
+## Summary
+
+Elite is a managed-services client that uses **Freshservice** as their helpdesk platform.
+TOGA 2 is the source of truth for tickets, notes, and file attachments. The integration
+keeps Freshservice and TOGA 2 in sync via webhook-driven worker jobs.
+
+## TOGA 2 API credentials
+
+Constants live in `worker2/Worker/Elite.php`:
+
+```php
+const CLIENT_UUID_ELITE = '97627e9d-605f-3ded-9e83-97a4ec7a0c15';
+const API_UUID_ELITE    = 'ee08cf3d-caee-462d-adb5-acc5c36b1f01';
+const API_SECRET_ELITE  = 'c2701d21-1908-4895-aaed-2e6456710e8d';
+```
+
+The library version of the sync methods (`library/app/api/toga2.php`) passes these in as
+parameters — they are not constants there.
+
+## Freshservice instance
+
+- Base URL: `https://eliteservicedesk.freshservice.com`
+- Configured in `_underscore` framework via `_Component_Api_Elite`
+- API logs written to the `ClientLogs` database (`ClientLogs.Api` table) — this DB must
+  exist locally when running worker2 scripts against production Freshservice
+
+## Key custom fields
+
+| TOGA 2 field | Purpose |
+|---|---|
+| `c_eliteTicketId` | Freshservice ticket numeric ID (on `Tickets`) |
+| `c_eliteConversationId` | Freshservice conversation ID (on `TicketNotes`) |
+| `c_eliteUserId` | Freshservice user ID (on `Contacts`) |
+| `c_eliteCategoryId` | Freshservice category ID (on `TicketCategories`) |
+| `c_eliteGroupId` | Freshservice group ID (on `TicketTeams`) |
+
+## Systems involved
+
+| System | Role |
+|---|---|
+| Freshservice | Client helpdesk — source of ticket/conversation events |
+| TOGA 2 (`api2`) | System of record — tickets, notes, contacts, files |
+| worker2 `_Worker_Elite` | Webhook processor — syncs Freshservice events into TOGA 2 |
+| library `App_Api_Toga2` | Shared sync helpers (note-file bidirectional sync) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.35",
+  "version": "1.0.37",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",