toga-ai 1.0.926 → 1.0.928
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/library/features/toga2-api-client-and-bridge.md +7 -2
- package/knowledge/2.0/apps/dbchanges2/architecture.md +2 -2
- package/knowledge/2.0/apps/worker2/INDEX.md +3 -3
- package/knowledge/2.0/apps/worker2/features/clickup-design-sprint-automation.md +9 -4
- package/knowledge/2.0/apps/worker2/features/clickup-general-automation.md +47 -28
- package/knowledge/2.0/apps/worker2/features/clickup-work-type-automation.md +32 -8
- package/knowledge/2.0/apps/worker2/features/committed-secrets-and-config-debt.md +14 -3
- package/knowledge/2.0/apps/worker2/features/netsuite-opportunity-sync.md +7 -1
- package/knowledge/2.0/apps/worker2/features/oneuptime-worker2-monitoring.md +11 -1
- package/knowledge/2.0/apps/worker2/features/qa-qc-review-pipeline.md +52 -16
- package/knowledge/2.0/apps/worker2/features/team-sprint-management.md +92 -69
- package/knowledge/2.0/apps/worker2/features/wje-freshservice-sync.md +9 -4
- package/knowledge/2.0/apps/worker2/workflows/sprint-clickup-folder-repoint.md +26 -15
- package/knowledge/INDEX.md +1 -1
- package/knowledge/clients/wje/INDEX.md +2 -1
- package/knowledge/clients/wje/features/oneuptime-ticket-writes-monitor.md +80 -0
- package/knowledge/clients/wje/profile.md +19 -5
- package/knowledge/standalone/apps/claude/workflows/mcp-tool-usage.md +4 -2
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@ project: Library
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-01
|
|
10
10
|
owners: [jcardinal, mhammontree, bala]
|
|
11
11
|
files:
|
|
12
12
|
- library/app/api/toga2.php
|
|
@@ -23,6 +23,7 @@ related:
|
|
|
23
23
|
- ../../worker/features/netsuite-togasupply-per-client-sync.md
|
|
24
24
|
- ../architecture.md
|
|
25
25
|
- ./error-capture-1-0.md
|
|
26
|
+
- ../../../clients/wje/features/oneuptime-ticket-writes-monitor.md
|
|
26
27
|
- ../../../clients/elite/features/togadesk-service-request-intake.md
|
|
27
28
|
---
|
|
28
29
|
|
|
@@ -145,7 +146,9 @@ A large bi-directional sync gated by **two checkpoint parameters** stored in 2.0
|
|
|
145
146
|
|
|
146
147
|
Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven independent enable flags** and an optional `$monitorTogadeskDepartmentIds[]`:
|
|
147
148
|
- **2.0 → TOGaDesk** (`$isEnabledToga2ToTogadeskIntegration`): pages 2.0 `/tickets` and `/ticket-notes` changed since the checkpoint; routes each to `syncToga2TicketIntoTogadesk1{RepairOrder|Ticket}` by `$integrationType`. Per-ticket exceptions are caught and sent to Sentry so one bad ticket doesn't abort the batch.
|
|
148
|
-
- **TOGaDesk → 2.0** (`$isEnabledTogadeskToToga2Integration`): queries the 1.0 `db_togadesk` DB directly (SQL over `repair_orders`/`repair_order_history` or `tickets`/`tickets_replies`/`comments`, using `GREATEST(...)` of reply/comment timestamps) for the client's department(s)
|
|
149
|
+
- **TOGaDesk → 2.0** (`$isEnabledTogadeskToToga2Integration`): queries the 1.0 `db_togadesk` DB directly (SQL over `repair_orders`/`repair_order_history` or `tickets`/`tickets_replies`/`comments`, using `GREATEST(...)` of reply/comment timestamps) for the client's department(s) — only the ones the cron passes (`$monitorTogadeskDepartmentIds`) — then calls `syncTogaDesk1{RepairOrder|Ticket}IntoToga2Ticket($id)`.
|
|
150
|
+
- **Link = `referenceId`.** `tickets.referenceId` = 2.0 ticket uuid; `tickets_replies.referenceId` / `comments.referenceId` = 2.0 ticket-note uuid. Written back **only after a successful POST**. Rows created by the 2.0 → 1.0 direction already carry it. So NULL/`''` `referenceId` = not (yet) in 2.0 — the basis of the [WJE ticket-writes monitor](../../../clients/wje/features/oneuptime-ticket-writes-monitor.md).
|
|
151
|
+
- A reply with a blank message falls back to `newstatus`; if still PHP-empty (NULL, `''`, `'0'`) it is skipped and never gets a `referenceId`. Comments are always sent.
|
|
149
152
|
- Five further one-way 2.0 → TOGaDesk sub-syncs, each its own flag: Contacts→People, Items/Units→Assets, Predefined-Replies→TicketsPr, Ticket-Teams→Groups, Ticket-Categories→Category/Subcategory/Items.
|
|
150
153
|
- Finishes by PUT-ing both checkpoint parameters back to 2.0.
|
|
151
154
|
|
|
@@ -167,6 +170,7 @@ Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven i
|
|
|
167
170
|
- **Two unrelated `sync_togasupply_<client>.php` families.** Files under `worker/crons/toga2/netsuite/` are the **NetSuite supply importer** (`require` the common engine). Files under `worker/crons/toga2/<client>/` (e.g. `aig/`, `wje/`) are this **bridge** (`syncWithToga`/`syncWithTogadesk`). Same filename, different integration, different parameter keys — they coexist harmlessly. Confirm which folder before editing.
|
|
168
171
|
- **`syncWithToga` and the bridge build SQL by hand** with `App_Database::sqlEscape()` rather than the `App_Model` layer; follow the surrounding escaping discipline when modifying.
|
|
169
172
|
- **Per-record error isolation** in the TOGaDesk sync is via `try/catch` → `\Sentry\captureException`; a thrown exception elsewhere (transport, checkpoint read) still aborts the whole run.
|
|
173
|
+
- **⚠ A failed ticket is never retried.** After a per-ticket exception the run still advances `TOGA_LAST_TICKET_INTEGRATION_DATETIME` past it, so the row stays unsent until someone touches the ticket again (new reply) or the checkpoint is moved back. A checkpoint-freshness check reads healthy through this — monitor NULL `referenceId` instead.
|
|
170
174
|
- **Naming `fields` can SUPPRESS a nested relation that a plain request returns.** Observed 2026-08-12 on `GET /states`: a bare request comes back with the nested `country`, but adding a `fields` list drops it. This is the mirror image of the reverse-hasMany rule below — when a nested object you expected is missing, suspect the field list before the data.
|
|
171
175
|
- **A reverse hasMany collection is only returned if you add it to the fetch `fields` whitelist.** The `/contacts` fetch requests an **explicit** `fields` list from the 2.0 metadata API; a reverse hasMany collection (e.g. `contactEmailAddresses.emailAddress`) is **not** returned unless added — so consuming a related collection means editing **both** the fetch field list **and** the consuming code. (The general 2.0-side metadata behavior — reverse relations resolved via the child model's `FOREIGNKEY_MODEL` and named as the camelCase plural of the child model — is a framework-level fact captured separately.)
|
|
172
176
|
- **⚠ Depth alone does NOT return every reverse-hasMany collection — `GET /sales-orders` at depth 4 omits `itemFulfillmentItems` and `invoiceItems`.** Verified 2026-08-17 against prod `Logs_Compass.Api` responses (no fields whitelist): a depth-4 sales-order GET returns `salesOrderItems` and the **PO-link bridge** collections (`salesOrderItemPurchaseOrderItems`, `purchaseOrderItemSalesOrderItems`) but **not** `itemFulfillmentItems` or `invoiceItems`. So any importer code that guards on `$salesOrderItem->itemFulfillmentItems` / `->invoiceItems` from that GET is a **silent no-op** — same reverse-hasMany trap as above, with a concrete consequence: the NetSuite SO renumber fix keys off a recorded "could-not-delete" set instead of trusting those collections (see [per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md)).
|
|
@@ -180,6 +184,7 @@ Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven i
|
|
|
180
184
|
- **Pre-existing (low severity, out of scope): a raw-vs-escaped comparison causes a redundant UPDATE every sync.** At the update path (~line 8584) `syncContactToToga1Customer` compares `$existingCustomer['emailAddress'] !== $emailAddress` — the DB value from `App_Database::fetchRow()` is **unescaped**, while `$emailAddress` has already been through `App_Database::sqlEscape()`. For any value containing `'`, `"` or `\` the two can **never** compare equal, so that column is rewritten on **every** run. Same pattern applies to `businessName` and `phoneNumber` just above. **Predates TRUE-79401**, is a redundant write (not data loss), and was deliberately left out of that ticket's scope — recorded here so a future reader knows it is known.
|
|
181
185
|
|
|
182
186
|
## Change history
|
|
187
|
+
- 2026-10-01 — Documented the `referenceId` link/write-back, blank-reply skip rule, and that a per-ticket failure advances the checkpoint (never retried). (mhammontree)
|
|
183
188
|
- 2026-09-21 — Recorded a pre-existing committed JWT (dev/NYCHH token) at `toga2.php:641` inside `send()` (the sync call path); flagged to rotate + delete, location-only. (jcardinal)
|
|
184
189
|
|
|
185
190
|
## Related
|
|
@@ -6,7 +6,7 @@ project: Database Changes
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: architecture
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-01
|
|
10
10
|
owners: [jcardinal, mhammontree, bala, ajean]
|
|
11
11
|
files:
|
|
12
12
|
- Core/
|
|
@@ -238,7 +238,7 @@ This lifecycle is automated by a local browser tool — see the [TOGa 2.0 Client
|
|
|
238
238
|
|
|
239
239
|
**Every UUID must be freshly, fully random — never derive one from another.** Do **not** take a known uuid and bump a digit (`…-0001`, `…-0002`, …), copy-and-tweak, or otherwise pattern them. When a file needs several literals, generate each one independently at random. (Patterned uuids are guessable and defeat the uniqueness guarantee.) The one intended exception is a fan-out `Client/` grant, which **reuses the same** literal per row across tenants on purpose for safe rollback — deliberate reuse, not incrementing (see the feature doc).
|
|
240
240
|
|
|
241
|
-
One earlier migration (`Core/2026-05-08 … CronJobs_WorkerCleanup_insert`) used `UUID()` — that is a known violation, not a precedent to copy. First applied correctly in `Core/2026-
|
|
241
|
+
One earlier migration (`Core/2026-05-08 … CronJobs_WorkerCleanup_insert`) used `UUID()` — that is a known violation, not a precedent to copy. First applied correctly in `Core/2026-09-29a - Record sprint and design CronJobs.sql` (new rows 83–86 carry pre-generated v4 literals; the recorded prod rows keep their existing prod uuids, which predate this rule).
|
|
242
242
|
7. **Guard every registration/seed insert with `NOT EXISTS` (or `INSERT IGNORE`).** Rows that register platform behavior — `ApiPayloadInterceptors`, `Core.RecordFields`/`CustomRecordFields`, cron jobs, ACL rows — must be inserted only if not already present. An **unguarded** registration is not merely non-idempotent: on a fresh environment it can run **alongside a second migration that registers the same behavior**, producing a **double registration** (e.g. two interceptor rows for the same record → the interceptor fires twice). A real instance: the Rate WH `prePost` interceptor is registered unguarded in `Client_Rate/2026-07-15 - PreInterceptor.sql` (record 191, `minDepth NULL`); a second guarded registration would have double-fired it. See `clients/rate/features/whole-home-warranty-purchase-guard.md`.
|
|
243
243
|
8. **Never reference a database other than the folder's own target.** No `Core.Table` in a `Client_<Name>/` file, no `Client_X.Table` in a `Core/` file, and **no database qualifier at all** in the fan-out folders (`Client/`, `Logs_Client/`, `_modules/`). These databases are on **separate production clusters**; the query works locally and dies in production. Resolve foreign ids with a hardcoded v4 UUID literal or an in-database slug/natural key, or split the work into one file per database folder. See *Database isolation* above — enforced by the `dbchanges2-cluster-isolation` hook.
|
|
244
244
|
9. **Inserting into `Core.Records` or `Core.RecordFields`? ASK the developer for the next `id`** (the developer takes it in Tools → Developers → Central IDs — the source of truth; never guess from MAX(id), branches hold unreleased ids). These are the **only two tables on the platform with team-maintained primary keys**. Write the assigned `id` as an explicit literal — **never** let `AUTO_INCREMENT` assign it. Conversely, **always hardcode** these `id`s where they are referenced, including in foreign keys (`recordId`, `recordFieldId`) from other databases. See *`Core.Records` / `Core.RecordFields`* above.
|
|
@@ -11,14 +11,14 @@
|
|
|
11
11
|
| [ClickUp Connectivity Watchdog](features/clickup-connectivity-watchdog.md) | Cron watchdog that emails when the ClickUp integration goes silent in business hours; open when touching ClickUp webhook health or `_Worker_Clickup_Health`. |
|
|
12
12
|
| [ClickUp Custom-Field Conventions (constant IDs, runtime option values, Task Type migration)](features/clickup-custom-field-conventions.md) | How worker2 code must reference ClickUp custom fields; open before reading/writing any ClickUp custom field or touching the Task Type migration. |
|
|
13
13
|
| [ClickUp Design Sprint Automation (Final Design Outcome)](features/clickup-design-sprint-automation.md) | `_Worker_Clickup_Design` drives the design-sprint workflow in ClickUp (FDO default on create, spawn iteration / complete task on Final Design Outcome, daily rev |
|
|
14
|
-
| [ClickUp General Automation (Sub-Environment reservation, Has UI / Has Code
|
|
14
|
+
| [ClickUp General Automation (Sub-Environment reservation, Has UI / Has Code, review-field reset)](features/clickup-general-automation.md) | `_Worker_Clickup_GeneralAutomation` keeps QA/QC fields consistent on ClickUp field/status changes (sub-environment reservation, Has UI / Has Code, review-field |
|
|
15
15
|
| [ClickUp GitHub-tab Auto-linking & Ticket-id Branch Naming](features/clickup-github-autolink.md) | How ClickUp surfaces branches/PRs/commits in a ticket's GitHub tab and the branch/PR-title naming that triggers it; open when naming branches for ticket work or |
|
|
16
16
|
| [ClickUp Project & Opportunity Multi-List Routing](features/clickup-project-routing.md) | Routes ClickUp tasks into secondary multi-list memberships by custom-field values; open for opportunity/epic routing, the stage-is-a-list constraint, or webhook |
|
|
17
17
|
| [ClickUp Rich-Text Custom Fields via Quill Delta (API)](features/clickup-richtext-api.md) | How to write formatted text into a ClickUp text/long-text custom field via the undocumented `value_richtext` Quill Delta key; open when a richtext ClickUp field |
|
|
18
18
|
| [ClickUp Subtask Activity → Parent Opportunity/Epic Comments](features/clickup-subtask-activity.md) | Mirrors subtask status/comment activity up to its top-level Opportunity/Epic as one comment; open for the subtask fan-out, its dedup ledger, or structural ances |
|
|
19
19
|
| [ClickUp Task Description Fluffer (Talos KB Investigation)](features/clickup-task-fluffer-talos.md) | Async worker that "fluffs" a new ClickUp task by running its name+description through the Talos `dev-core` KB agent and writing the report back; open for the fl |
|
|
20
20
|
| [ClickUp Webhook Health Monitor → OneUptime (Monitor/Clickup/CheckWebhook)](features/clickup-webhook-health-monitor.md) | worker2 cron that checks the team's ClickUp webhook subscriptions and pages via a OneUptime Incoming-Request monitor; open when tuning the ClickUp webhook alarm |
|
|
21
|
-
| [ClickUp Work Type Automation (Committed / Conditional / Stretch)](features/clickup-work-type-automation.md) | Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date, and sets
|
|
21
|
+
| [ClickUp Work Type Automation (Committed / Conditional / Stretch)](features/clickup-work-type-automation.md) | Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date, and sets sprint-list defaults (Work Ty |
|
|
22
22
|
| [worker2 Codeception integration tests (2.0 test-framework pilot)](features/codeception-integration-tests.md) | First 2.0 app-level automated test suite: Codeception 5 / PHPUnit 11 in worker2 (pilot, Jeff's direction), real-DB integration tests with hard local-only safety |
|
|
23
23
|
| [worker2 — Committed Secrets & Config Exposure (debt register)](features/committed-secrets-and-config-debt.md) | Standing list of credentials committed to the worker2 repo (locations only); open before adding a credential to worker2 or assuming a config value is private. |
|
|
24
24
|
| [Compass VIP Support Importer (worker2)](features/compass-vip-support-importer.md) | worker2 action that ingests Compass's quarterly VIP spreadsheet and sets each VIP's support tech (`Users.c_supportedByUserId` in `Client_Compass`); open when wo |
|
|
@@ -60,7 +60,7 @@
|
|
|
60
60
|
| [Talos Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | Consumer side of Talos: the endpoints, DevCore assistant, and JWT auto-refresh a worker2 dev tool needs to query the meeting-notes corpus; open when a tool must |
|
|
61
61
|
| [Talos Pricing Automation (worker2 Cron — Usage Import, AWS Actuals, Margins, Profiles)](features/talos-pricing-automation.md) | Monthly worker2 crons that turn raw usage + AWS cost into a measured platform unit price, per-client cost, and margins; open for the pricing pipeline, its AWS c |
|
|
62
62
|
| [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` cron pipeline ingesting Teams VTT transcripts into Talos Bedrock KBs + emailing organizer recaps; open for Export/Process/Retry/SyncK |
|
|
63
|
-
| [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` engine for TOGA's biweekly dev-sprint process + reporting (warnings, lock, daily dashboard, sprint-end scoring, AI release notes
|
|
63
|
+
| [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` engine for TOGA's biweekly dev-sprint process + reporting (warnings, lock, daily dashboard, sprint-end scoring, AI release notes, 7 PM In |
|
|
64
64
|
| [Teams Meeting Transcript Export](features/teams-transcript-export.md) | How `Team/Transcripts/Export` polls Microsoft Graph for Teams transcripts; open for the Graph organizer-resolution, GUID-vs-UPN, and Entra-permission details (t |
|
|
65
65
|
| [Centralized Tracking-Status Refresh (worker2 Sync cron, all clients)](features/tracking-status-refresh.md) | Platform-wide worker2 cron polling FedEx/USPS/UPS to keep `TrackingNumbers.status` current for every provisioned client via api2 PUT; open for the dispatcher/cu |
|
|
66
66
|
| [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | worker2 handler for VAPI's end-of-call report (writes results onto the `ContactAttempt`, dispatches the post-call action) + the outbound dialer eligibility rule |
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-01
|
|
10
10
|
owners: ["ajean", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Clickup/Design.php
|
|
@@ -24,7 +24,7 @@ related:
|
|
|
24
24
|
|
|
25
25
|
`_Worker_Clickup_Design` replaces a set of native ClickUp automations. When a task's **Final Design Outcome** (FDO) drop-down is set to an actionable value, the worker spawns an iteration task or completes the task; a daily cron reminds the Design Approver of pending reviews.
|
|
26
26
|
|
|
27
|
-
**Current state (2026-
|
|
27
|
+
**Current state (2026-10-01):** live on Design Sprints `90116585900` (`DESIGN_SPRINTS_FOLDER_IDS`) + Backlog Design lists. The old True Design Sprints folder `90115939842` was **removed** from the gate (old folders get no automation; open tasks there are closed by hand). FDO-on-create, approve→complete and iterate→spawn verified working since Jeff's 9/25 deploy; failures reported after that predated it. History of the 2026-07 no-op / no-spawn fixes (worker2 PR #98, #113 + _underscore PR #669) is in Gotchas. The native ClickUp automations masking the original no-op must stay disabled.
|
|
28
28
|
|
|
29
29
|
## How it works
|
|
30
30
|
|
|
@@ -47,7 +47,7 @@ related:
|
|
|
47
47
|
- `Worker/Clickup.php` — routes ClickUp webhooks.
|
|
48
48
|
- `shouldEmailDesignApprover($payload, $taskDetails)` — the single predicate gating the "Action Needed: Design Review" email (requires history_items[0] is a Design Approver custom_field change, task status still `iterated`, and user-driven: `user.id != -1`).
|
|
49
49
|
- `taskUpdated` path (~822) and `taskStatusUpdated` path (~230) both gate the email on this predicate.
|
|
50
|
-
- `Controller/ClickupDesignTest.php` — regression tests reached via the `?controller=ClickupDesignTest` route (worker2 has **no PHPUnit harness**; mirrors the `CompassPeopleTest` convention). Uses `ReflectionMethod` to reach the private methods.
|
|
50
|
+
- `Controller/ClickupDesignTest.php` — regression tests (updated 2026-10-01: the new-task case now expects `in progress`) reached via the `?controller=ClickupDesignTest` route (worker2 has **no PHPUnit harness**; mirrors the `CompassPeopleTest` convention). Uses `ReflectionMethod` to reach the private methods.
|
|
51
51
|
|
|
52
52
|
### Flow
|
|
53
53
|
|
|
@@ -80,7 +80,7 @@ Configuration lives as ClickUp identifiers (folder IDs, custom-field IDs, drop-d
|
|
|
80
80
|
- **Never use strict `===` when reading a ClickUp `drop_down` value/orderindex/id (ROOT CAUSE of the silent no-op).** ClickUp returns `value`/`orderindex` with **inconsistent int-vs-string typing** (`"2" !== 2`), so `resolveFinalDesignOutcomeOptionId` matched nothing, returned `null`, and `handleFinalDesignOutcome` hit its early return (job still `isSuccess=1`). Fixed by comparing **as strings**: `(string)$option->orderindex === (string)$field->value` (and the same for `$option->id`). **Every other** drop_down reader in `Worker/Clickup.php` relies on PHP **array-key coercion** (`$optionsByIndex[$option->orderindex]` then indexed by `$customField->value`) precisely because of this typing inconsistency — strict `===` anywhere reading a ClickUp drop_down value is a bug. The native automation `c25fd345` had been masking this no-op in prod.
|
|
81
81
|
- **`taskStatusUpdated` switches on `before->status`, NOT `after->status`** (Clickup.php ~71). A `case '<status>'` in that switch fires when a task **LEAVES** that status — the in-code comment claiming "just changed TO iterated" is misleading. This is the trap behind the second spurious-email vector: the `iterated` case fired on `completeTask()`'s `iterated → complete` transition during `spawnIterationTask`. Anyone adding status-transition logic must account for this direction. The "entering iterated" trigger added this session is correctly placed under `switch ($historyItem->after->status)` (`case ...::STATUS_ITERATED` → `handleTaskEnteredIterated()`) — distinct from the `before->status` switch.
|
|
82
82
|
- **Spurious "Design Review" email — fixed on a single predicate.** Both vectors (the `taskUpdated` Design-Approver-field path ~822 and the `taskStatusUpdated` `iterated` path ~230) now gate on `shouldEmailDesignApprover($payload, $taskDetails)`, which requires: `history_items[0]` is a Design Approver custom_field change, task status still `iterated`, and the change is **user-driven** (`user.id != -1`; ClickBot/automation writes carry `user.id == -1`). The dead `$isDesignApproverSet` block was removed. Consequence: a pure status change **into** iterated (approver already set, no field change) no longer emails from the status path — the legitimate review email flows from the `taskUpdated` field handler.
|
|
83
|
-
- **Native automations must stay disabled once the worker is verified.** The worker replaces: `c25fd345`, `272f743e`, `a64c863a`, `7739cc41`, `08570f31`, `70b09962`. Keep sprint-readiness automations only: `10df1786`, `f6214678
|
|
83
|
+
- **Native automations must stay disabled once the worker is verified.** The worker replaces: `c25fd345`, `272f743e`, `a64c863a`, `7739cc41`, `08570f31`, `70b09962`. Keep sprint-readiness automations only: `10df1786`, `f6214678` — these existed only on the old folder and **must be rebuilt by hand on Design Sprints** (open as of 2026-10-01). Re-enabling any of the replaced set reintroduces the duplicate+close and spurious-email behavior.
|
|
84
84
|
- **The spawn gate must admit Backlog Design lists — folder + name, not just the sprint folder (ROOT CAUSE of the 2026-07-15 intermittent no-spawn).** `isDesignSprintTask()` originally accepted only the True Design Sprints folder or two hardcoded `DESIGN_LIST_IDS`, so a design task in a **Backlog Design list** was silently skipped (no iteration spawned). The gate now also admits `BACKLOG_FOLDER_ID = '90116192732'` when the list name matches `DESIGN_LIST_NAME_PATTERN`. **Regex trap:** the pattern is `/(?<!non-)\bdesign\b/i` — a bare `\bdesign\b` wrongly matched **"Non-Design Overflow"** because the hyphen is itself a word boundary, so the negative-lookbehind `(?<!non-)` is required; the `\b` boundary also correctly excludes **"Redesign"**. `SendReviewReminders()` applies the same folder+pattern so new backlog Design lists get reminders without a code change.
|
|
85
85
|
- **Resolve the FDO from the webhook `after` value, NOT the per-process snapshot (read-after-write race — second root cause of the 2026-07-15 failure).** `handleFinalDesignOutcome` used to resolve the outcome from the cached `getTaskDetails` snapshot fetched early in the request, which can **predate** ClickUp reflecting the drop-down write → it read the OLD value → silent skip. The webhook's authoritative new value (`history_items[].after`) is now threaded through `handleFinalDesignOutcome(..., ?string $webhookFdoValue)` → `resolveFinalDesignOutcomeOptionId(..., ?string $overrideValue)` (override wins over the snapshot value; the option *list* is still read from the snapshot). `_Worker_Clickup::extractCustomFieldAfterValue()` normalizes the `after` shape and — critically — distinguishes a **missing** `after` (→ `null` → fall back to snapshot) from an **explicit null** `after` (→ `''` sentinel → field was cleared → resolve to no outcome; do NOT act on the stale previous option). The previously-silent resolver-null return now logs a `[ClickupDesign]` breadcrumb.
|
|
86
86
|
- **`_Component_Api_Clickup::send()` throws on transient failures — it no longer silently returns NULL (shared-core, `_underscore` PR #669).** The old retry loop never actually retried and returned NULL on HTTP `>= 502`, **silently dropping** the API call. Now transient failures — HTTP 502/503/504, 429, and transport-level errors (`responseCode === 0` / cURL failure) — **throw** so the worker job fails loudly and the **queue** retries; other non-2xx still throw. Retry is intentionally left to the queue (no in-worker sleep-and-retry; CTO Option C). **It throws the base `Exception`** — the same type every other `_Component_Api` client throws and the framework error handler / Sentry already track — NOT a bespoke exception class. (A `_Exception_Api_Transient` subclass was prototyped and dropped precisely because a new type sits outside the existing Sentry/error path.) Exception messages never embed the full `$response` body (task/user PII); the private `summarizeApiError()` surfaces only ClickUp `err`/`ECODE` with a bounded/truncated fallback. Affects **every** caller of `send()` across the 2.0 apps — grep `_Component_Api_Clickup::send(` before assuming a caller tolerated the old null-on-failure return.
|
|
@@ -90,3 +90,8 @@ Configuration lives as ClickUp identifiers (folder IDs, custom-field IDs, drop-d
|
|
|
90
90
|
|
|
91
91
|
- [ClickUp Work-Type Automation](./clickup-work-type-automation.md)
|
|
92
92
|
- [team-sprint-management](./team-sprint-management.md)
|
|
93
|
+
|
|
94
|
+
## Change history
|
|
95
|
+
- 2026-10-01 — Old True Design Sprints folder `90115939842` dropped from `DESIGN_SPRINTS_FOLDER_IDS`; natives 10df1786/f6214678 to rebuild on Design Sprints (ajean; worker2 PR #178)
|
|
96
|
+
- 2026-09-21 — Multi-folder gate (commit `6dcebe0`) (jcardinal)
|
|
97
|
+
- 2026-07-15 — Backlog Design lists + webhook `after` value fix (worker2 #113, _underscore #669) (ajean)
|
|
@@ -1,59 +1,78 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: ClickUp General Automation (Sub-Environment reservation, Has UI / Has Code
|
|
2
|
+
title: ClickUp General Automation (Sub-Environment reservation, Has UI / Has Code, review-field reset)
|
|
3
3
|
framework: "2.0"
|
|
4
4
|
repo: worker2
|
|
5
5
|
project: Worker
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
|
-
status:
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [jcardinal]
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["jcardinal", "ajean"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Clickup/GeneralAutomation.php
|
|
13
13
|
- worker2/Worker/Clickup.php
|
|
14
|
+
- worker2/Worker/Github.php
|
|
14
15
|
- _underscore/Component/Api/Clickup/Clickup.php
|
|
15
16
|
related:
|
|
16
17
|
- ./qa-qc-review-pipeline.md
|
|
17
18
|
- ./clickup-work-type-automation.md
|
|
18
19
|
- ./clickup-custom-field-conventions.md
|
|
20
|
+
- ./team-sprint-management.md
|
|
19
21
|
---
|
|
20
22
|
|
|
21
|
-
`_Worker_Clickup_GeneralAutomation`
|
|
23
|
+
`_Worker_Clickup_GeneralAutomation` keeps QA/QC fields consistent on ClickUp field/status changes (sub-environment reservation, Has UI / Has Code, review-field reset to Needs Review); open when one of those fields is wrong or you add a QA/QC field rule.
|
|
22
24
|
|
|
23
25
|
## Summary
|
|
24
26
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
1. **Sub-Environment auto-reservation** — when a Task needs a reserved environment.
|
|
29
|
-
2. **Has UI / Has Code roll-up** to the parent Feature.
|
|
30
|
-
3. **Has Code backfill** on "Development Finished" when a PR exists.
|
|
27
|
+
Action `Clickup/GeneralAutomation/Process`, fanned out from `_Worker_Clickup::Webhook` on
|
|
28
|
+
`taskUpdated` / `taskStatusUpdated`. Implements the "General Automation" section of the team QA/QC
|
|
29
|
+
plan as seven numbered rules (listed in the class docblock). Live in prod; Dev Sprints folder scope.
|
|
31
30
|
|
|
32
31
|
## How it works
|
|
33
32
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
33
|
+
### The rules (`GeneralAutomation.php`)
|
|
34
|
+
|
|
35
|
+
1. **Client Approval Required** checked → check **Reserve Sub-Environment**.
|
|
36
|
+
2. **Task Type = Feature** → check Reserve Sub-Environment.
|
|
37
|
+
3. Reserve Sub-Environment checked → claim the next free Greek letter from `SubEnvironments` with a
|
|
38
|
+
**guarded `UPDATE` + retry** (never select-then-update) and set **Sub-Environment**.
|
|
39
|
+
4. Reserve Sub-Environment unchecked → clear Sub-Environment, release the row.
|
|
40
|
+
5. A subtask's **Has UI / Has Code Changes** → roll up onto the parent Feature (any child = parent).
|
|
41
|
+
6. Status → **Development Finished**, a PR exists, Has Code Changes unchecked → check it.
|
|
42
|
+
7. Status **enters** `ui review` / `front-end review` / `back-end review` → reset the matching field
|
|
43
|
+
**UI Review** (`c08e3ca2…`) / **Front-end Review** (`e21c888a…`) / **Back-End Review**
|
|
44
|
+
(`227480b6…`) to **Needs Review**. Dev Sprints only, not Feature tasks (`FeatureReview` owns
|
|
45
|
+
those); skipped if already Needs Review. `QaReview` / `FeatureReview` treat Needs Review as a
|
|
46
|
+
no-op, so the reset does not loop.
|
|
47
|
+
|
|
48
|
+
### PR → "Has Code Changes"
|
|
49
|
+
|
|
50
|
+
- `hasPullRequest(string $customTaskId)` (public) — **exact** match on `PullRequests.title`, which
|
|
51
|
+
stores only the task custom id (`_Worker_Github::prTitleToCustomId`). A `LIKE` prefix match was a
|
|
52
|
+
false-positive risk (`TRUE-123` matched `TRUE-1234`). Shared with
|
|
53
|
+
`_Worker_Clickup::handleDevelopmentFinished`, which checks for a PR itself so it does not race rule 6.
|
|
54
|
+
- `Worker/Github.php` `setHasCodeChangesFromPr()` — checks Has Code Changes when a PR row is **first
|
|
55
|
+
recorded** (task resolved via `Team.Tasks` customClickupIdentifier → clickupIdentifier). Best-effort,
|
|
56
|
+
never throws: the GitHub webhook must keep returning 200.
|
|
57
|
+
|
|
58
|
+
Sprint-list defaults set on create / move (Condition, Has UI Changes, Work Type) live in
|
|
59
|
+
[ClickUp work-type automation](./clickup-work-type-automation.md#sprint-list-defaults-taskcreated--taskmoved).
|
|
49
60
|
|
|
50
61
|
## Gotchas
|
|
51
62
|
|
|
52
|
-
- **Reserve the sub-environment with a guarded `UPDATE` + retry, never select-then-update.**
|
|
53
|
-
|
|
63
|
+
- **Reserve the sub-environment with a guarded `UPDATE` + retry, never select-then-update.**
|
|
64
|
+
Concurrent webhooks would hand the same letter to two tasks.
|
|
65
|
+
- **Field IDs are constants, option VALUES are resolved at runtime** — see
|
|
66
|
+
[ClickUp custom-field conventions](./clickup-custom-field-conventions.md).
|
|
67
|
+
- **Review fields always reset on entry (owner decision 2026-10-01).** Do not "keep" an old
|
|
68
|
+
Approved/Denied when a task re-enters a review status.
|
|
54
69
|
|
|
55
70
|
## Related
|
|
56
71
|
|
|
57
72
|
- [QA/QC review pipeline](./qa-qc-review-pipeline.md)
|
|
58
73
|
- [ClickUp Work-Type Automation](./clickup-work-type-automation.md)
|
|
59
74
|
- [ClickUp custom-field conventions](./clickup-custom-field-conventions.md)
|
|
75
|
+
|
|
76
|
+
## Change history
|
|
77
|
+
- 2026-10-01 — Rule 7 review-field reset; Has Code Changes on first PR record; `hasPullRequest()` exact match (ajean; worker2 PR #178)
|
|
78
|
+
- 2026-09-16 — Initial rules 1–6 (jcardinal)
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["jcardinal"]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["jcardinal", "ajean"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Clickup.php
|
|
13
13
|
- worker2/Tests/Worker/ClickupWorkTypeTest.php
|
|
@@ -16,7 +16,7 @@ related:
|
|
|
16
16
|
- ../workflows/clickup-credential-rotation.md
|
|
17
17
|
---
|
|
18
18
|
|
|
19
|
-
Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date, and sets
|
|
19
|
+
Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date, and sets sprint-list defaults (Work Type, Condition, Has UI Changes) on create/move; open when work-type promotion/demotion or those default writes misbehave.
|
|
20
20
|
|
|
21
21
|
## Summary
|
|
22
22
|
|
|
@@ -42,12 +42,29 @@ Flow:
|
|
|
42
42
|
- `Stretch` with a due date inside the current sprint → set Committed/Conditional as above.
|
|
43
43
|
6. When a task is depended on by another (`$dependency->depends_on == $taskId`), it recurses into the blocking direction so completing a task re-evaluates the tasks it was blocking. The `$dependsOnTaskId` argument breaks two-node cycles.
|
|
44
44
|
|
|
45
|
-
###
|
|
45
|
+
### Sprint-list defaults (`taskCreated` + `taskMoved`)
|
|
46
46
|
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
47
|
+
Applied by `applySprintEntryDefaults()` when a task is **created in**, or **moved into**, a list whose
|
|
48
|
+
name starts with `Sprint` inside the Dev Sprints or Design Sprints folder
|
|
49
|
+
(`isInDevOrDesignSprintsFolder()`). Backlog and other lists get nothing.
|
|
50
|
+
|
|
51
|
+
- **Condition** = On Track if empty (`defaultConditionOnTrack()`), then **Previous Sprint Condition**
|
|
52
|
+
= Condition (`CLICK_UP_CUSTOM_FIELD_ID__PREVIOUS_CONDITION`).
|
|
53
|
+
- **Has UI Changes** checked (Dev Sprints only).
|
|
54
|
+
- **Work Type** = Unplanned if empty (Dev Sprints only; `defaultWorkTypeUnplanned()`, option resolved
|
|
55
|
+
by name).
|
|
56
|
+
- **Previous Sprint Work Type** = the Work Type name (or Unplanned) — **only on sprint lists** (it used
|
|
57
|
+
to fire on Backlog too).
|
|
58
|
+
- Every optional write goes through private `setFieldSafely($taskId, $fieldLabel, callable)`:
|
|
59
|
+
try/catch → `error_log` + Sentry, so one refused field never aborts the job (the later
|
|
60
|
+
Clickup/Fluffer/Process dispatch still runs). Dropdowns are set by option name
|
|
61
|
+
(`setDropdownByName()`).
|
|
62
|
+
- Design FDO default is dispatched from `taskCreated` too — see
|
|
63
|
+
[design sprint automation](./clickup-design-sprint-automation.md).
|
|
64
|
+
|
|
65
|
+
**Dev sprint data ingestion** (sprint tag, `ensureTaskExistsInCurrentSprint`, dependencies) requires
|
|
66
|
+
the task's **HOME** folder = Dev Sprints (`isHomeFolderInQaQcScope()`), not just any list membership —
|
|
67
|
+
so Design tasks multi-listed into a Dev list stay out of Dev reports (owner decision).
|
|
51
68
|
|
|
52
69
|
Data model: reads/writes the `Team` database (`Tasks`, `Developers`, `Tasks_Developers`, `Sprints`) and the ClickUp REST API. The Work Type itself lives in ClickUp as a custom field (`CLICK_UP_CUSTOM_FIELD_ID__WORKTYPE`), not in the DB.
|
|
53
70
|
|
|
@@ -58,8 +75,15 @@ Data model: reads/writes the `Team` database (`Tasks`, `Developers`, `Tasks_Deve
|
|
|
58
75
|
- **No PHPUnit harness** in worker2. The regression test `Tests/Worker/ClickupWorkTypeTest.php` is a plain-PHP script (reflection on `isStatusComplete`) run with `php Tests/Worker/ClickupWorkTypeTest.php`.
|
|
59
76
|
- **ClickUp dropdown custom-field `value` is the option ORDERINDEX, not its name.** The webhook payload for a dropdown/labels custom field delivers the selected option's integer `orderindex`, *not* the option's display name. Any name-keyed lookup (e.g. `$lookupPreviousSprintWorkTypeByName`) will silently never match if fed the raw `value`. Resolve the name first via the field's own `type_config->options` — build a map keyed by `orderindex` (`$workTypesByIndex[$opt->orderindex] = $opt`) and read `->name` from it. This is the proven pattern in `_Worker_Team_Sprint::SprintLock` (`Worker/Team/Sprint.php`, the Work Type resolve ~L3894 and the POST ~L3971). Guard the lookup with `isset($workTypesByIndex[$value])` so a stale/deleted option orderindex doesn't warn.
|
|
60
77
|
|
|
78
|
+
- **Home folder vs. any list.** `isTaskInQaQcScope()` matches any list the task is in (good for QA/QC
|
|
79
|
+
gates); sprint **data ingestion** must use `isHomeFolderInQaQcScope()`. Mixing them up leaks Design
|
|
80
|
+
tasks into Dev sprint reports.
|
|
61
81
|
- **2026-09-21..25 outage: every taskCreated failed (~113 jobs, 0 Fluffer jobs).** The automation user lacked edit rights on Previous Sprint Work Type → HTTP 401 `ACCESS_606`; the throw killed the rest of the job. Fixed by field permissions + `setFieldSafely`. Any new optional write must use `setFieldSafely`. Account-change check: [credential rotation](../workflows/clickup-credential-rotation.md).
|
|
62
82
|
|
|
63
83
|
## Related
|
|
64
84
|
|
|
65
85
|
- [ClickUp Project & Opportunity Multi-List Routing](./clickup-project-routing.md)
|
|
86
|
+
|
|
87
|
+
## Change history
|
|
88
|
+
- 2026-10-01 — Defaults also on taskMoved into a `Sprint*` list; Condition / Previous Sprint Condition / Has UI Changes added; Previous Sprint Work Type limited to sprint lists; ingestion gated on HOME folder (ajean; worker2 PR #178)
|
|
89
|
+
- 2026-09-25 — `setFieldSafely` after the ACCESS_606 taskCreated outage (jcardinal)
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["kyalamarthi"]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["kyalamarthi", "ajean"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/.gitignore
|
|
13
13
|
- worker2/Worker/Team/Github.php
|
|
@@ -43,7 +43,8 @@ the publish on a credential literal.
|
|
|
43
43
|
|---|---|---|
|
|
44
44
|
| GitHub personal access token, `agilantgit` account | `worker2/Worker/Team/Github.php:9` **and** `worker2/.ebextensions/git.json:4` (same token, two copies) | Committed since 2025-01-17 and present in **166 of 167 branches**, so scrubbing history is not a fix — **rotation is the only fix**. Org-wide reach. It is also interpolated into a `shell_exec` clone URL, so it shows up in the instance process list. |
|
|
45
45
|
| NetSuite EC private key | `worker2/Config/netsuite_ec_key.pem` | Tracked in git as a file. |
|
|
46
|
-
| TOGa Talos (`api.togaiq.com`) API key | `worker2/Worker/Team/Sprint.php
|
|
46
|
+
| TOGa Talos (`api.togaiq.com`) API key — **old** release-notes key | git history only: `worker2/Worker/Team/Sprint.php` before PR #179 (still visible at commit `8613ace`) | Code no longer uses it, but it was committed, so it **must be revoked** in TOGa Talos. Open until revoked. |
|
|
47
|
+
| TOGa Talos API key — **current** (`[talos] api_key`) | `worker2/Config/production.ini` `[talos]` group (~L230) | Shared by release notes and the transcript pipeline. Committed like every `Config/*.ini` (see Summary). |
|
|
47
48
|
| PayPal webhook signature verification **off** in production | `worker2/Config/production.ini:182` | Impact already written up — a caller who knows the URL can create a paid SalesOrder. See [Rate PayPal subscription webhook](../../../../clients/rate/features/paypal-subscription-purchase-webhook.md). |
|
|
48
49
|
|
|
49
50
|
### Fixed / no longer true
|
|
@@ -54,6 +55,12 @@ the publish on a credential literal.
|
|
|
54
55
|
`Authorization` header written into `Logs.Api`, are separate open items — see
|
|
55
56
|
[_ApiRequest gotchas](../../_underscore/features/apirequest-json-content-type.md#gotchas).
|
|
56
57
|
|
|
58
|
+
- **TOGa Talos key in `Sprint.php`** (2026-10-01, worker2 PR #179) — the `AI_API_KEY` constant is
|
|
59
|
+
**gone**. `aiApiKey()` reads `[talos] api_key`, the same `/api/ai/generate` key the transcript
|
|
60
|
+
pipeline already uses (no new `[togaiq]` group). This removed a **second** copy of a TOGa Talos key
|
|
61
|
+
from PHP — but `[talos] api_key` is itself committed in `production.ini` (table above), so treat it
|
|
62
|
+
as a refactor, not a fix.
|
|
63
|
+
|
|
57
64
|
## Gotchas
|
|
58
65
|
|
|
59
66
|
- **Config is not private here.** Moving a secret from a PHP constant into
|
|
@@ -75,3 +82,7 @@ the publish on a credential literal.
|
|
|
75
82
|
- [Team Sprint Management & Reporting](./team-sprint-management.md)
|
|
76
83
|
- [_ApiRequest — encode/decode & api-logging behavior](../../_underscore/features/apirequest-json-content-type.md)
|
|
77
84
|
- [Rate PayPal Subscription Purchase & Webhook Pipeline](../../../../clients/rate/features/paypal-subscription-purchase-webhook.md)
|
|
85
|
+
|
|
86
|
+
## Change history
|
|
87
|
+
- 2026-10-01 — TOGa Talos release-notes key moved to `[talos] api_key`; old key listed for revocation (ajean)
|
|
88
|
+
- 2026-09-21 — Register created from audit (kyalamarthi)
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-01
|
|
10
10
|
owners: ["dfranks", "kyalamarthi", "ajean", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Netsuite.php
|
|
@@ -253,6 +253,11 @@ Stage/close/reopen rules: [stage sync](./netsuite-clickup-opportunity-bidirectio
|
|
|
253
253
|
|
|
254
254
|
## Gotchas
|
|
255
255
|
|
|
256
|
+
- **Unverified: the folder filter on `GET /team/{id}/task` may be wrong here.** ClickUp documents the
|
|
257
|
+
folder filter as `project_ids[]`; `Worker/Netsuite/Opportunity.php` (~L1345) and
|
|
258
|
+
`Worker/Netsuite/OpportunityDocuments.php` (~L456) send `folder_ids[]`. If ClickUp ignores it, the
|
|
259
|
+
query returns team-wide tasks, not just the Opportunities folder. Check before trusting those scans
|
|
260
|
+
(follow-up, flagged 2026-10-01). The newer sprint code uses `project_ids[]`.
|
|
256
261
|
- **⚠ The `endCustomerName` 9-day total outage — a column-existence gate does NOT protect an ORM write (2026-09-01 → 2026-09-10).** Every `Netsuite/Opportunity/post|put` job threw `There is no field called 'endCustomerName' in the '_Model_Forecast_Opportunity' model.` (`_underscore/Model.php:498`) from 2026-09-01 12:01 to 2026-09-10 15:48 CDT. **The whole NetSuite→Forecast→ClickUp opportunity sync was dead for 9 days and every job was red.**
|
|
257
262
|
- **Cause — a deploy split across TWO repos.** worker2 shipped the `$values['endCustomerName']` write (PR #158) while `_underscore/Model/Forecast/Opportunity.php` had not yet declared the property. `_Model` builds its field map from **declared properties** in the constructor, so `__set` rejected the unknown field. The DB column existed (the migration had been applied) — the missing half was the model declaration, in a different repo.
|
|
258
263
|
- **Blast radius:** the throw lands in `importOpportunity()` **before** the Forecast save and **before** `maybeCreateClickupTask()`, so affected opportunities got **neither** a Forecast row **nor** a ClickUp task. Measured in prod `Core.WorkerJobs`: **611 failed jobs, 310 distinct opportunities, 264 still with no ClickUp task**, 1 (NS internalId 7442501) absent from Forecast.
|
|
@@ -322,4 +327,5 @@ Stage/close/reopen rules: [stage sync](./netsuite-clickup-opportunity-bidirectio
|
|
|
322
327
|
- [MCP Tool Usage](../../../../standalone/apps/claude/workflows/mcp-tool-usage.md) — the `toga-db` query traps hit while running the verdict census.
|
|
323
328
|
|
|
324
329
|
## Change history
|
|
330
|
+
- 2026-10-01 — Flagged `folder_ids[]` vs documented `project_ids[]` filter (unverified) (ajean)
|
|
325
331
|
- 2026-09-25 — Claim guards creation only; CU→NS routed by folder, link check, actor guard; orderindex-0 bug fixed (jcardinal)
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-01
|
|
10
10
|
owners: ["jcardinal", "mhammontree", "bala", "rgirish"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Monitor/Compass.php
|
|
@@ -18,6 +18,7 @@ files:
|
|
|
18
18
|
- worker2/Worker/Monitor/Prudential.php
|
|
19
19
|
- worker2/Worker/Monitor/Nycdoe.php
|
|
20
20
|
- worker2/Worker/Monitor/Staples.php
|
|
21
|
+
- worker2/Worker/Monitor/Wje.php
|
|
21
22
|
- worker2/Config/production.ini
|
|
22
23
|
- worker2/Config/beta.ini
|
|
23
24
|
- worker2/Config/sandbox-dev.ini
|
|
@@ -32,6 +33,7 @@ related:
|
|
|
32
33
|
- ../../../clients/staples/features/oneuptime-sftp-folder-monitors.md
|
|
33
34
|
- ../../../clients/compass-usa/features/oneuptime-ma-refresh-order-monitor.md
|
|
34
35
|
- ../../../clients/compass-usa/features/oneuptime-netsuite-duplicate-sales-orders-monitor.md
|
|
36
|
+
- ../../../clients/wje/features/oneuptime-ticket-writes-monitor.md
|
|
35
37
|
---
|
|
36
38
|
|
|
37
39
|
OneUptime push-metric monitor pattern for worker2 (2.0): a self-contained cron action decides pass/fail and POSTs a string token to a OneUptime Incoming Request monitor; open when building/tuning a worker2 monitor, its thresholds, or the OneUptime import JSON.
|
|
@@ -302,6 +304,13 @@ has no proven connection to the legacy `Common` DB today**, so a 2.0 monitor can
|
|
|
302
304
|
`MaRefreshOrderExceptions` counts rows in `Client_Compass` instead. If worker2 ever gets that
|
|
303
305
|
connection, a generic "1.0 cron has not checked in" monitor becomes possible.
|
|
304
306
|
|
|
307
|
+
This is about `Common` only — **worker2 does read other legacy DBs.** `worker2/_.php` registers
|
|
308
|
+
`TOGaDeskSupport` as `_underscore::DB_TOGADESK1` and `Vision_Log` as `DB_VISION_LOGS` at boot (from the
|
|
309
|
+
`[database1]` config group), already used in prod by `Worker/Sync/SalesOrderStatus.php` and
|
|
310
|
+
`Worker/Client/Aig/ClosedClaims.php`. A 2.0 monitor over TOGaDesk data is possible and preferred (it
|
|
311
|
+
runs outside the 1.0 worker it watches) — e.g. the
|
|
312
|
+
[WJE ticket-writes monitor](../../../clients/wje/features/oneuptime-ticket-writes-monitor.md).
|
|
313
|
+
|
|
305
314
|
### Match the lateness criterion to the flow's real time-scale
|
|
306
315
|
|
|
307
316
|
The sibling Compass monitors ship 6 criteria including a DEGRADED incident at **2 missed pushes (10
|
|
@@ -559,4 +568,5 @@ production deploy.**
|
|
|
559
568
|
- [OneUptime 1.0 worker uptime monitoring](../../../1.0/apps/worker/features/oneuptime-worker-uptime-monitoring.md) — the 1.0 push-heartbeat predecessor
|
|
560
569
|
|
|
561
570
|
## Change history
|
|
571
|
+
- 2026-10-01 — Clarified worker2 reads `TOGaDeskSupport` (`DB_TOGADESK1`) — only `Common` is unreachable; added `Monitor/Wje`. (mhammontree)
|
|
562
572
|
- 2026-09-25 — Added `NetsuiteDuplicateSalesOrders`, the first monitor that queries NetSuite (SuiteQL) instead of a Toga DB. (rgirish)
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [jcardinal]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["jcardinal", "ajean"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Clickup/QaReview.php
|
|
13
13
|
- worker2/Worker/Clickup/QcReview.php
|
|
@@ -15,10 +15,12 @@ files:
|
|
|
15
15
|
- worker2/Worker/Clickup/StageBatch.php
|
|
16
16
|
- worker2/Worker/Clickup/FeatureReview.php
|
|
17
17
|
- worker2/Worker/Clickup/BeReview.php
|
|
18
|
+
- worker2/Worker/Clickup/DevFinished.php
|
|
18
19
|
- worker2/Worker/Clickup.php
|
|
19
20
|
- worker2/Worker/Github.php
|
|
20
21
|
- _underscore/Component/Api/Clickup/Clickup.php
|
|
21
22
|
- dbchanges2/Core/2026-09-09a - Insert StageBatch CronJob.sql
|
|
23
|
+
- dbchanges2/Core/2026-09-29a - Record sprint and design CronJobs.sql
|
|
22
24
|
- dbchanges2/Team/2026-09-09a - Add isApproved to PullRequests.sql
|
|
23
25
|
related:
|
|
24
26
|
- ./clickup-general-automation.md
|
|
@@ -40,8 +42,10 @@ production** (deployed on `_production`).
|
|
|
40
42
|
- `_Worker_Clickup_QcReview` — QC defect review + client review; also handles the **Hotfix** defect
|
|
41
43
|
gate and its `DeployHotfix` entry.
|
|
42
44
|
- `_Worker_Clickup_QcBatch` — a weekday 4:30pm Central CRON that batches QA-Approved tasks into QC.
|
|
43
|
-
- `_Worker_Clickup_StageBatch` —
|
|
44
|
-
|
|
45
|
+
- `_Worker_Clickup_StageBatch` — batch that ships "To Deploy" tasks to production. **Not scheduled in
|
|
46
|
+
prod: Stage deploys stay manual (owner decision 2026-10-01).**
|
|
47
|
+
- `_Worker_Clickup_DevFinished::CatchUp` — every-30-min safety net for missed Development Finished
|
|
48
|
+
webhooks.
|
|
45
49
|
- `_Worker_Clickup_FeatureReview` — the **Feature** flow: rolls a multi-subtask feature through the
|
|
46
50
|
same gates as a unit, advancing only when **all** subtasks clear a gate.
|
|
47
51
|
- `_Worker_Clickup_BeReview` — **auto** BE-approve driven by GitHub PR reviews (removes the manual
|
|
@@ -79,17 +83,41 @@ when a task *left* a status; the QA/QC entry must be on entering the status.
|
|
|
79
83
|
(matches `QaReview::changedCustomFieldIds`), each in its own try/catch; safe because each skips the
|
|
80
84
|
other's task kind. Write failures are logged + Sentry, never thrown.
|
|
81
85
|
|
|
86
|
+
`handleDevelopmentFinished` checks for a PR itself (`_Worker_Clickup_GeneralAutomation::hasPullRequest`)
|
|
87
|
+
before picking the first review status, so it does not race the Has Code Changes backfill.
|
|
88
|
+
|
|
89
|
+
**Prod flow end to end:** Development Finished → UI / FE / BE review → QA Approved → (4:30 PM QC
|
|
90
|
+
Batch) QC Review → Stage or Client Review → To Deploy → **manual** deploy.
|
|
91
|
+
|
|
92
|
+
### Catch-up for missed webhooks (`_Worker_Clickup_DevFinished::CatchUp`)
|
|
93
|
+
|
|
94
|
+
- Cron id 86, `*/30 * * * *`. Scans Dev Sprints (`GET /team/{id}/task?project_ids[]=…&statuses[]=development finished&subtasks=true`)
|
|
95
|
+
and re-runs `_Worker_Clickup::RunDevelopmentFinished` for tasks whose **Environment** is empty, or
|
|
96
|
+
set but stuck ≥ 30 min (`STUCK_THRESHOLD_SECONDS`).
|
|
97
|
+
- Fresh re-fetch per task (`getTaskDetails($id, true)`); skips tasks updated in the last 2 min
|
|
98
|
+
(`RECENT_UPDATE_GUARD_SECONDS`) so it does not race a live webhook.
|
|
99
|
+
- Per-task `catch (Exception)` is **on purpose** (CSO decision, CodeRabbit agreed): each task is
|
|
100
|
+
independent and is retried next run; code errors (`Error`) still fail loud. Rescans from page 0
|
|
101
|
+
after a trigger (moved tasks shift pages); `MAX_PAGES` caps fetches. Returns
|
|
102
|
+
`{isSuccess, checked, triggered, failed}`.
|
|
103
|
+
- First run 2026-10-01 moved 10 tasks stuck since the 9/12–9/20 webhook outage (all → `qa approved`,
|
|
104
|
+
since neither Has UI nor Has Code was checked).
|
|
105
|
+
|
|
82
106
|
### QA review chain (`_Worker_Clickup_QaReview`)
|
|
83
107
|
|
|
84
108
|
- Three ordered review fields: **UI → FE (front-end) → BE (back-end)**.
|
|
85
|
-
- **Approve** advances to the next review field.
|
|
109
|
+
- **Approve** advances to the next review field. Entering a review status resets its field to
|
|
110
|
+
**Needs Review** ([general automation](./clickup-general-automation.md) rule 7); `QaReview` /
|
|
111
|
+
`FeatureReview` ignore Needs Review, so there is no loop.
|
|
86
112
|
- **Deny** sends the task to **Rework** and **reverts** the task's changes from the QA/QC branch.
|
|
87
113
|
- **Unit-test gate is a STUB today.** `runUnitTests()` currently returns `true`. Real gating (Cypress /
|
|
88
114
|
Postman / Codeception) is pending — do not treat the gate as enforcing anything yet.
|
|
89
115
|
|
|
90
116
|
### QC batch promotion (`_Worker_Clickup_QcBatch`)
|
|
91
117
|
|
|
92
|
-
- Weekday **4:30pm Central** CRON
|
|
118
|
+
- Weekday **4:30pm Central** CRON, prod id 75 (`Core/2026-09-08e - Insert QcBatch CronJob.sql`).
|
|
119
|
+
It was `isActive = 0` in prod until 2026-09-29a turned it on — until then QA Approved tasks never
|
|
120
|
+
moved to QC.
|
|
93
121
|
- Moves **QA-Approved** tasks → **QC-Task**.
|
|
94
122
|
- **Skips the run if QC is still busy** (does not pile a new batch on top of an in-progress one).
|
|
95
123
|
|
|
@@ -122,16 +150,15 @@ when a task *left* a status; the QA/QC entry must be on entering the status.
|
|
|
122
150
|
safe. The actual prod merge goes through `MergeTaskToProduction` — see
|
|
123
151
|
[QA/QC branch automation](./qa-qc-branch-automation.md).
|
|
124
152
|
|
|
125
|
-
### Stage
|
|
153
|
+
### Stage batch (`_Worker_Clickup_StageBatch`) — not scheduled
|
|
126
154
|
|
|
127
|
-
-
|
|
128
|
-
`Core/2026-09-09a - Insert StageBatch CronJob.sql`
|
|
129
|
-
- `Process` loops `Team.Tasks` "to deploy" rows and dispatches **one `DeployTask` per task
|
|
130
|
-
stays under the 300s worker cap
|
|
131
|
-
`
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
switch cases in `Worker/Clickup.php`).
|
|
155
|
+
- **Stage deploys stay manual (owner decision 2026-10-01).** Do not add or turn on a StageBatch cron
|
|
156
|
+
row without the owner, even though `Core/2026-09-09a - Insert StageBatch CronJob.sql` exists.
|
|
157
|
+
- When run: `Process` loops `Team.Tasks` "to deploy" rows and dispatches **one `DeployTask` per task**
|
|
158
|
+
(stays under the 300s worker cap; a re-run resumes via the deploy-claim). `DeployTask` ships via
|
|
159
|
+
`MergeTaskToProduction` then finalizes (`markProductionDeployed` + `releaseSubEnvironment`).
|
|
160
|
+
- Replaces the **retired** legacy status-driven prod-merge (old `to deploy` / `hotfix review` cases in
|
|
161
|
+
`Worker/Clickup.php`).
|
|
135
162
|
|
|
136
163
|
### Feature flow (`_Worker_Clickup_FeatureReview`)
|
|
137
164
|
|
|
@@ -157,7 +184,12 @@ when a task *left* a status; the QA/QC entry must be on entering the status.
|
|
|
157
184
|
the fan-out, `handleDevelopmentFinished`, `QcBatch`, and the taskCreated Work Type default. A task
|
|
158
185
|
outside that folder is ignored — verify the folder before expecting the flow to run.
|
|
159
186
|
- **Dev Finished with neither Has UI Changes nor Has Code Changes checked → `qa approved` (skips
|
|
160
|
-
review).** Intended (jcardinal confirmed); do not "fix".
|
|
187
|
+
review).** Intended (jcardinal confirmed); do not "fix". Sprint-list defaults now check Has UI
|
|
188
|
+
Changes on Dev tasks, and a recorded PR checks Has Code Changes, so this path is now rare.
|
|
189
|
+
- **A missed Development Finished webhook used to strand the task.** CatchUp now recovers it within
|
|
190
|
+
30 min. It skips multi-listed tasks whose HOME folder is still the old True Dev Sprints folder —
|
|
191
|
+
same gap as `ResetInProgress`, see
|
|
192
|
+
[team-sprint-management](./team-sprint-management.md#gotchas).
|
|
161
193
|
- **A Feature advances only when ALL subtasks clear a gate** — a single lagging subtask holds the whole
|
|
162
194
|
feature. Subtasks carry no code; only the feature branch does.
|
|
163
195
|
- **`_underscore CLICK_UP_CUSTOM_FIELD_ID__CLIENT_REVIEW` was repointed** to the Dev Sprints
|
|
@@ -186,3 +218,7 @@ when a task *left* a status; the QA/QC entry must be on entering the status.
|
|
|
186
218
|
- [ClickUp general automation](./clickup-general-automation.md)
|
|
187
219
|
- [QA/QC branch automation](./qa-qc-branch-automation.md)
|
|
188
220
|
- [ClickUp custom-field conventions](./clickup-custom-field-conventions.md)
|
|
221
|
+
|
|
222
|
+
## Change history
|
|
223
|
+
- 2026-10-01 — DevFinished CatchUp (cron 86); QC Batch turned on (id 75); review fields reset to Needs Review on entry; Stage deploys confirmed manual (ajean; worker2 PR #178, dbchanges2 PR #504)
|
|
224
|
+
- 2026-09-25 — Scope narrowed to Dev Sprints `90116585845` (jcardinal)
|
|
@@ -6,23 +6,25 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["jcardinal", "kyalamarthi"]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["jcardinal", "kyalamarthi", "ajean"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Team/Sprint.php
|
|
13
13
|
- worker2/Config/production.ini
|
|
14
14
|
- _underscore/Model/Team/Sprint.php
|
|
15
|
+
- dbchanges2/Core/2026-09-29a - Record sprint and design CronJobs.sql
|
|
15
16
|
related:
|
|
16
17
|
- ../architecture.md
|
|
17
18
|
- ./creating-worker-actions.md
|
|
18
19
|
- ./clickup-work-type-automation.md
|
|
19
20
|
- ./clickup-project-routing.md
|
|
21
|
+
- ./clickup-general-automation.md
|
|
20
22
|
- ./committed-secrets-and-config-debt.md
|
|
21
23
|
- ../workflows/sprint-clickup-folder-repoint.md
|
|
22
24
|
- ../workflows/php-runtime-upgrade-dependency-audit.md
|
|
23
25
|
---
|
|
24
26
|
|
|
25
|
-
`_Worker_Team_Sprint` engine for TOGA's biweekly dev-sprint process + reporting (warnings, lock, daily dashboard, sprint-end scoring, AI release notes); open for sprint actions, the scoring model, or Power BI-parity dashboard SQL.
|
|
27
|
+
`_Worker_Team_Sprint` engine for TOGA's biweekly dev-sprint process + reporting (warnings, lock, daily dashboard, sprint-end scoring, AI release notes, 7 PM In Progress reset); open for sprint actions, their cron schedule, the scoring model, or Power BI-parity dashboard SQL.
|
|
26
28
|
|
|
27
29
|
## Summary
|
|
28
30
|
|
|
@@ -50,11 +52,13 @@ everything.
|
|
|
50
52
|
### Key files / entry points
|
|
51
53
|
- `Worker/Team/Sprint.php` — the entire feature (one abstract class, static methods).
|
|
52
54
|
- `initialize()` — registers the `Logs_True` DB connection used for ClickUp API logging.
|
|
53
|
-
- ClickUp config
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
55
|
+
- ClickUp config: `CLICKUP_TEAM_ID`; Dev Sprints folder via public `devSprintsFolderId()` — reads
|
|
56
|
+
`[clickup] sprint_folder_id`, falls back to `CLICKUP_DEV_SPRINTS_FOLDER_ID` (`90116585845`);
|
|
57
|
+
`CLICKUP_DESIGN_SPRINTS_FOLDER_ID` (`90116585900`, used only by `ResetInProgress`). Token via
|
|
58
|
+
`_Component_Api_Clickup::accessToken()` (`[clickup] token`).
|
|
59
|
+
- AI config: `AI_API_ENDPOINT` (TOGa Talos `…/api/ai/generate`), `AI_MODEL_ID`
|
|
60
|
+
(`bedrock/us.anthropic.claude-haiku-4-5-…`), temperature, max-tokens, timeout. Key via private
|
|
61
|
+
`aiApiKey()` → `[talos] api_key` (the same key the transcript pipeline uses); throws if missing.
|
|
58
62
|
- Report distribution: `SEND_LEADERSHIP_REPORTS_TO` (leadership email list); `SEND_EMAILS_FROM` /
|
|
59
63
|
`SEND_EMAILS_FROM_NAME`.
|
|
60
64
|
|
|
@@ -62,29 +66,38 @@ everything.
|
|
|
62
66
|
`_Worker_Team_Sprint::SprintEnd`. New actions are only reachable once registered as a `CronJobs` row
|
|
63
67
|
(or webhook route). See [creating-worker-actions](./creating-worker-actions.md).
|
|
64
68
|
|
|
65
|
-
### The sprint lifecycle (
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
69
|
+
### The sprint lifecycle — cron schedule (prod `Core.CronJobs`, Central time)
|
|
70
|
+
All rows are recorded in git by `dbchanges2/Core/2026-09-29a - Record sprint and design CronJobs.sql`
|
|
71
|
+
(PR #504, applied to prod). Sprints launch **every other Wednesday**; cron cannot say "biweekly", so
|
|
72
|
+
weekly rows **self-gate in PHP** against `Sprints`.
|
|
73
|
+
|
|
74
|
+
| Id | Action | Schedule | Gate |
|
|
75
|
+
|----|--------|----------|------|
|
|
76
|
+
| 9 | `Team/Sprint/SprintDaily` | `0 9 * * 1-5` | skips if no `Sprints` row covers today |
|
|
77
|
+
| 11 | `Team/Sprint/SprintEnd` | `0 9 * * 1` (Monday) | skips if previous sprint ended > 10 days ago (the mid-sprint Monday) |
|
|
78
|
+
| 40 | `Team/Sprint/SprintLockScheduled` | `0 10 * * 3` | runs only if a sprint ended yesterday |
|
|
79
|
+
| 83 | `Team/Sprint/SendSprintLockWarningsScheduled` | `30 8 * * 3` | same gate as SprintLockScheduled |
|
|
80
|
+
| 84 | `Team/Sprint/SendSprintEndWarningsScheduled` | `30 8 * * 1` | same gate as SprintEnd |
|
|
81
|
+
| 85 | `Team/Sprint/ResetInProgress` | `0 19 * * 1-5` | none (weekdays incl. holidays) |
|
|
82
|
+
|
|
83
|
+
(Id 21 = Design review reminders, id 86 = `Clickup/DevFinished/CatchUp` — see
|
|
84
|
+
[design automation](./clickup-design-sprint-automation.md) and
|
|
85
|
+
[QA/QC pipeline](./qa-qc-review-pipeline.md).)
|
|
86
|
+
|
|
87
|
+
Most methods accept an optional `$sprint` number; when omitted they resolve it via `getSprint()` —
|
|
88
|
+
the sprint whose `dateStart…dateEnd` contains today, `ORDER BY dateStart DESC, id DESC`. It returns
|
|
89
|
+
**null** when no row matches; every caller returns a skip message instead of fataling.
|
|
90
|
+
|
|
91
|
+
Every action returns a string to `Core.WorkerJobs.output` — skips say why; `SprintDaily`, `SprintEnd`,
|
|
92
|
+
and the warning actions return a JSON summary (e.g.
|
|
93
|
+
`{"isSuccess":true,"sprint":2,"tasksCaptured":74,"emailSent":true}`). Read the output column to tell
|
|
94
|
+
a good run from one that did nothing.
|
|
83
95
|
|
|
84
96
|
### Public actions (entry points)
|
|
85
97
|
|
|
86
|
-
#### `SendSprintEndWarnings()` / `SendSprintLockWarnings()`
|
|
87
|
-
|
|
98
|
+
#### `SendSprintEndWarnings()` / `SendSprintLockWarnings()` (+ `…Scheduled` wrappers)
|
|
99
|
+
The crons call the `…Scheduled` wrappers, which apply the gates in the table above and then call
|
|
100
|
+
these. Look up the latest `Sprints` row, find every distinct developer with a task in that sprint
|
|
88
101
|
(`Developers ⋈ Tasks_Developers ⋈ Tasks`), and email each a templated reminder. End-warning = "clean
|
|
89
102
|
up your data before the 9:00 AM End capture"; Lock-warning = "have your commitments ready before the
|
|
90
103
|
9:45 AM lock; work added after lock is treated as unplanned/interrupt work." Both send via the
|
|
@@ -94,7 +107,10 @@ up your data before the 9:00 AM End capture"; Lock-warning = "have your commitme
|
|
|
94
107
|
Freezes the sprint's plan at launch time. Optionally re-captures ClickUp data first, then for each
|
|
95
108
|
task records the **"at lock" baseline** — `workTypeAtLock`, `sprintPointsAtLock`, `statusAtLock` — and
|
|
96
109
|
initializes the per-sprint tracking counters (rework events, unjustified status/work-effort changes,
|
|
97
|
-
in-progress time). These at-lock values are what later metrics measure *against*.
|
|
110
|
+
in-progress time). These at-lock values are what later metrics measure *against*. Per task it also
|
|
111
|
+
copies ClickUp **Condition → Previous Sprint Condition** (`CLICK_UP_CUSTOM_FIELD_ID__PREVIOUS_CONDITION`)
|
|
112
|
+
then resets **Condition = On Track** — each write in its own try/catch so one refused field does not
|
|
113
|
+
stop the lock. First lock running this code: 2026-10-14 (the 9/30 lock ran the old code). If the target sprint
|
|
98
114
|
list cannot be found in ClickUp it emails a leadership alert. With `$generateSprintLockReport`, emits
|
|
99
115
|
an Excel lock report of committed/conditional/unplanned/stretch points and tasks. When
|
|
100
116
|
`$allowUnplannedTasks` is false, tasks appearing after lock with an UNPLANNED work type are flagged.
|
|
@@ -109,10 +125,8 @@ Otherwise it returns a skip message (captured to `Core.WorkerJobs.output`) and d
|
|
|
109
125
|
`SprintLock()` itself is unchanged — this only decides *whether* to run it today. Action path
|
|
110
126
|
`Team/Sprint/SprintLockScheduled` → `_Worker_Team_Sprint::SprintLockScheduled`.
|
|
111
127
|
|
|
112
|
-
**Deploy coupling.**
|
|
113
|
-
|
|
114
|
-
`Team/Sprint/SprintLockScheduled` action, which must exist in the deployed code or the scheduled job
|
|
115
|
-
fails when it fires.
|
|
128
|
+
**Deploy coupling.** Deploy worker2 **before** applying any `CronJobs` row that names a new action —
|
|
129
|
+
otherwise the job fails when it fires.
|
|
116
130
|
|
|
117
131
|
#### `SprintDaily($sprint = null, …)`
|
|
118
132
|
In-sprint **leadership dashboard**, read-only with respect to metrics. Optionally re-captures ClickUp
|
|
@@ -127,14 +141,17 @@ Emailed to the leadership list as `TeamSprintDailyReport_Sprint{N}_{date}.xlsx`.
|
|
|
127
141
|
|
|
128
142
|
#### `SprintEnd($sprint = null, …)`
|
|
129
143
|
The largest method and the heart of the feature. Phases:
|
|
130
|
-
1. **Resolve** the target (previous) sprint
|
|
131
|
-
|
|
144
|
+
1. **Resolve** the target (previous) sprint. The 10-day mid-sprint skip runs **before**
|
|
145
|
+
`CaptureSprintEnd()` — capture writes back to ClickUp, so it must not run on the middle Monday
|
|
146
|
+
(it used to, and overwrote the last sprint's data). Then optionally capture.
|
|
132
147
|
2. **Score every developer** across the four metrics below.
|
|
133
148
|
3. **Per-developer report workbook** (Dashboard, Team Summary, Sprint History, Task Detail, and a
|
|
134
149
|
Score-Trends chart over the last `SPRINTS_BACK_FOR_AVERAGES` sprints), emailed to each developer.
|
|
135
150
|
4. **Leadership summary workbook** (team-wide dashboard, all-developer summary with a *recommended
|
|
136
151
|
commitment* for next sprint, and stalled-task detail), emailed to leadership.
|
|
137
|
-
5. Optionally call
|
|
152
|
+
5. Optionally call `SendReleaseNotes()`. Its real result sets `releaseNotesSent` /
|
|
153
|
+
`releaseNotesError` (capped at 300 chars) in the JSON summary; a failure there is caught, so
|
|
154
|
+
SprintEnd still returns.
|
|
138
155
|
|
|
139
156
|
#### `SendReleaseNotes($sprint = null)`
|
|
140
157
|
Pulls every **completed** task in the sprint (`statusNow IN (…DONE)`) into a CSV of `workType,
|
|
@@ -143,6 +160,14 @@ with a detailed system prompt that rewrites technical task titles into business-
|
|
|
143
160
|
stakeholder-grouped HTML release notes. The returned HTML (code fences stripped) is emailed to the
|
|
144
161
|
leadership list as "Sprint N Release Notes".
|
|
145
162
|
|
|
163
|
+
#### `ResetInProgress(): string`
|
|
164
|
+
7 PM weekdays: moves every **In Progress** task and subtask back to **To Do** in Dev Sprints
|
|
165
|
+
(`devSprintsFolderId()`) and Design Sprints (`90116585900`). No notification. Uses ClickUp
|
|
166
|
+
`GET /team/{id}/task` with `project_ids[]` (the documented folder filter — **not** `folder_ids[]`)
|
|
167
|
+
plus a home-folder check, rescans from page 0 after writes (moved tasks shift pages), throttles
|
|
168
|
+
writes (`RESET_IN_PROGRESS_THROTTLE_MICROSECONDS`), and caps page fetches per folder
|
|
169
|
+
(`RESET_IN_PROGRESS_MAX_ITERATIONS`).
|
|
170
|
+
|
|
146
171
|
### The scoring model (Sprint End)
|
|
147
172
|
Each developer receives four 0–100 sub-scores and a weighted **Final Score**:
|
|
148
173
|
|
|
@@ -339,45 +364,38 @@ see [TOGa Talos Sprint Dashboard API](../../api2/features/sprint-dashboard-api.m
|
|
|
339
364
|
**next** sprint's start (exclusive); the code stores `dateEnd` minus one day. **Proper fix for a
|
|
340
365
|
future ticket:** take the number from the name but the dates from the list object's `start_date` /
|
|
341
366
|
`due_date` (epoch ms), already fetched for the date-window match.
|
|
342
|
-
- **
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
reached. **Workaround (proven in production 2026-09-16):** `INSERT` one `Sprints` row by hand
|
|
348
|
-
(`sprint`, `dateStart`, `dateEnd`); `CaptureSprintDaily`'s "ensure the sprint record exists" block
|
|
349
|
-
(~L3478) self-heals from there. Anyone truncating these tables hits this — see
|
|
367
|
+
- **An empty `Sprints` table still cannot bootstrap itself.** Every entry point calls `getSprint()`
|
|
368
|
+
before the `Capture*` that would create the row. Since PR #178 it **skips with a message** instead
|
|
369
|
+
of fataling (`Attempt to read property "id" on null`), but it still never creates the row. Fix:
|
|
370
|
+
`INSERT` one `Sprints` row by hand (`sprint`, `dateStart`, `dateEnd`); `CaptureSprintDaily`
|
|
371
|
+
self-heals from there. See
|
|
350
372
|
[Repointing the sprint tooling](../workflows/sprint-clickup-folder-repoint.md).
|
|
351
|
-
- **`
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
373
|
+
- **`_underscore` `_Model_Team_Sprint` "last 4 sprints" math must not go below sprint 1.** After the
|
|
374
|
+
numbering restart, `sprint - 4` underflowed the unsigned sprint column (MySQL Error 1690) and
|
|
375
|
+
`SprintDaily` failed every day 9/22–9/30. Fixed on `_production` by `pastSprintsRange()`
|
|
376
|
+
(TRUE-82292). Any new "N sprints back" math must clamp at 1.
|
|
355
377
|
- **`SprintLock` is NOT a safe way to repopulate empty tables.** It captures before it reads, so it
|
|
356
378
|
looks like a bootstrap (its `getSprint()` call is gated behind `$generateSprintLockReport`), but
|
|
357
379
|
its capture block (~L4016-4030) **writes back to ClickUp for every task** — sets Previous Sprint
|
|
358
|
-
Work Type
|
|
380
|
+
Work Type, "Added at sprint launch" = true, and the Condition copy/reset. Run outside the real launch ceremony it
|
|
359
381
|
corrupts those fields across the whole sprint.
|
|
360
|
-
- **
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
`
|
|
364
|
-
|
|
365
|
-
(
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
- **The dev-sprints folder id lives in THREE places** — the `Sprint.php` constant, the
|
|
374
|
-
`Monitor/Oneuptime.php` fallback, and `[clickup] sprint_folder_id` in `Config/production.ini`.
|
|
375
|
-
They must move together, and `Worker/Clickup/Design.php`'s own Design-Sprints folder must not move
|
|
376
|
-
with them. Procedure:
|
|
382
|
+
- **Condition and Previous Sprint Condition are live fields (2026-10-01).** The 9/25 "Condition
|
|
383
|
+
retired" call was reversed by the owner. Both fields now exist on the Dev Sprints and Design Sprints
|
|
384
|
+
lists (Previous Sprint Condition used to exist only on the old True Design Sprints folder). Writers:
|
|
385
|
+
`SprintLock` (above) and the webhook defaults in
|
|
386
|
+
[ClickUp general automation](./clickup-general-automation.md). Any task in a sprint list without
|
|
387
|
+
these fields (or **"Added at Sprint Launch"**) fails the lock write — the 9/30 lock first failed
|
|
388
|
+
with `[FIELD_115]` on that field, retry succeeded.
|
|
389
|
+
- **`ResetInProgress` skips multi-listed tasks whose HOME folder is the old True Dev Sprints folder**
|
|
390
|
+
(owner accepted for now). The same gap exists in `Clickup/DevFinished/CatchUp`. Fix: walk the
|
|
391
|
+
folder's lists with `include_timl=true` instead of the team-task filter.
|
|
392
|
+
- **The dev-sprints folder id: change `[clickup] sprint_folder_id` first.** `Sprint.php` reads it via
|
|
393
|
+
`devSprintsFolderId()`; the `Sprint.php` constant and the `Monitor/Oneuptime.php` fallback are only
|
|
394
|
+
fallbacks but should still move with it. Design folder ids must not move with them. Procedure:
|
|
377
395
|
[Repointing the sprint tooling](../workflows/sprint-clickup-folder-repoint.md).
|
|
378
|
-
- **Credentials.**
|
|
379
|
-
|
|
380
|
-
|
|
396
|
+
- **Credentials.** Both come from config now: ClickUp `[clickup] token`, TOGa Talos `[talos] api_key`
|
|
397
|
+
(PR #179). The old hardcoded TOGa Talos key is still in git history and must be revoked — see the
|
|
398
|
+
[committed secrets register](./committed-secrets-and-config-debt.md).
|
|
381
399
|
- **Worker contract.** Like all 2.0 workers this must return cleanly and commit writes via the
|
|
382
400
|
`_Db`/transaction layer to be durable — uncommitted captures can be silently dropped.
|
|
383
401
|
- **ClickUp pagination returns duplicate tasks — the sync loop must be idempotent.** ClickUp's
|
|
@@ -406,3 +424,8 @@ see [TOGa Talos Sprint Dashboard API](../../api2/features/sprint-dashboard-api.m
|
|
|
406
424
|
- [PHP Runtime Upgrade on Elastic Beanstalk](../workflows/php-runtime-upgrade-dependency-audit.md)
|
|
407
425
|
- [TOGa Talos Sprint Dashboard API](../../api2/features/sprint-dashboard-api.md) — the productionized Record Scripts.
|
|
408
426
|
- [Record Scripts](../../api2/features/record-scripts.md)
|
|
427
|
+
|
|
428
|
+
## Change history
|
|
429
|
+
- 2026-10-01 — getSprint() deterministic + null-safe; JSON run summaries; SprintEnd skips before capture; release-notes key from `[talos] api_key`; SprintLock copies/resets Condition; new ResetInProgress + scheduled warning wrappers; cron rows recorded in dbchanges2 (ajean; worker2 PR #178, #179)
|
|
430
|
+
- 2026-09-25 — Condition write removed from SprintLock (reversed 2026-10-01) (jcardinal)
|
|
431
|
+
- 2026-09-16 — Repointed to the Dev Sprints folder; tables reset (kyalamarthi)
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: wje
|
|
7
7
|
type: client-feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["snaredla"]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["snaredla", "mhammontree"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Wje.php
|
|
13
13
|
- _underscore/Component/Api/Wje/Wje.php
|
|
@@ -24,9 +24,11 @@ files:
|
|
|
24
24
|
- worker/crons/toga2/wje/sync_togasupply_wje.php
|
|
25
25
|
- worker/crons/notifications/reports/wje/wje_common.php
|
|
26
26
|
- library/app/systemmonitor/wje.php
|
|
27
|
+
- worker2/Worker/Monitor/Wje.php
|
|
27
28
|
- dbchanges2/Client_Wje/2024-10-04 - WjeOnboarding.sql
|
|
28
29
|
related:
|
|
29
30
|
- clients/wje/profile.md
|
|
31
|
+
- clients/wje/features/oneuptime-ticket-writes-monitor.md
|
|
30
32
|
- 2.0/apps/worker2/features/elite-freshservice-sync.md
|
|
31
33
|
- 1.0/apps/library/features/toga2-api-client-and-bridge.md
|
|
32
34
|
- 2.0/apps/worker2/features/startech-webhook-handler.md
|
|
@@ -62,7 +64,7 @@ a **daily batch import** of supporting records. A separate 2-minute cron bridges
|
|
|
62
64
|
`crons/toga2/wje/sync_togasupply_wje.php` (2-min Togadesk bridge),
|
|
63
65
|
`crons/notifications/reports/wje/{wje_monthly_report,wje_quarterly_report,wje_common}.php`.
|
|
64
66
|
- **Models:** `_underscore/Model/Wje/*` (8 classes, see Data model).
|
|
65
|
-
- **Monitoring:** `library/app/systemmonitor/wje.php` (`App_SystemMonitor_WJE`).
|
|
67
|
+
- **Monitoring:** `library/app/systemmonitor/wje.php` (`App_SystemMonitor_WJE`, Freshservice/webhook errors); `worker2/Worker/Monitor/Wje.php` ([TOGaDesk → 2.0 write monitor](../../../clients/wje/features/oneuptime-ticket-writes-monitor.md)).
|
|
66
68
|
|
|
67
69
|
### Real-time webhook (`_Worker_Wje::Webhook`)
|
|
68
70
|
|
|
@@ -115,7 +117,7 @@ asset type "Laptop", urgencies (Low/Normal/High/Critical), system params
|
|
|
115
117
|
`TOGADESK_LAST_TICKET_INTEGRATION_DATETIME` / `TOGA_LAST_TICKET_INTEGRATION_DATETIME`.
|
|
116
118
|
|
|
117
119
|
Hardcoded IDs: `CLIENT_UUID_WJE=c2f5180f-…`, `API_UUID_WJE=4750650f-…` (and `API_SECRET_WJE`) in
|
|
118
|
-
`worker2/Worker/Wje.php`; TOGaDesk client `153`, dept `269` (
|
|
120
|
+
`worker2/Worker/Wje.php`; TOGaDesk client `153`, dept `269` (dept `290` is defined but not synced TOGaDesk → 2.0 — see `clients/wje/profile.md`); default group
|
|
119
121
|
`13000155420` / agent `13000771757`; category form field `13000126389`.
|
|
120
122
|
|
|
121
123
|
## Gotchas
|
|
@@ -134,6 +136,9 @@ Hardcoded IDs: `CLIENT_UUID_WJE=c2f5180f-…`, `API_UUID_WJE=4750650f-…` (and
|
|
|
134
136
|
- **W12** TicketStages use Freshservice status codes (Open=2 … Closed=5); status sync depends on those matching live Freshservice statuses.
|
|
135
137
|
- Secrets (`API_SECRET_WJE`, the Freshservice password) are hardcoded in source — read them from the file when needed; never copy the values into docs or commits.
|
|
136
138
|
|
|
139
|
+
## Change history
|
|
140
|
+
- 2026-10-01 — Corrected dept 290 (not synced 1.0 → 2.0); linked the new ticket-writes monitor. (mhammontree)
|
|
141
|
+
|
|
137
142
|
## Related
|
|
138
143
|
|
|
139
144
|
- `clients/wje/profile.md` — WJE client profile + the 2-min Togadesk bridge sync.
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["kyalamarthi"]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["kyalamarthi", "ajean"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Team/Sprint.php
|
|
13
13
|
- worker2/Worker/Monitor/Oneuptime.php
|
|
@@ -33,15 +33,20 @@ constant edit. Done once on 2026-09-16 under TRUE-82052 (worker2 + dbchanges2).
|
|
|
33
33
|
name with a regex that requires the year; a list created as `Sprint 1 (9/16 - 9/30)` does not
|
|
34
34
|
match and the run fatals. See
|
|
35
35
|
[team-sprint-management → gotchas](../features/team-sprint-management.md#gotchas).
|
|
36
|
-
2. **Update
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
- `
|
|
41
|
-
|
|
36
|
+
2. **Update the folder id everywhere it lives:**
|
|
37
|
+
- `[clickup] sprint_folder_id` — `worker2/Config/production.ini`. **Primary:** `Sprint.php`
|
|
38
|
+
(`devSprintsFolderId()`, also used by ResetInProgress and DevFinished CatchUp) and the
|
|
39
|
+
escalation cron read it.
|
|
40
|
+
- `CLICKUP_DEV_SPRINTS_FOLDER_ID` — `worker2/Worker/Team/Sprint.php` (fallback only).
|
|
41
|
+
- `CLICKUP_SPRINTS_FOLDER_ID_FALLBACK` — `worker2/Worker/Monitor/Oneuptime.php` (fallback only).
|
|
42
|
+
- `_Worker_Clickup::QA_QC_FOLDER_IDS` — `worker2/Worker/Clickup.php` (QA/QC + sprint-default scope).
|
|
42
43
|
|
|
43
|
-
|
|
44
|
-
|
|
44
|
+
Design Sprints is a different folder: `DESIGN_SPRINTS_FOLDER_IDS` (`Worker/Clickup/Design.php`)
|
|
45
|
+
and `CLICKUP_DESIGN_SPRINTS_FOLDER_ID` (`Sprint.php`) move only when Design moves.
|
|
46
|
+
|
|
47
|
+
**Also add the sprint custom fields to the new folder's lists** — at least Condition, Previous
|
|
48
|
+
Sprint Condition, Work Type, Previous Sprint Work Type, and "Added at Sprint Launch". A missing
|
|
49
|
+
field fails SprintLock (`[FIELD_115]`, seen 2026-09-30) and the webhook defaults.
|
|
45
50
|
3. **Run the reset migration alone, on a fresh connection**, against the `Team` database:
|
|
46
51
|
`dbchanges2/Team/2026-09-16a - Reset sprint tables for new ClickUp folder.sql`. It truncates
|
|
47
52
|
child → parent (`Tasks_Developers` → `Tasks` → `Sprints`) wrapped in
|
|
@@ -55,8 +60,10 @@ constant edit. Done once on 2026-09-16 under TRUE-82052 (worker2 + dbchanges2).
|
|
|
55
60
|
[creating-worker-actions](../features/creating-worker-actions.md)). From there
|
|
56
61
|
`CaptureSprintDaily`'s own "ensure the sprint record exists" block (~L3478) keeps `Sprints`
|
|
57
62
|
current on its own.
|
|
58
|
-
6. **Verify
|
|
59
|
-
|
|
63
|
+
6. **Verify.** `SprintDaily` returns a JSON summary to `Core.WorkerJobs.output` (e.g.
|
|
64
|
+
`"tasksCaptured":74`); a skip message means no `Sprints` row covers today. Confirm `Sprints` /
|
|
65
|
+
`Tasks` repopulated.
|
|
66
|
+
7. **Old folder:** no automation runs there. Close its open tasks by hand.
|
|
60
67
|
|
|
61
68
|
## Gotchas
|
|
62
69
|
|
|
@@ -68,12 +75,16 @@ constant edit. Done once on 2026-09-16 under TRUE-82052 (worker2 + dbchanges2).
|
|
|
68
75
|
- **Truncate throws away sprint history.** The alternative is renumbering the old rows. On
|
|
69
76
|
2026-09-16 the developer chose truncate and declined a backup — if history matters next time,
|
|
70
77
|
renumber, or back up `Sprints` / `Tasks` / `Tasks_Developers` first.
|
|
71
|
-
- **
|
|
72
|
-
|
|
73
|
-
|
|
78
|
+
- **Cron rows do not need changing.** They name actions, not folders. Schedule is recorded in
|
|
79
|
+
`dbchanges2/Core/2026-09-29a - Record sprint and design CronJobs.sql` — see
|
|
80
|
+
[team-sprint-management](../features/team-sprint-management.md#the-sprint-lifecycle--cron-schedule-prod-corecronjobs-central-time).
|
|
74
81
|
|
|
75
82
|
## Related
|
|
76
83
|
|
|
77
84
|
- [Team Sprint Management & Reporting](../features/team-sprint-management.md)
|
|
78
85
|
- [Creating Worker Actions](../features/creating-worker-actions.md)
|
|
79
86
|
- [dbchanges2 Architecture](../../dbchanges2/architecture.md)
|
|
87
|
+
|
|
88
|
+
## Change history
|
|
89
|
+
- 2026-10-01 — Folder id now config-first (`devSprintsFolderId()`); field-setup step; cron rows now in git (ajean)
|
|
90
|
+
- 2026-09-16 — First repoint, TRUE-82052 (kyalamarthi)
|
package/knowledge/INDEX.md
CHANGED
|
@@ -20,7 +20,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
20
20
|
## 2.0 framework
|
|
21
21
|
|
|
22
22
|
- **_underscore** (_Underscore) _(framework core)_ — 94 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
23
|
-
- **worker2** (Worker) —
|
|
23
|
+
- **worker2** (Worker) — 81 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
24
24
|
- **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
25
25
|
- **dbchanges2** (Database Changes) _(framework core)_ — 22 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
|
26
26
|
- **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
|
|
@@ -2,4 +2,5 @@
|
|
|
2
2
|
|
|
3
3
|
| Doc | Framework | Summary |
|
|
4
4
|
|-----|-----------|---------|
|
|
5
|
-
| [
|
|
5
|
+
| [WJE: TOGaDesk → TOGa 2 ticket-writes monitor (Monitor/Wje/TicketWritesToToga2)](features/oneuptime-ticket-writes-monitor.md) | 2.0 | worker2 OneUptime push monitor counting WJE TOGaDesk tickets/replies/comments never written to TOGa 2 (NULL referenceId); open when working on WJE bridge monito |
|
|
6
|
+
| [Wiss, Janney, Elstner Associates, Inc.](profile.md) | 2.0 | WJE (Wiss, Janney, Elstner / "WJE IT") Freshservice help-desk client, single hardcoded tenant; open for the three sync paths, the TOGaDesk bridge ids/department |
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "WJE: TOGaDesk → TOGa 2 ticket-writes monitor (Monitor/Wje/TicketWritesToToga2)"
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: wje
|
|
7
|
+
type: client-feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["mhammontree"]
|
|
11
|
+
files:
|
|
12
|
+
- worker2/Worker/Monitor/Wje.php
|
|
13
|
+
- dbchanges2/Core/2026-09-30b - Insert WJE Ticket Writes Monitor CronJob.sql
|
|
14
|
+
- worker/crons/toga2/wje/sync_togasupply_wje.php
|
|
15
|
+
- library/app/api/toga2.php
|
|
16
|
+
related:
|
|
17
|
+
- ../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md
|
|
18
|
+
- ../../../1.0/apps/library/features/toga2-api-client-and-bridge.md
|
|
19
|
+
- ../../../2.0/apps/worker2/features/wje-freshservice-sync.md
|
|
20
|
+
- ../profile.md
|
|
21
|
+
---
|
|
22
|
+
worker2 OneUptime push monitor counting WJE TOGaDesk tickets/replies/comments never written to TOGa 2 (NULL referenceId); open when working on WJE bridge monitoring, its departments, or clearing a stuck ticket.
|
|
23
|
+
|
|
24
|
+
## Summary
|
|
25
|
+
`Monitor/Wje/TicketWritesToToga2` (`_Worker_Monitor_Wje`, TRUE-80523) watches the 1.0 → 2.0 half of the WJE TOGaDesk bridge. It follows the shared [OneUptime push-metric pattern](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md) — read that for payload contract, criteria, and provisioning.
|
|
26
|
+
|
|
27
|
+
- **Status 2026-10-01:** committed on branch `TRUE-80523` (worker2 + dbchanges2 + worker comment), **not deployed**. Cron seed ships `isActive = 0`; `$ONEUPTIME_URL` is still empty (push credential — never record it).
|
|
28
|
+
- Runs in worker2, outside the 1.0 worker it watches. Reads `TOGaDeskSupport` via `_underscore::DB_TOGADESK1` (see [worker2 legacy DB access](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md)).
|
|
29
|
+
- OneUptime monitor name: **"WJE Ticket Writes to TOGa 2"**.
|
|
30
|
+
|
|
31
|
+
## How it works
|
|
32
|
+
|
|
33
|
+
### Why a backlog, not a checkpoint-freshness check
|
|
34
|
+
The bridge catches a per-ticket exception → Sentry only, and **still advances `TOGA_LAST_TICKET_INTEGRATION_DATETIME` past it**. The failed ticket is never retried unless touched again. A checkpoint-age monitor would read healthy. A row with NULL `referenceId` catches both a dead cron and silent per-ticket failures (link model: [bridge doc](../../../1.0/apps/library/features/toga2-api-client-and-bridge.md)).
|
|
35
|
+
|
|
36
|
+
### What it counts
|
|
37
|
+
WJE rows (`tickets.clientid = 153`) in `tickets`, `tickets_replies`, `comments` where `referenceId` is NULL or `''`, created on/after `BACKLOG_CUTOFF_DATE = '2025-05-01'`, in the configured departments, minus `excludedTicketNumbers`.
|
|
38
|
+
|
|
39
|
+
Skip rules mirror `syncTogaDesk1TicketIntoToga2Ticket` in `library/app/api/toga2.php` (~L12280-12290): a reply with a blank message falls back to `newstatus` and is skipped if still PHP-empty (NULL, `''`, `'0'`) — it never gets a `referenceId`, so the monitor skips it too. Comments are always sent.
|
|
40
|
+
|
|
41
|
+
### Decision + payload
|
|
42
|
+
- `alarm: HIGH` when any counted row is `>= alertThresholdMinutes` old; else `alarm: OK`.
|
|
43
|
+
- `status: error` on DB failure (payload carries **only** `errorClass` + `errorNumber` — the exception message can hold SQL, DB host, DB user) and on empty `departmentIds` (`error: no_departments_configured`).
|
|
44
|
+
- Body: `latePendingCount`, `pendingCount`, per-type counts, `oldestPendingMinutes`, `alertThresholdMinutes`, `lateTicketNumbers` (top 10), `departmentIds`, `excludedTicketCount`.
|
|
45
|
+
|
|
46
|
+
### Parameters (`Core.CronJobs.parameters`, no deploy to change)
|
|
47
|
+
`{"alertThresholdMinutes":15,"departmentIds":[269],"excludedTicketNumbers":[]}`
|
|
48
|
+
- `departmentIds` PHP default is `[]` **on purpose** — the DB row is the only home of the list. Missing list → `status: error`, never a silent hardcoded dept. The cron comment in `sync_togasupply_wje.php` points here; keep the two in step if the sync's departments change.
|
|
49
|
+
- `excludedTicketNumbers` — TOGaDesk ticket numbers the team gives up on, so one permanently failing ticket does not hold the alarm red.
|
|
50
|
+
|
|
51
|
+
### Clear a stuck ticket (runbook)
|
|
52
|
+
1. Fix the cause (Sentry has the exception).
|
|
53
|
+
2. Add a reply to the ticket in TOGaDesk so the sync picks it up again — or move the checkpoint back.
|
|
54
|
+
3. If abandoning it, add its ticket number to `excludedTicketNumbers`.
|
|
55
|
+
|
|
56
|
+
### Activation (in order)
|
|
57
|
+
1. Create the OneUptime Incoming Request monitor with **body rules only**: Contains `"alarm":"HIGH"` → Offline + incident; Contains `"status":"error"` → Offline + incident; Contains `"alarm":"OK"` → Online.
|
|
58
|
+
2. Paste its URL into `$ONEUPTIME_URL`.
|
|
59
|
+
3. Push and deploy worker2.
|
|
60
|
+
4. Run the Core seed `2026-09-30b - Insert WJE Ticket Writes Monitor CronJob.sql`.
|
|
61
|
+
5. Test HIGH: set `departmentIds` to `[290]` (has old unsent rows), hand-run. Restore `[269]`, hand-run to see recovery.
|
|
62
|
+
6. Set `isActive = 1`.
|
|
63
|
+
7. Only then add the not-received 10/15 min rules (see the activation-ordering trap in the shared doc).
|
|
64
|
+
|
|
65
|
+
## Gotchas
|
|
66
|
+
- **Dept 269 is very low volume** (Sept 2026: 3 tickets, 7 replies, 2 comments). "No writes" is normal — this is why it is a backlog monitor, not a "writes stopped" monitor.
|
|
67
|
+
- **Blind spot: field-only edits.** Priority/assignee change on an already-linked ticket creates no row, so a failed push is invisible. Status changes are covered (they arrive as replies).
|
|
68
|
+
- **Clock is US/Central.** `TOGaDeskSupport` `NOW()` and row timestamps are both server US/Central (the `_underscore` driver sets no session `time_zone`). Spring DST can false-alarm once; autumn can delay an alarm up to 1h.
|
|
69
|
+
- **Pre-cutoff history never drains:** 21 sendable unsent dept-269 rows 2024-12 → 2025-04-22 (early integration). That is why the cutoff is 2025-05-01.
|
|
70
|
+
- **Query cost (prod, verified):** `tickets` via index_merge(departmentid, clientid); `tickets_replies` via `ticketid` index; `comments` has **no** `ticketid` index → full scan of ~13k rows (acceptable). Timestamp columns have no `ON UPDATE`.
|
|
71
|
+
- **Do not add 290 casually.** Dept 290 is not synced 1.0 → 2.0 today (see [profile](../profile.md)); pointing the monitor at it would alarm on the known backlog, not a new failure.
|
|
72
|
+
- **The ClickUp TRUE-80523 description is fabricated** (Talos auto-generated) — see [MCP tool usage gotcha](../../../standalone/apps/claude/workflows/mcp-tool-usage.md). Trust this doc and the code.
|
|
73
|
+
|
|
74
|
+
## Related
|
|
75
|
+
- [OneUptime push-metric monitors for 2.0 workers](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md) — shared pattern
|
|
76
|
+
- [App_Api_Toga2 bridge](../../../1.0/apps/library/features/toga2-api-client-and-bridge.md) — `syncWithTogadesk`, `referenceId` write-back
|
|
77
|
+
- [WJE profile](../profile.md)
|
|
78
|
+
|
|
79
|
+
## Change history
|
|
80
|
+
- 2026-10-01 — Created: monitor built on branch TRUE-80523, not deployed, cron `isActive = 0`. (mhammontree)
|
|
@@ -4,22 +4,26 @@ framework: "2.0"
|
|
|
4
4
|
apps:
|
|
5
5
|
- worker
|
|
6
6
|
- library
|
|
7
|
+
- worker2
|
|
8
|
+
- dbchanges2
|
|
7
9
|
project: Library
|
|
8
10
|
client: wje
|
|
9
11
|
type: profile
|
|
10
12
|
status: active
|
|
11
|
-
updated: 2026-
|
|
12
|
-
owners: [jcardinal]
|
|
13
|
+
updated: 2026-10-01
|
|
14
|
+
owners: [jcardinal, mhammontree]
|
|
13
15
|
files:
|
|
14
16
|
- worker/crons/toga2/wje/sync_togasupply_wje.php
|
|
15
17
|
- worker2/Worker/Wje.php
|
|
16
18
|
- worker/crons/toga2/wje/import_supporting_records.php
|
|
19
|
+
- worker2/Worker/Monitor/Wje.php
|
|
17
20
|
related:
|
|
18
21
|
- ../../1.0/apps/library/features/toga2-api-client-and-bridge.md
|
|
19
22
|
- ../../2.0/apps/worker2/features/wje-freshservice-sync.md
|
|
23
|
+
- ./features/oneuptime-ticket-writes-monitor.md
|
|
20
24
|
---
|
|
21
25
|
|
|
22
|
-
WJE (Wiss, Janney, Elstner / "WJE IT") Freshservice help-desk client, single hardcoded tenant; open for the three sync paths
|
|
26
|
+
WJE (Wiss, Janney, Elstner / "WJE IT") Freshservice help-desk client, single hardcoded tenant; open for the three sync paths, the TOGaDesk bridge ids/departments, and its monitor.
|
|
23
27
|
|
|
24
28
|
## Summary
|
|
25
29
|
|
|
@@ -31,9 +35,19 @@ Data moves between WJE and TOGA on three paths:
|
|
|
31
35
|
|
|
32
36
|
1. **Real-time Freshservice webhook** → `_Worker_Wje::Webhook` (worker2) — creates/updates tickets, contacts, assets, groups, categories, and conversation notes, keyed by `c_wje*` custom fields.
|
|
33
37
|
2. **Daily batch import** (`worker/crons/toga2/wje/import_supporting_records.php`) — imports canned responses, requesters, assets, groups, and the 3-level category hierarchy.
|
|
34
|
-
3. **2-minute TOGaDesk bridge** (`App_Api_Toga2::syncWithTogadesk`) — the cron `worker/crons/toga2/wje/sync_togasupply_wje.php` enables the **full** set of bridge sub-syncs (tickets both directions, ticket-notes, contacts→people, items/units→assets, predefined-replies, ticket-teams→groups, ticket-categories) for TOGaDesk client id 153
|
|
38
|
+
3. **2-minute TOGaDesk bridge** (`App_Api_Toga2::syncWithTogadesk`) — the cron `worker/crons/toga2/wje/sync_togasupply_wje.php` enables the **full** set of bridge sub-syncs (tickets both directions, ticket-notes, contacts→people, items/units→assets, predefined-replies, ticket-teams→groups, ticket-categories) for TOGaDesk client id 153. **TOGaDesk → 2.0 syncs department 269 only.**
|
|
39
|
+
|
|
40
|
+
Monitoring: [ticket-writes monitor](./features/oneuptime-ticket-writes-monitor.md) alarms when a dept-269 TOGaDesk row is not written to 2.0.
|
|
41
|
+
|
|
42
|
+
## TOGaDesk departments
|
|
43
|
+
|
|
44
|
+
- **269** — tickets created in TOGaDesk (`153000xxx` numbers). Very low volume. Synced TOGaDesk → 2.0.
|
|
45
|
+
- **290** — WJE Freshservice tickets that arrive in TOGaDesk **from** 2.0 (ticket number = Freshservice INC number, e.g. `65239`; ~100-200/month, all carry `referenceId`). The cron defines `TOGADESK_WJE_INTERNAL_TICKETS_DEPARTMENT_ID = 290` but never passes it (`$monitorTogadeskDepartmentIds` not passed), so 290 is **not** synced TOGaDesk → 2.0. Confirmed with Jonathan 2026-09-30: only 269 is synced today.
|
|
46
|
+
|
|
47
|
+
**⚠ Known gap (not fixed):** a reply or comment typed in TOGaDesk on a dept-290 ticket is never written to 2.0 and never reaches WJE. Verified: ticket `65239`, a reply on 2026-08-18 has NULL `referenceId` and is absent from `Client_Wje.TicketNotes`. Since 2026-01 ~49 replies + 10 comments are stuck. **If 290 is ever added to the sync, its first run pushes that whole old backlog to WJE at once — decide what to do with it first.**
|
|
35
48
|
|
|
36
49
|
## Related
|
|
37
50
|
|
|
38
|
-
- **[wje-freshservice-sync](../../2.0/apps/worker2/features/wje-freshservice-sync.md)** — full integration: webhook handler, import cron, `c_wje*` models, API clients, schema, reports,
|
|
51
|
+
- **[wje-freshservice-sync](../../2.0/apps/worker2/features/wje-freshservice-sync.md)** — full integration: webhook handler, import cron, `c_wje*` models, API clients, schema, reports, gotchas.
|
|
39
52
|
- **[toga2-api-client-and-bridge](../../1.0/apps/library/features/toga2-api-client-and-bridge.md)** — the `syncWithTogadesk` bridge mechanism.
|
|
53
|
+
- **[oneuptime-ticket-writes-monitor](./features/oneuptime-ticket-writes-monitor.md)** — the 1.0 → 2.0 write monitor.
|
|
@@ -6,8 +6,8 @@ project: Claude Harness
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["jcardinal", "tcox", "ajean"]
|
|
9
|
+
updated: 2026-10-01
|
|
10
|
+
owners: ["jcardinal", "tcox", "ajean", "mhammontree"]
|
|
11
11
|
files:
|
|
12
12
|
- claude/.claude/skills/kickoff/SKILL.md
|
|
13
13
|
- claude/.claude/skills/capture/SKILL.md
|
|
@@ -106,6 +106,7 @@ SELECT name, clientIdentifier FROM Clients WHERE name LIKE '%<keyword>%';
|
|
|
106
106
|
- **⚠ "Beta" is `dev-sandbox`, NOT `client-beta`.** For the beta API tier (`api.beta.togahub.com`), `Core.DatabaseHosts` maps environment `beta` for `Client_*` schemas to the **dev-sandbox** cluster — so "the beta database" and "the dev-sandbox database" are the same host. The MCP's `client-beta` environment exposes **no schemas** and querying it looks like the data is missing. Query **`dev-sandbox`** whenever someone says "beta". Any frontend pointed at `VITE_API=https://api.beta.togahub.com/v2` is therefore reading/writing dev-sandbox data. (2024-era onboarding SQL shows beta on a us-west-2 `client-cluster`; that has since been consolidated — trust `Core.DatabaseHosts`, not old migrations.)
|
|
107
107
|
- **⚠ The write-blocker is a naive string scan — it rejects `UPDATE`/`REPLACE` even inside a string literal.** `toga_query` refuses the whole statement if those words appear anywhere in it, so a read-only query like `… WHERE output LIKE '%skipped update%'` fails validation. **A column alias trips it too** — `… AS updateDisabled` is rejected on the same scan, so the alias has to be renamed (`skippedDisabled`), not just the predicate. Work around it by matching on a fragment that avoids the keyword (`LIKE '%disabled%'`, `LIKE '%skipped%'`), or by pulling the rows and filtering client-side — don't conclude the data isn't there.
|
|
108
108
|
- **Production sessions render in US/Central (UTC−5); most other timestamps you're comparing are UTC.** GitHub/API/webhook timestamps are UTC, so a naive `dtStamp` vs. commit-time comparison is off by five hours — enough to make a healthy worker fleet look dead. Anchor both sides to UTC (or `CONVERT_TZ`) before drawing a conclusion about "when did this last run".
|
|
109
|
+
- **⚠ Talos auto-generated "KB investigation" text on a ClickUp ticket can be fabricated.** TRUE-80523's description called WJE "Write Job Entry" and cited a CloudWatch/Datadog ADR, a DLQ, and a Migration Sprint retro — none exist. Treat such write-ups as unverified; check every name and source against the code, this KB, and the data before acting on it.
|
|
109
110
|
- **`local` is not this MCP.** If someone means their own machine's database, use the `mysql` CLI via shell, not `toga-db`.
|
|
110
111
|
- **The client roster is wider than `registry.json`, and identifiers are not guessable.** `Core.Clients` held **46 clients** on 2026-07-30 against 17 in `registry.json`. A client absent from the registry is normal, not an error — the registry tracks clients whose *code knowledge* has been captured, not the full book of business. Identifiers do not follow from the display name, so **look them up, never construct them**:
|
|
111
112
|
|
|
@@ -123,6 +124,7 @@ SELECT name, clientIdentifier FROM Clients WHERE name LIKE '%<keyword>%';
|
|
|
123
124
|
Schema names are **case-sensitive** — copy the exact value from `toga_list_schemas` or `Core.Clients`. `Client_Aig` works; `Client_AIG` and `client_aig` do not.
|
|
124
125
|
|
|
125
126
|
## Change history
|
|
127
|
+
- 2026-10-01 — Added gotcha: Talos auto-generated ClickUp ticket write-ups can invent sources (TRUE-80523). (mhammontree)
|
|
126
128
|
- 2026-09-25 — corrected the "TOGa IQ" row: it now names the separate `toga25-iq` app, not the `talos` codebase (rebrand) (jcardinal)
|
|
127
129
|
|
|
128
130
|
## Related
|
package/package.json
CHANGED