toga-ai 1.0.220 → 1.0.222
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/worker/INDEX.md +1 -1
- package/knowledge/1.0/apps/worker/features/forecast2-netsuite-reconciliation.md +78 -1
- package/knowledge/2.0/apps/worker2/INDEX.md +3 -0
- package/knowledge/2.0/apps/worker2/features/clickup-richtext-api.md +133 -0
- package/knowledge/2.0/apps/worker2/features/talos-meeting-notes-integration.md +97 -0
- package/knowledge/2.0/apps/worker2/workflows/ticket-to-pseudocode-planning.md +60 -0
- package/knowledge/INDEX.md +1 -1
- package/package.json +1 -1
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
| [Worker (1.0 Framework) Architecture](architecture.md) | `worker` is the legacy (**1.0** `App_` framework) **background-job tier**. | worker/index.php, worker/_/app/framework.php, worker/crons/, worker/schedules/, worker/ebs/cron.worker.php, worker/.ebextensions/035_cron.worker.config |
|
|
6
6
|
| [Compass MA Sales Order Exception Report](features/compass-ma-sales-order-exception-report.md) | A worker cron that emails operations the "Compass Refresh Exception Report" — Compass `MA%` sales orders whose corresponding Office Depot (ODP) sales order has | worker/crons/toga2/compass/workflow/7_generate_ma_sales_order_exception_report.php |
|
|
7
7
|
| [Compass Partial In-Transit & Delivered Emails (per package)](features/compass-partial-in-transit-delivered-emails.md) | Compass USA and Compass Canada send a **per-package** in-transit email (and a matching delivered email) instead of one email listing the whole order. | worker/crons/toga2/compass/update_salesorder_status_from_odp.php, worker/crons/toga2/compasscanada/workflow/3_update_salesorder_status_from_grand_and_toy.php |
|
|
8
|
-
| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
|
|
8
|
+
| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/checker.php, test/@dave/looper.php, test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
|
|
9
9
|
| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql |
|
|
10
10
|
| [Prudential: Send Shipments for the Day report (daily cron)](features/send-shipments-for-the-day.md) | Daily cron (9:00 PM) that emails Prudential and Dell stakeholders an Excel report of all devices shipped that day, including tracking number, serial number, emp | worker/crons/notifications/reports/send_shipments_for_the_day.php |
|
|
11
11
|
| [Onboarding a Client to the NetSuite TOGa Supply Sync](workflows/onboarding-client-to-netsuite-togasupply-sync.md) | How to add a new TOGa 2 client to the per-client NetSuite → TOGa Supply importer (`worker/crons/toga2/netsuite/`). | worker/crons/toga2/netsuite/sync_togasupply.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql |
|
|
@@ -6,9 +6,11 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-06-
|
|
9
|
+
updated: 2026-06-26
|
|
10
10
|
owners: [dfranks]
|
|
11
11
|
files:
|
|
12
|
+
- test/@dave/checker.php
|
|
13
|
+
- test/@dave/looper.php
|
|
12
14
|
- test/@dave/reconcile_netsuite_totals.php
|
|
13
15
|
- test/@dave/fixer.php
|
|
14
16
|
- test/@dave/analyze_netsuite_forecast_diff.php
|
|
@@ -42,6 +44,37 @@ by reconciling a chosen tranDate range directly against NetSuite.
|
|
|
42
44
|
|
|
43
45
|
## Key files / entry points
|
|
44
46
|
|
|
47
|
+
- `checker.php [--from --to] [--category sales|openorders|opportunities|all] [--verbose] [--no-banner] [--fix --prod]`
|
|
48
|
+
— **hyper-fast drift detector** using a **moment-fingerprint drill-down** (default window = YTD).
|
|
49
|
+
Instead of materializing every row on both sides and diffing (the OOM trap that gates `reconcile`'s
|
|
50
|
+
per-txn diffs), each side computes a **5-number fingerprint PER TIME BUCKET** entirely in-DB in one
|
|
51
|
+
`GROUP BY`: `n` = COUNT of non-zero-measure records; `rev` = `ROUND(SUM(revenue),2)`;
|
|
52
|
+
`profit` = `ROUND(SUM(profit),2)` (Sales only); `idSum` = `MOD(SUM(MOD(id,P)),P)`;
|
|
53
|
+
`idSq` = `MOD(SUM(MOD(id*id,P)),P)`, where `P = 2147483647` (2³¹−1 Mersenne prime). `(rev,profit)`
|
|
54
|
+
catch **value drift**; `(n,idSum,idSq)` are a **power-sum fingerprint of the id SET** that catches the
|
|
55
|
+
**compensating case** a plain SUM total can't see (a missing txn masked by an extra txn of equal value).
|
|
56
|
+
Power-sums are pure integer arithmetic, so they compute **identically on Oracle-flavored SuiteQL and
|
|
57
|
+
MySQL** (no cross-engine hash-portability problem); `MOD` by the prime keeps the running sums inside
|
|
58
|
+
64-bit. Compared **top-down, descending only where fingerprints disagree**: L1 `GROUP BY` month → L2
|
|
59
|
+
`GROUP BY` day (only inside bad months) → L3 per-id list (`--verbose`, reporting only). A clean system
|
|
60
|
+
**stops at L1 in seconds pulling no rows**. Match identity is the **NetSuite internal id**
|
|
61
|
+
(`netsuiteTransactionInternalId` / `netsuiteSalesOrderInternalId` / `netsuiteOpportunityInternalId`);
|
|
62
|
+
date + category are only **scoping/localization**, never identity. Identity (count + id-moments) is
|
|
63
|
+
computed over **non-zero-measure rows only**, so checker's "in sync" verdict equals fixer.php's "nothing
|
|
64
|
+
to fix" **by construction** (a missing $0 row is a non-issue). On drift it hands `fixer.php` a **tight
|
|
65
|
+
day window** (window handoff, not id handoff — see decision below). `--fix` requires `--prod` (auto-runs
|
|
66
|
+
fixer on the localized window); a **bare run never writes**. Reads the prod core2 **read-replica**
|
|
67
|
+
directly (same pattern as `reconcile_netsuite_totals.php` / `fixer.php` — creds live in
|
|
68
|
+
`worker/config.worker.ini` / `CLAUDE.md`, not reproduced here).
|
|
69
|
+
- `looper.php [--interval=60] [--count=0] [--cmd "<verbatim>"] [--args "<extra checker flags>"] [--no-banner] [--no-fix]`
|
|
70
|
+
— **generic loop runner**, self-contained (no framework bootstrap; shells out via `passthru`). **DEFAULT
|
|
71
|
+
BEHAVIOR (bare `php looper.php`): loops `checker.php --no-banner --fix --prod` every 60s — i.e. it
|
|
72
|
+
continuously AUTO-CORRECTS PRODUCTION** (prints a bold warning Mode line; the once-only LOOPER ASCII
|
|
73
|
+
banner gets the subline "Looper / Checker / Fixer" in fix-prod mode). `--no-fix` gives a **read-only
|
|
74
|
+
detect loop**. `--count 0` = forever; `--cmd` loops a verbatim command instead of checker (no flags
|
|
75
|
+
added); `--args` appends extra checker flags. Suppresses checker's own per-run banner; prints per-run
|
|
76
|
+
dividers with run #, timestamp, exit code, elapsed. Rationale: continuous unattended drift monitoring +
|
|
77
|
+
auto-correction.
|
|
45
78
|
- `reconcile_netsuite_totals.php [from] [to] [--verbose]` — category grand totals NS vs Forecast2
|
|
46
79
|
(Sales, Sales Profit, Open Orders, Opportunities) with deltas. Read-only. Connects explicitly to
|
|
47
80
|
the **prod core2 reader**; NetSuite via SuiteQL SUMs. `--verbose` additionally prints the
|
|
@@ -141,6 +174,18 @@ by reconciling a chosen tranDate range directly against NetSuite.
|
|
|
141
174
|
stays bounded under the existing 1G `memory_limit`. (This corrected a prior docblock claim that `$sqls`
|
|
142
175
|
was never accumulated globally.)
|
|
143
176
|
|
|
177
|
+
- **checker → fixer handoff is WINDOW-based, not id-based (decision).** checker hands fixer a tight day
|
|
178
|
+
window and lets fixer **re-find** the discrepant ids inside it, rather than passing exact ids. Rationale:
|
|
179
|
+
maker≠checker independence (fixer re-derives the drift), and checker has already shrunk the window, so a
|
|
180
|
+
re-scan is cheap. Id-passing only wins when drift is **sparse across a wide span** (it would skip fixer
|
|
181
|
+
re-scanning clean gaps). **Deferred (NOT built):** (a) an `--ids` skip-find path so checker passes exact
|
|
182
|
+
ids; (b) a `--since` "recency-pruned fingerprint" mode that uses NS `lastmodifieddate` to choose which
|
|
183
|
+
`trandate` buckets to fingerprint, walking backward until clean. ⚠ A `--since` mode has **structural blind
|
|
184
|
+
spots**: pure NS **deletions** (orphan Forecast rows) and **old untouched drift** are invisible to a
|
|
185
|
+
lastmodified scan, so the full `trandate` fingerprint stays the authoritative backstop. Forecast tables key
|
|
186
|
+
on `tranDate`, not NS `lastmodifieddate`, so a **symmetric** lastmodified fingerprint isn't possible — it
|
|
187
|
+
would require NS-pull-then-id-lookup or the recency-pruned approach.
|
|
188
|
+
|
|
144
189
|
## Data model
|
|
145
190
|
|
|
146
191
|
`Forecast.Sales`, `Forecast.OpenOrderItems` on the **core2** cluster
|
|
@@ -206,6 +251,24 @@ None — Forecast2 is a single shared dataset.
|
|
|
206
251
|
concentrated **Dec 2025–Apr 2026**, driven by NetSuite re-valuing `costestimate` on older
|
|
207
252
|
invoices that no `lastmodifieddate`-based sync re-pulls. (Revenue reconciles to the penny; only
|
|
208
253
|
profit drifts.)
|
|
254
|
+
- **Open-orders "round-then-sum" vs "sum-then-round" — the most broadly reusable rounding gotcha.**
|
|
255
|
+
`Forecast.OpenOrderItems.revenue`/`profit` is `decimal(14,2)`: Forecast stores each open-order **LINE's**
|
|
256
|
+
revenue (`qtyOpen × rate`) **rounded to cents on store, then sums** (`SUM(round(line,2))`). A NetSuite-side
|
|
257
|
+
aggregate that does `ROUND(SUM(qtyOpen*rate),2)` (**sum-then-round**) diverges by up to ~½¢ **per line** —
|
|
258
|
+
benign rounding noise that **scales with line count**, so **no fixed aggregate-dollar tolerance can separate
|
|
259
|
+
it from real drift** (a busy bucket's benign band can reach whole dollars). This produced an **unfixable
|
|
260
|
+
phantom**: `fixer.php`'s FIND flags such a penny SO, but its FIX compares **per-line**, finds every line
|
|
261
|
+
already correct to the cent, writes nothing — and the SO churns forever in a loop. **FIX: round each line to
|
|
262
|
+
cents BEFORE summing on the NS side** — `SUM(ROUND(((-tl.quantity) - NVL(tl.quantitybilled,0)) * tl.rate, 2))`
|
|
263
|
+
— so NS matches Forecast's `decimal(14,2)` storage exactly; any remaining diff is then genuinely different
|
|
264
|
+
inputs = real drift. Applied to **both** `checker.php` (open-orders `nsSub` SELECT + HAVING) and `fixer.php`
|
|
265
|
+
`findOpenOrderDiscrepancies`. **Affects ONLY open orders** (a computed `qty×rate` product); Sales/Opportunities
|
|
266
|
+
sum pre-rounded `foreignamount`/`projectedtotal` and were already exact. Verified: full 18-month open-orders
|
|
267
|
+
window went from "drift" to cents-exact in sync after the fix. The previously-reported open-orders deltas were
|
|
268
|
+
entirely this artifact, NOT data drift: the ~$0.04 `reconcile` delta, and the per-SO 1¢ on **SO 6870666**
|
|
269
|
+
(NS 852.06 vs FC 852.07). **Correction to any prior note:** the open-orders **2025-04 "unlocalized month" was
|
|
270
|
+
this round-then-sum noise, NOT NS-live-vs-Forecast-replica timing jitter** — the rounding fix eliminated it,
|
|
271
|
+
disproving the earlier replica-jitter hypothesis.
|
|
209
272
|
- **Compare money at 2 decimals.** DB columns store 2dp but PHP `revenue - cost` carries float
|
|
210
273
|
dust (`313.6` vs `313.60000001`); raw `!=` produced thousands of phantom UPDATEs that re-wrote
|
|
211
274
|
identical values (1,839 on one open-orders run). Both tools now compare `round((float)$x, 2)`.
|
|
@@ -297,6 +360,20 @@ None — Forecast2 is a single shared dataset.
|
|
|
297
360
|
|
|
298
361
|
## Change history
|
|
299
362
|
|
|
363
|
+
- 2026-06-26 — **Added `checker.php` (moment-fingerprint drift detector) + `looper.php` (loop runner), and
|
|
364
|
+
found/fixed the open-orders round-then-sum rounding gotcha.** `checker.php` confirms sync in seconds and
|
|
365
|
+
localizes real drift via a 5-number per-bucket fingerprint (`n,rev,profit,idSum,idSq`; power-sums mod the
|
|
366
|
+
2³¹−1 prime → engine-portable) drilled top-down month→day→id, descending only into mismatched buckets,
|
|
367
|
+
pulling no rows on a clean book; hands fixer a tight **day window** (window handoff, not id — decision
|
|
368
|
+
recorded). `--fix` requires `--prod`; bare run never writes. `looper.php` defaults to looping
|
|
369
|
+
`checker --no-banner --fix --prod` every 60s (continuous prod auto-correction; `--no-fix` = read-only).
|
|
370
|
+
**Rounding gotcha:** `OpenOrderItems` `decimal(14,2)` stores each line round-then-sum, so a NS-side
|
|
371
|
+
sum-then-round aggregate diverges up to ~½¢/line (an unfixable phantom that loops fixer forever); fixed by
|
|
372
|
+
rounding each NS line to cents before summing (`SUM(ROUND(qtyOpen*rate,2))`) in both `checker.php` and
|
|
373
|
+
`fixer.php` `findOpenOrderDiscrepancies` — open-orders only (Sales/Opps already exact). This also
|
|
374
|
+
**disproves the earlier replica-jitter hypothesis** for the open-orders 2025-04 unlocalized month — it was
|
|
375
|
+
the same per-line rounding. Deferred: `--ids` skip-find and a `--since` recency-pruned fingerprint (blind to
|
|
376
|
+
NS deletions + old untouched drift). (dfranks)
|
|
300
377
|
- 2026-06-25 — **`trueup_open_orders` Step 4 now lists the exact per-statement SQL** (`<NS SO id> <SQL>`,
|
|
301
378
|
one line per statement) for both live and `--dry-run` passes, so a dry-run is auditable and any
|
|
302
379
|
unexpected insert/update/delete is traceable to its source order. Implemented via a global `$sqlLog`
|
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
| [Worker (worker2) Architecture](architecture.md) | Worker (repo `worker2`) is an AWS Elastic Beanstalk **Worker Tier** application that processes background jobs. | worker2/Controller/Index.php, worker2/Worker/, worker2/LambdaFunctions/, _underscore/Worker.php |
|
|
6
6
|
| [ClickUp Connectivity Watchdog](features/clickup-connectivity-watchdog.md) | A cron watchdog that emails when the ClickUp integration looks disconnected during business hours. | worker2/Worker/Clickup/Health.php, worker2/Database/ClickupHealthWatchdog.sql |
|
|
7
7
|
| [ClickUp Project & Opportunity Multi-List Routing](features/clickup-project-routing.md) | Routes ClickUp tasks into the correct **secondary multi-list memberships** based on their custom-field values, via the `clickup` webhook. | worker2/Worker/Clickup/Project.php, worker2/Worker/Clickup.php |
|
|
8
|
+
| [ClickUp Rich-Text Custom Fields via Quill Delta (API)](features/clickup-richtext-api.md) | ClickUp custom text fields (type `text` and long-text) support rich formatting only through a **Quill Delta** written to the undocumented `value_richtext` key o | test/@dave/clickup_md2delta.js, .claude/skills/plan-ticket/scripts/clickup.js |
|
|
8
9
|
| [ClickUp Work Type Automation (Committed / Conditional / Stretch)](features/clickup-work-type-automation.md) | The ClickUp webhook handler (`_Worker_Clickup`) automatically maintains each task's **Work Type** custom field — `Committed`, `Conditional`, or `Stretch` — base | worker2/Worker/Clickup.php, worker2/Tests/Worker/ClickupWorkTypeTest.php |
|
|
9
10
|
| [Creating Worker Actions](features/creating-worker-actions.md) | How to add a new callable Worker action — a PHP class whose `public static` methods are invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). | worker2/Worker/, worker2/Controller/Index.php, _underscore/Worker.php |
|
|
10
11
|
| [Elite Freshservice Sync (worker2)](features/elite-freshservice-sync.md) | `_Worker_Elite` processes Freshservice webhook events and syncs them into TOGA 2. | worker2/Worker/Elite.php, worker2/Config/dev-kmaramreddy-laptop.ini |
|
|
@@ -15,7 +16,9 @@
|
|
|
15
16
|
| [NetSuite → Forecast Open-Orders Sync (salesOrder webhook → OpenOrderItems)](features/netsuite-salesorder-open-orders-sync.md) | Webhook-driven, single-record port of the legacy open-orders importer (TRUE-79142). | worker2/Worker/Netsuite/SalesOrder.php, worker2/Worker/Netsuite.php, test/@dave/probe_salesorder_rest_shape.php, test/@dave/probe_open_order_lines.php, test/@dave/check_so_status.php, test/@dave/check_so_history.php, test/@dave/probe_so_rest_lines.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_open_order_gating.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
|
|
16
17
|
| [DB-Driven Notification (Internal) Email](features/notification-email.md) | Internal/notification emails (merge-conflict alerts, ops notices — anything system-generated, not client-facing transactional mail) are sent through one worker | worker2/Worker/Notification/Email.php, _underscore/Model/Client/EmailTemplate.php, dbchanges2/Client/2026-06-23a - EmailTemplateWrapper.sql, dbchanges2/Client_True/2026-06-23a - EmailTemplateWrapper.sql |
|
|
17
18
|
| [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Receives inbound webhook events from Startech (Easeedesk) and creates or updates the corresponding ticket in TOGA 2.0. | worker2/Worker/Startech.php |
|
|
19
|
+
| [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus programmatically. | .claude/skills/plan-ticket/scripts/talos.js |
|
|
18
20
|
| [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php |
|
|
19
21
|
| [Teams Meeting Transcript Export](features/teams-transcript-export.md) | `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for Teams meeting transcripts produced by a set of organizers, classifies ea | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
|
|
20
22
|
| [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | `_Worker_Vapi` ([worker2/Worker/Vapi.php](worker2/Worker/Vapi.php)) is the **PHP side of the AI-BDR call loop** — the webhook that receives VAPI's end-of-call r | worker2/Worker/Vapi.php, worker2/Worker/Ai/Bdr/Vapi.php |
|
|
21
23
|
| [WJE Freshservice Sync (worker2)](features/wje-freshservice-sync.md) | WJE ("WJE IT", helpdesk `wje.freshservice.com`) is a **Freshservice**-based help-desk client whose tickets, contacts, assets, groups, categories, and canned res | worker2/Worker/Wje.php, _underscore/Component/Api/Wje/Wje.php, _underscore/Model/Wje/Ticket.php, _underscore/Model/Wje/TicketNote.php, _underscore/Model/Wje/Contact.php, _underscore/Model/Wje/Unit.php, _underscore/Model/Wje/TicketTeam.php, _underscore/Model/Wje/TicketCategory.php, _underscore/Model/Wje/AssetType.php, _underscore/Model/Wje/PredefinedReply.php, library/app/api/wje.php, worker/crons/toga2/wje/import_supporting_records.php, worker/crons/toga2/wje/sync_togasupply_wje.php, worker/crons/notifications/reports/wje/wje_common.php, library/app/systemmonitor/wje.php, dbchanges2/Client_Wje/2024-10-04 - WjeOnboarding.sql |
|
|
24
|
+
| [Ticket → ClickUp Pseudocode Planning (Talos-grounded)](workflows/ticket-to-pseudocode-planning.md) | A repeatable procedure for turning a ClickUp ticket into a reviewed, formatted implementation plan posted back to the ticket's `📝 Pseudocode` custom field. | test/@dave/clickup_md2delta.js |
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: ClickUp Rich-Text Custom Fields via Quill Delta (API)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-06-26
|
|
10
|
+
owners: [dfranks]
|
|
11
|
+
files:
|
|
12
|
+
- test/@dave/clickup_md2delta.js
|
|
13
|
+
- .claude/skills/plan-ticket/scripts/clickup.js
|
|
14
|
+
related:
|
|
15
|
+
- ./netsuite-opportunity-sync.md
|
|
16
|
+
- ./clickup-project-routing.md
|
|
17
|
+
- ../workflows/ticket-to-pseudocode-planning.md
|
|
18
|
+
- ./talos-meeting-notes-integration.md
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Summary
|
|
22
|
+
|
|
23
|
+
ClickUp custom text fields (type `text` and long-text) support rich formatting only through a
|
|
24
|
+
**Quill Delta** written to the undocumented `value_richtext` key on the Set-Custom-Field-Value
|
|
25
|
+
endpoint. Markdown or HTML written to the documented `value` key renders as literal characters.
|
|
26
|
+
A companion `value` (plain-text fallback) must accompany every richtext write.
|
|
27
|
+
|
|
28
|
+
## How it works
|
|
29
|
+
|
|
30
|
+
### Endpoint
|
|
31
|
+
|
|
32
|
+
`POST https://api.clickup.com/api/v2/task/<taskId>/field/<fieldId>?custom_task_ids=true&team_id=<CLICKUP_TEAM_ID>`
|
|
33
|
+
|
|
34
|
+
Header: `Authorization: <CLICKUP_API_KEY>` — no `Bearer` prefix.
|
|
35
|
+
|
|
36
|
+
### Request body
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"value": "<plain-text fallback>",
|
|
41
|
+
"value_richtext": "{\"ops\":[...]}"
|
|
42
|
+
}
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
`value_richtext` must be the Delta JSON **serialized to a string** — not a nested JSON object.
|
|
46
|
+
The outer JSON body is a normal JSON object; only `value_richtext`'s value is a JSON-encoded string.
|
|
47
|
+
|
|
48
|
+
### Quill Delta format rules
|
|
49
|
+
|
|
50
|
+
A Delta is `{"ops":[...]}`. Each op is `{"insert": "...", "attributes": {...}}`.
|
|
51
|
+
|
|
52
|
+
**Block-level formats** attach to the `\n` op that **terminates the block**; they apply back
|
|
53
|
+
to the previous newline. Block attribute keys:
|
|
54
|
+
|
|
55
|
+
| Format | Key | Values |
|
|
56
|
+
|--------|-----|--------|
|
|
57
|
+
| Heading | `header` | `1`, `2`, `3` |
|
|
58
|
+
| Bullet list | `list` | `"bullet"` |
|
|
59
|
+
| Ordered list | `list` | `"ordered"` |
|
|
60
|
+
| List indent | `indent` | `1`, `2`, … |
|
|
61
|
+
| Code block | `code-block` | `true` |
|
|
62
|
+
| Blockquote | `blockquote` | `true` |
|
|
63
|
+
|
|
64
|
+
**Inline formats** wrap the text `insert` directly:
|
|
65
|
+
|
|
66
|
+
| Format | Key | Value |
|
|
67
|
+
|--------|-----|-------|
|
|
68
|
+
| Bold | `bold` | `true` |
|
|
69
|
+
| Italic | `italic` | `true` |
|
|
70
|
+
| Inline code | `code` | `true` |
|
|
71
|
+
| Link | `link` | `"https://..."` |
|
|
72
|
+
|
|
73
|
+
**No native table op.** Convert markdown tables to bullet lists. Convert horizontal rules
|
|
74
|
+
(`---`) to a blank line — there is no `hr` op in ClickUp's Delta renderer.
|
|
75
|
+
|
|
76
|
+
Every Delta must end with a bare `{"insert": "\n"}` op.
|
|
77
|
+
|
|
78
|
+
### Minimal example — a heading followed by a bullet list
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"ops": [
|
|
83
|
+
{ "insert": "Overview" },
|
|
84
|
+
{ "insert": "\n", "attributes": { "header": 1 } },
|
|
85
|
+
{ "insert": "First item" },
|
|
86
|
+
{ "insert": "\n", "attributes": { "list": "bullet" } },
|
|
87
|
+
{ "insert": "Second item" },
|
|
88
|
+
{ "insert": "\n", "attributes": { "list": "bullet" } },
|
|
89
|
+
{ "insert": "\n" }
|
|
90
|
+
]
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Reusable converter
|
|
95
|
+
|
|
96
|
+
`test/@dave/clickup_md2delta.js` — converts a Markdown file to a Quill Delta and posts both
|
|
97
|
+
`value` and `value_richtext` to a ClickUp custom field.
|
|
98
|
+
|
|
99
|
+
```sh
|
|
100
|
+
node clickup_md2delta.js <md-path> <TASK_ID> <FIELD_UUID> [--dry]
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
`--dry` prints the computed Delta without posting. Reads `CLICKUP_API_KEY` and
|
|
104
|
+
`CLICKUP_TEAM_ID` from the environment.
|
|
105
|
+
|
|
106
|
+
The converter was used to push a full implementation plan into the `📝 Pseudocode` custom
|
|
107
|
+
field (`ad0e4fc8-ff03-4e5f-a187-7d2cd6d0bc65`) on ticket TRUE-79868. The same mechanism is
|
|
108
|
+
reused by the `clickup.js` helper in the local `plan-ticket` skill — see the
|
|
109
|
+
[ticket-to-pseudocode planning workflow](../workflows/ticket-to-pseudocode-planning.md).
|
|
110
|
+
|
|
111
|
+
## API documentation status
|
|
112
|
+
|
|
113
|
+
ClickUp's official v2 API docs document only a plain-string `value` for text fields and list
|
|
114
|
+
markdown support as an unfulfilled feature request. The `value_richtext` write path is
|
|
115
|
+
**undocumented but empirically verified** (2026-06-26). It works for both `text` and
|
|
116
|
+
long-text field types.
|
|
117
|
+
|
|
118
|
+
## Tooling gotcha — PowerShell ConvertTo-Json
|
|
119
|
+
|
|
120
|
+
PowerShell 5.1 `ConvertTo-Json` pathologically inflates nested strings. A 21 KB
|
|
121
|
+
`value_richtext` string was inflated ~190x (to ~3.8 MB) due to recursive escape-doubling
|
|
122
|
+
of the inner JSON. Avoid `ConvertTo-Json` for ClickUp richtext payloads. Use one of:
|
|
123
|
+
|
|
124
|
+
- **Node `fetch`** — serialize the body with `JSON.stringify()` directly.
|
|
125
|
+
- **PowerShell + file** — write the JSON body to a UTF-8 file, then pass it to
|
|
126
|
+
`Invoke-RestMethod -InFile <path> -ContentType "application/json"`.
|
|
127
|
+
- **`System.Web.Script.Serialization.JavaScriptSerializer`** — correctly serializes a
|
|
128
|
+
pre-built string without re-escaping it.
|
|
129
|
+
|
|
130
|
+
## Change history
|
|
131
|
+
|
|
132
|
+
- 2026-06-26 — Cross-linked to the Talos meeting-notes integration and the ticket-to-pseudocode planning workflow; noted the `plan-ticket` skill reuses this richtext write path (dfranks)
|
|
133
|
+
- 2026-06-26 — Initial doc: Quill Delta richtext write path verified; converter tool documented; PowerShell inflation gotcha added (dfranks)
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-06-26
|
|
10
|
+
owners: [dfranks]
|
|
11
|
+
files:
|
|
12
|
+
- .claude/skills/plan-ticket/scripts/talos.js
|
|
13
|
+
related:
|
|
14
|
+
- ../../talos/architecture.md
|
|
15
|
+
- ./clickup-richtext-api.md
|
|
16
|
+
- ../workflows/ticket-to-pseudocode-planning.md
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Summary
|
|
20
|
+
|
|
21
|
+
How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus
|
|
22
|
+
programmatically. Talos itself (the LangGraph Agent Protocol server) is documented under
|
|
23
|
+
[`2.0/apps/talos/`](../../talos/architecture.md); this doc is the **consumer side** — the
|
|
24
|
+
endpoints, assistant, and auth-refresh flow a worker2-side dev script needs to ground its
|
|
25
|
+
output in meeting notes.
|
|
26
|
+
|
|
27
|
+
**Key distinction:** the `[talos]` key in worker2 config only reaches the **stateless**
|
|
28
|
+
`/api/ai/generate` and `/api/ai/chat` endpoints, which have **no meeting-notes access**.
|
|
29
|
+
Meeting-notes access comes only from **assistants wired to knowledge bases**, reached through
|
|
30
|
+
the Agent Protocol `/threads` + `/threads/{id}/runs/wait` surface with a **user Bearer JWT**.
|
|
31
|
+
|
|
32
|
+
## How it works
|
|
33
|
+
|
|
34
|
+
### Server
|
|
35
|
+
|
|
36
|
+
- Agent Protocol (LangGraph) server: `https://api.togaiq.com`
|
|
37
|
+
- Web app (login / token source): `https://talos.togaiq.com`
|
|
38
|
+
|
|
39
|
+
### Assistants
|
|
40
|
+
|
|
41
|
+
Meeting-notes / knowledge-base queries go to a knowledge-wired **assistant**, not the
|
|
42
|
+
stateless generate endpoint.
|
|
43
|
+
|
|
44
|
+
| Assistant | ID | Use |
|
|
45
|
+
|-----------|-----|-----|
|
|
46
|
+
| **DevCore** | `a5b1833d-b47c-5cb3-8cda-963f1cc74a33` | dev + meeting notes (plan-and-execute agent) |
|
|
47
|
+
| Talos One | — | general |
|
|
48
|
+
| Talos Sales | — | sales |
|
|
49
|
+
| Talos HR | — | HR |
|
|
50
|
+
|
|
51
|
+
### Querying DevCore
|
|
52
|
+
|
|
53
|
+
1. `POST /threads` → returns a thread id.
|
|
54
|
+
2. `POST /threads/{id}/runs/wait` with the assistant id and the user message.
|
|
55
|
+
3. DevCore is a **plan-and-execute** agent that **interrupts for plan approval**
|
|
56
|
+
(agent-inbox schema): the response carries `__interrupt__` with `action: plan_approval`.
|
|
57
|
+
4. **Resume** the run with `command: { resume: [ { "type": "accept", "args": null } ] }`.
|
|
58
|
+
5. The **final answer** is the last item in `messages[]` whose `type` is `"ai"`.
|
|
59
|
+
|
|
60
|
+
### Authentication & token auto-refresh (verified)
|
|
61
|
+
|
|
62
|
+
JWTs are issued by `api-writer.togahub.com` with audience `talos.togaiq.com`. This is the
|
|
63
|
+
standard **TOGA 2.0 `/v2/auth/*` flow** (mirrors `api2/Component/Api/V2/V2.php`).
|
|
64
|
+
|
|
65
|
+
- **Access token** — 1 hour TTL.
|
|
66
|
+
- **Refresh token** — 30 day TTL and **rolls forward** on each refresh (a new refresh token
|
|
67
|
+
may be returned and must replace the stored one).
|
|
68
|
+
- **Refresh call:**
|
|
69
|
+
`POST https://api-writer.togahub.com/v2/auth/refresh?transactionId=<unique-uuid>`
|
|
70
|
+
Header: `Authorization: Bearer <refresh token>`
|
|
71
|
+
Response: `{ isSuccess, data: { tokens: { access [, refresh] } } }`
|
|
72
|
+
|
|
73
|
+
**Gotchas (both verified empirically 2026-06-26):**
|
|
74
|
+
|
|
75
|
+
- The **`/v2` path prefix is REQUIRED** — omitting it returns error **EV-2 "invalid version"**.
|
|
76
|
+
- `transactionId` must be **globally unique per call** — reusing one returns error **EV-5**.
|
|
77
|
+
|
|
78
|
+
The browser obtains the initial token pair via the `talos.togaiq.com` login. A developer can
|
|
79
|
+
copy `accessToken` + `refreshToken` out of the browser session to seed a local tool, after
|
|
80
|
+
which the tool refreshes on its own.
|
|
81
|
+
|
|
82
|
+
## Credential storage (location only)
|
|
83
|
+
|
|
84
|
+
The consumer tool stores its token pair at **`~/.talos/credentials.json`** (user home,
|
|
85
|
+
**outside any repo**). Never commit, echo, or paste the token values anywhere — they are
|
|
86
|
+
secrets. Document only this location.
|
|
87
|
+
|
|
88
|
+
## Consumer helper
|
|
89
|
+
|
|
90
|
+
`.claude/skills/plan-ticket/scripts/talos.js` — performs the auth refresh and runs a DevCore
|
|
91
|
+
query (create thread → run → accept plan interrupt → extract final `ai` message). It is part
|
|
92
|
+
of the **local** `plan-ticket` skill, not the team repo; its source is intentionally not
|
|
93
|
+
mirrored into the knowledge base.
|
|
94
|
+
|
|
95
|
+
## Change history
|
|
96
|
+
|
|
97
|
+
- 2026-06-26 — Initial doc: consumer-side Talos meeting-notes access via DevCore assistant + the `/v2/auth/refresh` token-rotation flow; stateless `[talos]` config vs. assistant distinction; EV-2/EV-5 gotchas; credential location `~/.talos/credentials.json` (dfranks)
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Ticket → ClickUp Pseudocode Planning (Talos-grounded)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: shared
|
|
7
|
+
type: workflow
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-06-26
|
|
10
|
+
owners: [dfranks]
|
|
11
|
+
files:
|
|
12
|
+
- test/@dave/clickup_md2delta.js
|
|
13
|
+
related:
|
|
14
|
+
- ../features/clickup-richtext-api.md
|
|
15
|
+
- ../features/talos-meeting-notes-integration.md
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Summary
|
|
19
|
+
|
|
20
|
+
A repeatable procedure for turning a ClickUp ticket into a reviewed, formatted implementation
|
|
21
|
+
plan posted back to the ticket's `📝 Pseudocode` custom field. It spans three systems —
|
|
22
|
+
ClickUp (read ticket / write field), the codebase (investigation), and Talos DevCore
|
|
23
|
+
(meeting-notes grounding) — so it is a workflow, not a single feature.
|
|
24
|
+
|
|
25
|
+
It is implemented as a **local Claude skill** (`.claude/skills/plan-ticket/`) that is **not**
|
|
26
|
+
in the team repo. This doc records the *shape of the procedure* and the durable integration
|
|
27
|
+
mechanics; it does not duplicate the skill source. The two underlying integrations have their
|
|
28
|
+
own feature docs:
|
|
29
|
+
|
|
30
|
+
- [ClickUp rich-text custom fields via Quill Delta](../features/clickup-richtext-api.md) — how the plan reaches the `📝 Pseudocode` field.
|
|
31
|
+
- [Talos meeting-notes integration](../features/talos-meeting-notes-integration.md) — how the plan is grounded in meeting notes.
|
|
32
|
+
|
|
33
|
+
## Steps
|
|
34
|
+
|
|
35
|
+
1. **Fetch the ticket** by id from ClickUp (`custom_task_ids=true`, `team_id` from env).
|
|
36
|
+
2. **Investigate the codebase FIRST** — establish what already exists before asking Talos, so
|
|
37
|
+
the Talos query is code-informed rather than speculative.
|
|
38
|
+
3. **Query Talos DevCore** (assistant `a5b1833d-b47c-5cb3-8cda-963f1cc74a33`) for
|
|
39
|
+
meeting-notes context, passing the codebase findings as grounding. Accept the
|
|
40
|
+
plan-approval interrupt to get the final answer.
|
|
41
|
+
4. **Synthesize a ≤150-line phased plan**, saved to `test/@dave/approach/<TICKET>.md`.
|
|
42
|
+
5. **Preview & approval gate** — show the plan; iterate until the developer approves.
|
|
43
|
+
6. **Push to the ticket's `📝 Pseudocode` field** as a Quill Delta via the richtext write
|
|
44
|
+
path. The push is **overwrite-guarded**: it refuses to clobber an already-populated field
|
|
45
|
+
unless run with `--force`.
|
|
46
|
+
|
|
47
|
+
## Plan document format
|
|
48
|
+
|
|
49
|
+
`test/@dave/approach/<TICKET>.md` (see also the team convention to store approach plans
|
|
50
|
+
under `test/@dave/approach/`):
|
|
51
|
+
|
|
52
|
+
- Title heading.
|
|
53
|
+
- Bold header line: **Repos / Framework / Client / Sibling**.
|
|
54
|
+
- Sections, in order: **Summary** · **Meeting-notes context** · **What already exists** ·
|
|
55
|
+
**Architecture decision** · **Phases** (each with file paths + pseudocode) · **Testing** ·
|
|
56
|
+
**Risks** · **Owners / open questions** · **Key files**.
|
|
57
|
+
|
|
58
|
+
## Change history
|
|
59
|
+
|
|
60
|
+
- 2026-06-26 — Initial doc: ticket→pseudocode planning procedure (ClickUp read → codebase investigation → Talos DevCore grounding → phased plan → approval gate → guarded richtext push); records the shape, not the local skill source (dfranks)
|
package/knowledge/INDEX.md
CHANGED
|
@@ -17,7 +17,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
17
17
|
## 2.0 framework
|
|
18
18
|
|
|
19
19
|
- **_underscore** (_Underscore) _(framework core)_ — 17 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
20
|
-
- **worker2** (Worker) —
|
|
20
|
+
- **worker2** (Worker) — 20 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
21
21
|
- **api2** (API) — 7 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
22
22
|
- **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
|
23
23
|
- **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
|
package/package.json
CHANGED