toga-ai 1.0.490 → 1.0.492
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/tools/INDEX.md +3 -2
- package/knowledge/1.0/apps/tools/features/errors-curation-console.md +21 -3
- package/knowledge/1.0/apps/tools/features/legacy-email-notifier.md +112 -0
- package/knowledge/1.0/apps/tools/features/mvc-data-access-patterns.md +25 -2
- package/knowledge/1.0/apps/tools/workflows/deploy-to-elastic-beanstalk-al2023.md +68 -2
- package/knowledge/2.0/apps/_underscore/INDEX.md +1 -1
- package/knowledge/2.0/apps/_underscore/features/error-reporting-issue-event.md +42 -4
- package/knowledge/2.0/apps/worker2/features/error-escalation-cron.md +11 -5
- package/knowledge/2.0/apps/worker2/features/monitoring-framework.md +1 -1
- package/knowledge/2.0/standards/backend-php.md +33 -1
- package/knowledge/INDEX.md +1 -1
- package/knowledge/clients/compass-usa/INDEX.md +1 -1
- package/knowledge/clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md +48 -2
- package/package.json +1 -1
|
@@ -8,9 +8,10 @@
|
|
|
8
8
|
| [Design Demo Admin](features/design-demo-admin.md) | A self-serve admin UI at **`/design`** in the SSO-protected **Tools** app that lets the design team publish self-contained "Claude Design" HTML exports as **ver | tools/_/app/design/github.php, tools/mvc/design/get.php, tools/mvc/design/post.php, tools/assets/css/design.css, tools/assets/js/design.js, tools/_/app/frameworkindex.php, tools/_/app/nav.php, tools/composer.json |
|
|
9
9
|
| [Tools — Developers Folder (UUID & Password Generators)](features/developer-tools.md) | The first two tools shipped in the Tools app, both under the **Developers** folder and gated to personas **Development Team** / **TOGa Technology**. | tools/mvc/developers/uuid/get.php, tools/mvc/developers/password/get.php |
|
|
10
10
|
| [/errors Curation Console (Tools → shared Core Logs DB)](features/errors-curation-console.md) | Internal-only triage/curation screen for the 2.0 Issue/Event error-reporting pipeline, built as a 1.0 Tools MVC page reading the **shared Core Logs DB** through | tools/mvc/errors/get.php, tools/mvc/errors/post.php, tools/_/app/nav.php, tools/config.production.ini |
|
|
11
|
-
| [
|
|
11
|
+
| [Legacy Email Notifier (tools /email-migration/notify)](features/legacy-email-notifier.md) | An SSO-gated admin tool at **`/email-migration/notify`** (nav group **Email Migration** > **Legacy Notifier**, personas `['TOGa Technology','Development Team']` | tools/mvc/email-migration/notify/get.php, tools/mvc/email-migration/notify/post.php, tools/_/app/nav.php |
|
|
12
|
+
| [Tools MVC — Routing, CSRF & App_Database Access Patterns](features/mvc-data-access-patterns.md) | The load-bearing 1.0 (`App_`) framework conventions a developer needs when adding a page to the Tools app — URL routing, CSRF, and DB access through `App_Databa | tools/_/app/nav.php, tools/mvc/get.php, library/app/error.php |
|
|
12
13
|
| [Tools Persona-Gated Navigation (App_Nav)](features/persona-gated-navigation.md) | `App_Nav` is the Tools app's two-level, **persona-gated** navigation. | tools/_/app/nav.php, tools/mvc/get.php |
|
|
13
14
|
| [Tools SAML SSO Consumer & Persona-Gated Auth (App_Auth)](features/saml-sso-auth.md) | `App_Auth` is the Tools app's authentication layer: it consumes the SAML gateway `?saml=` handoff (see the 2.0 SAML downstream integration contract), establishe | tools/_/app/auth.php, tools/mvc/sso/initiate/get.php, tools/mvc/sso/get.php, tools/mvc/login/get.php, tools/mvc/login/post.php, tools/mvc/logout/get.php, tools/mvc/get.php, tools/config.production.ini, tools/config.local.ini |
|
|
14
15
|
| [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | > **PER-AI-MODEL, DATA-DRIVEN SCOPING (2026-07-29).** The KB-documents and Vocabulary admin > UIs were refactored from a single hard-coded **"development-team"* | tools/mvc/talos/kb-documents/get.php, tools/mvc/talos/kb-documents/post.php, tools/mvc/talos/knowledge-bases/get.php, tools/mvc/talos/knowledge-bases/post.php, tools/mvc/talos/vocabulary/get.php, tools/mvc/talos/vocabulary/post.php, tools/_/app/talos/s3.php, tools/_/app/talos/bedrock.php, tools/_/app/pg.php, tools/_/app/model/true/aimodels.php, tools/_/app/model/true/vectorindexes.php, tools/_/app/model/true/aimodels_vectorindexes.php, tools/_/app/worker.php, tools/_/app/nav.php, tools/config.production.ini, tools/config.alpha.ini, tools/.platform/httpd/conf.d/timeouts.conf, tools/.platform/hooks/prebuild/01-install-php-pgsql.sh, tools/.platform/hooks/postdeploy/01-restart-php.sh |
|
|
15
16
|
| [Talos Pricing UI (Onboarding, Dashboard, Benchmarks, Cost Factors + Estimator)](features/talos-pricing-ui.md) | The 1.0 (tools app) face of the **Talos Pricing Platform** — a "Talos Pricing" nav folder with four pages plus a client-side estimate engine. | tools/_/app/nav.php, tools/_/app/talos/estimator.php, tools/mvc/talos/onboarding/get.php, tools/mvc/talos/onboarding/post.php, tools/mvc/talos/pricing/get.php, tools/mvc/talos/benchmarks/get.php, tools/mvc/talos/factors/get.php, tools/mvc/talos/factors/post.php, tools/assets/css/style.css |
|
|
16
|
-
| [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the **Tools** 1.0 app boots on Elastic Beanstalk running `PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. | tools/.ebextensions/004_http_to_https.config, tools/.ebextensions/006_mount-s3fs.config, tools/.ebextensions/007_setup_export_cache_folders.config, tools/.ebextensions/008_setup_ldap.config, tools/.ebextensions/009_setup_phpini.config, tools/.ebextensions/020_setup_git_libraries.config, tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config, tools/ebs/git.json |
|
|
17
|
+
| [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the **Tools** 1.0 app boots on Elastic Beanstalk running `PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. | tools/.ebextensions/004_http_to_https.config, tools/.ebextensions/006_mount-s3fs.config, tools/.ebextensions/007_setup_export_cache_folders.config, tools/.ebextensions/008_setup_ldap.config, tools/.ebextensions/009_setup_phpini.config, tools/.ebextensions/020_setup_git_libraries.config, tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config, tools/ebs/git.json, tools/ebs/setup_export_cache_folders.php, tools/_/app/frameworkindex.php |
|
|
@@ -6,7 +6,7 @@ project: Tools
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-08-01
|
|
10
10
|
owners: ["jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- tools/mvc/errors/get.php
|
|
@@ -32,10 +32,10 @@ happening and were deliberately never ticketed. Nobody had that view before.
|
|
|
32
32
|
## How it works
|
|
33
33
|
|
|
34
34
|
- **Curate** subject, description, `minimumUrgency`, assignee, and mute window.
|
|
35
|
-
- **Manage per-client email recipients** (`
|
|
35
|
+
- **Manage per-client email recipients** (`IssueEmailAddress`). `clientId = 0` means **all
|
|
36
36
|
clients** and requires an explicit confirmation flag; the list view renders it as
|
|
37
37
|
**"ALL CLIENTS"**.
|
|
38
|
-
- **Merge fingerprints** — repoint an `
|
|
38
|
+
- **Merge fingerprints** — repoint an `IssueFingerprint` row at an existing Issue. Without this,
|
|
39
39
|
the fingerprint/Issue split is inert: a refactor that shifts line numbers produces a new Issue
|
|
40
40
|
and the curated one is orphaned.
|
|
41
41
|
- **Any save sets `isManaged`**, which does double duty: it protects the curated text from being
|
|
@@ -59,8 +59,21 @@ from `Core.Databases` id **11** (`worker2` `CORE_LOGS_DATABASE_ID`); `Logs` is o
|
|
|
59
59
|
`_underscore` *register alias*. Read the real name from `Core.Databases` before configuring a new
|
|
60
60
|
environment, or the console silently points at a database that may not exist.
|
|
61
61
|
|
|
62
|
+
## Table names are SINGULAR
|
|
63
|
+
|
|
64
|
+
The Core Logs tables this console queries are `Issue`, `Event`, `IssueFingerprint`,
|
|
65
|
+
`IssueClickupTask`, `IssueEmailAddress`, `IssueAreaOwner` — **singular**, unlike every other
|
|
66
|
+
table this app touches. That is the deliberate `Logs`-cluster convention (see the
|
|
67
|
+
[2.0 error-reporting doc](../../../2.0/apps/_underscore/features/error-reporting-issue-event.md)).
|
|
68
|
+
They were renamed from plural on 2026-08-01, after the pipeline was already in production.
|
|
69
|
+
|
|
62
70
|
## Gotchas / known issues
|
|
63
71
|
|
|
72
|
+
- **A 2.0 table rename breaks this page silently until it is run.** This console has **no model
|
|
73
|
+
layer** — `get.php` and `post.php` interpolate table names into hand-written `App_Database`
|
|
74
|
+
SQL (40+ literal occurrences between them). Nothing here fails at deploy time; it fails when
|
|
75
|
+
a curator opens the page. Any rename in the shared Core Logs DB must be applied to both files
|
|
76
|
+
in the same release as the migration.
|
|
64
77
|
- **This is a 1.0 app reading a 2.0 shared database.** Access is plain `App_Database` via the
|
|
65
78
|
`db_toga2logs` alias, not the `_Model` layer — so none of the 2.0 model protections apply.
|
|
66
79
|
Escape every interpolated value and allowlist every identifier (see the 1.0 back-end standard).
|
|
@@ -72,6 +85,11 @@ environment, or the console silently points at a database that may not exist.
|
|
|
72
85
|
|
|
73
86
|
## Change history
|
|
74
87
|
|
|
88
|
+
- 2026-08-01 — Updated every hardcoded table name in `tools/mvc/errors/get.php` (25+ refs) and
|
|
89
|
+
`post.php` (15+ refs) for the Core Logs singular rename (`Issues`→`Issue`, `Events`→`Event`,
|
|
90
|
+
`IssueFingerprints`→`IssueFingerprint`, `IssueClickupTasks`→`IssueClickupTask`,
|
|
91
|
+
`IssueEmailAddresses`→`IssueEmailAddress`, `IssueAreaOwners`→`IssueAreaOwner`). Recorded that
|
|
92
|
+
this 1.0 console has no model layer to absorb a 2.0 shared-schema rename. (jcardinal)
|
|
75
93
|
- 2026-07-30 — Built as part of TRUE-78188: new `/errors` triage/curation console over the
|
|
76
94
|
shared Core Logs DB via a new `db_toga2logs` alias; curation sets `isManaged` (protects text
|
|
77
95
|
and exempts from GC); fingerprint merging; per-client recipient management with an explicit
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Legacy Email Notifier (tools /email-migration/notify)
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: tools
|
|
5
|
+
project: Tools
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-08-03
|
|
10
|
+
owners: [bala]
|
|
11
|
+
files:
|
|
12
|
+
- tools/mvc/email-migration/notify/get.php
|
|
13
|
+
- tools/mvc/email-migration/notify/post.php
|
|
14
|
+
- tools/_/app/nav.php
|
|
15
|
+
related:
|
|
16
|
+
- ./mvc-data-access-patterns.md
|
|
17
|
+
- ../workflows/deploy-to-elastic-beanstalk-al2023.md
|
|
18
|
+
- ../architecture.md
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Summary
|
|
22
|
+
|
|
23
|
+
An SSO-gated admin tool at **`/email-migration/notify`** (nav group **Email Migration** >
|
|
24
|
+
**Legacy Notifier**, personas `['TOGa Technology','Development Team']`) that drives the
|
|
25
|
+
company-wide retirement of the legacy email domains. An admin uploads the weekly Microsoft 365
|
|
26
|
+
message-trace `.xlsx` exports; the tool emails each current employee a digest of messages still
|
|
27
|
+
being sent to their **old** addresses last week, asking senders to switch to `@togatech.com`.
|
|
28
|
+
|
|
29
|
+
Legacy domains it targets: `@goagilant.com`, `@asisystem.com`, `@agilantsolutions.com`.
|
|
30
|
+
The one client DB it reads is `Client_True.Users` (the internal Agilant/True org tenant) via the
|
|
31
|
+
`db_true` connection, **read-only**.
|
|
32
|
+
|
|
33
|
+
## How it works
|
|
34
|
+
|
|
35
|
+
1. **Upload + merge.** Accepts **multiple** `.xlsx` files (up to 10) and merges all rows into one
|
|
36
|
+
set. Parsing uses the library-vendored **PhpSpreadsheet** (see the deprecation gotcha below).
|
|
37
|
+
2. **Legacy filter.** Keeps only rows whose `RecipientAddress` ends in a legacy domain; everything
|
|
38
|
+
else is dropped.
|
|
39
|
+
3. **Recipient resolution (product decision — nothing is ever skipped).** A local-part map is
|
|
40
|
+
loaded once from `Client_True` over `db_true`:
|
|
41
|
+
|
|
42
|
+
```sql
|
|
43
|
+
SELECT
|
|
44
|
+
LOWER(SUBSTRING_INDEX(email, '@', 1)) AS lp,
|
|
45
|
+
email,
|
|
46
|
+
displayName
|
|
47
|
+
FROM Users
|
|
48
|
+
WHERE email LIKE '%@togatech.com'
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
`isActive` is deliberately **not** filtered — local-parts are unique. Each legacy recipient's
|
|
52
|
+
local-part is matched to that map (**user-match**). If it is **not** found, the tool **always**
|
|
53
|
+
domain-swaps to `localPart@togatech.com` — no row is skipped, by explicit product decision.
|
|
54
|
+
4. **Digest grouping.** Groups into **one digest per person per legacy domain** (a person who
|
|
55
|
+
received legacy mail on two old domains gets two digests).
|
|
56
|
+
5. **Chunked browser-driven send.** An AJAX loop sends **25 recipients per batch** with a progress
|
|
57
|
+
bar, per-step vertical connectors, and a **Cancel** that clears files/preview/in-progress send.
|
|
58
|
+
|
|
59
|
+
### Data facts (verified in prod `Client_True.Users`)
|
|
60
|
+
|
|
61
|
+
- `@togatech.com` is the migrated domain: **1960** users, **1039** active.
|
|
62
|
+
- Functional mailboxes / distribution lists (e.g. `serviceops`, `ariba`, `rumcsupport`) have **no**
|
|
63
|
+
user record, so they resolve **only** via domain-swap. That swapped address may hard-bounce if no
|
|
64
|
+
such `@togatech.com` mailbox exists — an **accepted tradeoff** of the never-skip rule.
|
|
65
|
+
|
|
66
|
+
## Gotcha — cross-instance job cache must live in S3, not local disk
|
|
67
|
+
|
|
68
|
+
Prod `tools` runs **multiple Elastic Beanstalk instances behind a load balancer**. The parsed job
|
|
69
|
+
is written at Preview and read at Send; if it is cached to local disk, Preview lands on instance A
|
|
70
|
+
and the Send AJAX is load-balanced to instance B, which returns **"That job has expired."**
|
|
71
|
+
|
|
72
|
+
`/var/www/cache` (created per-instance and `chmod 777` by
|
|
73
|
+
`ebs/setup_export_cache_folders.php`) is **local disk, not an s3fs/shared mount**, so it cannot
|
|
74
|
+
hold cross-request state. Fix: store the parsed job JSON in **S3** (reusing
|
|
75
|
+
`App_Talos_S3::client()`; key prefix `email-migration-jobs/<jobId>.json`), read it in the
|
|
76
|
+
send-batch step, and delete it on completion. Local dev has no vendored AWS SDK
|
|
77
|
+
(`tools/vendor` absent) and is single-instance, so it **falls back to a local file**; **prod
|
|
78
|
+
refuses the local fallback and fails loudly** ("Cross-instance job storage is unavailable").
|
|
79
|
+
Sessions work across instances only because they use a shared store — that is why login worked but
|
|
80
|
+
the local file cache did not. (See the deploy workflow for the multi-instance / local-cache facts.)
|
|
81
|
+
|
|
82
|
+
## Gotcha — `App_Email::send()` is unusable in the tools runtime; send via SES SMTP directly
|
|
83
|
+
|
|
84
|
+
Every send failed (`sent:0`, all "Send failed") because `App_Email::send()` also does a `Defaults`
|
|
85
|
+
lookup (Vision / `db_main`), a `Logs.Emails` write (`db_logs`), and an `EmailSuppressionList`
|
|
86
|
+
check (Common / `db_common`) — that 1.0 email-stack DB wiring is **not** present in the tools app,
|
|
87
|
+
so it threw for every recipient.
|
|
88
|
+
|
|
89
|
+
Fix: send with **PHPMailer directly** using `App_Email`'s SES SMTP **constants**
|
|
90
|
+
(`SMTP_HOSTNAME` / `SMTP_PORT` / `SMTP_SECURE` / `SMTP_USERNAME` / `SMTP_PASSWORD` — the same
|
|
91
|
+
proven SES SMTP creds, `us-west-2`) and **no DB calls**; From `no-reply@togatech.com` / "TOGa IT".
|
|
92
|
+
**Operational prerequisite:** the `togatech.com` sender identity/domain must be **verified in that
|
|
93
|
+
us-west-2 SES account** or delivery is rejected. Any future tools-app feature that needs to send
|
|
94
|
+
email hits the same wall — bypass `App_Email::send()` and go straight to SES.
|
|
95
|
+
|
|
96
|
+
## Gotcha — PhpSpreadsheet 1.6.0 deprecations abort the request
|
|
97
|
+
|
|
98
|
+
The vendored PhpSpreadsheet is **1.6.0** (library composer constraint `^1.6`, which supports only
|
|
99
|
+
PHP 5.6/7.0) and emits `E_DEPRECATED` all over on PHP 8.x. A single deprecation during the xlsx
|
|
100
|
+
parse aborts the request, because the 1.0 error handler promotes any PHP notice/deprecation to a
|
|
101
|
+
thrown exception **and** (when Sentry is absent) echoes "Sentry is not installed…" into the
|
|
102
|
+
response, corrupting the JSON. Fix: install a **scoped `set_error_handler`** at the top of the
|
|
103
|
+
endpoint that swallows non-fatal diagnostics (returns `true`) for the duration of the action; real
|
|
104
|
+
fatals and thrown exceptions still reach the `catch`. The framework mechanism is documented in
|
|
105
|
+
[mvc-data-access-patterns](./mvc-data-access-patterns.md).
|
|
106
|
+
|
|
107
|
+
## Change history
|
|
108
|
+
- 2026-08-03 — Built the Legacy Email Notifier: multi-file xlsx merge via vendored PhpSpreadsheet,
|
|
109
|
+
legacy-domain filter, `Client_True.Users` local-part resolution with always-domain-swap
|
|
110
|
+
(never-skip), one digest per person per legacy domain, and chunked (25/batch) browser-driven
|
|
111
|
+
send. Recorded the S3 cross-instance job cache, SES-direct send (App_Email unusable in tools),
|
|
112
|
+
and the PhpSpreadsheet-deprecation scoped error-handler fixes (bala)
|
|
@@ -6,11 +6,12 @@ project: Tools
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [jcardinal]
|
|
9
|
+
updated: 2026-08-03
|
|
10
|
+
owners: [jcardinal, bala]
|
|
11
11
|
files:
|
|
12
12
|
- tools/_/app/nav.php
|
|
13
13
|
- tools/mvc/get.php
|
|
14
|
+
- library/app/error.php
|
|
14
15
|
related:
|
|
15
16
|
- ./persona-gated-navigation.md
|
|
16
17
|
- ./saml-sso-auth.md
|
|
@@ -86,10 +87,32 @@ if (!headers_sent()) { http_response_code(401); } // matches App_MVC::routeTo'
|
|
|
86
87
|
path hit this too — see [saml-sso-auth](./saml-sso-auth.md). The clean long-term fix is to handle
|
|
87
88
|
status-setting routes **before any output** and `exit`.
|
|
88
89
|
|
|
90
|
+
## Gotcha — the error handler promotes ANY PHP notice/deprecation to a thrown exception (and corrupts JSON)
|
|
91
|
+
`App_Error::handleError` (`library/app/error.php`) **ignores the `error_reporting` mask** and
|
|
92
|
+
promotes **any** PHP notice/warning/deprecation to a thrown `ErrorException`. When Sentry is not
|
|
93
|
+
installed it **also echoes** "Sentry is not installed…" into the response — which **corrupts a JSON
|
|
94
|
+
response body**. So a single `E_DEPRECATED` (e.g. an old vendored package like PhpSpreadsheet 1.6.0
|
|
95
|
+
running on PHP 8.x) aborts the whole request and, on an AJAX endpoint, breaks the JSON the browser
|
|
96
|
+
is parsing. This is the same escalation behind the preloader-flush 500 above.
|
|
97
|
+
|
|
98
|
+
Fix for a page/endpoint that legitimately triggers non-fatal diagnostics: install a **scoped**
|
|
99
|
+
`set_error_handler` at the top of the action that **returns `true`** to swallow non-fatal notices
|
|
100
|
+
for the duration of the action; real fatals and thrown exceptions still reach your `catch`. Scope
|
|
101
|
+
it tightly — do not disable diagnostics app-wide.
|
|
102
|
+
|
|
103
|
+
```php
|
|
104
|
+
set_error_handler(fn(): bool => true); // swallow non-fatal notices/deprecations for this action only
|
|
105
|
+
```
|
|
106
|
+
|
|
89
107
|
## Autoloader
|
|
90
108
|
`App_Foo_Bar` → `app/foo/bar.php`, **all lowercase** (per the library CLAUDE.md). E.g.
|
|
91
109
|
`App_Talos_Estimator` → `_/app/talos/estimator.php`.
|
|
92
110
|
|
|
93
111
|
## Change history
|
|
112
|
+
- 2026-08-03 — Added the error-handler escalation gotcha: `App_Error::handleError` promotes ANY PHP
|
|
113
|
+
notice/deprecation to a thrown `ErrorException` (ignoring `error_reporting`) and, without Sentry,
|
|
114
|
+
echoes "Sentry is not installed…" into the response — corrupting JSON on AJAX endpoints; workaround
|
|
115
|
+
is a scoped `set_error_handler` returning true. Surfaced building the Legacy Email Notifier (xlsx
|
|
116
|
+
parse via PhpSpreadsheet 1.6.0 on PHP 8.x). (bala)
|
|
94
117
|
- 2026-06-30 — Added the preloader-flush gotcha: pages run after `App_Page::flushCapture()` flushes the output buffer, so `http_response_code`/`header`/`session_regenerate_id` from inside a page fatal on headers-already-sent (escalates to a 500) — guard with `!headers_sent()`. Surfaced by the SSO failure-path 500 fix. (jcardinal)
|
|
95
118
|
- 2026-06-29 — Documented tools MVC routing (GET→get.php / POST→post.php; writes via mvc/<route>/post.php, not the legacy actionHandler path), Origin/Referer CSRF (no token field), `App_Database` query/row/txn API + `sqlEscape()` name, the by-reference row-helper warning gotcha, the persona-narrowing gotcha, and the lowercase autoloader mapping. Discovered building the Talos Pricing UI. (jcardinal)
|
|
@@ -6,8 +6,8 @@ project: Tools
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [jcardinal]
|
|
9
|
+
updated: 2026-08-03
|
|
10
|
+
owners: [jcardinal, bala]
|
|
11
11
|
files:
|
|
12
12
|
- tools/.ebextensions/004_http_to_https.config
|
|
13
13
|
- tools/.ebextensions/006_mount-s3fs.config
|
|
@@ -17,6 +17,8 @@ files:
|
|
|
17
17
|
- tools/.ebextensions/020_setup_git_libraries.config
|
|
18
18
|
- tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config
|
|
19
19
|
- tools/ebs/git.json
|
|
20
|
+
- tools/ebs/setup_export_cache_folders.php
|
|
21
|
+
- tools/_/app/frameworkindex.php
|
|
20
22
|
related:
|
|
21
23
|
- ../architecture.md
|
|
22
24
|
---
|
|
@@ -100,6 +102,64 @@ roles, so `AWS_SECRET_KEY` / `AWS_ACCESS_KEY_ID` are not injected into `$_SERVER
|
|
|
100
102
|
s3fs credential write fails cleanly. Use `ignoreErrors` only on genuinely non-fatal infra
|
|
101
103
|
setup — the git clone (`020`) deliberately does **not** use it (see gotchas).
|
|
102
104
|
|
|
105
|
+
## Prod is multi-instance — never store cross-request state on local disk
|
|
106
|
+
|
|
107
|
+
Prod `tools` runs **multiple EB instances behind a load balancer** (see
|
|
108
|
+
`050_register_instance_to_shared_application_load_balancer.config`). Any state written on one
|
|
109
|
+
request may be read on another instance on the next request. Consequences:
|
|
110
|
+
|
|
111
|
+
- **`/var/www/cache` is per-instance LOCAL disk, not shared.** It is created and `chmod 777`ed
|
|
112
|
+
per instance by `ebs/setup_export_cache_folders.php`; it is **not** an s3fs/shared mount, so it
|
|
113
|
+
cannot hold state that a later request must read. Symptom seen: a Preview→Send flow that cached
|
|
114
|
+
the parsed job to `/var/www/cache` failed with "That job has expired" because Preview hit
|
|
115
|
+
instance A and the Send AJAX was load-balanced to instance B.
|
|
116
|
+
- **Cross-instance state must go to S3** (e.g. via `App_Talos_S3::client()`), or another shared
|
|
117
|
+
store. **Sessions already survive across instances** because they use a shared store (redis) —
|
|
118
|
+
that is why login works while a local file cache does not.
|
|
119
|
+
- Fail **loudly** in prod if the shared store is unavailable rather than silently falling back to
|
|
120
|
+
local disk (a local fallback is fine for single-instance local dev only).
|
|
121
|
+
|
|
122
|
+
## No composer install step on deploy
|
|
123
|
+
|
|
124
|
+
The deploy clones `agilantsolutions/library` and `resources` via
|
|
125
|
+
`020_setup_git_libraries.config` but has **no explicit `composer install` step**. `tools/vendor`
|
|
126
|
+
(AWS SDK, Sentry) therefore exists **only if** the EB PHP platform auto-runs composer. Do not
|
|
127
|
+
assume a composer package is present at runtime — any code path needing one must `require_once` the
|
|
128
|
+
autoloader itself and degrade gracefully when `tools/vendor` is absent (this is why S3-backed
|
|
129
|
+
cross-instance storage has a local-disk fallback path for dev, where the AWS SDK is not vendored).
|
|
130
|
+
|
|
131
|
+
## Config selection & DB wiring
|
|
132
|
+
|
|
133
|
+
- **Which config file loads is chosen by the `ENVIRONMENT` env var, NOT the hostname.**
|
|
134
|
+
`App_Config::getConfigFilename` resolves `config.<ENVIRONMENT>.ini`.
|
|
135
|
+
- Config maps DB key **`db_<x>` → `[database_<x>]`** section; `App_Registry::get()` connects
|
|
136
|
+
**lazily** on first use. `Client_True` is **`db_true`** (`reader1.client.database.togahub.com`).
|
|
137
|
+
- Prod php.ini caps `upload_max_filesize=25M` / `post_max_size=30M` (`009_setup_phpini.config`) —
|
|
138
|
+
multi-file uploads (e.g. the Legacy Email Notifier's xlsx trace exports) must stay under this.
|
|
139
|
+
|
|
140
|
+
## Local development
|
|
141
|
+
|
|
142
|
+
- **Run locally with the `ENVIRONMENT` var pointing at your local config:**
|
|
143
|
+
|
|
144
|
+
```sh
|
|
145
|
+
ENVIRONMENT=<name> php -d include_path=".;C:/WWW/library" -S localhost:8000 router.php
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
This selects `config.<name>.ini` and puts `library/` on the include path. Local dev is
|
|
149
|
+
single-instance and has **no vendored AWS SDK** (`tools/vendor` absent), so S3-backed features
|
|
150
|
+
fall back to local disk here.
|
|
151
|
+
- **The Vision copyright watchdog in the shared `resources/library.js` redirects local dev.**
|
|
152
|
+
`App_FrameworkIndex::render()` loads `resources/library.js`, whose obfuscated block
|
|
153
|
+
(`resources/library.js:848` → `jc()`/`jcr()`) pings the hardcoded **production** endpoint
|
|
154
|
+
`https://vision.asisystem.com/vision/cr.php` ~12–35s after load. `cr.php` checks the browsing
|
|
155
|
+
host against the `CopyrightHosts` table (`db_log`) and, for a host that is neither listed nor a
|
|
156
|
+
`*.agilantsolutions.com` origin (e.g. `localhost`), redirects to `vision/unauthorized.html`
|
|
157
|
+
("Unauthorized Access!"). This breaks local dev of **any** tools/1.0 page. Workarounds: browse
|
|
158
|
+
under a `*.agilantsolutions.com` hosts alias (cr.php short-circuits those), **or** neutralize
|
|
159
|
+
`window.jc`/`window.jcr`. A **dev-only** skip was added to `App_FrameworkIndex_Tools::render()`
|
|
160
|
+
(`tools/_/app/frameworkindex.php`), gated on `App_Registry::inDevMode()` (a no-op in prod), that
|
|
161
|
+
overwrites those globals so local dev is not redirected.
|
|
162
|
+
|
|
103
163
|
## Gotchas (durable AL2023 / EB rules)
|
|
104
164
|
|
|
105
165
|
- **php-fpm on AL2023 defaults to `clear_env=yes`, so `getenv()` cannot see EB environment
|
|
@@ -136,6 +196,12 @@ setup — the git clone (`020`) deliberately does **not** use it (see gotchas).
|
|
|
136
196
|
returns `200` fast. Reusable for any TOGA app behind Elastic Beanstalk.
|
|
137
197
|
|
|
138
198
|
## Change history
|
|
199
|
+
- 2026-08-03 — Recorded that prod Tools is multi-instance behind an ALB, so `/var/www/cache` is
|
|
200
|
+
per-instance local disk (not shared) and cross-request state must go to S3 — the root cause of a
|
|
201
|
+
Preview→Send "job expired"; noted the no-composer-install deploy, `ENVIRONMENT`-based config
|
|
202
|
+
selection + `db_<x>`→`[database_<x>]`/`db_true` wiring, and added a Local development section (run
|
|
203
|
+
command + the Vision `cr.php` copyright-watchdog redirect and its dev-only frameworkindex skip).
|
|
204
|
+
Discovered building the Legacy Email Notifier (bala)
|
|
139
205
|
- 2026-07-24 — Added the php-fpm `clear_env=yes` gotcha (read EB env properties from
|
|
140
206
|
`getenv()`/`$_SERVER`/`$_ENV`) and raised `application.ini` upload/post limits to 25M/30M for
|
|
141
207
|
multi-MB design-demo exports (jcardinal)
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
| [Re-pointing a DB alias mid-request (_Database::register park/restore)](features/database-alias-repointing.md) | `_Database` keys **all live per-database runtime state by the connection ALIAS** (`Client` / `_underscore::DB_CLIENT`, `ClientLogs`, `Archive`), **not** by the | _underscore/Database.php, _underscore/Query.php, api2/Component/Api/V2/V2.php, api2/Component/Api/CrossClient/CrossClient.php |
|
|
17
17
|
| [2.0 Email Send Pipeline (queue + Send worker)](features/email-send-pipeline.md) | In 2.0, `_Email::send()` **does not transmit** — it queues the message. | _underscore/Email.php, worker2/Worker/Infrastructure/Email/Send.php |
|
|
18
18
|
| [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
|
|
19
|
-
| [Error Reporting — Issue/Event Capture, Fingerprinting & Aggregation](features/error-reporting-issue-event.md) | Platform-wide error reporting for TOGA 2.0, built on an **Issue / Event** aggregation model in the **shared Core Logs DB**. | _underscore/Error.php, _underscore/Exception/Business.php, _underscore/Model/Core/Logs/Issue.php, _underscore/Model/Core/Logs/Event.php, _underscore/Model/Core/Logs/IssueFingerprint.php, _underscore/Model/Core/Logs/IssueClickupTask.php, _underscore/Model/Core/Logs/IssueEmailAddress.php, _underscore/Model/Core/Logs/IssueAreaOwner.php, dbchanges2/Logs/2026-07-30a - Error reporting Issues and Events.sql, dbchanges2/Core/2026-07-30a - Error escalation cron job.sql |
|
|
19
|
+
| [Error Reporting — Issue/Event Capture, Fingerprinting & Aggregation](features/error-reporting-issue-event.md) | Platform-wide error reporting for TOGA 2.0, built on an **Issue / Event** aggregation model in the **shared Core Logs DB**. | _underscore/Error.php, _underscore/Exception/Business.php, _underscore/Model/Core/Logs/Issue.php, _underscore/Model/Core/Logs/Event.php, _underscore/Model/Core/Logs/IssueFingerprint.php, _underscore/Model/Core/Logs/IssueClickupTask.php, _underscore/Model/Core/Logs/IssueEmailAddress.php, _underscore/Model/Core/Logs/IssueAreaOwner.php, dbchanges2/Logs/2026-07-30a - Error reporting Issues and Events.sql, dbchanges2/Logs/2026-08-01a - Rename tables to singular.sql, dbchanges2/Logs_Client/2026-08-01a - Drop Error table.sql, dbchanges2/Core/2026-07-30a - Error escalation cron job.sql |
|
|
20
20
|
| [Record-Changed Event Publishing (_Event::publish to SQS)](features/event-publish-sqs.md) | `_Event::publish()` (in `_underscore/Event.php`) is the PHP side of the real-time event pipeline. | _underscore/Event.php |
|
|
21
21
|
| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | worker2/Component/Forecast/SaleImport/SaleImport.php, worker2/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/verify_je_classification.php, test/@dave/probe_je_accounts.php, test/@dave/probe_je_shape.php, test/@dave/fixer.php, test/@dave/Junk Drawer/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/Junk Drawer/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
|
|
22
22
|
| [isFulfillable Propagation Up the SO↔PO Chain](features/fulfillable-item-propagation.md) | `Items.isFulfillable` is a boolean that gates whether a storefront line's **Qty Fulfilled** cell is actionable. | _underscore/Model/Client/Item.php, _underscore/Model/Compass/Item.php, dbchanges2/Core/2026-07-17 - Items isFulfillable RecordField.sql, dbchanges2/Core/2026-07-17 - RegisterItemIsFulfillableInterceptors.sql, dbchanges2/Client/2026-07-17 - ItemsisFulfillable.sql |
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-08-01
|
|
10
10
|
owners: ["dfranks", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Error.php
|
|
@@ -18,6 +18,8 @@ files:
|
|
|
18
18
|
- _underscore/Model/Core/Logs/IssueEmailAddress.php
|
|
19
19
|
- _underscore/Model/Core/Logs/IssueAreaOwner.php
|
|
20
20
|
- dbchanges2/Logs/2026-07-30a - Error reporting Issues and Events.sql
|
|
21
|
+
- dbchanges2/Logs/2026-08-01a - Rename tables to singular.sql
|
|
22
|
+
- dbchanges2/Logs_Client/2026-08-01a - Drop Error table.sql
|
|
21
23
|
- dbchanges2/Core/2026-07-30a - Error escalation cron job.sql
|
|
22
24
|
related:
|
|
23
25
|
- ../architecture.md
|
|
@@ -45,9 +47,23 @@ re-implementing persistence.
|
|
|
45
47
|
|
|
46
48
|
## Data model (shared Core Logs DB)
|
|
47
49
|
|
|
48
|
-
Six tables, created by `dbchanges2/Logs/2026-07-30a - Error reporting Issues and Events.sql
|
|
49
|
-
|
|
50
|
-
`
|
|
50
|
+
Six tables, created by `dbchanges2/Logs/2026-07-30a - Error reporting Issues and Events.sql` and
|
|
51
|
+
renamed to their final **singular** names by
|
|
52
|
+
`dbchanges2/Logs/2026-08-01a - Rename tables to singular.sql`:
|
|
53
|
+
`Issue`, `IssueFingerprint`, `Event`, `IssueEmailAddress`, `IssueClickupTask`, `IssueAreaOwner`.
|
|
54
|
+
|
|
55
|
+
> **⚠ The `Logs` cluster is the one place that uses SINGULAR table names.** Everywhere else on
|
|
56
|
+
> the platform tables are plural (`Orders`, `Customers`) — see the 2.0 back-end standard. In
|
|
57
|
+
> `Logs` the tables are named for the *thing* (`Issue`, `Event`), matching how they are referred
|
|
58
|
+
> to verbally ("the Issue table", "API Logs", not "APIs Logs"). **When you add a new table to the
|
|
59
|
+
> `Logs` cluster, name it singular.** Do not "fix" these to plural, and do not carry the singular
|
|
60
|
+
> form outside `Logs`.
|
|
61
|
+
|
|
62
|
+
Renamed 2026-08-01 (`Issues`→`Issue`, `Events`→`Event`, `IssueFingerprints`→`IssueFingerprint`,
|
|
63
|
+
`IssueClickupTasks`→`IssueClickupTask`, `IssueEmailAddresses`→`IssueEmailAddress`,
|
|
64
|
+
`IssueAreaOwners`→`IssueAreaOwner`). The migration renames the tables *and* their foreign keys
|
|
65
|
+
and indexes; the `const TABLE` on every `_Model_Core_Logs_*` model was updated to match. Older
|
|
66
|
+
notes below and in related docs may still quote the plural names — the shape is unchanged.
|
|
51
67
|
|
|
52
68
|
**An Issue is never client-scoped.** An Issue is identified by *code*, and code is global — the
|
|
53
69
|
same bug hitting six clients is **one** Issue. Per-client scoping lives on the occurrence
|
|
@@ -183,6 +199,18 @@ Sentry-removal ticket cannot complete until the 1.0 port does.
|
|
|
183
199
|
`varchar(255)` (`errorMessage`, `subject`) **throws** on `->save()`. Inside the handler's
|
|
184
200
|
`catch(Throwable)` the throw is swallowed and the row is silently dropped — precisely when a
|
|
185
201
|
long message matters most. Truncate to column width before `save()`.
|
|
202
|
+
- **`const TABLE` must track a table rename, and nothing warns you.** A `_Model` subclass whose
|
|
203
|
+
`const TABLE` points at a renamed table fails only at query time. When a `Logs` table is
|
|
204
|
+
renamed, the matching `_underscore/Model/Core/Logs/*.php` `const TABLE` must change in the
|
|
205
|
+
**same** deploy as the migration.
|
|
206
|
+
- **1.0 apps break on a 2.0 table rename.** Tools' `/errors` console reads the shared Core Logs
|
|
207
|
+
DB with hand-written `App_Database` SQL, so it has no model layer to absorb a rename — every
|
|
208
|
+
literal table name in `tools/mvc/errors/get.php` and `post.php` had to be edited by hand.
|
|
209
|
+
Grep the 1.0 side before renaming anything in a **shared** 2.0 database.
|
|
210
|
+
- **The per-client `Logs_Client.Error` table is dead.** Superseded by the Issue/Event pipeline;
|
|
211
|
+
the code that wrote it is commented out in `_Error.php`, the `Model/Core/Logs/Error.php` and
|
|
212
|
+
`Model/Client/Logs/Error.php` models were deleted, and the table is dropped by
|
|
213
|
+
`dbchanges2/Logs_Client/2026-08-01a - Drop Error table.sql`. Do not reintroduce it.
|
|
186
214
|
- **Shared-schema migration collision.** Because these tables live in the *shared* Core Logs
|
|
187
215
|
DB, only **one** migration may `CREATE` them; later migrations only `ALTER`.
|
|
188
216
|
- **Pre-existing committed secrets in this area (not fixed — out of scope, report only).**
|
|
@@ -239,6 +267,16 @@ clientUserId). **Neither was built.** As built instead:
|
|
|
239
267
|
|
|
240
268
|
## Change history
|
|
241
269
|
|
|
270
|
+
- 2026-08-01 — **Decided: `Logs` cluster tables use SINGULAR names.** Renamed all six
|
|
271
|
+
post-deployment (`Issues`→`Issue`, `Events`→`Event`, `IssueFingerprints`→`IssueFingerprint`,
|
|
272
|
+
`IssueClickupTasks`→`IssueClickupTask`, `IssueEmailAddresses`→`IssueEmailAddress`,
|
|
273
|
+
`IssueAreaOwners`→`IssueAreaOwner`) via
|
|
274
|
+
`dbchanges2/Logs/2026-08-01a - Rename tables to singular.sql` (tables + FKs + indexes), and
|
|
275
|
+
updated `const TABLE` on every `_Model_Core_Logs_*` model. This is a naming-convention
|
|
276
|
+
alignment, not a behavior change: the `Logs` cluster is named for the thing ("API Logs", not
|
|
277
|
+
"APIs Logs") while the rest of the platform stays plural. Also deleted the obsolete
|
|
278
|
+
`Model/Core/Logs/Error.php` and `Model/Client/Logs/Error.php` and dropped the per-client
|
|
279
|
+
`Logs_Client.Error` table. (jcardinal)
|
|
242
280
|
- 2026-07-30 — Reworked TRUE-78188: six-table schema in the shared Core Logs DB (Issue is
|
|
243
281
|
global by code; client scope lives on Events/IssueEmailAddresses); `_Error::captureException()`
|
|
244
282
|
as the single capture entry point; trace-frame fingerprinting with the message excluded
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-08-01
|
|
10
10
|
owners: ["jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Infrastructure/Errors.php
|
|
@@ -71,7 +71,7 @@ range contains today.
|
|
|
71
71
|
|
|
72
72
|
### Recurrence (episodes)
|
|
73
73
|
|
|
74
|
-
One `
|
|
74
|
+
One `IssueClickupTask` row per ClickUp **episode**. Closing the task fires a webhook that stamps
|
|
75
75
|
`dtResolved` and increments `recurrenceCount`. A later occurrence opens a **new** task linked to
|
|
76
76
|
its predecessors via `POST /task/{id}/link/{links_to}` — a **link, not a dependency**; a
|
|
77
77
|
dependency would *block* the task. A cooldown prevents recreating a task 60 seconds after
|
|
@@ -85,7 +85,7 @@ which is safe only because the automation never sets status itself.
|
|
|
85
85
|
|
|
86
86
|
### Area-level owner learning (observation only)
|
|
87
87
|
|
|
88
|
-
`
|
|
88
|
+
`IssueAreaOwner` learns an owner after **5 in a row** and ships **observation-only** — a hint in
|
|
89
89
|
the task body, never an auto-assignment. Misassignment is expensive: a few wrong tickets and
|
|
90
90
|
developers stop trusting the queue. Learning is per **area**, not per issue, because one issue
|
|
91
91
|
rarely recurs enough to learn from before it is fixed.
|
|
@@ -119,7 +119,7 @@ recipients away** after a quiet month.
|
|
|
119
119
|
|
|
120
120
|
- **The working-set query must be a UNION, not an OR.** It is a UNION of three separately
|
|
121
121
|
indexable branches. MySQL will **not** `index_merge` across an `OR` when one branch is a
|
|
122
|
-
correlated `EXISTS`, so the OR form **full-scans `
|
|
122
|
+
correlated `EXISTS`, so the OR form **full-scans `Issue` every 60 seconds, forever**.
|
|
123
123
|
- **`INSERT ... SELECT ... WHERE NOT EXISTS` is not a concurrency guard** under REPEATABLE READ.
|
|
124
124
|
The open-episode guard is a UNIQUE key on a VIRTUAL generated column, and the code **claims
|
|
125
125
|
the episode before calling ClickUp** so a losing run fails before creating a duplicate
|
|
@@ -131,7 +131,7 @@ recipients away** after a quiet month.
|
|
|
131
131
|
- **Business email bodies/subjects must not carry `errorMessage` or `trace`.** Both are seeded
|
|
132
132
|
once from whichever client created the Issue and are never re-scoped. An uncurated business
|
|
133
133
|
issue gets a **neutral** subject rather than another client's raw error text.
|
|
134
|
-
- **`
|
|
134
|
+
- **`IssueEmailAddress.clientId = 0` means "all clients"** and requires an explicit
|
|
135
135
|
confirmation flag in the Tools console; list views render it as **"ALL CLIENTS"**.
|
|
136
136
|
- **The ClickUp webhook has no HMAC signature verification** (pre-existing, unfixed). A forged
|
|
137
137
|
POST can mark issues acknowledged — freezing the neglect axis — or resolve episodes. That is
|
|
@@ -143,6 +143,12 @@ recipients away** after a quiet month.
|
|
|
143
143
|
|
|
144
144
|
## Change history
|
|
145
145
|
|
|
146
|
+
- 2026-08-01 — Table names updated for the Core Logs **singular** rename (`Issue`, `Event`,
|
|
147
|
+
`IssueFingerprint`, `IssueClickupTask`, `IssueEmailAddress`, `IssueAreaOwner`). Naming-only;
|
|
148
|
+
the cron's behavior is unchanged. See
|
|
149
|
+
[error-reporting-issue-event](../../_underscore/features/error-reporting-issue-event.md).
|
|
150
|
+
(jcardinal)
|
|
151
|
+
|
|
146
152
|
- 2026-07-30 — Built as part of TRUE-78188: action renamed `SyncWithClickup` → `Escalate`; new
|
|
147
153
|
`Worker/Clickup/ErrorTask.php`. Two-axis urgency (volume 20/50/100 + neglect 1/4/24) with a
|
|
148
154
|
`minimumUrgency` floor; fast-up/slow-down with release bands 12/35/70 over 3 windows; a
|
|
@@ -58,7 +58,7 @@ Lambda, the EB worker tier + SQS delivery (see [worker2 architecture](../archite
|
|
|
58
58
|
|
|
59
59
|
worker2 has **two** monitoring patterns and neither writes to the central **`Logs` Issues/Events**
|
|
60
60
|
tables. That is deliberate — **`Logs.Issues`/`Logs.Events` are strictly for application
|
|
61
|
-
errors/exceptions** that escalate to ClickUp or email (business-routed by `
|
|
61
|
+
errors/exceptions** that escalate to ClickUp or email (business-routed by `IssueEmailAddress`
|
|
62
62
|
presence; see the escalation-cron work). **Periodic health-check, integration-health, and
|
|
63
63
|
data-quality RESULTS do NOT go there** — they belong in the Monitor framework:
|
|
64
64
|
|
|
@@ -5,7 +5,7 @@ project: _Underscore
|
|
|
5
5
|
client: shared
|
|
6
6
|
type: standard
|
|
7
7
|
status: active
|
|
8
|
-
updated: 2026-
|
|
8
|
+
updated: 2026-08-01
|
|
9
9
|
owners: [jcardinal, mhammontree]
|
|
10
10
|
files: []
|
|
11
11
|
related:
|
|
@@ -292,6 +292,34 @@ WHERE
|
|
|
292
292
|
|
|
293
293
|
### **SQL Naming Conventions**
|
|
294
294
|
|
|
295
|
+
#### **Table Names — plural, except the `Logs` cluster**
|
|
296
|
+
|
|
297
|
+
* Table names are **plural**: `Orders`, `Customers`, `PurchaseOrderItems`. This is the default
|
|
298
|
+
and applies to every database on the platform with one exception.
|
|
299
|
+
* **Exception — the shared Core `Logs` database uses SINGULAR table names**: `Issue`, `Event`,
|
|
300
|
+
`IssueFingerprint`, `IssueClickupTask`, `IssueEmailAddress`, `IssueAreaOwner`. The `Logs`
|
|
301
|
+
cluster is named for the *thing being logged*, matching how it is referred to verbally —
|
|
302
|
+
"API Logs", not "APIs Logs"; "the Issue table", not "the Issues table".
|
|
303
|
+
* **When adding a table to `Logs`, name it singular.** Do not "fix" existing `Logs` tables to
|
|
304
|
+
plural, and do not carry the singular form into any other database.
|
|
305
|
+
* Foreign keys are unaffected: a key referencing `Issue` is still `issueId` (the FK rule already
|
|
306
|
+
singularizes).
|
|
307
|
+
|
|
308
|
+
```
|
|
309
|
+
# Good — non-Logs databases
|
|
310
|
+
Orders
|
|
311
|
+
Customers
|
|
312
|
+
Locations_LocationAttributes
|
|
313
|
+
|
|
314
|
+
# Good — the Logs database only
|
|
315
|
+
Issue
|
|
316
|
+
IssueFingerprint
|
|
317
|
+
|
|
318
|
+
# Bad
|
|
319
|
+
Logs.Issues # Logs tables are singular
|
|
320
|
+
Core.Order # non-Logs tables are plural
|
|
321
|
+
```
|
|
322
|
+
|
|
295
323
|
#### **Bridge Tables**
|
|
296
324
|
|
|
297
325
|
* Most bridge tables need to include an underscore between the two tables they are joining. If you are joining a parent with a child table, the parent should come first. Exceptions may be made for inherent child records like `SalesOrderItems` being a child yet also a bridge of `SalesOrders`
|
|
@@ -756,3 +784,7 @@ client sending a shallow depth.
|
|
|
756
784
|
### Code Documentation
|
|
757
785
|
|
|
758
786
|
* Ensure code is well-documented with comments and usage examples.
|
|
787
|
+
|
|
788
|
+
## Change history
|
|
789
|
+
|
|
790
|
+
- 2026-08-01 — Recorded the Logs-cluster singular table-naming exception to the otherwise-plural table convention. (jcardinal)
|
package/knowledge/INDEX.md
CHANGED
|
@@ -14,7 +14,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
14
14
|
- **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
|
|
15
15
|
- **test** (Test) — 13 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
|
|
16
16
|
- **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
|
|
17
|
-
- **tools** (Tools) —
|
|
17
|
+
- **tools** (Tools) — 13 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
|
|
18
18
|
|
|
19
19
|
## 2.0 framework
|
|
20
20
|
|
|
@@ -12,5 +12,5 @@
|
|
|
12
12
|
| [Compass USA](profile.md) | 2.0 | Compass USA is a TOGA client running a multi-tier supply-chain commerce operation. | |
|
|
13
13
|
| [Compass Cross-Kit Bundle Corruption — Detection & Repair](workflows/cross-kit-bundle-corruption.md) | 2.0 | A frontend regression in `toga2-commerce`'s edit-order bundle submission mis-attributed bundle (kit) line items and **fees/warranties** to the **wrong kit**, pe | src/api/syncSalesOrderItemsFromLocalStorageCartToApi.ts |
|
|
14
14
|
| [Compass Office Depot Duplicate PO-Line Cleanup (dual-catalog SKU)](workflows/odp-duplicate-po-line-cleanup.md) | 1.0 | The NetSuite fulfillment-sales-order importer duplicated Office Depot purchase-order lines on Compass USA because it reconciled the fulfillment SO against the e | library/app/api/toga2.php, dbchanges2/Client_Compass/2026-07-22a - CleanupOfficeDepotDuplicatePurchaseOrderItems.sql |
|
|
15
|
-
| [Compass ODP Order Pipeline to NetSuite (numbered worker crons)](workflows/odp-order-pipeline-to-netsuite.md) | 1.0 | The end-to-end **Compass Office Depot (ODP) order → NetSuite** pipeline as it actually runs through the 1.0 `worker` crons under `worker/crons/toga2/compass/`, | worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php, worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, worker/crons/toga2/compass/edi/1_download_edi_s3_create_po_toga.php, worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, library/app/client/compass.php |
|
|
15
|
+
| [Compass ODP Order Pipeline to NetSuite (numbered worker crons)](workflows/odp-order-pipeline-to-netsuite.md) | 1.0 | The end-to-end **Compass Office Depot (ODP) order → NetSuite** pipeline as it actually runs through the 1.0 `worker` crons under `worker/crons/toga2/compass/`, | worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php, worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, worker/crons/toga2/compass/edi/1_download_edi_s3_create_po_toga.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php, worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, library/app/client/compass.php |
|
|
16
16
|
| [Compass Order Lifecycle & Data-Integrity Invariants](workflows/order-lifecycle-and-data-integrity.md) | 2.0 | End-to-end map of how a Compass order flows through the `Client_Compass` (2.0) database and the **expected raw-data shape** at each link/ASN/IF level. | |
|
|
@@ -6,12 +6,13 @@ project: Worker
|
|
|
6
6
|
client: compass-usa
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["rgirish"]
|
|
9
|
+
updated: 2026-08-03
|
|
10
|
+
owners: ["rgirish", "bala"]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
|
|
13
13
|
- worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php
|
|
14
14
|
- worker/crons/toga2/compass/edi/1_download_edi_s3_create_po_toga.php
|
|
15
|
+
- worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php
|
|
15
16
|
- worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php
|
|
16
17
|
- library/app/client/compass.php
|
|
17
18
|
related:
|
|
@@ -69,6 +70,45 @@ ODP SalesOrder (customerId = 1)
|
|
|
69
70
|
- **computer-kit orders** (Bundles 187–196) wait for a **2nd PO** before syncing.
|
|
70
71
|
- a **30-minute delay** after the first ODP PO is created.
|
|
71
72
|
|
|
73
|
+
## Cron 5 NetSuite SO creation — failure modes & the two ODP error emails
|
|
74
|
+
|
|
75
|
+
Two different Compass ODP error emails exist, from **two different pipeline stages**. They look
|
|
76
|
+
alike but are distinct — check the subject to know which cron/stage failed:
|
|
77
|
+
|
|
78
|
+
- **`Office Depot PO Import error: <ODP PO number>`** — sent by cron **3a**
|
|
79
|
+
(`workflow/3a_import_office_depot_purchase_orders.php`, EDI 850 → PO import into Toga 2.0) via
|
|
80
|
+
its `sendErrorNotification()`. It writes the raw EDI x12 to a temp file and **attaches it**
|
|
81
|
+
(`App_Email_Agilant->addAttachment`).
|
|
82
|
+
- **`Compass Office Depot SO failed in NetSuite: <SO> / <CompassPO>`** — sent by cron **5**
|
|
83
|
+
(ODP PO → NetSuite SO creation). (Reworded 2026-08-03; was
|
|
84
|
+
`Office Depot Import error: <Compass SO> / <Compass PO>`, historically with no attachment.)
|
|
85
|
+
|
|
86
|
+
**Cron 5 has exactly two error sources**, both pushed onto `$errMessage[]`:
|
|
87
|
+
1. **`Item not found in NetSuite: <partNumber>`** — the SKU resolved to no NetSuite internal id.
|
|
88
|
+
2. **NetSuite `add()` rejection** — NetSuite refused the SalesOrder (see the Ext Price root cause
|
|
89
|
+
below, the common case).
|
|
90
|
+
|
|
91
|
+
### Root cause of the recurring "Please enter a value for Ext Price" rejection
|
|
92
|
+
In cron 5's item loop, `$soItem->rate` (NetSuite's "Ext Price") is set **only** when the SKU
|
|
93
|
+
resolves as a NetSuite **item group** via `App_NetSuite::getItemGroupInternalIdFromPartNumber()`.
|
|
94
|
+
For a **plain item** resolved via `App_NetSuite::getItemInternalIdFromPartNumber()`, no rate is
|
|
95
|
+
ever set, so NetSuite's `add()` rejects the whole order with USER_ERROR **"Please enter a value
|
|
96
|
+
for Ext Price."** The order then stays stuck: cron 5 only picks ODP SalesOrders with
|
|
97
|
+
`c_dtTransmittedToNetsuite IS NULL` and **re-attempts (and re-emails) every hour** until a success
|
|
98
|
+
stamps `c_dtTransmittedToNetsuite` + `c_netsuiteInternalSalesOrderId`. So a repeating hourly
|
|
99
|
+
"SO failed in NetSuite" email for the same order is this loop, not a new failure each time.
|
|
100
|
+
|
|
101
|
+
### Error-email rework (2026-08-03, cron 5)
|
|
102
|
+
- New helper **`buildNetsuiteErrorMessage(AddResponse $response)`** walks
|
|
103
|
+
`writeResponse->status->statusDetail[]` and returns `NetSuite rejected the order - <CODE>:
|
|
104
|
+
<message>` (multiple details joined by `; `) — replacing the old whole-`AddResponse` `print_r`
|
|
105
|
+
dump.
|
|
106
|
+
- The email body now appends the **no-rate item part numbers** (collected in `$itemsWithoutRate`
|
|
107
|
+
whenever `$soItem->rate` is unset) after the error line — these are the Ext Price culprits.
|
|
108
|
+
- An attachment **`NetSuiteError_<SO>.txt`** (written to `sys_get_temp_dir()`) carries the Compass
|
|
109
|
+
SO/PO, the Office Depot PO, and the full NetSuite request (SalesOrder) + response dump, attached
|
|
110
|
+
via `App_Email_Agilant->addAttachment()` (the same mechanism cron 3a uses).
|
|
111
|
+
|
|
72
112
|
## Constants & identities (`library/app/client/compass.php`)
|
|
73
113
|
- `App_Client_Compass::VENDOR_ID__OFFICE_DEPOT = 1` — `Vendors.id = 1` = "OFFICE DEPOT".
|
|
74
114
|
- `App_Client_Compass::CUSTOMER_ID__OFFICE_DEPOT = 1` — `Customers.id = 1` = "Office Depot".
|
|
@@ -101,6 +141,12 @@ wrong record makes every order look "stuck." Use the **join**, never a number ma
|
|
|
101
141
|
transmission feature.)
|
|
102
142
|
|
|
103
143
|
## Change history
|
|
144
|
+
- 2026-08-03 — Documented cron 5's "Please enter a value for Ext Price" root cause (`rate` set
|
|
145
|
+
only for NetSuite item groups, never plain items → NetSuite `add()` USER_ERROR, order re-emails
|
|
146
|
+
hourly while stuck), the two distinct ODP error emails (3a PO-import vs 5 SO-creation) and cron
|
|
147
|
+
5's two error sources, plus the cron-5 error-email rework (`buildNetsuiteErrorMessage`
|
|
148
|
+
`statusDetail` parsing, no-rate part numbers appended, reworded subject, `NetSuiteError_<SO>.txt`
|
|
149
|
+
request/response attachment). (bala)
|
|
104
150
|
- 2026-07-01 — Documented the numbered ODP→NetSuite worker-cron pipeline (crons 1/2/edi-1/5),
|
|
105
151
|
the AS2 850 S3 hand-off (`agilant-as2` / `OfficeDepot/`), the customerId=1-vs-2 NetSuite
|
|
106
152
|
writeback rule + cron-5 exclusion filters, and the "order not in NetSuite" diagnostic method
|
package/package.json
CHANGED