toga-ai 1.0.879 → 1.0.880
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/tools/INDEX.md +1 -0
- package/knowledge/1.0/apps/tools/architecture.md +3 -2
- package/knowledge/1.0/apps/tools/features/worker2-job-bridge.md +68 -0
- package/knowledge/2.0/apps/worker2/INDEX.md +1 -0
- package/knowledge/2.0/apps/worker2/features/stranded-worker-jobs.md +56 -0
- package/knowledge/2.0/apps/worker2/workflows/running-worker2-locally.md +2 -0
- package/knowledge/INDEX.md +2 -2
- package/knowledge/clients/compass-usa/INDEX.md +1 -0
- package/knowledge/clients/compass-usa/features/people-file-import-admin-page.md +62 -0
- package/knowledge/clients/compass-usa/features/people-file-user-lifecycle.md +4 -0
- package/package.json +1 -1
|
@@ -20,4 +20,5 @@
|
|
|
20
20
|
| [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | Tools (1.0) admin UI to browse/fix Talos KB documents, provision new Bedrock VECTOR KBs (S3 Vectors), and manage transcript-cleanup vocabulary — the human front |
|
|
21
21
|
| [Talos Pricing UI (Contracts, Pricing Dashboard, Usage, Settings + Estimator)](features/talos-pricing-ui.md) | The 1.0 (tools) face of the Talos Pricing Platform — Contracts / Pricing Dashboard / Usage / Pricing Settings pages + the estimate engine, reading the Team DB v |
|
|
22
22
|
| [App-Wide Colour Theme (light / dark / auto)](features/theme-light-dark.md) | App-wide light/dark/auto colour theme for the Tools app via a single semantic-token layer (`theme.css`), localStorage-persisted, applied before first paint; ope |
|
|
23
|
+
| [Tools → worker2 Job Bridge (App_Worker::enqueue + reading WorkerJobs.output)](features/worker2-job-bridge.md) | How a 1.0 `tools` page runs 2.0 worker2 work and reads the result back; open before adding any Tools page that triggers a worker action. |
|
|
23
24
|
| [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the Tools 1.0 app deploys/boots on Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023) — AL2023 package migration, INI/fpm session gotchas, git-clone of library |
|
|
@@ -6,8 +6,8 @@ project: Tools
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: architecture
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
10
|
-
owners: [jcardinal, kyalamarthi, ajean]
|
|
9
|
+
updated: 2026-09-23
|
|
10
|
+
owners: [jcardinal, kyalamarthi, ajean, mhammontree]
|
|
11
11
|
files:
|
|
12
12
|
- tools/index.php
|
|
13
13
|
- tools/.ebextensions/020_setup_git_libraries.config
|
|
@@ -79,6 +79,7 @@ Create `mvc/<folder>/<tool>/get.php` (copy `mvc/_TEMPLATE/get.php`), add one ent
|
|
|
79
79
|
|
|
80
80
|
## Gotchas
|
|
81
81
|
|
|
82
|
+
- **⚠ `tools` runs PHP 8, not 7.2 — check the vhost, not the framework label.** Despite being a 1.0 `App_` app, its vhost lives in the PHP 8 Apache (`DocumentRoot "C:/WWW/tools"`), and `App_Compass::tenantKey(mixed $key)` uses the `mixed` type hint (PHP 8.0+). The trap: `mixed` **passes** `php -l` on 7.2 (parsed as a class name) and only fatals at runtime — "Argument 1 passed to f() must be an instance of mixed, string given". So linting `tools` on 7.2 fails in the dangerous direction: it accepts code that would die.
|
|
82
83
|
- **Plaintext secrets were publicly web-readable (fixed 2026-07-24; rotation still owed).** `config.*.ini` (incl. `config.prod.ini`) holds plaintext production secrets (RDS master password, SMTP, Payeezy, NetSuite). The front controller routed through `index.php` on a `RewriteCond !-f` rule **only**, which is not access control — a direct `GET /config.production.ini` served the file **verbatim**. Fixed 2026-07-24 by adding `<FilesMatch>` `Require all denied` blocks (sensitive extensions incl. `.ini`, and any `config.*`) — see the [1.0 back-end security standard](../../standards/backend-php.md). **Action still owed:** rotate every secret in `config.production.ini` — the file was reachable, so treat all of it as compromised. (Location only — no values recorded here.)
|
|
83
84
|
- **Hardcoded GitHub Personal Access Token committed in plaintext (rotation owed).** `.ebextensions/020_setup_git_libraries.config` contains a GitHub PAT in plaintext, committed to the repo and therefore exposed in git history. **Action owed:** rotate the token and move it to an EB environment property (read via `getenv()`/`$_SERVER`/`$_ENV` per the deploy workflow's clear_env gotcha); a rotation does not un-expose git history. (Location only — no value recorded here.)
|
|
84
85
|
- **SSO initiation + replay defense are open items** — see `features/saml-sso-auth.md`.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Tools → worker2 Job Bridge (App_Worker::enqueue + reading WorkerJobs.output)
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: tools
|
|
5
|
+
project: Tools
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-09-23
|
|
10
|
+
owners: [mhammontree]
|
|
11
|
+
files:
|
|
12
|
+
- tools/_/app/worker.php
|
|
13
|
+
related:
|
|
14
|
+
- ../../../clients/compass-usa/features/people-file-import-admin-page.md
|
|
15
|
+
- ./mvc-data-access-patterns.md
|
|
16
|
+
- ../../../2.0/apps/worker2/architecture.md
|
|
17
|
+
- ../../../2.0/apps/worker2/features/stranded-worker-jobs.md
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
How a 1.0 `tools` page runs 2.0 worker2 work and reads the result back; open before adding any Tools page that triggers a worker action.
|
|
21
|
+
|
|
22
|
+
## Summary
|
|
23
|
+
|
|
24
|
+
`tools` never talks to a third-party system that worker2 already owns. It **enqueues a worker2
|
|
25
|
+
action** and reads the answer out of `Core.WorkerJobs.output`. `App_Worker::enqueue()`
|
|
26
|
+
(`tools/_/app/worker.php`) inserts the `Core.WorkerJobs` row and sends the SQS message.
|
|
27
|
+
|
|
28
|
+
## How it works
|
|
29
|
+
|
|
30
|
+
1. Page calls `App_Worker::enqueue($action, $parameters)`.
|
|
31
|
+
2. worker2 picks the job up, runs it, writes the result into `Core.WorkerJobs.output`.
|
|
32
|
+
3. Page polls the row by `workerJobId` and decodes `output`.
|
|
33
|
+
|
|
34
|
+
**`output` is a LOG LINE wrapping the JSON**, written by `worker2/Controller/Index.php`:
|
|
35
|
+
|
|
36
|
+
```
|
|
37
|
+
[requestId] Successfully Executed (Ns): {"...json..."}
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
So find the first `{` and decode from there. If no `{` is present, show a clear error — never an
|
|
41
|
+
empty result — because the wrapper format is worker2's, not a contract.
|
|
42
|
+
|
|
43
|
+
## Gotchas
|
|
44
|
+
|
|
45
|
+
- **`enqueue()` has THREE outcomes, not two.**
|
|
46
|
+
(a) insert fails → `ok=false`, `workerJobId=null`.
|
|
47
|
+
(b) insert succeeded but the **SQS send failed** → `ok=false` with a **real** `workerJobId` and
|
|
48
|
+
error "Job row created but SQS dispatch failed".
|
|
49
|
+
(c) both succeeded → `ok=true`.
|
|
50
|
+
A caller that tests only `workerJobId === null` reads (b) as success and then shows "still
|
|
51
|
+
running" forever, because no worker will ever pick that row up. **Test `ok`, not the id.**
|
|
52
|
+
- **`isSuccess IS NULL` means two different things** — running, or abandoned. Same trap as
|
|
53
|
+
[stranded worker jobs](../../../2.0/apps/worker2/features/stranded-worker-jobs.md).
|
|
54
|
+
- **Tools must not speak SFTP (or any integration worker2 owns).** Decided on CTO review
|
|
55
|
+
(TRUE-82250). Giving `tools` its own SFTP client would copy a production credential into a
|
|
56
|
+
second repo (rotation becomes a two-repo change forever), and **`library/phpseclib` is
|
|
57
|
+
phpseclib 1.x**: it offers only `diffie-hellman-group1-sha1`, `group14-sha1`,
|
|
58
|
+
`group-exchange-sha1`, `group-exchange-sha256` — no curve25519, no ecdh-sha2. Modern SFTP
|
|
59
|
+
servers commonly refuse sha1 key exchange, so 1.0 may not negotiate at all where worker2's
|
|
60
|
+
phpseclib3 can.
|
|
61
|
+
- **Timeouts do not match.** Tools' PHP `max_execution_time` is **120s**. A job enqueued outside
|
|
62
|
+
the CronScheduler has `cronJobId NULL` and therefore the **300s** action cap, not the cron's
|
|
63
|
+
`maxExecutionTime`. Never make a page wait for long work — poll, don't block.
|
|
64
|
+
- **Deploy worker2 first.** A Tools page that calls a new action is broken until the action
|
|
65
|
+
exists.
|
|
66
|
+
|
|
67
|
+
## Change history
|
|
68
|
+
- 2026-09-23 — Created from TRUE-82250: documented the three `enqueue()` outcomes, the log-line-wrapped `output`, and the CTO decision that tools never speaks SFTP (phpseclib 1.x sha1-only KEX + credential duplication). (mhammontree)
|
|
@@ -53,6 +53,7 @@
|
|
|
53
53
|
| [Service Request → Sales Order → Purchase Order generation (Sync/ServiceRequest)](features/service-request-sales-order-generation.md) | Worker `_Worker_Sync_ServiceRequest` that turns a Service Request → Sales Order → one PO per vendor for any tenant; open for the SR→SO→PO chain, its interceptor |
|
|
54
54
|
| [SSO Stability Monitor (Monitor/Operations/SsoStability)](features/sso-stability-monitor.md) | Reporter cron that GETs the saml `/healthcheck` and pushes an SSO heartbeat to a OneUptime monitor; open when working on the SSO uptime probe, its tokens, or it |
|
|
55
55
|
| [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Inbound Startech (Easeedesk) webhook handler that creates/updates the matching TOGA 2.0 ticket; open for the webhook flow, its payload, or ticket-type/stage res |
|
|
56
|
+
| [Stranded WorkerJobs (deploy-window job loss) and how to repair them](features/stranded-worker-jobs.md) | Jobs queued while worker2 restarts are dropped silently with no retry and no alert; open when a cron "never ran" or a webhook vanished. |
|
|
56
57
|
| [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | Consumer side of Talos (TOGa IQ): the endpoints, DevCore assistant, and JWT auto-refresh a worker2 dev tool needs to query the meeting-notes corpus; open when a |
|
|
57
58
|
| [Talos Pricing Automation (worker2 Cron — Usage Import, AWS Actuals, Margins, Profiles)](features/talos-pricing-automation.md) | Monthly worker2 crons that turn raw usage + AWS cost into a measured platform unit price, per-client cost, and margins; open for the pricing pipeline, its AWS c |
|
|
58
59
|
| [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` cron pipeline ingesting Teams VTT transcripts into Talos/TOGa IQ Bedrock KBs + emailing organizer recaps; open for Export/Process/Ret |
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Stranded WorkerJobs (deploy-window job loss) and how to repair them
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-09-23
|
|
10
|
+
owners: [mhammontree]
|
|
11
|
+
files:
|
|
12
|
+
- worker2/Controller/Index.php
|
|
13
|
+
- worker2/LambdaFunctions/JobScheduler/lambda_function.py
|
|
14
|
+
- worker2/Worker/Infrastructure/Worker.php
|
|
15
|
+
related:
|
|
16
|
+
- ../architecture.md
|
|
17
|
+
- ./creating-worker-actions.md
|
|
18
|
+
- ../../../1.0/apps/tools/features/worker2-job-bridge.md
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
Jobs queued while worker2 restarts are dropped silently with no retry and no alert; open when a cron "never ran" or a webhook vanished.
|
|
22
|
+
|
|
23
|
+
## Summary
|
|
24
|
+
|
|
25
|
+
A **worker2 deploy silently drops every job queued during the restart window.** The row stays
|
|
26
|
+
`dtQueued` set, `dtStarted NULL`, `isSuccess NULL` — forever. Nothing recovers it:
|
|
27
|
+
|
|
28
|
+
- JobScheduler Part A only re-queues rows where `dtQueued IS NULL`.
|
|
29
|
+
- The watchdog (Part B) only fails rows that already have `dtStarted`.
|
|
30
|
+
|
|
31
|
+
So there is **no failure, no alert, no `Logs.Issue`** — the job just never happened.
|
|
32
|
+
|
|
33
|
+
Measured in prod 2026-09-23: **95 rows** stranded between 09:15:45 and 11:05:22. Not only crons —
|
|
34
|
+
Clickup/Webhook 9, Netsuite/Webhook 6, Github/Webhook 5, Infrastructure/Email/Send/Run 4,
|
|
35
|
+
Infrastructure/Errors/Escalate 4, plus 2 each for eight `Monitor/Compass/*` actions.
|
|
36
|
+
|
|
37
|
+
## How it works — repairing them (crons and webhooks need OPPOSITE treatment)
|
|
38
|
+
|
|
39
|
+
**CRON row** — mark it `isSuccess = 0` with an explanatory `output`.
|
|
40
|
+
- Do **not** delete it: that destroys the audit record.
|
|
41
|
+
- Do **not** re-queue by nulling `dtQueued`: that re-runs stale time-based work.
|
|
42
|
+
- If the run is genuinely still needed, fire it deliberately as a fresh job.
|
|
43
|
+
|
|
44
|
+
**WEBHOOK row** — the payload is still in `parameters`, so
|
|
45
|
+
`_Worker_Infrastructure_Worker::Retry($workerJobId)` genuinely recovers it. Retry it.
|
|
46
|
+
|
|
47
|
+
## Gotchas
|
|
48
|
+
|
|
49
|
+
- **`isSuccess IS NULL` is ambiguous** — running, or abandoned. Check `dtStarted`: NULL plus a
|
|
50
|
+
`dtQueued` older than a few minutes = stranded.
|
|
51
|
+
- **After any worker2 deploy, sweep for stranded rows.** Nothing else will tell you.
|
|
52
|
+
- Same ambiguity on the enqueue side in `tools` — see
|
|
53
|
+
[Tools → worker2 job bridge](../../../1.0/apps/tools/features/worker2-job-bridge.md).
|
|
54
|
+
|
|
55
|
+
## Change history
|
|
56
|
+
- 2026-09-23 — Created: 95 prod jobs lost in a deploy window (2026-09-23), with the cron-vs-webhook repair rule. (mhammontree)
|
|
@@ -131,6 +131,7 @@ NetSuite or to `Forecast`.
|
|
|
131
131
|
|
|
132
132
|
- **⚠ Locally, worker2's `Core` resolves to the 1.0 `Core` database, not the 2.0 one.** `worker2/_.php:20` has `const DB_CORE = 'Core';` and `Core` is never explicitly registered, so the schema name is used literally. On a dev machine the 2.0 tables live in **`Core_2`** (48 tables) while `Core` is the 1.0 schema (10 tables) — symptom: `Table 'core.workerjobs' doesn't exist`. Workaround: flip `DB_CORE` to `'Core_2'` locally and **revert before committing**.
|
|
133
133
|
- **That workaround is incomplete — `_underscore/Email.php:435` hardcodes the literal prefix `FROM Core.Clients`.** So the summary-email step still throws `Table 'core.clients' doesn't exist` no matter what `DB_CORE` says. In the Compass PEOPLE cron that step runs **last**, after both tenants commit, so the data work is unaffected — but do not read the trailing exception as a failed run.
|
|
134
|
+
- **Locally, YOU are the queue — non-prod worker2 has no SQS consumer.** Nothing picks a `WorkerJobs` row up on its own; kick each job by POSTing `{"workerJobId": N}` to the worker2 host (Postman).
|
|
134
135
|
- **⚠ A stale local client database silently invalidates the test.** A local tenant that predates a migration can make broken code **pass** locally — e.g. local `Client_Compass` still had `contactId` NULLABLE, so a missing FK value was accepted where prod's `NOT NULL` turns it into `0` and a constraint error. Before testing anything schema-sensitive, drop and reload the tenant from production and **re-verify the constraint/index actually survived the copy**.
|
|
135
136
|
|
|
136
137
|
- **The ad-hoc SuiteQL runner `test/@dave/Junk Drawer/NetSuite/nsq.php` cannot start — stale bootstrap.**
|
|
@@ -181,4 +182,5 @@ NetSuite or to `Forecast`.
|
|
|
181
182
|
— the api2 sibling of steps 2–3.
|
|
182
183
|
|
|
183
184
|
## Change history
|
|
185
|
+
- 2026-09-23 — Noted that non-prod worker2 has no SQS consumer: kick jobs locally by POSTing `{"workerJobId": N}`. (mhammontree)
|
|
184
186
|
- 2026-09-23 — Added three local-run traps found on TRUE-82230: worker2's literal `DB_CORE = 'Core'` hits the 1.0 schema locally (2.0 lives in `Core_2`); `_underscore/Email.php:435` hardcodes `FROM Core.Clients` so the summary-email step throws regardless; and a stale local client DB can make broken code pass (reload from prod and re-verify constraints). (mhammontree)
|
package/knowledge/INDEX.md
CHANGED
|
@@ -15,12 +15,12 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
15
15
|
- **test** (Test) — 16 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
|
|
16
16
|
- **toga** (TOGa) — 3 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
|
|
17
17
|
- **store** (Store) — 3 doc(s) → [1.0/apps/store/INDEX.md](1.0/apps/store/INDEX.md)
|
|
18
|
-
- **tools** (Tools) —
|
|
18
|
+
- **tools** (Tools) — 21 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
|
|
19
19
|
|
|
20
20
|
## 2.0 framework
|
|
21
21
|
|
|
22
22
|
- **_underscore** (_Underscore) _(framework core)_ — 90 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
23
|
-
- **worker2** (Worker) —
|
|
23
|
+
- **worker2** (Worker) — 75 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
24
24
|
- **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
25
25
|
- **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
|
26
26
|
- **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
|
|
@@ -21,6 +21,7 @@
|
|
|
21
21
|
| [Compass: ODP / Strategic Systems ASN CSV email import (and the scientific-notation tracking-number loss)](features/odp-strategic-systems-asn-csv-import.md) | 1.0 | The 1.0 worker cron that reads Office Depot / Strategic Systems ASN CSVs from email, why it routes by column count, and why a tracking number can arrive as `8.7 |
|
|
22
22
|
| [Compass: MA refresh-order monitor (Monitor/Compass/MaRefreshOrderExceptions)](features/oneuptime-ma-refresh-order-monitor.md) | 2.0 | worker2 OneUptime push monitor counting Compass MA employee-refresh orders whose ODP SO has no NetSuite order yet; open when working on Compass MA monitoring or |
|
|
23
23
|
| [Compass USA — Fulfilled vs Partially Fulfilled (order total kept; per-line rule built then rejected)](features/order-fulfillment-status-per-line.md) | 2.0 | Why Compass USA shows Fulfilled on partly-shipped orders and why the per-line fix was built, measured, then reverted; open before touching Compass USA `_status` |
|
|
24
|
+
| [People File Import admin page (tools /compass/people-file-import → worker2)](features/people-file-import-admin-page.md) | 1.0 | Browser page that lists the Compass PEOPLE files on SFTP and runs the importer on demand; open before changing the page or the importer's listing/guard code. |
|
|
24
25
|
| [Compass PEOPLE-File User Lifecycle (duplicate accounts, reactivation grace window, raw-SQL deactivation)](features/people-file-user-lifecycle.md) | 2.0 | The nightly Compass PEOPLE-file cron's Users-row lifecycle (insert/reactivate/deactivate), why it creates duplicate accounts, and its raw-SQL/index traps; open |
|
|
25
26
|
| [Persona Model & Levy-Sector Gating (worker2 PEOPLE cron)](features/persona-model-and-levy-gating.md) | 2.0 | Compass USA catalogue visibility is driven by **personas** in `Client_Compass`. |
|
|
26
27
|
| [Compass Sales-Order Line-Number Renumbering (and why it hangs off the SO hooks)](features/sales-order-line-renumbering.md) | 2.0 | How Compass keeps SO lines numbered 1..N with no gaps, why the renumber rides the parent SO hooks, and the two-phase SQL that avoids a duplicate-key fatal; open |
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: People File Import admin page (tools /compass/people-file-import → worker2)
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: tools
|
|
5
|
+
project: Tools
|
|
6
|
+
client: compass-usa
|
|
7
|
+
type: client-feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-09-23
|
|
10
|
+
owners: [mhammontree]
|
|
11
|
+
files:
|
|
12
|
+
- tools/_/app/compass/peoplefile.php
|
|
13
|
+
- tools/mvc/compass/people-file-import/get.php
|
|
14
|
+
- tools/mvc/compass/people-file-import/post.php
|
|
15
|
+
- worker2/Worker/Client/Compass/PeopleFile.php
|
|
16
|
+
related:
|
|
17
|
+
- ./people-file-user-lifecycle.md
|
|
18
|
+
- ../../../1.0/apps/tools/features/worker2-job-bridge.md
|
|
19
|
+
- ../../../1.0/apps/tools/features/compass-user-persona-admin.md
|
|
20
|
+
- ../../../2.0/apps/worker2/features/stranded-worker-jobs.md
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
Browser page that lists the Compass PEOPLE files on SFTP and runs the importer on demand; open before changing the page or the importer's listing/guard code.
|
|
24
|
+
|
|
25
|
+
## Summary
|
|
26
|
+
|
|
27
|
+
`tools` page **/compass/people-file-import** (`App_Compass_PeopleFile`, Development Team persona
|
|
28
|
+
only) lists the PEOPLE files waiting on the SFTP server, with a **Refresh** button and an
|
|
29
|
+
**Import** button shown only when files exist. It exists so a stranded or failed nightly cron can
|
|
30
|
+
be re-run without a developer.
|
|
31
|
+
|
|
32
|
+
`tools` does **not** touch SFTP — it enqueues worker2 actions and reads `Core.WorkerJobs.output`.
|
|
33
|
+
See [Tools → worker2 job bridge](../../../1.0/apps/tools/features/worker2-job-bridge.md) for the
|
|
34
|
+
enqueue/decode rules and why SFTP stays in worker2.
|
|
35
|
+
|
|
36
|
+
## How it works
|
|
37
|
+
|
|
38
|
+
worker2 side (`Worker/Client/Compass/PeopleFile.php`):
|
|
39
|
+
|
|
40
|
+
- `gatherPeopleFiles(SFTP): array` — oldest first, **shared by both** the listing and the import,
|
|
41
|
+
so the page can never show a different "next file" than the importer takes.
|
|
42
|
+
- `Client/Compass/PeopleFile/ListFiles` — new read-only action returning JSON, used by the page.
|
|
43
|
+
- `isAnotherRunInFlight()` guard on `ProcessPeopleFiles`.
|
|
44
|
+
- `ProcessPeopleFiles` now **returns a JSON summary on every path**. It previously returned
|
|
45
|
+
nothing, so every run recorded "<no return message>".
|
|
46
|
+
|
|
47
|
+
**Deploy order: worker2 FIRST, then tools.** The page calls `ListFiles`, which does not exist
|
|
48
|
+
until worker2 ships.
|
|
49
|
+
|
|
50
|
+
Reference: prod `Core.CronJobs` id **8** = "Compass - People File Import", schedule `15 9 * * *`,
|
|
51
|
+
`maxExecutionTime` 3600, action `Client/Compass/PeopleFile/ProcessPeopleFiles`. A job enqueued
|
|
52
|
+
from this page is **not** scheduler-fired, so it gets the 300s action cap, not 3600.
|
|
53
|
+
|
|
54
|
+
## Gotchas
|
|
55
|
+
|
|
56
|
+
- **A failed run does not mean nothing happened.** The importer is not all-or-nothing — see
|
|
57
|
+
[PEOPLE-file user lifecycle](./people-file-user-lifecycle.md). Check the data before importing
|
|
58
|
+
the same file again.
|
|
59
|
+
- **Persona-gated to Development Team.** It runs a production import; do not widen it casually.
|
|
60
|
+
|
|
61
|
+
## Change history
|
|
62
|
+
- 2026-09-23 — Created with the page (TRUE-82250, tools 0cc6f99 / worker2 a81ad6c, branch TRUE-82250): shared `gatherPeopleFiles`, new `ListFiles` action, in-flight guard, and a JSON summary return on every path. (mhammontree)
|
|
@@ -91,6 +91,7 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
|
|
|
91
91
|
**Open product question, still open:** when a manager has genuinely left and HR supplied no replacement, what should the order do — block, escalate to the supervisor's supervisor, or route to an admin queue? The engine deliberately **skips** these as ambiguous rather than guessing.
|
|
92
92
|
|
|
93
93
|
## Gotchas
|
|
94
|
+
- **⚠ The import is NOT all-or-nothing — a failed job can still have written everything.** Each tenant's import is transactional, but everything **after** the tenant transactions commit runs outside it: `sendProcessingSummaryEmail()` (~line 264) and the SFTP delete. A run that fails at the email step records `isSuccess = 0` while all the data stays written, **and the file stays on SFTP**, so the same file can be imported a second time. Proven locally 2026-09-23: job 1076884 failed in `sendProcessingSummaryEmail` after 25.4s with 460 US + 26 CA users (and the same number of Contacts) committed and kept. **Check the data before re-running.**
|
|
94
95
|
- **⚠ Deactivation is raw bulk SQL — model hooks never fire.** User inactivation runs as `UPDATE Users SET isActive = 0 WHERE id IN (...)` through `_Query`, **not** the model layer. A `postPut` / `postPost` interceptor on the User model will **never** run on deactivation. Any deactivation-time side effect must be written **inside the importer itself** — adding a model hook and expecting it to fire is a silent no-op.
|
|
95
96
|
- **The same raw SQL is invisible to the change-audit log.** It produces no `RecordField` rows in `Logs_Compass`, so anything the importer should be accountable for must write its own `Record` **note** explicitly. See [Record Change Audit Log](../../../2.0/apps/_underscore/features/record-change-audit-log.md).
|
|
96
97
|
- **A duplicate account strands approvals, but not email.** The approval gate matches on user id, so a decision assigned to the old row is invisible to the same human's new account — while the notification email still arrives, because both rows carry the same address. Visible symptom: a person chased to approve an order they cannot see. Detail: [Approval-Decision Flow](./approval-decision-flow.md).
|
|
@@ -118,8 +119,11 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
|
|
|
118
119
|
- [Compass Approval-Decision Flow](./approval-decision-flow.md) — what a duplicate account breaks downstream.
|
|
119
120
|
- [Stranded Approval Reassignment](./stranded-approval-reassignment.md) — the engine that repairs the approvals a duplicate account stranded.
|
|
120
121
|
- [Record Change Audit Log](../../../2.0/apps/_underscore/features/record-change-audit-log.md) — why the importer's raw writes leave no audit trail.
|
|
122
|
+
- [People File Import admin page](./people-file-import-admin-page.md) — the on-demand re-run page in `tools`.
|
|
123
|
+
- [Stranded WorkerJobs](../../../2.0/apps/worker2/features/stranded-worker-jobs.md) — why the nightly cron row can never run at all.
|
|
121
124
|
- [Compass USA profile](../profile.md)
|
|
122
125
|
- [Users.email as login identity](../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md)
|
|
123
126
|
|
|
124
127
|
## Change history
|
|
128
|
+
- 2026-09-23 — Recorded that the import is **not** all-or-nothing (post-commit email + SFTP delete run outside the tenant transaction; job 1076884 failed yet kept 460 US / 26 CA users). TRUE-82230 confirmed in prod: 543 `Client_Compass` users created, **zero** null-or-0 `contactId`, two manual ACTION runs ~18s each — needed because the 09:15 cron row (job 1784367) was stranded. (mhammontree)
|
|
125
129
|
- 2026-09-23 — **Fixed the `Users.contactId` FK write order (TRUE-82230).** Jeff's `NOT NULL` + `Users_ibfk_1` on both tenants broke the importer from 2026-09-22 09:15 (last good run 2026-09-21); the whole run rolled back and Canada never executed. New `insertContactsForNewUsers()` creates the Contact before the `Users` batch and puts `contactId` in the INSERT column list; existing users missing a contact keep the create-then-UPDATE path. Also recorded: `LAST_INSERT_ID()` must be read on the write connection (not `_Query`); `TEST_MODE` vs `DRY_RUN` neither being safe alone; the FIFO/one-file/empty-is-silent-success facts; and that a stale local DB hides this class of bug. Verified against prod-reloaded local schemas: 80 new US / 17 new CA users, 0 with null-or-0 `contactId`. (mhammontree)
|
package/package.json
CHANGED